Hello and welcome to my channel! In this video, we'll explore what Istio is,
how it works, and how it can benefit your microservices architecture. First, we'll talk about istio architecture
and how it works under the hood. Then I'll show you multiple methods on how
to install istio on Kubernetes cluster, including helm installation. I'll walk you through a few examples of how
to manage traffic, request routing, and canary deployment. And we'll talk about multiple ways to inject
sidecars into the pod. One of the biggest topics of this tutorial
is how to use an ingress gateway to expose applications running in Kubernetes to the
internet. We'll install cert-manager and use letsencrypt
to automatically obtain TLS certificates and secure our APIs. Finally, I'll show you how to use Prometheus
and grafana to monitor latency, traffic, and availability not only of the services exposed
to the internet but also of internal applications. And, of course, we'll use kiali to visualize
the service mesh inside our cluster. In the end, we'll talk about Gateway API and
how to use it for both service mesh and instead of your typical ingress. So what istio is? Istio is an open-source service mesh that
provides a unified platform to connect, manage, and secure microservices. It was created by Google, IBM, and Lyft in
2017 and has since gained significant traction in the cloud-native community. Istio is built on top of Envoy - a high-performance
proxy that handles all the traffic between microservices. Istio works by deploying a sidecar container
alongside each microservice instance in your environment. The sidecar container intercepts all traffic
to and from the microservice, handling traffic routing, load balancing, service discovery,
and other important networking tasks. Istio also provides advanced traffic management
features like canary deployments, A/B testing, and fault injection. Now, why would you want to use Istio? Istio provides several benefits to modern
microservices architecture. Firstly, it simplifies network management
by abstracting away the complexity of service discovery, load balancing, and traffic routing. Secondly, Istio provides advanced security
features like mutual TLS authentication, role-based access control, and traffic encryption. Finally, Istio provides observability features
like distributed tracing, metrics collection, and logging. Since the istio operator is deprecated, most
people would prefer helm installation. It's pretty easy to include in your deployment
pipeline with other terraform resources. You can use istioctl to try it out, but realistically
you'd probably still use helm to deploy it in production environments. This step is optional only to show you how
to get default helm values. Let's go ahead and add the official helm repo. Now let's search for the base helm chart that
includes the custom resource definitions, which are mandatory to install before you
can deploy istiod. I highly recommend using the same version
for the first try; otherwise, the ingress gateway may not work for you. We can use helm show values to get defaults
for this helm chart and pipe it to the file. Let's call it istio-base-defaults.yaml. You can open it and find values that you want
to customize. It's pretty basic. For this video, I'm going to be using the
EKS cluster that you can create using my terraform code. Now I'll use terraform as well to deploy the
istio helm chart. You can put it in the same folder or follow
along. First of all, we need to declare the helm
provider. To authenticate with Kubernetes cluster such
as EKS, we can dynamically obtain a temporary token and then use it to install helm. Another way is just to point the provider
to Kubernetes config on your local machine. Next, create a new file for the first istio
base helm chart. You don't have to use terraform; you can copy
a command that exactly matches terraform and execute it in the terminal. I prefer to use terraform to make it easily
reproducible. Let's call this release my-istio-base. Point to the remote helm repository. Specify what helm chart you want to use. Then the Kubernetes namespace. If it does not exist, let's create it. Also, make sure to use the same version. Just in case if you want to override any variables,
you can use set statements. For example, set the default namespace for
the istio deployment. Recently istio team combined the pilot, citadel,
and gallery into a single executable called istiod. Let's get the default values for it as well
and use the same version. We would need to override a few variables
for the ingress gateway to work. Create another terraform file to deploy the
istiod helm chart. You can also find the helm command to deploy
it manually. Telemetry and namespace are default values,
but we need to override the ingress service and ingress selector for the cert-manager
to work correctly and be able to solve the http01 challenge from letsencrypt. Also, we need to explicitly depend on the
previous helm chart since istiod requires custom resources. In the terminal, you need to initialize terraform
first. Then run apply to deploy both helm charts. Let's check if the istio custom resources
were created. Also, we need to make sure that the istiod
pod is running. Alright, istio is deployed to our cluster,
and we're ready to start managing traffic using the istio control plane. Let's create a dedicated namespace for our
first example and call it staging. If you want istio to inject sidecars to all
pods in this namespace, you add an injection enabled label. In order for istio to manage traffic, each
service must have an istio sidecar. We'll have two deployments to simulate a canary. This deployment is version v1. Also, when we get to the kiali, you would
want these labels to show up in the UI. It's a simple golang app that is uploaded
to the docker hub. Since it's a public image, you can use it
as well. The second deployment is identical, except
that it uses the v2 version label. When we create this standard service in Kubernetes,
since it uses a single label app: first-app, it randomly routes traffic to both versions
v1 and v2. To manage and shift traffic, we actually need
a couple of istio custom resources. The destination rule defines what backend
applications are used. They call it subsets. We have a subset for the v1 version, which
is also called v1, and another one for v2. You also need to add a target host; in this
case, it's a service name first-app. You can get all the values from the official
istio documentation. The second custom resource is virtual service. First, we also need to use the same target
host. Same as in the previous example, it will target
Kubernetes service first-app. Then we can intelligently route traffic to
different backends. For this example, we'll simulate a typical
application upgrade when you need to roll out a new version of the app, in this case,
v2, and start shifting traffic to it. Sometimes it's called canary deployment. When the traffic hits the new version, you
can monitor status codes and decide if you want to proceed or abort. Alright, let's start routing traffic to only
version v1. Now to test, we need a client inside Kubernetes,
which also have a sidecar. Port forward won't work in this case. Later we'll use a gateway. Let's go ahead and apply the example-1 folder. Check the status of the pods in the staging
namespace. If you add a wide flag, you can get the internal
IP addresses of the pods. Take note of those two. We also have a single Kubernetes service in
that namespace. If you check endpoints on that service, you'll
get two pods. So if you access this service without a sidecar,
you would randomly receive traffic from both versions, v1, and v2. For the test, ssh to the client in the backend
namespace. That pod is based on a curl image and has
an infinite loop to prevent it from exiting. Let's use curl in the loop to hit our first-app
service in the staging namespace. Since we used istio virtual service, traffic
is only routed to v1 version. Later I'll show you how to visualize this
traffic in the UI. Now let's say we want to upgrade our app to
version v2. First, we want to route a small percentage
of the traffic to that version and make sure it's healthy. Add weight attribute with 90% value. Also, add another destination rule that will
send traffic to the v2 version but only 10% of the traffic. Everything is ready; let's apply the new destination
rules. On the bottom window, you can see that v2
versions rarely appear. Since we only route 10% of traffic to that
version. Without istio to implement the same task,
you would scale version v1 to 9 pods and create a single pod for v2. Well, you can have any number of pods just
need to keep the ratio. With istio, it does not matter how many pods
you have. You can still intelligently route traffic. Let's say we are now confident that the v2
version has no bugs. Now we can start routing half of the traffic
to the new version. And of course, you can automate this in your
pipeline, but first, you need to understand how it works. Let's apply it again. Now we receive an equal amount of traffic
on both versions. Also, you can implement similar routing with
Gateway API but with experimental custom resources only. We'll talk about technics at the end of the
video. At this point, we are confident in our new
version, and we want to route all the traffic to it. Apply it again, and you'll see immediately
that all the traffic is now routed to the v2 version. In this section, I want to talk about multiple
ways you can inject sidecars into your pods. The first one and most common is to use inject
label. You can also set the value to disable, and
other methods won't work either. The second method is to use a pod inject enabled
label. Now the third one is to inject manually. But you should use it only when you are trying
to test something. For example, this deployment does not have
labels, and istio, should not inject the sidecar. Let's apply and see the result. Alright. the first deployment has a sidecar, and the
second one does not. To manually inject sidecar, you need to use
istioctl cli and specify the path to that deployment. If you get the pods in a few seconds, you
should see that the istio sidecar was injected into the second pod. In the following section, I want to show you
how to expose an application running in Kubernetes to the internet using the istio ingress gateway. We'll also use the helm chart to deploy it
in our cluster. Similar to the previous charts, let's save
the default values locally. Just in case if you want to override some
of them. I'll keep all default values for the gateway. For example, you can use service annotations
to specify what kind of load balancer you want. In AWS, you probably want to upgrade classic
LB to a network load balancer. Create another terraform file for gateway
deployment. As you can see, I'm not going to override
any values in this deployment. Keep a note that we'll deploy it to the istio-ingress
namespace. Let's run terraform apply to install the helm
chart. Now we can check if the load balancer was
created. If it's in the pending state, try to describe
it and find any errors. You can also get events in that namespace. For the third example, we'll use the production
namespace and also inject the sidecar to all pods in that namespace. Then identical deployments for version v1
and v2. The same service, just a different name. Same destination rule that defines two subsets
for both versions. Now the virtual service must have the external
DNS name under the hosts. In my case, I have app.devopsbyexample.com
and the second one for internal Kubernetes routing. In this case, we have an additional attribute
gateway that uses the name of the gateway that we're going to create next. Similar to the previous example, we'll route
90% of the traffic to v1 and 10% to v2. You can also use different prefixes if multiple
services hide under your api, similar to a typical ingress. To expose the application to the internet,
we need to create a gateway object. This is an istio custom resource and not a
gateway api from Kubernetes. Keep this in mind; they are very similar on
purpose. On line 9 selector uses pods labels to identify
what gateway to use. To route plain http traffic, you select port
80 and http protocol. As a host, you put your dns name. You can use the same gateway to route multiple
domains just list them here. Let's check the pod labels on the ingress
gateway to make sure that it will match our label under the selector. Istio gateway looks like it should work. Let's go ahead and apply the example-3 folder. Check the status of pods. I want to show you a trick that you can use
to verify Ingress before you update DNS. Hostname is just a header. You can use curl and pretend that you already
have a proper dns. For the address, you specify the load balancer
hostname. Alright, it works; we got a response from
the app deployed in Kubernetes. Now let's create a CNAME record for our domain. I use google domains, but it does not matter. Just create a CNAME if it points to another
hostname or A record if you need to point to the IP address. In a few minutes, check if the DNS is updated. Finally, we can use our custom domain name
to reach the app. That was pretty simple. The next step is to secure our application
with a TLS certificate. First of all, we need to deploy a cert-manager,
including all custom resources that come with it. In some environments like GCP, you would also
need to create an additional firewall rule. In aws, it should work out of the box. Let's apply it. To automatically obtain TLS certificates from
letsencrypt, we need to create a cluster issuer. You can also create just Issuer, which is
namespace specific. When you are just getting started with cert-manager,
you should use a staging environment. It's almost identical to production, just
different URLs. Also, you must specify what ingress class
will be used to solve the http01 challenge. That's why we had to override some defaults
when we deployed istiod. When you test and be able to obtain a certificate,
you can switch to the production Issuer. They have a strict quota for certificates. Let's apply both staging and production cluster
issuers. Check if both of them are ready before creating
certificates. To automatically get certificates on a typical
ingress, you would use annotation. At this time, it's not supported on the istio
Ingress, but it works on gateway api. As a workaround, we can create certificate
resources separately. Use your domain name. To test certificates, update the name to staging
issuer. Also, it must be created in the istio-ingress
namespace, where you have your gateway pods. When you create these certificates, the cert-manager
will obtain a certificate from let's encrypt and store it in Kubernetes secret. The certificate is valid only for 90 days,
and the cert-manager will automatically renew it and update the secret. Let's apply it. If you get the certificate immediately, you
can see it's not ready. In case it is stuck in not ready state, you
can describe that certificate and find that the certificate request was created. Then you can describe the certificate request. You can see that order was created. Let's describe it as well. Then the challenge was created. Describe the challenge. In my case certificate was issued, and the
challenge was deleted already. If you get a certificate, it should be in
a ready state by now. Check that the Kubernetes secret was also
created. Now to secure the api, you can add another
rule for port 443 and use the https protocol. You also must specify the secret name that
was created by the certificate resource and a host. Let's apply it again to update the gateway. To check the certificate, you can use the
openssl tool. This is your certificate. You can try to access your api from the browser. It looks like it works. The certificate is valid and issued from letsencrypt. In the following section, we'll use prometheus
and grafana to monitor istio and applications. I'm not going to spend a lot of time on deploying
all monitoring components; you can watch the previous video for that. First, we need to create prometheus operator
custom resources. Then deploy the prometheus operator itself. Prometheus. And grafana. You can use either create or server-side to
apply custom resources. Then create a monitoring namespace. Deploy Prometheus operator. Prometheus. And finally, grafana. In the monitoring namespace, you should have
two pods. And a single pod for grafana. To monitor istio, we need to create a podmonitor
and use istio sidecar labels. For example, let's get one of the pods that
we want to monitor. First of all, to create a podmonitor Prometheus
object, we need a named port, in this case, http-envoy-prom. In the second part, we need to select those
pods based on the label, such as istio monitoring. Based on these two pieces, we can start monitoring
the istio service mesh. You need to specify the namespace where your
applications are deployed, staging, and production, as an example. Then use the selector and the port name. On line 8 prometheus main label must match
the one on the Prometheus resource. Let's go ahead and apply it. Since I don't have Ingress for prometheus,
I'll use port forward to access Prometheus UI. In a few seconds, maybe up to a minute, you'll
see that the prometheus operator converted podmonitor object to the native prometheus
config and reloaded the server. Under targets, you should have a new target
with a few pods. The next step is optional. I just want to show you how to use a service
monitor instead of a pod monitor. For example, to monitor the ingress gateway. Let's get the pod in yaml format. For example, you have a port, but the name
is missing, and you cannot add it for some reason. In this case, you cannot use podmonitor since
you need a named port. Instead, you can create a service and a service
monitor to target this port. Let's define a new Kubernetes service that
only uses prometheus port and give it a name metrics. Now we can create a service monitor and use
that endpoint and metrics port name. It's a useful workaround when you don't have
a port name and are not able to add it but still want to monitor the application with
prometheus. Let's apply both service and service monitor. If you refresh the prometheus UI, you should
get a new target with the gateway. There are a lot of metrics exposed by istio
sidecars, but we'll focus on requests metrics. Since we used podmonitor for internal metrics
and service monitor for the gateway, you can use a job label to filter them out. I'll use port-forward as well to access the
grafana dashboard. If you used my code to deploy grafana, the
username is admin, and the password is devops123. I prepared a dashboard to monitor Ingress
and mesh inside Kubernetes. You can copy json and import it to the grafana. Now let's simulate some traffic. On the top graph, you can find latency measured
in percentiles. The second one is traffic in requests per
second, and the bottom one is the ratio between successful and failed requests. Let's generate some failed requests by sending
requests to the not supported path. In the graph, you can see that availability
drops below 80%. For the second test, to monitor the service
mesh inside the Kubernetes, ssh to the pod and run another script. In the next section, we'll deploy kiali to
visualize the service topology inside Kubernetes. You can also deploy it using helm or yaml
files. You just need to watch out for external services,
especially you need to provide a valid prometheus URL. The topology graph is built based on prometheus
metrics. Let's apply. And verify that kiali is up. You may see some errors that you need to fix
by updating the config, but the main functionality is based on prometheus. It still should work. Since some of the requests are sent to the
wrong URL, you can see red lines that represent failed requests. In a large microservice deployment, it is
useful to find out what service is failing quickly. If we stop our script, in a couple of minutes,
you should see that the service is recovered. We have the gateway, internal service, and
different versions of our application. You can use UI to monitor traffic distribution
when you perform canary deployments. Istio has become the de facto service mesh
for modern microservices. With its advanced networking, security, and
observability features, Istio provides a unified platform to connect, manage, and secure your
microservices. If you're building cloud-native applications,
Istio is a must-have tool in your toolkit. If you want to use Istio in production in
the next couple of months, you should use istio custom resources such as destination
rules, virtual services, and others. Istio also has betta support for Gateway API
to use as Ingress. But keep in mind that it won't work with the
virtual service. Also, istio has experimental support to use
Gateway API in a service mesh. I have two other tutorials on how to implement
both. Thank you for watching, and I'll see you in
the next video.