If you want to understand what a service mesh is and learn one of its implementations, Consul as well as understand why this concept is so popular in cloud and DevOps space. And get your first hands on practice with it, then this crash course is exactly for you. You definitely want to stick around till the end, because this is going to be a value packed, super exciting crash course with lots of interesting concepts. First, we will see why we even need a service mesh technology like Consul and what it does exactly. We'll then see its practical use cases, including how it's used in multi data center and multi-cluster environments. We will understand Consul architecture so how it works and how it does all that. And finally we'll see a really interesting demo use case of deploying microservices application across two Kubernetes clusters on two different cloud platforms and configuring the connectivity of these services across different environments using Consul. And in case this sounds like complex topics and use cases, remember you are on Tech World with nano channel, so you can be sure that I will break down all the complex topics into a simple and easy to understand examples and explanations. So let's get into it. Let's say we have an e-commerce application like Amazon, which is a complex microservices application. It has services for various functionalities like product catalog to manage product information, pricing, product images, etcetera. We have a shopping cart service that allows adding products, removing them, maybe saving for later. We also have order management service to handle all the orders. We have user authentication and authorization services, obviously to manage the registration, user login and so on. You know, reviews and rating service, recommendation service. And let's say it also integrates with bunch of supplier APIs and allows others to set up their own shops or a payment gateway to integrate with external payment processors. So a bunch of stuff is going on where this huge application logic is broken down into microservices. And if you don't know what microservices are exactly and how they are designed, I actually have a separate video on that which I will link in here. And of course these microservices are interconnected. They need to talk to each other like when product is added to shopping cart product service needs to update the stock information on how many products are left. Shopping cart needs to talk to the payment service or user account. The user authentication service will talk to other services that require user authentication like verify user identity, doing order placement, or payment. For example, recommendation engine service will communicate with User authentication Service to personalize recommendations based on user preferences, as well as talk to product catalog service to fetch product details for recommended items. So as you see, it is a complex network of services that all need to talk to each other without issues to make sure that the entire application works properly and the user experience is smooth. And while moving from monolith to microservices, architecture introduced a lot of flexibility in developing and scaling such complex applications. One of the main challenges it introduced was the connectivity between those services. In monolith application is just one application and one code base, so it's all function calls between different parts of the application. But in microservices, you have multiple isolated micro applications, which introduces a couple of questions and challenges like how do they talk to each other, on which endpoints, what communication channel do they use? Do they send Http requests to each other? Do they use message broker synchronous or asynchronous communication? What about the communication bottlenecks? When one service is completely overwhelmed with requests from all other services? How to deal with a situation where one service is down and not responsive to other services? Like what happens if shopping cart service is down but other services depend on it to do their jobs and it's just not responding? And how do we even monitor such application? How do we know which services are up and running, which ones are having issues, which services are overloaded with requests or not responding or just slow in their response? So all of these are challenges that microservices infrastructure introduced. Now let's say we have our microservice application deployed within a Kubernetes cluster in ECS in one of the US regions. Let's say we are a US based company and we have mostly American users, but we grow and become super popular in Europe and Asia. So now we need to deploy our application in those regions as well, geographically closer to our new users, to make their user experience better. So we need the instance of our application in other geographic regions to make sure that our application loads fast and people in those regions have good user experience. Now, this is another layer of complexity because now the question is how do we manage communication between services across multiple regions, in multiple data centers? That introduces a whole different level of challenges of operating your microservices application, like networking and connectivity between those regions, making sure those connections are also secure, making sure the data is in sync between regions and data centers. And you don't have data inconsistencies. Now, as if that wasn't enough, our microservice is using two different databases for different purposes, and those are managed centrally. By database engineers on separate VMs, because let's say those databases are used and shared by the whole company. It's a legacy system. Everything's interconnected. So it's not easy to move away from virtual machines and migrate the database to a Kubernetes cluster. Let's say it's a large database with more powerful machines and many replicas. So it would be an enormous effort to migrate all that to a Kubernetes environment. So microservice application has to communicate with database services running on virtual machines in on premise data centers. So we have a hybrid environment. And that is even more challenging than managing connections between services on two different Kubernetes clusters. However, it is a very common real use case among many companies and projects. So that's a real challenge. Now our story continues. Let's say one day right before the holidays, AWS has an outage in multiple regions around the world, and we lose lots of business in our e-commerce application. So management decided to make sure this never happens again, to have a failover to a different cloud provider, like a backup, in case the main cloud provider has issues. So they replicate the entire application on Google Kubernetes Engine, which is a Google's managed solution of Kubernetes, because the chances of both and Google Cloud going down at the same time is very low. And this is great for business in case of any such issues. But a new headache and challenges for the engineers. Again, connectivity across multiple cloud providers now security, network configuration and so on. So as you see, the operations of microservices, especially in the modern, highly complex environments is a challenge. And that's where the service mesh technology like Consul comes in as the communication channel or communication network layer between microservices. That solves many of the above challenges. Now that's a bit simplified definition of service mesh. But essentially service mesh like Consul, is the whole infrastructure layer that has different features to solve these challenges of operating microservices applications and enabling communication between them across multiple environments. Great. So now that we understand conceptually what Consul is and why we even need a service mesh technology, let's actually understand how it works and how it solves these challenges that I talked about. And the great thing is that most service mesh technologies are pretty similar in their functionalities. So understanding the concepts of how Consul works will make it much easier for you to understand any other service mesh technology as well. So I'm a big fan of concepts before technologies concept to understand what problem you are solving and what is the need. And technology is then a tool for solving that problem and fulfilling that need. So let's understand all that to understand the way a service mesh like Consul works. Let's imagine we have a city with different buildings and roads, and those apartment buildings have a bunch of residents in apartments, and those residents each do their tasks, and sometimes they need information from other residents to complete those. So they send messages to each other to communicate. And each resident has an own registry book, like an old address book with a list of all the residents they talk to and their addresses, where they can send the messages. The city is the Kubernetes cluster, buildings are the nodes, and the apartments are pods of each microservice, while the residents are the service containers within the pods. And that phonebook is like a configuration file for the applications in a container where they provide information of all service endpoints with service name and port number. So basically where they have a list of all the services they talk to. Okay. So that's what we are working with. No Consul no service mesh in our imaginary city yet. Now let's see how our city residents or pods operate without a mesh. If a resident moves to another building, all residents who were sending messages to her need to now update their address book with the new address. Otherwise, they will be sending the messages to the wrong address and wonder why they never get an answer back. And this would happen when a microservice or database service gets a new endpoint or service name or port changes. Now the city is managed centrally, like when someone is administering Kubernetes cluster. So the administrators want all communication data going through the services to be transparent and gathered in one place so they can identify if there are any issues in the communication between the residents and fix those issues to see maybe how secure the city is, how responsive the residents are to each other, and so on. So this residents need to keep a protocol and report the city monitoring service about their communications. Each and every one of them needs to do that. So making sure each resident does their job properly and consistently and while keeping up with the messages, they do all these other administrative tasks as well, which is overwhelming for a lot of residents. So they're working over time. I know this sounds like a weird city with surveillance system monitoring its residents and making them work 24 over seven, but we are in a Kubernetes city, so it's fine. And in Kubernetes cluster, this is equivalent to adding a monitoring endpoint in our applications to expose metrics. So we can scrape and collect those metrics in Prometheus, for example, or adding logic to each microservice for how to handle communication with other services when they don't respond or when they get an error reply, like how do they retry the request, and so on. And some residents might miss to track some data. Some of them will just track part of the data and not all of it. They may all write in different formats or not readable handwriting's, so the central service will not be able to fit those reports or that metadata about the communication itself together, because they're all in different formats. So essentially anything related to communicate. With other services, making sure the addresses and endpoints are up to date, proper handling of when they don't get a response back or if there are any communication issues, and so on. The residents are responsible about all these themselves. Now to optimize this city and release some workload from our residents and let them focus on their main tasks. As city administrators, we introduce a service mesh like Consul. Basically in every apartment for every resident, we add a personal assistant. This assistants or agents are now saying to the resident, I will send all those messages for you. In fact, you don't even need to know the exact addresses of other residents that you are talking to. You just write their name on the envelope and I will find out where they live, and I will deliver the message. And when they respond back, I will receive the incoming messages and forward them to you. I will also keep a protocol of any information that goes through me. So all these administrative tasks around sending the messages are taken care of by those agents or assistants, and residents can focus on their main activities and the actual contents of the message. In Consul this assistants are envoy sidecar proxy containers injected into the pod of each service. As you know from Kubernetes, we have a service container running in Pod, and we can run helper or sidecar containers that will run alongside to support that main service container in its job. So these envoy proxies will act as those assistants living in the pod along the service. And by the way, I have another video where I explain service mesh with example of Istio. I explain the same concept but from a different angle, so you can check it out as well. To get even better understanding and compare two different service mesh tools. Also, if you are new to these concepts like running microservices applications in a complex Kubernetes environment, I actually have a complete DevOps bootcamp where you can learn all these with practical hands on projects as well as separate course focused specifically on microservices with mono repo, poly repo structures, and building a CI CD pipeline for the microservice application on the git. And in our latest program about DevSecOps, you can also learn the security focused approach of working with containerized applications, Kubernetes cluster, automating security and compliance checks for applications, and also learn about the production grade deployment of microservices application with a service mesh in Kubernetes using the security best practices, along with tons of other concepts. So if you are at the point where you want to take your engineering career to the next level, definitely check out our courses and trainings to really dive in and learn these tools and practices properly. And all these for a fraction of the price of what engineers with this skill set earn in salary anywhere in the world. So how do these assistants do their job? Because now they need to have an address book and a way to keep the protocol of things, instead of each resident having its own address book. Assistants actually have a shared network with a shared address book, so each assistant will add the information about their resident or their service and how to talk to it in this central registry so other assistants can read that information as well. So when a new pod gets scheduled with a new microservice, it will get assigned the assistant automatically. So proxy will be automatically injected by service mesh and proxy will say to all the other proxies or the shared network. Hey, we are new here. Me and my service. And this is how you can contact me if you want to talk to the service that I'm assisting. And I will then forward your message to the service. Now, when any service wants to talk to any other service, they can say to their personal assistant. Hey, I want to send this request or message to this service called payment. Please deliver it. The proxy looks at the shared registry to find the location of the intended service based on its name or tags, and it will send the message to the services address, where the agent or assistant of that service will open the door and accept the message, and that agent will then deliver it to the actual service inside the apartment or pod. So essentially that means services don't need to know each other's endpoints at all. They have the assistance for that. So we free the individual services from having to even know this information and extract it completely into the service mesh. And when we have new tenants in new apartments and when old ones move out of the building or the city, agents update this information dynamically. So instead of a static configuration file with endpoints, we have what's called a dynamic service registry that is always kept up to date by those Consul agents. Now let's say a resident gets sick like they get a burnout from too much work, in which case their assistant will update the information in the registry and say my service is sick. They can't receive and reply to any messages for now, and I will let you know when they're healthy and responsive again. So now if we have pod replicas of the same service on same or different nodes represented by buildings, other proxies will know to talk to one of the other healthy replicas of that service and not send the traffic to the unhealthy replica by reading this health information from the shared registry. And again, all of this is handled just between the assistance services. Don't need to worry about handling any of this logic, or keeping up to date with which service replicas are healthy or not, and trying to retrieve these health information from somewhere. They're completely unaware of all this. Now, let's say while agents are carrying these messages back and forth between services in different buildings, some malicious actors like hackers managed to sneak into our city. So they entered our Kubernetes cluster or our infrastructure and got access to our network somehow. Now we have these malicious actors on the streets roaming around freely who want to sniff these messages being sent between services, especially if they contain private, sensitive data. Maybe they want to steal payment information or personal user data. Maybe they want to mess up the systems. So if they snatch the envelope from the agents, open it and read it, they will see all the information inside. So ideally we want to encrypt that communication between the services. So even if hackers managed to get into our system and network and they were able to see those messages, they can't understand anything because instead of plain text, it's all encrypted and only our agents can decrypt them because they have the decryption keys. So that's another feature. Service mesh offers encrypting end to end communication between services using mutual TLS without mesh. If you had 20 microservices and you wanted to implement encryption between them, you would have to change the application code in every single service to implement encrypting the data or receiving encrypted data. Terminating encryption. You would have to implement the management of the encryption keys and certificates to make sure that they are also securely created and stored. And it's a lot of work on the development side. Again, that extra administrative work around secure communication that has nothing to do with the business logic directly. Plus, mostly these are the things that developers are probably not the most knowledgeable in and not the best at implementing this stuff. You kind of need specialized knowledge to implement this with proper security. So the fact that you get these out of the box in service mesh is pretty powerful. And this is interesting, the microservice itself is still sending the traffic, which is unencrypted, but before it leaves the apartment or pod, it's captured by the proxy. The proxy has a TLS certificate with encryption key to encrypt the message. So when the request leaves the pod, it's fully encrypted. When that encrypted request reaches the target service or in Consul term, upstream dependency of that service, that services proxy will then receive the message. And we'll do that TLS termination before routing it to its host service, which basically means it will decrypt the message with its encryption key and pass the plain text message to the microservice within the pod. So now, even if someone infiltrates did Kubernetes network and was sniffing the traffic between the pods, they won't be able to read the messages because they are all encrypted. And again, the microservices themselves have no idea that all this encryption decryption is happening. From their perspective, they're just sending and receiving unencrypted messages. The biggest advantage of service mesh is that all that functionality is built in the mesh itself, which means it doesn't matter how your applications are programed or how other people's third party applications are programed over which you anyways have no control. You can still use all these like end to end encryption and error handling, etcetera with Consul without relying on the applications implementing this logic or having support for TLS, for example. And that is super powerful and helpful when you're operating complex, heterogeneous systems. Now, these end to end encryption or mutual TLS between services gives us one more thing. Since each service proxy gets its own individual certificate to establish secure connection with other services. This individual certificate can also be used to uniquely identify the service and validate its identity. So, for example, each resident or service gets their own unique stamp or certificate. So when they send the message, the assistant stamps the envelope with the stamp or encrypts the message with their key. So when the receiving agent or proxy gets the message, that agent can verify with the central registry, is this stamp real or fake? Was it tampered with and which service does it belong to? So we know this message really comes from the proxy of the payment service. For example, since it's signed by its certificate, we can now use this information to define rules about who can talk to who, like define whether payment service is allowed to talk to user authentication service and frontend service is not allowed to talk to the database service, for example. So after the identity is verified as a second step, proxies will check the communication rules. So this is really a payment service sending this message that's verified. But is it actually allowed to talk to my user authentication service. So is this resident allowed to talk to my resident or does it maybe have a restraining order if proxy sees oh, it's not supposed to be sending message to my service, then it can block the message and the connection so it won't be forwarded to the service at all. If the rule allows it, then everything is verified, so it will forward the decrypted message. This is also called micro network segmentation. So instead of having a firewall on the security group level or a subnet level, we have firewall on an individual service level, which gives us a more granular control of who can talk to our services on which ports. ET cetera. That's why the term micro network segmentation. Now remember, city wants to have protocols of who is talking to whom, especially when we limit those connections with strict rules. We want to see who is trying to break the rules and talk to the services that they are not supposed to be talking to, or generally which tenants are unhealthy maybe? Or who is sending and receiving? How much traffic? Are there any bottlenecks in the system? What is the error rate and what error responses are we getting from different services? Maybe a few services are getting too many requests and are overloaded and on the verge of a burnout and proxies by being located exactly in that traffic path where the data exchange is happening, automatically end up with rich telemetry data, which they can then expose to an external system like Datadog or Prometheus. And here's a great thing about proxies being the ones that collect and expose this data. Consul proxies are all the same service, which is envoy proxy. So when they collect and expose the metrics in different services, they all do it in the same way because it's the same application. So they collect and expose the same metrics in the same format across all services. So it's easy to put together metrics of all services and build unified dashboards from them in Prometheus and Grafana, which are the monitoring tools. So this architecture gives us immense power to change and control things in the network without having to do any changes in the applications, which means we are flexible to do whatever we want very fast, and configure things very fast in a unified way for all the services. Now you're probably thinking proxies have a shared address list of all other services. They have certificate data and they know who can talk to who based on the rules configuration that they all share access to. So the question is how do they get all this data? Or when a new proxy starts up, who provides it with all this information? And where is this shared database and storage of information and certificates? Where is it located? And that's where the Consul servers come in. With our analogy. Imagine these assistants all worked for the same company, and they had a headquarter office in the city in its own building, separate from the assistance. This office is the Consul server. You can have a single room in a building, like a single Consul server instance or a single pod replica. But if you are managing many services and their proxies, you might need a bigger office. So maybe 3 or 5 Consul server pods. So these Consul servers push out all the needed data to the proxies or Consul clients, like service registry information, the configuration certificates. So we don't have to do anything to get certificates in all this data to the proxies. All of this is done and managed automatically by the Consul servers. So we have basically automated operations of the mesh itself. And as I mentioned, those personal assistants have a network. They talk to each other exchanging information and so on. And that network of personal assistants is called data plane. And the central office or cluster of Consul servers that manage these assistants network is called control plane. This means the data plane is managed centrally by Consul servers or the control plane, so that they too can focus on doing their job of handling the communication between services and if something changes or gets updated, like the address of a service or new service gets added or removed, certificates get rotated, they will get the update from the central office automatically. It's like these proxies are all working for the same organization, having access to the centrally managed resources and data so they can all do their work easily. And this control plane leaves separately, maybe in the same city, which would be the same Kubernetes cluster. Or maybe they even have an office in a different city, which means you can spin up a dedicated Kubernetes cluster just for the Consul control plane, and then connect it to the Kubernetes cluster where the data plane is running. Now let's say we have multiple Kubernetes clusters with our microservices like in different geographic regions, maybe replicated on different cloud platforms even. And this is like having allied cities or city allies where cities form a network and decide, you know, let's form a partnership. So we will allow your services to talk to ours and vice versa. In this case, you can have Consul control plane in each cluster. So own control plane office in each city. Or again you may have one dedicated cluster or the main headquarter Consul control plane. And it will manage all other clusters data planes from there which is a common setup. This way you can avoid replicated offices and resources. For example in Consul is especially powerful in such multi cluster multi data center environments. Connecting services across different environments which can be a really big networking and security challenge if you're doing this without a service mesh tool. So how does this happen with Consul? Think of Consul planting guards at the exit and entry of the city in Kubernetes cluster. This guard is called a mesh gateway. So if payment service from cluster one wants to talk to user authentication service in cluster two, it will be the same process for the services where the payment service just says to its proxy, hey, send this message to the user service, please. I don't know where it's running. Also, I don't care. You will figure out how to deliver this message. Proxy will have the list of available services provided by the Consul server, including services in all allied cities where it says okay, this user authentication service lives in another city, so it could hand the message over to the city guard. The guard will take it to the other cluster and hand it over to the guard at the entry of that cluster, which is going to be the mesh gateway of the second cluster, which will then deliver it to the proxy of the user service inside the cluster. And finally it will be forwarded to the user service within the pod itself. And the response will flow the same way back to the payment service. And we're going to see an example of this specific use case in the demo part, where we will connect two Kubernetes clusters on two different cloud platforms with each other using Consul. And the good thing is, it doesn't matter which cloud platform you use. It pretty much works the same all the time. Now when we talk about multi data center environments, it's not just Kubernetes cluster. Many companies, especially large established companies, have tons of applications that run on legacy systems directly on VMs. Or they have a large company wide database that database engineers team is managing centrally that also run on VMs. And often that team already has a strong expertise of how to manage and operate those services on the virtual machines. So the overhead of learning Kubernetes and then learning how to migrate and operate the service on Kubernetes is often too large. Or if it's a small legacy application, maybe the overhead is just not worth it. So these are real use cases where companies still have services that will run on VMs and may not be migrated to Kubernetes or cloud anytime soon, but these companies and projects still want to take advantage of the modern tools like Kubernetes and containers. So the teams in that company deploy their microservices in Kubernetes cluster, which now has to connect to the database on the VMs or connect with other legacy backends still running on premise virtual machines. And if connecting multiple Kubernetes clusters is a challenge, try throwing VMs in that mix that becomes even larger. Challenge of how do we connect those networks? And again, service mesh tools make that easier by abstracting this low level network configuration and letting you manage that on a service mesh level. What's great with Consul specifically is that while other service mesh tools also have this capability, Consul actually treats the VM environment with the same importance or as a first class citizen, same as the Kubernetes environment, and doesn't treat it as an uninvited or undesired guest serving it just because it's there and it has to. So how does Consul work on VMs? If we use our analogy again, an on premise data center would be its own city with a bunch of private houses where each house is a VM, the application or service will be the only resident in the house and the Consul proxy and. Consul client will be living in that house along the resident, and we will have its own house for the Consul server as main office. You will then configure the communication channel so that Consul server running on the VM can connect to the Consul server in the Kubernetes cluster, so they can share information and create connection channel for their residents. So now again you have Mesh gateway in the Virtual Machine City as well. Who will communicate with Mesh Gateway in Kubernetes cluster. This way it can connect to other allied cities like other VMs or Kubernetes clusters. And once the trust and secure communication channel is established between them. Now the residents of both cities can talk to each other through that secure channel. So again, now with payment service in Kubernetes, cluster wants to talk to the database on VM. They go through the same exact process where payment just says to its proxy, hey, send this message to a database please. Proxy will have the list of available services provided by the Consul server, and it sees their database lives in another city. So through the mesh gateways the message will get transported all the way to the database running on VM in a different data center. So as you see, a service mesh like Consul is essentially the whole infrastructure layer that has different features to solve these challenges of operating microservices applications and enabling communication between them. So in this demo part we're going to create a Kubernetes cluster on AWS using ECS service. So that's going to be our very first step. Once we have the Kubernetes cluster we're going to deploy a microservices application with lots of services inside the cluster. And we're going to use an open source microservices application project from Google. So it's a little bit more realistic, like a more complex microservice. And not just two services for the demo. And once we have that all set up, we're going to deploy Consul on ECS, and we're going to see how the proxies will be injected in each one of those microservices. What configuration changes we're going to have to make to the microservices Kubernetes manifest files in order for Consul to work in Kubernetes, and also explore a couple of features of Consul and what it gives us out of the box. Once we have all of that set up, we're going to create another Kubernetes cluster on a different cloud platform. And we're going to use fairly simple Kubernetes managed service on Linode Cloud platform. I like using because it's super simple to spin up a cluster there to create it manually. It's just very little effort compared to ECS, and it's also very fast. So we're going to use that as a demonstration for another Kubernetes cluster. But it could really be any other Kubernetes cluster that you want. So the concepts are the same. And then in that linode Kubernetes managed cluster, we're going to deploy the same exact microservice and same exact Consul configuration. And once we have that we're going to connect those two clusters together using Consul. And we're going to see the demo of or simulation of a service going down or crashing inside the cluster. And it failing over to the same service inside the Elk cluster. So basically a multi cluster environment with a service failover to another Kubernetes cluster. So let's go ahead and do that step by step. Where along the way I'm going to explain lots of different concepts related to Consul service mesh. So the first step is we're going to create an EKS cluster. But of course we don't want to do that manually because it's a lot of effort and it's not the easiest thing to do. So we're going to use infrastructure as code using Terraform and all the code configuration files, the Terraform script, the microservices application, all of that will be linked in the video description. So you can clone those repositories and follow along. So this is one repository where I'm going to have all my Kubernetes manifest files that we're going to use to configure Consul and deploy our microservices application. So all of that is going to be here. And we have the Terraform folder inside with the Terraform script for creating the EKS cluster. And I have this repository cloned locally so that I can work on it using my code editor with terminal. So this is where we're going to be doing most of the work of configuring stuff. And I also have my account ready where we're going to be creating the Elastic Kubernetes Service. Right now we don't have any. So let's create one. And before we do, let's actually go through a little bit of the Terraform script and what we're doing here. It's pretty straightforward actually. I'm using the modules to make my job easier. So I'm using the VPC module to create a new VPC for the EKS cluster. Our EKS cluster will be publicly accessible. That's very important. And therefore we have the public subnet in addition to the private subnet in our VPC. And then I'm just using the X module to create the cluster obviously referencing this VPC. And we're basically configuring it with a bunch of parameters to configure our cluster. So first of all as I said I want my Kubernetes cluster to be accessible externally. So with this attribute we can actually create a public endpoint. Or we can let create a public endpoint for our Kubernetes API server. So we can connect to the cluster using kubectl for example or browser whatever from outside the VPC. Right. So I'm setting this to true. To achieve that the next attribute is adding some security group rules. And this is actually important for Consul to be able to do its job. And there are some specific ports that we need to open on the worker nodes themselves, where the Consul processes will be running in order to allow Consul components to talk to each other, and for the Kubernetes control plane components to reach Consul processes as well. And I'm actually going to reference to the list of ports that we need to open for Consul. So you see what they are and why those ports are needed. However, just to make things simpler and to make sure that you guys do not have any networking issues with Consul, and just to make sure that things go smoothly, what I'm going to do is, as you see in the Terraform configuration itself, I'm actually going to open all the ports on my worker nodes, and I want to stress that I'm actually doing it for the demo, because the security best practice is to have only those ports open that you actually need exposed, and only those internal or external processes that need access to whatever service is running on that port needs to have access to that port and nothing else. However, this is a demonstration, and I just want to make it easier for you guys to follow along and to make sure you don't have any networking problems when deploying Consul. And finally, this is the managed node groups. So these are the actual worker nodes or the worker node group configuration for the cluster. And here we're basically just choosing the small instances. And we're going to have three nodes or three of those instances in the cluster. And finally we have these two configuration pieces which are also needed for Consul deployment. So these are basically the configuration for the dynamic volume configuration because Consul is a stateful application. So it actually deploys a stateful set component and it needs to store and persist some data. So it needs to create the volumes on whatever platform it gets deployed. And in this case, we are making sure that creation of Amazon Elastic Block Storage is enabled for the cluster by giving permission to processes on these nodes to create the storage. And in addition to this role, we have to enable what's called an add on on EKS cluster, which allows for automatic provisioning of the storage. And once the cluster is created, I'm actually going to show you all this information so we can see that visually as well. Apart from that, we have variables that we are setting and I have added some default values for most of the variables, so you don't have to set them in the TF vars file. So these are basically just Cidr blocks for VPC private and public subnets. The Kubernetes version. That's very important to make sure to choose the one of the latest ones. This is the latest one as of now. So that's what I'm using the cluster name because we use that in a couple of places within the main configuration. So I extracted that as a variable region. You can set whatever region is close to you. And there are two pieces of information or variables that you have to set yourself to execute the script. Everything else is configured and set already. So before you are able to execute this Terraform script, make sure to go to your AWS account and for your user, create an access key pair and you have to set those values for the Terraform. Object. So I have defined them and referenced them right here in the provider configuration, which means I can just set those variable values in my Terraform dot vars file, which I have done already. And once you have that, you should be good to go and terraform. Tfrs is a simple text file with key value pairs. So you have the key name, which is. This one equals whatever your key id is in quotes. And same for the access key. So set those two values in the tf vars file and we are good to go. That's the provider configuration. That's the version that I'm using. And now we can actually execute this Terraform script to create the EKS cluster. So I'm going to switch to the Terraform folder and I'm going to do terraform init. So terraform init basically downloads any providers that are specified here just like you download dependencies of your code. For example in order to run your project. As you see it creates this dot terraform folder where the modules and providers will be downloaded. Those modules and this provider and everything is green, which means our Terraform has been initialized. We also have the Terraform log file. And now. We can execute Terraform apply. You can do Terraform plan for the preview. But I'm going to do terraform apply immediately. And we have to pass in the variables file that defines any missing variables. So. VAR file is terraform tf vars. And let's apply. And this is our preview. All the things that will be created. I don't need to look through that. I'm going to confirm, and this is going to take a couple of minutes to create everything, because lots of components and things are being created. And once the cluster has been created and fully initialized, we can continue from there. So the Terraform script was executed and it took some time, but it successfully executed. So now if I switch back to my AWS account in the region that you have basically set in the variables file, I chose the EU central region, which is closest to me. So this is my EKS cluster that was created in the Frankfurt region. And if we go inside and check out the detailed view, I'm going to show you a couple of things that we have configured in our Terraform script that we can see in the UI as well. So I want to point out a couple of configuration details. First of all we have the cluster configuration details. So we have the role the IAM role for the cluster which is this one right here, as well as security groups for the cluster like this. And then we have the configuration details on the node groups or the worker nodes themselves. So if I go to compute we're going to see the three nodes that were created because that is our configuration. We have defined three nodes. So these are basically the work node instances that are running in our account. So if we go to EC2 dashboard, we're going to see these three instances here. And we have the security group configuration on the worker node level, which is this one right here. And note that this security group additional rule actually applies to the worker nodes. So this is the same security group that all the nodes share. So it will be same for each worker node. And we have also configured this additional policy for the IAM role which also applies to the node groups. So now the cluster the EKS cluster role or the control plane role. But the node group role that apply to the worker nodes. And again we can see that right here in the instance configuration. This is the role. And we should see this Amazon EBS, CSI driver policy listed here. And I'm pointing this out because first of all, you need to understand that these two things are configured separately. You have the control plane configuration with EKS which is actually running in its own network. And then you have the worker node configuration with its own role on port's own firewall configuration and so on. So if you have any networking issues and so on, this should help you troubleshoot and know where to look for things basically. And finally last thing I want to show you is these EBS CSI driver. Add on that we activated on our cluster. And you are going to see that in the evidence tab for the cluster. And right here we have Amazon EBS CSI driver which basically is needed in order to automatically provision the elastic block storage for persistent volumes inside the cluster. So in our case Consul stateful set actually needs a persistent volume. So this allows the cluster to automatically provision the Amazon block storage for those volumes. Awesome. So the cluster is already active. So we can connect to it using kubectl and deploy our application inside. So I'm going to switch back to my code editor. And I'm actually going to use the terminal here. So we have everything in one place. And we don't need the Terraform script anymore because we executed the provisioning already. So now the next step is to actually connect to our EKS cluster. And we do that using AWS command line interface. So this is basically a secure way to retrieve a cube config file from the EKS cluster without exposing any credentials and without having to download this kube config file, and so on using a simple command, which means you have to have installed. If you don't, it's pretty easy. Just go ahead and install command line interface on whatever operating system you have. And once you have that, you need to also configure your CLI to use the access keys, which can be the same access keys that your Terraform is using for this demo use case, because command line interface will need the access credentials to connect to the AWS account. Right? And I have already configured all of that with AWS configure command. So just make sure that the default region and credentials configured here are for the same account as for Terraform, and you should be good to go. So. With that setup I'm going to execute AWS X command. Update. Kube config. And you can actually provide the region here as well for where the cluster is running. So central one and we are going to need the cluster name as well. And we have that here we call the cluster. This generic name. So I'm going to copy the cluster name. So basically what update kube config subcommand does is it fetches the cube config file which is like a credentials file for Kubernetes cluster from AWS. And it stores it locally into a default cube config location, where cube CTL will look for it and the location is on your user's home directory in dot cube folder. So after executing this command you should find cube config file in there. So let's execute. And there you go. You see the output that the cube config was. Or the context of the cluster was added in this location in dot cube slash config. So if you don't have the dot cube folder already, it will basically create one and add the cube config file configuration in there. Or if you already have one, it will just append to the existing config because you may be connected to multiple clusters. So all of those configuration will be right here. That's how it works for Kubernetes in general. So nothing is specific here. And that means we now should be able to connect to the cluster using kubectl command. So let's see. Kubectl get node. And there you go. We have our three work nodes with this Kubernetes version which we have defined here. Awesome. The first step is done as a next step. We want to deploy microservices application into this Kubernetes cluster. So for that I'm going to actually switch to the Kubernetes folder where I have my manifests. And I'm going to close this up and expand this. So these are all the config files we're going to need in this demo. But we're going to start with the simplest one. So that's all we need to deploy our microservices. So actually we don't need the repository of the microservice application itself. We just need a reference to the images. So this is a Kubernetes config file that references images of all those microservices. But of course I'm going to link the microservices repository in the video description as well. So this is an open source microservices demo repository from Google. And all those images are public which makes it easy to use it for demos. And this currently happens to be the latest version. If the version has changed, you can check in the provided link and you can just update the version basically. So super simple actually, we just have a bunch of deployments for each service. The configuration is pretty similar for each microservice. They just run on different ports and have different names. All of them have cluster IP services which are basically internal services. And we have one entry point microservice, which is the front end that will then route the traffic to all the other microservices. And here we see basically the frontend talks to all other services, and it is the only one that has an external service of type load balancer. And all those microservices also share a Redis memory database. Again, pretty simple setup. Nothing crazy here. So that means once we apply this configuration file, all the images will be downloaded, all the pods will be created, deployment services and so on. And we're going to have one entry point service to the cluster through load balancer. Now for simplicity I'm not going to deploy an ingress controller in the cluster. So we're just going to use the load balancer service directly to access our application, which is going to be enough for our demo. So let's go ahead and apply this config file. Config dot Yaml. That's all we have to do. And by the way, there is one service that is misconfigured slightly on purpose, which is a payment service. It basically is missing one environment variable. So it's not going to be able to successfully start in a pod, but we're going to need that to demo that later in Consul. So let's execute and see the result. And we have the output of all the stuff that was created. So we have quite a few microservices here, and it will need a little bit of time. And we're going to check kubectl get pod. So they will all be created in the default namespace. So if we do kubectl get pod we should see all our pods are up and running except for the payment service which is going to stay in the error state which is fine. We're going to use that as an example of an error in microservice to see it in Consul. Okay. So that was pretty easy. Now what we actually want to see what we deployed. So I'm going to do kubectl get services. And as you see all our services are cluster IP type except for the frontend external. That means if you have watched my other Kubernetes tutorials, you would know that load balancer external service gets the internal cluster IP, but also the external IP because we need to be able to access it externally. And this is the external domain name for the load balancer that will then map to the external IP. So where does it come from. That's also pretty easy as it works in Kubernetes. When you create a load balancer on whatever cloud platform, you are creating this load balancer service. It will in the background use that Cloud Platforms load balancer service to create the native load balancer there. So that's where the external IP comes from. And that means if I switch back to AWS and go to EC2 service. That's where we have the load balancers. We should see our load balancer for the front end external right here. And this is basically the DNS name that we see right here right ending in 138 configured on Http port forwarding to this port which is configured right here. So pretty simple. We just grab this DNS or external DNS name. And since port 80 is the default Http port it will just open it like this. And there you go. Our microservice is deployed. Now the next step is to actually deploy Consul in our cluster and use Consul for our microservices. So how do we deploy Consul. There are several ways to install Consul on Kubernetes cluster. One of them is using Consul's Kubernetes CLI and another one is using Consuls. Official helm chart for Kubernetes, which is what I'm going to use. So if we search for Consul, helm, chart and open the installation guide on their official documentation, always try to refer to the official documentation instead of some blog posts because they are most up to date. So for the latest version, these are the instructions. We basically add the HashiCorp helm repository. We install the Consul chart and provide any parameters we need. And if you have watched my helm chart videos, you know that helm charts are configurable so we can actually provide any parameters, any configuration options in order to configure the service using those parameters. Right. So for example, with Consul service mesh I mentioned it has multiple features. And depending on which features you actually need, you can enable them by configuring them as the chart values or passing them as chart values, and to see what values are available to be set and parameterized. Obviously we need to see the values yaml file. So we have a chart reference here. So that's basically the chart values reference documentation. I usually actually prefer the values stored file in the repository. So for example this one. But as you see the repository has been archived. So this documentation is up to date and something that you should reference. But even though it's an advantage to have the chart highly configurable so you can actually tweak it to whatever desired configuration you have in mind. If you don't know what you're doing, there's this huge list of values that you have to basically understand what they're doing. And it's pretty difficult to understand all these configuration. So we're going to use a couple of these configuration options. And I'm going to explain to you what they're actually doing. But if you need any additional configuration options you can find those listed here with descriptions. I don't think it's the most comprehensive and understandable, but at least you have some reference point. So let's switch back to the project. And I'm going to walk you through the values file that I have configured for our specific Consul installation. Which is actually pretty simple configuration. We have this global attribute which applies to all the components that are part of the chart. As you know, chart usually holds or is a bundle of multiple components of different Kubernetes native components as well as custom resource definitions and so on. So this applies to multiple components. First of all, we have the Consul image with the version. We are enabling TLS communication between the components and services. And since we want to connect multiple clusters or multi data center environments basically with each other, we are also enabling the peering. Then we have the Consul server configuration. As I explained, the Consul server is the control plane that manages all the proxies, the Consul client and so on. And usually in a production environment you want to have at least three replicas, not just one, because if one replica dies, you want to have a failover. In our case, we're just going to use one server for our demo purpose. So you can configure that here. This is an important configuration option. Connect inject is basically the name, the technical term of the service mesh functionality of Consul. And what this configures is basically if we enable connect inject it will allow Consul to automatically inject the proxies, those helper containers, as I mentioned, into the pods of services. So that's what enabled true does. And then we have the second configuration that says default false which is actually the default value. But I wanted to specifically configure this. So if this is set to false then you would need an extra annotation inside the Kubernetes manifest files or deployment manifest files in order to actually inject the proxy into that pod. If we set this to true as well, like this, then it will actually inject those proxies, even if we don't have the Consul annotations in the deployments. And this is a good thing to have control over, because it could be that you have pods running in your cluster, that you don't want to have any proxy services in them, or you need to, or maybe you have some namespaces that should not have any Consul proxies applied. So if you set it to false, you basically decide per deployment per application where you want the proxy injected. So I'm going to set it to true. So we can see how that works. And we actually want all our services to have the proxy injected. And we don't want to add annotations one by one on each deployment. So I'm going to set it to true. And that's the connect inject configuration. Then we have mesh gateway, which as I explained is basically a connector between multiple environments. So mesh gateway is like a guard that is standing at the entry or exit of the city with our analogy. And you can also have multiple replicas of the mesh gateway, because if one of them goes down or has an issue or maybe becomes a bottleneck because of the number of requests, you can actually scale up the replicas. And again, it's a feature that you can enable if you actually need to. So if you don't have a multi cluster environment or just for security purposes, you need to stay within the cluster, then of course you want enable it. And finally we have UI which gives us a Consul, dashboard or UI where we can see the services and where we can even configure stuff which we are actually going to be using in our use case and type load balancer basically means that it will create a service, an external load balancer type of service, so we can access the UI. And we are also enabling that. So that's the configuration that we want to apply to Consul in our cluster to one, or inject the proxies in our microservices pods. And second, allow us to connect to Kubernetes clusters with each other and also have a dashboard where we can see stuff and configure some things. So I'm going to go ahead and actually install the Consul helm chart with these values. And let's see what we get. So I'm going to copy the first command and let's add HashiCorp repository. There you go. And now we can actually install helm install. And we're going to give our Consul chart deployment a name. You can call it whatever you want. I'm actually going to call this X to basically differentiate it later from the Linode Kubernetes Engine deployment. So I'm going to go with that name. And then obviously we need the chart name. I'm going to copy it from here. And actually when components are created it prepend Consul in the name. So that's why I'm not using X Consul or Consul in the chart name itself. And I'm actually going to add some additional parameters which can also be configured in the values. But I'm going to set them separately because I'm going to use the same values file for the Consul deployment as well. So I'm going to pass in the version first of all. And we're going to use. Version one. And then of course, we have to pass the values file. Consul values dot yaml. That's our file. And finally the last configuration is I'm going to set. A global config. So one of those global attributes called data center. And that's basically the name. So you can name the environment where Consul is running. You can give it a name of a data center. And I'm also going to call this x. And that's basically it. So I'm going to execute this command. And this should install all the Consul components in our cluster. And this gets executed pretty quickly actually I'm going to do kubectl get pod. So first of all I'm going to check the pods. And now that I'm deploying the Consul components in the same default namespace. But you could also have them in a separate namespace, especially if you have multiple applications. That would make sense. Let's check again. So we have these four pods. The first one is server. Obviously we need the server or the control plane to manage the proxies to inject the proxies and so on. And the chart name was taken as the prefix as you see here for all those pods. Then we have the mesh gateway. One instance of it we have the connect injector that is the one that is responsible for injecting pods. And we have the webhook certification manager. And as you see all the pods are running successfully. You can also check any other components that were deployed. So we have this stateful set which is a server itself and those deployments and we have the Consul services themselves. And one of them is we saw is the Consul UI which is created as a load balancer service type. And it also gets its own load balancer component on AWS with its own DNS name. So we can use that to access the Consul UI. So let's go ahead and do that. And I'm going to make this a little bit broader. And one thing I want to point out here is that this external service of Consul UI is actually accessible at the Https port 443. That means we have to access the service using Http as protocol. Like this. And of course our browser doesn't know the certificate which is signed by Consul CA so we can say it's all fine, we allow it. And there you go. This is our Consul UI. And as you see we have only two services displayed here. So basically Consul now is aware only of two services. One of them is a mesh gateway which we deployed as part of Consul. And the other one is Consul server itself. That means another interesting point. The proxies have not been injected yet in any of the microservices because we need to restart or recreate those services or deployments. So that's what we're going to do next. So I'm actually going to delete our microservice deployment completely. And I'm going to redeploy it with a little bit adjusted configuration. So let's do config dot Yaml. There you go. And let's see. Looks good. Now, I actually have already prepared a config file with a couple of changes for Consul deployment specifically. So I'm going to open this file and let's go through it and understand those changes. So first of all in order to configure anything in our deployment that is Consul relevant. So most of the things that are actually relevant for how Consul will treat our deployments, or whether it will inject proxies or how it will handle the communication between services, etcetera, we can configure those using the annotations is, you know, annotations are part of metadata of Kubernetes components like deployments. And these are going to be the annotations on the pod level. So inside the template metadata in the annotations we can add the Consul annotations basically. So this will communicate our desired configuration of our microservice sees to Consul, which is a pretty easy way to manage that using the Kubernetes native way. So the first annotation, which is probably the most used one, most seen one is connect inject. True. So basically this specific configuration is not relevant for us because we set the connect inject default to true. Basically, as I said we can say we don't want auto injection in every part in every namespace. We want to be able to decide which pods actually get those proxies. And we decide that by adding this annotation to the pod metadata, right. So every part that we want to have proxy injected, we can add this annotation and it will take care of it. However, when this is set to true which is what we configured. And that's what Consul that is deployed in our cluster actually knows, we don't need that. So I'm just going to comment this out. Let's go to the next microservice. So we have the same connect inject. We don't need this. And this is another annotation called Connect Service up streams which is another core annotation which basically defines which services does this service talk to. And how are those services called. Remember I mentioned that once the proxy is there, the service inside the pod does not or should not care about where the destination services are located and what their addresses are. They can just hand over the request with the name of the service, and then proxy will figure out where that service actually runs in. Kubernetes services are already referenced in this easy way, using the service name and port instead of static IP addresses. So this is basically not a huge improvement in this case because Kubernetes already manages that. However, we do need to communicate to Consul which services this one will be talking to. So that's kind of the metadata definition. And when we have this configured there is one more change we're going to do here, which is there is an environment variable that points to the service that recommendations service is talking to. And in the previous configuration this was actually the Kubernetes service name and the port. And again going back to my previous explanation, the service does not need to know what the Kubernetes service name of that microservices that he talks to because it talks to its proxy. And the proxy will listen to it to this request on localhost because it's within the pod. So the containers within the pod communicate via localhost and the same port where the upstream service is configured. So this request will basically go to the proxy instead of the Kubernetes service. But proxy knows that whatever that request points to is located here, which is the Kubernetes service name. So as I said, it's not like a huge game changer here because Kubernetes already manages the service names pretty well, but that's basically it. So that's the annotation that I have configured in services. That's the only annotation we use here. So I'm going to scroll through and comment out all of those connect inject annotations just to demonstrate that we don't need them. Like this. So I basically just commented out that annotation, but I'm still going to leave it in the configuration file for your reference in case you want to use that as well. And then we have two last services that have a bunch of upstream services they talk to. And it's the same idea. You can just provide a list with service names and ports. And then all of those will be accessible on localhost on different ports. So basically the service the checkout service will always be talking to the proxy instead of talking to all those different services as it was doing before right here. It will now only talk to the proxy. And then proxy will forward all those requests to the respective services based on whichever port the proxy receives that request. And that means obviously those ports need to be different within the localhost and the same for front end. So front end is the last service which also has a bunch of upstream services exactly the same concept. Change the localhost here. And there is one more annotation that we are using here for front end to actually be accessible, which is transparent proxy annotation set to false. So transparent proxy or transparent proxy true is a feature of Consul that makes it possible for services to communicate with each other through those proxies, without being aware that those proxies are actually there. So they are thinking that they're sending the request to the service, and proxies are capturing those requests in the middle, but services are unaware of that. And with transparent proxy set to false, basically, we are saying that the service needs to be aware of the proxy, and it has to explicitly send the traffic to the proxy and route the traffic through it. And that's our slightly modified configuration. And I'm going to apply. This config Consul file to deploy our microservices so CTL apply. Config. Consul and let's see. And let's give it a couple of seconds for the pods to come up. And. Let's do kubectl get pod. And there you go. So let's see what we have here. These are all the pods from the microservices. And you probably notice that for each service or each pod of the service, we have two out of two containers running inside instead of one. And that second container is basically the injected proxy. And we can just log one of the service containers to see what the proxy is doing. So I'm going to do kubectl. Logs and let's just take the edX service. So I'm going to do this. So basically when you have two containers it takes a default which is the main container. So this is the microservice itself. However we want to log the proxy container logs. And this is actually the Consul data plane. So this is the proxy that was injected in the pod. And we also have an init container. So you need container as you know already from Kubernetes concepts is basically a container that starts up before the standard containers actually run. So it's kind of preparing the environment before the rest of the actual containers will run and init container exits once it's done its job. And then the other containers in the pod will run and the init container. So this is the Consul process. Basically it prepares the environments for the proxy. So remember I told you that proxy needs information about what other services are there. So that if its host service wants to talk to other services, it knows how to reach them. It also gets the TLS certificate for the secure encrypted communication. So all of that is actually handled by the init container that injects all this information into the pod so that the proxy has access to them. So let's actually log the proxy container and what it does. And we're just going to provide the container name like this. And. There you go. As I mentioned, Consul's data plane is an envoy container. That's the technology behind it. So we're seeing the envoy logs and the proxy basically on the startup. What it's doing is it tries to find the service associated with the pod. So for the edge service, for example, it will find the associated service and it will register it with Consul so other proxies can talk to it. Awesome. That means if it doesn't find any related or associated service, it will actually give you an error that it couldn't find a service associated with that pod. And this now means that all those proxies actually did the work of registering these microservice pods with Consul. So this means if we go back to the Consul UI, we're going to see all the services listed here. Because now Consul knows about them. And for each service it also shows you how Consul is aware of those services. And in our case they have been registered with proxies. And we actually have a pod in our cluster that is crashing. So the proxy is up. However the service itself is not able to start up. That's why we have just one container which is the proxy up and running. And we see that here as well. Basically it's failing the health check which means it's not accessible. This means that all the other proxies will actually know about the issue of the service, without even sending a request to it. And we can click inside the service. And what this basically displays is not which service talks to which other service, because payment service is obviously not talking to all of them, but rather which service is allowed to talk to which service. And right now we have no rules in the cluster that limit any service to talk to any other service. And that's why everything is allowed. However, we can change that. So for example, if we go back to our configuration. And I'm going to go to the payment service. You actually see that the payment service is not initiating communication with any other service. So there is no service that payment service directly talks to. However, there are services that send the request or initiate request to the payment service, which is the checkout service. So this is actually the only service actually that talks to the payment service that initiates the request. That means all of these other connections here that we are allowing are actually not necessary. So by limiting those connections and basically saying payment service should not talk to anything other than the checkout service, we are reducing the attack surface in our cluster. So that means if there was a bug in the payment service, like a huge security vulnerability, we would actually limit the damage that someone can do by exploiting the vulnerability in the cluster, because we're limiting what payment service can do within the cluster and who it can communicate to and talk to. And we're going to use the concept of intentions here. So this is the micro network segmentation that I already explained, in which Consul basically allows us to define firewall configurations in a granular way on a service level. And that feature is called Intentions in Consul. And there is an intention CRD file that you can create as a Kubernetes manifest file, which is pretty simple to configure. And we're going to create an intention where we're going to say that the checkout service is going to be able to initiate communication to the payment service, which is what we have right here, because checkout Service will be talking to the payment service, not vice versa. And that's it. No other service is allowed to do that. And let's create that. And we can create another intention that says all the services are denied to talk to the payment service. So basically by default we disallow any communication and we only allow it for checkout service as an exception. If I go back you see that this diagram has changed. And we see checkout service is the only one that can talk to it. And going back, we can also create another rule where we can say the payment service itself to all other services is also denied. So let's save this. Go back to topology. And there you go. And as you see, this is on a specific service level. However, you can do this for your entire microservices application group. And we even have a warning here that basically tells us to configure that for all the services. And you can do that here directly as well. But as I said, if you want to automate this, if you want to have that again, configuration as code, which is the recommended way of working, you would create the Crds for Consul called service intentions. So we have successfully deployed Consul in the EKS cluster and configured it to inject proxies into all the microservices, plus the ready service that we're using, which is a third party service obviously. So it lets us have the proxy application in any service, whether it's our own third party or whatever, to have that consistency in the communication between the services which is actually great because we can apply the same kind of rules on third party applications as we can on our own application, so we can decide which services can talk to the database using intentions, and we can encrypt the connection with the database the same way as we do within our own applications. Now, as a next step, we're going to repeat the same exact deployment in another Kubernetes cluster on another cloud platform. So we're going to recreate the same exact state in another cluster. And then we're going to connect those two to simulate a failover. When a service here fails that the same service in another cluster on another cloud platform can take over its job. So we're going to deploy the same application in an Elk cluster. So I'm going to go ahead and log in into my linode account. So if you don't have a linode account yet, you can sign up. But also, as I mentioned previously, you can actually create this cluster wherever you want. The concept will be exactly the same, so it's not anything that is ECS or specific. This could work with any two Kubernetes clusters. I personally like linode because it makes the cluster creation super fast and super easy. That's why use it. But you can use whatever you want. So going to Kubernetes I'm going to create a cluster I'm going to call these Elk Consul. Not very creative. And I'm going to create these also in the Frankfurt region, but it could be in a different region as well. It really doesn't matter. Just choose the latest Kubernetes version. I'm going to choose no Aicha for the control plane, because we just need a demo here. And let's choose the cheaper linode machines by switching to the shared CPU. And let's take the four gigabyte sized VMs, and let's choose two nodes. Let's confirm that. Create a cluster. And this should be up and running pretty fast. So let's wait for the provisioning of those worker nodes. Our two nodes are running and we can now download the cube config file to access our cluster. So I'm going to go back to my Visual Studio Code and open a new terminal. And let's go into the Kubernetes. Folder again, and we're going to have to export that cube config file to connect to the cluster. So right here in this execution environment basically we have executed the AWS command to add the cube config file in a default location where kubectl will look for it. So wherever I execute kubectl now in my local environment it will connect to the eks cluster because that's set in the dot cube default location. So basically we're going to configure this specific terminal session or this environment to point to the cube config file of LCC. And then when we execute kubectl commands here specifically it's going to connect to the LCC. And that's very easy. We're just going to export an environment variable called cube config. But again kubectl will pick up on and we're going to set its location to wherever that downloaded. Alki kube config file is that's what it's called. And there you go. So now if I do kubectl get node this should point me to those nodes. Very simple. So basically kubectl will first check is kube config environment variable set and pointing to a specific kube config file. If it is then that's the cluster it connects to. If the environment variable is not said, it's just going to look in the default location dot kube and try to find the cube config file there. And for security reasons we would limit the permissions on this file. So right now it's readable not only for the owner but also for group and any other user. So I'm going to do change mode. We can set it to 700. So basically remove any permissions from the file to anyone other than the owner. And that's it. Awesome. So we are connected to the cluster, which means we're going to repeat the same exact steps. Install or deploy the Consul helm chart inside the cluster and deploy our microservices application with the annotations. Let's do it. And this is going to be a command here. We're calling this helm installation of the Consul chart. We're using the same version. The same values file. And this time we are setting the data center to l k value. So calling the cluster where Consul will run. So let's execute this and let's check. Everything that was created. We have our deployments. The four pods starting up the same exact thing is in ECS. What Consul also deploys along which I briefly mentioned are the Crds. So we can also check those get CRD. And there are a bunch of kids here that are from the LCC itself, so we can actually filter. And. Let's find only the things that have Consul in it. So basically you have those service intentions that I showed you and so on. So it actually creates a bunch of crds in the cluster. So you can configure Consul and different components of Consul in the Kubernetes native way with manifest files. And you can find the configuration of all the Crds in their official documentation as well. So let's give it some time to start up. Let's see if the pods are ready. There you go. And now we can deploy. The microservices application. Apply config Consul and enter. In the same way, we see that two containers are starting up in every part of microservices, which means the proxy containers were injected in each service. And while this is starting up, let's actually. Check the services. Because with the same configuration, we have the Consul UI for this cluster as well as the front end service for this cluster. And as I told you, it doesn't really matter which cloud platform you use because the concepts are really similar. So the same way is on AWS. The load balancer component in Kubernetes actually links to the cloud native load balancer in linode called node balancer. So this one right here was created. That's the IP address the external IP address. And we see that here. This is for the frontend on port 80. And then we have the Consul UI which is accessible with Https on this endpoint. This one right here. So exactly same configuration, which means we can actually access this and see the Consul UI in LKY. And as you see, that's what we called the data center. That's the name we gave to the cluster environment when we deployed Consul. So it says okay here and here it says x. So it's going to make it a little bit easier for us to differentiate when we do stuff to connect those to. And as you see all the services are listed here. And all those parts except for the payment service have successfully started. Awesome. Now we have basically recreated the same exact environment in a different cluster, on a different cloud platform that can allow us to now have a failover. So if something happens to the cluster, we can always fall back to the cluster. However, we need to first establish that connection because obviously now these are two separate clusters. So we need to connect them so that the services in EKS cluster can communicate with services in cluster. And for that we are going to create what's called a peer connection. So we're going to make those two clusters peers. And since both of them have full Consul deployments inside on the Consul level, we are going to connect them so that services inside those two clusters can communicate with each other. And it is actually pretty easy to do. Again, there is an option to do this using the CRD components. for this demonstration we're going to use a more visual and simpler approach of Consul UI to establish that peer connection. And in the Consul values file, remember we enable the mesh gateways. These are the components that are actually going to help us connect those two clusters together. So the mesh gateway in EKS cluster will connect to the mesh gateway in cluster. So those two are the connection links. And when we have Consul clusters on different networks like we do here, completely different networks, different cloud platforms, then we're going to need to set the Consul servers or the control plane, basically to use mesh gateways to request or accept the peering connection. So on both sides we're going to actually configure Consul to use the mesh gateway component to send or accept the peering connection from the other cluster. And we can do that with one of the crds called mesh. So this one here, and we're going to have to apply that on both Consul deployments. So going back to my code. I actually have that configuration file already prepared right here. And you see how simple it looks like. This is the CRD from HashiCorp. By the way, make sure to check the latest version in the documentation if you are watching this a little bit later. So we're creating the mesh component with a specification that enables or basically tells Consul to use the mesh gateways for the peering connection. And we're going to apply this on both clusters which means I'm going to do kubectl apply. Consul. Mesh gateway right here. And I'm going to switch to and apply it here as well. There you go. We can also check that the CD was created. And as you see, we have this mesh component in the cluster, which, as I said, will allow to route the peering traffic through the mesh gateways. And now we are actually ready to pair those two clusters. So first going to the ECS Consul deployment. So that's our main data center. So to say our main cluster. Not technically but just theoretically for us. And we're going to go to the peers section. And we're going to add a peer connection. And we can give the peer a name. I'm going to use LCM. That's going to be the peer. So that's the name that the peer will be represented by. And we're going to click on Generate Token. And this is basically a secure token that will allow the other peer to connect to this one. So I'm going to copy this and let's close. And as you see this is the peer name. And it's pending because the appear basically has to make a connection as well using that token. So we're going to go to the now the peers at Peer Connection. And now instead of generate token we're going to do establish peering again name of the peer. We're going to call the other peer X and the token that I just copied at peer. As you see, super simple. And here we have this status as well as the health check. Is the peer accessible or not? And if I switch back this one is active as well. Very simple and straightforward as you see. Now as the next step we're going to add an exported service configuration in the LCK. So basically we're going to use an example of one specific service. And we're going to expose that service or export the service from this pier to make it accessible for the EKS cluster. That means the services here in the cluster will be able to talk to that exported service. And we're going to use an example of the shipping service actually, which means we're going to export this shipping service from the cluster to make it accessible from ECS services. And I also have a configuration for that. Let's go back right here. I have the exported service. And as you see the configuration is also pretty simple. We have this CD called Exported Services. This is the name of the service that we are exporting. And this is the name of the peer that is going to consume the service. Pretty straightforward. So basically just to demonstrate how this is going to work right now, if the shipping service failed, a specific feature related to that service will not work anymore because the pod is not available, the service is not available. So let's click in one of the items. And if I click on Add to Cart, as you see, everything works because shipping service is up and running. Let's go back. I'm actually going to. Yeah, I'm going to switch back to the EKS cluster and I'm going to delete deployment. Let me check the name. Shipping. Service. There you go. The shipping service pod should be gone and should be gone from here as well. And now let's actually try to access the same function again. And as you see, it's not working because the shipping service is not available. You get the error here as well. So what we're going to do now is that if this happens, like some of the services in this cluster basically crash and they're not available, we're going to direct or we're going to forward the traffic to the peer cluster that has the same service. So the shipping service of the cluster will basically take over instead of that deleted or crashed shipping service that was running here. So that's what we want to achieve. So I'm going to bring up the shipping service deployment. Again I'm just going to apply. The config again. Like this. There you go. It works again. And now let's configure that failover. So in the LCK. So we have this exported service for shipping service. And we're going to. Apply this. And if I switch back to my Consul. UI for deployment. You see that we now have one exported service, which is this shipping service basically. It also shows the topology of the connections and the same way in each cluster. It shows that as an imported service from that cluster. So now if we go back to the list of all the services, so we have all those other services through proxy, and we have this one here that shows that the service is actually coming from the peer connection. So we have two shipping services available for this cluster. Now. Now there's one more thing that we need to do from the case side to create what's called a service resolver. And again service resolver is its own CRD. And this is the configuration for the service resolver component. So basically we're configuring the service resolver for the shipping service since we have two shipping services now. Right here. And we're saying that we're going to use the failover to the peers shipping service service. So we are going to need to apply the service resolver in the EKS cluster, because we have those two instances of shipping service in the EKS cluster. And we are defining a service resolver for them, saying that this should be basically a failover. So switching back to EKS I'm going to apply. The service resolver. Let's do that. Create it. And now, the moment of truth. I'm going to delete the shipping service deployment again in the cluster. And that add to cart feature should still be working. So let's do that again in the EKS cluster. I'm going to. Delete the deployment shipping service again. Switch back as you see that one is gone. And now let's actually refresh again just in case. Going to any product. And if I click on Add to Cart, it should work by failing over to this imported service. And let's do that. Awesome. As you see, it used the service from a pure cluster as a failover. Awesome. So that was basically our demo. I hope I was able to give you lots of new insights and lots of new knowledge about service mesh technology generally, as well as what concepts and small details are involved in all of this. If you made it till the end of the video, congratulations on gaining a lot of valuable insights and knowledge in this area. We put a lot of work and effort in creating this video, so I will absolutely appreciate. If you like this video, leave a comment with your feedback and even share with your colleagues or anyone who you think will benefit from learning these concepts. And with that, thank you for watching till the end and see you in the next video.