let's say you've just set up a bunch of new servers and need to get them ready install software make folders add configuration files doing this by hand or even with scripts can quickly get messy and very hard to manage that is where automation tools come into picture today we're going to take a look at Anzible which many of you already use to manage servers and repeat tasks but as your setup grows maybe you need to run several playbooks in order handle errors or connect your automation to other tools you need something more to keep it organized that is where Kestra comes in in this video I'll show you how to take your Ansible playbooks and use Kestra to run them as a part of a bigger automated workflow we're going to start with a quick anzible example then see how Kestra can help you schedule organize and track your automation from start to finish so it's going to be a hands-on demo lot of uh coding so you can follow along as well because it is very simple kestra is open source so you can use it for free and folks who are beginner who may feel like these are just buzzwords don't worry we're going to cover the basics as well let's get started now before we jump in obviously we'll be using a lot of tools in this video so I'll be doing a demo of all of those so you can follow through but uh before that a huge shout out to Kestra for collaborating with Vmake devs and uh we are bringing a new series called hack sprints where we pick open source projects like real world open source projects being used by companies around the world and you sort of like hack onto it for over a week and we're going to give away some amazing prizes not to give you spoilers but like really expensive nice prizes in the past we've given like MacBooks and stuff so a separate uh live stream video will be going out for that uh so make sure you like share and subscribe and uh join the Vmake devs community all the links can be found in the description below so before we jump into the automation part let's quickly talk about what Ansible actually is well at its core Ansible is simply an automation tool you can use it to install software manage configurations update your systems or even provision cloud infrastructure what's the best part it does all of this without requiring agents on the target machines that basically means that you just need SSS access that's a tongue twister ssh access and Python and you're good to go so a basic example would be let's say you have 10 servers and you want to install htop on all of them now normally you would let's say SSH into each one and run the install command manually but with Ansible what you can do is write a playbook which is just like a YAML file describing the task once and then Ansible runs it across all of your servers easy simple let's see it in action so here is a simple Ansible playbook it's just uh going to be printing hello world and nothing else don't worry we will be looking at some real world examples and the host machine now this is local host means it's running on my own machine you can uh follow along this demo as well it's going to be very simple i can also connect to virtual machines in the cloud like uh a Civo instance or some other cloud instance it would work perfectly like that as well you just need to give it SSH access um the command is very simple i'm just going to say anible playbook and the name of this file is what simple run this so here you can see that uh it just ran it there was only one task print hello world message and it's just saying okay this is done local host message hello world very simple right but that is not cool we want to see something real world we want to see something that you know people would actually use so this is another file this is playbooky let me go through this uh and show you how it works don't be overwhelmed because you look at the structure you just give it a name host means where it's running connection this is my local connection gather facts right now no you You don't have to worry about this if you're just getting started and then in the tasks uh field you just put the name debug message and the other things i will show you how that works so you can see you can say debug and the message you can add placeholders you can add uh for example a shell command right so let's say you want to do pip install something you want to do pseudo install something or whatever right uh you can set variables and things like that you can say again debug message and all those things right again another shell commands so what does this file do now this real world example is uh sort of similar to our simple YAML file here I only have one task but here I have multiple tasks and this is essentially just giving me let's say you have 1,000 servers and you want the information about those servers like how much uh disk space is free what is the serial number what is the what are the ports that are active what all applications are installed on it what operating system it's using what's the RAM and things like that and you know uh if you do that manually for 1,000 or 10,000 servers that's not good so you can just simply write this one file you can say anible I'm writing this playbook please give me all the information about my system now this can be really good for system audits right troubleshooting or just keeping track of what is running on your machine at any point in time so I'm going to run this but I'm going to remove the system profile i don't want you to see my you know serial number and things like that and but you can you can you can run the command that I just removed and I will actually also hide my the IP address part okay you can you can check that yourself the command that I just removed just run that so if I run the Ansible playbook you can see it's going to give me bunch of information about uh see system overview i'm running Mac OS this is my memory information this is the free memory that I have uh my CPU information right i have 16 cores and uh show internet connectivity uh system load top processes what all things I have installed using homebrew and uh bunch of other things right docker information anything else that is running firewall information and whatnot so you can get you know as uh creative as you want with this but uh yeah if you if you are new to Ansible this one I just wanted to mention quickly but uh I want to show you the main part which is uh infrastructure automation using Kestra and uh you know introduce you to a nice amazing open source project but what exactly is infrastructure automation why do we care about that well in simple terms it means you're turning repetitive infrastructure tasks into code so things like spinning up cloud virtual machines with Terraform building and pushing container images running Ansible to configure those machines and rolling things back if something breaks now in most DevOps or infrastructure teams automation isn't just a nice to have it is essential so you automate infrastructure to avoid manual errors you keep the environments consistent and you scale resources when the demand spikes but as your stack essentially grows so does your complexity you are probably juggling Terraform for provisioning you're using Anzible for config management and docker for builds maybe a cloud CLI here and there and somehow it all needs to work together that is where orchestration for workflows comes into picture and in this video I'll be uh using Kestra to show you how it can help for those of you who don't know Kestra is a modern open-source orchestration engine you can think of it like a central brain that connects all of your infrastructure tools in one unified uh repeatable flow as they like to call it so everything as code is key here with Kestra you can chain together docker builds terraform deployments anible playbooks and cloud provisioning all inside a YAML workflow um which is essentially called a Kestra flow right it can uh run on a schedule you can respond to a git push or it can be triggered by an external event like a web hook or an API call now these flows can be fully version controlled integrated with git and terraform so you know no brittle UI hacks even and uh everything fits uh properly into your software development life cycle now why would you want to use Kestra i think it does more than just connect your automation steps it manages dependencies retries scaling and failures for you so for example if a Docker build fails Testra can automatically retry it when a Terraform task finishes you can pass its outputs like uh image tags or IP addresses straight to the next step so heavy tasks don't really need babysitting here kestra has scalability built in secrets and variables move securely between steps and you get builtin error handling logging and alerts for teams who are using GitOps your workflows are versioned with your code plus Kestra works across any cloud or if you want on-prem setup so you are never really vendor locked in that is the real gap with legacy tools and low code platforms you know they don't scale in depth or structure also ever since Broadcom took over VMware using uh area automation has gotten way more expensive it's no longer sold on its own the SAS version is gone and some companies are seeing five to 10x price hikes that is not cool okay that is why a lot of teams are looking at alternatives like Kestra because Kestra is open sourced easy to set up and works across tools like Anzible Terapform Docker and a lot of the others you don't get vendor locked in and you don't need to pay crazy license fees just to automate your infrastructure so if you are frustrated with VMware and you know where it's headed or you just want something that is flexible modern and developer friendly I would say Kestra is definitely worth checking it out and it's also open source so take a look at their GitHub repository as well by the way all the links I'll leave in the description below let's actually take a look at the demo all right let's get started with the demo uh running Kestra is very simple so we saw a demo of Ansible running locally some real nice examples now let's run Ansible on scale automate our infrastructure using Kestra and uh running Kestra is very easy just this simple Docker command and uh what you need to do is uh uh it's essentially just going to pull the uh Docker uh image from Docker Hub runestra you can access it on port at now this is what the dashboard looks like very simple and clean here are a bunch of things uh which we'll take a look at uh later but some things that are self-explanatory like name spaces logs executions I'll show you what the executions mean when you when you execute your flow flow is what you have a bunch of tasks that are that you want to run right you will be able to see okay I tried to execute this one task what happened what were the logs what was the output things like that blueprints are just like uh you know if you are running some tasks again and again you want to share it with the community you can do that as uh and there are some plugins speaking of plugins we will be using the Ansible CLI plug-in okay so there are various ways by which you can do this i'll obviously show you a real world example let's take a simple example for now okay so I go to my flows and you just go to create and here you paste right you paste this so here you can see what is happening is you just give it an ID you give it a name space and here in the task section you just list down what tasks you want to perform so here I have the Anzible task what is the type of the task anzible CLI which you can find over here the comprehensive documentation right commands uh anible config before commands container image all sorts of things right so this is the anzible CLI command that's why I've put this same thing as you can see over here now we had this file locally right the playbook file we had this file locally and we were just running the command how would Kestra know about this also there's the inventory.ini file which basically tells you which server you are connected to this basically means are connected to local host right um if you're using like a cloud instance you will add the IP public IP your uh you know secrets and the SSH information and things like that so here I'm basically going to say first of all I'm going to save this and here I'm going to go to files now what I'm going to say is that um over here essentially um I can see that uh there's the anible uh the ID is anible and the command is anible C cli that is fine that is the type now input files what files do you want to take from it so because this is an anible command it's going to ask me the inventory.ini inventory.ini ini i can just copy this and I will add my file over here i can also upload it okay inventory.ini i'm doing it step by step so you can see save this not a problem what else do we need so if we have our inventory.ini in my flow as you can see available over here next thing I need is my playbook.yamel okay it's going to have it saved under this variable create a file my playbook you can see like parent folder and do other things just going to save this empty for now that is fine and uh what I'm going to do is I'm going to actually take my simple hello world playbook okay let's just run this for example for now so what it's going to do is here you can see my files inventory simple hello world playbook uh my Kestra will have uh flow will have access to this and uh you can see that it's pulling this docker image and it's just going to run the anzible command isn't this the command we were running uh we didn't have to specify this directly because it was taken as default but wasn't this the command we were running in our local repository on our local terminal right let me just execute this command and see what happens finished as simple as that now I can go into this task i can see what happened so it pulled this image container was created volume was created and message is hello world very simple this is not cool though okay i mean it is pretty cool but it's not something like real world okay so let me give you like a nice example a better example so this is an example that I have for uh Kestra right uh what I have set up over here is a simple Kestra workflow it will run an Ansible playbook using the official uh Anzible Docker image right uh as we just saw it was running it for the basic stuff as well and uh this is also running locally you can also SSH into remote servers and the environment is also isolated in a container which is important because no setup is needed on your host so here's what it does it prints a simple debug message just to confirm that the playbook starts up correctly then it installs the edgetop package this uh basically simulates a basic provisioning task like uh let's say you're installing some monitoring tools or system dependencies next it's going to create a directory structure for uh an app under the /op/myapp uh folder with subfolders for config logs and data it's also going to drop in a sample configuration file inside the config directory with basic key value settings and finally it's going to list out the entire directory structure and print it into the logs just to verify that everything was created as expected now the whole playbook is uh passed as an inline YAML inside the Kestra workflow as you can see I am not saying uh you know like I was um let me show you the previous flow here where I was saying that hey my inventory file actually is in this file my playbook EML is in this file so I was separating this but here what I have done is Kestra allows me to essentially add or copy paste the entire playbook file my anzible playbook file in the flow itself okay so once I trigger this flow Kra is going to handle the container spin up it will run the playbook it will capt capture the output including any debug messages and any command results as well right so I'm going to copy this whole thing i'm going to actually create a new flow okay paste this and I'm going to save this now I do I need to add any files over here no i mean I can if I want to but here you can see that I have directly pasted the files over here okay cool stuff execute this let's run this simple took 4 seconds let's take a look at the logs so image pull container created volume done and uh message hello from Ansible so that is running fine my task install package running fine that is done show package installation results uh failed false so it worked perfectly installed the packages and uh here you're getting all the other information create a main directory uh it created the directory and uh it added all the things all the things that I did see opt my app/ data it did everything over here you can take a look at the logs as well the attempts and filter it if it fails you can filter it accordingly you can check the topology so this is the drag and drop sort of like command uh structure that I wanted to show to you like if you have multiple stuff this is an anible example let's say you are running uh terapform and stuff and other cloud stuff uh you know you can essentially just uh click and drag and do that as well um here you can find the outputs oh by the way let me show you what I mean by the outputs okay let me show you what that means so for example I want all the logs in an output so I can just say output log file I I can set that to true i will save this and I will run this go to my outputs output files here I have one file i can preview this and I can download this now a question people would be asking me is kunal if we have 100 files do I just do 100 inline files or do I write this uh thing as like 100 times no there's this concept in Kestra called namespace files where users can uh basically orchestrate their you know entire terraform anzible or python projects okay and not just quick tasks like this so let me show you an example of what that looks like so instead of input files I'm actually going to delete my input file section okay and I'm going to say namespace files and here I'm going to say enabled true hopefully the spelling is correct save execute that should run fine as well easy so it says that it loaded two namespace files from company team if you don't know what namespaces are you can take a look at the documentation you're just making a collection of your project okay so you can see that it's uh like a folder for your flows okay you can organize your flows into logical categories you can give it a company name and things like that okay simple stuff so this is another thing I wanted to show you like if you have complete like you know entire Python projects or anible projects or things like that you just mention it under one name space uh add all your files in a folder using just this one line it will have access to all of those things easy so I showed you how the outputs move as well into files and things like that here are the logs overview you can check out some metrics and other things over here also cool this was my production example file available over here okay this is another example it's like a business automation so you have working directory you make a query you save it to CSV you uh save it to Excel and things like that so you can essentially run the code in any language not just some sandboxed JavaScript or Python snippet um and you can use this drag and drop UI right so Kestra basically handles real scripts and full coding projects i gave you a nice little example of uh you know we did a simple example just to show you how the Kestra flows are working and I gave you a nice big file example as well with some more challenging and real world uh tasks right but here are some of the other examples like you can see when you basically set up a uh you know Kestra you will be able to see like uh a lot of lot of examples in their ecosystem and I would recommend checking out the blueprints as well last thing I want to mention is u if I have my you know like enzible tag uh sorry my enzy flow over here you can also set triggers right so triggers like I can schedule this and I can just essentially just say like under my tasks when do I want to do this task so I can set a schedule I want to do it hourly you can also do other things like whenever someone makes a git commit I want you to run this execution whenever someone does this or that I want you to or make an API call or things like that you know that's another thing I wanted to show you but uh very cool project what I would recommend next is go to the Kestra GitHub repository over here they are very close to 20,000 stars so make sure you do that um I would recommend checking out their examples other things that they have and uh a lot of other uh you know uh communitydriven examples i would recommend checking out the issues they have a lot of uh good first issues for you to contribute to as you can see there are so many so you can check that out so make sure you give it a star and uh what I mentioned earlier is we are going to be running a hack week with Kestra so you can follow this tutorial to get familiar with the tool i would say create your own flows use anible definitely but try to use other projects as well and um we will see you very soon in the Kestra hack week i am very excited about that so as you can see they have over 600 plugins so I highly recommend checking that out but yeah uh all the links can be found in the description below let me know what you think of this uh tutorial if there's any specific example you want me to cover I can do that as well so make sure you uh comment that down below or share it on social media tag me tag Kestra and I will see you in the next one have a great day bye