hey everyone today we're going to have a look at how you can design a modern cicd pipeline so if you're a software engineer and devops engineer if you're designing a pipeline this is going to be a good video for you so in this video I'm going to go over the general concepts of Designing a modern cicd pipeline so no matter what industry you're in no matter what type of application you're creating this should all be applicable to you so let's go ahead and start diagramming this out so basically every pipeline is going to to start with what's known as the source stage and the source stage is a pretty simple one to understand basically this is just getting your code from your code repository such as something like GitHub or bit bucket now there are a few steps that I like to include in my source pipeline so the first thing I like to Define in my source stage is just having some basic Branch protection rules and when I say Branch protection I just mean like simple rules where you have to have some sort of PR review before a developer can merge their code to the main branch so you never want it so your developers can just push up to the main branch you want some Branch protection you want some sort of PR process so that's the first thing the next thing I like to implement at the source stage is linting so linting if you don't know about it basically this is just checking your code for santax errors there's many different linters out there basically every single programming language yaml document any type of configuration language has a linting program that you can Implement into your cicd pipeline so one practical example of how I would implement this into a pipeline is I would just have a GitHub action and it would give either a passing or failed status check if the linting checks failed and then if it has a failing status check the pr can't be merged until the linting problem has been resolved now one other thing I like to do it actually happens before the source stage and it actually happens before the developer can even commit his code so let me explain this with the diagramming here so let's add a developer and this developer is going to commit their code to GitHub but one thing I like to do is have what's known as a pre-commit hook that's built into the developer's IDE that will actually run the these linting checks locally on their machine before it even lets them make the commit so what we have here so far is the developer goes to write his code before he can actually commit and push it to GitHub they're going to have to pass their local pre-commit checks and then once it gets to your Source control repository then that's what triggers The Source stage then we have our PR checks the linting checks and then we can move on to what's known as the build stage so with the build stage this is all about getting an artifact that you can then test and then release so the first part of the build stage is actually compiling your coat and building your container images if you're using a technology like Docker which if this is a modern cicd pipeline you definitely are creating container images of your application now actually the order of this you can either compile your code first and then build the image or you can build the image first and then compile your code within it so usually actually how I like to do this is I will actually build my container images first using my Docker file and then I'll actually compile the code within my Docker image so this is all personal preference how you do it so all that really matters is you have your code built and you have a container image that you can start using to run your unit tests so our output here is going to be a container image and that container image we're going to use to actually run our unit tests on so unit tests are basically just a way to test the basics of your code so every time a developer submits new code or a new function or anything like that they should be building unit tests and then those unit tests run during this stage of the pipeline so if you had a calculator app basically a unit test would be does 5 + 5 equal 10 does 5 - 3 equal 2 you can think of a unit test to be just simple checks like that now once your unit test are done one other thing that I like to do in the build stage is to check for code coverage so let's add a node here and just scroll over maybe zoom out a little bit and go code coverage check and usually I like to check for about 80 to 90% And the percentage is really going to depend on the organization and the type of application now if you're not familiar with what code coverage is basically it's just checking your code to make sure that your developers are putting in unit tests so if you have unit tests for 90% of your code then you would have a code coverage of 90% if you only had unit tests for 80% of your code then your code coverage would be 80% so this is one thing that I like to gate if the code coverage ever Falls below a certain percentage then I like to fail the pipeline there now once your build stage is done it's time to move on to the testing stage where the real testing happens so I'm going to actually just zoom out here a little bit and let's add another stage to our pip pipeline so we'll add the test stage and let's go ahead and add some images here to make this diagram a little nicer all right so like I said in the test stage this is where the real heavy testing happens so in the build stage I do the simple testing like the unit tests and the code coverage checks but in the test stage we want to actually test out our application and run what is called integration tests so we'll add this here so basically what an integration test does is it's going to test the functionality of your application so if we had sort of like a social media application one of the functions that we may test is can a user post a new message can someone comment on that message can they upload photos can they delete a photo what happens if this happens this is where all those complicated tests start to run and your functionality of your application gets fully tested so this is actually one of the longer stages of your pipeline because running all those integrated tests does take a little bit of time now one other thing to keep in mind here is if you're building an image and then testing a container for a micros service that microservice probably has a lot of other dependencies it probably depends on a lot of other microservices in your infrastructure it probably depends on databases so you have to make sure that you're able to bring up an integrated testing environment Now using something like Docker compose can do this for you but you can also run this in a full testing environment in a kubernetes cluster this is really going to depend on the organization their testing Suites that they need to run and the type of application so I like to leave this one a little bit open uh for how you actually integrate it into your pipeline so that's all I wanted to mention about the testing stage let's now move on to the final stage of the C ICD pipeline which is the release stage so the next stage here is release this stage is all about releasing your application so your servers can start using it so what happens in the release stage well this is usually when you ship your image to registry so let's get uh ECR registry here and point this to our registry so so when your image is available on your registry that's when all your other environments can start pulling it down and running your application so this is going to be something like a QA environment so your QA Engineers can run extra tests to your application this might be a pre-production environment such as a staging environment as well as your production environment so let's plug those in here so all these different environments can get the image from the registry now let's scroll out here and have a look at this whole thing so everything we've went over here is what's known as a continuous integration and continuous delivery pipeline but in the next video I'm going to talk about how we can actually get to the next step which I was getting to at the end here which is how we can continuously deploy this through a continuous deployment pipeline something along the lines as a get Ops Pipeline and I'll talk about how we can integrate our cicd pipelines our continuous integration and continuous delivery pipelines with our continuous deployment pipelines and how the different pipelines can interact and how you can go from a developer committing code to that getting automatically pushed out to production so join me in the next video where we go over continuous deployment and get Ops