Transcript for:
Overview of AWS Developer Tools for CI/CD

Mondays yes um open source to so open source tools are not tools that are are specific to a to a platform like the team C like the bit pocket like the um um gitlab the Jenkins they obviously owned by projects some of these projects have been doned to the cloud native uh Foundation the cncf but right now since we are uh um looking into AWS we want to also look at AWS specific tools which we can use to achieve um uh the same um goals right so you can use set up uh pipelines you can use to completely um set up um flow of our uh software release cycle from development to testing to um uh deployment okay and AWS um has a bunch of uh developer tools which we can see give me a second I share my screen with you so where is the icon looking for the shap so a has a bunch of developer tools which we can use to BAS basally set up the same system the same software relas cycle from development to pushing it to S repository to triggering a build orchestrating that bill and deploying it to soft to our end users when we were talking about the open source tools we remember we said that um developers will push the code to GitHub in this in this case then Jenkins is going to Trigg that will also trigger the Jenkins either with a web hook or Jenkins using pull SCM to be pulling our source code or sem from time to time and once it notices that there is a change or new commit has been uh added to the repository which is watching then it can uh pull whatever has been put into the repository build that and deploy that to the N users we saw that right hello yes we did yes sir but so sir you keep saying P SCM that's a terminology that we used severally before right P SC CM pool source code management system okay source code management system okay so our SCM is get help yes yeah yeah I was going to say I think we did that we did that in Jenkins in our Jenkins server right yes we did that in our Jenkins server so Jenkins is going to pull our sore management system if there are changes in there it builds our changes and pushes and deployers out to depending on how the pipeline is configured so if if we want to do the same thing but using AWS we can use um a suit of tools from the AWS developer developer section so where is see developer tools you can see that AWS has developed a bunch of tools for they're not sharing [Music] though sorry I I thought I was can you see what do you see we see we see it now the the developer tool I thought it was sorry so a has developed bunch of tools which you can use so there is the code artifact which you can use for to manage s s code revisions so if you look at the AWS code artifact so it say secure and scalable artifact management system for software development you can is this an equivalent of net Nexus maybe ex a ex exactly equivalence of Nexus so there is the AWS code um view which we are going to see an example of going you see the code Catalyst code deploy so the code build is an equivalent of Marvin so code build is not just U for Marvin but it's um a build tool for AWS which comes with preconfigured build environments and these build environments already has necessary run times necessary um packages which you can use to de to build different uh possible projects based on different languages right so we looked we remember we talked about Marvin Marvin is one of them that's already be pre pre pre configured and there's gr there other two other languages that they pre could bill supports but once you're talking once you hear code bill you should be thinking about the bill to and one of the build tools which we did talk about was Marvin we together yeah so code deploy um code deploy is a tool which you can use to basically deploy automate the deployment of applications to instances so if you remember we already used a tool when once we um discussing with Jenkins we used the plugin which we were able to use that to actually deploy our application to a Tom card server remember can somebody um remind me of that pluging I think it's called code deploy Noe no the the plugin for the plugin on Marvin deploy to container yeah deploy to container yeah so there's a plugin which we use a Jenkins called deploy to container and that plugin enabled Jenkins to be able to deploy our executable to the end users or to to the servers which the users um and users are to access so an equivalent to that in um the i space is the code deploy okay good so the code deployer with the code deployer you are able to deploy your artifacts to different um types of uh deploy deployment environment so ec2 Lambda functions ECS and stuff so we're going to see an example with E and also code pipeline so a code pipeline is like jenin so it's an e stration tool right so the code pipeline is the continuous delivery service that enables you to model um automate the different steps that you can uh use to actually release software to your end users we together well that's the one that's deprecated then we are not using anymore right we are not using Code commit oh code commit sorry okay so we are not using Code commit good so these different tools have um some common or some core features which I would like us to talk about so if we look at can you still see my screen you still see it sir yes what do you see no nothing we just went off just the black screen that uh but we know it's your screen though I think it has some iPad thing at the bottom oh there you go good so with the different um developer tools like code commit you can already said that it's a manage tool from AWS that you can use to compare uh sorry code build it's a manage tool from AWS which you can use to compare your source code run uh do unit testing produce artifacts and actually send deploy those artifacts so you can use code be to push your artifacts to um artifact store like the AWS um artifact or to other third parties which you can integrate with with code build so these tools even though they AWS native they do not only integrate with ads based tools you're a able to integrate them with Jenkins you're able to integrate them with bit bucket you're also able to integrate them with other third party tools okay that's why we will be able to integrate with uh GitHub and actually uh pull our information from GitHub so for code bill it has a concept they have a some main concept which I want us to talk about like the build project so the build project when it comes to code build is actually um the it actually includes um information about the source code where the build environment which build commands to run where to store the build output and stuffs like that so once you you want to set up code build the first thing that you code build needs is um something called the build project so the build project you supply it um once you're setting up code bu to actually do a build okay so this build project contains information that could build needs for example where to get the source code so the build project would have that information um which build environment to use so the build environment is basically um um information about um the configuration of your builds where to run the build is it OS what programming language to use what tools to use and stuff like that so that's the build environment okay so with code build most of this build environments are already prepackaged like I said so they are prepackaged in Docker images start Docker at the end of this week so you already have a Docker image in that Docker image you've uh pre-installed all the necessary operation system the version the version of of Marvin which if you're using Marvin for example the version of grade if you're using gr all the dependencies all these things are already P prepackaged and um made available to code deploy and that will be the build in okay are we together yes sir sir yes sir we are wait it doesn't make sense please stop me so I could reiterate or try to will you be able to share this again sir if you want yes but that was not the plan we can change the plan so um it also has something called the build spec so the build spec is our build specification so um if you want to build the code you actually need to tell code deploy uh inform uh information so you have a build spec in that build spec you can have all the collection of the commands that you you would the build should use some settings and this build spec could be um part of the build project or if you're setting your U environment so that it pulls your source code from uh an sc then the bill spe can also be passed to the SCM in a yamama format so you will see that this uh demo project which we have we have yamy that contains our bill spe and everything is um available inside the bill spe okay so you can see and a high level setup of this is actually from from from the documentation is from AWS so I did not do this cre created is you so we have an a high level of how code works so if you see code build you provide a code build build project as I said and code build uses the build project to actually create a build environment and in that build environment you like I said has um all the necessary dependencies the run time the OS and stuff like that which could be need to actually build the project so code build then downloads the source code into the build environment and uses the build spec to actually run the bill okay are we together yes sir I'm trying to my P is not working I have no clue why trying to highlight so Prof yes so code build I'm just trying to imagine what's going on so code build you provide code bu build with the build project and is that the developers doing that or is that us the you know the automation operation guys that um that do that because you said something about um providing the code well the actual code base so the code base comes from the developers correct yes the code base comes from the developers but now setting up the steps for code build to actually pull that code base build um the project and all the stuffs and actually the build specifications you actually um like you say it's it's a Dev culture and nobody's actually a silo there's always a interaction some information in order that you need to other to set up the cicd pipelines is not coming from you but they are not setting it up you need to get the information from the dev team integrate that to you set up and actually have that and and system running running okay yeah good fortunately my pen seems full now good so um yes so the code build is going to actually pull your SCM here and use the code in your source code and the build back to actually run the build okay so once the build like we saw there is an output so you can use S3 you can use other artifac um stores to actually push the build so that you can save out your artifacts and once the build is actually running since you're using Code build and it's in IVs and an Ed and AWS you're able to integrate the build code build with systems like cloudwatch so that pushes metrics for whatever is happening in your environment to Cloud watch and also integrate that to with some s um SNS notifications so that during the build you're able to receive information if there was a failure in the build process um if there was success whatever um system which notification which you want to set up there then that's possible for you to also integrate that okay together good if you remember we said the build environment it's a Docker container and in that Docker container even though you running the build in a Docker container once you do Docker you understand that Docker is um and high level extraction of a server so it's small it's actually small I would say small computer on its own or a small server on its own but it's also able to during the build process still feed information to code build about uh the builds okay how what is happening during the build and it's also able to push information to um to to Cloud wash then you can use the code build console you can use CLR you can use whatever sdq SDK this is known as software development kit okay so Prof um from what you're saying is um when not for the code build that's where it house the project there is no um e to instance is actually a doer image even though we haven't done doer I'm trying to yes in my head okay so there's no is to instance if you remember when we were when we were um doing mavin we had to spin up an instance install Marvin in there to actually have that running right code build is actually using Docker images to do that same thing so you already have the precompiled environment in your Docker image the dependencies in the doer image and code bill is using that as your build environment um U to actually uh do the bill okay are we together any question hello yeah yes sir no no question not from me so it's not just adding up thing is it's not coming up I'm not because of the darker image part right something is missing I don't know I'm not connecting the do in the so what what what we are talking about here right now is just code bill and we said code build is a build tool right if you remember that we already talked about the build tool which was Marvin right yes yes so code build is an equivalent of Marvin like our build environment remember that Marvin when we dealing with Marvin Marvin it was not just building your software we were able to also use Marvin to push our software to sonak we're also to use Marvin to push um artifacts to Nexus all yes so all of them has this these features so code bill is your equivalent in in AWS so with code bill you still able to pull that information if you remember when we dealing with the Marvin you have the settings.xml and whatever information that you need there all those things are still uh important here with could be yes and Marvin is actually still using that information to pull that source code build the source code in your build environment and in that Marvin we used to have an is2 instance that Marvin is running in but Cod bill is not using an is2 instance Cod Bill already comes being a managed platform already comes with the possibility to have prepackaged build environments which are Docker images so when we do Docker you see that Docker is actually a mini computer or a Mini Server right so inside Docker image you are able to uh run command you're able to install software you're able to package whatever dependencies which you have so code build is using these darker images as its build environment okay so this they they what they are manage images there are some manage images which AWS provides but as a company you can also do your uh pre you prepackage your images or pre-build your images so you have custom images which would meet your specific requirements and you have these images in Docker hob and code bill is still able to use the docker images as um your custom document images as um your build environment to actually run your bill all right so code Bill uses that pulls your SCM gets the source code builds that source code and if it has artifacts from that environment it's able to use S3 as an artifact store it's also able to use artifact service from RVs is also able to integrate with third party um um integr integr integr third party tools all right so like like um I I did not check Nexus okay specific but we could also look into third party tools once we go to the um handon we can check that since I'm using my my tablet now okay so in this case now the S3 um is an equivalent to NEX for the AC some environments are actually using S3 as the artifact store because if using a and um they using um AWS developer tools and they able to basically make use of S3 without cost of Enterprise Nexus and stuff like that some many projects actually using that some environments are actually that are still starting a project or they in the death phase they in still in the very beginning of the project and developers are still trying things they have all this set up and they're using S3 as the artifact and as the project gets matured and they might decide to switch to something else okay so you can have projects where um we're trying out something all right once we still trying out something some Po's we're trying to keep cost at the minimum right so we use tools that um would keep our cost at minimum so we actually have this set up and they using this then once we are sure about what we doing the project would not be killed or the project would not be um basically we are not going the project will not be stopped either because of fonts or either because it it met the initial requirements then they start by using something like stre okay for the artifact so the code Bill also integrates with SNS it also integrates with cloudwatch so that it can push locks to cloudwatch and you can set up what sort of notifications would you have around there so this picture is not an exhaustive picture it's just phasically telling you the possible Integrations that you can have with code okay sure good does it make sense now that makes sense now you know the energy make the difference what energy sorry the energy made the difference I'm not sure understand what you mean I don't either that the energy that you used to deliver now make big difference thank you okay yes because he smiled because when he started he was not smiling know he was sleepy I feel like yeah so you smiling alone makes a point so Prof keep smiling now who was sleepy no just that you not smiling that's all you not sleepy but you were not smiling the second the second explanation that's my proof let's keep going thank you yeah please smile till the end of the class good great and um now let's have a short conversation about code deploy so code deploy we already said it is a automation tool right that automates the deployment process are we together yes sir M so code deploy automates the deployment process so you can integrate code deployer into your cic pipeline and once the build happens then code deployer is able to pick those artifacts from wherever you have the artifacts stored and deploy those artifacts to the end users make sense please can you go over that again so so sir what I'm trying to think what the equivalent of code deploy is to this CV project we think about it and tell me what you think Jin Jenkins I was thinking about Jenkin however I think maybe I'm not right yeah C you think about it yes somebody already mentioned that in here when we started uh um container what you been able to do this was I yeah I did it good so there's a pluging we always use when we want tooy to Tom card oh yeah is it the container deploy deploy to container deploy to container sorry good so the code deploy is the pluging or is the manage deploy M service from AWS which you can use to actually deploy your artifacts to the and users okay okay so code deploy has some um components we have something called the application we'll see that as we we get into the handson so the application is basically a name that you code deploy would use to uniquely identify whatever um software application which you you're building and this application will ensure that a combin a correct combination of give me a second my battery is low my headphones I wanted to tell you your B is getting low so I think you you were talking about the battery for my tablet I think that was that yeah the one with the tablet I think that's the one I see on screen yeah but my headphones is telling me that I running out of battery don't forget to plug in the one your tablet before you we lost leave him alone why is that me class he always wants to worry leave me alone Victor I miss you can you hear me [Music] so like I said COD deploy has what called an application and the application basically is a name that you need identify the appc the software application which we building and it ensures that a com a correct combination of um revision is used deployment config is used and the deployment group are referen during the deployment so what are these a revision we've talked about revision before can somebody tell me what the revision is so when we say software revision what what comes to mind like update so simple it's it is the version of our yeah so please get used to this this um terminology so okay so when I talk about software revision is basically saying talking about the software version so each time you run a pipeline each time you build your software you're developing a version you're developing producing a new revision of that okay and you see that in with many tools especially when you start doing uh using hem they will always be talking about what hem revision are you uh are you currently using and stuff like that so the application ensure that the correct version the correct deployment configurations and the deployment groups are always referenced during deployment phase so what is this revision we said it's a version what are the deployment configurations deployment configurations are basically a set of rules and deployment success or failure conditions that code deploy would always use when he wants to deploy um software to um um to a deployment group right and the deployment group is basically a set of each two instances okay so a set of each two instances which could deploy or which the pipeline or which coulde deploy will have to use in order to deploy the artifact or or your your software version does it make sense hello yes it does make sense so you have what we call deployment configuration um quick question Prof yes can the deployment group be used as well to categorize um end users or staging environment yes you can actually use that but you would have to um configure that in your pipeline right it's a pipeline and it's configurable so the deployment group just from what we say it's a basically set of these two instances so you can actually configure the pipeline to deploy to different deployment groups um based on uh environment and you can actually have the pipeline Target those deployment groups based on something like isue tax and stuff like that all right and you would know which deployment group actually represents what what environment right so to answer your question that is possible yeah thank you good sorry just to back up a little bit this term deployment group I'm just trying to see the context that you would you would want to use it um in in the cicd um end to end pipeline deployment group so at that definition I'm not quite clear what it is so a deployment group is basically a set of is two instances I want to just keep it simple set of in set of individual instances so each deployment grou can have individually tged these two instances these two instances in an auto SC SC Auto scaling group or board so forget about all the long history uh literature then a deployment group is you have each two instances that you want them to be where or servers that you want them to poost the application once you you build the application right because on once you build an application you have the software run developers develop the software you have your continuous integration happening you have your continued deployment there is an environment that you want that is going to host that application at the end right for example our Tomcat server so if you have so many different to Tomcat servers and let's say 10 Tom card servers and three them you T them um de three you T Pro three talk uat these three or these groups or these different set of these two instances are different deployment groups gotcha uh Pro I just want to ask again you said that you where to com um compute platforms where to deploy the applications I know you mentioned ec2 um instances being a set of um deployment groups but then if you also have TS and because these are all way areas um resources where um you can deploy your um application to like Lambda EC right and on Prem so the deployment group do they still have to be ec2 instances no so e code deployer can deploy to these three categories of compute platforms so code deployer can deploy to servers which is either e to in IVs or on premises servers so code is able to actually read um reach onlo you can deploy to Lambda functions or it can deploy to ECS okay so each of these n complete complete platforms would then constitute your deployment group okay how you you you you tell code deploy to find these deployment groups will depend on the type of compute platform which you're using so where let use the case of is2 you can tag each2 instances and code deployer is able to find these is2 instances based on the tax okay this is interesting together yes so using tags as well I'm just curious I know we haven't yes yet I mean are these also T taggable resources um um in in AWS what do you mean by what what T table ECS is it a tarable resource I mean can I ECS is actually running in the background on uh some is2 instances just like eks okay okay so you can find that but um am I jump in the G by saying how would it identify that particular Resource as its Target is there a different there's probably a different mechanism okay okay all right let's leave it alone for now I shall revisit so like I said we have the um application components you have the compute platforms you have the deployment configuration which is basically a set of rules set of conditions set of um uh um whatever standards whether you want code deployed to actually use on it's deploying your your application to the deployment group then you have the deployment Ty what type of deployment do we want do you want to deploy everything in place or do you want to have what we call Blue Green um deployment so in place deployment is possible when you're using each two instances and with in place deployment what happens is um um code deployer is going to basically replace the application that is running on your is instances so the old um Environ instance with the old version or the old revision of your code it's actually stopped the application is stopped the new revision installed started then made available to the end users but with blue green deployment you have um what you call a blue environment and a green environment and the blue environment is is to instance with the old version of the code and that is still made available to end users once you have a new revision coulde deploy spins up new set of um is instances you uses a new deployment group gr installs the application and once it is ready then it um basically um stops or makes decommissions the old version of the o e to instances all right have you guys ever heard about blueing deployments before yeah about it so code deploy is actually able to use that so code deployer also uses what we call the I am instance profile so you know what an instance profile is right so it's an AM r that is attached to an each two instances and that am R has permissions and these permissions are actually required by code deployer to make the connections which he needs to go to S3 to talk to GitHub repositories and to make uh to talk to other services which he integrates with so we are going to use um the instance profile today um to actually set up our access to uh our GitHub repository and the iion we already talked about the revision so they have a service rle so Cod deploy like I said integrates with uh uh different services like the tax he has r tax that apply to these two instances it might need to actually perform some operations on each two instances um also trigger load balances and stuff like that so uh code deploy uses what we call um service roles so service roles are different from I am I the instance profile which is yes also a role but a different kind of role it's a different kind of role so for AWS if you remember once you want to uh have two Services talking together then you have what we call Service rules service link rules so those are Service rules so you're using Service rules from one service to be actually make able to make API call or another service okay good great so that's basically it then you have a code Pipeline and the code pipeline is basically the our Jenkins equivalent it is the orchestrator of our different um tools and um with the code pipeline we able to set up stages and use that to um build our uh our software from SC M right down to uh deploying it to the end users any question so code pipeline is like Jenkins basically right exactly code pipeline is like Jenkins it is our orchestrator it is the guide to trigger um the build is uh uh the tool that would discover that there's a change with in s c m will pull the change and Trigger could build to do a build could deploy to do a deploy and basically it's Orr on hope when I say that that makes sense yep so these are different tools and to be honest with you if you want to sit down on this different tools it's [Music] um we cannot we can definitely not cover them in in one class but uh the CH about it is for you to understand we we spend a lot of time talking about the open source tools which in my opinion 80% or 90% of companies out there are using and we also want to uh make you understand that even though we concentrated on these open source tools um AWS has uh these tools which you can actually use to still achieve the same thing now the in my opinion they are not as feature Rich as most of the open source tools that's why I think most companies to are not going for it small teams would use it for small developments but it's good to also know that okay if you're using a IVs these are the possible tools which you can use all right for those taking interviews I don't know if some of you you've come across questions they actually asking you about whatever what about what about what cicd tools and stuff like that you can use in in AWS have you yes they care about Dev so much of the CSD to you can use in had so much that interview get question asking about CV tools like in devop I mean those are potential interview questions is what Prof is saying I guess one of them specifically asked me about code deploy I mean code pipeline so even though some of them might not be using it but because you're coming from Ed experience um some of them would want to know if you actually know some of those tools from my experience personally and um could quite be different from some other person I haven't used any of this AWS development Tools in my projects so but some some some um I know about it small small team small application team in one of the projects which you're using and the way you just had the setup and for the testing but the farm life environment we using completely something completely different so Prof let me ask a different question in a way that I think I understand this so these tools that are AWS native you say are managed tools just to explain that I mean from perspective of how it will be used managed because the other ones open source and we have to do most of the work right how is manage by AWS a plus for us so um basically the admin overhead when the people are talking about manage to from um being managed it's first of all from the fact that there is um already a lot of the configuration overhead which has been taken away from you there is a lot of the for example code pipeline if you remember you build your Jenkins environment we spin up a server we install Jenkins and we are using Jenkins and from time to time you guys are complaining that Jenkins is slow that's not something you will experience with C pipeline because it is a managed tool it is already built for scalability it's built for resilience and stuff like that so that's the advantages that come with using manag platforms that's why um even though Jenkins is is out there people also go for manage environments like get up actions there the get up actions which is you are not you doing the self self self-hosted Runners you're using the manag that's a manag solution from KOB if you're doing bit bucket pipelines that's a money solution from bit bucket as far as you're not also um having those Runners you're not managing those Runners even if you're managing the runners to an extent eost having a small manage to or SM manage service which comes with the uh better S or let me put it that way best better user experience because they have a complete team that is M or that is there just to run that or just just to ensure that that service is running smoothly so those are the advantages that come with using this managed managed tools okay gotta thank you if you're using Code pipeline you will never say that oh code pipeline is slow why is it slow AWS already has that build into the tool for Autos scaling for it skills based on the project so the small project obviously is going to as your project grows the artifacts become big your bu build spec becomes the tool automatically scals based on Project needs so that's the manage part of it does that answer your question yes it does thank you great good do you still see my screen no no if no question then I would want also to I shared um a link on the chart and this is a link to simple bit bucket project which we Sorry simple get up uh Tom Cat application which we going to build and I also share a link to the cloud formation so what are we going to do as our handson I have a cloud formation stack and in that cloud formation stack we already have um uh uh resources that is going to build our code build is going to create create our C build is going to create our C deploy deploy environment is also going to uh set up our U code pipeline so the code pipeline is going to be able to orchestrate these three these other tools okay give me a second EV is everything good yes sir yes sir everything is okay thanks for checking yes sir so far so good thank you good so can you see my screen what do you see code small but um yes code is it yeah okay so I shared you um links to these two repositories so I have a repository called move this way sample Tom Card application so I did not build this application I basically clone it from another another repository and uh there is also um Cloud foration stack and let's have a conversation about this Cloud foration stack and this Cloud foration stack I could I refactored it because the initial Cloud information St fors was um integrating this with code commit so I refactored the cloud information stack to be able to talk to GitHub now um this is not a cloud formation class I remember we already spent some time to look Cloud for ISU so I I expect that most of you would already understand um this by looking at it am I correct to assume that yeah yes this is familiar stuff good so we have the parameters that defining our parameters so in the parameters uh if you want to integrate that with an SNS we have an or email we have a repository branch that actually talks to the branch of our repository in in GitHub the format of the repo which is the par which will be expected by the stack and what what T keys that and T values that will be attached to each two instances do we want want to attach an is keypad to is2 instance Define defining a parameter for that okay and the metadata section remember we talked about metadata basically um more information about the resources and the resource section this is the only required section of our Cloud information stock and in here we are having our code build rule so the code build rule is where um we are setting up the build permissions for for code bill to be able to uh make our bill right so we have a basically this is familiar thing it's a role we have a trust relationship and this is the trust relationship we as as the uh asking the code Bill service to be able to assume that trust that role right and this sets up the permissions of the rule we also have a codee pipeline and code pipeline rule does the same thing it's a role once you create a role from the conso you need to give a trust relationship for a service to be able to assume that role right yes right right sir Prof um question you're looking the file you're looking at this um cloud formation file in the repo you sent to us where is it exactly it is in the it is the report just give me a second let's discuss it then I'll show you in the repos okay okay I sent you two reports one called giop something and the other one called um the application so let's let's have the conversation about what the code is doing then we'll talk to look solve your worry so we are creating the role and we establishing the trust relationship for the role and we are also giving the permissions for our code pipeline rule so we're telling the our code pipeline rule that you're giving it permissions to be able to orchestrate talk to code build to able to talk to code deployer to able to send our stuff to S S3 to able to also talk to SNS if you want to use Lambda we integrate that with Lambda it is via Cloud information so we want it to also able to talk to Cloud information and um we want code pipeline to also able to talk to Secrets manager why because um we need an Authentication token to be able to authenticate to our GitHub so and that token will placed in secret manager and code Pine should be able to actually go to secret manager and get that token okay so what resources of the services we'll just keep it St for now okay just back up a second please one up so above the code block above this one you just described actually identifies the service is that what you said where you you talking about here yeah I'm talking about line I think between 92 and 98 I think is that where you define what service that you are trying to assume the IM r or make assum so this is the r this is just a service um you know this type AWS I am R so you're saying that I'm creating a rle now who can assume this rule if remember when you're creating a rule you need to tell it from the console that what is the role for is it an E2 E2 rule is it a Lambda rule is it for S3 so this is where we are doing the assume rule okay so STS assume rule watch service code pipeline gotta gotta okay this ex exactly the same thing above we're creating a role we're telling it that service coulde build should be able to assume that role that rule yes okay exactly and the next part is basically policies so permission policies right you understand what this is you're giving a creating the role is one thing giving the role the permission to do stuff is another thing so the policies give the role the permission to do what you expect rle to be able to do in your AWS account am I too fast umon is it a best practice to have them roles um separate since it's the same service you're assuming why don't you have them like a list all together do we have to do SE what do you mean by the same service uh this basic look at look at the permissions that okay I think I I I I probably during the B I changed this but if you having different Services they have the they they need different levels of access the permissions that could bu need it's not the same permissions that could deploy right okay okay okay so it's it's best practice the principle of of best it call least privilege give who give who uh permissions to the service to be able to only do what they need to do what they need to do yeah yeah exactly so but this is not what you would do in your typical environment because this is an admin Ro that I'm telling C to have so that was I I changed that probably during my debug because it failed so but exactly um for the code pipeline I'm telling you to be able to talk to code bu code deploy S3 SNS Lambda cloud formation and a secret manager now this is also very permissive because I'm saying go to codb and do anything go to and do anything yes so this is also permissive so for handson that's okay once you get into your environment um um always use the principle of Leist privilege okay yeah now with our code Bill remember we said the code bill needs what we call what do I say build project so this is where you create the build project for code build you see that yes so if you go um this is not uh how you call I did not invent this wheel right guys you know AWS how it called can somebody direct me to it uh that's the AWS documentation isn't it yes how do I get the resource cloud formation could bu right just search it ons build that's the three this is The Bu project that we we actually creating so all the bu project that has all the properties you want artifacts configure the artifacts is it bash enabled is it Bill config is it cash description and we have encryption keys and all this stuff so that's where it's coming from just FYI so we are creating our build project and the build project it's a code build uh service code build remember we created a role up there right which we call the code build rule so we saying that it depends on that role basically it needs that role to be able to do what it needs to do and the name we're saying we're using C parameters for the stack name the description service role and stuffs like that so type of artifacts here we're not producing artifacts but we're going to trigger artifacts to be actually generated in the code pipeline because we are going to use AWS code pipeline to orchestrate all this thing all right so we also said that code build what does it need it needs the build environment and we set this build environments are based on Docker Imes can you see that so this is the darker image which going to actually represent our build environment but this is our C no this is our managed Docker image the code build is able to actually use custom Docker images so if you're in your environment you have your setup in such a way that you uh application needs some specific um customizations which the AWS manage or AWS provided images do not meet your need then you can build your um own Docker image put all the dependencies there and make that image available to code build to use right by the end of this week I expect you guys to be able to build images so this is something we will not uh go home on Friday night if we are not a able to to build our images okay okay good so you build the image and make that image available to code build so like I said darker image is basically a mini at a very low level a server on its own and in here you can package whatever dependencies you want to package into the image okay once you run the image then um it makes everything available for code build actually to be able to build your project okay so now what is the source we're saying that the source is from GitHub all right so it's passing the source from GitHub time out for our bill is 10 minutes so the time out in minutes is 10 minute and if want notifications we have that uh configured with SNS we can actually do all these go to the console and we we we do all this but I thought it's faster we just have a cloud formation St that orchestrates everything to us with us for us okay good so code deploy now also has what we call um code deploy uh environments so it needs an environment to be able to uh deploy our instances in let me try to find this a second working with working with these two instances taking these two instance so you can see that um Cod deblo actually uses the tag for instances I'm trying to look for give me a second a specific link and um they could deploy each two instances that's the problem with Json with Json you can you're not able to put in comments oh you can't uh do they can't comment in Json F that sucks unfortunately I think yes this is the E two instances so you have um um some already pre complied cloud formation templates from AWS which you can able to use to set up an is an environment for deployer so most this is instances that code deployer needs to talk to it needs to have what we call the code deploy agent so the code deployer agent already has to be running inside the is2 instance so that code deployer can actually make calls to that um is2 instance and actually deploy into those is2 instances so AWS already has some pre-built um code deployed cloud formation templates which you can actually use to uh set up this so we are just using that so it's cloud formation it's um basically um bash script that is running in the cloud formation template with the CFN in need and CFN H and CFN stuff which you talked about do you remember that or it's completely forgotten BR yes I speak for myself but I have no recollection of what you just described forgot do you even remember the words I just used I'll have to I'll have to refresh thatn yes I remember word what the CFN we talked about cloud formation helper scripts when we talking about cloud formation here CFN in CFN H CFN what the other one oh oh oh oh my goodness yeah I I know every day we are piling more information I understand the feeling and now do you remember I never we never talked about CFN help CL helper scripts in this session we did we did I just have to check my notes that's the thing help scripts so lesle Flor abdala we or I'm assuming we did Sir you're not zo in good so B basically um basically these um uh cloud formation launch temp uh could deploy templates are also using all this uh B scrips to be able to make the agents um available or the is to instances so now this is where we have the nest stack and this is an example of a nest stack that is actually calling one of those um uh making that um calling one of those manag templates to actually create an E2 instances R for COD deploy basically that's what I was talking about that I I I went through all that just so you could understand this information so we are deploying a cloud information template and in the nested stock which we did talk about and for you to be able to deploy n stack then you need to call the results AWS Cloud information stack right I don't have a great feeling we don't have total recall right now but we have notes that's the thing so we have to reference those notes good so um uh the code deployer application we said that it needs an application and we calling this is the code deploy application in it's a deployment group we also setting up the deployment group here and the deployment group is using um the app uh application the code deploy application which we just talked about we just said that code deployer has what call the application the deployment group and this stuff right and um now we have the code pipeline stack and in the code pip L stack it is what is orchestrating every um resource which you've already created cre so the Cod pipel start now is having our different stages and um we creating the Cod pipelin stack here it depends on the build project and we having the different stages so we having where Cod Pine should keep our artifact so we setting up an artifact store and it's S3 and the different stages we have the Source stage so where this code pipeline watching for our source code is watching it in the g g GitHub and it's using the provider GitHub to be able to to talk to GitHub and it needs [Music] um so how does it get triggered is there a trigger anywhere for the pipeline so pools sources for change do you see that so this is how you tell it to do your own pool SC so it pulls uh for Source changes so it's true you can also have different types of trigger so this is how we are also setting up the pool ACM here and we it needs an O toen to be able to authenticate so this is how it's talking to secret manager it's looking for a Tok token called GitHub token which we are going to create and um uses that toen to be able to authenticate to our GitHub repository okay then we have the build stage in the build stage the code pipeline should trigger the provider called code build to be able to make our build and um put the build then output an artifact called my build app and the deploy stage is going to use that output artifact if you see the uh the the first stage which was the source stage output an afca artifact called my app are we together let's STI this all over again so the source T is where actually pulls our code from SS it outputs an artifa C might have the build stage now uses that output from The Source stage as its input artifact my app builds it using the provider code which we already set up above then it outputs an another artifact called my build my app build okay and it's using the build stage should be using provider code be us and the deploy stage now uses as input artifact from our build stage then deploys our environment to the deployment group are we together I have a question sure so so I'm looking at this code right it's like what almost uh 300 lines of code and and on the light of Cl because he blown out everything on the same line but yes un understood I'm just I'm just trying to put a scenario in in the real world of an office where we are expected to kind of be familiar with this this type of thing are we the ones expected to write this thing or are we just going to take the template and tweak it accordingly okay if you're in a very new environment and they do not have that then you have to find templates from uh different sources documentations AWS documentations and stuffs and TW it this is a template that I just TW okay okay okay so I did not write it from start to end this is what was a prebuild project by AWS but this was using Code comit so what I have done is I've disintegrated it from c k and I've introduced C deploy sorry okay giop makes sense okay makes sense so most environments yes and um trust me when when you're setting up a pipeline this is not a one day two day three day task you be doing it and testing it's failing doing it and testing this is not even just a Sprint maybe when once you become that experienced then you can have us one week two weeks to set this up okay especially for completely uh new environments that they do not have um um repositories because for example if you join an environment like us we have different versions of cicd pipeline so all we need you is just go to pick one tweak it and make it do what you want it to do so in that case we will not be giving you two weeks to get that done right earlier they were maybe patient to have a week two weeks but now there's also Char GPT that will help you do what you used to do in 10 days in two days or even in three hours so yes everybody it's out that to be used so use it your AI tools use it so the the the goal is do not make it do everything understand what it is doing it should help you uh be um efficient I have I have personally have an issue with with because I've seen one or two people that are just they don't even want to know what CH is doing they don't even read the explanation they just put it there take it and give it and when you ask them the Code that they pushed and what is this they you can see that they have no idea so that's the problem but it is a faster faster if you use it um properly is really going to help you it's really going to because from time to time I remember before we used to use stack Overflow that's what we had when you have this problem you have to go to stack Overflow and you basically have to read from one uh thread to another one thread to another but now charp I think they've also trained the charp model with all the data from stock overflow correct so yes so it's it will basically able to spit out that information for you in less than no time and in a coherent manner because before you had to go to stock overflow you read like 10 different threads and you basically uh put that information together yourself so use it properly please that's just what I'm all that long story also search maker yeah that's from AWS right page maker I see that in my environment yes s maker it's an AWS tool Sage maker oh s s maker yeah s a s a g e right yeah yeah yes yes it's an AWS to so there there are a bunch of tools out there even um cicd platforms oh no wonder it doesn't really pull up nice what you what did she say no wonder it doesn't work work well because it'sss AWS to no um AWS tools are especially when it comes to AI they are very powerful yeah they are when it comes to comparatively I don't think so when it comes to data analysis I data analytic tools um I think I'm not an experienced person in that that space But from the trend that I see especially in our environment I think that aures better but when when it comes to Big Data AWS why do I say so I have a lot of companies or a lot of projects that you used to use um um Azure Cloud for their um AI things and they basically moving to migrate their data into AWS I remember telling you people that we had a set of Jenkins that they not are in a different cloud and stuff like that right and those were actually AI environments that they trying to run their mods in different cloud from and is not working and they want to B migrate them TOS that's where we come in interesting but um why do you say why do you think IVs is not good enough maybe you're right I don't know that's not my maker the one I'm talking about okay you're talking about a specific maker yes this the one have it doesn't the output when you compare when I do it on my personal computer I do it there the output is quite not similar like it has very big gap when it comes to the Quality so your personal computer is much more powerful thans instance no because I use I used chat DP all my personal computer I do the same thing I do with search ma but difference is there with the output yeah I can't use on my environment yeah she's CH GPT and S maker and she's thinking that oh that's what you're comparing I get it that's what I'm comparing that a difference output okay this is a very minute feature from your s Maker Now was Sage maker built like that I don't know to be honest good any question if no question then let's move on to the demo is it that it's not clear or it's understood where processing according to evet it's okay so we can move to the handson I'm sure we understand more when we are doing hands on but it's so um I think somebody should share or should I do it it's a very short hands on it's a really Shar sh the the repository yeah Pro the repository you shared doesn't look like the one you have on the screen no I was not sharing a report that was my vs code that's what I was pushed to the repository are you still seeing my screen yeah yeah you you plan to push this one to the repo and then it's already there oh okay let's take a look so this is the repository I shared with you right two of them actually so this is the code you see this not seeing anything what are we seeing do you see my screen no we're seeing your vs code we're seeing your vs code actually that's all we're seeing why are you seeing my vs code yeah we are seeing the vs code now I'm switching you should be seeing the screen and not my vs code again still what do you see now vs code tools GitHub Json that's the file we're looking at right now okay yeah he shared the same um Theo in the chat so there okay it's the second one yeah not the first one just has read me and [Music] Jason now you can see my screen right no no Noe yes should good so this is where you have the developer tools and G of integration and there is also um repository for the application like I said I didn't build it I cloned it from somewhere from the initial project good so you have the this repository which I said AWS could deploy sample Tom card and in here you have now the r me which we can use to actually set this up so please somebody should share I want to go through the process of doing all this um doing this or somebody it's a very short hand son so I don't think uh anything where's Mya boy of greatness they not here who me Franchesca you you have never shared so go ahead and share what makes you think I've never shared go ahead on my [Music] phone you think I have everything on my this then okay don't need any it's a clation stack right so it's a CL you just pardon what do I need it's not going to cost a lot of money right it's uh after that you can delete the stock once we done oh so it's because of many people are running away how much is it Prof it's not going to cost you $2 ah it's a it's a T1 instance that is going to spin up hello EV is sharing now I thought Francesca was sharing okay let me stop sharing then you do it oh let me stop was somebody sharing yeah was sh so oh IET was sharing now I just I just started sharing yeah really yeah oh so that means you override me then then share continue with your baby space continue disconnected actually continue okay that's a pretty baby yeah okay so here we are my so um the reason you have this GitHub repository is just so you you able to copy the code once we get to deploying it so you just come here and you have raw let me restart the recording oh we've been recording S I thought I posted no no so um the repository with the r me the other repository they could deploy sample I think that's where it is so here we have the r just scroll down to the r me yes one scroll down just hold scroll down to the end of the file you should see the read me so read me always be displayed here right you don't have to yes at the bottom yes yeah so the first thing is we need to create an S3 bocket so just create an S3 bucket in in US is one and you should use the syntax called code Pipeline and you have the region code and you have the account ID why because if you remember when we were talking about the P the discussing the template you saw that the code pipeline is expecting to put artifacts in an S3 bucket right so this is the um basically the syntax of the bucket that Cod pipeline will be uh looking for so that's the the bucket name what mean yes that's the that's the bucket name okay sorry what's the bucket name the number no this is an example account ID I said the syntax is should be code pipeline oh region the region Ione ID yeah code pipeline Us East one and then account ID so that's just an example but won't you have the account ID after you've created the bucket no you have your account already right your own now oh okay oh I see yeah your own your own and your own region oh I see it should be us East Us East one please do in North Virginia I think this template was built for North Virginia so it might be looking for Amis in if you deploy it in another region it might look for Amis that are not available [Music] I I already started I don't know if I'm supposed to be waiting for you to you you have you have um Place Hold Us in this thing that you have here do you see that yeah so I'm supposed to replace this with us East one and then my account ID exactly yeah without the account ID without the iPhones that been easier for you to copy the example um the example and then remove the account ID that is there and replace it with yours it's a lot easier cleaner because you still Us East one right yeah the EG on the right that one on the right is what you need yeah copy the whole thing and replace the account ID with yours after you copy it why she's doing that if there's any question any clarification which you need please do not hesitate your account ID evet just click on evet the top right that yeah then you can copy the first one yeah all of it right what's the I East one that's East one oh sorry sorry sorry not East one uh Prof you said the the iPhones are supposed to be right no for no hones in the account ID if you look at account her account ID here there isn't any no there's always this but I do not want this inside the the name do you see what I mean so without theones I think it's not there it's not there if you copy it like that it's not there okay yeah you're right we don't need any tax right no tax just create a bucket that's fine or the pipeline would be looking for is a Buckhead with that name that you credentials so configure credentials we need to generate a GTH up token so this token will be used by our pipeline to authenticate with GitHub so how do you generate a Tok token so get has something apart from your username and password and your s Keys you also has something which you call p8 personal um she has to for for the repo first right no no this is for your GitHub this is GitHub it's not it's not it's not repo specific okay so the steps are in the Run book so go to the Run book and let's see what it says it says go to GitHub you're already there so you can just open it in the new tab so you need to go to the on your gith up page to your top right is what we call your profile there right MH is it top right or top left which one top right right yeah that's right yeah the that logo yeah yes click on that so scroll down to settings that is correct all the way down and then you're going to do s um where is it tokens on your left scroll down to something they call developer settings so scroll down you should look for developer settings at the bottoms exactly then you we [Music] want P so personal access tokens at the bottom so I want the token classic but I don't think we've done something like this before right we have not yeah okay go ahead we have we haven't I would remember so we pardon I thought we did tokens for son Cube itens cube is it the same concept yeah it's a token right is um um a token is just some another type of credential which is not username and password but you using it to get um access to some sort of restricted platform or endpoint it falls under the Ambit of security right when you're talking exactly of security it's the same thing that rules do remember that when you're using rules AWS rules to access something you're not using AWS username and password you're not using access Keys secret access keys because what AWS does in the background is it generates a token that has a duration so once you assume that rule that token is passed to your identity because you assumed the rule and that rule has a token now that token has a a validity period maybe 24 hours maybe 10 hours and if you uh using that role and you accessing an environment after the validity period Then it tells you that please you need to resign relog in or something like that I don't know if you guys have experienced or something like that when you're using the so basically a token is some another way of passing credentials to an entity okay yeah okay just is that the same thing they say when they reference jit what they call just in time is that the same concept just just in credentials I mean where you have to you know it's it's based on a time time span and it's it's temporary you use it and it expires and then you have to actually kind of you know a new one a new one yes yes um from your explanation that's the same concept but your jit that's the acronym I'm not so first I saw that acronym in one of the interviews that I Was preparing for exactly so a token just basically now you can have tokens that are indefinite but the the goal is um you're trying to give credentials to an entity with without actually using something like username and password right so if you generate a token with this with this token you're still able to access GitHub okay and tokens are are mostly used by endpoints like two different endpoints trying to make API calls between each other use always uh great to have um use tokens it's easier to use tokens okay at least those that develop those apis my is here a developer you should tell us I think it's it's easier for you to integrate um uh a token Frameworks rather than having to make um username and passwords and stuff like that right absolutely sir good so we need to generate a token has generate new token so I can create the password right you're not creating a password this is your password need your password for your G he wants to be sure that you are who you say you are V token so you can describe the name of the token what is this token form you can say this is for a qu C pipeline or something like that now you can see here this is an expiration can you click on this drop down so you can tell um basically how long do I want this token to be valid for do you want the token to never expire then you say no expiration do you want it for three months do you want it for six months or two months and stuffs like that okay are the advantages of using um usernames and passwords but if you create a username and password for somebody on the platform um anyway I haven't seen a tool that you can actually expire the username and password but it's easy for you to do that with tokens okay so we want to uh check Ripple so Scopes basically tells you how to scope how to the boundaries of the permissions which you're giving to this token so that's basically the scope all right so repositories so I want you to check there's a check boox there so that's why I have a checkbox in the in the r me so check the repo and you also check the admin. repo so there are different Scopes if you're using GitHub has something they call GitHub organizations so if you're using GitHub organizations then you have to come and check the admin. organization and stuff like that admin. repo this one right admin. repo yes is there any admin. repo again I think that's all wait SC wa wa wait I think I saw something admin repo did I so we have this and delete repo okay then that should be good wait so say if you if you you can read on the Scopes and actually see what it it says go to the Scopes which you just checked so the repo basically gives you full control of private repositories access to uh commit status access to deployment status access to public reports access to repository invitations and read write permissions and and for security events and stuff like that and all the different Scopes so basically they scoping the permission right do you understand what the Scopes here yes sir setting like the boundaries or the exactly boundaries for what the token can be used for okay so once we create the token once you Scope the token then you can then generate the token so click on generate token then it will generate our token token so you need to copy the token because if you close this page you won't find this token again you have to come here and generate it again once you generate the token let's now um uh keep it somewhere that code pipeline can be able to actually access it and the best way for us or the best place for us to keep it will be actually AWS right because good pipeline can then access so let's use secret manager manager go to your AWS and uh Secrets manager there you go I new Secret yes go to secret manager um I think we we treat the SEC manager right Dr chilos I see your hand raised are you on mute doesn't it doesn't seem so at least no icon shows that he's he's muted all right maybe maybe has some o issues we can't hear you just FYI what I'm saying yes I said that can you show Meo where get that password on repo use the password it's my password that I created when when I was doing my when I created my GitHub account oh oh oh I think I was trying to understand were you talking about the token she just copied all the passw passw you know he wanted to sign password yes it's your password because yeah think about it this way you're trying to create credentials on a platform the the platform actually really needs to know that you're the owner of that account that you're trying to create Creations in right so that's why it's asking you to authenticate again so it is your the password of your G up account ah okay okay now understand okay yes security okay I understand so which one do you select here for the secrets manager other type so you can say just give me a second let's just discuss it since we are here so you can see secrets for ours so if you want to create secrets for RDS did we is this the first time we coming to Secrets manager yeah so I don't remember any of this okay the AWS has what like 200 something tools did did we tweet this guys I don't remember I think Prof suan talked about it I'm very sure about it it wasn't in class so if you want to create secrets for for RDS databases then you're going to use this if you want to use document DB if you want to use red shift if you want to create student for other types of databases now since our GitHub doesn't fall in any of these then we go to other types of Secrets you can see it here it says API Keys o credentials o to others right so we can create our give me a second so so sir it's not we're not creating anything we're supposed to be storing it right yes we are storing so we are creating a secret here to store our token oh so creating a secret in secret manager is not generating a secret you're just creating a container to actually host your secr to store our secrets in okay yes got so secret manager is not used to generate Secrets no it's just used to keep your secrets yeah right so you go to other types of secret so now the secret is a um key value pairs I in the key what is the key I said you use you said we were going to use a uh token right token so it's not going to be key value right no it's a key value pair okay what's the key though so so the key give me a second I want to see okay o o token o yes okay o so you type O upper case oer please copy it from the wrong book it from the wrong book okay but if you have sorry do you already have your uh okay you did not close the page so it's fine yeah she saved it on the notepad too so scroll down this one yes what talken uh I think it's one word not no space okay scroll down scroll down somewhere you actually find it hey uh there is exactly yeah yeah so the key is O and the value is the token we Rec created we generated yes then your key your token interesting good so once you have that do we want to encrypt that we are not encrypting our our [Music] our uh we are not encrypting using our own manage key so if you do not specify a key by default AWS using an AWS generated key to actually incr your secret so just leave it at default that's why it says AWS secret manager that's an a owned key then you go to next so you can give it a secret name and our pipeline we remember the the pipeline will be looking for a secret with a particular name and it's called giop token I think give me a second to be to be sure yeah that's what in the wrong yes exactly gith hop token so now these names are here but if you look at the pipeline especially the the code pipeline stack on the block of code pipeline you will see that it's looking for this information here once she we go that then Ure there's no space before and after please no white space you can give it a description if you want that's not important for us now um generate uh store the uh click on next and store our tool welcome interesting so um okay she already skipped the page um there is um a feature one of the great features of secret manager is what we call secret rotation so you can actually configure for applications that are able to handle secret rotation configure the rotation of the secret so that just in case just in case your your secret is exposed or somebody is in your environment and the person leaves and is angry and wants to do something secrets are rotated but your application in the background should be able to handle secret rotations right so those are just the features of it now uh the application using this should also be able to handle that so that's done as our secret go to Secrets just to show make sure we have our secret so click on secret itself so we can see secrets and make sure secr I st first I Haven you st yeah stall please I didn't St beautiful so we've successfully stored our secret in secret manager so that's our token then now we need to go for rep repository so you go back go back to give up and the application side of things fog that repository into your space this one or the other rep the one where you have the read book The read me I think the second one right no this is the one the application part that's what I want to F yes for into your space yeah go on now click on that and then uncheck that that box if you can the one that says main only I think it's safe to do that I think yes go ahead create Fork So Pro the uh secret manager that rotation thing is like if they were to ask you a question um how um will you implement although there are many other things involved with it for uh to uh rotate um access keys for say an application or an individual that's where that rotation area will become exactly so it's an added um security feature right to actually rotate is actually best practice for example you should have an sensitive application that application should not be using one credential for the whole year right from time to time every two weeks you need to rotate the credentials just so that if something happens then you're trying to reduce your blast radius if that those credentials are leaked whoever is using the credential at some point should uh um should be invalid so secret manager can do that secret rotation but now um not all applications and trust me so many applications out there are not able to handle secret rotation the developers don't want to embed that into their code when we tell them oh we need to do that is a please that's a whole another project so the application and the application side should be able to handle secret rotation really yes okay so when you run into that says that they wouldn't do it I mean how do you get around that now there are so many things right it's from an infrastructure perspective I can't force you to do it okay question if I can't force you to do it it's a feature which is good now the stakeholders the owners of the application can force you to do it if they they determine that the security of that is Paramount then they can force you to do it the developers on their own side can tell you okay for us to implement secret rotation maybe in the initial contract package of the your demand for the software that was not part of the deal and they say oh to to um add this feature to our our our application you need maybe 20 P PT how do you call it 20 working days that's cost right that you the stakeholder or the project produ product owner should be able to Bey now if you say okay yes we'll pay you for the 20 days for you to implement that then the developers implement it then they can use the feature so the feature is there but not not all applications can do it now we we we had a customer it's a banking environment and one of the applications which we are hosting it's not using it and we propose that to the customer because as an infrastructure team we're talking to the customer directly the customer went back to the developers and said we won this and the develop said no um the Frameworks which we use if we want to implement this when just be it's better to build a new application from scratch and that was the end of the topic end of the conversation wow if they know that I mean they can also still find ourselves that they they are not selling the truth that they can that was not the truth right that you want to be yeah that's their story and they're sticking that's the story of the de team that's not our problem we're talking to stakeholder they so they came back to us and said the developer say it's it's not possible and they need to be new application if they want to integrate that because they using Legacy blah blah blah blah blah okay so it's there for you to use the application should be able to handle it no where did we stop okay we are waiting for you the application so you fued the repository did you Fu it already yes sir once you have that foged then we need to now deploy our template so in the other repository you have the cloud formation template there does she have does she have to go to vs code you do not have to go to vs code okay at least for the template for the uh Cloud information template you do not have to but for us to see our changes then you would the reason why you fed it is because you would clone this make the changes then you push it back so you can see the changes as um the Cod pipeline is orchestrating the whole thing but can she can she make the changes in her own um git yes you can actually make the changes on vs code itself on um geub yeah that's what I meant yes so you don't have to go to vs so exactly so we we don't bore you with that okay good so let's go to here please go to uh raw so select the AWS developer tools GitHub Json which is our Cloud information template do we have to F or no you don't have to it if you don't if you don't need okay I'm putting it here just so you can access the template itself I could also just give you a zip file here or copied it fire and give you but I want you there because I'll be developing it since I completely removed it from the original uh template that was having Cod kit so I made my changes here good so you can copy raw and it's a template so we need to go to cloud formation now and actually spin up this template awesome this is quite interesting which one say we should one of them which one you don't have to Fu the template repository the cloud information template but the application you can Fu okay or you should Fu okay because you can't make committee my own right okay right okay you have tax tax here what in you've been busy hm you've been very busy I've been trying to delete this ones and they don't want to delete I don't know why so these are nested Stacks nested Nest nested Stacks can when you delete the parent stack that this nested Stacks will go where are the parent Stacks I I've deleted it for sure they are not here so must not be here it must can be in another region are you sure this is not a cross region something go down this is amplify amplify amplify amplify scroll down yeah all of it is amplify cck click on the last one last no go back radio button the what the what button go back to radio button right select select the last no not that one last last the bottom the the bottom yeah so that's what we call a radio button exactly click on delete a drift we please let's go to our so let's let's create a stack yes so scre start in designer so with standard resources that's fine oh no okay so we want to build nine for standard resources so we want to build from infrastructure composer this part is not in the Run M because I assume you know this right yeah we've done this so many times infu so uh create infrastructure computer so we can go to template and we space our template the template you just complete security good so now scroll up a little bit to the block of code that says code pipeline this is not very exactly stop I wanted to show you where this is being used anyway let's go ahead I'll show you at the end if you go to line give me a second I think I can find it and maybe direct you hopefully uh the line numbers are the same across platforms what are you looking for sir I wanted to show you where those token is being called okay so if if you go to that the build do [Music] deploy there is a source uh the source section of the pipeline there something called the o o um artifacts so line from my side 235 so there's a 235 slowly slowly slowly please you're too fast there is it exactly talk you can see is looking for the token called GitHub token okay interesting are we together so the GitHub is is a that's a standard name it's some it's not something you you give it it's so this is the name we gave to the Token we saved you remember itop token is that yes and this is the key will give to the Token or token so it's using the name and this key then once he gets these two things then it can retrieve the value that is attached to this key got and that value will be the token got and if you scroll up please can you scroll up slowly I just also want to show you why we created an S3 bucket stages artifact store exactly here artifact store you see the type here is S3 what where is the S3 bucket scroll down it's expecting a bucket that has join you remember this join which we we were talking about functions is joining the region and the account name to the prefix code pipeline do you see that interesting okay exactly so that's why we were creating a bucket with that specific name because this block of code would be looking for that if it doesn't see it then the pipeline will fail fa exactly so now um create our template please when you see quotes like this don't be scared you have to would you have to oh you validated already yes she validated already when you see Cod like this don't say 500 lines of code don't be scared basically minimize every section and take it one step step at a time that's why when we're doing cloud formation we started one step at a time there are parameters there those parameters are being called there functions there they split they join they insert they whatever those all those pieces put together is what is being built here now once you understand that it's just AIT matter of repetition that's that gets it to that level right so were we I was I supposed to select the the bucket we created no okay is going to look for it the statement there saying that is going to look for it so next let's go to next basically now we give your your stack and name any name any name and now it goes to what we call the um GitHub reposit it needs to be able to talk to a repository and the repository should be in this form so based on the uh script which I put in there it needs to be in the form of owner and repo so if you go to your GitHub repository copy the string you always use to clone so go to the GI up repository for the yes go go to clone yes copy just copy the string so we just hold hold here so this is where I mean by owner and repo so this here is the owner part of the string and this is the repo part of the string so we need to throw this out so let's just copy that into your clipboard and take it back to cloud formation we can then edit it there yeah so you can see the format owner repo highlighted up there please follow that format remove the first part and the Slash the slash here no the slash is good and the dogit good so that's the the entire thing is a repo she has yes this is the repo okay inside the code itself it's already joining there's a joint Commander will join https G to all this thing now to give you the final um string to actually talk to the repository if you add that in here then the string will not be correct okay good so um the name of your branch so if your branch is mean they leave it mean because by default I put it mean if it's Master then you need to change it to master but I think it is mean and the deployment key ke an is2 instance key so in the report in the read me fire you're supposed to create a key pair but if you have a key pair in this region that should also be fine we have a drop down there select one of your key pairs whatever that's good so uh you need to T the is instance with the T key of name t value of code deploy is to t you can give me an email address because I was trying to with SNS but that's not fully build so you can just put it in the Dom email there you put in your email that's a g and click on next and we deploy that so the email part will not work though it's not fully integrated into uh into the notification because I was trying to make it to be able to notify you even when when there's a build happening and stuff like that okay gotcha but for end to end project after microservices this everything will be there but the goal here is actually just to see make you see how code pipeline or streate and you can use so you need to acknowledge the capabilities but do we do we write our the the the valid the email email address do give your valid email that's no problem okay see any issue with so submit good so this will then create our stack it takes some time it takes a couple of minutes about about U five minutes so you're very confident it's going to pass uh hopefully it pass everybody should start praying we should I've had so many nightmare scenarios with this uh cloud formation we'll be holding our breaths then at the last minute it doesn't pass everyone should go and sleep there'll be a but I think this one will be fine I think we did everything correctly so keep the invent up can you minimize a bit so I can actually see do control minus there you go no that's not one we want the same page just minimize the Fone size so we can see more of the events exactly you can uh close this uh side of your screen or pull it this way pull this this way please so while we wait um there's any question please show no question for me my before before we do the um Theo we have to it first before because you need to make changes there you can't make changes to M you don't have right privileges in my repository right okay I have a question yes so what do you think might be the reason why my stacks W delay it's a drift it's saying there there that it's saying there that there's an I am r that is looking for is not saying did you read that error message in red I did what did he say I canot remember but something like what you said why did anyway let's not have your debu in the recording what once this is done then we'll look at it about how long does take I think takes about five minutes or so I don't know I pause the recording right yeah if you can good so we said that um it's complete because if you scroll it from bottom to top this is how events are locked in the stack so the first events from bottom then as resources are being created so these are the logical ideas remember we already talked about all these things you should already remember them right so great I have a what what St can you give me can you explain more on those P um give me a second I don't want to but nested stack just from the the the the name you're nesting a stack in another stack right so the parent stack is the the stack which is calling another a child stack all right did you okay you said you were not part of a CL foration let's let's let me clarify this to you after the the recording is stop okay let let I want to be I want to be on record please no there is a record for that not this one I don't want others to um um except there you guys are okay with it for me I don't have a problem put it there please I want you to be recording there's a recording for that particular if you if you like can you put leave it leave it this please um I already said it Dr chilos we cannot do that and other the others that are going to listen to this please there's a recording for the cloud information thing but I said I would clarify this to you once you stop the recording so the others that want to watch the recording do not have this we already have so many recordings that's that's trying to clarify that topic I just said I just said give me a second please okay okay I just said you have what we call a stack and if you remember this um stack which evet created EV created that stack and she gave the stack a name called jjag right so this is our parent stack but if you look at the template of this tack um if you go to the template if I if if you go to the template of that stack please don't if you go scroll down there's a part of the template that is calling another template so you're nesting a template inside a template so that's a nested stack all right good so now let's go to the topic of today if you go to um our stack is complete so if you go to uh pipelines go to AWS pipel yes pipelines so just leave this leave this um TP open double uh duplicate the tab so go to pipelines just pipelines right or AWS Pipelines code pipeline sorry code pipelines yes code pipelines that's it good pipeline so you've created a p it should be a a pipeline now here yes created 6 minute ago so uh it's coming from if you look at the name uh JJ Tech code Pipeline and stack ID and stuff like that all this naming convention is actually build into the template itself so you can scroll you can click on the pipeline so we can see a pipeline here you see this is the pipeline that takes the source that pulls our information from um gith giop it pulls our information from GitHub if you remember the pipeline stack if you look at the template itself you have different stages there but the different stages will represent the different stages of your build so if you want to have a build that pulls the source you want to have some integration testing all those other things are not integrated in our simple pipeline here right but if you want to embed testing into it if you want to embed whatever other tool you want to integrate into that then you can have this different pipeline so you can have a pipeline with like seven different stages so as is a simple Pipeline with three stages so you have the source that pulls our code from SCM and you have the bill stage and the bill stage this is where we are invoking uh code bill to actually do the bill so code Bill bills our code and produces an output and code deploy then deploys it so you have this three different stages so code deploy actually then uses code deploy to actually deploy our code to an ec2 instance so there was a a part of the template that actually already prepared the deployment environment if you remember when I talked about it in the template when we discussing the template right yes sir and good so if you go to an go to E2 you should see an is2 instance there which the template created using the uh uh uh Stacks which I showed you so that was a nested stack inside here that actually prepared this deployment environment so can you click on it uh good so copy the is2 instance IP go to the uh um your browser and and part 80 and we should be able to see the application which is available there is Pam still with us P I'm here is it going yes it is it's gone good good so this is the application that has been deployed to our is two instance any question so far if I think you have low battery no sir it's not I always bring down my brightness no I'm not talking about brightness yeah the power on your laptop oh it's okay it will survive it's okay okay so if no questions to this point then we are fine I said the is two instance is being deployed by or is been prepared by the embedded stock that's inside this template right so now we can actually test our code pipeline by making some changes so this is I did not do this this is a jav application which I like I said so okay so you go to the repository go to your GitHub repository and let's make changes here so in the uh Java application that is using a landing page no this is this is I want the application repository please so but the the the the normal procedure will be for you to pull this your your your your PC you clone the repository you make changes there you push it back right yeah good so if you go to source so we talked about the structures when we discussing Marvin so if you go to source source go to web apps and go to Pages there's this index.jsp so This index.jsp is actually the file that has information concerning the our landing page so the landing page is called uh it's written something is written on doggies or something so let's find that and actually make changes so you can go to uh go to edit mode since we want to do it directly on the console so you go to edit mode and um Place whole I'm trying to look for can you screw can you go to the landing page please just leave leave this St open go to the landing page what I mean by landing page is where you actually uh shows what is deployed in the application on the web browser the web browser exactly the dogs yes scroll up can you copy this into your clipboard peep best book suits for dogs just copy the string so we can find it in that yes um on this landing page we we develop our Mode work where you can see the HTML code back in F12 I don't know try it please so I want to look for that string here do I paste it search no inside inside please go no no no no no here so click inside like you in edit mode you're already in edit mode just click in here then you do uh command f contrl f you mean yeah contrl F then you paste in past that the string enter so you can see that and um I want you to paste it in replace go to replace the replace dialog box and let's change those two strings so it sees that there have four places no no no no no no that find and replace you should be able to use this find and replace so whatever string that you're looking for you place it in the find and if you want to replace it with something you place that in the replace okay okay okay sir so this is the same feature that's on vs code so can you pleas um uh just again replace what is in your clipboard no go to replace pleas the simp same thing I just want to add a few letters there and let's just put um B suit for docks whatever add some some some strings here exactly so now click on replace all so that should replace that in um exactly everywhere that that original string was found wait to together yes so that's it then in Tech so you can commit that information so there's a orinal commit miss that update JSP that's committed so that should be committed now that we've made our change to the application let's go back to code Pipeline and see what happens why do you have so many different browsers why why not one t it's go back to C pipeline refresh the code pipeline where is the code pipeline please so code pipeline should be able to see that okay um some changes have been made click on it list and let's wait to see if could pipeline PS refresh let give it give some few minutes and let's see what happens like that so I let closing in 10 seconds I expect Cod P line to be able to pick our changes and starts doing the B exactly do you see what so is in progress yes oh okay it happens in real time with that trigger set yes it happens with that trigger set so you can also see it here so it has saw it it changes then pcks picked up the changes now the bill is happening because of our changes then once the bill is done it has an artifa sends out thata the orchestrator which is code pipeline picks that arria calls code deployed that please take this and put it to the the deployment group and could deploy and deploys our application to the end users make sense so there are no um let's put it this way in in in cloud formation there are no um what's the word I'm looking for um something like uh P requests in terraform I mean in um in a GitHub that will have somebody make sure that the code change was validated by senior a senior um senior person where would that be where you'd have to before it commits in GitHub it'll be in GitHub right okay yes remember that when you were setting this you were setting pipeline to watch the master brand oh yes the normal flow would be these changes she's going to do it in a feature Branch creates a PO request and you as a senior or whoever is the owner of the project is going to review that P request if it's okay approves that P request then she merges that into the master Branch only after that merge that the pipeline would see that there's a change in that Master Branch but now for the demo purposes we've lipped all those those okay gotta gotta gotta gotta but you understand the flow right we've already treated this flow a couple of times several times yes yes well so this change would actually happen after she must have cloned the repository done her changes done her testing pushed it to her own Branch creates a pull request somebody reviews it if it's a critical environment or a critical application you can actually set it in GitHub that I want at least two approval so it means two different colleagues of yours must approve your pull request before it can be merged into Master if you have just one approval the merch the merch icon will not be highlighted until you get the second approval then you can merge into Master then pipeline can see that there's a new commit into in master then it pulls that and deploys yes understood so if your deploy is build is successful is deployed done yeah so deploy is also successful you can go back to your browser where you have the application refresh and let's see if we actually have um our changes in JJ Tech exactly that is beautiful thing to see so uh Prof so the cloud formation the code we use in cloud formation is like the um pom.xml file that we using Jenkins sort of no it's not the cloud formation is just remember we're building a Tom card server building Nexus server building Marvin building all those things so with AWS now we are building all that in one go that clation stack right but if you go back to the application code that's where you have the application the p. XML file is an application specific thing right so that's where it is so if you look at the the repository with the application you should see the the the the form the XML file there so it should be at the root of the directory can you go to your your your GI up so that I could answer that question or she can see what I mean so exactly so if you go to the source scroll down no the pom.xml file and the source should always be at the same level except if not you have to tell the pipeline where to look for that P XML um collapse this thing so you can see this is our pom.xml file here so if you remember we talked about be specs you can have your bill specifications actually uh embedded or packaged with your source code right and the build could build is going to use those Bill specifications and stuff like that interesting are we together yes sir good great so um 99 or 90% of the cases you wouldn't you as a devop engineer you're not responsible for this file this an applications thing but like I said it's good for you to know why because you're responsible for the pipeline when the pipeline fails you are the one to debug the pipeline so you might debuging the pipeline and it's telling you that it needs a certain dependency then you need to know that I have to go to the pipeline and actually see if this dependency is there or is not not there if it's not there you can tell the responsible people that my pipeline is failing you send them a failure message because this is is's looking for this and he's not seeing and stuff like that good if we are good then I think yes so yeah um okay what if you in a situation whereby like you just mentioned the last statement that if a pipeline fail you just want to see what the ER message is saying and then get to the um engineer the developers that buil the code right to fix it so what if you are in a situation whereby the stakeholders want you to fix it they want you to do the configuration change and all that meanwhile it's not your job line what do you do how do you know it's not your job line in the sense that you did not build the code for you to change the code right you don't know the dependencies that is in involved the people that built it should know some of the dependencies that you maybe you may not be aware of so now um you're you're making a general statement and I'm assuming that yes let me answer it from a general perspective yeah and I want to use this scenario I joined a company and they gave me this pipeline or this application to build right and I set up the pipeline now the pipeline part of it you as a devop engineer your job is what is in that cloud formation template because that's where we are setting up the pipeline are we together yes sir now as the application guys they are responsible for all this p. XML file and stuff like that like I said from the beginning of all these Devils we said it's a culture that um uh for STS collaboration so as a devop engineer you will not just sit in your room shut your doors and do everything I expect it to work there's always that collaboration between teams now even the guys building the um the the p. XML thing they might they would always collaborate with you because you're setting up the pipeline you must have some requirements are you responsible as a devops engineer to actually build the pom.xml file from scratch no why because you do not have good knowledge of the application it's dependencies so those application dependencies are should come from the application team you as a devops guy are you responsible for writing test the test of the application no because you're not a tester and secondly you're not an application um uh expert however you might find yourself in environments that you have developers that have switched and some of them will assume those rules I don't try to um what's the word meet up with that I tell you categorically I've never been a developer I don't know that do that and give me now why do I say it's important for you to know the structure of the form. XML file like I said if you are a devops engineer and you're working in a team and use a Java application and you're using mavin to do your bill please or grer whatever Bill to using and you're responsible for the pipeline please spend some time to actually deep dive into that tool understand the structure understand some uh inep have some inp knowledge about it because if the pipeline fails you the the first point of contact is you as the engineer who set up the pipeline the devops engineer you need to know why the pipeline F and it's somehow um embarrassing if you carry your pipeline and say oh this pipeline failed because of the application and they go back to the application and it's actually a problem from you so you need to be aware and be sure before talking to the application guys that oh this is your problem now all it depends on the relationship you do have there are some teams or some application guys that we at that Cod here that immediately the pipeline face I said see that thing has failed again please come I'm tired let's look at it together and they will join me to a call there are others that they only ping them when you're sure that the problem problem is theirs so it all depends does that answer a question yes sir but I think sometimes you know some of these stakeholders MH they could if you're not very careful they could be kind of mixing up mixing things up I feel like they kind of want to set people off for failure no it also it depends on what you told them the interview exactly depending on what you sold yourself I I don't really think so because there was a team that is in charge of this that I just said in in charge of the application and then there was this guy he is he is a tier one tier two support right but then