Jenkins and DevOps Pipeline Overview

Jenkins is an open source automation server, which makes it easier to build, test and deploy software. In this course, you'll learn how to build a full DevOps pipeline using Jenkins, Linode servers and other tools. Gwynne Faraday is your teacher for this course. Gwynne is an experienced software developer, and she's created a bunch of popular courses on both her own channel and the free CodeCamp channel. Jenkins can help developers automate their software development process and improve their productivity. It can also help users obtain a fresh build of their software project more easily. Jenkins is an important tool for creating a DevOps pipeline. A DevOps pipeline is a set of processes and tools that enable the continuous delivery of software applications. The term DevOps is a combination of the words development and operations. DevOps pipelines are used to automate the build, test, and deploy phases of the software development lifecycle. The goal of a DevOps pipeline is to make the software development process more efficient and effective. DevOps pipelines help to automate the tedious and manual tasks that are often involved in software development. This automation can help to speed up the software development process and make it more reliable. Jenkins is a flexible tool that can be used to create custom DevOps pipelines. And that's what you're going to learn about in this course. So at the end, consider leaving a comment with something you learned in this course. Hi, everyone. My name is Gwen. And in this course on Free Code Camp, I'm going to be building a full DevOps pipeline using tools like Jenkins, Linode servers for hosting our application as well as our Jenkins server. We'll be using GitHub as our code repository. Docker and Docker Hub and some related tools. Now to follow along with this tutorial, you can see the app that I will be using for deployments on my Faraday Academy GitHub organization. It's called the curriculum app. If you followed my YouTube channel, you might have seen that I actually built this application and maybe 50 or so live streams a few years ago. There is also a readme here on GitHub. where I have each step in the process listed out as well as links and relevant resources and this architecture diagram of how everything fits together in the process. Jenkins is a tool used for automation with continuous integration and continuous deployment pipelines. The project started in 2004, so it's been around for almost 20 years, and it's completely open source. Right now, it claims that it's the most commonly used automation server in software development. So why would you want to use Jenkins as an app developer? It allows you to automatically watch for certain events in your repository and react to those events. You can do things like... build your code, run scripts, perform testing, and then deploy your app if everything passes, or reject the deployment if the test fails. Either way, you will be able to see logs of every step throughout the Jenkins pipeline and the results of the deployment. Jenkins can ensure that all of your tests are run in the same environment or as many environments as you want. for consistency. Another benefit is that Jenkins has a very large community that it has amassed over almost the last 20 years. So there are lots of tutorials and community questions and answers that you can find. Another feature of Jenkins is its plugin architecture. To use Jenkins effectively, You will want to install the plugins that make sense for your project. These plugins can include options for compiling or testing. They have a Docker plugin which we will be using as well as a Git plugin that allows us to watch our GitHub repository and react to changes. Jenkins is self-hosted by default. You don't need to pay for an enterprise tier. This gives you a lot of options for configuration. It's also a tool that has been proven to scale since it's used by so many large companies around the world. It's not just large companies that use Jenkins however, many small or mid-sized companies use this tool as well. Why do companies use it? Many of them say that it improves their software delivery cycles by making them faster and more performant. What are some of the benefits of using Jenkins versus other alternatives? There are plenty of new tools in the CI CD pipeline space. You could choose Travis CI or CircleCI. There's also GitHub Actions. There's GitLab CI and many more. Some of the differences are that Jenkins is open source. Many of the other options are not. A big one is that you can self-host without paying for an enterprise plan and you have total control over configuration. One thing that might be helpful if you're considering Jenkins as an option and comparing it to other alternatives is their Jenkins YouTube channel. They have a lot of information about Jenkins and they also have this playlist of real users of Jenkins, real companies. talking about why they chose Jenkins and what benefits they see from using it. We talked about some of the benefits, now let's talk about some of the cons or potential drawbacks of using Jenkins. So plugins I listed in the pros column because there are hundreds of plugins covering all different kinds of use cases and you can usually find a plugin for what you need in Jenkins. You also have the option to create one yourself. That can also be a drawback though since you are relying on community supported plugins that might not be updated or documented well. There can also be some confusing overlap between plugins since anyone can create their own Jenkins plugin. A common complaint of Jenkins is the outdated UI. More modern tools have interfaces that are a little bit easier to use. than the standard Jenkins interface. Now Jenkins does have a newer interface that you can install as a plugin and use for a lot of things but not everything and that is the Blue Ocean plugin. We're going to demo that in this tutorial as well. Another potential drawback is that since it's been around for so long and it's community supported sometimes the documentation or answers that you find can be out of date. This is not an uncommon problem for software tools in general. Overall I think Jenkins has pretty good documentation but it is something to be aware of. Checking and making sure that the documentation you're reading is up to date with the version of Jenkins that you are using. Another huge consideration to keep in mind when choosing Jenkins is that it's completely self-hosted and maintained. Unless you are using a third-party solution to manage your Jenkins server, you will be the one installing Jenkins, keeping it up to date, managing the controller and agents and all the different aspects of the Jenkins pipeline, and also ensuring security, which can be tricky sometimes using a tool like Jenkins. where you need to install so many different plugins to use it. Before we really dive in here, I want to make sure we're all on the same page by defining some terms that I'll be using in this tutorial and common terms that you will see when you are using Jenkins as well. Let's go over some of the terms that we will be using in this tutorial just to make sure everyone is on the same page. First, we use the terms continuous integration and continuous delivery, commonly abbreviated as continuous delivery. CI CD my favorite definition for continuous integration actually comes from Atlassian it says continuous integration is the practice of automating the integration of coding changes from multiple contributors into a single software project. Developers can frequently merge code changes into a central repository where it builds and then the tests run. Continuous integration is a precursor for continuous delivery. Continuous delivery is responsible for packaging the app and getting it ready to deploy using automated build tools. This whole process is known as a pipeline. A pipeline is basically a well-defined set of steps that build, test, and deploy applications automatically. It basically takes you through the continuous integration and continuous delivery processes. You will see that we also have a build step in our pipeline that we are going to be using. If you are new to development or have only used something like JavaScript, maybe you aren't familiar with a build step, but generally the build step will spin up an environment, compile code, and create an executable if that's necessary. When we talk about testing code, that is a step of the pipeline that will run any unit end-to-end integration or whatever other tests that are needed for the application to make sure that it's ready to deploy into production. if the tests pass. There are two other terms that are Jenkins specific here. The first is controller and that's our main instance of Jenkins that is running. The Jenkins controller is responsible for configuration, key management, plugins, and it's a centralized hub to manage all of the agents connecting to it. Agents are usually container environments that will run your pipeline steps. These steps are also called a job which means some work that needs to be done. The Jenkins controller is responsible for assigning it to an agent and then the agent will either spin up and run all the steps of that job through to failure or deployment. You can also set the agent to just stay running all the time instead of spinning up for each job that you want to run. This is a simple diagram of the architecture for this tutorial so you can visualize what we're going to be doing. To the left we have the app development side. This is a full stack JavaScript. application used for the demo and code is committed directly to a mono repo in github on the other side we have a Linode account and two servers running in Linode these are called called the nodes in that interface. One of them is running Jenkins, also known as the Jenkins controller. And the other server is using Docker to run our application. The Jenkins controller will be configured to watch the GitHub repo. And when it pulls and sees changes, it will initiate the CI CD process in one of the agents. The agents will run all of the steps in the pipeline to check out the code from the branch we designate in GitHub, run tests, and deploy if everything was successful. If the pipeline fails for any reason it will immediately stop and of course not deploy the application. And either way we can see the build results and the logs in the main Jenkins controller. The first step is to sign up for a Linode account. If you are a first time user of Linode, when you sign up for an account they will give you a $50 credit. that is good for the first 60 days for you to try out. That will be more than enough to follow along with everything that we are doing in this tutorial. You can sign up for your account with whatever service provider you want or using your email. I already have an account so I'm going to skip to the dashboard view. You can see I have some servers already running here that I used for testing. In this left-hand menu, you can see the services that Linode has to offer from Linodes, which are servers that you can spin up, volumes for storage, other things like firewalls, if you want to connect your domains. They also have an object storage solution for things like images or static resources, similar to using AWS's S3, and a marketplace. that gives you one-click installs for a lot of popular solutions. We are going to be using a couple apps from this marketplace in the tutorial today. You can see I clicked marketplace and it went ahead and took me through to create a new node. Select a marketplace and shows me all of the options here. The first server that I am going to create is one that's already set up with Jenkins. If you were installing Jenkins yourself, you would have a couple options to install directly or use their official Docker container. The Linode marketplace option here just installs it directly on the server that we're going to spin up. I'm going to scroll down to more options. It gives me the option for the Linux distro. Now, depending on what option you choose here in the marketplace, which one of these you choose, You're going to see different options available down here. For the Jenkins container, these are the ones I have to select. I'm going to see what's available. Right now it's only Debian 10 that's available for this Jenkins server. And now I need to select a region. I'm just going to select the region closest to me, in New Jersey, USA. And then I need to select what kind of server. By default it'll show you all of the options with dedicated CPUs. How many CPUs, how much RAM, and these are a little bit more expensive. For this tutorial I'm just going to use the cheapest one, the nano 1 gigabyte, which if you look at the Jenkins documentation is more than enough to run Jenkins. So I'm going to choose this, scroll down. This is the name. This Linode label is the name that you saw in that first table view. So I'm going to give it the name FreeCodeCampJenkins. You can optionally add tags here about the type of servers, the technologies, or something else you want to differentiate between Linodes that you spin up. I'm just going to leave it blank. And now I can choose a root password. This password that I enter here is something I can use when I SSH into the server. I can also add an SSH key and SSH with the key. I'm just going to generate a root password here. So I'm going to use my password manager which is bitwarden. I'm going to come down to generator. It generated this unique password it's 30 characters i'll be deleting this server after the tutorial so don't need anything crazy let me do 10 characters copy and paste i'm not going to add an ssh key for this i'll just use the password i do want to be sure i save the password though so what i like to do is come in here i'm gonna add this username and password Let me paste in the password. And then I called the Linode FreeCodeCamp Jenkins. So I'll just put that in the username. Save it. And then I have the password stored. So I can copy it later when I go to SSH. I don't need any of these options. I can go ahead and hit create. It will take a minute. It says provisioning. While that's going, I'm going to create a second Linode. Now for the second one, I could just create a custom one. but I'm going to go ahead and use their marketplace again and create one with Docker already installed and set up. So I'm going to use their Docker Linode from the marketplace. I clicked on that and scroll down for the options. So now you can see I have special options for the Docker server. For this I'm going to leave these options blank. Instead of using an SSH key again I'm just going to use a password. And I'll select the latest WN11 region. Again, I'll do Newark, New Jersey, shared CPU, nano. And this is the server where I'm going to host and run my application. So I'm going to say free code camp application. For the root password, I'm going to do the same thing, generate a password. copy, enter, and then I'll go ahead and add that password. Create. That's provisioning. If I go back to my list, my FreeCodeCamp Jenkins is up and running. And this one should be done in a minute. Now let's check out our running instance of Jenkins. You can go into the Jenkins server I just spun up. And I can actually access the running instance of Jenkins at this IP address. I'm going to go ahead and do that. It should be at port 8080. And usually when you would pull this up, you would see a login screen but because this is our first time it's prompting us to enter a password. Now this password we will have to get from this file when we ssh into our server. I'm going to go ahead and do that. Copy this ssh. Now I can ssh into Linode and it's asking for my password. Since I didn't set up a key for this instance, so I'm going to go ahead and use password authentication and I did save it in my password manager. So I'll go ahead and copy the password. I'll enter it in here and now I'm inside the server. So let me go back to Jenkins and it said to copy the password from this file. Let me take a look at that. I'm going to cat that file and that prints out the contents so now I can copy this password. Now if I go back to the browser and I can just paste that password here, click continue, and since I'm just setting it up for the first time it's asking what plugins I want to start with. You can either do this manually, Or if you're just getting started with Jenkins, it's probably best to install the suggested plugins. You can always add or remove plugins later, but as you're using Jenkins you'll realize that a lot of plugins depend on other plugins and these suggested plugins just give you a good base to start with. So I'm going to click on that. It's installing. This takes a few minutes. Now I can create the first admin user. I'm going to call this Jenkins FCC and now I need a password. I'll go ahead and generate this password. Just a short simple password. Copy it. I'm going to just use a dummy email address here. Jenkins FCC. This is just a public email address provider. And before I go, I'm actually going to save this user. The username. There we go. So I'm going to click save and continue. I don't have a domain or another configuration so I'll leave this how it is right now. I can always access it via this IP at port 8080. So I'm going to save and finish and now Jenkins is ready to start using. Next let's do a quick tour of the Jenkins interface. To start, you can see they have this top navigation and then also this sidebar to the left. I'm going to start off with the top since it's a lot simpler. So first I have this Jenkins FCC, that's my username. So this is just a quick link to my user profile. I'm going to go back and I also can see any notifications or warnings or error messages linked here. Because I clicked on it, it took me to my settings page. I am going to go back for now and come back here in a minute. Go back to dashboard. And starting from the top, if I click on new item, you can see this is a quick link to create a freestyle project or a pipeline. So if I want to create jobs, I can come here and do that. I'll come back here to do that in a minute. Notice whenever I go into one of these navigation items, you can see I get breadcrumbs here where I can navigate back up. This people navigation item just lets me manage any users that have access to this Jenkins. Right now I just have the one admin user, but really you might have DevOps and developers and other people with different levels of access to this interface. and you can actually give quite fine-grained permissions on what they can or cannot do. Build history will display information and feedback of the jobs that we run. We'll be coming back to that later as well. I'm going to skip down to views now. You can see there's nothing in this views page yet. This is basically a dashboard view. of all of your jobs and you can also use this new view link to create kind of custom dashboards for viewing different kinds of jobs or maybe for different users to see different information. So I'm going to go back to dashboard. Now I saved manage Jenkins for last because this is where all of the configuration and plugins, key management, This is the page we will be spending most of our time on initially to set everything up so that we can create pipelines and run jobs. First at the top you can see that it will let you know when there is a new version of Jenkins available. Actually this is a little bit out of date or the version that I have in this linode is out of date anyway so it's letting me know. And you can see if I scroll down at the bottom here, it says the current version that I have, which I believe is like maybe 25 minor versions out of date. So I do need to go into my server and update that. If I want to see information about my current version of Jenkins, there is an About Jenkins section in this Manage Jenkins dashboard, which will take me to a page that also shows my version and information about the version of Jenkins that I'm using. I'm going to go back. Now some of the main settings here are your settings to globally configure different things about Jenkins and also paths. This configuration page will let you manage a lot of the core settings and features of Jenkins as well as give you setting options for some of the plugins that you install. I'm not going to touch any of these things right now. I'm just going to leave them as the default. Go back to Manage Jenkins. Manage Plugins. This is where we can search for, install, or update plugins or even remove plugins that we don't need. We can do all that from here. I'll come back here in a minute because we will be installing a few more plugins. I'll go back. Managing nodes. See we have the default node. which is the controller. Now to start I will be running some jobs on this default built-in node, which is good for trying things out and getting started, but it is recommended that you create nodes just for running jobs and then use this default built-in as the controller without running jobs on this node itself. We'll come back to that. The other important area that I'm going to point out is this manage credentials. When you create other nodes that are going to be able to run jobs, you will need to add credentials here to give those nodes permission to access whatever resources they need to and for the controller to communicate with the nodes. This is a very simple overview. of the Jenkins interface. Next I'm going to add a few plugins and then there's going to be another part of the interface called Blue Ocean to demonstrate. I want to point out here that Jenkins has kind of a dated look and feel but it is completely customizable. You can use a theme plug-in, use a theme built... by somebody else or create your own css and javascript to make this look and feel however you want i'm not going to be installing a theme but i will be installing kind of a new ui section of this jenkins interface to manage pipelines which is called like i mentioned blue ocean Now I'm in the Plugin Manager and you can see the tabs here. Updates are for the plugins that I already have and have available updates. So I could come and select plugins I want to update and click download now and it will install the update after I restart Jenkins. Which as I'm downloading it does give me the option if I want to go ahead and restart Jenkins. I'm going to deselect these. Available is just a list of plugins that I can check off and install as well. I'm going to come back to this. I can see the plugins that I already have in this installed tab and advanced is some options that I'm not going to look at right now. So let me go back to available. Actually I'm going to go ahead and filter here and first I'm going to look for Blue Ocean. You can see there are several plugins here. I'm going to go ahead and install the main BlueOcean plugin. I'm going to say download now and install after restart. It is in the process of downloading and installing. You can see and I'm going to check this box at the bottom. If you can see it says restart Jenkins when installation is complete. So now it says Jenkins is going to shut down. It will just take a minute for Jenkins to restart now. You can see that this restarting Jenkins is running. Now that it's finished, I can log back in. So I'm going to go back to the plugin section in Manage Jenkins, Manage Plugins. And I'm going to go to available and this time I'll filter by Docker. So this first option is the main Docker plugin and there is a second one for Docker Compose. I'm going to do the same thing and download now and install after restart. You can see that... Some of the plugins have dependencies of other plugins. The only ones I selected were actually this Docker one and the Docker Compose one. But because they have dependencies of Docker API and Docker Commons, Jenkins is also installing those plugins. So it will install the plugin you select as well as any dependent plugins if they aren't installed already. This is going to take a few minutes to... Install these. Now that we installed those plugins, You will see some additional options in some of the settings pages in Jenkins. And you can also see this new sidebar item saying Open Blue Ocean. Let me click on that. You can see this is a completely different interface because it's a modern UI for pipeline management in Jenkins. It's still linked to the old interface. So if you click on some links here like administration, you can see it takes you back to the old interface because Blue Ocean doesn't have a Managed Jenkins section. Let me go back to Blue Ocean. You can also click this kind of exit icon and that takes you back to the regular Jenkins dashboard. Normally you would see a list of the pipelines you create here. but since there are no pipelines it gives us this box prompting us to create a new pipeline. So that's what we're going to do next. Let me start by creating a new pipeline. It's asking where my repository is hosted, which is GitHub, and in order to connect to my repository and be able to run jobs I need to use my GitHub access token so I need to create one. This warning in red here is just because I had previously created an access token you probably will not see this. I'm going to come here to create an access token. It will take me to my GitHub account and I can go straight to the page where I can create an access token. If you want to get here on your own You can access this page through settings and developer settings and then personal access tokens. You can see that by clicking on the link the necessary permissions that I'm going to need for Jenkins are already pre-populated and selected here. So I don't need to change anything. I just need to give a name to my personal access token and then I can also change the expiration if I want. This is just a name so when I look through my tokens in GitHub I can know what the token is for. I won't be using this name anywhere in Jenkins. I'll just be using the token itself. So everything else is set. already and I'm going to go ahead and generate the token. The token does disappear if I leave this page so I'm going to go ahead and copy it and now that it's created I can go back over to Jenkins and enter it here and click connect. So that worked and now that I'm connected to the GitHub account with my access token, it shows me the list of all the organizations that I belong to and I'm going to select the Faraday Academy organization because that's where the rebuild I want to use is located and in that organization It shows me all 14 repos and I'm going to be using the curriculum app for this tutorial and then I just hit create pipeline. Now it takes me to this interactive pipeline creator screen and you can see a message just popped up that said no Jenkins file located. So let me just go to the project in GitHub so you can see what it's talking about and in this repo in GitHub You can see there is no file called Jenkins file. So by default, Jenkins will look for a file called Jenkins file in the root directory of a repository. Now what a Jenkins file is, is a file that contains instructions for a pipeline written as code. So you can read the file. and see each step defined in the pipeline going from top to bottom divided into stages and steps which we'll talk about in a minute. Now because this is a file it's checked into git and versioned and can be managed or rolled back just like any other file in git. Now since the file doesn't already exist in our repository, when we create the pipeline in Blue Ocean and save for the first time, it will automatically create a new Jenkins file for us and commit it to the repository. So let's go through that process now back in Jenkins. You can see the message down at the bottom here saying A stage is required. Basically the pipeline is broken down into stages which are like sections. Every stage has to have at least one step and the steps are what contain the scripts, calls to Jenkins plugins, or other actionable tasks to be carried out in the pipeline. Both stages and steps can be run in parallel or one after the other. You will see this as we go along in the pipeline creator. I'm going to name this stage something like checkout code and now I need to add steps. So I'm going to add a step here and find the git plugin that I installed earlier. And it's asking for the URL of my git repo. So let me grab that, paste it, and I need to denote which branch I'm using. For the repo I use dev as the default branch. So I'm going to use that. And now I can click save. It says that saving this pipeline will commit a Jenkins file to the repository, which is exactly what I want. And since there is no Jenkins file, of course, it will create a new one for us. I'm going to leave the description blank, so it will just give me a default commit message. And it's... pointing to dev so it's going to commit directly to the dev branch. So let me save and run that. And it failed the first time it was trying to run. Sometimes it passes on the second try so let me retry. And it failed again so let me look at the logs and it says that the recommended git tool is none. It gives me some information here on what's going wrong and why it's failing. It's saying that git does not exist. Basically, I spun up this server using Jenkins from the Linode marketplace, but that server does not have git pre-installed on it. So I will need to install that directly on my instance. And since I don't have any kind of automated setup for the instance, I'm just going to SSH directly into it. So in Linode, I'm going to go to my server and instead of using my own terminal this time, I'm going to use the built-in one that they provide in Linode. It's called Lish, so I can click launch Lish console and it prompts me to log in here. So ignore the previous incorrect logins. I'm going to log in as root and then grab the password. that I saved in my password manager and I successfully ssh'd in. Let me make sure that git is not installed. I can do that with the git dash dash version command and it's telling me that git is not found. So before I install it, I'm going to do an apt update. to make sure I have the latest versions of the packages. It's telling me that I can upgrade some of the packages I already have installed or they have new versions, but I'm not going to do that right now. For now, I'm just going to do apt install git. It's asking me if I want to continue installing 38.3 megabytes and now I should have git installed. So let me check again and yes I have git. version 2.20. And now that I have git installed, I'm going to try to rerun my pipeline over here. While it's running, notice that the colors change. So you have the color blue while it's running and then you can see things turn green as they pass. And then when everything passes, this whole thing turns green. As you've seen before, red means failure and there's also a yellow state. For certain cases like test failure, then it can turn yellow. So everything is very visual here. And you can see that now it passes since I have the correct dependencies installed on the server. So this is the most basic version of a pipeline. to create in Jenkins. So let me add a few more steps now. Now when Jenkins created this pipeline as I mentioned before it also created a Jenkins file. So when I refresh my GitHub here it added the Jenkins file and you can see where it says added Jenkins here. That is the default commit because we didn't type in a custom one and you can see if we go into the Jenkins file this is the stage that we created and the step to check out code with the GitHub plugin. Jenkins files are usually pretty easy to read because you can see well-named options like which agent you are running the pipeline on, a list of stages, and then the steps inside of each stage. And for the GitHub plugin here, I'm using the API that they provide and passing in the GitHub URL for the repo and which branch I want to check out. So let's explore the interface a little bit and then try to update our pipeline. From the top left, you can see which pipeline is running. I only have one right now. If I click on it, I can see the history of the jobs that have run and the ending state if they passed, failed, or are still running. And if I go into one of them, I can see that it has all the information. I don't have any changes yet. and I don't have testing that really runs. And you can see the fourth tab is for any artifacts that are produced. So if I click on this file, this is just automatically created by Jenkins. It's not something I explicitly have to create. But if we did create some kind of output, it would be here with the artifacts on this page. The next icon here. that looks like refresh will rerun any jobs. With the pencil I can edit the pipeline. If I click on the gear icon it takes me to settings in the old Jenkins UI. It takes me directly to the settings for this pipeline and also here if I go to branches and hover over any of the branches in the list here. Then I get a couple icons here and one of them is edit. So I'm going to go ahead and click on that edit icon. Now there's different ways to arrange a pipeline. I can add more stages. I can go back to the previous stage and add more steps. or I can add stages that run in parallel depending on what I want to do. Now there are a lot of options for different types of things that I can do in a step. Some of these like Docker and Git are from plugins that I've installed. I'm just going to use the regular shell script here and for right now I'm just going to list contents of the directory and this will show up in the Jenkins logs. It's going to create another commit to update the Jenkins file. So I'm going to add a commit message this time and I'll save and run that. And if I go over to my app, I can see that next to the Jenkins file, it shows the new commit message. And go into the Jenkins file and the new... Stage and step is updated. I guess I forgot to name my stage. I thought it was required. You can see in Jenkins here that when I make updates it's running twice. That's because I am committing with the Jenkins file directly to the dev branch. So that runs but there is also an open PR in my repository, this PR90. and I'm making the PR into dev. Every time I'm updating the Jenkins file it's rerunning all of the PRs. So let me go into here and it's passing. It defaulted to the name error here but anyway I can see that it printed everything out. Let me change that name. error. So let me rename the stage to log and save that. Of course now it's going to run again even though I'm just updating the wording in the Jenkins file. One thing to note here is that services like GitHub do have restrictions on API usage. So if you start to see failing jobs here, it might not be because of anything you've done wrong in Jenkins, but you might have hit the limit with... the GitHub API and then you'll either have to upgrade your account or wait until it resets. I think it resets every hour. So sometimes jobs fail because of a cap on API usage. But here it looks like everything went fine. So now I want to add another script to the pipeline. I'm going to come to branches and click edit again and I have this simple log script but let me just run one in parallel so you can see how the Jenkins file works. I'm going to call this front end unit test, add a step here, another shell script, and I'm going to paste in the command. I'm going into the curriculum front. directory where my front end is and then run an npm install to install the dependencies and then run npm run test unit so run the unit test and i'll click save there So this is going to run and I'll follow it here. I think it is going to fail because I don't actually have node or npm installed on my Linode server. So it failed. Let me look at the logs. and indeed it says npm not found. So let me go back to Linode and launch the Lish console again and I'm still logged in from last time so instead of apt install this time I'm going to install Node via the recommended command with curl. This is for version 16. You don't only have to use version 16, but this is just the version that I tested it with. It's a stable version. It's giving me more instructions here to apt-get. install node. So I'm going to enter that command and now I'm going to check the node version. It says version 16 and let me check the npm version because npm comes with node and I have version 8 of npm. So that should be all I need here. Let me try to rerun this now. All the other steps passed. This one's still going. It hasn't failed yet so that's a good sign. While it's running I'm going to switch over to GitHub and go to the Jenkins file and you can see that now it adds this parallel command. and denotes that these two stages are running in parallel and gives it the title of the first stage for both. And then each stage in parallel has its own title and steps. Of course you could just put this in one stage and have two steps as well, but I just wanted to show how it works to run things in parallel in Jenkins. This is taking a little while so I'm going to fast forward. So I actually re-ran this because I closed that other screen. You can see that everything passed this time including the new one for tests that I added. Now that we have a couple basic scripts working, let's add some stages so that we can finally deploy the app. to Docker Hub. Now I'm going to add a build step to our pipeline. I'm just going to create a new stage called build and then I can add a regular shell script. I'm going to use this shell script to run Docker commands. So I'll use the regular Docker build command to build a Docker image from this. docker file that I'm pointing to because the docker file isn't in the root directory here. It's in a subdirectory. So I need to point to that with this dash F flag. And I also have to let docker know what is my current context, which in this case is the directory I'm already in. So I'm just using the period here. And now I'm going to commit this. Now it's building and I'm waiting for it to build. I re-recorded this a few times so you can see the numbers skip around a little bit but I'm still going through each step in the process. Now that the build passed I need two more steps to be able to deploy this to Docker Hub. First I need to log in with my Docker Hub credentials to make sure I'm authorized to push images to my Docker Hub account. To do that I need to use the docker login command and the default place that it points to is Docker Hub. So I don't have to do anything special here. but I do need to use my credentials like my username and password. But of course I don't want to store my username and password for docker hub in the Jenkins file so I am going to use variables in Jenkins. So I'm going to add environment variables to my pipeline and then I will just point to those variables. from my Jenkins file. So you can see I can add the Docker Hub user and the Docker Hub password. I'm going to just add those values in Blue Ocean. If you aren't using Blue Ocean or you want to use the standard Jenkins interface, there is also a key manager in the settings over there. I think it's a bit easier to do directly in Blue Ocean here. Of course, I'm not going to show these variables on screen. You'll have to enter in your own credentials here. And now I'm building and testing just to make sure that my login worked and indeed it passed. The only thing left to do is to add a new stage here to actually push my updated image to Docker Hub. So I'm going to add another shell script and this time I'll use the docker push command. I just need to add the name of the container and optionally the tag but the only tag I have right now is the default latest tag so I'm just going to leave that. And it failed because it says that an image does not exist. So with my username and the name of that image. Now, since I already built and deployed this app a few years ago, I already have this in my Docker Hub. However, now that I want to push a new image from Jenkins, I need to tag it during the build step here. So I'm going to add a tag to my build script and I can just use the dash t option and I'm going to tag it with the same name that I'm using so my username and then curriculum front and I'll just add latest. Now let me save and run that and see if it passes. and it passed and pushed correctly to Docker Hub. Let me check over in my Docker Hub interface. It's at hub.docker.com and I'm already logged in. So I can see that my curriculum app was updated less than a minute ago, or rather my curriculum front app, which is just the front end. Congratulations on making it to the end of this video tutorial. I hope you were able to learn a lot about Jenkins and DevOps and Linode. As always if you have questions or comments please leave them in the comment section below. I hope you are also able to check out the follow along readme that is linked in the description. It has all of the instructions for of course following along with every step in this video. as well as some notes that I took along the way about things that I learned in the process of creating this tutorial. I want to thank Linode once again for their grant to Free Code Camp and making educational content like this video possible. I hope all of you have a great rest of your week and we'll see you in the next video.

It can also help users obtain a fresh build of their software project more easily. Jenkins is an important tool for creating a DevOps pipeline. A DevOps pipeline is a set of processes and tools that enable the continuous delivery of software applications. The term DevOps is a combination of the words development and operations.

DevOps pipelines are used to automate the build, test, and deploy phases of the software development lifecycle. The goal of a DevOps pipeline is to make the software development process more efficient and effective. DevOps pipelines help to automate the tedious and manual tasks that are often involved in software development.

This automation can help to speed up the software development process and make it more reliable. Jenkins is a flexible tool that can be used to create custom DevOps pipelines. And that's what you're going to learn about in this course.

So at the end, consider leaving a comment with something you learned in this course. Hi, everyone. My name is Gwen. And in this course on Free Code Camp, I'm going to be building a full DevOps pipeline using tools like Jenkins, Linode servers for hosting our application as well as our Jenkins server.

We'll be using GitHub as our code repository. Docker and Docker Hub and some related tools. Now to follow along with this tutorial, you can see the app that I will be using for deployments on my Faraday Academy GitHub organization.

It's called the curriculum app. If you followed my YouTube channel, you might have seen that I actually built this application and maybe 50 or so live streams a few years ago. There is also a readme here on GitHub. where I have each step in the process listed out as well as links and relevant resources and this architecture diagram of how everything fits together in the process.

Jenkins is a tool used for automation with continuous integration and continuous deployment pipelines. The project started in 2004, so it's been around for almost 20 years, and it's completely open source. Right now, it claims that it's the most commonly used automation server in software development. So why would you want to use Jenkins as an app developer? It allows you to automatically watch for certain events in your repository and react to those events.

You can do things like... build your code, run scripts, perform testing, and then deploy your app if everything passes, or reject the deployment if the test fails. Either way, you will be able to see logs of every step throughout the Jenkins pipeline and the results of the deployment.

Jenkins can ensure that all of your tests are run in the same environment or as many environments as you want. for consistency. Another benefit is that Jenkins has a very large community that it has amassed over almost the last 20 years.

So there are lots of tutorials and community questions and answers that you can find. Another feature of Jenkins is its plugin architecture. To use Jenkins effectively, You will want to install the plugins that make sense for your project.

These plugins can include options for compiling or testing. They have a Docker plugin which we will be using as well as a Git plugin that allows us to watch our GitHub repository and react to changes. Jenkins is self-hosted by default. You don't need to pay for an enterprise tier. This gives you a lot of options for configuration.

It's also a tool that has been proven to scale since it's used by so many large companies around the world. It's not just large companies that use Jenkins however, many small or mid-sized companies use this tool as well. Why do companies use it? Many of them say that it improves their software delivery cycles by making them faster and more performant.

What are some of the benefits of using Jenkins versus other alternatives? There are plenty of new tools in the CI CD pipeline space. You could choose Travis CI or CircleCI.

There's also GitHub Actions. There's GitLab CI and many more. Some of the differences are that Jenkins is open source. Many of the other options are not. A big one is that you can self-host without paying for an enterprise plan and you have total control over configuration.

One thing that might be helpful if you're considering Jenkins as an option and comparing it to other alternatives is their Jenkins YouTube channel. They have a lot of information about Jenkins and they also have this playlist of real users of Jenkins, real companies. talking about why they chose Jenkins and what benefits they see from using it. We talked about some of the benefits, now let's talk about some of the cons or potential drawbacks of using Jenkins. So plugins I listed in the pros column because there are hundreds of plugins covering all different kinds of use cases and you can usually find a plugin for what you need in Jenkins.

You also have the option to create one yourself. That can also be a drawback though since you are relying on community supported plugins that might not be updated or documented well. There can also be some confusing overlap between plugins since anyone can create their own Jenkins plugin.

A common complaint of Jenkins is the outdated UI. More modern tools have interfaces that are a little bit easier to use. than the standard Jenkins interface.

Now Jenkins does have a newer interface that you can install as a plugin and use for a lot of things but not everything and that is the Blue Ocean plugin. We're going to demo that in this tutorial as well. Another potential drawback is that since it's been around for so long and it's community supported sometimes the documentation or answers that you find can be out of date.

This is not an uncommon problem for software tools in general. Overall I think Jenkins has pretty good documentation but it is something to be aware of. Checking and making sure that the documentation you're reading is up to date with the version of Jenkins that you are using. Another huge consideration to keep in mind when choosing Jenkins is that it's completely self-hosted and maintained.

Unless you are using a third-party solution to manage your Jenkins server, you will be the one installing Jenkins, keeping it up to date, managing the controller and agents and all the different aspects of the Jenkins pipeline, and also ensuring security, which can be tricky sometimes using a tool like Jenkins. where you need to install so many different plugins to use it. Before we really dive in here, I want to make sure we're all on the same page by defining some terms that I'll be using in this tutorial and common terms that you will see when you are using Jenkins as well.

Let's go over some of the terms that we will be using in this tutorial just to make sure everyone is on the same page. First, we use the terms continuous integration and continuous delivery, commonly abbreviated as continuous delivery. CI CD my favorite definition for continuous integration actually comes from Atlassian it says continuous integration is the practice of automating the integration of coding changes from multiple contributors into a single software project.

Developers can frequently merge code changes into a central repository where it builds and then the tests run. Continuous integration is a precursor for continuous delivery. Continuous delivery is responsible for packaging the app and getting it ready to deploy using automated build tools. This whole process is known as a pipeline.

A pipeline is basically a well-defined set of steps that build, test, and deploy applications automatically. It basically takes you through the continuous integration and continuous delivery processes. You will see that we also have a build step in our pipeline that we are going to be using.

If you are new to development or have only used something like JavaScript, maybe you aren't familiar with a build step, but generally the build step will spin up an environment, compile code, and create an executable if that's necessary. When we talk about testing code, that is a step of the pipeline that will run any unit end-to-end integration or whatever other tests that are needed for the application to make sure that it's ready to deploy into production. if the tests pass. There are two other terms that are Jenkins specific here.

The first is controller and that's our main instance of Jenkins that is running. The Jenkins controller is responsible for configuration, key management, plugins, and it's a centralized hub to manage all of the agents connecting to it. Agents are usually container environments that will run your pipeline steps. These steps are also called a job which means some work that needs to be done.

The Jenkins controller is responsible for assigning it to an agent and then the agent will either spin up and run all the steps of that job through to failure or deployment. You can also set the agent to just stay running all the time instead of spinning up for each job that you want to run. This is a simple diagram of the architecture for this tutorial so you can visualize what we're going to be doing. To the left we have the app development side. This is a full stack JavaScript.

application used for the demo and code is committed directly to a mono repo in github on the other side we have a Linode account and two servers running in Linode these are called called the nodes in that interface. One of them is running Jenkins, also known as the Jenkins controller. And the other server is using Docker to run our application. The Jenkins controller will be configured to watch the GitHub repo.

And when it pulls and sees changes, it will initiate the CI CD process in one of the agents. The agents will run all of the steps in the pipeline to check out the code from the branch we designate in GitHub, run tests, and deploy if everything was successful. If the pipeline fails for any reason it will immediately stop and of course not deploy the application.

And either way we can see the build results and the logs in the main Jenkins controller. The first step is to sign up for a Linode account. If you are a first time user of Linode, when you sign up for an account they will give you a $50 credit. that is good for the first 60 days for you to try out.

That will be more than enough to follow along with everything that we are doing in this tutorial. You can sign up for your account with whatever service provider you want or using your email. I already have an account so I'm going to skip to the dashboard view.

You can see I have some servers already running here that I used for testing. In this left-hand menu, you can see the services that Linode has to offer from Linodes, which are servers that you can spin up, volumes for storage, other things like firewalls, if you want to connect your domains. They also have an object storage solution for things like images or static resources, similar to using AWS's S3, and a marketplace. that gives you one-click installs for a lot of popular solutions.

We are going to be using a couple apps from this marketplace in the tutorial today. You can see I clicked marketplace and it went ahead and took me through to create a new node. Select a marketplace and shows me all of the options here.

The first server that I am going to create is one that's already set up with Jenkins. If you were installing Jenkins yourself, you would have a couple options to install directly or use their official Docker container. The Linode marketplace option here just installs it directly on the server that we're going to spin up. I'm going to scroll down to more options. It gives me the option for the Linux distro.

Now, depending on what option you choose here in the marketplace, which one of these you choose, You're going to see different options available down here. For the Jenkins container, these are the ones I have to select. I'm going to see what's available. Right now it's only Debian 10 that's available for this Jenkins server.

And now I need to select a region. I'm just going to select the region closest to me, in New Jersey, USA. And then I need to select what kind of server. By default it'll show you all of the options with dedicated CPUs.

How many CPUs, how much RAM, and these are a little bit more expensive. For this tutorial I'm just going to use the cheapest one, the nano 1 gigabyte, which if you look at the Jenkins documentation is more than enough to run Jenkins. So I'm going to choose this, scroll down.

This is the name. This Linode label is the name that you saw in that first table view. So I'm going to give it the name FreeCodeCampJenkins. You can optionally add tags here about the type of servers, the technologies, or something else you want to differentiate between Linodes that you spin up.

I'm just going to leave it blank. And now I can choose a root password. This password that I enter here is something I can use when I SSH into the server.

I can also add an SSH key and SSH with the key. I'm just going to generate a root password here. So I'm going to use my password manager which is bitwarden.

I'm going to come down to generator. It generated this unique password it's 30 characters i'll be deleting this server after the tutorial so don't need anything crazy let me do 10 characters copy and paste i'm not going to add an ssh key for this i'll just use the password i do want to be sure i save the password though so what i like to do is come in here i'm gonna add this username and password Let me paste in the password. And then I called the Linode FreeCodeCamp Jenkins. So I'll just put that in the username. Save it.

And then I have the password stored. So I can copy it later when I go to SSH. I don't need any of these options. I can go ahead and hit create.

It will take a minute. It says provisioning. While that's going, I'm going to create a second Linode.

Now for the second one, I could just create a custom one. but I'm going to go ahead and use their marketplace again and create one with Docker already installed and set up. So I'm going to use their Docker Linode from the marketplace.

I clicked on that and scroll down for the options. So now you can see I have special options for the Docker server. For this I'm going to leave these options blank.

Instead of using an SSH key again I'm just going to use a password. And I'll select the latest WN11 region. Again, I'll do Newark, New Jersey, shared CPU, nano. And this is the server where I'm going to host and run my application. So I'm going to say free code camp application.

For the root password, I'm going to do the same thing, generate a password. copy, enter, and then I'll go ahead and add that password. Create. That's provisioning. If I go back to my list, my FreeCodeCamp Jenkins is up and running.

And this one should be done in a minute. Now let's check out our running instance of Jenkins. You can go into the Jenkins server I just spun up.

And I can actually access the running instance of Jenkins at this IP address. I'm going to go ahead and do that. It should be at port 8080. And usually when you would pull this up, you would see a login screen but because this is our first time it's prompting us to enter a password. Now this password we will have to get from this file when we ssh into our server.

I'm going to go ahead and do that. Copy this ssh. Now I can ssh into Linode and it's asking for my password.

Since I didn't set up a key for this instance, so I'm going to go ahead and use password authentication and I did save it in my password manager. So I'll go ahead and copy the password. I'll enter it in here and now I'm inside the server.

So let me go back to Jenkins and it said to copy the password from this file. Let me take a look at that. I'm going to cat that file and that prints out the contents so now I can copy this password. Now if I go back to the browser and I can just paste that password here, click continue, and since I'm just setting it up for the first time it's asking what plugins I want to start with. You can either do this manually, Or if you're just getting started with Jenkins, it's probably best to install the suggested plugins.

You can always add or remove plugins later, but as you're using Jenkins you'll realize that a lot of plugins depend on other plugins and these suggested plugins just give you a good base to start with. So I'm going to click on that. It's installing.

This takes a few minutes. Now I can create the first admin user. I'm going to call this Jenkins FCC and now I need a password. I'll go ahead and generate this password.

Just a short simple password. Copy it. I'm going to just use a dummy email address here.

Jenkins FCC. This is just a public email address provider. And before I go, I'm actually going to save this user. The username. There we go.

So I'm going to click save and continue. I don't have a domain or another configuration so I'll leave this how it is right now. I can always access it via this IP at port 8080. So I'm going to save and finish and now Jenkins is ready to start using.

Next let's do a quick tour of the Jenkins interface. To start, you can see they have this top navigation and then also this sidebar to the left. I'm going to start off with the top since it's a lot simpler. So first I have this Jenkins FCC, that's my username.

So this is just a quick link to my user profile. I'm going to go back and I also can see any notifications or warnings or error messages linked here. Because I clicked on it, it took me to my settings page.

I am going to go back for now and come back here in a minute. Go back to dashboard. And starting from the top, if I click on new item, you can see this is a quick link to create a freestyle project or a pipeline. So if I want to create jobs, I can come here and do that. I'll come back here to do that in a minute.

Notice whenever I go into one of these navigation items, you can see I get breadcrumbs here where I can navigate back up. This people navigation item just lets me manage any users that have access to this Jenkins. Right now I just have the one admin user, but really you might have DevOps and developers and other people with different levels of access to this interface. and you can actually give quite fine-grained permissions on what they can or cannot do.

Build history will display information and feedback of the jobs that we run. We'll be coming back to that later as well. I'm going to skip down to views now.

You can see there's nothing in this views page yet. This is basically a dashboard view. of all of your jobs and you can also use this new view link to create kind of custom dashboards for viewing different kinds of jobs or maybe for different users to see different information. So I'm going to go back to dashboard. Now I saved manage Jenkins for last because this is where all of the configuration and plugins, key management, This is the page we will be spending most of our time on initially to set everything up so that we can create pipelines and run jobs.

First at the top you can see that it will let you know when there is a new version of Jenkins available. Actually this is a little bit out of date or the version that I have in this linode is out of date anyway so it's letting me know. And you can see if I scroll down at the bottom here, it says the current version that I have, which I believe is like maybe 25 minor versions out of date. So I do need to go into my server and update that. If I want to see information about my current version of Jenkins, there is an About Jenkins section in this Manage Jenkins dashboard, which will take me to a page that also shows my version and information about the version of Jenkins that I'm using.

I'm going to go back. Now some of the main settings here are your settings to globally configure different things about Jenkins and also paths. This configuration page will let you manage a lot of the core settings and features of Jenkins as well as give you setting options for some of the plugins that you install. I'm not going to touch any of these things right now. I'm just going to leave them as the default.

Go back to Manage Jenkins. Manage Plugins. This is where we can search for, install, or update plugins or even remove plugins that we don't need.

We can do all that from here. I'll come back here in a minute because we will be installing a few more plugins. I'll go back.

Managing nodes. See we have the default node. which is the controller.

Now to start I will be running some jobs on this default built-in node, which is good for trying things out and getting started, but it is recommended that you create nodes just for running jobs and then use this default built-in as the controller without running jobs on this node itself. We'll come back to that. The other important area that I'm going to point out is this manage credentials. When you create other nodes that are going to be able to run jobs, you will need to add credentials here to give those nodes permission to access whatever resources they need to and for the controller to communicate with the nodes.

This is a very simple overview. of the Jenkins interface. Next I'm going to add a few plugins and then there's going to be another part of the interface called Blue Ocean to demonstrate. I want to point out here that Jenkins has kind of a dated look and feel but it is completely customizable. You can use a theme plug-in, use a theme built...

by somebody else or create your own css and javascript to make this look and feel however you want i'm not going to be installing a theme but i will be installing kind of a new ui section of this jenkins interface to manage pipelines which is called like i mentioned blue ocean Now I'm in the Plugin Manager and you can see the tabs here. Updates are for the plugins that I already have and have available updates. So I could come and select plugins I want to update and click download now and it will install the update after I restart Jenkins.

Which as I'm downloading it does give me the option if I want to go ahead and restart Jenkins. I'm going to deselect these. Available is just a list of plugins that I can check off and install as well.

I'm going to come back to this. I can see the plugins that I already have in this installed tab and advanced is some options that I'm not going to look at right now. So let me go back to available. Actually I'm going to go ahead and filter here and first I'm going to look for Blue Ocean.

You can see there are several plugins here. I'm going to go ahead and install the main BlueOcean plugin. I'm going to say download now and install after restart.

It is in the process of downloading and installing. You can see and I'm going to check this box at the bottom. If you can see it says restart Jenkins when installation is complete. So now it says Jenkins is going to shut down. It will just take a minute for Jenkins to restart now.

You can see that this restarting Jenkins is running. Now that it's finished, I can log back in. So I'm going to go back to the plugin section in Manage Jenkins, Manage Plugins. And I'm going to go to available and this time I'll filter by Docker. So this first option is the main Docker plugin and there is a second one for Docker Compose.

I'm going to do the same thing and download now and install after restart. You can see that... Some of the plugins have dependencies of other plugins.

The only ones I selected were actually this Docker one and the Docker Compose one. But because they have dependencies of Docker API and Docker Commons, Jenkins is also installing those plugins. So it will install the plugin you select as well as any dependent plugins if they aren't installed already. This is going to take a few minutes to... Install these.

Now that we installed those plugins, You will see some additional options in some of the settings pages in Jenkins. And you can also see this new sidebar item saying Open Blue Ocean. Let me click on that.

You can see this is a completely different interface because it's a modern UI for pipeline management in Jenkins. It's still linked to the old interface. So if you click on some links here like administration, you can see it takes you back to the old interface because Blue Ocean doesn't have a Managed Jenkins section. Let me go back to Blue Ocean.

You can also click this kind of exit icon and that takes you back to the regular Jenkins dashboard. Normally you would see a list of the pipelines you create here. but since there are no pipelines it gives us this box prompting us to create a new pipeline.

So that's what we're going to do next. Let me start by creating a new pipeline. It's asking where my repository is hosted, which is GitHub, and in order to connect to my repository and be able to run jobs I need to use my GitHub access token so I need to create one.

This warning in red here is just because I had previously created an access token you probably will not see this. I'm going to come here to create an access token. It will take me to my GitHub account and I can go straight to the page where I can create an access token.

If you want to get here on your own You can access this page through settings and developer settings and then personal access tokens. You can see that by clicking on the link the necessary permissions that I'm going to need for Jenkins are already pre-populated and selected here. So I don't need to change anything.

I just need to give a name to my personal access token and then I can also change the expiration if I want. This is just a name so when I look through my tokens in GitHub I can know what the token is for. I won't be using this name anywhere in Jenkins.

I'll just be using the token itself. So everything else is set. already and I'm going to go ahead and generate the token. The token does disappear if I leave this page so I'm going to go ahead and copy it and now that it's created I can go back over to Jenkins and enter it here and click connect. So that worked and now that I'm connected to the GitHub account with my access token, it shows me the list of all the organizations that I belong to and I'm going to select the Faraday Academy organization because that's where the rebuild I want to use is located and in that organization It shows me all 14 repos and I'm going to be using the curriculum app for this tutorial and then I just hit create pipeline.

Now it takes me to this interactive pipeline creator screen and you can see a message just popped up that said no Jenkins file located. So let me just go to the project in GitHub so you can see what it's talking about and in this repo in GitHub You can see there is no file called Jenkins file. So by default, Jenkins will look for a file called Jenkins file in the root directory of a repository.

Now what a Jenkins file is, is a file that contains instructions for a pipeline written as code. So you can read the file. and see each step defined in the pipeline going from top to bottom divided into stages and steps which we'll talk about in a minute. Now because this is a file it's checked into git and versioned and can be managed or rolled back just like any other file in git. Now since the file doesn't already exist in our repository, when we create the pipeline in Blue Ocean and save for the first time, it will automatically create a new Jenkins file for us and commit it to the repository.

So let's go through that process now back in Jenkins. You can see the message down at the bottom here saying A stage is required. Basically the pipeline is broken down into stages which are like sections. Every stage has to have at least one step and the steps are what contain the scripts, calls to Jenkins plugins, or other actionable tasks to be carried out in the pipeline. Both stages and steps can be run in parallel or one after the other.

You will see this as we go along in the pipeline creator. I'm going to name this stage something like checkout code and now I need to add steps. So I'm going to add a step here and find the git plugin that I installed earlier. And it's asking for the URL of my git repo. So let me grab that, paste it, and I need to denote which branch I'm using.

For the repo I use dev as the default branch. So I'm going to use that. And now I can click save. It says that saving this pipeline will commit a Jenkins file to the repository, which is exactly what I want. And since there is no Jenkins file, of course, it will create a new one for us.

I'm going to leave the description blank, so it will just give me a default commit message. And it's... pointing to dev so it's going to commit directly to the dev branch.

So let me save and run that. And it failed the first time it was trying to run. Sometimes it passes on the second try so let me retry.

And it failed again so let me look at the logs and it says that the recommended git tool is none. It gives me some information here on what's going wrong and why it's failing. It's saying that git does not exist. Basically, I spun up this server using Jenkins from the Linode marketplace, but that server does not have git pre-installed on it. So I will need to install that directly on my instance.

And since I don't have any kind of automated setup for the instance, I'm just going to SSH directly into it. So in Linode, I'm going to go to my server and instead of using my own terminal this time, I'm going to use the built-in one that they provide in Linode. It's called Lish, so I can click launch Lish console and it prompts me to log in here.

So ignore the previous incorrect logins. I'm going to log in as root and then grab the password. that I saved in my password manager and I successfully ssh'd in. Let me make sure that git is not installed. I can do that with the git dash dash version command and it's telling me that git is not found.

So before I install it, I'm going to do an apt update. to make sure I have the latest versions of the packages. It's telling me that I can upgrade some of the packages I already have installed or they have new versions, but I'm not going to do that right now.

For now, I'm just going to do apt install git. It's asking me if I want to continue installing 38.3 megabytes and now I should have git installed. So let me check again and yes I have git. version 2.20. And now that I have git installed, I'm going to try to rerun my pipeline over here.

While it's running, notice that the colors change. So you have the color blue while it's running and then you can see things turn green as they pass. And then when everything passes, this whole thing turns green. As you've seen before, red means failure and there's also a yellow state.

For certain cases like test failure, then it can turn yellow. So everything is very visual here. And you can see that now it passes since I have the correct dependencies installed on the server. So this is the most basic version of a pipeline. to create in Jenkins.

So let me add a few more steps now. Now when Jenkins created this pipeline as I mentioned before it also created a Jenkins file. So when I refresh my GitHub here it added the Jenkins file and you can see where it says added Jenkins here. That is the default commit because we didn't type in a custom one and you can see if we go into the Jenkins file this is the stage that we created and the step to check out code with the GitHub plugin.

Jenkins files are usually pretty easy to read because you can see well-named options like which agent you are running the pipeline on, a list of stages, and then the steps inside of each stage. And for the GitHub plugin here, I'm using the API that they provide and passing in the GitHub URL for the repo and which branch I want to check out. So let's explore the interface a little bit and then try to update our pipeline. From the top left, you can see which pipeline is running.

I only have one right now. If I click on it, I can see the history of the jobs that have run and the ending state if they passed, failed, or are still running. And if I go into one of them, I can see that it has all the information.

I don't have any changes yet. and I don't have testing that really runs. And you can see the fourth tab is for any artifacts that are produced. So if I click on this file, this is just automatically created by Jenkins. It's not something I explicitly have to create.

But if we did create some kind of output, it would be here with the artifacts on this page. The next icon here. that looks like refresh will rerun any jobs.

With the pencil I can edit the pipeline. If I click on the gear icon it takes me to settings in the old Jenkins UI. It takes me directly to the settings for this pipeline and also here if I go to branches and hover over any of the branches in the list here.

Then I get a couple icons here and one of them is edit. So I'm going to go ahead and click on that edit icon. Now there's different ways to arrange a pipeline.

I can add more stages. I can go back to the previous stage and add more steps. or I can add stages that run in parallel depending on what I want to do. Now there are a lot of options for different types of things that I can do in a step. Some of these like Docker and Git are from plugins that I've installed.

I'm just going to use the regular shell script here and for right now I'm just going to list contents of the directory and this will show up in the Jenkins logs. It's going to create another commit to update the Jenkins file. So I'm going to add a commit message this time and I'll save and run that.

And if I go over to my app, I can see that next to the Jenkins file, it shows the new commit message. And go into the Jenkins file and the new... Stage and step is updated. I guess I forgot to name my stage.

I thought it was required. You can see in Jenkins here that when I make updates it's running twice. That's because I am committing with the Jenkins file directly to the dev branch.

So that runs but there is also an open PR in my repository, this PR90. and I'm making the PR into dev. Every time I'm updating the Jenkins file it's rerunning all of the PRs.

So let me go into here and it's passing. It defaulted to the name error here but anyway I can see that it printed everything out. Let me change that name.

error. So let me rename the stage to log and save that. Of course now it's going to run again even though I'm just updating the wording in the Jenkins file.

One thing to note here is that services like GitHub do have restrictions on API usage. So if you start to see failing jobs here, it might not be because of anything you've done wrong in Jenkins, but you might have hit the limit with... the GitHub API and then you'll either have to upgrade your account or wait until it resets. I think it resets every hour. So sometimes jobs fail because of a cap on API usage.

But here it looks like everything went fine. So now I want to add another script to the pipeline. I'm going to come to branches and click edit again and I have this simple log script but let me just run one in parallel so you can see how the Jenkins file works.

I'm going to call this front end unit test, add a step here, another shell script, and I'm going to paste in the command. I'm going into the curriculum front. directory where my front end is and then run an npm install to install the dependencies and then run npm run test unit so run the unit test and i'll click save there So this is going to run and I'll follow it here. I think it is going to fail because I don't actually have node or npm installed on my Linode server. So it failed.

Let me look at the logs. and indeed it says npm not found. So let me go back to Linode and launch the Lish console again and I'm still logged in from last time so instead of apt install this time I'm going to install Node via the recommended command with curl.

This is for version 16. You don't only have to use version 16, but this is just the version that I tested it with. It's a stable version. It's giving me more instructions here to apt-get.

install node. So I'm going to enter that command and now I'm going to check the node version. It says version 16 and let me check the npm version because npm comes with node and I have version 8 of npm.

So that should be all I need here. Let me try to rerun this now. All the other steps passed. This one's still going. It hasn't failed yet so that's a good sign.

While it's running I'm going to switch over to GitHub and go to the Jenkins file and you can see that now it adds this parallel command. and denotes that these two stages are running in parallel and gives it the title of the first stage for both. And then each stage in parallel has its own title and steps. Of course you could just put this in one stage and have two steps as well, but I just wanted to show how it works to run things in parallel in Jenkins. This is taking a little while so I'm going to fast forward.

So I actually re-ran this because I closed that other screen. You can see that everything passed this time including the new one for tests that I added. Now that we have a couple basic scripts working, let's add some stages so that we can finally deploy the app.

to Docker Hub. Now I'm going to add a build step to our pipeline. I'm just going to create a new stage called build and then I can add a regular shell script. I'm going to use this shell script to run Docker commands.

So I'll use the regular Docker build command to build a Docker image from this. docker file that I'm pointing to because the docker file isn't in the root directory here. It's in a subdirectory. So I need to point to that with this dash F flag.

And I also have to let docker know what is my current context, which in this case is the directory I'm already in. So I'm just using the period here. And now I'm going to commit this.

Now it's building and I'm waiting for it to build. I re-recorded this a few times so you can see the numbers skip around a little bit but I'm still going through each step in the process. Now that the build passed I need two more steps to be able to deploy this to Docker Hub. First I need to log in with my Docker Hub credentials to make sure I'm authorized to push images to my Docker Hub account.

To do that I need to use the docker login command and the default place that it points to is Docker Hub. So I don't have to do anything special here. but I do need to use my credentials like my username and password.

But of course I don't want to store my username and password for docker hub in the Jenkins file so I am going to use variables in Jenkins. So I'm going to add environment variables to my pipeline and then I will just point to those variables. from my Jenkins file. So you can see I can add the Docker Hub user and the Docker Hub password. I'm going to just add those values in Blue Ocean.

If you aren't using Blue Ocean or you want to use the standard Jenkins interface, there is also a key manager in the settings over there. I think it's a bit easier to do directly in Blue Ocean here. Of course, I'm not going to show these variables on screen.

You'll have to enter in your own credentials here. And now I'm building and testing just to make sure that my login worked and indeed it passed. The only thing left to do is to add a new stage here to actually push my updated image to Docker Hub. So I'm going to add another shell script and this time I'll use the docker push command. I just need to add the name of the container and optionally the tag but the only tag I have right now is the default latest tag so I'm just going to leave that.

And it failed because it says that an image does not exist. So with my username and the name of that image. Now, since I already built and deployed this app a few years ago, I already have this in my Docker Hub.

However, now that I want to push a new image from Jenkins, I need to tag it during the build step here. So I'm going to add a tag to my build script and I can just use the dash t option and I'm going to tag it with the same name that I'm using so my username and then curriculum front and I'll just add latest. Now let me save and run that and see if it passes. and it passed and pushed correctly to Docker Hub. Let me check over in my Docker Hub interface.

It's at hub.docker.com and I'm already logged in. So I can see that my curriculum app was updated less than a minute ago, or rather my curriculum front app, which is just the front end. Congratulations on making it to the end of this video tutorial.

I hope you were able to learn a lot about Jenkins and DevOps and Linode. As always if you have questions or comments please leave them in the comment section below. I hope you are also able to check out the follow along readme that is linked in the description.

It has all of the instructions for of course following along with every step in this video. as well as some notes that I took along the way about things that I learned in the process of creating this tutorial. I want to thank Linode once again for their grant to Free Code Camp and making educational content like this video possible. I hope all of you have a great rest of your week and we'll see you in the next video.

Transcript for:Jenkins and DevOps Pipeline Overview

Transcript for:
Jenkins and DevOps Pipeline Overview