In this video we're going to create a Raspberry Pi IoT or Internet of Things server which will allow you to collect any kind of sensor data you like, send it over your network to this Raspberry Pi server and visualize it on a nice looking Grafana dashboard. In our example we're going to use an Arduino Uno paired with a BME 680 sensor to collect the temperature, pressure, humidity and air quality data every two or so seconds. This data is wrapped into a JSON message and is sent over our network to the server that we're going to create in this video and that's done via the MQTT message protocol.
In this video we're going to focus on the server aspects of this application. If you want more information about the sensor node let us know in the comments if you want a follow-up video specializing in that. We do have the source code linked in the description as well if you want to poke around.
All in all this project should take you roughly half an hour to an hour to implement. From a top-level view, this system could consist of a series of sensor nodes that send data in a JSON message over your network to this Raspberry Pi via the MQTT network protocol. And the MQTT protocol is basically where a client publishes data to a broker or a server such as Mosquito, which is the application we're going to be using in this video, and that broker pushes the data that has been sent to it to any client.
that has subscribed to that topic. A topic is essentially a way to categorize the data that has been sent. For example, I have this sensor in my office, so I could use a topic such as home slash office, and any client that is subscribed to home slash office will be forwarded the MQTT message that is sent to the broker.
In our implementation, an application called Node-RED will subscribe to this topic, interpret the JSON message data, and then send the data to a database to be stored and eventually used. In this case we're going to use influxdb to store the sensor data and then we're going to use a grafana front end which is going to query the database in order to display the data we want to see and in a nice format. So on our Raspberry Pi server we're going to need four main applications. This is Mosquito, the MQTT broker, Node-RED, influxdb and grafana.
We are going to run these applications within containers using Docker. Running an application inside a container means it's really quick to set up and deploy, either to deploy again on the same device or on different devices, and to remove or manage certain applications. Containers are not quite virtual machines, as virtual machines emulate a machine all the way down to the system hardware, whereas a container runs above the operating system.
It's not as confusing as it sounds if you haven't used Docker before, so don't worry, as we're going to use a program called IoTStack to get the containers ready for us to use, and that makes the process super simple. Before we get into the tutorial covering how to implement this, please do consider subscribing if this video is going to help you out. We have a number of interesting IoT videos coming up, such as a review of this drop-in DIP replacement for the Arduino Uno which adds Wi-Fi functionality straight onto the board.
So stay tuned for that. Okay, so we are starting with a fresh install of headless Raspbian OS. So that's the 64-bit lite version and we've enabled SSH. We have a video linked in the cards above if you need to find out how to set that up.
I should note that you will need a static IP address for your Raspberry Pi and you can do this by adjusting the DHCP configuration as mentioned in that video. The first thing I'm going to do is update everything. with sudo apt update and sudo apt upgrade.
Then we're going to download IoT stack using this curl command. This will also install docker, docker compose and some other necessary packages that you might need. So if you get a prompt telling you to install something just say yes to those.
This might take a little time to download and install but once it's finished it will prompt you to restart and you can do this with sudo shutdown minus r now. It's important that you do restart so that all the environment variables are set correctly. You'll get a few errors if you try and run this without restarting.
To note, there is a written guide to this video on our website where we list all these commands you'll need to get started, so by all means head to there and copy and paste those commands. Once the Pi is restarted, log back in and enter the IoT stack folder. Open the menu with ./.menu.sh and use Enter to select Build Stack.
This is where we can select the packages that we want to use and we can navigate using the up and down arrow keys and spacebar to select. Just to refresh your memory we're going to be using Grafana, InfluxDB, Mosquito, Node-RED and Portainer. Portainer is something we haven't mentioned yet and that's going to give us a graphical interface to manage these docker containers. When you select Node-RED you might see a warning.
Press the right arrow key to enter the options list and the defaults are fine. Press enter to build these lists. and then navigate to return to the menu and finish your choices. This isn't quite displaying properly on my screen that's because I've zoomed in on the terminal to make it easier for you guys to see, but it should be fine on yours. Once you've selected all of your components, so all the five, you can press enter again and that's going to build a file which essentially contains all the different components we've set and how to configure them.
If you navigate to the docker commands section Select Start Stack and this will start all of these containers. This will take some time as all the images are downloaded. With all these downloaded, all your applications should now be running.
And you can test this with docker-compose ps and that should show you your five containers. Another way to look at this is to open Portainer. By opening a web browser, navigate into the IP address of your server and use the port 9000. If you click on the option to see the local docker containers, you can then click the containers and see a list of them.
This gives you a way, sort of a graphical interface of how to interact with these containers. So you can open things like a console in them, or you can turn them off if you want to. Have a play around with this interface.
So, let's start configuring our applications. First things first, we're going to create an InfluxDB database in order to store all of our sensor data. Let's go back to our terminal. and use the command docker exec dash it influxdbinflux.
And what this is going to do is enter the command prompt that we're using into the Docker container so the commands that we run are executed inside that container. We use the command create database and then the name of the database you want to make. I'm going to call this sensor data and then hit enter and it'll create the database.
We can then exit using quit. Next, we're going to configure Node-RED. Navigate to your Pi via a web browser by entering the IP address and the port 1880. And this is essentially a visual sort of programming thing where you can build a workflow by dragging and dropping boxes from the left-hand panel and linking them up. We're going to need three units, MQTT input, change node, and InfluxDB output.
To configure the input we double click on the MQTT input button and we need to add a server by clicking on the pencil. We can name the server and type the IP address of the Pi into the server address box. Leave the port as the default 1883. Localhost won't quite work how you expect in this situation so it's easier just to type out the IP address of your server. Press add server and then select this server from the drop down list. We can then type the topic of the data that we want to subscribe to.
In my case I'm going to use slash home slash sensors and we want a past JSON object from this block. You can also name this block. It's helpful if you have multiple of these set up so I simply use the topic I'm subscribing to.
In the change block we want to change the transmitted message into a JSON expression and we select the JSON expression from the list and press the button with the three dots. and in this expression we're going to change the keys in the original message to something a little bit more descriptive. You can obviously adapt this to the specific data you are transmitting. You can save and close this window. Last but not least we have to configure the output node.
Open the console, press the pencil next to the server and add the IP address of the Pi and use the port 8086 if it's not there as default. Then we use the database name that we previously created, press add, and then we need to name the measurement we're writing the data to. You can put what you like on here, it's essentially a way of naming the table that everything's stored in. I just use sensor data again. You can close this panel and hit deploy to save and implement this configuration.
I've made sure that my sensor node is transmitting data and we can test if this node red configuration has worked. by checking that there are some entries in the table. If we head back to the terminal and enter the InfluxDB container again, We can enter the commands use sensor data, then show measurements, and then select asterisks from sensor data, and you should see now some data being populated.
And this means that we're ready to set up our graphical interface. If we exit out the docker container and open up Grafana, the Grafana dashboard, which is the Pi's IP with the port 3000, it might prompt you for some credentials. In this case, just use admin and admin.
and you'll be asked to change your password. The first thing we need to do is add a data source. If you don't see this on the front page you can hover over the cog in the bottom left. Select influxdb and then enter the IP address of the Pi and the port 1886. You'll need to put http at the beginning.
Scroll down and enter the database name and then the get http method. Press save and test. and you should see a nice green bar saying that the data source is configured and confirmed. Next we can create a dashboard which is where we will see all the information we have recorded.
Select create new dashboard and then add a panel. In the measurements box in the from parts of the data source you can select sensor data and then under the field option you can pick the variable that you want to plot and as you do this you should see some data being populated in the graph. You can name the panel on the left hand side and click apply. Repeat this process for as many times as you have data to plot.
In my case that's four graphs. You can adjust plot types and various settings in the right hand side panel so I would recommend having a play around with that. And once you're happy with your dashboard do make sure to save it because the number of times I've set something up and then exited away, closed my browser window and it's not saved this panel is annoying.
So, do make sure you save it. So basically that's it! You've now created a nice IoT server which will take sensor data sent from a node using the MQTT protocol and it will be plotted in a nice graphical dashboard.
As a reminder, all these commands are written out on our website, linked below. If this video has helped you, please consider leaving a like and subscribing and do let us know how you intend to use this server down in the comments below. Thank you very much for watching, and as always, have a nice day.