i reckon that simulations are one of the most interesting and important parts of robotic development having a good simulation environment can be a really valuable tool because it's often expensive or time consuming or even dangerous to be testing algorithms on real hardware especially when you're testing it for the first time and so because of this time spent investing in a good simulation environment is often well worth it even better than this is when someone has already made one and they give it to you for free and that's what we have with gazebo it's a free robotic simulation environment that's created by open robotics that's the same group that's looking after ross even though they're made by the same people the projects are managed separately and so gazebo isn't quite a part of ross this distinction can be a bit confusing sometimes but gazebo and ross integrate very well together even if they sometimes handle things a little bit differently with gazebo we can create a virtual world and load simulated versions of our robots into it simulated sensors can detect the environment and publish the data to the same ros topics that real sensors would allowing easy testing of algorithms then forces can be applied to the simulated actuators on the robot and they can take into account physics and things like friction gazebo has recently been rewritten and replaced with a new simulator called ignition gazebo sometimes just referred to as ignition this is kind of like the transition that's been made from ros1 to ros2 except that ros 2 is still compatible with gazebo classic unfortunately there are a couple of key plugins that aren't quite compatible with the new version at the moment so we'll be sticking with gazebo classic for these tutorials but hopefully in future we'll be able to upgrade to the new one so today we're going to learn how to use gazebo how to simulate a virtual robot and how to integrate it with ross so let's get started we'll start by installing gazebo and seeing what it can do without the ros integrations since we've already got ros installed the easiest way to get gazebo installed with all its dependencies is to go sudo apt install ross foxy gazebo ross packages okay and once that's installed running gazebo without ross is actually pretty easy we just type gazebo and then the path to a world file if we want to start with a world file so we're going to use one of the ones that's built in called seesaw so it's in user share gazebo 11 worlds seesaw world and so that'll start up gazebo and there we see a little seesaw now take a minute to just get used to the environment so left click to pan middle click to orbit right click or scroll to zoom around so you can go around take a look and what we've got here is the gazebo world so if we look up here we can see all the things that are in the world and in particular we've got some models in the world so if we open up that models drop down we can see we've got five models here there's the ground plane which is the big grey plane we've got cube one over here cube two on this end the fulcrum which is there and then the plank that is balancing on top of the fulcrum and so we've got these five models inside the world and what we can do this is just a physics simulator so we can grab these we can start moving it around and then the seesaw is unbalanced and it tips over so we can go to reset world you can use control r and say what we can do we can do all sorts of stuff to you know make the world do various things we can actually pause time for a moment and then we can you know lift something up rotate something set things up how we want and then hit go and then see what happens we can apply forces to these objects so let's apply a force of minus 2 million newtons to this one and i'm sure you'll be able to guess what's going to happen when we do that so it launches that one up into the air and so at this point this is just a physics simulator what we want to do now is see how we can use ross to get our robots into this simulation environment so the first thing we want to understand is the gazebo model structure now in the last tutorial we created a urdf for a robot and gazebo uses a similar format called sdf it's it's very similar to urdf but it's just a little bit different the good news is that we don't have to write two different files because gazebo comes with a tool that can convert urdf to sdf automatically unlike urdf which just describes a robot sdf is used to describe the world that is being simulated as well as the models inside that world so for example with that seesaw that we just looked at the seesaw.world file was an sdf file but then each of those little cubes inside there each of those models if they wanted to could have been their own sdf file and this is a really flexible approach because it means we can reuse models inside different worlds or we can reuse worlds over and over again and test different robots inside those worlds something else to understand is that every time gazebo wants to interact with something outside of itself something like ross it needs to use what's called a plug-in and this is an extra little piece of code that does one particular thing that we can tell gazebo to execute at a certain time so for example if we needed to control the robot externally or get information out of a simulated sensor we would use plugins to do that now you might remember from the last few tutorials that we have a robot state publisher which takes in the urdf description in an exacto file it takes some joint state messages and it broadcasts the resulting transforms as well as publishing the description to a topic now until this point we have been sending fake messages to the joint states to move the transforms but now we can move closer to a properly integrated system so now we've got gazebo and gazebo represents kind of the real world gazebo has a spawn script that can read the description from the topic that's being published and simulate the robot it will then use a plugin to see how the joints are moving in the simulation and publish those to the joint state topic and then another plugin can take input from the rest of ros and force the joints to move in certain ways on top of that any sensor plugins that we have can publish to other topics let's take a look at all of this in action and hopefully it'll start to make a bit more sense so here's where we finished up in the last tutorial we had our robot state publisher then we had arviz running and it was looking at that and then we had joint state publisher gui that was running and we were faking our joint state so we could make our robot move around in the visualization so what we're going to do now is we're going to get rid of that and upgrade our uidf so that we can put it in the simulated virtual environment we'll be able to control the joints and the coolest bit is that we'll be able to simulate a virtual camera and be able to see what the world looks like from the robot's point of view so we'll start by firing up gazebo now this time instead of just running it normally because we've got to run it with the ros integrations we're going to use a launch script that has been provided for us so we're going to run ros2 launch gazebo ross gazebo launch dot phi so this is going to run gazebo and it's just going to launch it there with an empty world then what we want to do is we want to spawn our robot and so the gazebo ross package also comes with a script that can take our urdf file convert it to sdf and spawn it into the gazebo world so we'll type ros2 run gazebo ross spawn entity now we want the topic that it's going to read from to be robot description and the uh the name of the entity so the name of the robot is going to be we'll just call it my bot and so what this should do is spawn our robot within the world now immediately we can kind of see that something's not quite right uh we've got it's all white if we head back into rviz and reset we'll see that all of a sudden our transforms aren't there properly because nothing's publishing the joint states anymore also something else that's worth noting is if we have a look down here in the models we can see our bot my bot but it's only got three links and you might remember that our robot originally had five links and that's because any links that were joined by a fixed joint gazebo is just going to take and collapse into a single link and so that means that they're all treated by one link traders one link by gazebo and at least one of those has to have inertial properties set otherwise it won't work properly so in this case our world and our base got collapsed into one and our arm and the camera got collapsed into one so yeah so we've got this here it's there but it's not quite right we're going to look at what we need to do to to make this work so we're going to close this down before we do that we're actually going to swap our launch script because you'll notice to do that we had to run robot state publisher and then run gazebo and then run the spawner also we'll need to control c to kill gazebo sometimes it can be a bit stubborn so here where we had our robot state publisher launch file from from last time i've got a new one that is robotstatepublishersim.launch and it starts off the same it has this used sim time parameter added to robot state publisher and then it also includes the um gazebo ros launch file and the spawn entity so it's going to do all three of these things for us so we'll start with that now let's upgrade our urdf so here's here's the one that we had last time and here i've got a little template of the things that we're going to put in it so to start with we're going to add and include so we're going to include a new one called example gazebo.exacro and we're going to start by just making that an empty uidf now the way this works is that any code that is specific to gazebo we put inside gazebo tags and that way when gazebo is reading the urdf it knows to look in there and it can find things that it needs but other things can just ignore it and there are a couple of different types of gazebo tags you can have gazebo tags that are attached they're connected to a link or a joint or you can have gazebo tags that are just kind of generally telling you something about the whole file and so to start off with we're going to fix up the the colors because we saw that the colors were all white before so we're going to grab these and put them in our uidf file this is in this gazebo one so these ones are gazebo tags that are referenced to a link so the base link the slider link and the arm link and then the materials use gazebo materials so the colors that we had set before that were working before in rvs they won't work in gazebo for technical reasons so we'll start by doing this so we've got that file included in this one we've got our new launch file so let's now instead of running that we'll run our new launch file and hopefully what will happen is we've now got colors isn't that great now if we take a look at our views we'll see that these are still not working we're still not publishing our joint state so that's the next problem we've got to solve so we'll take a look at this template again and grab this bit so we've got another gazebo tag here and this one is not for a particular link or a particular joint this is just in general saying that in general we want to be publishing joint states and we're going to use a plugin here called the gazebo ros joint state publisher and it's just got a couple of parameters we set the update rate to 20 hertz and then the names of which joints we want to publish so in this case it's the slider joint and the arm joint now it's worth pausing here for a second and saying that there are a few different plugins that can solve this the best way to do this is using the ros2 gazebo ros 2 control plugin which is part of the big rost2 control system and that helps handle the the publishing side of the joint states as well as controlling the joints themselves it's a great system but it's just a little bit too complicated to set up for this quick demo so we're going to go for this simpler approach instead but it is the better way so we're just going to use this joint state publisher plugin so we'll close gazebo kill our launch file and we'll run it again with this new bit added in looks the same in here but when we swap back to rvs we can see that all our states are being published now um and so what we'll do is we'll put that on one side and that on the other side let's just hide these to get a bit more room and now what we'll see is if we start playing with the physics simulation this will start responding so it's looking at what's going on in the physics and then it's figuring out what the joint states need to be publishing them to the topic which uh river state publisher is then sending to rvs and broadcasting transforms and that sort of thing now one thing you might notice is that the bass keeps oops the bass keeps jumping back to the origin and that's because we set our first link as a world link so what gazebo does is if it sees that there then it's going to say i don't care what the physics is saying i know that this has to be fixed to the origin and so anytime you've got a robot that's got a fixed kind of base like this then you probably want to have a world link in there so that gazebo keeps it at the origin if you've got a robot that's moving around then don't have that one in there so that's all great but now we'd like to be able to control this now the approach we're going to take here is a little bit messy like i said the better way to do it is using the rost2 control system but we're going to take this next plugin which is the joint pose trajectory plugin so we'll add it here and so the only parameter we've got here is the update rate which that sets into let's make that 20 as well what i'm also going to do here is i'm going to add some damping to our joints because you might have noticed as we were doing this the joints kind of flop around a fair bit and when we go to use this plugin it's just going to make it look a bit funny so we're going to go back into our original uidf file and look at our our moving joints so we've got the arm joint and we're just going to add these uh some friction to those joints to the arm joint and the slider joint and that'll stop them bouncing around quite so much so now that we've got that new bit in there we'll close gazebo run it again and now what we'll be able to do is we've got this little message here so it's going to publish a joint trajectory message it's a bit complicated but basically all it's doing is it's going to say put the slider at 0.8 meters and put the arm at 0.6 radians so if we do that give that a sector run and it's placed it at that trajectory and you'll see if we zoom right in you might be able to tell that this is actually still falling due to gravity so even though we put those frictions in the that friction in the joints um the physics is still going to going to overcome that but if we hadn't put the friction in then this would be bouncing around all over the place so we've now got our system we've just got to hit reset because we restarted gazebo so we've got our system kind of integrated now we're able to use ros to move the simulator and then it gets the physics applied and then those results are being fed back into rvs using the publisher the next thing that we're going to do is simulate a sensor and this is the bit that i think is the most interesting we're going to simulate this camera and so we'll be able to see what the world looks like from the camera's point of view but at the moment that's actually going to look pretty boring because it's just going to be this big grey plane so we're going to use gazebo to create a more interesting world to look at with our camera so what we can do is we can go over here to the insert tab and you can see it's connecting to these model databases to get some models this can take a really long time so i'm not quite sure why it takes so long and it has to do it again every time but we'll just wait a minute until they connect okay so those have connected now um we can open up this one and we see all these models in here so why don't we grab a construction barrel i like to use them you can see it downloads the model off the internet i've got pretty patchy wi-fi where i am right now so it might take a minute and let's grab a construction cone as well and then if we keep clicking these it'll keep downloading them so what we can do now is if we go up the top because we've downloaded them once they should show up in that list and they're not i'm not quite sure why they're not showing up there but we'll just leave it as that for now what we can also do is we can use the model editor and the building editor now these will both create model files the model editor is used for taking models that already exist or like cubes and spheres and stuff and building them together into new models the building editor is for creating a building with walls and it's a little bit fiddly but let's grab it and we'll try and make a bit of a wall i think it's a bit of a funny system and then we can put some colors or textures on our walls we can put a door in there or a window and then what we can do is we can save that so you have to save it for it to work so let's call this my building and then we exit the building editor and we'll see now that's become part of our world now what we don't want to have to do is to redo this every single time that we we want to rerun gazebo so what we'll do now is we'll actually delete our robot from the world then we can save the world let's just save it here and call it my world dot world and then let's quit gazebo oop that was the wrong one that was obvious and then what we can do when we relaunch it we can go we can add this argument to specify the world we want to launch so we'll specify what we call it home slash my world dot world and so it reruns you can see those models are now in that list they just didn't load reload before and so we can now keep adding them nice and quickly without having to wait for them to download so now that we've got a nice little world to look at let's add our camera so what we're going to do when we're using cameras in ross we have to put this extra link in called the the optical joint this isn't a tutorial on camera so we're not going to go into detail on that right now for now we just copy and paste that and put it we'll add that extra joint and link in there and then we're going to grab this tag which is what we need to simulate our camera now this looks pretty complicated to start with but i'm just going to close these up so what we can see is we've got a another gazebo reference tag so we're saying okay this sensor is attached to a particular link in this case the camera link the sensor type we're actually going to change this to a regular camera you'll see why in a second rather than a depth camera it's got some parameters that are common to all sensors then any parameters that are specific to cameras um and then the plug-in that we're using to connect it to ross in this case gazebo ros camera and any parameters that are specific to that plug-in so we're not worrying too much about what these parameters are at the moment this is just for the demo but yeah so we've got this we've changed it to camera so let's close that and rerun then we'll also rerun rvs and what we'll see now in gazebo is we get this nice little preview of what the camera is seeing so it's seeing this brick wall and then if we add an image display here we can set the topic to if we scroll down here we can say my camera image raw and there it is so that's being simulated in gazebo and then the image is being passed to ross and we can display it in rvs so that's being published to that topic but what's even cooler than a regular camera is a depth camera so we're going to close gazebo again we're going to change that back to depth and then we're going to rerun gazebo now the reason that i had you change it back before is because for some reason when you're using depth camera you don't get the nice little preview i'm not sure if that's a bug or what it's a bit of a shame but if we reset our views now we've still got our image here but we can also add a point cloud and so we set that to the my camera points topic that now exists and what we should see is the 3d point cloud of what that depth camera is seeing and so what we can do now is go into gazebo let's say we shift that cone over then it'll jump there in the simulation if we have it bounce around you can see the depth camera is seeing it roll about we can even move our actual obvious didn't like that not sure what happened there i just re-ran everything so uh yeah we should be able to move our robot around and see it respond in our views and you know whatever it's looking at is now going to look a little bit different so we can see that appears there now before we finish up there are a couple of other important things that are worth noting when we're using gazebo the first is you might remember that back when we were looking at our launch file we set this use sim time parameter for robot state publisher and what that means is that normally when ros is running it uses the unix system for keeping time so it's counting the number of seconds since the 1st of january 1970. but when we're running a simulation we often want gazebo to be able to control time you can see right now there's this real-time factor 0.93 so sometimes we want gazebo to run faster or slower depending on how we're doing the physics simulations and so we need all of the systems uh all the nodes that are running in the system to understand that gazebo is in charge of keeping time and so when gazebo is running it'll actually be publishing to a topic called clock and so and so you can see gazebo is constantly publishing to this topic and so all the other nodes are able to look at that and keep time so we need to make sure that use sim time parameter is set on all our nodes in ros1 there was a way you could just set it across the board for every node in ros2 you have to set it for each node individually but that's where launch files come in handy because we can make it a parameter to the launch file and then we can toggle it on and off there when we run our launch file and have it pass it down to all the nodes so that's the first thing the second thing is that gazebo isn't actually just one program it's two programs there's a server and a client and for more advanced users there are really good reasons that you want to do that but it's sometimes confusing for someone using it for the first time so if we take a look at the processes that are currently running on this computer we'll see here we've got the terminal bash ros2 we can see this was our launch script that's got the robot state publisher the gazebo server and the gazebo client and sometimes one or both of these might crash and it's not very clear what's going on and gazebo won't start again and that sort of thing so if you get stuck in that situation you can kill all gz server gz client if it's really bad you might need to use nine or even sudo and that will just say kill all of gazebo and then you can you know start it up again and hopefully it works next time now that we know how to simulate our robot in gazebo we're ready to start building one in real life and so that's the end of this tutorial series at least for now but pretty soon we'll be kicking off a new series on how to build your own autonomous mobile robot i'm really excited for this project and it'd be even better if you wanted to join in and build it along with me so if you don't want to miss out on that make sure you subscribe if you want more information about this stuff you can check out the blog post that's linked in the description below and if you've got any other questions feel free to ask them in the comments otherwise i'll catch you next time [Music] you