the quality of code from the United States government's been a bit of a meme for a while for those of y'all who are at least my age you probably remember the healthcare.gov Fiasco where a new healthcare service was put up and it just didn't work there's been a lack of quality code from the government for a while now but it seems like they really want to change it earlier this year they did some really interesting research trying to figure out which languages would be the safest as they wanted to overhaul their cyber security and defense systems and also improve the quality and resilience of their services some research was published that I thought was really interesting the specific part that this calls out is that in order to reduce memory safety vulnerabilities at scale creators of software and Hardware can better secure the building blocks of cyberspace this report focuses on the programming languages as a primary building block and explores hardware architectures and formal methods as complimentary approaches to achieve similar outcomes so what was their conclusion from this research their conclusion is we should be using memory safe languages you've probably seen a couple videos about this already but we're not talking about that we're talking about something much more exciting as we get in but here they're talking about the languages that they after their analysis determine to be Memory safe to potentially reduce the safety issues that could exist in their platforms they chose C go Java python rust and Swift as the languages that are memory safe enough to be used for the things that they are building I thought this was a really interesting call out but what does it actually mean they still have a bunch of code that's in C they still have a bunch of code that's unsafe what are they going to do about it that's what the video is about today tractor translating all C to rust after more than two decades of grappling with memory safety issues in C and C++ the software engineering community has reached a consensus it's not enough to rely on bug finding tools the preferred approach is to use safe programming languages that can reject unsafe programs at compile time thereby preventing the emergence of memory safety issues the tractor program aims to automate the translation of Legacy C code to rust the goal is to achieve the same quality and style that a skilled rust Dev would produce thereby eliminating the entire class of memory Safety and Security vulnerabilities present in C programs this program may involve novel combinations of software analysis such as static analysis and dynamic analysis and machine learning techniques like large language models this is a very interesting development I did not think that this would happen ever much less from the US government translating code to safe rust and then going all in on rust is a huge change in any software organization much less one the size of DARPA in the United States like government in research but how big of a deal is this how unsafe is C well I'm going to show you guys one of my favorite videos on YouTube we're going to go to my homie L level learnings Channel and sort by most popular coding and C until my program is unsafe it took him 42 seconds to make his program unsafe hilarious but also very reminiscent of the problem here when you write C code if you're not good about making sure you're referencing memory in a way that's locked down and isolated and you're not cleaning things up when you're done it's very easy to reference something for memory that has actually been changed by something else and now you can force execution of random behaviors in the code and if somebody's trying to exploit your services it's one of the best ways to do it is to figure out where memory is leaking where things are being read where they shouldn't and then use that to abuse the safety of your application let's watch this quick actually I think that'll be fun not actually sure if the sound is working for that y Great Sound is broken fun can't say I didn't try no luck I'll figure out my sound routing after stream it breaks all of the time because routing sound is hard I wanted to watch this really funny comment from chat here low level learning's fatal mistake was attempting to use strings everyone knows that to write safe code in C you have to handle all user inputs and outputs through integers and floats this hurts me so deeply so deeply God I do actually like this particular report the government put out to justify why they're putting the effort in here they actually got quotes from a couple different software architecture people and people who work at security and other technical places and they had a lot of important things to say here this new technical report takes a positive step forward on a critical issue the need for foundational safeguards against the root cause of many vulnerab abilities across the software supply chain addressing vulnerabilities across systems and infrastructure and ensuring resilient and diverse connectivity options are vital to National Security interests space adds an important layer to the diversity but by its nature faces a distinct set of security challenges good John Delmore who works in Accenture said memory safety vulnerabilities pose a significant security risk to software systems and are a root cause of many of the most damaging cyber attacks to address this we need to adopt memory safe programming languages for new applications and rewrite code using modern memory safe languages with secure development practices from the start we're pleased to see the oncd raise the issue because the Integrity of the global software supply chain is critical for National and International Security the report's focus on memory safe programming is a game Cher for cyber security underscoring the urgent need to prioritize safety in our coding practices and drastically reduce vulnerabilities at trail of bits We Stand firmly Behind The crucial shift towards a safer digital future leveraging formal methods to prove that these techniques are not nearly theoretical ideals but practical Necessities our experience has demonstrated that formal methods combined with memory safe programming languages provide a robust framework for eliminating vulnerabilities with unparalleled precision as you can see it's a pretty Universal agreement that memory safe programming languages are a really helpful part of making safer systems but we just looked at that list there's a bunch of other things in here we have go Java C Python and Swift why' they go with rust it's kind of an interesting choice right CU as we all know though as we talk about a lot here rust is far from the fastest language to iterate on that said this is the US government they're not moving very fast anyways so the speed of iteration is not anywhere near as important for government software where rust is strongest is when the specific behaviors you want to encode are really well defined and aren't going to change so you want to write the code in a way that will be relatively static and maintained for so long it will Rust I have two big rust puns I want to make with this that I think will help you all understand what the value here is rust USA so rust equals code that you let rust is a really clever name in that sense because the goal of rust code isn't that you change it every day that the way you build things in it changes constantly like it is in I don't know JavaScript where JavaScript today looks very different from JavaScript 10 years ago Russ is trying to avoid that is trying to make code that is just as maintainable in 10 years as it is today they want your code to be able to get old in an elegant way and never ever break so you can run some code that was written in Rust and just leave it running forever and not worry about it it's a phenomenal name in that sense but there's another fun pun here I want to make trust one of the biggest values of rust is that you can encode the safety as behaviors in your actual application so you can on a given function specify what it can and can't do when it does and doesn't interface with memory and the idea of the borrow Checker surely makes maintaining and editing your code a little bit harder but it makes your code much more trustworthy because you're encoding the exact behaviors that you're expecting it to do and if your code breaks those expectations it will not compile but this comes at costs the costs are iteration speed because when you're working in a system that's this strict it's inherently going to take more time to make changes if you know exactly what you're building day one you can get there pretty fast still but if the thing you're building changes it can get rough and don't take that from me somebody who doesn't write a whole lot of rust take it from somebody like Prime who tried really hard to write a lot of rust and ran into these issues when it came to maintaining the code bases and specifically pivoting what they were expected to do as soon as you need to make a change like make something async that was sync before rust starts to fall apart not because it's bad but because it's so strict and arguably so good that it slows you down and makes you really think through every one of those changes speaking of thinking things through compilation times the borrow Checker and all the fun things that the rust compiler does take a lot of time it's funny because we're using rust as the language of choice to rebuild all the JavaScript build tools so you could use something like RS pack instead of webpack and go from 20 to 30 minute JS builds to 10 to 30 seconds instead I find this really funny because Russ projects are notoriously slow for building when you try to build a project with rust it's going to take a bit nextjs itself so so to be clear not an app you built with nextjs the actual nextjs code base has a lot of rust stuff in it and building next now takes hours to do because the rust Builder happens to be very very slow according to chat the generics are another one of the largest hits to the compilation times that checks out but man the compilation times suck and there's no pretending otherwise there's also some other factors that affect here like compilation times so we said generics hurt also cargo won't allow binaries so if you have some Rust code that takes a while to compile and you release it as a package you can't ship the package with the binary you have to ship the package as source which means if somebody installs your package it's going to take significantly more time for their whole project to build because that's to build all of the code that you shi them this is a terrible thing if you're trying to move fast but that's an awesome thing if you're the US government because now you're actually building all of the source the likelihood that a bad binary gets snuck in is significantly lower so when it comes to the trust part of Rost that's great and the rest of the things here like iteration speed and compilation times those aren't anywhere near as big of a deal for the US government because 90% of the time actually here's an interesting Spectrum on a given project what percent of the time is being spent on planning and process versus is being spent on everything else so I'm going to change I'll leave that as red so if we have these two boxes so I'm going to do red and green because I think it's funny so red is planning and green here is building for most projects these are not an even split here's what my process looks like we have a little bit of planning we have some building we realize our plan sucks so we do some more planning we do a tiny more bit of building before realizing we're entirely we then plan a good bit and then we actually build the thing this probably looks somewhat familiar to uh the thing we love to call Agile this is the reality of a lot of software devs lives so this is most devs reality this is not how the government works government software here's how government software works a lot of planning a little bit of building but the building didn't know that there was still planning going on so this happened underneath and then they had to kill it because it turns out they were building the wrong thing so this gets killed the planning continues then we get an actual approval so we actually start building and then whenever somebody tries to propose a change they get told no you we already have this approval and once the like spec is completed and the Project's goals are determined nothing changes from that point forward in fact whoever was doing this planning is now working on other things now so we spend an insane amount of time planning we might start building a little bit but it's almost certainly going to get killed you finally get to the point where you have the spec and then you actually spend far too much time building the thing after the plan has been ironed out but this more iterative back and forth style that's how most of us are used to coding applications this doesn't happen in the government there is a planning process Sometimes some exploratory Dev work going on during planning that almost always gets killed and then the actual project starts this is the reality of government software and yes this results in things taking significantly longer but it also enables them to theoretically have things that are much safer and built in very different ways this is hopefully hopefully you can see why building rust here doesn't make a lot of sense cuz if I put all the time into building here and we realize we were slightly wrong now I have to try and rebuild it realize that porting this to the new idea sucks so we go back and plan a whole bunch on how to actually do it and we build it right the third time the government doesn't do things that way they don't rebuild a thing during development once development starts it almost always is going to ship eventually and then if they want to fix things they do another one of these giant planning arcs and then they fix it so if we' make this timeline like over time the next step is like Decades of nothing and then another one of these planning arcs and then another one of these building arcs and that's the reality of government software so much of it gets built then you sit in this empty void for an indefinite amount of time until things are on fire enough then exploration starts often because of a change in government leadership then they actually start building point from Chad is that the build time's a lot shorter hopefully you're not saying that for the government because it's not not especially with a project like translating all C code to rust this is like a 10 plus year project this is insane this going to take so much time to do the potential for this is nuts especially if they open source it but there's a lot of work here to be done another important point that came up is that the government uses contractors a lot of the employees who are working on these things aren't full-time so once this building is done the whole team probably gets AED at this point the team leaves contractors leave which means there's basically no iteration to happen cuz people who built it aren't there they might hire one person or some new person to do whatever updates and they have no context to history of why any of the previous decisions were made and their lives just get harder and harder and harder as a result this is the reality of government software and this is why so many government sites are so miserable and ancient because they go through this Loop but again this is why rust makes sense when you think of the strengths of rust it's code you can trust takes a bit to write but it's also a lot safer doesn't have weird behaviors hidden in binary you have no access to and it's code you can leave for a long long time and not have to worry about as much and if you write code in a way that isn't safe it yells at you so when you do get to this point where the team that made the thing left where the contractors are all gone the likelihood the new person you bring in things up is significantly smaller if your options at this point let's say they have to add a new feature and this hasn't be made by one of those contractors that comes in afterwards that doesn't know the code base let's say we theoretically have two options it takes this long but uh has has a high chance to break or it can take this long but as a low chance to break if you brought a contractor into an old like rails code base and said add this feature they can add that feature really quickly the likelihood that feature breaks other things or potentially doesn't even work at all relatively high with rust it will likely take significantly longer to implement that feature or that change but the benefit is that it's much less likely to fail because the things that could fail are encoded as rules here to prevent you from doing that it's almost like a level Beyond testing I think these things make rust a weakish choice for devs working at traditional companies where where resilience reliability are much more important than iteration speed but for the government the incentives here make perfect sense to me I'm aligned I think this is a good call this might be my single favorite use case for rust I'm a little scared of what adding llms to our government code basis is going to look like for translating code but that's the only scary part is it sounds like they're going to use AI to generate rust code out of C code if Prime lowle learning or somebody smarter than me does their own equivalent of this video what I would want them to do is take a bunch of C code that is unsafe throw it at all the different llms Claude and chat GPT and all the others tell it to translate to rust and see how safe the output rust code is I'm very curious I'm not good enough at rust to give honest takes on those things but if prime or low level learning want to I'd love to see them do that until next time keep your op safe peace nerds