This is jigsaw. He's a friendly little robot that's really, really good at only one thing: Putting together any jigsaw puzzle, no matter how complicated. Really, really fast. It's taken us three years to get to this point, but according to our initial tests, we have hopes he might be 200 times faster than the fastest competitive jigsaw puzzler in the world. So today, we're going to walk through what it took to get ready for the ultimate robot versus human face off. And along the way, we may just discover some tricks you might find helpful as a mere human jigsaw puzzler yourself. But before we unpack how jigsaw does what he does I first want to give us humans some well-deserved credit because while this seems pretty straightforward of me to be able to pick up and arrange these 12 pieces of a puzzle, I'm actually doing four very complicated tasks. The first is just picking up a piece. Have you ever stopped to think just how amazing our hands are? Hiding beneath that skin are 27 bones and 34 muscles, which makes them flexible and strong, but they're also incredibly precise and dexterous thanks to the high concentration of nerves in our fingertips for sensing pressure textures, and temperature. We've also evolved to have opposable thumbs, which makes it way easier to hold tools and to manipulate and pick things up than if it was just five fingers all side by side. Step two is rotating the piece to the correct orientation, which again is pretty straightforward when you have all the abilities I just mentioned. For step three, we need to move the piece into position, and that requires our whole arm. As I’ve mentioned before most vertebrae have the same basic arm bone configuration from a human to a bat, to a chicken to a turtle to a dolphin. But when you include the hand, our configuration is the most technically capable arm of any living thing to have ever existed on this planet. If you imagine a large cube in front of me, it's wild that us humans can move this puzzle piece to any position and orientation within that cube. That's really hard for a robot or pretty much any other species to do for that matter. For step four, we have to decide where this piece should go and for us humans, it's hard it’s hard to explain how but when we look at this, it just very quickly feels super obvious this piece should go here. What's actually happening, though, is our eyes communicate visual perception to our brains, which then subconsciously synthesize a complicated combination of pattern recognition, spatial reasoning, visual memory, and executive function, and as a result, in fractions of a second? The answer just feels obvious. And not to brag. But this is once again where us humans are the undisputed champions. Our complex brains are what make us special. Physically, we're kind of unremarkable in the animal kingdom. We're not faster than cheetahs or stronger than bears. We can't swim as well as dolphins or fly like an eagle. It's our brains that make up 2% of our body weight, yet they consume 20% of our energy every day. And that ratio is higher than any other living thing ever. And it's the reason we're the ultimate apex predators because it allows for the huge survival advantages that come from tool use planning, problem solving, language, and large scale cooperation with other humans. It's also what has possibly, up until this moment in history, made us the best... at jigsaw puzzles. So if we wanted to make a robot that was as good or better than us at puzzles, our daunting challenge was to take the 200 million years of evolution that enabled those four steps, and figure out how to translate it into things a robot can do For number one, to pick up a piece in lieu of an opposable thumb and 27 hand bones we used a tiny, specialized suction cup that's often used to manipulate small objects on assembly lines. This solenoid here can cut off and then connect to this vacuum pump, which means we could turn on the suction exactly when we want to pick it up, and then turn it off exactly when we want to let it go For number two we attached the suction cup grabber of jigsaw to this very fine tuned donut motor. That's precise to an incredible 0.005 degrees. That means if you attached an infinitely sharp knife to the end of the motor, it could slice your circular birthday cake into over 65,000 pieces. For number three to move a piece around, we modified our avid CNC router we're constantly using around here for our large builds by upgrading the motors to ClearPath industrial servo motors. These are the same motors we used on the Dominator. Our autonomous Domino robot. Once we did this, we saw it could accurately place a puzzle piece down to .0005 inches That's one tenth the width of a human hair, which means, as you can see here, jigsaw can take the lead out of a mechanical pencil and move all around the table, and then come back and put the lead right back in. So now that jigsaw could pick up rotate, and move any piece with terrifying precision. The only thing he lacked was step four, knowing exactly where to place the pieces. And as you might guess, this was by far the hardest one to solve because all the subconscious work performed by neural pathways in our brain that handle pattern recognition and spatial reasoning, that makes finding that makes finding the right piece feel so obvious to us is a really, really hard problem to solve using just computer logic and code To make matters worse just as we were really struggling to come up with a good solution to this problem, the most devastating thing that can happen as a YouTuber actually happened. My friend Shane from the YouTube channel Stuff Made Here released a video about, you guessed it, a machine that solves jigsaw puzzles. And what you should know about Shane is he's probably the most technically capable engineer I know. If there are like 15 categories that make you a good engineer, my knowledge goes pretty deep in maybe four, and I'm a generalist in the rest But then I have people like Ian and others on my team who help fill in the gaps. Shane, however, is somehow an expert in all 15 which is why you should subscribe to his channel. When I told him he beat me to the finish line, he told me he personally wasn't satisfied with where he landed the project and encouraged me to keep trying. So I persevered. And that's when our fortunes improved, because while filming a video in Rwanda covering the work done at Zipline, where they deliver life saving blood using autonomous drones, I got to spend some time with Ryan, their co-founder, who's responsible for all their complicated software algorithms. And after hearing all of our challenges by the time the plane landed on the way back home, he'd coded up a solution that ended up being the backbone that unlocked everything for us. Basically, instead of trying to compete with our brains very complicated pattern-matching neural network we took a much more simple approach by completely ignoring what was printed on the puzzle and just looked at the edges so while painting it white makes it more impressive and way harder to solve for us humans. It would make no difference to jigsaw. So to do this edge analysis, we first need a set of eyes. but not those eyes. Those are just decorative because it's no secret I love a good pair of googly eyes. And these were our most advanced set of googly eyes ever. Because the googly is controlled by two servo motors jigsaws real eyes were just a cell phone camera. The idea was to run a serpentine pattern over all the pieces and take a picture of each one, and then isolate them against the background then convert the edges of every piece into four splines. So if we wanted to find a match, say for this edge, we would just need to take that spline and then find the puzzle piece that had a corresponding edge spline that matched up perfectly. Because matching pieces share identical edges. Now, even for a small, simple puzzle like this, that's over 200 edge comparisons that need to be made. So the first step is just to look at the overall length of each spline and disregard any edges that were significantly longer or shorter, and that typically reduced the solution space by about 50%. Then for the remaining candidates, we would overlay the splines and then quantify the mismatch by calculating the area between the two edge splines. So two pieces that were obviously not a fit would have a lot of overlapping area, whereas two pieces that were a perfect match the overlap area would essentially be zero. From there, we looked at all the possible spline matchups and ranked the pieces as potential matches with each other from least overlap area to most. Then jigsaw would start with one of the four corner pieces and map out the potential solution space, which you could picture visually like this. So the correct solution is the only one that finds a good fit for all 12 pieces of the puzzle. But as you can see, there's lots of forks and dead ends. And that's because sometimes it was really obvious what the matching piece was. But sometimes there were 3 or 4 close contenders because the edges are so similar, and our pictures weren't always perfect, so jigsaw would always pick what seemed like the best match for a set of two pieces. But at some point the puzzle would only be halfway done and there would be no more good candidates to match the edges to continue solving the puzzle. And that's a dead giveaway he chose the wrong piece somewhere previously up the chain, so he would work his way back to the closest fork and then choose the next best option, and he would keep doing this over and over until he found a path that finally connected all 12 pieces with essentially zero spline mismatches at every connection. And then at that point, because he took pictures of all the pieces, he knows exactly where each of them is starting from over here. And he can simply assemble the puzzle using the precision he demonstrated back in steps one through three. So it was working on a 12 piece puzzle, but now for the real challenge. Could we scale that up to a 1000 piece, all-white puzzle? Since our ultimate goal was to dominate the world's fastest human puzzle solver. We figured maxing out the number of pieces would give the advantage to the non-overwhelmable robot. So after a couple more long weeks filled with plenty of late nights working through all the challenges that come from scaling up a simple prototype 100 times over... we landed here. It's 3 a.m. after three years. We're at this moment about to fully solve it for the first time. What's our confidence level? I'd say 100%. Wow! The code always works on the first try And so with that very heavy dose of sarcasm, jigsaw got to work starting first with taking all the pictures. And that took about 1.5 hours. And after we had the pictures, and we did the prep work on them I mentioned earlier it was time for jigsaw to actually try and solve the 1000 piece puzzle. And astoundingly, while running on just a simple laptop the actual time to figure out the correct placement of all 1000 pieces was less than a minute. And I love how you can actually see him working his way around the puzzle from the top corner solving, and then going back each time he hits a dead end to try and find a different fork in the solution space, until he finds the only combination that makes them all fit perfectly. So at this point, all that was left for jigsaw to do was get to work placing the pieces, since he now knew where every single one needed to go and things were looking really good. And just as we made the mistake of having some kind of glimmer of hope, a single piece didn't quite snap into place. And then a bunch more followed right behind. And upon troubleshooting, what we discovered was there were a few very small sources of error that tended to compound on each other. The more the puzzle was assembled. So, for example, there's always a little slop between any two puzzle pieces, which adds up over a lot of pieces. Or another source of error is the whole puzzle itself shifting slightly when certain pieces were laid down. And this led us the realized jigsaw was actually still missing one final important feature us impressive humans possess Because if you think about it, when we assemble a puzzle we first just approximately place the piece where it needs to go and then we rely on those highly sensitive nerves in our fingertips to form a feedback loop with the brain to make really tiny adjustments until we feel the piece fall into place and then a double tap is the customary move to make it official But since jigsaw doesn't have nerves in his gripper, we approximated that with the z height encoder on a spring-loaded linear slider and that encoder is so accurate if you slice a human hair lengthwise into 50 pieces, jigsaw could feel that tiny hair slice if it was resting on top of the table. So now if jigsaw went to place a piece and he could feel it hadn't quite snapped into place because it was resting too high, he would employ a wiggle routine where it would just barely translate the piece into various directions until he got the feedback from his finger that the piece was fully set down at which point he would give it the customary final tap like any puzzle solver worth his salt. And so, with Jigsaw's new hardware and software upgrades he was ready for one more final attempt. And I should mention, by the way, from a combined hardware software perspective, this is by far the most challenging build ever on my channel, and that includes the auto Bullseye dartboard and the automatic Domino Robot Dominator. So if you really want to appreciate all the juicy details, including all of the source code, you can find a link to the full write up in the video description. You boys ready to try this again officially for the 48th time? Better work this time. What's the confidence levels at? 100%. Have you learned nothing Ian?! Hit the dang button So jigsaw got to work and after taking pictures of all the pieces, he once again actually solved the whole puzzle in a mere 55 seconds which meant it was time to put them all together. And thanks to the wiggle routine you got this jigsaw. The first 20% was knocked out in less than an hour. Okay, I guess we could sit back down And that meant while jigsaw was doing all the hard work, I just get to kick back and relax in a state of Zen gazing at this polar gantry sand garden that you can build yourself. Because unlike the CrunchLabs build box, which is made for kids, we just launched HackPack, which is kind of similar It's just more advanced and created specifically for teenagers and adults. With HackPack, you get a series of really fun programable robots delivered right to your door, where we build it together and learn, step by step, the kinds of engineering skills that go into making the builds on my channel. They're all super fun and even useful, and they'll work right out of the box. No programing required, but since my goal is to get you to always level up your skills with some simple code tweaks we walk you through, you can level up the functionality of your robot, or get totally creative with some hacks of your own. So if you want to enhance or even just take the first step of unlocking the really fun and rewarding hobby of making stuff, just go to CrunchLabs.com, or use the link in the video description where we're giving away one free box as an early subscriber special. and back over with Jigsaw Things were looking very promising for the world record attempt of an off the shelf puzzle, completely solved and assembled by a robot with no human intervention. Some pieces would go right in Uno Wow Some required a little bit of wiggle. Oh, actually, yeah. And some required a lot of wiggle But in the end. let’s go Jigsaw C’mon last piece Jigsaw always came through Yes!! Now feeling more confident in Jigsaw’s abilities It was time for him to meet the greatest jigsaw puzzler the human race had to offer What? Are you done? Who, fun fact also happens to be a national sudoku champion Tammy McLeod I’m the Human Benchmark according to my child I first wanted her to face off against the best jigsaw puzzler I personally know my dear friend Kristen Bell aka Anna, or Veronica mars, or Eleanor, or a bunch more Kristen ever since I told you I was making a puzzle robot, you've been talking so much trash. A little bit of trash. Yeah. The puzzle robot isn't here today. So instead, I have a friend I want you to... challenge. This is Tammy. She holds the Guinness World Record for the fastest puzzle Ever solved by a human being. Hi, Kristen. We kick things off with a simple 30 piece puzzle. And to be honest, Three, two, one. Go! Kristen started off pretty strong. Where's the elephant? Tammy, you don't talk while you're doing it? I mean, I could trash talk, but- Do it! You're gonna lose Tammy. And then you hit me back. Wow, that was one minute and five seconds for a 30 piece puzzle. I'm sweating so much and that was a pretty decisive win. But I wanted to see one more face off, only this time with a 500 piece puzzle and a huge advantage in Kristen's favor. I think I'm going to help you on this one. Okay? And with Kristen's renewed confidence, the clock started and out of the gate Tammy was so confident in this 2-v-1 matchup. She's just doing flipping? Oh my god, that's crazy organized. She saw no problem in helping our cause. What is that little guy? his tail She’s right SHE’S RIGHT Mark, can I ask you a sincere question? Is this the robot? Admittedly, I did have an ulterior motive for this matchup and that was espionage. Tammy, I noticed you're using the box. What! Tammy, get real. You have to look at the picture. It’ll just distract me. I look at the pictures. I have good color acuity. I have very bad shape memory. So if you say had to go against an all white puzzle... That would maybe be my Achilles heel. And this was a very useful bit of information. But she didn't stop there. Because she also told me the four tips she tells amateur jigsaw puzzlers to help them solve puzzles faster. First off, dump out all the pieces and turn them over so you can see them all at the same time. For number two most people start with the edge pieces, and that's not a bad move, but it's not always the best. For example, if there's a puzzle that has something like a gold frame around the outside you should set all the edge pieces aside and then leave them till the end, where there's a lot more information from the interior pieces to help with the border For tip three look for groupings that catch your eye and organize them into piles. So for example, you could organize by colors, textures, or patterns. But either way, you're trying to reduce the search space into smaller chunks because scanning and pattern matching are different parts of the brain. And this way you don't have to do them at the same time, which makes it easier. And finally, if you're down to a bunch of pieces that all look the same like a blue sky, sort by shape based on how many ins and outs they have and orient them all the same way. This will leave you with six total piles and will make it much easier to find the final matches. And I needed to look no further than our current match up- Oh no. I look away for one minute. Don't look at Tammy's, Kristen Things have gotten dire. -to understand there must be some validity to these four steps. I’m not seeing your hands move or your eyes look at the board I’M PANICKING I'm not trying to- DONE!? 34 minutes and 2 seconds. 500 pieces. Congratulations. Congratulations. Your next puzzle match will not be against mere humans. And indeed, after setting things up at CrunchLabs, we were ready to go. Tammy, I'd like to officially introduce you to jigsaw. This is my upgrade from Kristen And Jigsaw was in no mood for conversation as he got straight to work. All right, then! As Tammy entered the matchup of her life with the weight of all humanity resting on her capable shoulders. What's your strategy here? The only shape I can make use of is the edge. Okay, how are we doing, jigsaw? not really sure what I'm supposed to do now... But that’s when I remember I have a specialized skill set of my own I might need to take measurements on your neck later. I'm making you something. That's why you don't have any pieces placed yet on your team. Wow, Tammy you're familiar with the saying “measure once, cut twice.” No, that's not it. And as if on cue, My teammate, wrapped up his measuring. There's a corner piece! and began cutting. did you have to make him so noisy? Three pieces! Four pieces! Tammy, ignore me... Saying five pieces! It ain’t over till it's over. You're still winning, Tammy. Don't let it get to your head. And while I was tempted to count all 1000 Nine pieces! I decided to take up woodworking instead. Are you nervous, Tammy? Even if I lost today, my family would still love me. Jigsaw. If he lost today, would you still love him? No My love is very conditional. Having properly reminded jigsaw what was at stake. Jigsaw has now officially taken the lead. Jigsaw continued to make steady progress while Tammy... Where’s that last edge? Oh, you need one edge piece faced the limits inherent in biology I really want to actually help find this for you That way I can say you needed my help. Without the precise search capabilities... I found it, I’m done with the edge! Of a smart phone to laptop nervous system. Click. I love that sound. Is there any way I could get some water. Jigsaw. Do you need a water break? You notice we are missing a piece in the middle that's on purpose I was the ideas guy and jigsaw was the muscle. Jigsaw. That's your next piece right there buddy. I'm a team player. The only thing is he could be a bit of a control freak So I opted to help Tammy instead. Tammy, do you need this piece? Do you need this piece? Do you need this piece? Do you need thi– And so while jigsaw continued to widen the gap I made myself useful by woodworking. Solitairing Finding Waldo. Tammy right here! Nah I’ll look later and just generally reverting to the role of my youth as the annoying youngest sibling. I love doing puzzles And after all that, Tammy had made a lot of progress- Tammy, I have good news and bad news. The bad news? There's only eight pieces left -but not enough. The good news is I made you a scarf! In fact, if you assume it takes one second per side to try all the remaining combination of pieces she had left working 12 hours a day, skipping weekends. It would take her one and a half months to finish which made this a good time to throw in the towel. Unless you're about to finish I think I'm good. Oh, this is a moment! 1000 pieces in four hours. Okay, Tammy, you represented us humans well, But alas, it’s time we welcome our benevolent robot overlords. It's an honor to lose to him That was beautiful. I gotta say! Good work, jigsaw. At ease So while jigsaw took his victory lap I put the final touches on the trophy I'd carved her. You are the best the human race has to offer. And you did a great job. Just, you know, not that great if you're a teenager or adult and you've always wanted to make and build cool stuff but just have it figured out that first step This is it. It's called the CrunchLabs Hack Pack and it's basically a series of really fun programable robots that get delivered right to your door. Where we build it together and learn, step by step, the kinds of engineering skills that go into making the builds on my YouTube channel, and they all work with no programing required. But since my goal is to take you from wherever you're at and level you up, you can easily hack the microcontroller brains of any of these robots in a bunch of ways to completely level up the functionality. There's also a community where you could share your builds or post questions, as well as an AI chat bot named Mark Robot that will check your code for you and help you implement your most creative ideas. On top of all that, each box has a chance to contain a platinum diploma. If you're box has it, congratulations because college is now free for you or a loved one you want to transfer it to plus you get to come out here and brainstorm one of your own ideas with me and my team for a day. So if you want to enhance or even just take the first step of unlocking the really fun and rewarding hobby of making stuff, just go to CrunchLabs.com or use the link in the video description where to say thank you We're giving away that free box as an early subscriber special Thanks for watching.