Transcript for: Motion Amplification and Pulse Measurements
Give it 30 seconds. - Are we turning it off
and turning it on again? - We are. - Great. A while back I made a video
about motion amplification a video processing
technique that amplifies tiny movements in a video. And look, here's the original video. The human eye can't detect any motion but it's possible to
take those tiny motions and amplify them so they can
be detected by human eyes. You can also amplify changes in color until it's possible to
see someone's heartbeat. Link to that video in the description. But when my friend Andrew
Steele saw the video, he said, "I've got an idea. Can I strap electrodes to
your chest and film it?" So he explained the idea further. And you know, smart watches, they have a sensor on the back and flashing lights and stuff. Well, what the sensor
sees when those lights are flashing is incredible. And Andrew wanted to
see if we could recreate what a smartwatch does when
those lights are flashing just using a camera pointed at my face and some software that
he wrote on his computer. The most common light you'll see on the back of a smartwatch is green and that helps to measure your pulse but you may also see a red light. In fact, if your watch has
a red light on the back, it has another light that you can't see. It has an invisible light on top of that. And the red light and the
invisible light together help to measure the level
of oxygen in your blood. But let's look at the
pulse measurement first because that's the one we're
trying to recreate here. - I'm gonna be filming you. I've got a nice bright
direct light source. My wife's a doctor. This is her stethoscope. And what we've done is, highly technical, I've just shoved a microphone and that's gonna allow us to listen to the sound of your heart. - Okay.
And also under here, I believe, we've got a heart rate monitor. Because you're measuring the
electrical activity directly. - Yeah.
Like it's gonna be much better than the rubbish
I get off this camera. - So this is to see
whether we're any good. But first things first, I need to check whether I've got this in the right place. Do you know what?
(heart beats) I think I've nailed it. - I'm gonna start recording from that chest strap.
Yeah. - I'm gonna set my camera going. - Yeah.
And we're gonna see if we can detect your heart
rate using just your face. - Okay.
Try and hold still. All right, that'll do. There is your face. It's going through and
it's processing this video. So it's measuring the color
of all the different pixels on your face. - Your software is trying to find the ups and downs in those red,
green, and blue channels. And then it's amplifying them.
[Andrew] Exactly. - [Steve] And laying it on top, okay. - So you can imagine this is actually quite medically useful. Imagine you've got a
prematurely born baby. It was nice if you could
just remotely monitor them by filming them basically. - And obviously if the premature
baby has a beard like this, you would shave it. Ooh, this is cool. These individual frames that you generated using your Python code will
then turn into a video. - [Andrew] You look a bit unwell in this. - I do, yeah. Yeah. I mean I look a bit unwell in this. - These are the raw RGB values. So I've subtracted that
slowly moving background. I've smoothed it out
a little bit and then. - [Steve] Look at that. That's great, isn't it? - [Andrew] What really interested me, I thought one of the reasons
they use green lights is 'cause the green
signal is the strongest. And I've read this in papers. And as you can see that probably is true. - [Steve] Yeah.
The green peaks and troughs do tend to be a bit bigger and smaller, but at the same time, there's actually quite a lot
of data there in that red and blue channel as well. - A green LED is cheaper than a white LED. - But the main reason is
actually about how deep these various different
colors go into your skin. Red's got a very long wavelength. That means it travels really
deep into your arm for example. And so what that means is that you are, you know, potentially
getting distracting signals from other places basically. And blue light, by contrast, it's got a very short wavelength. And that means it doesn't go
very deep into the skin at all. So it might not get deep
enough to see the blood. Whereas green is this
sort of nice compromise. It goes in deep enough to see the blood, but not so deep it's
basically just detecting muscles or something. So if you had like really sensitive eyes, you'd just be walking around and people's face should
be getting brighter and darker and brighter and darker. - Sounds like what it's
like to be Superman. That's weird.
It must be. - He never talks about it. - He can probably hear it as well. (Steve laughs) What a nightmare. Because of that
stethoscope, remember that, we've got the sound of
the heartbeat as well. So we've got that. That act as a gold standard and prove that I've not
just totally faked this. So here we go. You ready? (heart beats) - [Steve] Look at that
(heart beats) - [Andrew] And what you can see. Oh, and then you get a little twitch, flash green.
[Steve] Yeah. (heart beats) It's very twitchy obviously. (heart beats) - [Andrew] Yeah, it
goes woo-dump, woo-dump. - Yeah. That's not bad, is it? The heart rate that you can see on my face seems to match up with
the audio of my heart that you can hear. You can see why this reminded Andrew over the motion amplification video. The difference is we
are using the whole face to work out what the pulse is and then converting that pulse data into a color overlay so we can
see what's going on visually. We also took an ECG using
that strap around my chest as an extra test to make sure that this thing was really working. - It's got two little contacts
on your chest here and here. - Yeah. - And it's measuring
the voltage across that is basically the voltage, you know, from one side of your heart to the other. - And when you made me lick them, you weren't just being weird. - No. Cause it's really important for the electrical contacts. - Okay, sure it is. - [Andrew] Absolutely beautiful. - [Steve] Yeah.
It's so clean. But these massive big spikes, then we can just overlay this lovely, clean, beautiful ECG trace. - Yeah.
With that RGB video data. So what you can see is the ECG. Well, basically I can
believe those peaks line up, can't you?
Yeah. That's not bad, is it? I reckon we nailed that. Here's a question though. We were using a constant white
light to illuminate my face but watches use a green
light that is flashing. Why is the light flashing? - They want to flash the green light on. They take a measurement with their little one pixel sensor.
Yeah. - And then when the green light's off, they can then measure the ambient light. So the surrounding light. Let's imagine a worst case
scenario, you're in a disco. Disco light's are flashing on and off. They don't want that to
distract their measurement. - [Steve] Yeah.
So they need to subtract that flashing disco light and therefore by turning
off that green LED, they can subtract the constant
ambient light around you. So the other thing these
watches can do, right, is they can measure your breathing rate using your heart rate measurement. So when you breathe in.
[Steve] Yeah. - [Andrew] Your lungs
get bigger obviously. - [Steve] Yeah.
[Andrew] And that means the space inside your chest for your heart gets a little bit smaller. - No.
So if it wants to pump the same volume of blood around your body per minute, it's gonna have to beat
a little bit faster. And what you can do is you can watch for the
changes in heart rate over time. And that allows you then to
extract the breathing rate. - So that's what the green light is for. It's your pulse and as it turns out, your breathing rate as well. But what about the red light? Well, you may have seen
one of these before. It's a pulse oximeter, a device
that attaches to your finger and it will measure how much
oxygen there is in your blood, how saturated your blood is with oxygen. And your smartwatch is
doing the same thing. They both do it with
the help of a red light. The reason they use a red light is because the more oxygenated your
blood is the more red it is. So it shines a red light onto your body and there's a sensor that detects how much of that red light is reflected back. The only difference with these things is that the sensor is on the
opposite side of the light source. So it's detecting how much
light passes through your finger as opposed to how much is
reflected off your wrist. And because the amount of
blood in your wrist goes up and down a little as your heartbeats, you'll expect to get a signal from the sensor that
looks a bit like this. So the higher the line is the more oxygenated your blood is and the lower the line is the less oxygenated your blood is. But here's the problem. That line will also go up and down depending on
how thick your veins are, how much ambient light
there is around you, how thick your skin is, how close the sensor is
to one of your veins. So there's actually no
way to look at this line and know how much oxygen
there is in your blood. So how do we fix that? Well, the trick is to use a second light that's a different color. So different, in fact,
that you can't see it. It's infrared. That's the invisible light
I was telling you about at the start. This footage is from an infrared camera. So the flashing light you can see here you can't see with your own eyes. And here's that infrared
light on the pulse oximeter. You remember I said
that blood gets more red the more oxygen there is in it. Well, the opposite is true for infrared. The more oxygenated your blood is the less infrared it becomes. So if you were to plot
what the light sensor sees in infrared, it would be up
here for deoxygenated blood and down here for oxygenated blood. So if you were to look at
both readings together, this is what oxygenated
blood would look like. And then as you slowly
remove oxygen from the blood one line would go down and the other would go
up until they cross over. And so now you don't have to worry about, you know, ambient light, where exactly is the sensor
pointing on the wrist, all that sort of stuff. Because look, as these external factors like ambient light vary, well, these two lines move up and down but they move up and down together. The relative height of the
two lines stays the same. So that's the thing you measure and that's the thing that can tell you how much oxygen there is in your blood. How clever is that? Now because blood is typically 90 something percent oxygenated and because oxygenated blood
strongly absorbs infrared light with an infrared torch
and an infrared camera, we can see what your smartwatch sees. So this is what your smartwatch sees when it looks at your wrist. Of course, it's a one pixel
image that your smartwatch sees. We've got many more pixels here so we can pick out the detail
of the veins and arteries. How cool is that? But even freakier than that, look what happens when
I shine it into my head. - [Andrew] That's absolutely terrifying. - Me and Andrew had quite
a wide ranging conversation in the end. So I've taken all that extra footage and turned it into a
bonus video for Patreon's, link to my Patreon page
in the description. There's also a load of outtakes at the very end of this video,
so stick around for that. I've recommended Andrew
Steele's videos before. If you haven't checked
out his channel yet, this is the perfect opportunity to do it. His follow up video to this one about empirically testing whether
these different watch brands are any good at all, is brilliant. The link is in the description and it's also in the card there. Guess what's coming in the post. - [Child] What? - More KiwiCo. (kids cheer) One of the greatest
challenges of parenting could be summed up in a Venn diagram. Here are all the things that
kids enjoy and want to do and here are all the things
that are good for kids that nourish them and enrich them. And the overlap between
the two is razor thin. It feels so good to find
something in that overlap, which is why I've been
getting KiwiCo for so long. KiwiCo is a subscription service where you get a STEM project
in the post once a month. Everything you need for the
project is there in the box. And there are nine
different subscription lines for every possible age group. This is from the Kiwi subscription line which is for five to eight year olds. It's actually got two projects in it. One is this disappearing coin box. My son loves magic tricks, figuring out how they
work, and explaining them. So this one is perfect for him. This one is a game that you can play with a visor that has a mirror in it. So you're doing the puzzle
that's on the ground but you're looking straight
ahead through a mirror. And it sparked all these
conversations about how your brain communicates
with your limbs because you tried to move something up but it moves down because you're looking through this mirror. Because we've been getting it for so long, I feel like I can talk about the long term effect it's had on my kids. Like there's such tinkerers now. We just got this dancing singing Rudolph out of storage for Christmas. And my daughter didn't want it to sing, she just wanted it to dance
and for its nose to flash. So she said, can we take it apart and figure out where the voice box is? So we unscrewed everything and we saw the bit that makes it dance, the wires that go to the flashing nose, and we found the wires
that go to the speaker. We yanked them out, put it
all back together again, and everything worked
except for the speaker. And that's just how my
kids look at the world now. They see something and they start thinking
about how does it work. The offer on this one is really good. If you go to kiwico.com/stevemould, you'll get your first month for free. You can also buy individual
boxes one off as gifts. I've done that before. Best on clever. The link is also in the description. So sort out your holiday shopping today. - Now I'm listening to
your abdomen sounds. - I'm hungry, okay. It's mansplaining my heart. - Not that sort of doctor. - No, no. You've got PhD.
I've got PhD. - Oh, show off. I'll get emails from people
sometimes saying professor. - Professor Mould. - Yeah.
Totally. - Thanks. - Professor of YouTube. (Steve laughs) - Could you make the text smaller though because it's almost like- - [Andrew] Okay, I mean, it's
a waste of space, isn't it? It's gonna be beautiful, isn't it? I mean, look at those colors. - [Steve] It's just, I
mean, it looks horrendous. - [Andrew] Look smurf. - [Steve] Yeah. Showing your code in a video is just asking for code review in the comments. - [Andrew] Oh, it's absolutely, I mean, what is all this
commented at, nonsense. - [Steve] And what are you doing? - [Andrew] Why have I, I mean, why have I written all these
functions that I never use at the top of the file? - [Steve] Oh, it's self-documenting it. - [Andrew] It certainly is. Yeah, yeah, totally. So this is a picture of- - It's not a picture, it's a graph. I know you're a nerd,
but that's not a picture. - That's true. It is a graph. And what we're looking at here. Yeah, so-
You run at a very steady pace. - I mean, I guess most people do. I don't know if I'm-
I don't know. - Am I the Terminator? - Yeah, it does. I'm a bit freaked out by- - It's remarkable.
How robotic you are. - This wasn't a perfectly
controlled experiment. - Yeah.
You had a scary guy just out of shot. - There was. I dunno why, well, it's 'cause while you
were filming I suppose, so yeah. You had to be there, didn't you? - I had to be there.
Yeah. - Not much I could do. - Yeah. It was very intimidating.