WEBVTT 00:00:00.840 --> 00:00:05.641 For this section we're going to take the techniques we've built up so far, with the 00:00:05.641 --> 00:00:10.210 for-loop and the if-statement and the average technique to detect color areas, 00:00:10.210 --> 00:00:14.996 and put them together to do a real movie special effect. So this special effect is 00:00:14.996 --> 00:00:19.477 called blue screening. And I should point out that what you think of as video data 00:00:19.477 --> 00:00:23.905 is really just made of a series of regular still images, just showed something like 00:00:23.905 --> 00:00:28.493 twenty to 60 times per second. So for our purposes we'll just do the special effect 00:00:28.493 --> 00:00:33.294 on still images, and you can see how it'll generalize to movies. So the idea is it'll 00:00:33.294 --> 00:00:37.561 be a very simple model. Imagine I have the stop sign image. So we've, we've talked 00:00:37.561 --> 00:00:41.551 about the code to detect the red area inside of here. So for the blue screen 00:00:41.562 --> 00:00:45.830 effect, we're gonna have a second image which I'll call the, the background image. 00:00:45.830 --> 00:00:50.234 And the idea is, we'll detect the red areas, as we've done before. But whenever 00:00:50.234 --> 00:00:54.352 I get a pixel over here on the red area, I wanna think about the sort of 00:00:54.352 --> 00:00:59.043 corresponding pixel over from this other image. And I'm gonna copy that pixel over 00:00:59.043 --> 00:01:03.676 to sorta fill in all the red area over here with pixels over here from the other 00:01:03.676 --> 00:01:07.747 image. Sorry, it's a slightly more detailed diagram of it. So, I'll do 00:01:07.747 --> 00:01:11.959 examples in a little bit. But we're going to have a for-loop where we're kind of 00:01:11.959 --> 00:01:16.379 looping through this image. And we'll have an if-statement which is going to identify 00:01:16.379 --> 00:01:20.538 some of the pixels up here. So let's say we've identified this pixel in the left 00:01:20.538 --> 00:01:24.542 image. So that pixel has some x-y-coordinate and we haven't messed with the 00:01:24.542 --> 00:01:28.650 x-y very much up to now but it, you know, maybe x is 200 and y is 100 or whatever. 00:01:28.806 --> 00:01:33.212 So what we want to do is get those two numbers. And then find the corresponding 00:01:33.212 --> 00:01:37.750 pixel over here from the other image. So, maybe a (200,100), or whatever it is. 00:01:37.750 --> 00:01:42.840 There's some pixel that matches up. So once we've got that corresponding image, a 00:01:42.840 --> 00:01:47.868 corresponding pixel from the other image. Then we wanna copy it over, to set the 00:01:47.868 --> 00:01:53.079 pixel over here. And you know, how's that going to work, how do you copy a pixel 00:01:53.079 --> 00:01:57.143 over? You have to think about, 'Well, what defines a pixel? What makes a pixel look 00:01:57.143 --> 00:02:01.029 the way it looks?' And the answer is it's just the three numbers. It's just red, 00:02:01.029 --> 00:02:04.965 green and blue. So we're going to write code to sort of get the red value from 00:02:04.965 --> 00:02:08.951 this pixel over here and then set that as the red value over here. Likewise for 00:02:08.951 --> 00:02:13.139 green and blue. And by setting all those three numbers over we're going to make the 00:02:13.139 --> 00:02:17.327 pixel over on this image just look exactly the same as the pixel over here on this 00:02:17.327 --> 00:02:22.184 side. So, here's some code, here's our first code example to do this. And rather 00:02:22.184 --> 00:02:27.493 than sorta describe the effect, I think in this case I'll just run it so we can see 00:02:27.493 --> 00:02:32.417 it, and then I'll talk about what the lines do. So, here we see the sorta as 00:02:32.417 --> 00:02:37.598 outlined before, the stop sign with all its all the red areas of the stop sign. 00:02:37.789 --> 00:02:42.970 We've substituted in pixels from the leaves image instead. So let me point out the, 00:02:43.162 --> 00:02:48.686 the, how the parts of this work. So first off In all of examples up to now, I think 00:02:48.686 --> 00:02:52.866 we've just opened one image, but it turns out you can have multiple images. So here 00:02:52.866 --> 00:02:56.588 I open a second image, for the leaves.jpeg, and I store that in a variable 00:02:56.588 --> 00:03:01.232 called back. So then all this code we've seen before. I'm just looping over the 00:03:01.232 --> 00:03:06.259 stop sign and detecting the red pixels. So the interesting action is here inside of 00:03:06.259 --> 00:03:11.169 the if-statement. What do we do when we have a red pixel. And it sorta breaks down 00:03:11.169 --> 00:03:16.730 to three parts. So these first two lines just call pixel.getX() and pixel.getY(), and 00:03:16.730 --> 00:03:22.005 store the resulting values in variables x and y. Now I don't think I've used getX() 00:03:22.005 --> 00:03:26.694 and getY() up to now, but what tho-, what those do is just go to a pixel and 00:03:26.694 --> 00:03:31.840 retrieve whatever its x, y is out of it, so very analogous to getRed() and getBlue(). 00:03:32.311 --> 00:03:37.904 So I'm just gonna store those in these variables x y. And then on this line. I 00:03:37.904 --> 00:03:42.681 take those two numbers, x and y. Actually I'll sorta read it left to right here. So 00:03:42.681 --> 00:03:47.340 I go to the back image, so that's the, the leaves image. And I call getPixel, so I 00:03:47.340 --> 00:03:52.116 wanna retrieve a pixel out of it. And then I have to give it some x, y, and in this 00:03:52.116 --> 00:03:56.775 case the x and y I wanna use is the x, y that was set right here. So essentially, 00:03:56.775 --> 00:04:01.433 it's the x, y of the pixel from the stop sign image, saying, whatever that x, y is, 00:04:01.433 --> 00:04:06.574 go get that s, the pixel at the same x, y from the leaves image. So. Once we've got 00:04:06.574 --> 00:04:12.017 that pixel, then I'm just going to start another variable called pixel2. So 00:04:12.230 --> 00:04:16.131 a natural question would be like, oh well couldn't we just call that pixel? Well we 00:04:16.131 --> 00:04:20.131 can't call it pixel, because we're already using pixel to refer to the pixels from 00:04:20.131 --> 00:04:23.984 the stop sign image. So, pixel2 just seemed like sorta the most obvious other 00:04:23.984 --> 00:04:29.122 name to use. All right, so at this point, I've got pixel2 and pixel2 refers to 00:04:29.122 --> 00:04:34.156 the pixel from the background image, from the other image. And now, these three 00:04:34.156 --> 00:04:39.003 lines do what I was describing before as a copy. So let's just look at the first one. 00:04:39.003 --> 00:04:43.445 So it says pixel.setRed. So we've seen that, like, a 100 times. I'm gonna set 00:04:43.445 --> 00:04:48.118 the red value of the pixel from the stop sign image. And what, what am I gonna set 00:04:48.118 --> 00:04:52.837 that red value to be? And what I'm gonna set it to be, is, pixel2.getRed(). 00:04:52.837 --> 00:04:57.699 So I'm getting the pixel from the other image, getting it's red value. So, 160, or 00:04:57.699 --> 00:05:02.253 whatever it is. And I'm just gonna set that into the stop sign image. So 00:05:02.253 --> 00:05:07.176 repeating that for green and blue, we're just, essentially just, we're copying the 00:05:07.176 --> 00:05:11.787 three numbers over. So, in effect, this copies the pixel. So, this is fairly 00:05:11.787 --> 00:05:16.885 complicated. I want to do a few examples of this. So let me do a second example. 00:05:16.885 --> 00:05:22.109 This is a picture of baby Abby, when she's like six months old. Happy little baby on 00:05:22.109 --> 00:05:27.715 her bouncy chair. And later when she's a teenager she can be mad at me for using 00:05:27.715 --> 00:05:33.474 this example. So what I'm going to do is, or, what I want to do in this case is, 00:05:33.690 --> 00:05:39.089 notice the green areas of the bouncy chair. I want to kind of copy over the 00:05:39.089 --> 00:05:45.064 leaves so we get this kind of nature baby leafy effect. And, as, as I recall, there 00:05:45.064 --> 00:05:50.934 were basically two things that I needed to do in the loop here. Well first I should 00:05:50.934 --> 00:05:56.539 point out, so I'm, I'm going to call pixel.getGreen(), for the, the if-test. So 00:05:56.539 --> 00:06:01.677 the first problem was this. Get the, get pixel2. Get the corresponding pixel. 00:06:01.677 --> 00:06:07.216 And here I'm going to write it just as one line. So I say, back - so I go to the other 00:06:07.216 --> 00:06:12.451 image - back.getPixel. And I'll sort of space it out here. I want to get the 00:06:12.451 --> 00:06:16.922 pixel from the other image, and then I have to specify the x, y. And here what I 00:06:16.922 --> 00:06:21.220 want to do is, well, I wanna specify the x, y, of the pixel from the stop sign 00:06:21.220 --> 00:06:25.863 image. And previously I did that by having x, y variables, and well, that's fine. But 00:06:25.863 --> 00:06:30.391 I'm, in this case I'm gonna compact it down to just do it in one line. So really 00:06:30.391 --> 00:06:34.804 I can just say, pixel.getX(). So pixel is the pixel from the stop sign image. And 00:06:34.804 --> 00:06:39.504 I'll just call getX() and that gets the x-value and then I'll just, I'll put that 00:06:39.504 --> 00:06:45.547 directly here inside of. The back.getPixel call. So this it's the same idea as before 00:06:45.547 --> 00:06:50.623 but I've just compacted it down to one line. All right, so that one line sets 00:06:50.623 --> 00:06:56.366 pixel2 to be the corresponding pixel. And then the second thing we need to do in 00:06:56.366 --> 00:07:01.776 the loop is this copy over operation, and that is just literally the same so I'll 00:07:01.776 --> 00:07:09.000 just, I'll copy that from up here. Alright. So now get rid of the blank. 00:07:09.000 --> 00:07:15.371 These blank lines are harmless. Alright, so let's try, let's try that. Alright. So 00:07:15.371 --> 00:07:21.579 you can see we have, we're getting a little bit of the leaf data in here but 00:07:21.579 --> 00:07:29.451 it's not it's a little thin. So we need to make this a little smaller. .4 it's a 00:07:29.451 --> 00:07:45.790 little bit more, I want more. .2. Just a little bit more and. 00:07:45.790 --> 00:07:53.717 Too much. [laugh] All right. So let's try 1.05. Too much for me. Okay, I think 00:07:53.717 --> 00:07:57.812 that's pretty good. So you could see, it's sort of, you know, there was green arrows 00:07:57.812 --> 00:08:01.554 over here. There's sort of the shadow area that's, I mean, maybe just barely 00:08:01.554 --> 00:08:05.448 greenish. And then there was some green blankets over here that we sort of 00:08:05.448 --> 00:08:09.240 sprinkled these leaves in. So it's, it's a nice effect. Or, it's sort of eerie, 00:08:09.240 --> 00:08:13.437 right, that we've gotten these pixels from the leaf image, and sort of put them into 00:08:13.437 --> 00:08:17.229 this other image, and it looks, well, not totally realistic. But you could see 00:08:17.229 --> 00:08:20.970 where, with tuning, you could get this real effect. Alright, so let's try one 00:08:20.970 --> 00:08:28.858 like the movies. So here is our movie star, monkey. And I've taken a picture of monkey 00:08:28.858 --> 00:08:33.765 in front of a blue background. It's just a blue towel. You'll see that monkey is 00:08:33.968 --> 00:08:39.322 brown and has kind of a light tummy here and here, this banana is sort of a light 00:08:39.322 --> 00:08:44.693 yellow. So if you think of brown and tan and yellow, those are all kind of. On near 00:08:44.693 --> 00:08:48.608 yellow, right? So there's a lot of red/green used to make this up. Not a lot 00:08:48.608 --> 00:08:52.946 of blue. So separating the monkey from the blue background, in terms of, thinking of 00:08:52.946 --> 00:08:57.126 it in terms of RGB, is gonna work pretty well. So here's what I wanna use as the 00:08:57.126 --> 00:09:01.676 background. This is a candidate for one of the most famous pictures ever taken. This 00:09:01.676 --> 00:09:05.697 is a picture taken by the Apollo 8 astronauts as they were, they were in 00:09:05.697 --> 00:09:10.088 orbit. They were flying around the moon, and as they came around here's, the Earth 00:09:10.088 --> 00:09:14.585 loomed. Sort of showing the Earth just over the horizon of the moon. Emphasizing it's 00:09:14.585 --> 00:09:19.895 just a little lifeboat we're all on. So what I want to do is start with the monkey 00:09:19.895 --> 00:09:25.315 image, and for all these blue pixels, I want to pull in, pixels from the moon 00:09:25.315 --> 00:09:31.815 image. And so it'd look like the monkey is like, on vacation on the moon. And so I'll 00:09:31.815 --> 00:09:40.053 just grab. This code to get started. Let's see. Okay, so what I wanna do is, for the 00:09:40.053 --> 00:09:46.779 if, I wanna say if getBlue(). So I wanna detect the blue background right? And if 00:09:46.779 --> 00:09:53.590 it's the blue background, then I wanna pull it over pixels from the, the moon. So 00:09:53.590 --> 00:09:59.847 here our image is monkey back is moon.jpg I've got my if- statement. And 00:09:59.847 --> 00:10:04.093 then, this code is unchanged, right? It's just, get the corresponding 00:10:04.093 --> 00:10:08.457 from, it's unchanged from the earlier example. So just get the corresponding 00:10:08.457 --> 00:10:12.529 pixel, copy over red, green, blue. So that, that requires no change at all. 00:10:12.703 --> 00:10:18.409 Alright, so let's try it. Alright, so it, at 1.5 what's happening here is the if 00:10:18.409 --> 00:10:23.953 statement is never firing. I've specified, I've made the hurdle too high. Okay so 00:10:23.953 --> 00:10:32.819 let's try 1.3. Oh, just a little teeny hunt. So I'm too high. Let's try 1.1. Huh, 00:10:32.819 --> 00:10:38.333 okay. So now you can see, you know, for s, the bluishness of this part of the blue 00:10:38.333 --> 00:10:43.847 background was maybe just a little bit more, so we're getting that, but not down 00:10:43.847 --> 00:10:49.081 here. So I need to try 1.0. Now it's getting there. We have a little bit less, 00:10:49.081 --> 00:10:54.595 so actually we can go under we can go under 1 here so I'll try 0.9 as the 00:10:54.595 --> 00:11:00.109 hurdle where I'm just lowering the hurdle. Ooh, that's pretty good. See, there's a 00:11:00.109 --> 00:11:05.651 little teeny bit of moon on his chin there. 0.95. There we go, that's pretty 00:11:05.651 --> 00:11:10.365 good. We could, we could tweak this [inaudible], I think, three, there's a 00:11:10.365 --> 00:11:16.157 little bit of, there we go. I think that's perfect. So, you can see that's, you know, 00:11:16.157 --> 00:11:22.512 movie star in laser battles, spaceship, whatever, like. All right. So just to 00:11:22.512 --> 00:11:27.246 kinda summarize. In the code you're gonna have two images instead of one and then 00:11:27.246 --> 00:11:31.361 you loop through the main image and just kinda the way we've done before, 00:11:31.361 --> 00:11:36.151 identifying areas of some color. And then, once you've identified an area that's in 00:11:36.151 --> 00:11:40.717 the color you care about, then there's this operation to locate the corresponding 00:11:40.717 --> 00:11:45.282 pixel from the other image, and then copy over its red, green, blue values to, to 00:11:45.282 --> 00:11:49.678 get the effect done. And as you can imagine. You know, it's pretty easy to 00:11:49.678 --> 00:11:52.440 make up exercises that work on this technique.