1 00:00:00,840 --> 00:00:05,641 For this section we're going to take the techniques we've built up so far, with the 2 00:00:05,641 --> 00:00:10,210 for-loop and the if-statement and the average technique to detect color areas, 3 00:00:10,210 --> 00:00:14,996 and put them together to do a real movie special effect. So this special effect is 4 00:00:14,996 --> 00:00:19,477 called blue screening. And I should point out that what you think of as video data 5 00:00:19,477 --> 00:00:23,905 is really just made of a series of regular still images, just showed something like 6 00:00:23,905 --> 00:00:28,493 twenty to 60 times per second. So for our purposes we'll just do the special effect 7 00:00:28,493 --> 00:00:33,294 on still images, and you can see how it'll generalize to movies. So the idea is it'll 8 00:00:33,294 --> 00:00:37,561 be a very simple model. Imagine I have the stop sign image. So we've, we've talked 9 00:00:37,561 --> 00:00:41,551 about the code to detect the red area inside of here. So for the blue screen 10 00:00:41,562 --> 00:00:45,830 effect, we're gonna have a second image which I'll call the, the background image. 11 00:00:45,830 --> 00:00:50,234 And the idea is, we'll detect the red areas, as we've done before. But whenever 12 00:00:50,234 --> 00:00:54,352 I get a pixel over here on the red area, I wanna think about the sort of 13 00:00:54,352 --> 00:00:59,043 corresponding pixel over from this other image. And I'm gonna copy that pixel over 14 00:00:59,043 --> 00:01:03,676 to sorta fill in all the red area over here with pixels over here from the other 15 00:01:03,676 --> 00:01:07,747 image. Sorry, it's a slightly more detailed diagram of it. So, I'll do 16 00:01:07,747 --> 00:01:11,959 examples in a little bit. But we're going to have a for-loop where we're kind of 17 00:01:11,959 --> 00:01:16,379 looping through this image. And we'll have an if-statement which is going to identify 18 00:01:16,379 --> 00:01:20,538 some of the pixels up here. So let's say we've identified this pixel in the left 19 00:01:20,538 --> 00:01:24,542 image. So that pixel has some x-y-coordinate and we haven't messed with the 20 00:01:24,542 --> 00:01:28,650 x-y very much up to now but it, you know, maybe x is 200 and y is 100 or whatever. 21 00:01:28,806 --> 00:01:33,212 So what we want to do is get those two numbers. And then find the corresponding 22 00:01:33,212 --> 00:01:37,750 pixel over here from the other image. So, maybe a (200,100), or whatever it is. 23 00:01:37,750 --> 00:01:42,840 There's some pixel that matches up. So once we've got that corresponding image, a 24 00:01:42,840 --> 00:01:47,868 corresponding pixel from the other image. Then we wanna copy it over, to set the 25 00:01:47,868 --> 00:01:53,079 pixel over here. And you know, how's that going to work, how do you copy a pixel 26 00:01:53,079 --> 00:01:57,143 over? You have to think about, 'Well, what defines a pixel? What makes a pixel look 27 00:01:57,143 --> 00:02:01,029 the way it looks?' And the answer is it's just the three numbers. It's just red, 28 00:02:01,029 --> 00:02:04,965 green and blue. So we're going to write code to sort of get the red value from 29 00:02:04,965 --> 00:02:08,951 this pixel over here and then set that as the red value over here. Likewise for 30 00:02:08,951 --> 00:02:13,139 green and blue. And by setting all those three numbers over we're going to make the 31 00:02:13,139 --> 00:02:17,327 pixel over on this image just look exactly the same as the pixel over here on this 32 00:02:17,327 --> 00:02:22,184 side. So, here's some code, here's our first code example to do this. And rather 33 00:02:22,184 --> 00:02:27,493 than sorta describe the effect, I think in this case I'll just run it so we can see 34 00:02:27,493 --> 00:02:32,417 it, and then I'll talk about what the lines do. So, here we see the sorta as 35 00:02:32,417 --> 00:02:37,598 outlined before, the stop sign with all its all the red areas of the stop sign. 36 00:02:37,789 --> 00:02:42,970 We've substituted in pixels from the leaves image instead. So let me point out the, 37 00:02:43,162 --> 00:02:48,686 the, how the parts of this work. So first off In all of examples up to now, I think 38 00:02:48,686 --> 00:02:52,866 we've just opened one image, but it turns out you can have multiple images. So here 39 00:02:52,866 --> 00:02:56,588 I open a second image, for the leaves.jpeg, and I store that in a variable 40 00:02:56,588 --> 00:03:01,232 called back. So then all this code we've seen before. I'm just looping over the 41 00:03:01,232 --> 00:03:06,259 stop sign and detecting the red pixels. So the interesting action is here inside of 42 00:03:06,259 --> 00:03:11,169 the if-statement. What do we do when we have a red pixel. And it sorta breaks down 43 00:03:11,169 --> 00:03:16,730 to three parts. So these first two lines just call pixel.getX() and pixel.getY(), and 44 00:03:16,730 --> 00:03:22,005 store the resulting values in variables x and y. Now I don't think I've used getX() 45 00:03:22,005 --> 00:03:26,694 and getY() up to now, but what tho-, what those do is just go to a pixel and 46 00:03:26,694 --> 00:03:31,840 retrieve whatever its x, y is out of it, so very analogous to getRed() and getBlue(). 47 00:03:32,311 --> 00:03:37,904 So I'm just gonna store those in these variables x y. And then on this line. I 48 00:03:37,904 --> 00:03:42,681 take those two numbers, x and y. Actually I'll sorta read it left to right here. So 49 00:03:42,681 --> 00:03:47,340 I go to the back image, so that's the, the leaves image. And I call getPixel, so I 50 00:03:47,340 --> 00:03:52,116 wanna retrieve a pixel out of it. And then I have to give it some x, y, and in this 51 00:03:52,116 --> 00:03:56,775 case the x and y I wanna use is the x, y that was set right here. So essentially, 52 00:03:56,775 --> 00:04:01,433 it's the x, y of the pixel from the stop sign image, saying, whatever that x, y is, 53 00:04:01,433 --> 00:04:06,574 go get that s, the pixel at the same x, y from the leaves image. So. Once we've got 54 00:04:06,574 --> 00:04:12,017 that pixel, then I'm just going to start another variable called pixel2. So 55 00:04:12,230 --> 00:04:16,131 a natural question would be like, oh well couldn't we just call that pixel? Well we 56 00:04:16,131 --> 00:04:20,131 can't call it pixel, because we're already using pixel to refer to the pixels from 57 00:04:20,131 --> 00:04:23,984 the stop sign image. So, pixel2 just seemed like sorta the most obvious other 58 00:04:23,984 --> 00:04:29,122 name to use. All right, so at this point, I've got pixel2 and pixel2 refers to 59 00:04:29,122 --> 00:04:34,156 the pixel from the background image, from the other image. And now, these three 60 00:04:34,156 --> 00:04:39,003 lines do what I was describing before as a copy. So let's just look at the first one. 61 00:04:39,003 --> 00:04:43,445 So it says pixel.setRed. So we've seen that, like, a 100 times. I'm gonna set 62 00:04:43,445 --> 00:04:48,118 the red value of the pixel from the stop sign image. And what, what am I gonna set 63 00:04:48,118 --> 00:04:52,837 that red value to be? And what I'm gonna set it to be, is, pixel2.getRed(). 64 00:04:52,837 --> 00:04:57,699 So I'm getting the pixel from the other image, getting it's red value. So, 160, or 65 00:04:57,699 --> 00:05:02,253 whatever it is. And I'm just gonna set that into the stop sign image. So 66 00:05:02,253 --> 00:05:07,176 repeating that for green and blue, we're just, essentially just, we're copying the 67 00:05:07,176 --> 00:05:11,787 three numbers over. So, in effect, this copies the pixel. So, this is fairly 68 00:05:11,787 --> 00:05:16,885 complicated. I want to do a few examples of this. So let me do a second example. 69 00:05:16,885 --> 00:05:22,109 This is a picture of baby Abby, when she's like six months old. Happy little baby on 70 00:05:22,109 --> 00:05:27,715 her bouncy chair. And later when she's a teenager she can be mad at me for using 71 00:05:27,715 --> 00:05:33,474 this example. So what I'm going to do is, or, what I want to do in this case is, 72 00:05:33,690 --> 00:05:39,089 notice the green areas of the bouncy chair. I want to kind of copy over the 73 00:05:39,089 --> 00:05:45,064 leaves so we get this kind of nature baby leafy effect. And, as, as I recall, there 74 00:05:45,064 --> 00:05:50,934 were basically two things that I needed to do in the loop here. Well first I should 75 00:05:50,934 --> 00:05:56,539 point out, so I'm, I'm going to call pixel.getGreen(), for the, the if-test. So 76 00:05:56,539 --> 00:06:01,677 the first problem was this. Get the, get pixel2. Get the corresponding pixel. 77 00:06:01,677 --> 00:06:07,216 And here I'm going to write it just as one line. So I say, back - so I go to the other 78 00:06:07,216 --> 00:06:12,451 image - back.getPixel. And I'll sort of space it out here. I want to get the 79 00:06:12,451 --> 00:06:16,922 pixel from the other image, and then I have to specify the x, y. And here what I 80 00:06:16,922 --> 00:06:21,220 want to do is, well, I wanna specify the x, y, of the pixel from the stop sign 81 00:06:21,220 --> 00:06:25,863 image. And previously I did that by having x, y variables, and well, that's fine. But 82 00:06:25,863 --> 00:06:30,391 I'm, in this case I'm gonna compact it down to just do it in one line. So really 83 00:06:30,391 --> 00:06:34,804 I can just say, pixel.getX(). So pixel is the pixel from the stop sign image. And 84 00:06:34,804 --> 00:06:39,504 I'll just call getX() and that gets the x-value and then I'll just, I'll put that 85 00:06:39,504 --> 00:06:45,547 directly here inside of. The back.getPixel call. So this it's the same idea as before 86 00:06:45,547 --> 00:06:50,623 but I've just compacted it down to one line. All right, so that one line sets 87 00:06:50,623 --> 00:06:56,366 pixel2 to be the corresponding pixel. And then the second thing we need to do in 88 00:06:56,366 --> 00:07:01,776 the loop is this copy over operation, and that is just literally the same so I'll 89 00:07:01,776 --> 00:07:09,000 just, I'll copy that from up here. Alright. So now get rid of the blank. 90 00:07:09,000 --> 00:07:15,371 These blank lines are harmless. Alright, so let's try, let's try that. Alright. So 91 00:07:15,371 --> 00:07:21,579 you can see we have, we're getting a little bit of the leaf data in here but 92 00:07:21,579 --> 00:07:29,451 it's not it's a little thin. So we need to make this a little smaller. .4 it's a 93 00:07:29,451 --> 00:07:45,790 little bit more, I want more. .2. Just a little bit more and. 94 00:07:45,790 --> 00:07:53,717 Too much. [laugh] All right. So let's try 1.05. Too much for me. Okay, I think 95 00:07:53,717 --> 00:07:57,812 that's pretty good. So you could see, it's sort of, you know, there was green arrows 96 00:07:57,812 --> 00:08:01,554 over here. There's sort of the shadow area that's, I mean, maybe just barely 97 00:08:01,554 --> 00:08:05,448 greenish. And then there was some green blankets over here that we sort of 98 00:08:05,448 --> 00:08:09,240 sprinkled these leaves in. So it's, it's a nice effect. Or, it's sort of eerie, 99 00:08:09,240 --> 00:08:13,437 right, that we've gotten these pixels from the leaf image, and sort of put them into 100 00:08:13,437 --> 00:08:17,229 this other image, and it looks, well, not totally realistic. But you could see 101 00:08:17,229 --> 00:08:20,970 where, with tuning, you could get this real effect. Alright, so let's try one 102 00:08:20,970 --> 00:08:28,858 like the movies. So here is our movie star, monkey. And I've taken a picture of monkey 103 00:08:28,858 --> 00:08:33,765 in front of a blue background. It's just a blue towel. You'll see that monkey is 104 00:08:33,968 --> 00:08:39,322 brown and has kind of a light tummy here and here, this banana is sort of a light 105 00:08:39,322 --> 00:08:44,693 yellow. So if you think of brown and tan and yellow, those are all kind of. On near 106 00:08:44,693 --> 00:08:48,608 yellow, right? So there's a lot of red/green used to make this up. Not a lot 107 00:08:48,608 --> 00:08:52,946 of blue. So separating the monkey from the blue background, in terms of, thinking of 108 00:08:52,946 --> 00:08:57,126 it in terms of RGB, is gonna work pretty well. So here's what I wanna use as the 109 00:08:57,126 --> 00:09:01,676 background. This is a candidate for one of the most famous pictures ever taken. This 110 00:09:01,676 --> 00:09:05,697 is a picture taken by the Apollo 8 astronauts as they were, they were in 111 00:09:05,697 --> 00:09:10,088 orbit. They were flying around the moon, and as they came around here's, the Earth 112 00:09:10,088 --> 00:09:14,585 loomed. Sort of showing the Earth just over the horizon of the moon. Emphasizing it's 113 00:09:14,585 --> 00:09:19,895 just a little lifeboat we're all on. So what I want to do is start with the monkey 114 00:09:19,895 --> 00:09:25,315 image, and for all these blue pixels, I want to pull in, pixels from the moon 115 00:09:25,315 --> 00:09:31,815 image. And so it'd look like the monkey is like, on vacation on the moon. And so I'll 116 00:09:31,815 --> 00:09:40,053 just grab. This code to get started. Let's see. Okay, so what I wanna do is, for the 117 00:09:40,053 --> 00:09:46,779 if, I wanna say if getBlue(). So I wanna detect the blue background right? And if 118 00:09:46,779 --> 00:09:53,590 it's the blue background, then I wanna pull it over pixels from the, the moon. So 119 00:09:53,590 --> 00:09:59,847 here our image is monkey back is moon.jpg I've got my if- statement. And 120 00:09:59,847 --> 00:10:04,093 then, this code is unchanged, right? It's just, get the corresponding 121 00:10:04,093 --> 00:10:08,457 from, it's unchanged from the earlier example. So just get the corresponding 122 00:10:08,457 --> 00:10:12,529 pixel, copy over red, green, blue. So that, that requires no change at all. 123 00:10:12,703 --> 00:10:18,409 Alright, so let's try it. Alright, so it, at 1.5 what's happening here is the if 124 00:10:18,409 --> 00:10:23,953 statement is never firing. I've specified, I've made the hurdle too high. Okay so 125 00:10:23,953 --> 00:10:32,819 let's try 1.3. Oh, just a little teeny hunt. So I'm too high. Let's try 1.1. Huh, 126 00:10:32,819 --> 00:10:38,333 okay. So now you can see, you know, for s, the bluishness of this part of the blue 127 00:10:38,333 --> 00:10:43,847 background was maybe just a little bit more, so we're getting that, but not down 128 00:10:43,847 --> 00:10:49,081 here. So I need to try 1.0. Now it's getting there. We have a little bit less, 129 00:10:49,081 --> 00:10:54,595 so actually we can go under we can go under 1 here so I'll try 0.9 as the 130 00:10:54,595 --> 00:11:00,109 hurdle where I'm just lowering the hurdle. Ooh, that's pretty good. See, there's a 131 00:11:00,109 --> 00:11:05,651 little teeny bit of moon on his chin there. 0.95. There we go, that's pretty 132 00:11:05,651 --> 00:11:10,365 good. We could, we could tweak this [inaudible], I think, three, there's a 133 00:11:10,365 --> 00:11:16,157 little bit of, there we go. I think that's perfect. So, you can see that's, you know, 134 00:11:16,157 --> 00:11:22,512 movie star in laser battles, spaceship, whatever, like. All right. So just to 135 00:11:22,512 --> 00:11:27,246 kinda summarize. In the code you're gonna have two images instead of one and then 136 00:11:27,246 --> 00:11:31,361 you loop through the main image and just kinda the way we've done before, 137 00:11:31,361 --> 00:11:36,151 identifying areas of some color. And then, once you've identified an area that's in 138 00:11:36,151 --> 00:11:40,717 the color you care about, then there's this operation to locate the corresponding 139 00:11:40,717 --> 00:11:45,282 pixel from the other image, and then copy over its red, green, blue values to, to 140 00:11:45,282 --> 00:11:49,678 get the effect done. And as you can imagine. You know, it's pretty easy to 141 00:11:49,678 --> 00:11:52,440 make up exercises that work on this technique.