0:00:04.671,0:00:12.887 People are remarkable in all sorts of ways, but our attention in short-term memory are limited resources. 0:00:12.887,0:00:15.030 Fortunately, there’s hope. 0:00:15.030,0:00:21.927 The representations provided by language, gesture, drawings, and objects help us communicate and reason. 0:00:21.927,0:00:29.370 And, importantly for design, different representations can facilitate or hinder different thoughts. 0:00:29.370,0:00:31.984 The insight of today’s lecture is: 0:00:31.984,0:00:36.123 The ways in which we [inaudible] in the world organize and represent ideas[br] 0:00:36.123,0:00:41.211 can have a drastic impact on our cognitive abilities, for better and for worse. 0:00:41.211,0:00:45.858 Things can help us think and you can leverage this as a designer. 0:00:45.858,0:00:50.931 Let’s start with an example that comes from Don Norman and Jiajie Zhang: 0:00:50.931,0:00:57.486 It’s a really strange diner, and in this diner the wait staff is not so good.[br] 0:00:57.486,0:01:04.865 And the three people at the diner ordered oranges but they ended up on the wrong plates. 0:01:04.865,0:01:12.710 And your job — you can try this at home — is to help sort out which orange belongs on which plate. 0:01:12.710,0:01:16.485 And so, the way that the Oranges Puzzle works 0:01:16.485,0:01:22.543 is you need to order the oranges by size: largest to smallest, left to right. 0:01:22.543,0:01:25.449 Only one orange can be transferred at a time. 0:01:25.449,0:01:29.822 An orange can only be transferred to a plate on which it’ll be in the largest. 0:01:29.822,0:01:34.012 And only the largest orange on a plate can be transferred to another plate. 0:01:34.012,0:01:35.954 And so, I have three oranges here. 0:01:35.954,0:01:42.312 Our recording got a little bit delayed because somebody ate this little orange 0:01:42.312,0:01:45.039 so I had to go back to the market and get another one. 0:01:45.039,0:01:48.092 But we have an orange now and let me show you how this works. 0:01:48.092,0:01:50.217 So, I’m going to lay this out. 0:01:54.171,0:01:55.535 I’ve my diner coffee… 0:02:07.904,0:02:11.266 Here you have the three oranges as delivered by the wait staff. 0:02:11.266,0:02:12.883 And you can see they’re in the wrong order — 0:02:12.883,0:02:16.008 The largest one’s in the middle, and the middle one’s over here, 0:02:16.008,0:02:19.307 and the smallest one, which needs to be over here, is over on this side. 0:02:19.307,0:02:23.134 And so, to help save these diners, 0:02:23.134,0:02:29.215 one of the TA’s for the HCI online class, Alex, is going to come and sort this out. 0:02:32.154,0:02:37.295 >> Oh man, seems very difficult. So I have to only transfer one orange at a time, right? 0:02:37.295,0:02:38.180 >> One orange at a time. 0:02:38.180,0:02:41.026 >> And I can only put in to the biggest plate. 0:02:41.026,0:02:43.869 >> It has to be the largest orange on the plate. 0:02:43.869,0:02:47.072 >> Okay. So, can I transfer this here? 0:02:47.072,0:02:48.852 >> No, ’cause then it won’t be the largest orange. 0:02:48.852,0:02:50.850 >> Okay. So, can I transfer it here? 0:02:50.850,0:02:51.405 >> You bet! 0:02:51.405,0:02:53.511 >> Okay. And then can I transfer this one over here? 0:02:53.511,0:02:54.335 >> Sure! 0:02:54.335,0:02:55.536 >> And then this one here? 0:02:55.536,0:02:57.091 >> You got it! Well done! 0:02:57.091,0:02:59.390 >> Thank you! 0:03:00.682,0:03:07.208 >> And there’s a couple other diners who’s got similar orders; they all wanted bagels, and… 0:03:12.424,0:03:15.712 So, their first course is oranges; the second course is going to be bagels. 0:03:18.728,0:03:23.259 The real bagels puzzle is “Why you can’t get a decent bagel in California?” 0:03:23.259,0:03:27.882 But the Bagels Puzzle that we have for today is a little bit different: 0:03:27.882,0:03:33.066 So, we have our largest bagel, medium bagel, smallest bagel. 0:03:33.066,0:03:38.708 So, the plan is the same, except with the bagels, you have the ability to stack them: 0:03:38.708,0:03:45.859 You can out a larger bagel on top of a smaller bagel, but not a smaller bagel on top of a larger one. 0:03:45.859,0:03:51.411 And so, at any point in time, just like the oranges, it needs to be the largest one on the plate. 0:03:51.411,0:03:55.564 And these are in reverse order, and Alex is going to help us again, 0:03:55.564,0:04:01.887 rearranging the bagels so that the largest is on the left and the smallest is on the right. 0:04:09.471,0:04:11.960 >> Cool. So it has to be the largest, right? 0:04:11.960,0:04:12.849 >> Largest on the left. 0:04:12.849,0:04:14.081 >> So I can move this one here? 0:04:14.081,0:04:14.872 >> You got it! 0:04:14.872,0:04:15.947 >> I can move this one here? 0:04:15.947,0:04:16.656 >> Yup! 0:04:16.656,0:04:18.566 >> And I can move this one here? 0:04:18.566,0:04:20.515 >> Yup! There you go, well done! 0:04:21.238,0:04:25.565 Now all of our diners are happy — presuming of course, these are New York bagels, 0:04:25.565,0:04:28.133 which they are not, but that’s close enough for now. 0:04:28.133,0:04:30.493 So, that’s our bagels game for today. 0:04:30.493,0:04:32.244 You can try these at home, and what you see 0:04:32.244,0:04:41.691 is the fact that the bagels represent some of the constraints of the problem in the physical structure 0:04:41.691,0:04:44.084 makes it much easier to remember the state: 0:04:44.084,0:04:49.576 You don’t have to remember which of these two bagels is larger and therefore movable 0:04:49.576,0:04:51.357 because the larger on is on top — 0:04:51.357,0:04:54.450 you’re not going to move this medium-sized bagel; it’s underneath. 0:04:54.450,0:04:56.065 And that makes it all a lot easier — 0:04:56.065,0:05:02.687 when a representation of a problem enforces the constraints and the rules of the problem. 0:05:02.687,0:05:07.926 And you can think of this a lot of ways, like, when you leave your keys by the door[br] 0:05:07.926,0:05:11.781 so that right as you walk out the door, you bring your keys with you. 0:05:11.781,0:05:16.501 That’s a way of structuring the physical world to serve as a reminder 0:05:16.501,0:05:23.614 and to embed the constraints that you need —to leave with your keys — in the physical space itself.[br] 0:05:23.614,0:05:27.672 And, as some of you may have realized, you can think of these as being Towers of Hanoi. 0:05:27.672,0:05:29.345 And what’s interesting thing about this — 0:05:29.345,0:05:36.439 Towers of Hanoi is the kid’s game where you can move circles between pegs — that look a lot like bagels. 0:05:36.439,0:05:42.808 And what we’ve done is we’ve turned this Towers of Hanoi game into something where the rules are the same; 0:05:42.808,0:05:45.010 it’s just the representation has changed. 0:05:45.010,0:05:49.351 And let’s play a game — we’re going to play a card game now. 0:05:49.351,0:05:51.210 Two players. 0:05:51.210,0:05:58.318 Each of you is going to draw a number, one at a time such that you have three numbers that add up to 15.[br] 0:05:58.318,0:06:04.607 And your draw is without replacement; so, if I’ve picked five, that’s no longer available. 0:06:04.607,0:06:10.085 For example, if I was able to get four, five, and six, 0:06:10.085,0:06:14.623 together those add up to fifteen, and that would be a winning hand. 0:06:14.623,0:06:24.479 So, let’s say for example, I pick five for starters, and the other player picks six. 0:06:24.479,0:06:38.114 So I pick three, and the other player picks two, and then I pick seven. 0:06:38.114,0:06:41.653 That would be a winning hand. 0:06:41.653,0:06:48.852 And, this is hard enough for one player. As you can see, it’s even harder to try and play both sides.[br] 0:06:48.852,0:06:50.849 So we can make this a little bit easier. 0:06:50.849,0:06:56.780 So let’s say, instead of having it be numbers imaginary in our head, 0:06:56.780,0:06:59.892 we’ll do the same thing, but with playing cards. 0:06:59.892,0:07:05.983 And so I’m going to be able to have the numbers one through nine here — or ace through nine. 0:07:05.983,0:07:09.512 So, here we have the playing cards, ace through nine. 0:07:09.512,0:07:14.064 And again, two players, and this time we can lay ’em out in the table. 0:07:14.064,0:07:18.656 So I’ll lay these out so I can see ’em. 0:07:18.656,0:07:27.719 And again, start out by picking five, And the other player might pick six, and so I’ll go three. 0:07:27.719,0:07:30.780 And at this point — especially [as] the other player can see my numbers — 0:07:30.780,0:07:36.068 they’ll know that seven is what I want next, so they’re going to go seven, 0:07:36.068,0:07:39.510 and I’ll go two to prevent their 15. 0:07:39.510,0:07:43.244 And — alright, now they’re going to head off in another [tactic]… 0:07:43.244,0:07:47.386 So… maybe we can do… 0:07:52.417,0:07:58.420 Let’s try this: one. I haven’t thought it through, but that might help. 0:07:58.420,0:08:04.069 And at this point, I start to wonder whether this is going to work. 0:08:04.069,0:08:11.725 So, take — I’m not really playing here — but I’ll take nine — 12, 14… almost… 0:08:11.725,0:08:20.506 and it’s kind of nothing you can do at this point, and so… No dice! 0:08:20.506,0:08:26.157 A lot easier, especially early on, but still somewhat challenging. 0:08:26.157,0:08:31.284 Let’s play another game. We’ll pick an easier this time. How about Tic Tac Toe? 0:08:31.284,0:08:37.478 So, this is pretty easy. I think many of you know this game: So we can do x’s and o’s. 0:08:37.478,0:08:52.319 Somebody can go x, and somebody can go o. We can do x, and o, and x, and o, and x wins. 0:08:52.319,0:08:57.616 That game is a whole lot faster than either of the two cards we play, 0:08:57.616,0:09:04.047 but what might shock you is that these two games are isomorphs. 0:09:04.047,0:09:10.086 So I can fill in the numbers one through nine and have them be exactly equivalent to our card game. 0:09:21.117,0:09:26.767 Any row or diagonal that adds to fifteen also produces a winning game in Tic Tac Toe. 0:09:26.767,0:09:33.942 So for example, I can pick this right column, that’s 15, 15, all the way through. 0:09:33.942,0:09:39.459 And what we see from this is that the way that you represent the problem — 0:09:39.459,0:09:44.590 drawing numbers in our head, cards on the table or numbers in a grid — 0:09:44.590,0:09:52.198 has drastic influences on our ability to solve that problem fluently and to see alternate solutions 0:09:52.198,0:09:56.309 and to be able to coordinate our action with another person.[br] 0:09:56.309,0:10:02.190 And this is why Herb Simon, the famous artificial intelligence and cognitive science researcher, 0:10:02.190,0:10:08.715 writes that “Solving a problem simply means representing it so as to make the solution transparent.” 0:10:08.715,0:10:11.653 For example, if you think about proofs in mathematics, 0:10:11.653,0:10:16.458 what’s true at the end of the proof is also true at the beginning. 0:10:16.458,0:10:19.201 The only thing that’s changed along the way 0:10:19.201,0:10:28.479 is that the fact that the proof is indeed true has been rendered clear to the reader. 0:10:28.479,0:10:35.238 Why were the Oranges Puzzle and the number selection game so difficult? 0:10:35.238,0:10:39.364 It’s because both of them tax our working memory — 0:10:39.364,0:10:45.179 we had to keep in mind what moves are possible, what numbers we held, 0:10:45.179,0:10:51.239 what the other person was thinking and maybe doing — and that was a lot to keep in mind all at once.[br] 0:10:51.239,0:10:56.794 Human beings are incredible in all sorts of ways; working memory is not one of them. 0:10:56.794,0:11:00.749 And one of the most important things that we can do with user interfaces 0:11:00.749,0:11:05.109 by embedding the constraints of the problem in the user interface itself 0:11:05.109,0:11:12.862 is to offload working memory so that those limited resources become available for other people. 0:11:12.862,0:11:19.266 You may have heard the famous “seven plus or minus two” as a limit of working memory, 0:11:19.266,0:11:25.796 that we can keep seven plus or minus two things available at any point in time. 0:11:25.796,0:11:33.691 And that’s not exactly true — the real story is a bit more complicated — 0:11:33.691,0:11:38.317 but if you want to have a rule of thumb to work with as a designer, 0:11:38.317,0:11:41.991 I suggest two plus or minus two — 0:11:41.991,0:11:50.076 that basically don’t require users to keep anything in mind that you can possibly put on the screen. 0:11:50.076,0:11:55.409 So far the examples that we’ve looked at — oranges, bagels, cards, Tic Tac Toe — 0:11:55.409,0:11:57.335 are kind of toy problems. 0:11:57.335,0:12:02.572 And, you know, they may make for some good entertainment at a dinner party 0:12:02.572,0:12:05.914 but it’s not real-world stuff. 0:12:08.960,0:12:13.685 They demonstrate, however, principles of the power of distributing cognition — 0:12:13.685,0:12:16.850 That absolutely have real world impact. 0:12:16.850,0:12:21.289 One great example of this is that task management system Getting Things Done. 0:12:21.289,0:12:25.681 It’s very much designed in the principles of distributing cognition in mind. 0:12:25.681,0:12:31.638 So, for example, one of the rules of the Getting Things Done system is that 0:12:31.638,0:12:34.013 whenever something comes to your mind that you need to do, 0:12:34.013,0:12:37.808 the first thing you do is write it down — somewhere, anywhere. 0:12:37.808,0:12:41.291 And the reason for that is that if you have something in mind 0:12:41.291,0:12:44.750 that needs to be completed and you haven’t written it down yet, 0:12:44.750,0:12:50.529 you’re spending a lot of cycles of working memory, remembering to “I need to do my laundry”, 0:12:50.529,0:12:52.987 the same thing — “I need to do my laundry”, “I need to do my laundry”… 0:12:52.987,0:12:57.328 and that’s chewing up resources that could be better deployed elsewhere. 0:12:57.328,0:13:02.124 “Write it down” gets it out of your mind, and you can move on. 0:13:02.124,0:13:11.279 One of the words that people toss around all the time in terms of effective user interfaces 0:13:11.279,0:13:14.653 is that “this user interface is ‘natural.’” 0:13:14.653,0:13:18.041 And when we say that, we mean a couple of different things, 0:13:18.041,0:13:23.655 but one of the things that we mean [that] we can see in our example of the oranges and the bagels 0:13:23.655,0:13:31.097 [is] that the bagels, as a task, is more natural — because the properties of the representation — 0:13:31.097,0:13:38.786 bagels can stack — matches the properties of the thing that’s being represented.[br] 0:13:38.786,0:13:45.753 One of my user interfaces for this perspective is the Proteus indigestible pill. 0:13:45.753,0:13:49.730 This is a pill that, when you swallow it, 0:13:49.730,0:13:59.068 it sends out a little signal so that your phone — or the Internet — knows that you’ve taken this pill. 0:13:59.068,0:14:09.380 And the ability of keeping track of whether you’ve taken the pill or not automatically, by virtue of taking the pill, 0:14:09.380,0:14:11.184 is a really natural user interface — 0:14:11.184,0:14:20.084 the ingestion act serves both to take the pill and to mark that you have done so. 0:14:20.084,0:14:26.022 There’s another example of a really natural interaction in this system which is 0:14:26.022,0:14:32.323 that, in addition to the pill, there’s a transmitter that you need to stick to your body. 0:14:32.323,0:14:37.326 And the transmitter has a limited battery life. 0:14:37.326,0:14:39.738 So, how do you turn the transmitter on? 0:14:39.738,0:14:44.398 It’s a Band-aid-like system, and the way that you turn on the transmitter is 0:14:44.398,0:14:49.101 you peel back the Band-aid bandage, and that turns it on. 0:14:49.101,0:14:53.242 And so, again, that action that you need to do to stick it on yourself 0:14:53.242,0:14:57.881 is exactly the same action that you use to turn the [transmitter] on. 0:14:57.881,0:15:04.408 And so by integrating the necessary step with the step that’s easy to forget, 0:15:04.408,0:15:07.876 you don’t forget the step anymore. 0:15:07.876,0:15:11.786 So let’s bring this back to normal user interfaces. 0:15:11.786,0:15:16.584 Here’s an example of the Print dialog on a Macintosh. 0:15:16.584,0:15:23.260 And one of the things that you can see here is that there’s a world-in-miniature strategy in this Print dialog box. 0:15:23.260,0:15:29.690 So, if I want to look at the Stanford Academic Calendar, I can print that here. 0:15:29.690,0:15:32.137 And, what we’re going to look at is: 0:15:32.137,0:15:40.296 here’s our world in miniature, where we see the entire legal-sized page shown inside the dialog box. 0:15:40.296,0:15:42.636 And one of the things that that makes clear 0:15:42.636,0:15:48.771 is that I’m not going to be able to fit the top and the bottom of the page on a letter-sized page 0:15:48.771,0:15:53.754 because a letter sized page is three inches shorter than a legal page, 0:15:53.754,0:15:58.716 and so consequently, you know, it’s probably not something I want to do. 0:15:58.716,0:16:04.387 And, one of the things that I like about this is that by showing you the world in miniature, 0:16:04.387,0:16:11.012 the challenges of working that you’ve got to decide here become much clearer. 0:16:11.012,0:16:17.392 Here’s another example; this is from Microsoft Word, and what we can see here is — 0:16:17.392,0:16:23.174 I’m again about to print this, and the dialog box that I get from word says 0:16:23.174,0:16:31.574 “A footer of section 1 is set outside the printable area of the page. Do you want to continue?” 0:16:31.574,0:16:37.643 And I can click Yes or I can click No. 0:16:37.643,0:16:41.682 Now, in order to answer this problem, I need to know: 0:16:41.682,0:16:48.826 is it just that the bounding box of the footer is outside what the printer can print and so it’s irrelevant 0:16:48.826,0:16:53.963 or is there actually content that I need that won’t be printed? 0:16:53.963,0:16:55.597 None of that is available from this dialog box, 0:16:55.597,0:16:57.621 and so, unlike the previous print dialog 0:16:57.621,0:17:01.246 where we saw the world in miniature and could see what was being cut off, 0:17:01.246,0:17:03.803 here we have no idea what’s being cut off, 0:17:03.803,0:17:07.291 and consequently, this is a much less effective user interface. 0:17:08.107,0:17:12.191 And that’s the end of part one.