WEBVTT 00:00:04.671 --> 00:00:12.887 People are remarkable in all sorts of ways, but our attention in short-term memory are limited resources. 00:00:12.887 --> 00:00:15.030 Fortunately, there’s hope. 00:00:15.030 --> 00:00:21.927 The representations provided by language, gesture, drawings, and objects help us communicate and reason. 00:00:21.927 --> 00:00:29.370 And, importantly for design, different representations can facilitate or hinder different thoughts. 00:00:29.370 --> 00:00:31.984 The insight of today’s lecture is: 00:00:31.984 --> 00:00:36.123 The ways in which we [inaudible] in the world organize and represent ideas 00:00:36.123 --> 00:00:41.211 can have a drastic impact on our cognitive abilities, for better and for worse. 00:00:41.211 --> 00:00:45.858 Things can help us think and you can leverage this as a designer. 00:00:45.858 --> 00:00:50.931 Let’s start with an example that comes from Don Norman and Jiajie Zhang: 00:00:50.931 --> 00:00:57.486 It’s a really strange diner, and in this diner the wait staff is not so good. 00:00:57.486 --> 00:01:04.865 And the three people at the diner ordered oranges but they ended up on the wrong plates. 00:01:04.865 --> 00:01:12.710 And your job — you can try this at home — is to help sort out which orange belongs on which plate. 00:01:12.710 --> 00:01:16.485 And so, the way that the Oranges Puzzle works 00:01:16.485 --> 00:01:22.543 is you need to order the oranges by size: largest to smallest, left to right. 00:01:22.543 --> 00:01:25.449 Only one orange can be transferred at a time. 00:01:25.449 --> 00:01:29.822 An orange can only be transferred to a plate on which it’ll be in the largest. 00:01:29.822 --> 00:01:34.012 And only the largest orange on a plate can be transferred to another plate. 00:01:34.012 --> 00:01:35.954 And so, I have three oranges here. 00:01:35.954 --> 00:01:42.312 Our recording got a little bit delayed because somebody ate this little orange 00:01:42.312 --> 00:01:45.039 so I had to go back to the market and get another one. 00:01:45.039 --> 00:01:48.092 But we have an orange now and let me show you how this works. 00:01:48.092 --> 00:01:50.217 So, I’m going to lay this out. 00:01:54.171 --> 00:01:55.535 I’ve my diner coffee… 00:02:07.904 --> 00:02:11.266 Here you have the three oranges as delivered by the wait staff. 00:02:11.266 --> 00:02:12.883 And you can see they’re in the wrong order — 00:02:12.883 --> 00:02:16.008 The largest one’s in the middle, and the middle one’s over here, 00:02:16.008 --> 00:02:19.307 and the smallest one, which needs to be over here, is over on this side. 00:02:19.307 --> 00:02:23.134 And so, to help save these diners, 00:02:23.134 --> 00:02:29.215 one of the TA’s for the HCI online class, Alex, is going to come and sort this out. 00:02:32.154 --> 00:02:37.295 >> Oh man, seems very difficult. So I have to only transfer one orange at a time, right? 00:02:37.295 --> 00:02:38.180 >> One orange at a time. 00:02:38.180 --> 00:02:41.026 >> And I can only put in to the biggest plate. 00:02:41.026 --> 00:02:43.869 >> It has to be the largest orange on the plate. 00:02:43.869 --> 00:02:47.072 >> Okay. So, can I transfer this here? 00:02:47.072 --> 00:02:48.852 >> No, ’cause then it won’t be the largest orange. 00:02:48.852 --> 00:02:50.850 >> Okay. So, can I transfer it here? 00:02:50.850 --> 00:02:51.405 >> You bet! 00:02:51.405 --> 00:02:53.511 >> Okay. And then can I transfer this one over here? 00:02:53.511 --> 00:02:54.335 >> Sure! 00:02:54.335 --> 00:02:55.536 >> And then this one here? 00:02:55.536 --> 00:02:57.091 >> You got it! Well done! 00:02:57.091 --> 00:02:59.390 >> Thank you! 00:03:00.682 --> 00:03:07.208 >> And there’s a couple other diners who’s got similar orders; they all wanted bagels, and… 00:03:12.424 --> 00:03:15.712 So, their first course is oranges; the second course is going to be bagels. 00:03:18.728 --> 00:03:23.259 The real bagels puzzle is “Why you can’t get a decent bagel in California?” 00:03:23.259 --> 00:03:27.882 But the Bagels Puzzle that we have for today is a little bit different: 00:03:27.882 --> 00:03:33.066 So, we have our largest bagel, medium bagel, smallest bagel. 00:03:33.066 --> 00:03:38.708 So, the plan is the same, except with the bagels, you have the ability to stack them: 00:03:38.708 --> 00:03:45.859 You can out a larger bagel on top of a smaller bagel, but not a smaller bagel on top of a larger one. 00:03:45.859 --> 00:03:51.411 And so, at any point in time, just like the oranges, it needs to be the largest one on the plate. 00:03:51.411 --> 00:03:55.564 And these are in reverse order, and Alex is going to help us again, 00:03:55.564 --> 00:04:01.887 rearranging the bagels so that the largest is on the left and the smallest is on the right. 00:04:09.471 --> 00:04:11.960 >> Cool. So it has to be the largest, right? 00:04:11.960 --> 00:04:12.849 >> Largest on the left. 00:04:12.849 --> 00:04:14.081 >> So I can move this one here? 00:04:14.081 --> 00:04:14.872 >> You got it! 00:04:14.872 --> 00:04:15.947 >> I can move this one here? 00:04:15.947 --> 00:04:16.656 >> Yup! 00:04:16.656 --> 00:04:18.566 >> And I can move this one here? 00:04:18.566 --> 00:04:20.515 >> Yup! There you go, well done! 00:04:21.238 --> 00:04:25.565 Now all of our diners are happy — presuming of course, these are New York bagels, 00:04:25.565 --> 00:04:28.133 which they are not, but that’s close enough for now. 00:04:28.133 --> 00:04:30.493 So, that’s our bagels game for today. 00:04:30.493 --> 00:04:32.244 You can try these at home, and what you see 00:04:32.244 --> 00:04:41.691 is the fact that the bagels represent some of the constraints of the problem in the physical structure 00:04:41.691 --> 00:04:44.084 makes it much easier to remember the state: 00:04:44.084 --> 00:04:49.576 You don’t have to remember which of these two bagels is larger and therefore movable 00:04:49.576 --> 00:04:51.357 because the larger on is on top — 00:04:51.357 --> 00:04:54.450 you’re not going to move this medium-sized bagel; it’s underneath. 00:04:54.450 --> 00:04:56.065 And that makes it all a lot easier — 00:04:56.065 --> 00:05:02.687 when a representation of a problem enforces the constraints and the rules of the problem. 00:05:02.687 --> 00:05:07.926 And you can think of this a lot of ways, like, when you leave your keys by the door 00:05:07.926 --> 00:05:11.781 so that right as you walk out the door, you bring your keys with you. 00:05:11.781 --> 00:05:16.501 That’s a way of structuring the physical world to serve as a reminder 00:05:16.501 --> 00:05:23.614 and to embed the constraints that you need —to leave with your keys — in the physical space itself. 00:05:23.614 --> 00:05:27.672 And, as some of you may have realized, you can think of these as being Towers of Hanoi. 00:05:27.672 --> 00:05:29.345 And what’s interesting thing about this — 00:05:29.345 --> 00:05:36.439 Towers of Hanoi is the kid’s game where you can move circles between pegs — that look a lot like bagels. 00:05:36.439 --> 00:05:42.808 And what we’ve done is we’ve turned this Towers of Hanoi game into something where the rules are the same; 00:05:42.808 --> 00:05:45.010 it’s just the representation has changed. 00:05:45.010 --> 00:05:49.351 And let’s play a game — we’re going to play a card game now. 00:05:49.351 --> 00:05:51.210 Two players. 00:05:51.210 --> 00:05:58.318 Each of you is going to draw a number, one at a time such that you have three numbers that add up to 15. 00:05:58.318 --> 00:06:04.607 And your draw is without replacement; so, if I’ve picked five, that’s no longer available. 00:06:04.607 --> 00:06:10.085 For example, if I was able to get four, five, and six, 00:06:10.085 --> 00:06:14.623 together those add up to fifteen, and that would be a winning hand. 00:06:14.623 --> 00:06:24.479 So, let’s say for example, I pick five for starters, and the other player picks six. 00:06:24.479 --> 00:06:38.114 So I pick three, and the other player picks two, and then I pick seven. 00:06:38.114 --> 00:06:41.653 That would be a winning hand. 00:06:41.653 --> 00:06:48.852 And, this is hard enough for one player. As you can see, it’s even harder to try and play both sides. 00:06:48.852 --> 00:06:50.849 So we can make this a little bit easier. 00:06:50.849 --> 00:06:56.780 So let’s say, instead of having it be numbers imaginary in our head, 00:06:56.780 --> 00:06:59.892 we’ll do the same thing, but with playing cards. 00:06:59.892 --> 00:07:05.983 And so I’m going to be able to have the numbers one through nine here — or ace through nine. 00:07:05.983 --> 00:07:09.512 So, here we have the playing cards, ace through nine. 00:07:09.512 --> 00:07:14.064 And again, two players, and this time we can lay ’em out in the table. 00:07:14.064 --> 00:07:18.656 So I’ll lay these out so I can see ’em. 00:07:18.656 --> 00:07:27.719 And again, start out by picking five, And the other player might pick six, and so I’ll go three. 00:07:27.719 --> 00:07:30.780 And at this point — especially [as] the other player can see my numbers — 00:07:30.780 --> 00:07:36.068 they’ll know that seven is what I want next, so they’re going to go seven, 00:07:36.068 --> 00:07:39.510 and I’ll go two to prevent their 15. 00:07:39.510 --> 00:07:43.244 And — alright, now they’re going to head off in another [tactic]… 00:07:43.244 --> 00:07:47.386 So… maybe we can do… 00:07:52.417 --> 00:07:58.420 Let’s try this: one. I haven’t thought it through, but that might help. 00:07:58.420 --> 00:08:04.069 And at this point, I start to wonder whether this is going to work. 00:08:04.069 --> 00:08:11.725 So, take — I’m not really playing here — but I’ll take nine — 12, 14… almost… 00:08:11.725 --> 00:08:20.506 and it’s kind of nothing you can do at this point, and so… No dice! 00:08:20.506 --> 00:08:26.157 A lot easier, especially early on, but still somewhat challenging. 00:08:26.157 --> 00:08:31.284 Let’s play another game. We’ll pick an easier this time. How about Tic Tac Toe? 00:08:31.284 --> 00:08:37.478 So, this is pretty easy. I think many of you know this game: So we can do x’s and o’s. 00:08:37.478 --> 00:08:52.319 Somebody can go x, and somebody can go o. We can do x, and o, and x, and o, and x wins. 00:08:52.319 --> 00:08:57.616 That game is a whole lot faster than either of the two cards we play, 00:08:57.616 --> 00:09:04.047 but what might shock you is that these two games are isomorphs. 00:09:04.047 --> 00:09:10.086 So I can fill in the numbers one through nine and have them be exactly equivalent to our card game. 00:09:21.117 --> 00:09:26.767 Any row or diagonal that adds to fifteen also produces a winning game in Tic Tac Toe. 00:09:26.767 --> 00:09:33.942 So for example, I can pick this right column, that’s 15, 15, all the way through. 00:09:33.942 --> 00:09:39.459 And what we see from this is that the way that you represent the problem — 00:09:39.459 --> 00:09:44.590 drawing numbers in our head, cards on the table or numbers in a grid — 00:09:44.590 --> 00:09:52.198 has drastic influences on our ability to solve that problem fluently and to see alternate solutions 00:09:52.198 --> 00:09:56.309 and to be able to coordinate our action with another person. 00:09:56.309 --> 00:10:02.190 And this is why Herb Simon, the famous artificial intelligence and cognitive science researcher, 00:10:02.190 --> 00:10:08.715 writes that “Solving a problem simply means representing it so as to make the solution transparent.” 00:10:08.715 --> 00:10:11.653 For example, if you think about proofs in mathematics, 00:10:11.653 --> 00:10:16.458 what’s true at the end of the proof is also true at the beginning. 00:10:16.458 --> 00:10:19.201 The only thing that’s changed along the way 00:10:19.201 --> 00:10:28.479 is that the fact that the proof is indeed true has been rendered clear to the reader. 00:10:28.479 --> 00:10:35.238 Why were the Oranges Puzzle and the number selection game so difficult? 00:10:35.238 --> 00:10:39.364 It’s because both of them tax our working memory — 00:10:39.364 --> 00:10:45.179 we had to keep in mind what moves are possible, what numbers we held, 00:10:45.179 --> 00:10:51.239 what the other person was thinking and maybe doing — and that was a lot to keep in mind all at once. 00:10:51.239 --> 00:10:56.794 Human beings are incredible in all sorts of ways; working memory is not one of them. 00:10:56.794 --> 00:11:00.749 And one of the most important things that we can do with user interfaces 00:11:00.749 --> 00:11:05.109 by embedding the constraints of the problem in the user interface itself 00:11:05.109 --> 00:11:12.862 is to offload working memory so that those limited resources become available for other people. 00:11:12.862 --> 00:11:19.266 You may have heard the famous “seven plus or minus two” as a limit of working memory, 00:11:19.266 --> 00:11:25.796 that we can keep seven plus or minus two things available at any point in time. 00:11:25.796 --> 00:11:33.691 And that’s not exactly true — the real story is a bit more complicated — 00:11:33.691 --> 00:11:38.317 but if you want to have a rule of thumb to work with as a designer, 00:11:38.317 --> 00:11:41.991 I suggest two plus or minus two — 00:11:41.991 --> 00:11:50.076 that basically don’t require users to keep anything in mind that you can possibly put on the screen. 00:11:50.076 --> 00:11:55.409 So far the examples that we’ve looked at — oranges, bagels, cards, Tic Tac Toe — 00:11:55.409 --> 00:11:57.335 are kind of toy problems. 00:11:57.335 --> 00:12:02.572 And, you know, they may make for some good entertainment at a dinner party 00:12:02.572 --> 00:12:05.914 but it’s not real-world stuff. 00:12:08.960 --> 00:12:13.685 They demonstrate, however, principles of the power of distributing cognition — 00:12:13.685 --> 00:12:16.850 That absolutely have real world impact. 00:12:16.850 --> 00:12:21.289 One great example of this is that task management system Getting Things Done. 00:12:21.289 --> 00:12:25.681 It’s very much designed in the principles of distributing cognition in mind. 00:12:25.681 --> 00:12:31.638 So, for example, one of the rules of the Getting Things Done system is that 00:12:31.638 --> 00:12:34.013 whenever something comes to your mind that you need to do, 00:12:34.013 --> 00:12:37.808 the first thing you do is write it down — somewhere, anywhere. 00:12:37.808 --> 00:12:41.291 And the reason for that is that if you have something in mind 00:12:41.291 --> 00:12:44.750 that needs to be completed and you haven’t written it down yet, 00:12:44.750 --> 00:12:50.529 you’re spending a lot of cycles of working memory, remembering to “I need to do my laundry”, 00:12:50.529 --> 00:12:52.987 the same thing — “I need to do my laundry”, “I need to do my laundry”… 00:12:52.987 --> 00:12:57.328 and that’s chewing up resources that could be better deployed elsewhere. 00:12:57.328 --> 00:13:02.124 “Write it down” gets it out of your mind, and you can move on. 00:13:02.124 --> 00:13:11.279 One of the words that people toss around all the time in terms of effective user interfaces 00:13:11.279 --> 00:13:14.653 is that “this user interface is ‘natural.’” 00:13:14.653 --> 00:13:18.041 And when we say that, we mean a couple of different things, 00:13:18.041 --> 00:13:23.655 but one of the things that we mean [that] we can see in our example of the oranges and the bagels 00:13:23.655 --> 00:13:31.097 [is] that the bagels, as a task, is more natural — because the properties of the representation — 00:13:31.097 --> 00:13:38.786 bagels can stack — matches the properties of the thing that’s being represented. 00:13:38.786 --> 00:13:45.753 One of my user interfaces for this perspective is the Proteus indigestible pill. 00:13:45.753 --> 00:13:49.730 This is a pill that, when you swallow it, 00:13:49.730 --> 00:13:59.068 it sends out a little signal so that your phone — or the Internet — knows that you’ve taken this pill. 00:13:59.068 --> 00:14:09.380 And the ability of keeping track of whether you’ve taken the pill or not automatically, by virtue of taking the pill, 00:14:09.380 --> 00:14:11.184 is a really natural user interface — 00:14:11.184 --> 00:14:20.084 the ingestion act serves both to take the pill and to mark that you have done so. 00:14:20.084 --> 00:14:26.022 There’s another example of a really natural interaction in this system which is 00:14:26.022 --> 00:14:32.323 that, in addition to the pill, there’s a transmitter that you need to stick to your body. 00:14:32.323 --> 00:14:37.326 And the transmitter has a limited battery life. 00:14:37.326 --> 00:14:39.738 So, how do you turn the transmitter on? 00:14:39.738 --> 00:14:44.398 It’s a Band-aid-like system, and the way that you turn on the transmitter is 00:14:44.398 --> 00:14:49.101 you peel back the Band-aid bandage, and that turns it on. 00:14:49.101 --> 00:14:53.242 And so, again, that action that you need to do to stick it on yourself 00:14:53.242 --> 00:14:57.881 is exactly the same action that you use to turn the [transmitter] on. 00:14:57.881 --> 00:15:04.408 And so by integrating the necessary step with the step that’s easy to forget, 00:15:04.408 --> 00:15:07.876 you don’t forget the step anymore. 00:15:07.876 --> 00:15:11.786 So let’s bring this back to normal user interfaces. 00:15:11.786 --> 00:15:16.584 Here’s an example of the Print dialog on a Macintosh. 00:15:16.584 --> 00:15:23.260 And one of the things that you can see here is that there’s a world-in-miniature strategy in this Print dialog box. 00:15:23.260 --> 00:15:29.690 So, if I want to look at the Stanford Academic Calendar, I can print that here. 00:15:29.690 --> 00:15:32.137 And, what we’re going to look at is: 00:15:32.137 --> 00:15:40.296 here’s our world in miniature, where we see the entire legal-sized page shown inside the dialog box. 00:15:40.296 --> 00:15:42.636 And one of the things that that makes clear 00:15:42.636 --> 00:15:48.771 is that I’m not going to be able to fit the top and the bottom of the page on a letter-sized page 00:15:48.771 --> 00:15:53.754 because a letter sized page is three inches shorter than a legal page, 00:15:53.754 --> 00:15:58.716 and so consequently, you know, it’s probably not something I want to do. 00:15:58.716 --> 00:16:04.387 And, one of the things that I like about this is that by showing you the world in miniature, 00:16:04.387 --> 00:16:11.012 the challenges of working that you’ve got to decide here become much clearer. 00:16:11.012 --> 00:16:17.392 Here’s another example; this is from Microsoft Word, and what we can see here is — 00:16:17.392 --> 00:16:23.174 I’m again about to print this, and the dialog box that I get from word says 00:16:23.174 --> 00:16:31.574 “A footer of section 1 is set outside the printable area of the page. Do you want to continue?” 00:16:31.574 --> 00:16:37.643 And I can click Yes or I can click No. 00:16:37.643 --> 00:16:41.682 Now, in order to answer this problem, I need to know: 00:16:41.682 --> 00:16:48.826 is it just that the bounding box of the footer is outside what the printer can print and so it’s irrelevant 00:16:48.826 --> 00:16:53.963 or is there actually content that I need that won’t be printed? 00:16:53.963 --> 00:16:55.597 None of that is available from this dialog box, 00:16:55.597 --> 00:16:57.621 and so, unlike the previous print dialog 00:16:57.621 --> 00:17:01.246 where we saw the world in miniature and could see what was being cut off, 00:17:01.246 --> 00:17:03.803 here we have no idea what’s being cut off, 00:17:03.803 --> 00:17:07.291 and consequently, this is a much less effective user interface. 00:17:08.107 --> 00:17:12.191 And that’s the end of part one.