People are remarkable in all sorts of ways, but our attention in short-term memory are limited resources.

Fortunately, there’s hope.

The representations provided by language, gesture, drawings, and objects help us communicate and reason.

And, importantly for design, different representations can facilitate or hinder different thoughts.

The insight of today’s lecture is:

The ways in which we [inaudible] in the world organize and represent ideas

can have a drastic impact on our cognitive abilities, for better and for worse.

Things can help us think and you can leverage this as a designer.

Let’s start with an example that comes from Don Norman and Jiajie Zhang:

It’s a really strange diner, and in this diner the wait staff is not so good.

And the three people at the diner ordered oranges but they ended up on the wrong plates.

And your job — you can try this at home — is to help sort out which orange belongs on which plate.

And so, the way that the Oranges Puzzle works

is you need to order the oranges by size: largest to smallest, left to right.

Only one orange can be transferred at a time.

An orange can only be transferred to a plate on which it’ll be in the largest.

And only the largest orange on a plate can be transferred to another plate.

And so, I have three oranges here.

Our recording got a little bit delayed because somebody ate this little orange

so I had to go back to the market and get another one.

But we have an orange now and let me show you how this works.

So, I’m going to lay this out.

I’ve my diner coffee…

Here you have the three oranges as delivered by the wait staff.

And you can see they’re in the wrong order —

The largest one’s in the middle, and the middle one’s over here,

and the smallest one, which needs to be over here, is over on this side.

And so, to help save these diners,

one of the TA’s for the HCI online class, Alex, is going to come and sort this out.

>> Oh man, seems very difficult. So I have to only transfer one orange at a time, right?

>> One orange at a time.

>> And I can only put in to the biggest plate.

>> It has to be the largest orange on the plate.

>> Okay. So, can I transfer this here?

>> No, ’cause then it won’t be the largest orange.

>> Okay. So, can I transfer it here?

>> You bet!

>> Okay. And then can I transfer this one over here?

>> Sure!

>> And then this one here?

>> You got it! Well done!

>> Thank you!

>> And there’s a couple other diners who’s got similar orders; they all wanted bagels, and…

So, their first course is oranges; the second course is going to be bagels.

The real bagels puzzle is “Why you can’t get a decent bagel in California?”

But the Bagels Puzzle that we have for today is a little bit different:

So, we have our largest bagel, medium bagel, smallest bagel.

So, the plan is the same, except with the bagels, you have the ability to stack them:

You can out a larger bagel on top of a smaller bagel, but not a smaller bagel on top of a larger one.

And so, at any point in time, just like the oranges, it needs to be the largest one on the plate.

And these are in reverse order, and Alex is going to help us again,

rearranging the bagels so that the largest is on the left and the smallest is on the right.

>> Cool. So it has to be the largest, right?

>> Largest on the left.

>> So I can move this one here?

>> You got it!

>> I can move this one here?

>> Yup!

>> And I can move this one here?

>> Yup! There you go, well done!

Now all of our diners are happy — presuming of course, these are New York bagels,

which they are not, but that’s close enough for now.

So, that’s our bagels game for today.

You can try these at home, and what you see

is the fact that the bagels represent some of the constraints of the problem in the physical structure

makes it much easier to remember the state:

You don’t have to remember which of these two bagels is larger and therefore movable

because the larger on is on top —

you’re not going to move this medium-sized bagel; it’s underneath.

And that makes it all a lot easier —

when a representation of a problem enforces the constraints and the rules of the problem.

And you can think of this a lot of ways, like, when you leave your keys by the door

so that right as you walk out the door, you bring your keys with you.

That’s a way of structuring the physical world to serve as a reminder

and to embed the constraints that you need —to leave with your keys — in the physical space itself.

And, as some of you may have realized, you can think of these as being Towers of Hanoi.

And what’s interesting thing about this —

Towers of Hanoi is the kid’s game where you can move circles between pegs — that look a lot like bagels.

And what we’ve done is we’ve turned this Towers of Hanoi game into something where the rules are the same;

it’s just the representation has changed.

And let’s play a game — we’re going to play a card game now.

Two players.

Each of you is going to draw a number, one at a time such that you have three numbers that add up to 15.

And your draw is without replacement; so, if I’ve picked five, that’s no longer available.

For example, if I was able to get four, five, and six,

together those add up to fifteen, and that would be a winning hand.

So, let’s say for example, I pick five for starters, and the other player picks six.

So I pick three, and the other player picks two, and then I pick seven.

That would be a winning hand.

And, this is hard enough for one player. As you can see, it’s even harder to try and play both sides.

So we can make this a little bit easier.

So let’s say, instead of having it be numbers imaginary in our head,

we’ll do the same thing, but with playing cards.

And so I’m going to be able to have the numbers one through nine here — or ace through nine.

So, here we have the playing cards, ace through nine.

And again, two players, and this time we can lay ’em out in the table.

So I’ll lay these out so I can see ’em.

And again, start out by picking five, And the other player might pick six, and so I’ll go three.

And at this point — especially [as] the other player can see my numbers —

they’ll know that seven is what I want next, so they’re going to go seven,

and I’ll go two to prevent their 15.

And — alright, now they’re going to head off in another [tactic]…

So… maybe we can do…

Let’s try this: one. I haven’t thought it through, but that might help.

And at this point, I start to wonder whether this is going to work.

So, take — I’m not really playing here — but I’ll take nine — 12, 14… almost…

and it’s kind of nothing you can do at this point, and so… No dice!

A lot easier, especially early on, but still somewhat challenging.

Let’s play another game. We’ll pick an easier this time. How about Tic Tac Toe?

So, this is pretty easy. I think many of you know this game: So we can do x’s and o’s.

Somebody can go x, and somebody can go o. We can do x, and o, and x, and o, and x wins.

That game is a whole lot faster than either of the two cards we play,

but what might shock you is that these two games are isomorphs.

So I can fill in the numbers one through nine and have them be exactly equivalent to our card game.

Any row or diagonal that adds to fifteen also produces a winning game in Tic Tac Toe.

So for example, I can pick this right column, that’s 15, 15, all the way through.

And what we see from this is that the way that you represent the problem —

drawing numbers in our head, cards on the table or numbers in a grid —

has drastic influences on our ability to solve that problem fluently and to see alternate solutions

and to be able to coordinate our action with another person.

And this is why Herb Simon, the famous artificial intelligence and cognitive science researcher,

writes that “Solving a problem simply means representing it so as to make the solution transparent.”

For example, if you think about proofs in mathematics,

what’s true at the end of the proof is also true at the beginning.

The only thing that’s changed along the way

is that the fact that the proof is indeed true has been rendered clear to the reader.

Why were the Oranges Puzzle and the number selection game so difficult?

It’s because both of them tax our working memory —

we had to keep in mind what moves are possible, what numbers we held,

what the other person was thinking and maybe doing — and that was a lot to keep in mind all at once.

Human beings are incredible in all sorts of ways; working memory is not one of them.

And one of the most important things that we can do with user interfaces

by embedding the constraints of the problem in the user interface itself

is to offload working memory so that those limited resources become available for other people.

You may have heard the famous “seven plus or minus two” as a limit of working memory,

that we can keep seven plus or minus two things available at any point in time.

And that’s not exactly true — the real story is a bit more complicated —

but if you want to have a rule of thumb to work with as a designer,

I suggest two plus or minus two —

that basically don’t require users to keep anything in mind that you can possibly put on the screen.

So far the examples that we’ve looked at — oranges, bagels, cards, Tic Tac Toe —

are kind of toy problems.

And, you know, they may make for some good entertainment at a dinner party

but it’s not real-world stuff.

They demonstrate, however, principles of the power of distributing cognition —

That absolutely have real world impact.

One great example of this is that task management system Getting Things Done.

It’s very much designed in the principles of distributing cognition in mind.

So, for example, one of the rules of the Getting Things Done system is that

whenever something comes to your mind that you need to do,

the first thing you do is write it down — somewhere, anywhere.

And the reason for that is that if you have something in mind

that needs to be completed and you haven’t written it down yet,

you’re spending a lot of cycles of working memory, remembering to “I need to do my laundry”,

the same thing — “I need to do my laundry”, “I need to do my laundry”…

and that’s chewing up resources that could be better deployed elsewhere.

“Write it down” gets it out of your mind, and you can move on.

One of the words that people toss around all the time in terms of effective user interfaces

is that “this user interface is ‘natural.’”

And when we say that, we mean a couple of different things,

but one of the things that we mean [that] we can see in our example of the oranges and the bagels

[is] that the bagels, as a task, is more natural — because the properties of the representation —

bagels can stack — matches the properties of the thing that’s being represented.

One of my user interfaces for this perspective is the Proteus indigestible pill.

This is a pill that, when you swallow it,

it sends out a little signal so that your phone — or the Internet — knows that you’ve taken this pill.

And the ability of keeping track of whether you’ve taken the pill or not automatically, by virtue of taking the pill,

is a really natural user interface —

the ingestion act serves both to take the pill and to mark that you have done so.

There’s another example of a really natural interaction in this system which is

that, in addition to the pill, there’s a transmitter that you need to stick to your body.

And the transmitter has a limited battery life.

So, how do you turn the transmitter on?

It’s a Band-aid-like system, and the way that you turn on the transmitter is

you peel back the Band-aid bandage, and that turns it on.

And so, again, that action that you need to do to stick it on yourself

is exactly the same action that you use to turn the [transmitter] on.

And so by integrating the necessary step with the step that’s easy to forget,

you don’t forget the step anymore.

So let’s bring this back to normal user interfaces.

Here’s an example of the Print dialog on a Macintosh.

And one of the things that you can see here is that there’s a world-in-miniature strategy in this Print dialog box.

So, if I want to look at the Stanford Academic Calendar, I can print that here.

And, what we’re going to look at is:

here’s our world in miniature, where we see the entire legal-sized page shown inside the dialog box.

And one of the things that that makes clear

is that I’m not going to be able to fit the top and the bottom of the page on a letter-sized page

because a letter sized page is three inches shorter than a legal page,

and so consequently, you know, it’s probably not something I want to do.

And, one of the things that I like about this is that by showing you the world in miniature,

the challenges of working that you’ve got to decide here become much clearer.

Here’s another example; this is from Microsoft Word, and what we can see here is —

I’m again about to print this, and the dialog box that I get from word says

“A footer of section 1 is set outside the printable area of the page. Do you want to continue?”

And I can click Yes or I can click No.

Now, in order to answer this problem, I need to know:

is it just that the bounding box of the footer is outside what the printer can print and so it’s irrelevant

or is there actually content that I need that won’t be printed?

None of that is available from this dialog box,

and so, unlike the previous print dialog

where we saw the world in miniature and could see what was being cut off,

here we have no idea what’s being cut off,

and consequently, this is a much less effective user interface.

And that’s the end of part one.