0:00:03.375,0:00:09.641
There are several different ways that interfaces can help people think more fluidly
0:00:09.641,0:00:14.706
by distributing their cognition into artifacts in the world.[br]
0:00:14.706,0:00:20.403
When interfaces help people distribute a cognition, it can encourage experimentation;
0:00:20.403,0:00:25.207
it can scaffold learning and reduce errors through redundancy;
0:00:25.207,0:00:28.073
it can show only the differences that matter;
0:00:28.073,0:00:32.409
it can convert slow calculation into fast perception;
0:00:32.409,0:00:35.679
it can support chunking, specially by experts;
0:00:35.679,0:00:37.822
it can increase efficiency of interactions;
0:00:37.822,0:00:40.566
and it can facilitate collaboration.
0:00:40.566,0:00:44.047
Let’s go through these one at a time.
0:00:46.031,0:00:52.613
Here’s a video game that, I bet, many of you all know: This is the Tetris video game.[br]
0:00:52.613,0:00:59.861
And there was a very clever study that David Kirsh and Paul Maglio at UC San Diego ran,
0:00:59.861,0:01:05.704
where they looked at Tetris players playing Tetris, and what they found was really interesting.[br]
0:01:05.704,0:01:10.829
In Tetris, as you may know, you can use keys to move objects around on the screen.
0:01:10.829,0:01:18.456
And at a certain point you could hit the space bar to drop the object and try and get a row of a few blocks.[br]
0:01:18.456,0:01:21.594
Moving and rotating pieces on the screen may seem like a waste of time
0:01:21.594,0:01:25.303
because you have a limited amount of time before the block hits the bottom.
0:01:25.303,0:01:28.850
So, how do you use that most efficiently?
0:01:28.850,0:01:34.004
It turns out that people move the block around the screen
0:01:34.004,0:01:39.072
more than they — in some purely theoretical sense — need to.
0:01:39.072,0:01:44.437
So, in essence, I’m trying out different places that the blocks could go.
0:01:44.437,0:01:50.623
Maybe this is just for novices that when you’re learning Tetris you need to feel things out
0:01:50.623,0:01:54.430
but as you become more of an expert that’s no longer the case?
0:01:54.430,0:01:55.919
Exactly the opposite!
0:01:55.919,0:02:02.900
Kirsh and Maglio found that experts actually relied more heavily on moving objects in the world.
0:02:02.900,0:02:07.906
And what they were doing is they were distributing the cognitive effort of “what if” scenarios —
0:02:07.906,0:02:10.666
What if I placed it over here? What if I placed it over there?
0:02:10.666,0:02:13.175
You absolutely can do that in your mind,
0:02:13.175,0:02:18.615
but it turns that in this case, for most people, it’s cheaper to do it out in the world[br]
0:02:18.615,0:02:24.418
to be able to turn the cognitive task of reasoning about all of the what-if scenarios
0:02:24.418,0:02:29.606
into a perceptual task of “Oh yeah, that block would work perfectly right there.”
0:02:29.606,0:02:34.628
It saves you the effort of having to, in particular, mentally rotate the different pieces
0:02:34.628,0:02:38.318
to figure out how they would fit into the screen.
0:02:38.318,0:02:43.955
Here’s another example from the learning sciences: There are called Montesorri blocks.
0:02:43.955,0:02:51.349
And what we have here are beads that provide a physical representation for number[s].
0:02:51.349,0:02:57.291
Especially for young children, numbers are really abstract concepts, difficult to get your head around.
0:02:57.291,0:03:04.274
And these physical instantiations can help teach addition, multiplication and other simple arithmetic operations.
0:03:04.274,0:03:11.317
So, for example, if I’m going to take three and multiply it by three to get three squared,
0:03:11.317,0:03:18.328
well, I can see that I have three by three — I have a square and I can see that it’s composed of nine beads.
0:03:18.328,0:03:21.776
And there’re other addition and multiplication options that you could with these also.[br]
0:03:21.776,0:03:24.317
And by having this redundant information —
0:03:24.317,0:03:28.648
by taking an abstract concept and “realfying” it and making it concrete —
0:03:28.648,0:03:32.344
it can help scaffold learning and reduce errors.
0:03:32.344,0:03:36.023
The Montessori blocks example and the Tetris example
0:03:36.023,0:03:42.138
show us the power of providing a visual or physical instantiation of abstract ideas.
0:03:42.138,0:03:45.615
So what makes a good representation of this sort?
0:03:45.615,0:03:50.960
Well, you should show the information that you need and nothing else.
0:03:50.960,0:03:54.752
And, what these representations should do is
0:03:54.752,0:04:01.056
it should enable to kinds of tasks that users want to do like comparison and exploration and problem solving.
0:04:01.056,0:04:03.790
And if that seems too abstract or maybe obvious,
0:04:03.790,0:04:07.122
let’s take a look at this example from the London Underground.
0:04:07.122,0:04:10.489
This subway map was introduced about a century ago
0:04:10.489,0:04:16.560
and it was one of the very first maps to introduce a brand-new idea in map design:
0:04:16.560,0:04:24.548
of abstracting the layout of the tracks from the underlying physical geography.
0:04:24.548,0:04:29.586
Prior to this London subway map, the maps would show what the geography was
0:04:29.586,0:04:31.921
and so long things were long and short things were short;
0:04:31.921,0:04:34.872
and if the tracks wandered because that’s the way that it works;
0:04:34.872,0:04:38.934
then the tracks on the map would wander because that’s the way things worked.
0:04:38.934,0:04:42.918
And with the Underground map designers realized was that
0:04:42.918,0:04:50.018
the most common task for subway riders is to figure out how to get from point A to point B,
0:04:50.018,0:04:54.881
and all of this additional detail of faithfulness to the underlying topology
0:04:54.881,0:05:00.521
was getting in the way of that A-to-B task more than it was helping it.
0:05:00.521,0:05:04.617
And so what they did is they stripped that a lot of that unnecessary detail,
0:05:04.617,0:05:09.604
turning this into vertical, diagonal, and horizontal lines.
0:05:09.604,0:05:15.846
So there are some representation between the layout on the map and the layout on the real world —
0:05:15.846,0:05:18.184
North is north, and south is south,
0:05:18.184,0:05:22.192
and things roughly head in the direction that they do in the real world,
0:05:22.192,0:05:24.252
but the detail is stripped out.
0:05:24.252,0:05:29.646
And this makes it much easier to be able to figure out how to get between connections.
0:05:29.646,0:05:32.088
Another thing that they did is they introduced
0:05:32.088,0:05:38.082
what a century later we would call a “focus plus context” representation for the map.
0:05:38.082,0:05:42.445
In the center of London, the subway stations are very densely packed.
0:05:42.445,0:05:46.355
So that area is expanded out: it consumes more of the map real estate.
0:05:46.355,0:05:50.869
As you get out toward the suburbs, the stations are fewer and further between.
0:05:50.869,0:05:55.355
As oppose to that taking up 90% of the map because it’s 90% of the space.
0:05:55.355,0:05:57.991
Those stations are actually scrunched,
0:05:57.991,0:06:00.829
because if you know you need to go northeast to a particular station,
0:06:00.829,0:06:06.504
then the exact distances involved are most of the time less relevant.
0:06:06.504,0:06:11.281
Now, of course with constitutes of good representation is of course task-specific.
0:06:11.281,0:06:15.811
And so what you can see is that by making some tasks easier —
0:06:15.811,0:06:19.433
like getting from A to B when you know A and B,
0:06:19.433,0:06:23.650
or being able to navigate the center of London more effectively —
0:06:23.650,0:06:26.285
you’ve compromised on other tasks.
0:06:26.285,0:06:28.871
And so, for somebody who may need to make decisions
0:06:28.871,0:06:32.748
about what stop to get off at based on some underlying topography,
0:06:32.748,0:06:39.739
or another task that’s compromised is by virtue of the distance between the stations on the map
0:06:39.739,0:06:43.998
not lining up between the distances between stations in the world,[br]
0:06:43.998,0:06:48.219
you can make poor judgments about what’s close and what’s far:
0:06:48.219,0:06:53.177
In the center you may believe things are far apart when they are really close,
0:06:53.177,0:06:58.197
and out in the suburbs, you may believe from the map that things are closer than they actually are.
0:06:58.197,0:07:03.806
And so nearly all representation design is about fitness to task.
0:07:03.806,0:07:06.878
Here’s a temperature map from the Weather Underground.
0:07:06.878,0:07:10.177
It shows the temperature at each location in the Bay area,
0:07:10.177,0:07:17.011
geo-referenced so that the temperature number is placed right on top of that physical location.
0:07:17.011,0:07:21.302
What do you think are the benefits and drawbacks of this representation?
0:07:21.302,0:07:26.039
What’s it good for and what’s it a problem for?
0:07:30.670,0:07:35.438
If you know the physical coordinate that you’d like the temperature for,
0:07:35.438,0:07:39.376
say for example, “What temperature is it along the coast?”
0:07:39.376,0:07:45.707
and I don’t care or don’t know the exact name of the town, this works incredibly well.
0:07:45.707,0:07:50.024
It’s also a reasonable interface, in some sense,
0:07:50.024,0:07:54.481
for trying to get a good [inaudible] of what are the temperatures like in the area overall:
0:07:54.481,0:08:00.425
I can see, for example, as I head inland the temperature tends to get warmer.
0:08:00.425,0:08:03.958
There are a lot of ways in which we could make this better.
0:08:03.958,0:08:10.147
So, right now, every temperature is shown identically no matter what temperature it is
0:08:10.147,0:08:15.138
which means it’s hard to scan: I have to read every single temperature one by one.
0:08:15.138,0:08:22.289
I could make this better if instead I had the temperature number somehow —
0:08:22.289,0:08:29.088
the color or size of the temperature number — correspond to the weather,[br]
0:08:29.088,0:08:36.533
But if we’re going to start mapping the variables of the map to something like color,
0:08:36.533,0:08:39.448
we need to be careful to get it right.
0:08:39.448,0:08:43.353
This is an example that comes from Edward Tufte.
0:08:43.353,0:08:49.315
His books on visual design and the graphical representations of data are fantastic
0:08:49.315,0:08:50.856
and I strongly encourage you to read them.
0:08:50.856,0:08:58.596
In this map, we see how a computer scientist might make a mapping for Japan.
0:08:58.596,0:09:04.235
This is showing the height above or below sea level as color,
0:09:04.235,0:09:11.963
and what you can see is the depth below sea level is represented by the color spectrum Roy G. Biv.
0:09:11.963,0:09:21.007
Now, one of the challenges of hue is that it’s not an additive representation.
0:09:21.007,0:09:26.901
So it’s not really a strong ordering that we give to the colors perceptually.
0:09:26.901,0:09:32.961
It’s a substitutive representation that red and yellow are qualitatively different,
0:09:32.961,0:09:37.113
but we don’t automatically have a more-than or less-than relationship between the two.
0:09:37.113,0:09:39.095
Another problem on this representation,
0:09:39.095,0:09:45.093
is it’s very difficult by glance to tell what’s higher and what’s lower in the sea.
0:09:45.093,0:09:55.938
Conversely, the individual isosurfaces — the individual chunks of a particular depth — really pop out.
0:09:55.938,0:10:01.948
Like, to me, for example, the yellow depth pops out very strongly
0:10:01.948,0:10:05.846
and that shape really comes to the attentional fore,
0:10:05.846,0:10:10.942
which, if you are making a rock-and-roll poster for the Fillmore, would be awesome,
0:10:10.942,0:10:15.266
but if you’re trying to get a sense of the contours of the sea,
0:10:15.266,0:10:23.435
what become salient to you, the outlines for what’s 400 meters below sea level, it’s probably just not that relevant.
0:10:23.435,0:10:28.301
So how could we continue our theme of using color as a representational cue
0:10:28.301,0:10:32.805
but have it be more meaningful than you might see in this case
0:10:32.805,0:10:35.565
where we’re mapping it to the color spectrum?
0:10:35.565,0:10:39.643
And here’s Edward Tufte’s redesign which I think is much better.
0:10:39.643,0:10:43.701
There’s a couple of things that I really like about the representation here.
0:10:43.701,0:10:51.250
The first one is that all of the things that are above sea level are brown — are kind of an earth tone.
0:10:51.250,0:10:56.566
So, we’re leveraging our intuitions about the physical world and using that metaphorically for the map.
0:10:56.566,0:11:00.874
So, the land-colored stuff is land.
0:11:00.874,0:11:05.577
Similarly, all of the water is blue — the water-colored stuff is water.
0:11:05.577,0:11:15.846
And furthermore, we can see how the intensity — or the luminance — of that color blue changes with depth.
0:11:15.846,0:11:22.773
And the deeper blues are darker blue, which corresponds to our physical intuitions.
0:11:22.773,0:11:29.651
And of course, water doesn’t really get that much darker at the kind of depth that we’re talking about here —
0:11:29.651,0:11:34.762
our intuition about darker colors being deeper comes from much shallower depths.
0:11:34.762,0:11:41.224
But the idea — you can leverage this thing that we all know that water right by the shore is a paler color
0:11:41.224,0:11:44.359
and as you get more of it gets darker.
0:11:44.359,0:11:51.860
So this is a really wonderful way to see that these darker areas here are deeper than these shallow areas here.
0:11:51.860,0:11:58.196
With the London Underground map, we saw how the representation of the map —
0:11:58.196,0:12:03.869
what makes a good representation — was intrinsically tied up with the task that the user is doing.
0:12:03.869,0:12:11.972
Similarly, what makes a good representation is also dependent on what a user’s expertise is.
0:12:11.972,0:12:18.696
And a wonderful example of this comes from Herb Simon and [Bill] Chase in the early 1970’s.
0:12:18.696,0:12:24.965
They were looking at chess as an exemplar domain for trying to understand expertise.
0:12:24.965,0:12:28.229
One of the things that they observed was that
0:12:28.229,0:12:34.817
expert chess players were much better at being able to remember the configuration of a chess board.[br]
0:12:34.817,0:12:38.580
You can imagine a couple hypotheses for this.
0:12:38.580,0:12:46.659
So, one of them would be “Experts were born with a better memory for that kind of thing”;
0:12:46.659,0:12:51.643
Or, similarly, “Experts by virtue of their ten thousand hours of training
0:12:51.643,0:12:56.656
have trained themselves up to build up that muscle and be very good at remembering that kind of thing.”
0:12:56.656,0:13:00.265
Neither turns out to be the case.
0:13:00.265,0:13:12.124
Experts are much better at remembering the configuration of a board, but only if it’s an actual game![br]
0:13:12.124,0:13:18.978
So if the configuration of the chess board is the configuration of how a chess board could be,
0:13:18.978,0:13:22.332
experts do a fantastic job of remembering it.
0:13:22.332,0:13:28.536
But if you arranged the pieces on the board in a way that a chess play could not ever achieve,
0:13:28.536,0:13:32.433
the experts actually do no better than novices at all.
0:13:32.433,0:13:41.286
And so what we’re seeing is that the ability of experts to chunk things and have higher memory capacity
0:13:41.286,0:13:46.597
is because they are able to leverage their knowledge about the domain.
0:13:46.597,0:13:51.547
Game design and user interface design are both concerned
0:13:51.547,0:13:56.035
with how easy or hard it’s for a user to accomplish a particular task.
0:13:56.035,0:14:00.524
The difference is that often designers want to make it hard, the right hard;[br]
0:14:00.524,0:14:04.325
and interface game designers want to make things easy.[br]
0:14:04.325,0:14:07.024
And so we can learn from this chess example
0:14:07.024,0:14:10.047
and we can ask this question as interface designers:
0:14:10.047,0:14:13.748
“Can we make interfaces more chunkable?”
0:14:13.748,0:14:17.807
Can we make interactions that can be accomplished in one chunk
0:14:17.807,0:14:23.665
and therefore place a lower load on our memory and make it easier for users to work with?
0:14:23.665,0:14:26.455
A great example of this comes from Bill Buxton
0:14:26.455,0:14:32.650
who looked at being able to move text between locations on a document.
0:14:32.650,0:14:37.055
And in a common desktop user interface today,
0:14:37.055,0:14:43.543
one common operation would be either a keyboard shortcut or a menu command to cut some text,
0:14:43.543,0:14:47.588
and then you move the cursor to a new location, and you paste that text.
0:14:47.588,0:14:54.479
That’s three different operations and, in between, if you got interrupted,
0:14:54.479,0:14:59.351
you might forget what’s in the copy buffer — in fact, I’m sure that’s happened to all of us at some point.
0:14:59.351,0:15:08.553
What Buxton realized is what if you could turn all of this, through a gestural interface, into one command?
0:15:08.553,0:15:16.572
So, maybe I could grab a text that I want, draw a new location and drop it right there.
0:15:16.572,0:15:22.652
That’d be much better: There’s never a time where I could be interrupted and lose state
0:15:22.652,0:15:28.092
because all of the state is maintained in this continuous gesture.
0:15:28.092,0:15:31.709
We’ve all heard the saying that a picture is worth ten thousand words.
0:15:31.709,0:15:39.801
As interface designers, we’re tasked with the project of representing the information to the user
0:15:39.801,0:15:44.831
and one task that we commonly have to deal with is:
0:15:44.831,0:15:50.518
Should we represent information visually, or should we represent information textually?
0:15:50.518,0:15:54.025
The answer of course is it depends.
0:15:54.025,0:15:59.909
But one time when representing information visually can be much more effective
0:15:59.909,0:16:06.026
is when you can convert slow reasoning tasks into fast perception tasks
0:16:06.026,0:16:09.950
by virtue of making them visually salient.
0:16:09.950,0:16:15.396
We saw that with the map example: In that case, both the colorings of the map were visual,[br]
0:16:15.396,0:16:21.482
but [in] one, it was much easier to just add a glance to understand what’s going on in good coloring.
0:16:21.482,0:16:27.095
And the poor coloring, you have to reason about it much more slowly and the wrong things kept popping out.
0:16:27.095,0:16:31.365
If you think about a table of numbers, it can often be difficult to see trends,
0:16:31.365,0:16:34.936
whereas if you represent that same information visually,
0:16:34.936,0:16:40.482
it can often pop out what the high points, the low points, trends, outliers —
0:16:40.482,9:59:59.000
all of that becomes salient and automatically visible to you.