The Machine That Changed the World; Interview with Alan Kay, 1990

I don't want to go back, the first computers were large expensive machines that had been built to do calculations. How did people think about this new technology? Well, I think that some people thought they were never going to get any smaller. There were calculations about how many Empire State buildings you needed full of vacuum tubes to do this and that. Other people at Bell Labs who had been working on the transistor had a sense that this could be used as a switch because AT&T wanted something to replace vacuum tubes and so I think there are several perceptions of how big the thing was going to be. I think it wasn't until the 60s that people started thinking that computers could get quite small. Part of the problem was the size. The other problem was just what it was supposed to be for this machine. Yeah, I think that when they were doing ballistic calculations, there's this notion that you would run a program for a very long time, it was really after the Unibrac in the 1950s that people started thinking, oh, we might have to do some programming and people most

notably Grace Hopper very early on started thinking about higher level programming languages. So that was sort of one flow and I think the other thing was the urgency of the Cold War got people thinking about air defense and so scientists at MIT started working on systems that could assess radar and show it to you on a display and those are the first computers that had displays and pointing devices. Now for the most part, those were kind of exceptions. Can you describe what the batch processing experience was like? What it would have been like? Well, I've always thought it was something like a factory. You have this very large thing and there are raw ingredients going in the front and there are some very complex processes that have to be coordinated and then sometime later you get what you're hoping for out of the rear end and you can think of them also as like a railroad, somebody else is deciding the schedules.

I can certainly remember having maybe one run a day or two runs a day and you can only get your results then and you have to work around and so forth. So it's very much of an institutional way of thinking about computing. Did this put many people off of computing? I don't think so. I think the happiness is how much expectations, how much the reality exceeds expectations. I think that most people were happy in the 50s there are still, in business, predominantly business computing was done with punch card machines and IBM was a company that made those punch card machines. So there's an enormous disparity between what you could do with computers and what most people did. But still for most people, the physical size and appearance of the machine was a compelling thing.

Well, I think my feeling is that anything that's larger than human scale invokes mechanisms concerned with religion. So you have a priesthood with white coats, all the paraphernalia over there and some people thought it would take over the world and some people wanted it to take over the world and some people were afraid it would take over the world and none of those things happen. But that was what it was when it was this large thing and towards the late 50s many people in different places started saying, well, this thing has the potential of being more like a partner, maybe a complimentary partner. It can do some things that we can't do well in vice versa. So we should find some way of bringing it down to human scale. But an apartment to the individual. Partner to people, not so much directly connected to the institutions. Although of course the first way it was done using time sharing, the mainframe was still all owned by the institution.

Okay, now, all the people of these very early visionaries, you've divided them into sort of two groups. How about Chip? Yeah. Could you talk a bit about that? Which two groups? Well, one group was the ones who were interested in amplifying intelligence. Yeah, there are lots of different groups. I'll rephrase that. Okay, so to move on. Okay, must be okay because we're still rolling. So well, I think in the late 50s there are a variety of different conceptions of what you should do with it and some of those conceptions are had by the very same person. So for instance, John McCarthy, who is a professor at MIT, both wrote memos suggesting we should time share the computer and he also thought more into the future that we'd be all networked together and there would be these huge information utilities that would be like our power and

lighting utilities that would give us the entire wealth of the knowledge of man. And he suggested on that that what we'd have to have is something like an intelligent agent and an entity, not maybe as smart as us, but an expert in finding things. And we would have to give it advice, he called it the advice taker. Tell me about time sharing because one obviously one thing people would say is how could everybody have their own computer, wouldn't they? Well, I think that was it. They cost several million dollars back then and people had been fooling around with things a little bit like time sharing before sort of the official invention of it at MIT. And part of it was just to debug programs. The idea is that debugging is one of the most painful processes, incidentally you probably know that there wasn't actually a bug. The first bug was one found by, it was a moth, I think, found by Grace Hopper. So it actually has an official and a friend of mine just recently was having trouble with

his laser printer and nothing that they thought of could possibly work and finally somebody decided to open it up and look at it and what it was was a mouse. The mouse had moved in, it was nice and warm, it had on top of the circuit board had set up various places, you know, so it was like a real, yeah, it was real and of course that nobody understood it meant when they said that this guy's computer has a mouse problem because my student had problems. So it's an ill wind that blows nobody good. Tell me about the basic concept of time sharing, what was the idea? Well the idea was that when humans use computers the best way of having them use it is instead of using all of the power for five minutes a day is to take those five minutes of computer power and spread it out over several hours because people are slow at typing and when

you're debugging you don't need to do long runs of things and what you want is lots of interaction, lots of finding out that you're wrong and this is wrong, you can make a little change and this is wrong. So you want to go from something that requires enormous planning to something that's much more incremental and this is the urge of the people who develop time sharing had. But how could you give an individual access, exclusive access? Well you couldn't, so the idea is if the thing costs several million dollars as McCarthy pointed out one of the things that you could do is roll in one person's job and do it for a couple of seconds then roll in another person's job and do it for a couple of seconds and if you had a fast enough disk for holding all of these jobs then you'd be able to handle 20, 30 or more people at once and they wouldn't feel that the lag of a few seconds was going to hurt them. So computers switching between the jobs so fast is that the way it works?

Right. Well that's the way it's supposed to work and of course the thing that drove personal computing in the 60s into existence was that it proved to be extremely difficult to get reliable response time. But this is one of the early ideas of the use of having an illusion? Yes, indeed. Right. In fact the best one of these was a system called Joss at Rand Corporation in which the system was devoted entirely to running this single language and that made things much better. The language was designed for end users. It was the first thing that affected people the way spreadsheets do today, it was designed had eight users on this 1950s machine but the guy who did it Cliff Shaw was a real artist and the feeling of using this system was unlike that of using anything else on a computer and people who used that system thought of wanting all of their computing to have the same kind of aesthetic warm feeling that you had when using Joss.

If you had a list, if you were a very clear thinking person back then in the late 50s who knew everything they wanted computing to be, what would they have with a small interactive real time? What would be the... Well I think in the 50s the emphasis, you know these things go in way, so the emphasis in the 50s was first on being able to do things incrementally. Second people wanted to share, so the idea of electronic mail was very early at MIT and at some point people started thinking that the form of interaction started to have a lot to do with how good you felt and how puzzled you were and so forth. The invention of computer graphics, modern computer graphics by Evan Sutherland in 1962 had a lot to do with people's perceptions because once you saw that you couldn't go back. It established a whole new way of thinking about computer interaction and to this day

it has remained an inspiration. Tell me about that piece of work. He was a graduate student. Yes, he had gone to Carnegie Tech, now Carnegie Mellon. And came to MIT to do his PhD, one of his advisors was Claude Shannon and another one was Marvin Minsky. As he has told the story several times is that they were then in just the early stages of Jack Licklider's dream to have the computer be a sort of a symbiotic partner and he went out to Lincoln Labs, people were starting to think about that and there was a marvelous machine there called the TX2, one of the last computers in the US large enough to have its own roof, one of these enormous machines originally built for the air defense system and I've been made friends with people there and started thinking about something having to do with computer graphics, the air defense system used displays for putting up radar

pilots and so forth and the light guns were already around and so forth and so we started thinking about maybe doing a drafting system and as I recall it one of his original reactions when seeing the kind of graphics you could put on a screen because the screens couldn't even draw lines, when they put up a line it was put up with lots of individual dots and done fairly slowly so it would flicker and it was pretty awful looking and I've been at that point said the best words a computer scientist can ever say which is what else can it do and so he got in fact having the display not be great helped what he tried to do on it because he started thinking of what was the actual power and value of the computer. Now today we have a problem because the displays are good so our tendency is to simulate paper but what I've been started thinking about is what could the computer, what kind of value could the computer bring to the transaction so it would be even worthwhile to sit down and

use such an awful looking display and the thing he came out with was that the computer could help complete the drawings so you could sketch, this is where the idea of sketch bet you could sketch something in you could sketch in if you're trying to make a flange well you just put in the lines for the flange and then you would tell sketch pad to make all of these angles right angles and you could make these two things proportional to each other and so forth and sketch pad would solve the problem and straighten the house to drawing in front of your eyes and something that was more like what you wanted so that was a terrific idea and then he took it another step further because he realized he could solve real world problems so if you put a bridge into sketch pad it had never been told about bridges before but you could put bridges in and tell sketch pad about pulling and pushing of things and hang a weight on the bridge and sketch pad would generate the stresses and strains on the bridge so it was now acting as a simulation you could put an electric

circuit sketch pad never heard about electric circuits before but you could put in ohms law and what batteries do and it would in order to settle the constraints or one of the nicest things was you could put in mechanical linkages so you could do something like a reciprocating arm like on a locomotive wheel for going from reciprocating to circular motion and sketch pads problem solver if it had a problem that it couldn't get an exact solution for there wasn't just one solution of course there isn't for this what it would do is iterate through all the solutions so it actually animate this thing and right on the screen you'd see this thing animating was the very thing that you were thinking of 62 yeah well you can't buy a system today that does all the things that sketch pad could back that's what's really made it had first system that had a window first system that had icons certainly the first system to do all of its interaction through the through the display itself and

for a small number of people in this community the advanced research projects agency research community this system was like seeing a glimpse of heaven because it had all of the kinds of things that the computer seemed to promise in the fifties and practically everything that was done in the sixties in that community and into the seventies had this enormous shadow cast by sketch pad that or you could maybe think of it better as a light it was sort of showing us the way and that's one thing to have another glimpse of heaven I suppose is the work of Doug Engelbart yes and he was also funded by the advanced research projects agency and his original proposal I think was also in 1962 1962 is one of those amazing years and Engelbart had read about the Memex device that Vannevar Bisch who is president

Roosevelt science advisor and former professor at MIT he had read an article in 1945 called as we may think and most of the article is devoted to predictions of the future and one of them was he said that sometime in the future we'll have in our home a thing like a desk and inside the desk on optical storage will be the contents of a small talent library like five thousand books there'll be multiple screens there'll be pointing devices and ways of getting information and and pointing to things that are there and he said that you can form trails that would connect one piece of information to another he invented a a profession called pathfinding that there be people called pathfinders who sold paths you could buy a path that would connect some set of interesting things to some other set of interesting things and this is a complete vision in 1945 and a lot of people read that

I read it in the 50s when I was a teenager because I had seen it referred to in the science fiction story Engelbart had read it fairly early when he was in military service and once you once you read that thing you couldn't get out of your mind because it was the thing that anybody who deals with knowledge would desperately like to have and so Engelbart in the early 60s started writing proposals and he finally got ARPA interested in funding it and they started building a proposal of his and a couple of years later 1964 invented the mouse to have a both a better pointing device than the light pen and a cheaper one and they built a system that by 1968 was able to give a very large scale demonstration to three thousand people in San Francisco I was there but I had I had seen the system

beforehand because of course I was a graduate student in this community but still even having seen the system the scale of the demo and the impact it had is unbelievable I remember it started off it was about three thousand people in this auditorium is it one of the I was at the fall joint computer conference I think and all you could see on the stage was this figure with something in his lap and a box in front of him and a couple of things that look like TV cameras around him and he had on a headset and he started talking he said I like to welcome you into our demonstration all of a sudden his face appeared twenty by thirty feet wide on this enormous screen because they borrowed one of the NASA one of the first video projectors on this and they used this huge situation display then they used video editing so you could see while he was giving this demonstration what he was

doing with his hands with the mouse and the key set what was going on and the screen and so forth and that that is videotape to me that's something that you can use for the year so you don't know what did he what did he demonstrate what sort of things did he show Douglas Douglas Engelbart he started off just showing us how you could point to things on the screen and indicate them and started off actually fairly fairly simply just showing how you could look at information various ways he showed something very much like hypercard so he had a little map of how he was going to home that night he was going to go to the library and the grocery store and the drug store and so forth he could click on each one of those things and it would pop up and show him what he had to get there and what he demonstrated were the early stages of what we call hypertext today lots of response in the system one of the big shockers was midway through the thing he started doing

some collaborative work and all of a sudden you saw the in an insert on the screen you saw the picture of Bill Paxton who was 45 miles away down the Menlo Park live both of them had their mouse pointers on the screen and they were actually doing the kinds of things that people still dream about today so this is a complete vision and I think of it as the vision of what we today call personal computing or desktop computing except for the fact that the processor was a big time shared computer all of the parafilm Engelbart used a black and white display 19 inch display using video to blow up a calligraphic screen and had a mouse and if you look at the thing today you'd see something that looked like somebody's office that you could walk into and what remained was to do something about the problems in response time and all of that stuff and that was something that I had

gotten interested in a few years before and the first machine that I did was a thing called the Flex Machine and yeah what was the reaction like among the computer? Well I didn't take a, in fact I was actually, as I recall I actually was rather, I had the flu or something I was determined to go see this because Arpa had spent something like 175,000 on this demo and everybody in the Arpa community wanted to show up. I recall that the crowd you know he got a standing ovation and won the best paper in the fall joint computing conference and so forth and what was even better is that he had bought up four or five terminals to this system and had them in a room and people could go in and actually learn to interact with the system a bit so it was a large scale demonstration. I don't think that anybody has ever traced what people did over the next 15 or 20 years of the result of having been at that demonstration.

That would be interesting. Doug sort of hoped it would change the face of computing and it's very disappointed that he thinks it didn't really have much of it. Well I mean there are a couple of things that Doug, I mean we thought of Doug as Moses opening the Red Sea you know he was like a biblical prophet and like a biblical prophet he believed very much in his own particular vision. That vision was not a 100% good idea. One of the things that they neglected completely was the learnability of the system. So the people who use this system were all computer experts who loved it were willing to memorize hundreds of commands in order to be, if you memorize hundreds of commands and you learn how to use the key set, you could fly through this indimensional space. It was quite exhilarating to watch people do it. It was exhilarating to learn how to do it. The problem with it though is that there are so many barriers towards learning and there

are many other things that were problems. It wasn't a particularly good simulation of paper partly because he didn't want it to be. And so the idea that there would be a transition period where you would be producing documents, of course they printed documents out, but there was no notion of desktop publishing there. So the whole system was rather like a violin and if you were willing to learn how to become the violinist you could play incredible music through the thing. And of course all of us were completely sold on this system in the late 60s and the first machine that I did, the FlexMachine, was an echo of this. FlexMachine, the other element which is in this apart from these fantastic software achievements of Sutherland and Engelberg is of course the size of the machine. Now there was some precedent when that came out, yes. The first personal computer in my opinion was the machine called the link.

If you include size as one of the important things, of course you could say that the world win was a personal computer at MIT or the TX2. Some people tried to get Ivan Sutherland to write a paper called when there was only one personal computer and that was him using Sketchpad on the TX2 which is this thing bigger than a house. But in 1962 West Clark did a machine in which part of the design parameter was it was small enough to look over when you were sitting down at a desk. So it was not supposed to flume over you. So some of you could actually see over it and many important things were done on that machine. In fact quite a few hundred if not a few thousand of them were built and used in the biomedical community. It was designed for biomedical research, designed to be programmed by its users who are not computer scientists, even designed to be built by non-computer science. They used to have gatherings in the summertime where 30 people or so would come and build

their own links and then take them back and stuff. It was a great little machine. Had a little display and display editors and so forth. So it was something that you could point to when you were trying to think of what to do. And there are other small things. There was a machine called the IBM 1130 which was really an abortion of a machine. It was sort of a key punch keyboard hooked to one of the early removable disk packs and I mean this was a mess of a machine. But it was the size of a desk and you could sit down. It wasn't designed really to be programmed except by storing programs on a very funny. You can only store data on the disk and IBM forced you to put the programs on a bunch of cards. That's the only way you could feed them in as really hilarious. So there are lots of different kinds of things like that. So what were you trying to do with Flex?

What was the? Well I worked on this machine with a guy with a name of Ed Sheedall who was he was trying to do really trying to invent what today we call personal computing. He had a little machine and he had a little Sony television set and what he wanted was something for engineers that would allow him to flex, he was an engineer and he wanted to allow them to flexibly do calculations beyond the kinds of things that you do with a calculator. So you should be able to program the machine and something. You should be able to store the program as a way and you should be able to get it to do things. Then I sort of came in and corrupted the design by wanting it to be for people other than engineers. So I'd seen Joss and I'd also recently seen one of the first object-oriented programming languages and I realized how important that could be and then Sheel and I escalated. The problem is that he and I got along really well and so we escalated this design beyond

the means of the time to build it practically. But we did build one and it had many things that people associate today, it fit on top of a desk, special desks, the weighed hundreds of pounds. But it had a fan likened to a 747 taking off because the integrated circuits back there had maybe eight or 16 gates on a chip so this thing had about 700 chips in it and had a high resolution calligraphic display and had a tablet on it. And it had a user interface that included multiple windows, things like icons and stuff. But it was rather like trying to assemble a meal, maybe make an apple pie for random things and you find in the kitchen like no flour so you grind up Cheerios, you know. We wind up with this thing that looks sort of like an apple pie but it actually isn't

very palatable. So the result of this machine was a technological success and a sociological disaster and it was the magnitude of the rejection by non-computer people we tried it on that got me thinking about user interface for the first time and I realized that what Cliff Shaw had done in Joss was not a luxury but a necessity and so it led to other ways of looking at things. So we go back to originally you saying people sort of mainframes as like factories, right? These early attempts at personal computers and what like model teases. Yeah, I think Engelbart, I think one of the ways that we commonly thought about Engelbart stuff is that he was trying to be Henry Ford, that you could think of the computer as a railroad and the liberating thing for railroad is a personal automobile. And Engelbart thought of what you were doing on his system as traveling so you're moving

around from link to link in the hyperspace and he used terms like thought vectors and concept space and stuff. Nobody knew what it meant and I'm not sure he did either but it was that kind of a metaphor. What's wrong with that metaphor? Well I don't think there's anything particularly wrong with it but when you're doing things by analogy you always want to try and pick the right analogy because there are so many ways of making them and the thing that was limiting about it when you apply it to humanity as an example is a car you expect to take a few months to learn how to do it that was certainly true, it's something that doesn't extend into the world of the child, there's a whole bunch of things but of course we didn't think of it that way, we thought of the car was one of the great things of 20th century and it changed our society and everything and so we were definitely using that as a metaphor.

And in fact the thing that changed my mind had nothing to do with rejecting the car as a metaphor was finding a better metaphor that one that was completely possessed me and that came about from seeing a quite a different system. I had called the Flex Machina Personal Computer, I think that was the first use of that term and while I was trying to figure out what was wrong with it I happened to visit Rand Corporation over here in Santa Monica and saw sort of a follow on system to Joss that they had done for their end users who were people like Rand economists. These people love Joss but they hated to type and so in the same year the mouse was invented, the Rand people had invented the first really good tablet, it was a high resolution thing and they decided that the thing to do was to get rid of keyboards entirely and so the first really good hand character recognizer was developed there and they built an entire

system out of it called Grail for graphical input language so there's no keyboard at all. You interacted directly with the things on the screen, you can move them around, you did a square on the screen, if you drew a square it would recognize you're trying to draw a square it would make one, if you put in your hand printed characters it would recognize them and straighten them up and the system was designed for building simulations of the kinds that economists and other scientists would like to build and using this system was completely different from using the Engelbart system and this system it felt like you're syncing your fingers right through the glass of the display and touching the information structures directly inside and if what Engelbart was doing was the dawn of personal computing what the Rand people were doing was the dawn of intimate computing. At intimate computing you forget that it's a machine and you think of it as a machine. One of the things that completely took hold of me in using the Grail system was it felt more like a musical instrument than anything because the musical instrument is something

that most musicians don't think of their instruments as machines and it's that closeness of contact, the fitness that you're directly dealing with content rather than the form of the content that possessed me very strongly and I saw that was in 1968 as well and I saw another several things I saw see more papperts early work with logo where here were children writing programs and that happened because they had taken great care to try and combine the power of the computer with an easy to use language in fact they used the ram joss as a model and used the power of lisp which had been developed a few years before as an artificial intelligence language put them together and that was the early logo and to see children confidently programming just blew out the whole notion of the automobile metaphor and the thing that replaced it was that this is a medium this is like pencil

and paper we can't wait until the kids are seniors in high school to give them drivers the ed on the thing they have to start using it practically from birth the way they use pencil and paper and it was destined not to be packaged on the desktop because we don't carry our desk with it it had to be something much smaller and that was when I first started seriously thinking about a notebook computer and of course the the first thing I wanted to know after deciding that had to be no larger than this was when would that be possible if other and so I started looking at what the integrated circuit people were doing Gordon Moore and Bob noise and stuff and there are these nice projections that they as confident physicists have made about where silicon can go and what's wonderful is these projections have only been off by a few percent now more than 20 years later and those and of course I was very enthusiastic I would believe anything that was in the right direction so I took

this hook line and sinker and said okay 20 years from now we'll be able to have a notebook size computer that we can not only do all the wonderful things on computers we can do mundane things too because that's what paper is all about you can write it a shake experience on it on it but you can also put your grocery list on it so one of the questions I asked back in 1968 is what kind of a computer would it have to be for you to do something so mundane as to put your grocery list on it be willing to carry it into a supermarket and be willing to carry it out with a couple of bags of groceries there's nothing special about that you can do it with paper so the the question is is see the question is not whether you replace paper or not the question is whether you can cover the old medium with the new and then you have all these marvelous things that you can do with the new medium that you can't do with the old. Now many can find the idea of a medium tricky concept what in the sense you're talking about writing and so forth or music what do you mean?

Well I think most of my thoughts about media were shaped by reading McClune which not everybody agrees with but one of his points is that the notion of intermediary is something that is not added on to humans it's sort of what we are we deal with the world through intermediaries we deal with our we can't fit the world into our brain we don't have a one-to-one representation of the world in our brain we have something that is an abstraction from it and so we the very representations that our mentality uses to deal with the world is an intermediary so we kind of live in a waking hallucination and we have languages and intermediary we have closes and intermediary so this whole notion of what we think of as technology could also be replaced by the word medium and I think there's even though media has this connotation of you know the news business and stuff like

that I think it's an awfully good world because it where because it gives us this notion of something being between us and direct experience now both as media writing and printing obviously as a media we all think about the great power music as well yeah and what does that maybe consist well I think that you know the trade off with with using any kind of intermediary is that every time you put something between you and direct experience you're alienating a bit and what you hope to get back from it is some kind of amplification various people who have studied evolution talk about the difference between the direct experience you get from kinesthetically touching something to the indirect experience you get of seeing it and one is less involving than the other the seeing is a good one because it means you don't have to test out every cliff by walking over it and so there's an enormous survival

value about stepping back to have a brain that can plan and envision things that might happen whether as images or even more strongly in terms of symbols is of tremendous survival value because it allows you to try out many more alternatives and so as the existentialists of this century have pointed out that the we have gained our power over the world by at the cost of alienating ourselves so the thing that we computer is anything but a machine you know so I wonder why should that be a medium well I think well I think machine is a word that has a connotation that's unfortunate and you know maybe what we should do is either elevate what we think of when we say machine you know mechanism kind of thing or maybe use

a different word to most scientists machines are not a dirty word because we one of the ways we think about things even in biology these days is that what we are is an immensely complicated mechanism that's not to say that it's predictable in a sense of free will because there's a notion now that people are familiar with called chaotic systems and systems that are somewhat unstable and there's a lot of instability and stuff built in it's a little more stochastic in a way but the idea that there are things operating against each other and they make larger things and those things are part of larger things and so forth to most scientists seems like a beautiful thing not it's not a derogatory term. I think it's also that most machines were built to do useful work and this machine in

processing information has properties which is sort of linguistic as well as yeah well I think the connotation of machine is something that has to do with the physical world. Most people most people who haven't learned mathematics have never encountered the notion of an algorithm which is a process for producing and you could think of you're moving around physical things called symbols I mean something physical is always being moved around it's the way scientists look at the world and it's the real question is is this a trivial example of it or is this an unbelievable interesting example of it so a flower is a very interesting example of a machine and a lever is somewhat trivial and more understandable notion of a machine. Now these machines can move around symbols presumably like the electronic voltages or whatever so fast that they can dynamically simulate virtually a lot of things.

Lots of things yeah one way to think of it is the a lot of the what a computer is is markings and just as you doesn't matter whether you make the markings for a book on metal or paper or you can make them in clay all kinds of ways of making the equivalent of a book the main thing about a marking is you have to have some way of distinguishing from other markings and once you've got that then they can act as carriers of different kinds of descriptions and the range of the markings that you can have in a book and the range of markings you can have in a computer are the same. So like in the computer it's talking with punch cards that doesn't matter what it is you can make there's a wonderful computer build out of tinkertoy in the computer museum of Boston is done by some MIT grad students you can make them out of anything and so that's one set of ideas that it's the representational properties of things are very very much

like a book and then the computer has this even more wonderful thing is that it's a book that can read and write itself and moreover it can do it very quickly so it has this self reflexivity that you usually only find in biological systems and it can carry through these descriptions very very rapidly and then you get everything that's the big nirvana experience where you suddenly realize holy smokes this thing is a pocket universe and it has a nice complimentary role to the way we deal with our physical universe as scientists the physical universe especially in the 19th century it was thought that it was put there by God and it was the job of scientists to uncover this glorious mechanism so the computer what you do is you start off with a theory and the computer will bring that theory to

life so you can have a theory of a universe that is different than our own like it has an inverse cube law of gravity and you can sit down and not too much time you can program up a simulation of that and discover right away that you don't get orbits anymore with planets and that's curious and a little more thinking and deeper delving will soon make you discover that only the inverse square law will actually give you orbits the other way you put it once is you could invent a universe with no gravity right yes and in fact the I think it's again the difference between a physical book which when you can drop it and it will fall down to the floor and the story that's in the book that could be about a place that has no gravity like out in space and the computer is at once a physical thing and the stories that it holds don't have to have anything to do with a physical universe its components reside in so in other words you can lie both with books and

with computers and it's important because if you couldn't lie you couldn't get into the future because that we are living a world of things that were complete lies 500 years ago now the thing about the computer as well it's apart from being dynamic it's a bad birdies simulate means that it's can embody all other media it's the it is the first matter yes yes I called it the first methamedium in the in the early 70s when I was trying to explain what it was as being distinct from big main frames with whirling tape drives that the physical media that we have like paper and the way it takes on markings are fairly easy to simulate the higher the resolution display you get the more you can make things look like a high resolution television camera looking at paper that was one of the things that we were interested in at at Xerox Park was to do that kind of simulation and they're

deeper simulations than the simulation of physical media there's the simulation of physical processes as well now if then this is a pretty mind-blowing because it's the computer then is not just important as a versatile tool but a new machine and what we're talking about is something of the of the gravity of the invention of writing or printing in terms of terms we're talking about something really pretty fantastic right and it's sort of sneaking its way in the book did the the Catholic Church didn't I think the Catholic Church thought that the the book might be a way of getting more viables written in Latin and so at the time they started thinking about suppressing it was too late because of course the you could print all these other things and all of a sudden you go from having three or four hundred books in the Vatican Library in the year 1400 to something a hundred or a hundred

and fifty years later where there are some forty thousand different books in circulation in Europe and eighty percent of the population could read them and so all of a sudden you have something totally at odds with most ways of dealing with religion which is multiple points of view what can we learn looking at the the history of the book really what can we learn clearly took a lot longer to happen right for one thing but what were the stages it had to go through yeah well I think in doing in doing these analogies analogies can be suggestive you know and of course the the the amount that analogy carries through from one place to another depends on on lots of things I happen to like the analogy to the book quite a bit because you have several stage you have the invention of writing which is a huge idea incredibly important and the difference between having it and not

having it is enormous the difference between having computers even if they're big just big main frames and not having them as enormous they're just things you cannot do without them then the next stage was the Gutenberg stage and Gutenberg just as McLuhan like to say when a new medium comes along it imitates the old so Gutenberg's books were the same size as these beautiful illuminated manuscripts that were done by hand and big like this and in the libraries of the day back then the books generally weren't shelved there so few books in a given library that they were actually had their own reading table and if you look at woodcuts of the day it looks for all the world like a time-sharing bullpen you know you go into a lot there's one a library in Florence it set up that way so you go over to the table that the particular book is at they were chained in because they're too valuable they're priceless and Gutenberg imitated that but of course he could produce many more

books but he didn't know what size they should be and it was some decades later that Aldous Manuchius who was a Venetian publisher decided that books should be this size and he decided they should be this size because that was a size that saddlebags were in Venice in the late 1400s and the key idea that he had was that books could now be lost and because books could now be lost they could now be taken with you and they couldn't be like this they had to be something that was a portable size and I think in very important respects where we are today is before Aldous because this notion that a computer can be lost is not one that we like yet you know we protect our computers we bolt them to the desk and so forth but they're still quite still quite expensive they'll be really valuable when we can lose them the point about the point of literacy I mean is also well taken there while there

are very well there are no books to read I assume there's no point in right well if reading and writing is a profession you don't have the notion of literacy and illiteracy like there's no word in our language that stands for ill medicine right there's medicine which is practiced by professionals and there's no notion of ill medicine now if we all learned medicine that we're part of and it may be someday that staying healthy might be something that becomes an important part of everyone's life then there'll be a notion of medicine and ill medicine and so it wasn't until the there was a potential for the masses to read that you had the notion of literacy in ill literacy and to me the literacy has three major parts that you have to contend with one is you have to have a skill to access material prepared for you by somebody else regardless of what it is that you're you're trying to be literate in in print medium it's called reading reading accessing and I don't think many

people in our society would think that a person who just could read was literate because you have to be able to create ideas and put them into the information strain so you have creative literacy which is the equivalent of writing and then finally after a medium has been around for a while you have the problem that there are different genre of these things so that you have to go into a different gear to read a Shakespearean play now is a different way of doing it the way they wrote essays 200 years ago is different than the way they write essays today and so you have to have some set some sort of literacy in the different print styles for representing things and so I think those three things when when something new comes along since we have we're talking about media we can talk about media see and ill media see or computer is computer is see and ill computer is see or something like that and I think each one of those things is a problem that has to be solved is the problem

of how can you when you get something new made for a computer be able to do the equivalent of reading of access what it is you go to a computer store and get some new wonderful thing slap it into your computer and if you have to learn a new language in order to use the thing then things are not set up for you being computer literate so rocking and relaxing with the analog just yeah I think it has a lot to do with that and again to sort of strain the analogy we're making with with printing my other hero back then was Martin Luther who a few years later had this intense desire to let people have a personal relationship with God without going through the intermediaries of the of the church and he wanted people to be able to read the Bible in their own language and knew that German which was a bunch of different regional dialects back then mostly for talking about farming

and and stuff was not going to be able to carry the weight of the Bible as it was portrayed in Latin which is a much more robust and mature language so Martin Luther had to invent the language we today call high German before he could translate the Bible into it and so the language of the German speak today is is in a very large part the invention of a single person who did it as a user interface so that he could bring the medium much closer to the people rather than trying to get everybody in Germany to learn Latin now with that sort of insight we can think of this whole process literally took a long time to many centuries and we can as you say the early phases of the computer have gone quite much more rapidly haven't they? Right. Well I think we're part you know things are going more rapidly but also to the extent that analogies help you know certainly that progression I happen to know be a big fan

of books and I happen to know that progression of illuminated manuscripts which I absolutely adored when I was a child the Gutenberg story the eldest story is one of my great heroes because he was the guy who decided you had to read more than one book and then Martin Luther those immediately sprang to mine as soon as I had decided that the computer wasn't a car anymore and then the thing becomes the two most powerful things that you have to do right away you don't have to wait to gradually figure them out you have to get the thing small and you have to find a language that will allow people that will bring what it is closer to the people rather than the other way around. Now an opportunity came for you I know you were going to make your dining book one way or another but an opportunity came for you to go to a brand new center. Yeah I thought up the dining book in 1968 and made a cardboard model of it and a couple

of years later I was about to go to Carnegie Mellon University to do the thing and I had been consulting a little bit for the newly formed Xerox Park and it just got to be too much fun so I never wound up going to Pittsburgh. What was Xerox Park? Xerox Park was an attempt by Xerox to spend part of its research and development money on really far out stuff that was not going to be particularly controlled by that. In the late sixties they bought themselves a computer company they bought a whole bunch of publishers. They're trying to get into a larger sphere than just office coffeeers the one of the ringing phrases of the late sixties was Xerox should be the architects of information. They had this expansive feeling about getting into other areas and they got Jack Goldman in who was a scientist at Ford and as the new chief scientist is Xerox and Jack realized

that what we actually had to have at Xerox was something like a long range research center Xerox was a big company even then and what Jack decided to do is to hire George Pake who was the chancellor of Washington University in St. Louis and George was quite familiar with ARPA and so he hired Bob Taylor who had been one of the funders of ARPA in the sixties. Taylor was one of the champions of Engelbart, one of the champions of the work at Rand and so forth. What Taylor did was to instead of hiring random good scientists what he did was to decide to set up a miniature concentrated ARPA community so he went out after all of the people he thought were the best in the ARPA research community and those people were somewhat interested because the Mansfield amendment was shutting down, constricting the kinds of far out things

that ARPA could do so it was one of those things where the Mansfield amendment was an over reaction to the Vietnam War a lot of people were worried about various kinds of government and especially military funding on campus and secret funding and all of this stuff and the ARPA information processing techniques projects got swept up in this even though every single one of them was public domain and it had already been spun off into dozens of companies including Deck and so this is just one of these broad brush things done by Congress that happened to kill off one of the best things that's ever been set up in this country and so a lot of smart people in universities who believe more or less the same dream about the destiny of computers were gathered up by Taylor and I was one of those.

So how good would you be talking about the people that were gathered, I mean how good were they? Well I thought they were the best. Okay. I thought the people that were there were among the very best in my personal pick by 1976 which is about six years after the place it started their 58 out of my personal pick of the top 100 in the world there and divided into a couple of labs people who had very strong opinions who normally would not work with each other at all in any normal situation were welded together by having known each other in the past having come up through the ARPA community believing more or less the same things of course when you have that kind of level of people the disagreements in the in the 10th decimal place become significant

but it worked out very well because we were able to get together completely on joint hardware projects so we built all of our own hardware as well as all of our own software there and the hardware was shared amongst the many different kinds of groups that were there. So there's one set of hardware as we moved on from year to year and then many different kinds of software design. Zero, have this idea of architecture of information rather general concept what did you guys think you were doing? Well the psychology of that kind of person is would be thought of as being arrogant in most situations we thought we knew we didn't know exactly what we're going to do but we're quite sure that nobody else could do it better whatever it was and the architect of information that was a phrase that sounded good it didn't actually mean anything and there was some

mumbling also about if paper goes away in the office in the next 25 years there should be the company to do it because that was their heartland business but Taylor had was setting the place up to realize the the arpa dream which was to do man computer symbiosis and in all of the different directions that arpa have been interested in the past which included interactive computing which included artificial intelligence in the notion of agents. So there are projects that range from being intensely practical ones like Gary Stark whether doing the first laser printer and Metcalf and other people doing the ethernet first packet switching local area net, Thacker being the main designer of the alto which is the first workstation and the first computer to look like the Macintosh. I was head of a group that was interested in the dinobook design and a lot of the things that we came up with for the dinobook were simulated on the alto and became part of the

first notion of what workstations and personal computers should be like and those included things like the icons and the multiple overlapping windows and and a whole the whole paraphernalia that you see on machines like the Macintosh today. And the basic idea was the sort of machine you were interested in was one which had the capacity to handle rich graphical interfaces but could be networked together. So we had to have more stand-alone power than time-sharing etc. Yeah in fact one of the things we're incredibly interested in was how powerful that it had to be. We had we used phrases like we didn't know how powerful it had to be but it had to be able to handle something like 90 or 95% of your the things that you would generally like to do before you wanted to go to something more because we didn't know what that meant. We thought it would be somewhere around 10 mips, sort of the rule of thumb.

The machine we built was effectively somewhere between one and three mips depending on how you measured it and so it was less powerful than we thought. On the other hand it was about 50 times as powerful as a time-sharing terminal. So the magnification of what we could aspire to was enormous and the machine was not done blindly. It was done after about a year of studying things like fonts and what happens when the eye looks at little dots on screens and what animation means and we did a bunch of real-time music, very early synthesis kinds of stuff. So we knew a lot about what the machine should be and how fast it should run. So to design this machine and the software, were you studied human beings? Yeah, a lot of us did. What was nice here is I was one of the drivers I wanted these small machines.

There are other groups at this time in 1971 when Park was forming who were building a large time-sharing system to act as a kind of an institutional resource for Park. There are lots of different things going on. I wanted to do stuff with children because my feeling after doing the Flex machine was that we and I, especially, were not for a good at designing systems for adults. So then I thought, well, children have to use this thing. Well, forget about adults for a while, we'll just design a tool for children, a medium for children, and we'll see what happens. So I desperately wanted to get a bunch of these systems, I needed like 15 of them, I needed to have them in a school somehow with children because I wanted to see how children reacted to these things and improve the design that way.

Yeah. We'll keep going. So the idea was that when you use children, it's through into focus and many of the problems you were trying to set what was there. Yeah, I don't know that through into the shock when you go out to real users of any kind is enormous because technical people live in this tiny little world, actually. We like to think it's a big world, it's actually a tiny little world and it's full of phrases that we learn when we're taking math classes and it's hermetic and it's full of people who like to learn complicated things, they delight in it. And so one of the hardest thing is for these kind of people to do any kind of user interface for people who are not like that at all. That's the number one victim of user interface design is the users are not like us. And so what you need to have is some way of constantly shocking yourself into realizing that the users are not like us and children do it really well because they don't care

about the same kinds of things that adults do and they're not like this or not like that. They can always go out and play ball. They haven't learned to feel guilty about not working yet. And the last thing you want to do is subject the children to more school. You don't want to put them through a training course to learn a computing system. So we use the term forcing function, it was a set up where in many ways it's harder to design a system for children it is for adults, along a couple of avenues and it forced us to start thinking about how human mentalities might work. These were your small talk kids? These eventually became the small talk kids and the lead into that was that Seymour Pappert had gotten a lot of good ideas from having worked with Piaget and Piaget had a notion of the evolution of mentality in early childhood as being sort of like a caterpillar into a butterfly.

And the most important idea of it was that at each stage of development that the kid was not a deficient adult that was a fully functioning organism, just didn't think about the world quite the same way as adults did. And other psychologists like Jerome Brunner had similar ideas also inspired by Piaget. What were these basic changes for? Well, if you break them down into just three stages it's one in which a lot of thinking is done just by grabbing a hole is to dig it and object is to grab it. And then the next stage is very visual in which many of the judgments that you make about the world are dominated by the way things look. So in this stage you have the Piaget water pouring experiment, you know you pour from a squat glass into a tall thin one and the kid in this stage says there's more water in the tall thin one. He doesn't have an underlying notion, a symbolic notion that quantities are conserved, doesn't have what Piaget calls conservation.

And the conservation is not a visual idea, visual it's what does it look like when you compare it. And then later on you get a the stage of Piaget called the symbolic stage where facts and logic start to dominate the ways of thinking and a child in that stage knows as a fact that water doesn't appear and disappear or nor does anything else. And so it's able to give you judgments that are inferences based on these facts rather than by the way things appear. Now Brunner did a lot of these experiments over again and added some sort of side trips into them. As he did in the water pouring experiment was to interspers a cardboard after the child that said there's more water in the tall thin glass and the kid immediately changed what he had just said he said oh no there can't be more because where would it come from. And Brunner took the card away and the kid changed back he said no look there is more. Brunner put the card back and the kid said no no there can't be more where would it come

from. So if you have any six year olds you like the torment, you know this is a pretty good way of doing it and Brunner, the way Brunner interpreted this is that we actually have separate ways of knowing about the world. One of them is kinesthetic, one of them is visual, one of them is symbolic of course we have other ways as well. And these ways are in parallel they really constitute separate mentalities and because of their evolutionary history they are both rather distinct in the way that they deal with the world and they are not terribly well integrated with each other. So you can get some remarkable results by telling people in English sentence which they'll accept and then showing them a picture that represents exactly the same thing and have an enormous emotional reaction. How is this going to help in face design? Well you know we're clutching its draws, you know you do anything when you don't know what to do but it seemed like a reasonable thing to since we're going to be dealing

with humans, it seemed like a reasonable thing to try and get something out of the newly form cognitive psychology to do it and the thing that got me, the PGA stuff didn't help me much because one of the ways of interpreting it is that for example you shouldn't probably teach first graders arithmetic because they aren't quite set up to receive it yet you'd be better off teaching them something that's more kinesthetic like geometry or topology. They're just, they're learning, they're tying their shoes and stuff and they're a whole bunch of things you can do with knots and things that are higher mathematics but the particular symbolic manipulations of arithmetic are not easy for them at that stage. Well that's important when you're designing curriculum and schools but it didn't tell us much about a user interface. The burner thing though was really exciting because one of the ways of interpreting it is that at every age in a person's life there are these multiple mentalities there.

Now when the kid is six years old the symbolic one may not be functioning very much but there are tricks that you can get it to function and burner had this notion that you could take an idea that's normally only taken taught symbolically like some physical principle and teach it to a six-year-old kinesthetically and to a 10-year-old visually and later on to a 15-year-old you could teach it symbolically it's notion of a spiral and Pappert had the same idea that if you want to learn how gyroscope works the last thing you want to do is look at the equation what you want to do is do a little bit of visual reasoning about the thing and then have one in your hand in the form of a big bicycle wheel and when you turn that bicycle wheel and feel what it does it clicks together with a small amount of reasoning you're doing and all of a sudden you understand why it has to flop the other way but there's

nothing that mysterious about it but from the outside and looking at the equation and thinking about cross-products and stuff like that it just doesn't make any sense so burner's idea was when you don't know what else to do in developing a curriculum try recapitulating the Piagetian stages cast this adult level knowledge down into a kinesthetic form and recast it later on teach it over and over again recast it later on in a visual form and he thought of the these lower mentalities kinesthetic and iconic mentalities as being what we call intuition but they can't talk much but they have their own figurative ways of knowing about the world that supply a nice warm way of thinking about physical problems. But not a good one it's okay but if you only use it to point on the screen it's not very it's not very kinesthetic because the major operation in in kinesthesia is grabbing

something and moving it and Engelbart did not have that and that was one of the first now of course people have been moving things around in computer graphics and most people would have ever done this liked it better so the first little paper I wrote about the dining book talked about moving things around and grail move things around sketchpad moves things around it's a natural thing when you're doing when you're thinking about things from a graphic center which Engelbart wasn't even though he had a little bit of graphics he was thinking about it from text think about things from a graphic center it's natural to want to just move something around and so the connection with what had already been done with what Brenner was saying got it's thinking a little bit more strongly about what the what the mouse was supposed to be about the mouse was not just for indicating things on the screen it was to give you a kinesthetic entry into this world now the important thing about it is that the I don't know how all scientific discoveries are made they

usually are connected together with little flashes of insight over a line periods of time but in fact the we're when I started off I was quite sure that Brenner I had the key and we got the ingredients rather early for doing the user interface but it is remarkable to me how many people we had to try ideas out on for years three or four years we had to not only build the system but in fact the early versions of the system were not as good as later versions of the system it actually took about five years and a hundred dealing with hundreds of people to come out with the first thing that was sort of looked like the Mac and this is like the desktop this is yeah the desktop is action idea we didn't like particularly that was something that was developed as a when Xerox was thinking specifically of going at offices yeah this was the high on windows yeah and the in the in the dinerbook

idea so when you have a diner book and you have simulations and multidimensional things as your major way of storing knowledge the last thing you want to do is print any of it out right because you're destroying the multidimensionality and you can't run them so if you have any so one of the rules is if you have anything interesting in your computer you don't want to print it and that's why if your computer is always with you you don't want you won't print it when you give it to somebody else what you're sending to them is something that was in your diner book to their diner book okay and the user interface that we worked on was very much aimed at that and so it had a different notion than then the desktop is sort of a smaller version of it what we had was this you can think of it as multiple desks if you want but we had this notion that people wanted to work on projects and I especially like to work on multiple

projects and people used to accuse me of abandoning my desk you know and get piled up and I go to another one and pile it up and when I'm working on multiple things I like to have a table full of all a crap that goes there and another table and so forth so I can everything is in the state it was when I last left it and that was the user interface that we wound up doing where you could move from one project to another project and you could think of it on the if you did it on the Macintosh you could think of it as having multiple Macintosh desks each one of which was all the tools for that particular project the reason why these you used to turn it intuitive before or we feel we feel things that use our kinesthetic and iconic maintenance intuitive also the term leverage is used can you tell me what you mean when you say when when when you succeed when you can never join user yeah well I think I mean another part of my background before I was heavily into computer science was theater and if you think about a theater is kind of an absurd

setup because you have perhaps a balding person in his 40s impersonating a teenage and triumph holding a skull in front of cardboard scenery right it can't possibly work but in fact it does all the all the time and the reason it does is that theater the theatrical experience is a is a mirror that blasts the audience's intelligence back out at them so it's a as somebody once said people don't come to the theater to forget they come tingling to remember and so what you what it is is an evoker and again the having the thing go from an automobile to a medium started me thinking about the user interface as a theatrical notion and the what you do in the theaters take something that is impossibly complex like human existence or something and presented in something that is more like stereotypes which the user can then recentize human

existence out of the the I think it was Picasso said the that art is not the truth art is a lie that tells the truth and so I started thinking about the user interface as something that was a lie namely what really isn't what's going on inside the computer but none of the less tells the truth because it evokes it's a different kind of computer it evokes what the computer can do in the user's mind and that's what you're trying to do so that's the computer they might have going and that's that's what I think of as leverage suggest as with time share you go evolution you were okay he paused for a second this is happening we're probably build a computer out of duct tape yes it just as with time sharing you sort of had the illusion that you had exclusive use as a computer yeah you have a different illusion that you're moving things on it but there are objects on your screen yes in fact we use the term user illusion I don't think we ever use the term

user interface or at least certainly not in the early days and we did not use the term metaphor like as in desktop metaphor that was something that I think came along later the the idea was what what can you do theatrically that will cover the amount of magic that you want the person to be able to experience because we again the understanding was in the theater you don't have to present the world as physicists see it but if you're going to have magic in act three you better foreshadow it and act one and so there's this whole notion of how do you dress the set in order to set up expectations that will later turn out to be true in the in the the audience's mind now are you working on these things other people are pop building a machine they're building various machines and it was it was kind of funny because in the way it worked out as I had come up with a took west clerks link design and came up with a version of sort of a modern

version of that in a little handheld you know it was sort of like a suitcase that I called minicom and has done some design of how things would run on and the problem was it wasn't a powerful enough but it was something I figured I could get fifteen or twenty of them for the experiments that we wanted and I had designed the first version of small talk for this machine and much to my amazement I got turned down flat by a nameless Xerox executive and I was crushed big you know no we're doing time shit sorry okay so there's a I couldn't believe that this name nameless Xerox executive turned me down flat as far as the idea of building a bunch

of these things and I was crushed and now it's only it's 31 years old then and just it's terrible and so then I evolved another scheme of getting a bunch of little mini computers like data general novice and I could only afford five of those I had something like 230,000 dollars or so I figured five of those and take the system that we're experimenting with fonts and bitmap displays on and I'll replicate that I'll get five of them it's incredibly expensive way to do it but that will give me five machines to do stuff and then this Xerox executive went away on a task force and Chuck Thacker and Butler Lampson came over to me in September of 1972 and said to me if you got any money and I said well I've got about 230,000 they say well how would you like us to build your machine and I said I'd like it fine and so what I gave them was the results of the experiment side done in fonts and bitmap graphics plus

we had done some prototyping of animation and music so to do all of these things they had to be so so fast and what happened there was a combination of my interest in a very small machine in Butler Lampson's interest in a $500 PDP 10 someday and Chuck Thacker wanted to build a machine 10 times as fast as a data general nova and Chuck was the the hardware genius so he got to decide what it was in the end and he went off and in three and a half months with two technicians completely built the first machine that had the bitmap display and he built it not as small as I wanted it but 10 times faster than I had planned and it was a larger capacity machine so it was simultaneously both the first thing it was like a Mac and the first workstation and absolute triumph yeah this was the alto just what was the bitmap display why was that well it's I mean that was fairly controversial because the problem is that bits were expensive back then and before about

1971 bits were intolerably expensive because they're in memory they're in forms of these little magnetic cores and just even a modest memory was many cubic feet required expensive reads since amplifiers and power supply was a mess and in the early 70s intel came out with a chip called 1103 that have a thousand bits not right the chips you buy today have like four million bits and the Japanese are making ones with 16 million and 64 million bits this is a thousand bits on a chip this is the best thing that ever happened because all of a sudden everything changed this is one of these predictions of Bob noise and Gordon Moore that completely changed the relationship of us to computing hardware and the people who wanted to build a big time sharing machine at park including Chuck Thacker and Butler-Lampson immediately started building one out of this memory and it was also the right kind of thing to build the alto out of but we still couldn't have

a lot of it this is 1972 that we're talking about and so this notion that bitmap display is something that we didn't invent people had talked about there even were small versions of it television it's television is something that is discrete in one dimension and analogue and another dimension but you can think of it as being 300,000 separate pixels on a screen and it's something everybody would like because it's perfectly general the nicest thing about televisions you don't have to to determine beforehand what kind of picture you're going to show on it you can show anything you don't have to worry whereas every other kind of display you had to have some notion about what you were going to display in order to know what kind how much power you should build in a display so we spent a year and a half or so trying different kinds of displays and making calculations and stuff and at some point we realized oh heck with it

you know we just have to go for a bitmap display and of course we didn't know that people would settle for tiny little displays like this you know so the first display in the alto actually was about two and a half times the size of the first Mac it was about 500,000 pixels and it was an eight and a half by 11 page display the exact size of the dining book was supposed to be and is it your program the cookie monster? the cookie monster the first I invented the first painting program in the summer of 1972 and it was programmed up by Steve Purcell who was one of our students and again the idea was to get away from having just crisp computer graphics as they had been in the past because the idea is that people don't design on computers and one of the reasons they don't design on computers is they can't make things messy enough and my friend Nicholas Negropondi at MIT was also thinking similar thoughts he wanted people to be able to sketch as well as to render and so I invented the

the painting program the painting program was a direct result of thinking seriously about bitmap displays it's maybe the first thing you think of because you are always painting on it when you're putting up characters your painting characters and it was very much like Mac paint turned out to be many years later I have a nice video showing it being used and the second thing I thought up was that having a bitmap display would combine with multiple windows like the flex machine had would automatically allow us to do this overlapping idea which we all thought was terribly important because the we thought that this eight and a half by eleven display it was impossibly small for all the stuff you want to put on and the overlapping windows would give you a way of having much more effective screen space as you as you move things around this machine the also the prototype produced in 1973 how does it compare in terms of what it's got compared with the macintosh is the machines we have now well I think the the first thing is the the macintosh is

represents an amount of work even to get the first mac out an amount of work equivalent to what we did at Xerox added on to it so the screen design and the macintosh is unique to the macintosh and much nicer than we used and I mean they're all of these extra extra things on it and of course the number of tools you can buy there's something like nine thousand pieces of software rendered in the macintosh user interface that you can buy for it so the scope is enormous but in terms of the categories it's about the same you know we had both painting with bitmaps we had Chris computer computer graphics we did the first desktop publishing the first wizzy wig editors and most of the paraphernalia that people work with one exception were part of the original concept of the software that went on out out on that exception with spreadsheets so one of the things

is quite interesting for an outside they see this only mid 70s most of the key ideas of that we now regard as sort of into a computing are there yeah Xerox is not a computer company what did you have hoped what was your hopes when the alto was built did you hope that the world would see this product? I for one oscillated between because I thought of the alto is just an intermediate stage I wanted the diner book and I thought of that user interface as a stopgap measure for the real thing which is see I was basically interested not in the access literacy part which the user interface represents but in one of the things that we realize early in the game with Xerox and its executives that we're going to have to do a lot of demonstrations

because we originally started out trying to talk to them and we wrote some a bunch of papers called the pantry papers that expressed sort of what we were trying to do and those did not get received into any worthwhile corners of their minds so we thought that we would have to do demonstrations so we were set up to do demonstrations we did lots of demonstrations also we didn't want to be cut off from the research community that we could come from so we wanted to be able to give demos and the experience with Xerox went from where the lab being fairly open to periods of paranoia where the lab was completely closed to visitors and we used to say to Xerox look you know we're years ahead of everybody else you know lead time is everything in this business and you just carry through on the stuff don't worry you know you can show everybody in the world this stuff and it doesn't matter they won't be able to build it in time and still true I mean if you look at the Macintosh which came out years later look how long it's taken IBM to not quite get

there yet and they're still pushing that idea and Windows 3.0 is just come out you know take this stuff takes years to do if you have a few years lead time on anybody and your people are capable you can sustain it well I don't know about that because it's really hard to predict that kind of future I was amazed at the number of people for instance who bought IBM PCs considering how useless I thought they were and the the interest of everybody in switching over to the overlapping window interface with icons and mouse I think shows that what people bought millions of is not what they want they want what we wanted early in in the 70s but I think predicting what people will do under conditions of expediency is impossible in those days right there were just a few hours you couldn't buy it right there wasn't that many and Xerox didn't seem to have

any immediate plans yeah well we actually set up a production line for the things because I as I say I needed quite a few and they got started getting popular and so we set up a production line and I believe that upwards of 2,000 of them were built in the decade of the 70s certainly by 1975 or 76 almost everybody at park including the secretaries had altos and they're connected together by ethernet and their page a second laser printers nearby that you could go to so is it a very similar model to the the offices that you see that you see today there were enough when Xerox made its fateful turn down in 1976 there are at least 500 machines operating the way machines do in the in the 1980s so that we're not turning down a paper idea or a single prototype you're turning down something that had been functioning for three years and this was turning down 90 to some computer with the word presses as opposed to a well the story the story is is

complicated it has to do with a bunch of things I think a simplest way of characterizing what happened is that just as when IBM in the 50s turned down their chance at the Xerox machine because they ultimately thought they're in the computer business rather than the communications business Xerox when push came to shove thought they were a copier company thought you know so the problem with railroads right as they tried to make faster locomotives rather than investing in the airlines and so there was that problem there was a problem of slight more cost in doing the alto versus a word processor that they wanted to do all these things contributed I think one of the main problems is that the executive who made the final decision had no way of personally evaluating what he was being told from different sides so it was complicated I mean the the best they could have

1976 when they had to make the decision you figured at under best conditions it would take a couple years to get a machine out they would have had to charge around 20,000 or so for it which is not bad that early because that's what word processors were selling for but it was it was right on the right on the edge and by the time the star came out which was too little too late it was a I guess a $20,000 machine that should have been sold for around 10,000 but Lisa had some of the same problem it was a $10,000 machine that should have sold for a couple of thousand all right I think we're going to connect to where we were yesterday we're talking about 0's power do you look back on it as a sort of a golden age in computing that the 70 people gathered for a period together well I think I don't know I sort of feel it would be a little bit presumptuous to declare it a golden age in computing it was certainly a golden age for me I think the five years between 71 and 76 were the most fun I've ever had during research and I think that

there have been five or six critical places in computing over the last 40 years that have affected the way people think about things in park was one of them now you weren't the the run of the mill 0's executives and Stuart Brown wrote an interesting article. Oh yes well that was a that was a hilarious little side excursion Stuart Brown was the person who put the whole earth catalog together and he got interested in computer people because he lived right across the street from Engelbart's work at SRI and so he decided he would do a piece about the culture and one of the places that was just starting up then in 1971 and 72 was Xerox Park and so we yeah we all knew him and he was a friend we invited him and he wrote this great article which was published in the Rolling Stones which is really considered to be a rag back then especially by the East Coast establishment. And this described the youth people sitting on bean bags

yes the whole culture and their photographs are all taken by Annie Leapowitz and you know it was a very Hollywood's California type scene and the Xerox people did not like it at all and they had an enormous reaction so so larger reaction to it that when Stuart republished the article in his book they forced him to not use the name Xerox and so he referred constantly in this article to Shy Corporation which is what he called it. Now you were talking yesterday about having done all this work on the outside the questions of whether you really wanted to get it out there and so forth and you made lots of demos you said now one in particular big effort was made at a sales meeting in Bocca Ratton wasn't it? Yes that was rather late in the game though that was quite a few years after the alto have been around but their constant recirculings of trying and what this wasn't so much trying to get the alto out as a product although there were people who are very interested in it. A lot of these

meetings particularly the Bocca Ratton one had to do with just getting Xerox at the top and its major executives to have some sense of how the world was changing and what their future was likely to be. Did you feel disappointed not so much as you say that they might have had commercial reasons for not backing it but did you feel that they really understood what you've done that they got it? No I don't think they got it but I think the important thing about the Xerox deal is what they did promise to do was to give us relatively unsupervised support for a period of 10 years and that is what they did. We had our skirmishes with them and there are some people who have to sacrifice their body and soul to keep the umbrella over the group but in fact Xerox turned out to be a very good funder and there is no promise or even any intent that I ever heard in the beginning that Xerox is going to turn these into products. Most large corporations use their

long range research more as a hedge against future disaster than as pure product development. So in a sense you were particularly surprised at the thing? Yeah I was surprised. Sure I mean it's one thing for them not to make any representations about doing as a product but the whole setup was so obviously the way it was going to be that I was surprised sure. I was amazed. No one has been going on 10,5 and done the work on on the microchip and so forth and hobbyist machines started to appear. What did you guys make of that hobbyist well you know there was something that with versus a four-bit wide chip and an eight-bit chip and and so forth and they were sort of inevitable that they would appear. For me I was surprised that so many people were interested in them. I realized that an enormous number of people felt disenfranchised from computers completely and these were a way of getting in. I could

can touch the new technology and so forth but the the kinds of software you could run on them was really limited and so the various opinions at park Larry Tesla was much more interested in the eight-bit machines than I was as an example. My feeling was is that you had to have at least a 16-bit machine that went so and so fast in order to do all of this stuff and in fact that's exactly the way it worked out. The Macintosh was a 16-bit machine that went thus and so fast and you had to go that fast in order to do all the things that we were doing at park and so from my standpoint I would have just been just as happy if no machines had been built up until 1984 or so just from the standpoint of you think of all of the unfortunate standards that were set like MSDOS and so forth that are holding back the world even today. I think it would have been but you never know. The other head as you say they released this pent-up

sort of a massive disenfranchised people. I mean that was what was surprising even though as you said it was toys. Yeah and I think that depending on how you you look at it certainly one great thing was done on an eight-bit micro and that was Visicaelk. That was one of the best thing it's ever been done on a personal computer as an idea. The reaction to that at park was both admiration and shock. We couldn't believe that we hadn't seen it. That's how arrogant we were but I think that the aside from that almost everything else done on the eight-bit micros is sort of a reversion back to the early 60s when machines are very weak. Most of the eight-bit micros had either no operating system or terrible operating system. It's not clear which is worse and many of those operating systems are still hanging around today. Now what you saw happen after this work

was done at Karm. We saw the in the next years ahead the this was going to be exported from a company like Xerox. It wasn't really a computer company into one of these new fledgling popular companies and that was a remarkable transfer. Yeah I actually thought that Xerox is the right place to launch from since it wasn't in the computer business. It really didn't have any history to give up there whereas I think it was more remarkable for a company that was deeply wedded to a certain kind of eight-bit computing like Apple was to be willing to throw it all out and start in a completely new way. But that is very much one of Steve Jobs's biggest strengths. Did you have any recollections of the eventually enough to jobs as visit when you involved at all? No. I was at the famous demo that Larry Tesla gave Steve Jobs and some of the other people from Apple. What did you heard? Did you heard that Xerox was interested in buying a steak and Apple? No Xerox had a steak and Apple by then. There's a company called Xerox

Development Corporation and they had steaks in various companies. I forget how much it was 10 or 20 percent or something like that. But I don't think, you know, of course we thought the at park we thought the Apple was the the better of the eight-bit micros that were around and so forth. But it was not unusual to have, we gave many, many demos. So it's not unusual to have somebody like Steve Jobs and other people come over. Do you remember talking jobs that day? You have any recollections? Sure. Did he get it? We got it right away. I mean there are two kinds of demos. There are one where you're struggling to get the person to see what's going on. We found for many people we had to put our, they weren't used to looking at a computer screen. So we had to put our finger on the screen and get them to watch our finger and then see if they could focus in another quarter of an inch and stuff. And then there are other people who tell you what is going to happen next. I mean they're so on top of the demo that they know. And now you're going

to show me, yes, here it is, you know, those are great demos. I had the great pleasure of showing Small Talk to Kristen Negard who is the inventor of Simula, which is one of the major influences on Small Talk. And it was that kind of demo. He just knew everything I was going to show him. There was stuff that he had been dreaming about for years and here it was. And we've been friends ever since. Now what do you think Steve Jobs great achievement is then? Because he took this, he got it and he took it back. Well I think his, I mean what he took back was an idea that things could be done a different way. And particularly Steve who is such a visual person, very sensitive to how things look and appear and stuff. And the whole notion of the graphics interface was something that really captured his imagination. And eventually some people from Park went over to Apple. Larry Tesler was one of them. But what happened there was that they pretty much took what we had done as a departure point and did a completely new design.

And I remember I didn't see any of it until Elisa came out in 1983. And when I saw it I thought it was just the best thing I'd ever seen. It was just incredible. But I mean Elisa was beautiful. No it's a bird. The Lisa was still too expensive and so really the McIntosh was the threshold machine. Yeah there are, I mean the McIntosh in many ways is not as good a design as Elisa but it was a severe compromise. And the thing that was great about it is that it used the weak power of the then 16-bit 68,000 to great advantage where the Lisa couldn't quite make it with all the things that was trying to do. And so in many ways you can think of the McIntosh too as the

Apple's return to Elisa. Now when the McIntosh came out and certainly the history of the computer since then has been very much a sort of a vindication of everything you did. In a sense haven't the main victory been won by that point. People have realized computers at least could be different from that one. Well I don't think any of us at Park were fighting a war so it wasn't clear who were victorious against but if it meant getting lots of people convinced I don't think that it was one because the majority of people who compute today still don't interact with a computer in that way. There's this big urgency for hang on one second I can hear my McIntosh digging it. It needs paper. Okay we can hear that actually put the cat in the garage. Yeah I think the important thing is that what we did at Park

was not an ultimate answer to the problem of interacting with computers. So I think that a lot of people are going to be interested in it more than the millions that are now and what will inevitably happen is that people continue to be interested in long after it is worthwhile interacting with computers that way. So you have this thing where people have this tendency to once they like something they get religious about it then it hangs on and hangs on long beyond its actual use. The time McIntosh came out Apple essentially was betting the company on it and there was this very popular standard MS-DOS in existence on millions of computers. Right. Talking a bit about why do you think what are the what are the reasons why most people find one more intuitive and easier than the other and I want you to talk about some things. Yeah well I

don't think we ultimately know but the I don't think we ultimately know what what the reasons are but we certainly were guided by some theories that we have different ways of knowing about the world and only one of them is through language. We have a kinesthetic way of knowing about the world through touch. We have a visual way of knowing about the world and the kinesthetic and visual ways seem to be more intuitive for people there. They're less crisp. They deal more with analogy and the touch thing makes you feel at home you're not isolated from things when you're in contact with them you're grounded and so I think that for me the major reason the McIntosh works is because the mouse gives you just the tiniest way of putting your hand into through the screen to touch the objects that you're working on. The screen gives you a theatrical representation

of something much more complex something that computer designers don't even want to think about as a computer executing two million instructions per second with thousands and thousands of instructions in there and then finally the least successful thing that we did that we're still working on is symbolic underpinning for all of this that allows you to express back into this little pocket universe yeah so I think so that that film of the of the little girl 22 months old using a Mac very confidently she'd already been using it for about six months or so strikes a lot of people in a way that words don't because they see here's this little child about 70% literate in the access part of the McIntosh user interface she can start up the applications and do things and then save them away and all of those things and that's what we were trying to do

is to extend this thing from being something like a specialized tool or a car to something that's more like media that extends down into childhood and up through the the elderly. What do you turn a bit now to the practice of writing those interfaces and applications somewhere so and the difference that some of these problems are differences between software and hardware people basically how would you characterize the difference between software and hardware because the word software as far as I can see any any grew up about 1960 I mean it's not something it was something new in a way that's true although the you know the notion of the stored program goes back a long way but I think the for me there isn't any real difference that hardware is just software that's crystallized early because basically what you always have is something that you can remember in terms of some medium that can take on markings and there are different kinds of

markings and then you have to have something that allows you to change those markings and read them and change them and the simplest kind of computer you can build is one that only has a couple of states in it and it's all memory it's practically a clock and everything is out on the memory so it's almost no hardware there at all and the main reason that there's a fair amount of bits in the hardware of today's computers is there a lot of functions you'd like to compute rapidly and so there's special purpose little pieces of logic in there for doing fast arithmetic and so forth yeah that's a special second to do to do arithmetic and stuff yeah I think I think it definitely is true and there there have been computer designs and computers built that look at their memory in a completely different way of doing content-addressed

storage and having many thousands of processors looking at storage at the same time where hardly anything resembling arithmetic is done most of the time but in fact it doesn't matter because arithmetic itself is just a manifestation of a particular way of putting the logic elements together so it's a very basic way like sure I thought about this is the general purpose in the nature of symbols what you need to take some markings transforms them and puts it out right right and there's a trade-off between the number of different kinds of markings that you want to have the memory store and the amount of logic that you need to be able to carry out computing functions we take sort of average sort of computer it's got it's hardware can carry out or it's just about a hundred basic functions sometimes yeah these days the usually up in the range of 300 or so I think for for most people trying to understand how a computer works

down inside it's it's actually mostly memory is a vast amount of very undifferentiated stuff that is just for holding the markings then there's a very small amount of logic that carries out a few basic instructions and you can think of all the rest of the hundreds of instructions as things that the basic instructions could carry out but have been encoded specially for speed and then what the program has to do is using these available facilities as to constructed to do yeah well most good programmers are relatively lazy and so the last thing they want to do is spend a lot of time grubbing around down in the machine code of the computer so usually what they do is they write a couple of pieces of software one is called an operating system another one is called a programming language and often they are both the same as small talk was and what what that piece of software does is to create a virtual machine which is much more hospitable

and so it's a the machine simulating a much nicer machine and all of a sudden life is much more fun and often you will use that machine to simulate an even nicer machine eventually things slow down so that the the most wonderful machine you could have might run too slowly to be interesting but some people still like to program in terms of these highly idealized machines because maybe they'll be built one of these days so in the sense that you're telling us that what we see on our macintosh is a much pleasant information yes and in fact the user interface you can think of as the last layer that is designed to eat to further sweeten the the realities of all these different layering that are going on most important thing is it's it's when you're doing when you're cooking a cake baking a cake in the kitchen you don't have to worry about the details of organic chemistry because they're already subsumed by the ingredients that you have in the recipes are there to make sure that what you'll have converges on a cake rather than a mess

and for this exactly the same reason people like to confine where they're working into an area where they know what roughly is going to happen and if they that area doesn't work out well and they'll go to a deeper level and make a few changes now all of this of course depends on the computer being able to carry out instructions fast I mean right otherwise we probably wouldn't be having a simulation would not be that much fun if it were really slow right as it was when people when they had to just calculate simple trajectories of shells in World War II they'd have three or four hundred people on desk calculators just calculating a simple ballistic trajectory that was a simulation and in wartime it was deemed important enough to put these four hundred people to work for days doing these things now before the computer was built I'm ensuring approaching the subject from a different point of view you know spoke of a very simple machine which could indicate all other machines the digital computer is an example of one such sort of machine

right and what does it mean then that sense to call it a universal machine well I think it's one of the niftyest things to wrap your your head around and that is that regardless of what kind of thing you have if it has a few basic properties like a memory and the ability to put symbols into memory and take them out and make changes and make a few tests on them that is enough machinery to enable you to simulate any computer that is ever existed or any computer that will ever exist how interesting the result will be depends on how fast the simulation runs but in fact it's people sometimes think that a little machine on a desk or even a wizard sharp wizard calculator is a different kind of thing than a big cray computer but in fact they are the same kind of thing you could calculate one of the greatest pieces of 3D graphics on a sharp wizard given enough centuries to do it and enough external memory to put parts of the calculation so this machine

even though it was built for mainly to do scientific calculations some people that are threatened by getting it had just enormous potential yes in fact the the first person who seemed to realize it was Ada Augusta who was the sidekick of Babbage and and maybe history's first programmer and she wrote in one of her papers that she said the the analytical engine weaves algebraic patterns just as the jacquard loom weaves patterns and silk and she understood that the generality of representation was the same kind as you can represent in books which is namely the the kinds of things that we can talk about it wasn't restricted to just numeric calculations but it extended into the realm of general symbolization of models this this amazing capacity is a very seductive quality about it it is to us and we're thinking of saying the late 50s at MIT some of the early

hackers some of you know it once people people who realize the potential but they could make this machine do this that the other or I'm Southern or whatever you mentioned yes and it was it was like yeah I think two two things happened the the Turing thing was there and most of the early computer people either were mathematicians or had a lot of math in their background and there are other formulations like that of Turing girdels theorem also was a way of building a universal machine and there is a thing called post-production systems which is a similar they're all studying mathematical problems but they translated well to computer machinery and most computer people were aware of them then the second thing that had to happen was that there had to be some way of believing that the computer could get big enough and fast enough so that anything that Turing said made any practical sense you know so you have to have both of those things and what's happened

is that very early on in the even in the 60s but especially in the 70s people regularly would build very simple computers and then use those simple computers that ran very quickly to simulate the kind of hardware they really wanted that's what we did at Xerox Park Chuck Thacker who did the alto built a machine with almost no logic for that period of time I mean 1972 it had two had about 160 chips two boards worth of chips so I was very very few gates but it ran extremely fast and ran about five times faster than its than its main memory and because of that the alto could be many different machines it was a small talk machine when we ran small talk on it was a Mesa machine when they ran Mesa on it and it could take on different personalities and I've always thought that that was a really good way to go until you actually

know what the the biblical truth is on what your computer architecture should be why not have a computer that you can mold into the kind of virtual machine that you want right at the lowest level this tractability is moldability gives the computer a romanticism that other machines lack yeah well I don't I I agree on the romantic part but of course I I'm fairly romantic about musical instruments so that you know most musicians are most musicians adore their instruments and so I think that the from from ice standpoint my romance is very much connected to the same way I think about other marvelous contraptions that we've made including musical instruments but sail planes the kind of marvelous inventions that Paul McCready makes and so forth no no in that with a musical instrument you can build a private universe for somebody else you know that as you can you can make one but it doesn't have the tangibility that

that the computer has on the other hand it's it's worthwhile remembering though that no matter what we build on the computer no matter what appears on the screen it doesn't have any particular sense to it unless some human is there comprehending it so it does have something in common with with building a beautiful piece of music that there has to ultimately there's a listener might be just the composer but it what what comes out has to in some way come back into the human uh sensorium and be understood now you use a metaphor which is quite helpful and understanding programming of the pop and CHI I want you to do this yeah I think I got talked into that by some some magazine one of the traditional ways of of programming on a computer is to think of the memory part of it as being inert like ingredients in a kitchen and to think of the programs that you write as being like the recipes and then the central processor is kind of like the cook who's

looking at the rest of the book and then goes over and stirs the ingredients and if you're lucky you wind up with with something good to eat and another way of thinking about that is that's that's like a puppet theater because the puppets are all inert and there are puppet masters going around in the computer it's a very energetic puppet master because there's one generally goes around and twitching all of the strings of all the puppets fast enough so that it seems like something real is going on um but another way of looking at uh programming is to say well why not let the puppets pull their own strings so we'll let each puppet have an effect its own little computer within this larger computer and we'll let them be much more self-contained there won't be puppet masters from the from the outside and that's called object-oriented programming and uh the benefits of it are simplicity and ease of writing the programs so in procedural program if I want to tell my puppet master what to do I have to list everything absolutely in the right order yeah I build

my objects give them behavior and yeah well there's I think there's a continuum from totally locked-step proceduralism to trying to deal with multiple processes to having objects which have many of their processes inside of them and act much less sequential and so forth to what's coming in the future which is a something called agent-oriented programming or modular control programming or hasn't got a good name yet but of something where the the elements are much more like biological cells that they're quite self-contained it may spend 90% of their energies just maintaining their own equilibrium and maybe only 10% of them contributes to the larger system as a whole these are presumably even higher levels of virtual machines that were yes that's a very good way of thinking about them now just to get one point every time we go further out to a high level of virtuality that suits us it generally means the computer has to

work even harder yeah well I don't think it's up the computers is is only working at one level you know any given computer it's always executing two million instructions in per second see you can't make it work harder but it's like if you have a a five horsepower go-cart and you put it up various grades of hills it'll be a hill eventually that it won't be able to climb it's always putting out five horsepower and it needs 10 to get up that particular hill and that's what happens you just things start slowing down as you put more and more burden on an old-style processor I'm looking at the screen and I want to do something like open a fire on the MAC desco I mean some solution of how many operations might be well it's you can figure it out because the typical MAC these days executes about two million instructions per second and if it takes well say opening a file it has to go out to the desk so it's complicated because you now

depends on the speed of the desk moving stuff back and forth but suppose you're just in the multi-finder and you go you're in this window doing something and you put the mouse in another window and the window comes up to the top of the screen and you can easily just get out a stopwatch if it takes us if it's a big lumbering thing and it takes about a second to rebuild the screen then two million instructions have been executed it takes two seconds then four million instructions have been executed and it's always executing that many instructions per second even when it's just idling yeah and often the complexity is something that is a byproduct of the way the system was programmed rather than being intrinsic okay well the way I think about tools and agents is need some sort of perspective to think about them and I think about them in terms of the way we've

extended ourselves over the last several hundred thousand years and when we say extension to somebody they almost always come back and say tools and indeed there have been levers and wheels and physical tools but there also been mental tools like language and mathematics and I think of tools as being extensions of the gesture is a way of manipulating things that you're manipulating symbols when you're dealing with language you're bringing things that are hard to deal with into your power via your hand or something like your hand so the M word to me for tools is manipulation and tools are things that you watch while you manipulate them and then the other main way people have extended themselves is by getting other people to take on their goals Mumford called this making mega machines said that for most of human history most machinery made by humans has had other humans as it's moving parts we make cities and cultures and our groups

trying to do this and groups trying to do that and there are fewer goals in those kinds of groups and there are people they've taken on each other's goals traded off one way or another and they are communicating and the kind of entity that can take on your goals and act on your behalf we call an agent so an agent is something that watches you and you manage it so the M word is management for agent and manipulation for tools and in the agents of generally being people up to now where is tools that mean yeah you could you could say that that a shepherd dog or maybe a horse maybe a thermostat is something you have to work really hard with to build a thermostat to get it to take on the goal of what temperature that you that you want but by and large they've been people up to now and the interesting thing about computers when you're building agents on them is the agents don't have to be as smart as people just like a thermostat does not have to be

as smart as a person to be useful the main thing you want it to do is to be able to take on some collection of goals and be able to deal with those goals while you're not around so an agent has to have some artificial intelligence in it yeah if you like to use that that term you could call it flexible competence and make it sound a little less loaded so how would this work so you you feel that the computers at the next 10 20 years in addition to having the sort of rich interfaces that we'll have personal agents yeah I mean the way one of the ways I think about looking ahead into the into the future is to try and find analogies that might actually make some some sense and also to look for driving forces so the the one of the driving forces for the part type user interface came out just from there being inexpensive ice integrated circuits around start getting a proliferation of computers that are inexpensive enough for people to buy and all

of a sudden the kinds of people who might want to use computers changes completely and so all of a sudden you need a much easier to use user interface there's a driving force now to do something because it isn't just graduate students anymore to me the driving force for agents is pervasive networking because the techniques used on the Macintosh don't work well when you're connected up to a trillion objects scattered all over the world you need something looking for potential objects that will further your own goals and you need that something to be looking 24 hours a day so we think that what what we'll have is you know 10 15 20 or more little agents many of them not particularly intelligent but able to flexibly take on a goal that we have like an example of one is an agent that goes out and finds you the newspaper you'd most like to read

a breakfast every morning so all night line works it can touch dozens of different new sources the associated press New York Times and so forth looking for things that are relevant to you it can go to other sources for getting photographs and so forth they can do the news gathering with a particular interest in the kinds of things that you're that you have been involved in a headline could say new fighting in Afghanistan or might say your three o'clock meeting was cancelled today because news now could involve your own electronic mail a sidebar might say your children slept well last night and this is an interesting example of an agent because it's one that was built about 10 years ago it did not require a large amount of intelligence in order to work it's major strength was its ability to work 24 hours a day while you weren't

there and with a limited ability of doing matching against what you said you wanted it and what it thought you wanted it could do a good great deal of useful work for you. So we think of there being a number of revolutions in the history of computers so far but we're thinking now most people think of the computer as a standalone desktop object right where do you see tying this in with your dining book concept what you've been saying about networking and agent where do you see the next sort of thing taking it. Yeah well I think the way I think about that is the these three very different ways of relating the the human to the computer one is this institutional way of the time sharing mainframe one is the desktop way where you control all the stuff and then the third way is the intimate way which is the is the dining book way which is continuously connected into the worldwide informational network so it's part well a dining book is a sort of a figment of

imagination it's it was a holy grail that got us going it was a cardboard model that allowed us to avoid having meetings about what we were trying to try to do it was a lot of different things but it was basically a service concept not a box concept so there are actually three physical renderings of the dining book we thought about one was the notebook one was something in when in your pocket that had a head mounted display in glasses as I had worked with Ivan Sutherland's head mounted display in the 60s and then one was Nicholas Negro Pondy's idea of the sensitive wristwatch that in the 20 years future or so when there's a network wherever there's an electric plug in the wall then your user interface will follow you from room to room as as you go that everything has become pervasive you don't need to carry a big computer or even a tiny computer around with you so the the whole idea behind the dining book was the kinds of service and your

relationship to it which should be intimate casual mundane you should be able to aspire to the heights on it just as you can when you learn English you can aspire to the heights of Shakespeare but you're not forced to do what Shakespeare did every time you use the language so it's this idea of having a nice connected ramp without lots of bumps and so forth in it you know as see more paper it likes to say low threshold no ceiling now given you have that concept of the future your current work at the moment of the school here and essentially with children who may inherit such a model of computer what what is the objective of this work well several one is that Apple traditionally has been a company very interested in

the educational process and helping children in in schools and so we do a lot of things in the school that have to do with thinking about how schooling might be in the future then specifically one of the things that we do is a project that's been going on for about four years now to try and help find ways that will allow children to be able to write in the computer as fluently as they can now read using the Macintosh user interface and what we do since artificial intelligence is coming along we're trying to find ways to both understand artificial intelligence and understand how to program it by putting together a set of tools that allow children to do the kind of artificial intelligence programming normally only done by adults yeah well our all right we're trying to you think of adults trying simulate humans and and so forth humans are pretty

tough nobody's done simulation yet I've always felt it would be a good idea to work our way up through the food chain and start off with fairly simple animals see how they interact with the world that's something that children are interested in and so quite a few years ago we got the idea that would be really great if we could give children an environment where they could create ecologies to study animals and plants and then build those models into the computer and see if those models reflected what they thought they understood from the from the real world so there's a constant comparison between the real world and the in the computer model the the school has torn up part of its playground to make a a life lab which has both animals and plants in it the classrooms have animal cages and aquariums and and so forth and so there's a lot of real animals to study and then we also have macintoshes with a new system we've designed called playground

that tries to bring some of the techniques of artificial intelligence programming to eight nine and ten year olds right now are you trying to achieve literacy computer literacy in children well yeah I'm here you know in the in one sense I think think so in the sense that I've always wanted to close the loop at least with something that was like reading and something that was like writing and right now the something that's like reading is using the macintosh user interface language to deal with nine or ten thousand applications that are out there that seems to work reasonably successfully right now and the equivalent of writing should be something that allows children to aspire to the same kinds of things that are built on the mac now they may not sit down and do all this page maker or something like that because

that's that's something like a large play or something but they should be able to see a continuity between what they're doing and these tools that adults make for them we want to do something like what Dewey was talking about in the in the last century which is you pointed out that from in most of the ages of mankind the games that children played were serious simulations of what the adults did so the African child practicing with a spear or the Eskimo child learning how to kill birds because he's eventually going to have to go and kill seals for food is doing something that is content rich relative to the adult world but the 20th century child dressed up in a nurses suit and playing nursing with her doll has been cut off from the content part of that adult activity only the form is left and so the kids are very disenfranchised from most things that are happening to adults and one of the things that was noticed right away with computers is

that when you put a child on a computer they know they're doing the real thing that this is they can see instinctively the continuity between it and the other things that are going actually much better than adults do the computer many people claim it's going to have a big role in saving American education and so forth and this is a worrying thing because you said yesterday that the computer was a metanidium so it can be what we choose it to be right how would you yeah well I mean the the story I always tell is imagine the parents were afraid that their children wouldn't make it in life unless they were musicians and the state legislatures said well okay we'll put a piano in every classroom but we don't have enough money to hire musicians so we'll give the existing teachers two-week refresher courses and music doesn't get into the the classroom and I think we have a very similar problem when we want to think of the technology as being the

magic ointment musicians will tell you the music is in the piano if it were we'd have to let it vote so at best what we have is an amplifier and often these things that could be amplifiers will turn people away piano is often turned people away from music rather than turn them towards it so I think the most important thing is to have in for people who want healthy schools is to have parental involvement because down deep it's the value system that the children pick up about what's important in life that they mainly get from their parents that is going to condition what they do with their time it's hard to learn things in school there are lots of things going on school to me is basically about a resource for finding out that various things existed that you didn't think existed but as far as learning them most of the learning I think is done outside of school and what the child decides to do outside of school with his time is going to depend on the value system once you have that going really well then it is fairly easy to use all kinds of

technology because then they will simply amplify out from this interest in getting stuff happening in here interesting that this medium should come along at a time when a previous important meeting or a medium like writing everyone is so concerned about literacy in the schools yeah well I think that was one of McCluen's tongue in cheek jokes is that we've had all of these great inventions like the book and they've hardly affected education at all because if you go into most schools in the northern hemisphere you find 30 humans writing down what another human is saying and that's what was going on in Oxford in the in the 12th century so you know where's the book in all of this so I think that the the kinds of social whirlpools that exist when you get different kinds of humans together like teachers and children are going to have a lot to do

whether technology gets used at all I think the most important aspects are and have to do with areas of control and other kinds of things which are theoretically outside the domain of education but a lot of school is about controlling the kids now we go we're looking into this future some of the things that might be important can we can we count from the hardware continuing to improve for another decade yeah yeah we can definitely count on the hardware continuing to improve for another decade and probably more but I mean just this is just extrapolation the current kinds of hardware that we know how to build has a very stable outlook for the for the next 10 years well psych project is one of my favorite projects partly because it's done one of the by one of the smartest guys in the US and computer science and partly because it's one of

the hardest projects that anybody is trying to do right now there are not a lot of researchers working on what I would call really frontier difficult projects and but this is one of them and it his success will be and is is about turning up new ideas for representing things whether the system is actually able to turn into what its design goals say it is which is a a model of human common sense I don't think is is nearly as important as the wake that it's throwing up when you get a smart guy working on something really hard and a bunch of people being ingenious about it you're always going to get good things one would it be important to incorporate consensus representation of common sense in future yeah it's it's that's a good question because common sense is frequently wrong scientific discoveries of the last 300 years have been against common sense but whatever common sense is it's a kind of an underlying fabric

for getting between things that we know in much higher detail so one way of thinking of it is there things little things that were expert in our like islands and then there's this ocean that we can paddle around in from island to island there's a way to get from one island to another and the problem with most expert systems up to now is that they're an island completely surrounded by a cliff if you go anywhere off off what it's good at it just drops you completely there's no way of paddling to the next thing and I think as I said yesterday somebody Picasso I think said art is not the truth art is a lie that tells the truth and common sense is not the truth but it's a lie that provides a continuous fabric to work around in that we can then delve deeper into another thing about common sense that's kind of interesting is that it might be possible to use the computer to enlarge what we think of as common sense by giving

us sensory contact with things we've never been able to have sensory contact with before like things that are a million times smaller than us because common sense I think has a lot to do with the sensory domain and reasoning from things that are on our scale science in a very literal sense is nonsense because it's outside of the sensory domain almost everything that happens in science is very far you know this common sense says this is solid should hit here common sense says this is solid but science says it isn't common sense says the sun is going to come up tomorrow morning and science says no the earth is turning and yet we still say the sun what time is sunrise tomorrow so I think the importance of psych using common sense has a lot to do with that regardless of whether we're scientists or not we have this one way of knowing the world right or rightly or wrongly that is very comprehensive and gives us sort of a universal way of weekly getting from one topic to

another what you you've sort of known a lot of the people in artificial intelligence curious what do you think the sort of main legacy of artificial intelligence is I mean do you think it's been very well I think yeah some of my favorite crazy people are AI people AI in the 60s was once defined as all that stuff we don't know how to do yet and the to some extent it's been a moving targets thing the things that were AI problems in the 50s and 60s are now taught as part of computer engineering courses and but as far as the AI has something mimicking in a strong way human intelligence we're very far away from it and so it's a good goal gives you something to to reach for and I think for people in the field who have some biological motivation to their interest it's a good goal because it has partly to do with understanding more how we do it wonder if there are alternate ways to do it can you only do it the way we do it at what level do we do it

you know do we have to do it at the absolute neuronal level do we have to simulate every neuron in a human brain to get artificial intelligence or is there a a higher level that we can do it and those are good questions to ask because if you look at the way for instance biochemistry is done by nature it is impolingly inefficient very low energy transfer the absence of some watcher from the outside seeing oh yeah be much simpler to do it this way so the way we do chemistry in a lab in the way nature does biochemistry is completely different we do it much more efficiently nature does it much more ingeniously because of the way it's contrived and you can learn a lot from looking at the comparison between the two and so there there's a lot of reason to expect that you don't have to go to the neuronal level to be able to do the kinds of things that we do but nobody knows whether what level you actually have to go to

now you mentioned just earlier that you worked with Ivan Sutherland on the head-up display one one group of people argue that the stage we've got to with the the human computer interface 2d 2.5d is just a sort of a stepping stone but beyond that there is this virtual reality what's your view on that well I think I think many people will enjoy virtual reality since many people don't enjoy the current reality television is a kind of a virtual reality and I think things that go further in that direction will be very popular with a lot of people I think the best thing about virtual reality is that you can deal with these things outside of the normal senses you can take a trip down into a cell and see how incredibly agitated the thermal properties are down there I think all the things that you only can read about now in terms of

symbols you can actually go there and get a much more kinesthetic and visual hit on doing those things I think the use of it in fantasy will certainly certainly happen but if you look at what you actually have to do to get good dramatic situations in the theater then it's going to be a while before something good can happen in a in a in a movie that's partly being generated by your your presence on the other hand if you look at a typical Arnold Schwarzenegger shoot him up then those will be easy to do because I forget what was the name of that movie I can't remember the name of that movie that he just total recall you know total recall somebody said if you like road accidents you'll love this picture and that kind of stuff you know where you have

you know five people you have to kill every 30 seconds or so it's very easy to set up in virtual reality I'm sure that a large percentage of the population will enjoy it computer do a computer simulation that kids can understand from that one we're going to give a speech because that's an interesting thing but clearly the fact that you can make real thought experiments so you could you know many of the classic sort of thought experiments of Einstein and Bob you could right you could do right yeah and of course you have to be careful because because simulations are lies in a sense then there is nothing that says the simulation has to be like real life I mean you can make there've been plenty of thought experiments that are wrong and most of the great scientists have been good gassers and so you can you can also set up simulations of situations that don't have anything to do with the physical world right so you can dilute yourself as well as help yourself along anything that may be one of the

dangers for you no I don't think that's a danger I think that the anytime people try and make models try and look at their beliefs from different points of view and just one point of view I think is good if we look back I know you saw a lot about the problems of predicting what's going to happen if you look back at the history of the computer it seems that almost everybody's being quite quite seriously wrong with every so every stage you know we're very few exceptions um what lessons can we can we learn it well I think prediction there are lots of different ways of doing prediction but the the worst one in the 20th century has been extrapolation so just because something is x and 10% in some other direction of x gets you here and so forth doesn't mean a thing it's like if a computer could do so and so it would have to be the size of the empire state milling that was when people's imaginations were limited by vacuum tubes so the extrapolative way I think is out but the reason the predictions that we made in the late 60's were

so good and the reason bushes predictions in the 40's were so good had to do a completely different way of predicting which had to do with thinking about things that amplify human endeavor and the amount of horsepower available that is interested in making human endeavor be amplified is very very large so you can hook into something like saying if you can say oh the computer is a medium then all of a sudden you start seeing what the powers of amplification are and you also start getting ideas about what to do next you look at say Freud's model of human mentality which is a good one but it's all about drives that doesn't help you much in doing user interface you look at runners mental model which is about different ways we have of knowing the world and all of a sudden you get ideas one of the arguments which said there'd never be many computers was the one that we'd never be able to find enough things for them to do yes

and that's another one of those things is the again it was looking at the kinds of things that computers are doing now and as people used to say that's right you numbs skull you know but there's all the new things that they can do it's not we can do we can do payroll on the main frame when the personal computer came along it's all those things we can't do on the main frame like spreadsheets and desktop publishing and and so forth when you see the future of the computers becoming totally unremarkable well I would hope so nowadays well 10 years ago if you wanted to somebody's office and you didn't see a phone that would have been remarkable was the absence of the thing that would have been remarkable nowadays if you go into somebody's office and you don't see a phone you assume they're wearing one but you do not assume that there is no phone in in a person's office because it's it's something it's noticeable when it's absent and the computer right now is still more noticeable by its presence than its absence

when you go somewhere and somebody doesn't have a computer on them and that becomes a remarkable thing then I think the computer will have made it it's destiny is to disappear into our lives like all of our really important technology the things that we don't think of as technology like wristwatches and paper and pencil and clothing and all of those things I think the computer's destiny is to be one of those and it will disappear into our lives embodying old previous media or many previous media I think so I think we have to be careful because the when you simulate one thing by another you usually give up something and anybody who's ever seen the book of Kells in for real realizes what you don't get from photographs realizes what you don't get from printed books and also realizes what you do get from printed books that the the compelling charisma the transcription of the oral event

that was the a book like the book of Kells is completely different from the alienating regularity of machine type and both of those things have their place in the world you would hate to get rid of one completely and say well we're replacing it with the new because I don't think it works that way I'm building a rather large broke pipe organ even though you can quote unquote simulate them on synthesizers and stuff today and the answer is you can't simulate them you can't get all of this stuff yeah and even if you could even if you could prove beyond a shadow of doubt that the wave forms from it you still don't get something it looks as neat and so I think if you include all of our senses into an experience that when you simulate something as you always do in science what you're saying is I'm going to give up this in order to get that and that's my trade-off right now but a person who says I'm going to use this and

I'm not going to give up any of that stuff is just fooling themselves because there isn't a complete interconvertibility between one medium and another how would you rate the computer I mean you've studied a lot of human history I mean is this something we've been privileged to live through is this really a very remarkable yeah well I don't think it and I think one way of rating the computer is to say it's definitely a thing like the printed book it is definitely in an unremarkable stage like 30 years after Gutenberg and almost certainly if its promises realize and it's not just turned into television because that's one of the things that can simulate as well but if it can if it can deal with all of the range of things that can deal with and people use it to express all of those things then it very likely will have the same kind of effect on

our civilization as the as the printed book did whether it has that effect on any given person though is a completely different question because as people have noted the Renaissance has come and gone and we have what we like are pleased to call civilization now and a large percentage of the population not just in third world countries but in our own country have never touched the 20th century as far as as its ideas are concerned and in spite of the all the libraries with all the books and all the things and what books have done to us a very large percentage of people have never been carried along with them and that is very likely to happen with the computer so we haven't really succeeded with the last major yeah I doubt it's probably the case that we never succeed with those things that civilization gets transformed and a certain critical mass of people get transformed by and they are the ones who transform the civilization and then for

one reason or another a large number of people don't get transformed by it but another way of thinking about it is if you take a look at what the average person who has not been transformed by it but who has gone to college today thinks about the world it is a little bit better a little bit richer I think than what people thought about the world in a thousand AD but there's a problem with this literacy thing you said because I know you think of television as like a medieval stained glass window but actually you can you don't need any you don't need to learn anything to do it right well you have to learn something because we know that when DW Griffith first invented close-ups and moving cameras and montages and stuff that the audience was a little bit startle but it didn't take long once one pass through it and they got the idea of what was what was going on and so the amount of learning in the visual domain is pretty

low compared to what you have to do in doing reading and writing and that is a that is a big barrier the biggest problem though I think is that many people believe that there is an in equilibrium between the two media that what you can say in a book you can say on television and all of the evidence is against that that what you can do with television are some very important things you can get people interested you can give them an emotional hit you can get them to know somebody in a way they didn't think they could do it before you can maybe get them to look interested enough to look deeper but it's very strength is its weakness is strength is its involvement its weakness is its involvement because in order in order to think scientifically about the world you have to be romantic it's true but you also have to be able to step back and

say gee I wonder what else it is looks this way but I wonder what else it is when what else is going on and I don't think television gets people to to do that kind of connect connected thought away from the dominance of the senses there's also a thing you write about the amplitude problem there's a thing that some media require more concentration than others don't they and the written media do right and some forms of television and some forms of gangs and so forth then it's the this and the question is will the computer disappear into the television yeah I'm not sure whether it requires more concentration but it it may very well be that like you know some people read well and it's nothing for them to read a book a day and other people this is a big deal I mean it's a struggle and it's what's likely is going on is that most people never learn to read very well and so the amount of concentration they have

to put into the mechanics is what defeats them with the amount of material that has to be read and I think it's much more like like that because it's remarkable how much concentration you have to put into something like tennis until you learn to play it or how much concentration you have to put into music on mechanics until you actually get fluent at playing but then then the hour once you've gone to that place and the hours go by without even realizing because you're deep into what the content of the medium is the last question is what's surprised you most about the history of computing what surprised me most well I think the I think the thing that surprised me the most is how long it is taken to get ideas out of laboratories into the commercial world of how many different kinds of inertia there are both for good reason and for bad reason but just the sheer amount of time where a decade after an idea is a very short period to see it emerging

in the commercial world that is certainly surprising to me because most of the scientists who work on these things work on them because they were obvious you're so obvious that they just want to make them and have them not be obvious and to have them have go something it was revolutionary obvious to have to go through an evolutionary stage that make 10 or 20 years is quite surprising. Do you think there's any way of cutting down that period? I don't see it right now because it's I think it's I think it's a question of having people grow up being interested in multiple points of view rather than being disturbed when they're shown something outside of their single way of looking at the world and our civilization is very prone towards single-minded ways of looking at the world we we come from a monotheistic religious background which says there's a right

and a wrong and you're either for God or against God and so forth and these attitudes trickle over into our reactions to everything for doing what we're doing now is right then something that's different from it can't possibly be right. The Japanese seem to be a little more flexible in some of those scores they have several religions existing side by side in Japan and many the people adhere to several of them at once they don't see any big conflict and I think any civilization that can treat ideas as interesting as more interesting in an array than as treated singly is going to is going to make it into the future