It’s Time to Reinvent the Personal Computer

“In chaos, there is opportunity.”
—Tony Curtis, Operation Petticoat (and also Sun Tzu)

“Chaos” is a pretty good word to describe the personal computer market in 2013. Microsoft is trying to tweak Windows 8 to make it acceptable to PC users (link), its Surface computers continue to crawl off the shelf (link), PC licensees are reconsidering their OS plans and business models (link), and Apple’s Macintosh business continues a genteel slide into obscurity (link, link).

No wonder many people say the personal computer is obsolete, kaput, a fading figment of the past destined to become as irrelevant as the rotary telephone and steam-powered automobile (link).

I beg to differ. Although Windows and Macintosh are both showing their age, I think there is enormous opportunity for a renaissance in personal computing. (By personal computing, I mean the use of purpose-built computers for productivity tasks including the creation and management of information.) I’m not saying there will be a renaissance, because someone has to invest in making it happen. But there could be a renaissance, and I think it would be insanely great for everyone who depends on a computer for productivity.

In this post I’ll describe the next-generation personal computing opportunity, and what could make it happen.


What drives generational change in computing?

Let’s start with a bit of background. A generational change in computing is when something new comes along that makes the current computing paradigm obsolete. The capabilities of the new computers are so superior that previous generations of apps and hardware are instantly outdated and need replacement. Most people would agree that the transition from command line computers to graphical interface (Macintosh and Windows) was a great example of generational change.

What isn’t as well understood is what triggered the graphical interface transition. It wasn’t just the invention of a new user interface. The rise of the Mac and Windows was driven by a combination of factors, including:

—A new pointing device (the mouse) that made it easier to create high-quality graphics and documents on a computer.
—Bitmapped, full-color displays that made it easy for computers to display those graphics, pictures, and eventually video. Those displays also made it easier to manage file systems and launch apps visually.
—New printing technology (dot matrix and laser printers) that made it easy to share all of those wonderful new documents and illustrations we were creating.
—A new operating system built from the ground up to support these new capabilities.
—An open applications market that enabled developers to turn all of these capabilities into compelling new apps.

All of those pieces had been around for years before graphical computing took off, but it wasn’t until Apple integrated all of them well, at an affordable price, that the new paradigm took off. The new interface and new hardware, linked by a new or rebuilt OS, let us work with new types of data. That revolutionized old apps and created whole new categories of software.

Windows and Mac took off not because they were new, but because they let us do new things.

Although later innovations, such as the Internet, added even more power to personal computing, it’s amazing how little its fundamental features and capabilities have changed since the mid-1990s. Take a computer user from 1979 and show them a PC from 1995, and they’ll be completely lost in all the change. Take a computer user from 1995 and show them a PC from 2012 and they’ll admire the improved specs but otherwise be feel very much at home.
   
Maybe this slowdown in qualitative change is a natural maturation of the market. After an early burst of innovation, automobiles settled down to a fairly standard design that has changed only incrementally in decades. Same thing for jetliners.

But I think it’s a mistake to look at personal computers that way. There are pending changes in interface, hardware, and software that could be just as revolutionary as graphical computing was in the 1980s. In my opinion, this would be a huge opportunity for a company that pulls them all together and makes them work.


Introducing the Sensory Computer

I call the new platform sensory computing because it makes much richer use of vision and gestures and 3D technology than anything we have today. Compared to a sensory computer, today’s PCs and even tablets look flat and uninteresting.

There are four big changes needed to implement sensory computing.

The first big change is 3D. Like desktop publishing in the 1980s, 3D computing requires a different sort of pointing device, new screen technology, and a new kind of printer. All of those components are available right now. Leap Motion is well into the development of gesture-based 3D control. 3D printers are gradually moving down to smaller sizes and more affordable price points. And 3D screens that don’t require glasses are practical, but have a limited market today because we keep trying to use them for televisions, a usage that doesn’t work with the screen’s narrow viewing angle.

But guess what sort of screen we all use with a very narrow viewing angle, our heads perched right in front of it at a fixed distance? The screen on a notebook computer.

Today we could easily create a computer that has 3D built in throughout, but we lack the OS and integrated hardware design that would glue those parts together into a solution that everyone can easily use.

You might ask what the average person would do with a 3D computer. Isn’t that just something for gamers and CAD engineers? The same sort of question was asked about desktop publishing in the 1980s. “Who needs all those fonts and fancy graphics?” many people said. “For the average person Courier is just fine, and if I need to send someone an image I’ll fax it to them.”

Like that skeptical computer user in the 1980s, we don’t know what we’ll do when everyone can use 3D. I don’t expect us to send a lot of 3D business letters, but it sure would be nice to be able to create and share 3D visualizations of business data and financial trends. I’d also like to be able to save and watch family photos and videos in 3D. How about 3D walkthroughs of hotels and tourist attractions on Trip Advisor? The camera technology for 3D photography exists; we just need an installed base of devices to edit and display those images. And although I don’t know what I’d create with a 3D printer, I’m pretty sure I’d cook up something interesting.

Every time we’ve added a major new data type to computing, we’ve found compelling mainstream uses for it. I’m confident that 3D would be the same.

The second big change is modernizing the UI. User interface is ultimately about increasing the information and command bandwidth between a person and a computer. The more easily you can get information in and out of the computer, the more work you can get done. The mouse-keyboard interface of PCs, and the touch-swipe interface of tablets, were great in their time, but dramatically constrain what we can do with computers. We can do much better.

The first step in next-generation UI is to fully integrate speech. This doesn’t mean having everything controlled by speech, but using speech technology where it’s most effective.

Think about it: What’s the fastest way to get information in and out of your head? For most of us, we can talk faster than we can type, and we can read faster than we can listen to a spoken conversation. So the most efficient UI would let us absorb information by reading text on the screen, but enter information into the computer by talking. Specifically, we should:
—Dictate text to the computer by via speech, with an option to use a keyboard if you’re in public where talking out loud would be rude.
—Have the computer present information to us as printed text on screen, even if that information came over the network as something else. For example, the computer should convert incoming voice messages to text so you can sort through them faster.

We can do all of these things through today’s computers, of course, but the apps are piecemeal, bolted on, and forced through the funnel of an old-style interface. They’re as awkward as trying to do desktop publishing on a DOS computer (something that people did try to do for years, by the way).

Combine speech with 3D gestures and you’ll start to have a computer that you can control very richly by having a conversation with it, complete with head nods and waves of the hand. Next we’ll add the emerging science of eye tracking. I’m very impressed by how much progress computer scientists are making in this area. It’s now possible to build interfaces that respond to the things we look at, to facial expressions, and even to our emotional response to the things we see. This creates an incredibly rich (and slightly creepy) opportunity to build a computer that responds to your needs almost as soon as you realize them.

Once we have fully integrated speech, gesture recognition, and eye tracking, I’m not sure how much we’ll need other input technologies. But I’d still like to have the option to use a touchscreen or stylus when I need precision control or when a task is easier to do manually (for example, selecting a cell in a spreadsheet or drawing something). And as I mentioned, you’ll need a keyboard option for text entry in public places. But these are backups, and a goal of our design should be to make them options rather than a part of the daily usage experience.

The third change is a new paradigm for user interaction In a word, it’s time to ship cyberspace. The desktop metaphor (files and folders) was driven by the capabilities of the mouse and bitmapped graphics. The icons and panels we use on tablets are an adaptation to the touchscreen. Once we have 3D and gesture recognition on a computer, we can rethink how we manage it. In the real world, we remember things spatially. For example, I remember that I put my keys on my desk, next to the sunglasses. We can tap into that mental skill by creating 3D information spaces that we move through, with recognizable landmarks that help to orient us. Those spaces can zoom or morph interactively depending on what we look at or how we gesture. Today’s interface mainstays such as start screens and desktops will be about as relevant as the flowered wallpaper in grandma’s dining room. Computer scientists and science fiction authors have played with these ideas for decades (link); now is the time to brush off the best concepts and make them real.

The fourth change is to modernize the computing ecosystem. The personal computer software ecosystem we have today combines 20-year-old operating system technology with a ten-year-old online store model created by Apple to sell music. There’s far more we could do to make software easy to develop, find, and manage. The key changes we need to make are:

—The operating system should seamlessly integrate local and networked resources. Dropbox has the right idea: you shouldn’t have to worry about where your information is, it should just be available all the time. But we should apply that idea to both storage and computer processing. We shouldn’t have web apps and native apps, we should just have apps that take advantage of both local computing power and the vast computational resources of the web. An app should be able to run some code locally and some on a server, with some data stored locally and some on the network, without the user even being aware of it. The OS should enable all of that as a matter of course.

In this sense, the advocates of the netbook have it all wrong. The future is not moving your computing onto the network; it’s melding the network and local computer to produce the best of both worlds.

—Discovery needs work. App stores are great for making apps available, but it’s very hard to find the apps that are right for you. Our next generation app store should learn your interests and usage patterns and automatically suggest applications that might fit your needs. If we do this right, the whole concept of an app store becomes less important. Rather than you going to shop for apps, information about apps will come to you naturally. I think we’ll still have app stores in the future because people like to shop, but they should become much less important: a destination you can visit rather than a bottleneck you must pass through.

—Security should be built in. The smartphone operating systems have this one right: each app should run in a separate virtual sandbox where malicious code can’t corrupt the system. No security system can be foolproof, but we can make personal computers far more secure than they are today.

—Payment should be built in as well. This is the other part of the software and content ecosystem that’s broken today. Although the app and content stores have made progress, we’re still limited to a small number of transaction types and fixed price bands. You can’t easily sell an app for half a cent per use. You can’t easily sell a software subscription with variable pricing based on usage. As an author, you can’t easily sell an ebook for more than $10 or less than 99 cents without giving up 70% of your revenue. And you can’t easily sell a subscription to your next ten short stories. Why? Because the store owners are manipulating their terms in order to micro-manage the market. They mean well, but the effect is like the worst dead-hand socialist market planning of the 1970s. The horrible irony is that it’s being practiced by tech companies that loudly preach the benefits of a free market.

It’s time for us to practice what we preach. The payment system should verify and pass through payments, period. Take a flat cut to cover your costs and otherwise get out of the way. The terms and conditions of the deal are between the buyer and the creator of the app or content. Apple or Google or Amazon has no more business controlling what you buy online than Visa has controlling what you buy in the grocery store. The free market system has been proven to produce the most efficiency and fastest innovation in the real world; let’s put it to work in the virtual world as well.

Adding it up. Let’s sum up all of these changes. Our next-generation computer now has:
—A 3D screen and 3D printing support built in, with APIs that make it easy for developers to take advantage of them.
—Speech recognition, gesture recognition, and eye tracking built in, with a new user interface that makes use of them.
—A modernized OS architecture that seamlessly blends your computer and the network, while making you more secure against malware.
—An app and content management system that makes it easy for you to find the things you like, and to pay for them in any way you and the developer agree to.

I think this adds up to a new paradigm for computing. It’s at least as revolutionary as the graphical computing revolution of the 1980s. We’ve opened up new usages for your computer, we’ve enabled developers to revolutionize today’s apps through a new interface paradigm, and we’ve made it much easier for you to find apps and content you like.

Why can’t you just do all this with a tablet? You could. Heck, you could do it with a smartphone or a television set. But by the time you finished adding all these new features and reworking the software to make full use of them, you would have completely rebuilt the whole device and operating system. You’ll no longer have a cost-efficient tablet, but you’ll still have all the flaws and limitations of the old system, jury-rigged and still adding cost and inefficiency. Windows 8, anyone?

It’ll be faster and cheaper just to design our new system from scratch.


When will we get a sensory computer?

If you agree that we’re overdue for a new computing paradigm, the next question is when it’ll arrive. Unfortunately, the answer is that it may not happen for decades. Major paradigm changes in technology tend to creep forward at a snail’s pace unless some company takes on the very substantial task of integrating and marketing them. Do you think ebooks would be taking off now if Amazon hadn’t done Kindle? Do you think the tablet market would be exploding now if Apple hadn’t done the iPad? I don’t think so, and the proof is that you could have built either product five years earlier, but no one did it.

So the real question is not when we’ll get it, but who might build it. And that’s where I get stuck.

Microsoft could do it, and probably should. But I doubt it will. Microsoft is so tangled up now in tablet computing and Windows 8 that I find it almost impossible to believe that it could take on another “replace the computer” initiative. I think there’s a very good argument that Microsoft should have done a sensory computer instead of Windows 8, but now that the decision’s made, I don’t think it can change course.

Google could do it, but I don’t think it will. Google is heavily invested in the Chrome netbook idea. It’s almost a religious issue for Google: as a web software company, the idea of a computer that executes apps on the web seems incredibly logical, and is emotionally attractive. Google also seems to be hypnotized by the idea that reducing the cost of a PC to $200 will somehow convert hundreds of millions of computer users to netbooks. I doubt it; PC users have been turning up their noses for decades at inexpensive computers that force them to compromise on features. The thing they will consider is something at the same price as a PC but much more capable. But I don’t think Google wants to build that.

One of the PC companies might take a stab at it. Several PC companies have tried from time to time to sell computers with 3D screens. Theoretically, one of those companies could put together all the features of a sensory computer. I think HP is the best candidate. It already plans to build the Leap Motion controller into some of its computers, and I can imagine a beautiful scenario in which HP creates a new ecosystem of sensory computers, low-cost home 3D printers that render in plastic, and service bureaus where you can get your kid’s science fair design converted to aluminum (titanium if you’re rich). It would be glorious.

But it’s not likely. To work right, a sensory computer requires a prodigious amount of new software, very careful integration of hardware and software features, and the courage to spend years kick-starting the ecosystem. I don’t think HP has the focus and patience to do it, not to mention the technical chops, alas. Same thing for the other PC companies.

Meg, please prove me wrong.

Apple is the company that ought to do it, but does it have the will? Apple has the expertise and the market presence to execute a paradigm change, and its history is studded with market-changing products. I love the idea of Apple putting a big 3D printer at the back of every Apple store. Maybe you could let Sensory Mac users sell their designs online, with pickup of the finished goods at any Apple store, and Apple (naturally) taking a 30% cut...

But I’m not sure if today’s Apple has the vision to carry out something like that. The company is heavily invested in smartphone and tablet computing, with an ongoing hobby around reinventing television. There’s not much executive bandwidth left for personal computing. The company’s evolution has taken it steadily toward mobile devices and entertainment, and away from productivity.

Think of it this way: If Apple were really focused on personal computing innovation, would it be letting HP take the lead in integrating the Leap Motion controller? Wouldn’t it have bought Leap Motion a year ago to keep it away from other companies?

I think personal computing is a legacy market to Apple, an aging cash cow it’ll gently milk but won’t lead. I hope I’m wrong.

We’re out of champions, unless...  At this point we’ve disposed of most of the companies that have the expertise and clout to drive sensory computing. I could make up scenarios in which an outlier like Amazon would lead, but they’re not very credible. I think the other realistic question is whether a startup could do it.

It’s hard. Conventional wisdom says that you need $50 million to fund a computer system startup, and that sort of capital is absolutely positively not available for a company making hardware. But I think the $50 million figure is outdated. The cost of hardware development has been dropping rapidly, driven by flexible manufacturing processes and the ability to rapidly make prototypes. You could theoretically create a sensory computer and build it in small lots, responding to demand as orders come in. This would help you avoid the inventory carrying costs that make hardware startups so deadly.

The other big barrier to hardware startups has been the need to get into the retail channel, which requires huge investments in marketing, sales support, and even more inventory. Here too the situation has been changing. People today are more willing to buy hardware online, without touching it in a store first. And crowdfunding is making it more possible for a company to build up a market before it ships, including taking orders. That business model actually works pretty well today for a $100 gizmo, but will it work for a $2,000 productivity tool?

Maybe. I’m hopeful that some way or another we’ll get a sensory computer built in this decade. At this point, the best chance of making it happen is to talk up the idea so one or more companies will make it happen. Which is the point of this post.

[Thanks to Chris Dunphy for reviewing an early draft of this post. He fixed several glaring errors. Any errors that remain are my fault, not his.]


What do you think?  Is there an opening for a sensory computer? Would you buy one? What features would you want to see in it? Who do you think could build it? Please post a comment and share your thoughts.