The Internet is built wrong.
Well, OK, not so much “wrong” as “not the best it could be.” But for all its problems, we can’t rationalize a radical departure from the technologies that currently comprise the Internet. Why?
There’s a software engineering philosophy called Worse Is Better that’s been around for a good 18 years. In that time, it’s come to mean that an inferiorly designed system or piece of software may be more successful than its better-designed competitor, based on subjective or unexpected market criteria. A perfect modern example is that, while most Web applications are poor facsimiles of their desktop counterparts, they’re successful because there’s nothing to install and collaboration is a snap.
The Internet is the “Worse Is Better” poster child. We now know, for example, that IPv4 won’t scale to the projected size of the future Internet. We know too that near-universal deployment of technologies with inadequate security and trust models, like SMTP, can mean millions if not billions lost to electronic crime, defensive measures, and reduced productivity.
We know these things now, but it doesn’t matter. The Internet as we know it is long since “in production”; it works most of the time for most people; and, above all else, it makes enough money that it seems foolish to question whether or not it’s acceptably engineered. Worse is better when worse is profitable.
In a keynote speech in 1997, Alan Kay affirmed the success of Arpanet, the Internet’s predecessor, on which he worked. Yet almost as an aside, he decimated our conceptions of the worth of standard Web technologies, such as HTML: “HTML on the Internet has gone back to the dark ages because it presupposes that there should be a browser that understands its formats... You don’t need a browser, if you followed what this Staff Sergeant in the Air Force knew how to do in 1961. You just read [data] in. It should travel with all the things that it needs, and you don’t need anything more complex than something like X Windows.”
“Worse Is Better” happens because systems are multi-layered. Users interact with the topmost layers of a system, and this superficial interaction characterizes their perception of the system’s value. In the case of the Internet, users valued easy publishing of information (the layer provided by HTTP and HTML). But the engineering value in the Internet is in its scalable, resilient design, not the surface-layer technologies that have come to comprise the Web. But the Internet’s market value is now tied up in the Web. Worse is better.
So when will better actually be better?
Van Jacobson, an award-winning specialist in networking to whom the Internet owes its continued existence, gave a talk at Google in 2006 outlining a content-centric approach to networking. Jacobson’s approach leverages wide distribution of data across a variety of devices and media, while baking in security and simplifying the development model for networked applications.
If the majority of Internet usage continues to be about content, an approach like Jacobson’s would be not just prudent, but necessary. You needn’t do more than attempt to watch a streaming video on a busy office LAN or oversubscribed DSL circuit to understand that even the best-served markets for Internet connectivity are struggling to keep up with demand for networked content. Add to this that providing adequate security models for such content is a virtual impossibility on today’s Internet, and the need for a better approach is even clearer.
Yet it’s only seasoned engineers like Kay and Jacobson that have the guts and experience to question the decisions behind today’s popular Internet technologies. The majority is content to subscribe to the iterative philosophy of “Worse Is Better”: As users demand it, small improvements will be made. Users will be so conditioned to accepting mediocrity that even iterative improvements will seem like miracles. Technologists are then rewarded as miracle-workers for delivering the bare minimum.
We know the Internet could be better. We can quantify what we lose daily because it isn’t. The question remains: What will it take to push us forward?
— Alex Payne is API Lead at Twitter, where he works on the site’s stability, security, and developer outreach.
This blog is part of Internet Evolution’s IT Clan, which addresses the continuing impact of the Internet on enterprise networks, applications, and management. Register here to join the IT Clan’s conversation, and you just might win something unspeakably cool.