Watch the CEO of Google take a stab at defining the new generation of the Web:

What do you think of his definition?

A couple of weeks ago, I wrote two pieces on Web 3.0: ‘Semantic Web Part I: The Data‘, which covered the way information on the Semantic Web differs from the way we currently access information, and ‘Semantic Web Part II: The Interface‘, which discussed the challenges of processing the extraordinary quantities of information made available through a data-driven framework.

Let’s assume that you’ve read the first two posts and agree with the following basic premises:

1. The web of the future will offer information as data that can be manipulated in a virtually infinite number of ways. This may or may not be the RDF framework! But the amount of information and our ability to manipulate it will continue to grow exponentially.
2. If people want to be able to elegantly access the richness of this vast realm of data, they will need a means of interacting with it that offers an optimum level of choice.

If you don’t agree with either of those two statements, please let me know below! If you do, though, it’s time to delve further into that second point: a means of interacting with it that offers an optimum level of choice. What this means is that we want people to have unlimited flexibility, accessible in a simple enough manner that people can get their heads around it.

This may sound pretty straightforward, but in reality it’s the next great challenge of the Internet. Remember in Hitchhiker’s Guide to the Galaxy there was that torture machine that showed you the infinite vastness of the universe and your own size relative to it? Fried people’s brains (except Zaphod Beeblebrox—he was just proven to be as great as he always thought he was).

The Web is its own universe, and our minds would likely collapse if we tried to perceive its vastness. Without a meaningful interface and a meaningful filter, we would be utterly lost in information overload. As any artist can tell you, the value of the work lies as much in what is left out as in what is kept in.

The word for what should be kept in and what should be left out is ‘relevance’, and there are nearly as many different approaches to relevance as there are websites. Some people use demographics. Some people use history. Some people use social tagging. Some use contextual recommendations.

VortexDNA’s approach is to use the humanity of the individual.

In reality, we carry our own filters everywhere we go. This can be shown objectively; for example, we can only see a small part of the light spectrum, and we can only hear a tiny fraction of sound waves. Of that tiny fragment that we are capable of perceiving, we only consciously register yet another fraction, and we remember a smaller portion still.

Those limitations on our perception and awareness are not negative things. They are highly necessary boundaries that allow us to process the world we live in. Without them, we would likely go insane.

Our inherent filters also extend to who we are as people. Why do birds of a feather flock together? Because they can all see each other; they’re all using the same filter. I don’t mean ’see’ literally, of course; I mean ‘understand’.

VortexDNA’s technology works with these filters mathematically and brings them online. The same filters that allow us to process the physical world that we live in without losing our minds can be applied to our webworlds.

The Semantic Web can offer a richness of information beyond what our minds are capable of digesting. We place boundaries on how much we can take in by using effective interfaces and external relevance systems. The next step is for us to become the filter, to allow us to see those parts of the Web most aligned with who we are.

Does this sound like a useful framework? Do you believe that you create the world around you? Or is it a bit too airy-fairy for you? Either way, I’d be delighted to hear your thoughts.

Last week, I wrote a post describing the data behind the Semantic Web. The basic premise was that the Semantic Web breaks all of our information down into little bits that we can manipulate as much as we want. As I’ve said before, though, having that much data can be messy. In order for the Semantic Web to be able to serve us rather than the other way around, we need an effective interface that allows us to navigate intuitively through a sea of infinite information.

The choice paradox
We tend to think choice is good, and the more choice the better. Our experiences, however, tell us otherwise. In the book ‘Blink’, Malcolm Gladwell tells the story of an experiment conducted by Sheena Iyengar:

She once conducted another experiment in which she set up a tasting booth with a variety of exotic gourmet jams at the upscale grocery store Draeger’s in Menlo Park, California. Sometimes the booth had six different jams, and sometimes Iyengar had twenty-four different jams on display. She wanted to see whether the number of jam choices made any difference in the number of jams sold. Conventional economic wisdom, of course, says that the more choices consumers have, the more likely they are to buy, because it is easier for consumers to find the jam that perfectly fits their needs. But Iyengar found the opposite to be true. Thirty percent of those who stopped by the six-choice booth ended up buying some jam, while only 3 percent of those who stopped by the bigger booth bought anything. Why is that? Because buying jam is a snap decision. You say to yourself, instinctively, I want that one. And if you are given too many choices, if you are forced to consider much more than your unconscious is comfortable with, you get paralyzed. Snap judgments can be made in a snap because they are frugal, and if we want to protect our snap judgments, we have to take steps to protect that frugality.

Think of a situation in which the amount of consideration that had to go into a purchase seemed overwhelming—car-buying is a good one for many people. I remember the very first car I bought. I agonized over it for weeks. In the case of cars, the paralysis is for a different financial reason than it is for jam: you’re spending a good amount of money, and so you want to be sure to make the right decision. But the level of expertise required to make a ‘right’ decision in such a vast realm of choices leaves most of us feeling uninformed and uncertain about our ultimate selection. This, I suspect, is where ‘buyer’s remorse’ comes from: the inability to be sure that one option among millions is correct.

Not to have choice is anathema to most of us. Too much choice can paralyze us. Clearly, there’s a bell curve here, a point at which users and consumers have an optimum number of choices: enough that they can feel independent, not so much that they become despondent.

The paradox of choice will become increasingly important in the context of the Semantic Web. If all of the information on the Web is available as infinitely manipulatable data, how can we find the optimum point on the choice bell curve? I believe the answer lies in the interface.

Visible choices
A user interface is the means by which people can access the choices available to them. It essentially offers the user that part of the iceberg that is above water. By selectively revealing options that reveal more and more as the user progresses, an interface can deliver infinite choice in appropriate-size chunks for users to process.

Wizards (like mail merge wizards) are a great example of how a user interface can be used to make many choices seem manageable. I’ve been managing the development of a web tool that allows users to create promotional pdfs based on pre-set templates. The users have 36 possible templates to choose from, but our tool doesn’t show them 36 templates, or 18, or six. It shows them three. Then it asks if they want black & white or color. Then it asks which of three themes they want. Then it asks if they want a bold or minimalist design. Each of these questions is a ‘jam choice’, a snap decision from a small pool of options. Taken together, though, they lead the user comfortably down a path towards a confident decision.

If all of us who create the Web—every site owner, developer, widget maker, and blogger—adapt our material to the standards-compliant RDF framework, the fertile entrepreneurial ground will lie in creating interfaces that allow users to access the richness that is the Semantic Web without losing their minds. Sites like FaceBook encounter the challenge with thousands of personalization options; Google, inasmuch as they’ve got News and Blogs and Images along the top, are still, at their core, a single page with a simple text box. The lack of choices on the page allows users to feel comfortable with the infinite choices available in the query.

I had intended to touch on VortexDNA’s contribution to the interface in this post, but I suspect it’s getting rather long-winded, so I shall park it for another day. For now, how much choice do you want in your personal Web experience, and what do you think is the best way for providers to address varying preferences?

The best and the brightest Internet-oriented minds, of people like Sir Tim Berners-Lee, are talking about the Semantic Web and Web 3.0, but most of us either shut off or shut down when we hear the terms. They’re simply so nebulous that it’s easier to reject them by saying that they’re ‘mere hype’. I’ve already had a crack or two at it, but I’m still mulling the concepts over, and I’m going to give it another go here.

First, let’s review what we mean by Web 3.0. There are lots of theories out there, including a web that allows read-write-execute or one that is always on and controllable.

For purposes of this post, I’m going with Sir Tim Berners-Lee’s Semantic Web definition. Essentially, the Semantic Web is comprised of data, rather than documents. The difference may seem subtle, but it’s profound.

The Word Web
When information is on the web in the form of documents, the users rely on a programmer or developer having put information together in a certain way. I can use this blog to demonstrate what I mean:

3+3=6

I’ve just written that, and if you happened to want to know what 3+3 is you’d be in luck, but if you wanted 3×3, oh well. Because it’s text and not data, you are relying on what I wrote and have no option to adapt it to your needs.

If you write a formula in Microsoft Word, like 3+3=6, you can search for it, but you can’t do anything with it. You can’t change it to 3×3 or 3-3. You can’t take the result and apply further calculations to it. You’re pretty much stuck with what you’ve got.

In the post I referred to above, I included a video of Berners-Lee describing the Semantic Web, in which he uses the example of coffee shops in the neighborhood. In the Word version of the Web, I wait until someone has compiled a list, or overlaid little coffee mug icons on a map, and then I can reference the information. But I can’t just decide to switch to nightclubs; for that, I have to wait until someone compiles another list, or overlays little DJ icons on a map, or whatever.

The Excel Web
If you wanted to manipulate numbers, you wouldn’t use Word; you’d use Excel. You’d break your information down into little bits so you can put the numbers together however you want. This is the same concept Tim Berners-Lee is talking about with RDF (the Resource Description Framework).

If we were just talking about numbers, then Excel would give us all the manipulation power we need. But on the Web we’re not just talking about numbers; we’re talking about information. So perhaps a better analogy is a database program like Access.

Word=no manipulation.
Excel=moderate manipulation.
Access=massive manipulation.

The Access Web
Those of you who use Access know that it is not an intuitive program. If you’ve got no experience with relational databases, you’re likely to have no idea what you’re looking at and no concept that the structural decisions you make at the beginning can have a big impact on what you’re able to do at the end. But you’ll also know how powerfully you can manipulate the data, at a moment’s notice, on an enormous scale, without running into those problems you do in Excel (’OMG, I sorted my data and now all of my VLOOKUPS don’t work!)

If information is on the web as data, it becomes infinitely accessible. You can add it, multiply it, overlay it, correlate it… manipulate it however you want. You can switch from coffee shops to nightclubs.

The Interface
In order for Access to be useful to mere mortals and not just the technogeek gods, we have to construct forms and pre-saved queries and user interfaces. Having that much data and that much ability to manipulate can be messy. Same with the Semantic Web. In Part II, I’ll talk about the interface, which is where VortexDNA comes in.

Do you think this is a reasonable explanation? Too simplistic? Delightful? Misinformed? I welcome your comments.

Want to see something incredible? Got seven minutes?

For techheads, this video is a phenomenal representation of what can happen when software takes advantage of collective information. Why does Photosynth work? It’s not just because it’s spectacular technically; it’s because it uses the community as its data source.

As Marc Andreessen writes in his blog:

No single closed service, no matter how good, and no matter how big, could compete with the diversity of thousands and then millions of web sites that were customized to every conceivable user interest and need.

Marc was referring to FaceBook and the ability of developers to create custom apps on their platform, but I think that statement can be easily applied to this general concept:

Software and products that take advantage of our collective dataset and creativity will always be more powerful than software with a private dataset.

If you don’t believe me, try Apple vs. PC, Britannica vs. Wikipedia, or, for that matter, Web 2.0 vs. Web 1.0.

For car buffs, notice anything about the content of the Bimmer ad? Like that it doesn’t refer to BMW at all? The German auto manufacturer quashed rumors last year about being interested in buying Aston Martin, but I can’t imagine that this ad could be a coincidence or a mistake, especially since it’s at a conference sponsored by BMW.

Ford recently sold their stake to a consortium including some Middle East heavyweights—could they be looking to flip it?

Following a post in which I had commented on Google’s and VortexDNA’s approaches to Web 3.0, a reader wrote in asking what other solutions are out there.

Fortunately, ZDNet’s David Berlind interviewed Sir Tim Berners-Lee last Wednesday at the MITX (Massachusetts Innovation and Technology Exchange) Technology Awards, and then obligingly posted the video on his blog. Berners-Lee gives a convincing and relatively plain English (he does use the term ‘data bus’) explanation of the Semantic Web that’s been his pet project—I found it worth the 11 minutes I spent watching it.

One of the many things he touched base on was the difference between the Semantic Web and mash-ups of APIs. Basically, he explained, with mash-ups, people have to intentionally take two different databases and decide to put them together. An example might be someone taking a database of maps and a database of coffeehouses and slapping one on the other so you get maps of coffeehouses.

By contrast, Berners-Lee’s vision of the Semantic Web is that all of the information is just data, accessed by the universal query language SparQL. Once the work has been done to convert documents to standardized data, anyone can pull up anything or combine it in any fashion to see relationships.

In his writeup of the interview, though, Berlind raised some doubts as to whether it would be to the benefit of webcos to be on board with the data standards being proposed by Berners-Lee.

Given the popularity of API-driven access, in the back of my mind, I couldn’t help wonder if there wasn’t a bit of a race going on. On one side, there’s the W3C with the work its doing on the Semantic Web (based very much on something known as RDF or the Resource Description Framework).

On the other, a lot of big Internet companies would probably prefer developers go the non-standard API route because of the way API-dependencies can result in developer loyalty (ok, “lock-in”). After all, once code is written and reliant on APIs (and it works), API extrication (in favor of using SparQL against RDF) will invariably entail a rewrite. That is unless developers are anticipating the Semantic Web and modularizing their code in such a way that they have query modules that abstract query specifics. In that case, so long as the module returns the same information, it’s only the guts of the module that have to be fixed (trust me, it’s much more complicated that I’m making it seem).

Berlind isn’t the first to raise these questions—they cut to the heart of the Semantic Web vs. Semantic Search debate. It reminds me of the early days of Mac against PC. Yes, in that case the clear winning strategy was to open the standards to everyone, but in that case the standards hadn’t been defined or entrenched. To achieve Berners-Lee’s proposition, billions of web contributors would have to shift the way they operate. It will take a powerful tipping point to get there.

Do you think it will be possible to establish universal standards, like RDF, to make web data infinitely accessible? Or is the smarter bet on semantic tools to process non-semantic data?