I just read a post called ‘Trust is the new vertical search‘ on SearchAnyway. The topic was a new venture called WeTrust, which is, in their words, “a safe shopping search and directory for web sites that have secure shopping facilities.”

1. Your site must have a secure server with valid digital certificate so it can encrypt all customer personal information, including credit card information.

   By having a secure server, your site will be using technologies such as SSL (Secure Socket Layer) or SET (Secure Electronic Transmission) ensuring that sensitive information such as personal details passed between your customer’s computer and your computer cannot be read by a third party.

   Visitors to your site should be able to see that this technology is in place by seeing the locked padlock when reaching the order page, as illustrated below:

   

2. Visitors should be able to see “https” at the start of the address bar on their browser when completing your order form page. The "s" in "https" stands for secure.

So basically, if you’ve got a valid Thawte certificate or equivalent, and pay them £99 a year, you can be on this directory. I was moved to leave the following comment:

While I applaud the idea of pre-vetted shopping sites that are trustworthy, it seems to me that having a secure server with a valid digital certificate is a pretty minor requirement. Just because data is safe while in transit doesn’t mean that a site won’t then do something unscrupulous with it.

I think your post brings a bigger issue to light, though. I love the title ‘Trust is the new vertical search’. Wouldn’t it also be fair to take that a step further? Something like, ‘Trust is the new e-currency’? People are fearful of Google’s personalization efforts, they’re worried that they’re getting biased info on Wikipedia, and you just can’t find a good deposed Nigerian ambassador to invest in anymore. Trust is worth more than gold online these days. In fact, I’m going to write a post about this… I think it’s that important!

So here I am… living up to my word by writing this post. And I do think it’s important, so I’m going to say it again, in bold:

In a world with few boundaries, trust is the only currency worth having.

Why is trust so important? Other than the obvious, feel-good, look-I’ll-count-to-three-then-fall-back-and-you-catch-me sort of team-building aspect of it?

Because trust is not a touchy-feely, airy-fairy commodity. Trust is what drives the world economy, and without it, our entire financial system would collapse.

I’ve been reading a lot about the demise of various financial institutions thanks to Marc Andreessen’s accessible commentary; we’ve had five investment companies go under in the past three months here in New Zealand. Yes, many poor decisions were made. Yes, the economics of it didn’t work. Yes, the people who ran the funds really screwed things up. But the only reason these funds were able to operate in the first place was because people trusted them, and the reason they are going under now is because people don’t trust them.

Trust me (hah!)—if every American went to the ATM right now and withdrew their money, the banks would collapse, followed closely by everyone else. Trust is the lifeblood of any economic system.

I had the interesting experience as a child of visiting Argentina several years in a row during one of their hyperinflationary runs. The first year I went, the exchange rate was 14 Australes to the US dollar. The second year, it was 1,400. The third year, 2,500, and the fourth year it hit 10,000 and they changed to the peso.

Living in an economy where there’s no trust in the currency is a different way of life. There were no price tags on anything in the shop—there were codes, and the shop assistant looked up your item on an intricate matrix that included the most up-to-the-minute exchange rate. As soon as you made a purchase, the shop would close so the attendant could run to the bank and exchange the cash for dollars; if they waited until the end of the day, they could lose 30%.

So the economy is one thing that’s utterly dependent on a bedrock of trust. Online, though, you could say that your wealth is defined by the trust you engender.

If you want to sell something on eBay, you better have a trustworthy profile. If you want to be a power user on Digg, you have to build up trust. If you want to have a blog that people care about (I do, I do!), then you have to earn people’s trust and keep earning it.

If someone said to me, “Look, I’ve got a blog and a bunch of money,” that wouldn’t be very interesting. But if someone said, “Look, I’ve got a blog and 50,000 loyal visitors who trust what I say,” then I would perk right up. Online, it doesn’t matter where you are, what kind of accent you have, whether you’re male or female, or how much money you’ve got in the bank. What matters is how much people trust you.

Do you agree? Or do you think traffic trumps trust? And how do you decide who you’re going to trust on the Internet? I’d really like to hear from you.

It started with a challenge.

Some of the most creative minds, from the front lines of search, contemplated in silence.

Without rules or regulations, they created their visions of what search would look like in 2010 and beyond, and committed them to paper.

Over the past week, I’ve run many of the individual pieces, and I am pleased and proud to present to you the complete report:

The Future of Search from the Rising Star Dream Team

The complete report includes the synopsis, which draws together some of the recurring themes that emerged. It also includes additional pieces by Ephraim Schwartz of Reality Check | InfoWorld and Ran Geva from Omgili.

Here is the table of contents:

1. Introduction and synopsis
2. Andrew Matthews
3. Branton Kenton-Dau
4. Raf Manji
5. Nitin Karandikar
6. Charles Knight
7. Ephraim Schwartz
8. Ran Geva I
9. Ran Geva II

Read it. Peruse it. Enjoy it. Discuss it. Pass it on!

And now the challenge is extended to you. What is your reaction? What is your vision? What do you see as the next generation of search?

Comment here. Send me an email. Talk about it on your own blog. Express yourself in whatever way is best suited to you, but above all I ask you: participate.

The Web is nothing without you.

This piece was written by Raf Manji, of VortexDNA and Sustento. It is the sixth installment in the Rising Star Dream Team Future of Search series. The complete report, which includes additional contributions from Ephraim Schwartz and Ran Geva, will be available for download later today.

The major change in the area of search in 2012 will be the name itself. No longer will it be known as search; it will be known as receive. Yes, that’s right. We will no longer search for stuff—we will simply receive what we need. Search is an external process: I am looking for something. Receive is an internal process: I have everything I need inside me.

The world of receive will be simple, painless and wonderful. With the right personalisation processes you will simply find that stuff finds you and not the other way around. Search is so laborious and often you have to wade through lots of irrelevant and meaningless rubbish until you find what you want or just give up. At the moment personalisation works on you loading up websites you want to see and blogs you want to read. With receive you will have a personal profile, probably a VortexDNA one, and a simple filter of areas of interest: sport, shopping, news, blogs etc. Within those headings you will have sub filters. Then you will receive information on a constant basis as required and formatted.

So far so simple, but the best is to come. Within those filters you will get only relevant content as decided by your DNA profile. This means you will not get 100 stories about US politics or movies but ones which are relevant to you. If it’s not, just dump it and that gets recorded as a miss. Over time you will find yourself receiving everything you need. If you want a change then you just alter your settings. It’s easy. No more time wasting searching for stuff.

Just sit back, relax and receive.

This piece was written by guest author Nitin Karandikar, who writes Software Abstractions. It is the fifth guest installment in the Rising Star Dream Team Future of Search series.

I’ve been writing about Search technologies for a while now, so when Kaila Colbin of VortexDNA offered me the chance to participate in a future-of-search marathon, I jumped at the chance!

What will the search for information look like in the future - in five years, ten, twenty? Is it just more of the same, or will it look radically different?

Looking Back

Before looking to the future, let us first look at how far we have come. Danny Sullivan has a great post looking at a decade of search history and the various tribulations of past and present search engines - AltaVista, Ask Jeeves, Microsoft, Yahoo! and of course, the early Google. We owe a huge debt of gratitude to the tremendous contributions of these and other early pioneers of Search; Google, in particular, deserves a great deal of the credit for making web search ubiquitous outside the tech community. Indeed, “to Google” as a verb has become virtually synonymous with the idea of Web Search, much as the Xerox brand became synonymous with the idea of the photocopier in a bygone era.

Google’s venerable PageRank algorithm is certainly best-of-breed for the present, and Google keeps tweaking its results continuously. Given this progress in the Search area, can we still expect to see major improvements in search in the forseeable future? As an analogy, consider the DC-3 airplane - the first truly modern airliner, it was powerful, safe, reliable and economical (indeed, some of these are still flying today). It revolutionized air travel, and with its introduction, many considered the aviation age to have arrived for the general public. And yet, early jet aircraft had already appeared on the horizon, so to speak; within a decade, this reliable workhorse was obsolete, overtaken by jet aircraft in the competition for public air travel.

It could easily be the same with search. The key question that a search engine addresses is: what results do the maximum number of users find most useful for a given search query? PageRank is simply an approximation of the Wisdom of Crowds to answer this question. Is there a richer abstraction? Is Engagement the new black? Whatever the new approach is - in order to provide accurate results, it must work as implicitly as possible.

We have only to envision the possibilities …

Looking Forward

So let us take a speculative look at search, circa 2015 . To look at it systematically, we can separate the search engine into the following components from a user perspective:   (To see this breakdown in visual format, check out my earlier post on an abstract architecture for search )

• Query specification
• Base Index
• Relevance Algorithm
• Results Visualization
• Ongoing Interest

Let us consider the possible future directions for each one in turn.

Future Directions

Query Specification
(Input)

Google pioneered the keyword-centric, minimalist approach for specifying the search query, and all the major search engines follow that lead. But the search criteria could be so much richer ; instead of experimenting with different types of keyword searches to find the information they need, users could simply provide additional criteria up front to qualify their request.

Admittedly, this approach does not work for everyone. The casual user would get reasonable defaults, which would automatically get updated with regular use to their favorite values; the topical researcher, on the other hand, would actively tinker with these widgets in a “power user” mode. (Google already supports this type of functionality in a limited fashion.)

Some possible advanced features for specifying the query are given below:

1. Content Spec:  Enabling the user to dynamically specify the data sources to be included, based on domain, reputation, social network, and so on
   
2. Scope:  Input for seamlessly limiting the scope of the search, to Enterprise or personal data
3. Qualifiers:  Allow the user to add more information to disambiguate result matches, e.g. qualifying if “Java” means the programming language or the island
4. Parameter ranges:  Domain-specific parameters can be extremely valuable even to a general-purpose engine (see #5 in the section on Relevance Algorithms below)
5. UI paradigms:  Text keywords are a limited form of input. The actual input mechanisms could be more visual, in the form of sliders, buttons, fields and other UI widgets. Imagine, for example, that as you move a slider, the search results change or an increasing number of results appear on the page!
6. Multiple Profiles:  Personalization does not always have to be implicit. A user could explicitly set up profiles to represent different interests - professional, hobby, personal and so on, so that switching the profile would quickly change the areas of interest

Base Index
(Content)

This is a core area of concern for search engines: what is the scope of content to be considered when searching for information?

The standard approach currently is to build web crawlers that continuously scan as many web sites and web pages as possible; the scanned content is then used to build a master content index that is then updated regularly. This index is then used as the basis when searching for information.

For the base index, the big changes in the future are likely to involve both the scope and understanding of the content; here is a short list:

1. Rich media search, e.g. true indexing of audio and video content
2. Dynamic content search (searching the invisible web )
3. Integration of personal, web and corporate information
4. Perspective-based search, e.g. conservative vs liberal, hard news vs opinion, and so on
5. Subset creation, on-the-fly, e.g. to search for domain-specific data

Relevance Algorithm
(Mechanism)

This is, of course, the most-debated topic when discussing the future of search engines. Clearly, many different approaches and technologies show promise; some of these are noted below:

1. Personalization (but without storing personal info )
2. Social Input / Wisdom-of-Crowds (which has its pitfalls )
3. Social Graph: where your selected network of people help improve search results (Robert Scoble has recently gotten religion about this concept; Danny Sullivan rebuts)
4. Semantic Processing: of both, the query AND the content   (will this let the Search Engine find answers that we never knew we had?)

5. Parametric Search:  Vertical search engines already routinely offer domain-specific parametric search; for example, job search engines allow the user to specify the all-important location of the job as a primary criterion. Can this type of feature be generalized, so that as a user drills down deeper into search results, an increasing number of parameters can be offered?

6. Human-powered Search, for either the short head or the long tail of search
7. Swarm Intelligence: Mimicking biological search, such as Ant colony optimization, particle swarm optimization, and so on

Results Visualization
(Output)

Again, Google leads the way with its minimalist approach: simple headings, links and snippets of text. This is slowly changing, with the new “Universal Search” approach from Microsoft, Google and others; Ask.com is a leader in this area.

Search engines of the future will likely implement completely new paradigms for users to navigate and view search results. Often, meta-results - representing information about the results - are as important as the results themselves: users can figure out where a given result fits into the overall universe of results, and find the related results to an item of interest that has been found.

Some possibilities for results display are given below:

1. Tag clusters is not a new concept, but has yet to gain traction among the majors. Quintura, with its dynamic tag cloud display, has one of the best examples.
2. Organize results information by content type, is something every search engine will have to think about in the future. For example, should news stories be presented in an “overview capsule” fashion, or organized as a timeline-based view? Dale Dougherty at O’Reilly Radar has a brilliant article on this topic: Journalism is burning.
3. Follow-up actions - on viewing search engine results, a very common user action (as Greg Linden points out ) is to modify the current query, either to drill-down further or to try a different approach to find the required information. Google’s “did you mean …” feature is a step in this direction (although it leaves much to be desired).

4. Domain-specific visualization can significantly enhance the understanding of results. This is similar to the data organization point above, but focused on the display itself; results from different vertical domains may require very different visualization techniques, such as colors, graphs, images, trend lines, heat maps, topographic charts, and so on. [For a list of the more exotic variations, check out this amazing list from Smashing Magazine.]

5. Dynamic scoping - enabling users to widen or narrow search results, based on different criteria - such as geography (local or global), site authority, timeliness, point of view, domain, and so on - is a powerful feature, that will continue to grow in importance.

Ongoing Interest
(Notification)

This can be best explained as a Reverse search, where it is the content that finds the user - thus turning the concept of search on its head.

Most of us have ongoing interests in certain areas; they could be professional, social or personal. It makes a great deal of sense for the search engine to keep track of these interests and pro-actively notify the user at some periodic interval of new items that fit those interests. Google Alerts is an early example in this direction. But enhancements to its functionality in the future could significantly boost its utility.

Some day, search engine notifications could support the following features:

1. Diverse Mediums:  Many search engines already support email notifications. What’s to stop them from adding support for many additional delivery mechanisms, such as IM, SMS, widgets, the twitter API, and so on?

2. Levels of Detail:  Allowing users to set the scope and organization of information presented.

3. Prioritization:  This is a key feature! Once users are able to set priorities for different types of searches and for different areas, this can be used to drive the other features. For example, send me the headline about a breaking news event directly relevant to my blog, as an instant message, but email me a digest of the day’s results for baseball scores.
4. Schedules:  Some search results make sense only at certain times of the day; e.g. traffic search results are only relevant at commute times on work days.
5. Dynamic Control:  Finally, empowering users to assert dynamic, granular control over their search alerts would make this functionality truly powerful. For example, once I’ve been notified about a breaking news story, I might want to artificially boost its priority and delivery method to continually get updates quickly and efficiently.

Power and Responsibility

As search engines start including a few or many of the features described above, search will grow increasingly more powerful. It will get easier to find any information we want, quickly and easily. Whether the information is high-level or detailed, global or local, general or specific, past or present, in any domain - no nugget of human knowledge shall escape this relentless spotlight.

Is shining a light on the darkest corners of the web always a good thing? As a webbed superhero once told me (and a few billion others) - “With great power comes great responsibility!”. Privacy advocates are rightly concerned with the growing power of global web search engines; ongoing efforts from official and community channels are essential in minimizing abuse. A related issue is that web content can be archived and searched in perpetuity - the societal effects of this phenomenon have not yet begun to be understood.  A recent New York Times column highlighted this issue (paid content; here’s a perspective on it from Slate magazine ).

Conclusion

Clearly, search engines will continue to evolve, and a future engine might well have many of the improvements described above, in the next ten years. But how about even further out - say, 2020 or 2030? Will disruptive changes in networking, computing and information technologies radically change the way search engines operate? A change in the nature of human thinking, interaction and social customs would be even more dramatic, and could cause a change in the nature of search itself.

This is, of course, a fertile area of speculation more in the realm of Science Fiction (for now): for example, will we one day need a galactic search engine? Can we create microscopic information-matching agents, either biological or atomic? Results that suddenly become available to the user as knowledge in the brain? An “implicit” search engine that finds information as we need it? Why not?

This piece was written by Branton Kenton-Dau, the director of VortexDNA. It is the fourth installment in the Rising Star Dream Team Future of Search series.

A couple of days ago Kaila Colbin, VortexDNA’s dream blogger, asked me how I saw the future of search. I’m not the only one with a view on this and I hope you will make a contribution as well. I say that not just to be polite but because the one thing I have learnt since I last wrote about search is the power of mass collaboration.

Two weeks ago I bought a book called Wikinomics, How Mass Collaboration Changes Everything, by Don Tapscott and Anthony D. William. On the flight back to New Zealand from San Francisco I read the first 100 pages (it had been a long week). It certainly changed - or rather confirmed - everything my awareness has been inching towards.

Mass Collaboration is the reason why Marc Andreessen can write:

No single closed service, no matter how good, and no matter how big, could compete with the diversity of thousands and then millions of web sites that were customized to every conceivable user interest and need.

However brilliant the engineers are at Google, or Hadar Shemtov and his Natural Language team at Yahoo! are (and they ARE BRILLIANT), there is no way of outsurpassing the creative brilliance of millions of people collaborating to create a better search.

When I read something I believe, there does not seem to be any point continuing in the old mode any longer. As soon as I got off the plane we opened up the next generation of the MyWebDNA browser extension to the world. If VortexDNA is in the market to provided a more relevant Web - a totally personalised Web - there is now in my mind only one way to make that happen - by harnessing you and me and you and you… Jimbo Wales, founder of Wikipedia has known this for a long time. He has just recieved $10 million from Amazon to build the next search on these principles. I believe, any serious contender to be the next Google has to embrace the power of mass collaboration. To my mind there is no other way. What do you think?

This piece was written by guest author Charles Knight, the editor of AltSearchEngines. It is the third guest installment in the Rising Star Dream Team Future of Search series.

Collaborate or Perish

My wife and I were walking down the hallway of the local hospital the other day, and the in-house newsletter was displayed in a box attached to the wall. The box was notched so that the headline could be seen as people passed by. This day, the headline consisted of only three words, so they were especially large. It read, “Collaborate or Perish.” I never did read the article, but I did take a copy of the newsletter home, cut out the headline, and placed it on my keyboard, where it sits right now. “Collaborate or Perish,” it reminds me, every time that I sit down to work. It sits on my keyboard because I think it answers Kaila’s question; “What is the Future of Search?

I haven’t read any of the other “Dream Team” essays (because they are not available), but I imagine that they cover well trends in Search such as Verticals, Personalization, Semantic Search, Natural Language Processing, etc. I may be wrong. Regardless, if they do, it would establish, in my mind, that view of the future which takes existing trends and mentally extrapolates them a few years into the future. Then you just describe what you “see.”

Extending today’s trends makes perfect sense; in fact, it makes the most sense. After all, if there is a trend towards “green” fuel alternatives (to combat the trend of global warming), then isn’t it incumbent upon the futurists to tell us that in 2010 we’ll all be driving cars that run on Ethanol? This can be called the majority view, i.e. common sense. Whatever is happening now will likely continue. To argue against it, against trends that are here, now, puts a tremendous burden of proof on the naysayer.

One such minority view is called “Science Fiction.” I could claim, as Sci-Fi movies do, that in 2010 we will have solved all of today’s linguistic hurdles. That you will be able to talk to your computer as easily as you talk to a friend; how could you disagree? Maybe there will be a scientific breakthrough, and maybe there won’t. Maybe Semantic Search will succeed, and maybe it won’t. Sci-Fi just presumes that there will be a breakthrough. I don’t mean to imply that this approach is always wrong. For my money, Jeff Han’s multi-touch invention (Microsoft’s Surface) is the fulfillment of the Sci-Fi movie Minority Report.

I also haven’t read the “other” Dream Team’s articles, those written by Google, Yahoo!, Microsoft and Ask executives (I imagine that they won’t read this one either). I don’t attend those conferences either. No excuses, but I am just not wired to listen to major search engines talk about the future of major search engines. As I have said repeatedly, it just reminds me too much of the Big 3 U.S. automakers smoking fat cigars, only to go bankrupt for not imagining that they could ever have—wait for it—competition. Yahoo!, MSN and Ask have already, in the year 2007, shown us that they cannot compete with Google. Even if all three of them merged into one I doubt that they could do it. They are second tier search engines, and they always will be.

So, Kaila, my prediction for Search in the year 2010 is not a prediction at all, it is a possibility.

This probably won’t surprise anybody, but the Future of Search rests with the 100 or so best Alternative search engines.

Visit my blog (”Visit my blog, please!”) and you can scroll to your heart’s content until you are convinced that they already have all of the innovation that they need to go after the Big G. But, and this should be the “but” heard ’round the blogosphere, they have been, they are, and they will be just so many gnats on an elephant’s back as long as they remain separate.

What would worry you most: 100 little BBs, 10 bullets, or one cannonball coming at you? Which would you prefer on your arm, the soft rays of the beach sun, or the same rays focused to a super-hot point with a magnifing glass? (There’s that new UI again!) How about a drag race between a Corvette and thirty 10-hp mopeds, or one 300-hp Camaro? There is strength in numbers, but not when they are individually wrapped. Collaborate or Perish. If the Alts don’t begin their collaboration now, the Google cycle will just continue unchecked.

[The Google cycle:

1. Look at the Top 100 Alternative Search Engines.
2. Imitate their approach, hire their talent, or just buy some of them outright.
3. Replace the now-missing Alts with lesser quality ones.
4. Repeat steps 1-3.]

Collaborate or Perish. The Future of Search in the Year 2010 cannot be known, because that decision has yet to be made.