Archive for the ‘Predictive search’ Category

Attitudes, beliefs, predictive search and behavioral targeting

Tuesday, October 16th, 2007

Summary: Database giant Acxiom has just confirmed VortexDNA’s value proposition by including attitudes and beliefs in their relevance technology. The predictive value of who you are and what you believe is significantly greater than that of context.

Those of you who follow this blog regularly know that the foundation of VortexDNA’s relevance technology is that an individual’s purpose and values can be used to predict relevance.

Going by a piece by Laurie Petersen at Online Media Daily today, we’re not the only ones who know this to be true:

DATABASE MARKETING GIANT ACXIOM TODAY officially launches its Relevance-X products designed to allow marketers to make online media buys…

“We’re really excited about this,” said Rich Howe, Acxiom’s chief marketing and strategy officer. “We’re bringing our knowledge and experience in direct marketing to the online channels to give clicks context–going far beyond basic information such as age, gender and household income to include the attitudes, beliefs and lifestyles of consumers that are much more predictive.” [emphasis mine]

According to the article, Acxiom has been seeing click-through rates double or triple in tests of the Relevance-X system: powerful numbers.

Even without the boost from tapping into consumer attitudes, behavioral targeting is far more effective than contextual targeting. Consider what this Research Brief from the Center for Media Research has to say:

…a study on consumer receptivity to online advertising… found that more online consumers are consistently more receptive to behaviorally targeted ads than to contextual advertising, outperforming contextual by as much as 22 percent in some categories.

Marla R. Schimke, vice president of marketing at Revenue Science, said “… (this report shows that) behavioral targeting is more effective than contextual advertising for advertisers, publishers, and for consumers… This study… reaffirms our belief that Internet users favor advertising relevant to them personally…” [emphasis mine]

When that brief says, “relevant to them personally,” they’re saying we know you like computers so we’ll show you ads about computers. The validation of VortexDNA’s technology has shown that it’s possible to go far, far deeper than that: link relevance can be accurately predicted based on who you are, what your purpose in life is, and what you value above all.

And VortexDNA technology can do this without ever tracking history.

This concept is not standalone, either; companies can use it to augment their current recommendation technology rather than replace it.

Imagine the power of a search engine that integrates VortexDNA technology with existing keyword relevance matching. Imagine how gratifying it would be for an ecommerce site improve their recommendations to you based on what you really care about.

Do you have an ecommerce site? A search engine? An ad platform? Do you just find this topic intriguing? Leave a comment below or email me privately (kaila @ vortexdna.com). Let’s begin a conversation.

This is a dramatic shift in how we look at relevance, and we’d love for you to participate with us.

Google’s Norvig sees same future as VortexDNA

Friday, July 20th, 2007

Peter Norvig, Google’s director of research, gave us a glimpse on where Google is focusing its efforts for the future in an interview with Kate Greene from MIT Technology Review this week:

The core of what we do is still search and advertising. A lot of researchers are working on that. They’re working to give better-quality search results and to match ads better. Another area of research is gathering more sources of information, such as text in books, still images, video, and now audio in terms of speech recognition. I think another focus is to understand how people interact with Google and interact with each other on the Web, in general. How do people operate in these social networks? Understanding that question can help us serve them better.

These statements don’t reveal anything new or secret about Google, but they do reinforce what we’ve been saying for some time: the relevance arena, which is where search quality and ad matching live, is a vital and vibrant piece of the search equation, and there’s still a lot of room to grow.

His point about understanding how people interact with Google and each other on the Web is also an important one, and I’m glad to see that he included it in his initial statement. In The first principle of search relevance, I discussed the need to focus on the user in order to deliver true relevance:

What is relevance, if not caring? What is relevance, if not a reflection of the user’s needs, wants, and values? Without caring about the user, our search for relevance would fall dramatically short.

It’s great that someone as senior as Norvig is reinforcing that idea. The purpose of technology is not technology. The ability to do things differently is not an imperative to do so. At the end of the day, the question is and must continue to be: how will this affect people?

One of the things I love about mywebDNA is that it brings those two concepts, relevance and people, together on so many levels. With mywebDNA, the users are the filter that determines relevance.

I recently did one of those team-building exercises where we got split into groups of three and given a bucket, a bit of pvc pipe, and two balls. Our first challenge was to use the pipe to try to whack the balls into the bucket. Our second was to use the bucket and the pipe any way we wanted to get the balls into the bucket. The second was easier, of course, and the message was that these two things made it easier:

  1. Usability, and
  2. Control.

It’s important to note that when I talk about relevance, I’m not suggesting we should only be allowed to see certain things. We should be able to see anything we want! We should have access to a Web that allows us excellent usability and total control over our own experience.

The fact remains that, at the end of the day, we’ll only be able to access the minutest sliver of what the world has to offer us, online or otherwise. In economics, people talk about scarce resources; in life, our attention is the scarcest resource of all. Personally, I like to direct mine towards things I care about.

Google does a brilliant job at indexing, ranking, and (if you’ve got it turned on and aren’t opposed to it on moral grounds) personalization based on geography and demographics. Those calculations spit out a result that they hope will appeal to you at the deepest level. With mywebDNA, though, who you are at the deepest level serves as the filter; relevant Google results are circled because you are who you are.

Later in the piece, Greene asks Norvig what the outstanding problems in search are. He responds:

In general, we think there are two aspects of it. One is understanding users’ needs more. The other is understanding the contents of documents, whether they be Web pages or video. Mostly we look at what the user types in, treat the input as individual words, and count them up on pages and weigh those pages with different kinds of evidence. But we don’t look only at words they type in. We also look at spelling variants, and if a user types in a long query, we break it into pieces. Maybe a user meant some words, but didn’t really mean others.

Here is a simple three-step equation:

  1. Understand users more.
  2. Increase relevance.
  3. Serve them better.

Do you think this is a valid goal?

Should Google stay in or out of my history?

Monday, June 4th, 2007

The astonishing complexity and incredibly advanced algorithms behind Google’s search capabilities were explored today by Saul Hansell in the New York Times. While most of the article focused on the many parameters used to determine which search results enter the coveted top ten, he also touched on Google’s push towards personalization:

Increasingly, Google is using signals that come from its history of what individual users have searched for in the past, in order to offer results that reflect each person’s interests. For example, a search for “dolphins” will return different results for a user who is a Miami football fan than for a user who is a marine biologist. This works only for users who sign into one of Google’s services, like Gmail.

(Google says it goes out of its way to prevent access to its growing store of individual user preferences and patterns. But the vast breadth and detail of such records is prompting lust among the nosey and fears among privacy advocates.)

The privacy concerns arise because of the nature of the information that Google is collecting: essentially, search history and demographics. Read every ESPN article on the Miami Dolphins? Google knows you’re a fan.

The thing is, Google has to collect and maintain all of the individual bits of information about you in order to provide you with personalized search results. For example, it isn’t enough to know you’re a Miami Dolphins fan if you’re searching for a furniture store. Google also has to know where you live.

If you then search for “beetle”, Google will add to its growing list of data about you whether you prefer cars or bugs. Each search adds a different clue to their quest to provide greater relevance.

Do you think this is appropriate? Do you really care if Google, or anyone for that matter, knows whether you prefer cars or bugs?

VortexDNA has the unique characteristic and firm policy of never tracking users’ histories—the VortexDNA profile is an aggregated genome representing the totality of your purpose and values. Because the technology tracks something totally different to the personal information tracked by Google, it’s possible that the combined techniques could result in even more relevant search results. Is this something that users want, though?

Do you prefer to keep your history private, or would you rather see a technology like MyWebDNA combine with history and demographics to produce the most relevant results?

What are your privacy concerns?

Solve for semantics at the search engine level

Tuesday, May 29th, 2007

I’ve put up a few posts about the controversial semantic web or ‘Web 3.0′. Most people have a gut reaction that the concept is buzzword-heavy and lacking in practicality, or even a clear definition. Dr Riza C. Berkan summed up the issues today with intellectual rigor in a ReadWriteWeb post:

The two basic views of a semantic search are identified by the location of the semantic resources to be implanted. The first view is to embed the semantic resources in the Web pages themselves. It is called the “Semantic Web”. Why not compose Web pages in a structure that is semantics friendly?

…The “Semantic Web” approach has been around for a long time now. Unfortunately, it is based on an unrealistic assumption that every Web author will abide by the complex rules of semantics - not to mention the education it requires - and place content in the correct buckets of mysteriously unified standards. Another form of this approach may be to design Web factories that crank out refined Web pages once fed by ordinary Web pages. Of course if there is more than one factory, you have the standards issue again. In this day and age of fast content production, the Semantic Web seems to be more idealism than realism.

Dr Berkan goes on to discuss the pros of focusing efforts to understand the user at the search stage:

Without relying on statistics, long-tail queries can be analyzed by semantic algorithms on the fly, and bring search results with the accurate context… a semantic approach is very effective in handling dynamic content and can unleash its full power the second the content is born.

The argument, highly valid, is that it is easier to make one search engine intelligent than billions of web pages.

Dr Berkan’s company, hakia, offers a semantic search engine, as do Cognition Search and Lexxe. Powerset is working on theirs.

VortexDNA shares Dr Berkan’s view—in fact, we’re taking one step further away from the content. The idea behind MyWebDNA is not to create a new search engine, but a universal measure of relevance that can be overlaid onto any search engine.

Our tactics are different: the means of determining relevance can be through context, meaning, or, in our case, the purpose and values of the user. But our fundamental approach is the same: create the right lens, and the results will come into focus.

VortexDNA and MyWebDNA in the news

Monday, May 28th, 2007

Nearly every one of my posts references another piece of journalism, but it’s truly a delight to point you in the direction of an articles written about VortexDNA and MyWebDNA. YahooXtra News commented on our validated search results with an article entitled ‘NZ start-up sees itself as the visa of the net’:

VortexDNA has developed a self-profiling system that will allow search engines like Google, or social sites like MySpace, to predict your search and hugely increase your “click rate” on links that come up as a result of an internet search.

I’m excited to see the media pick up on the story, which I had commented on in a previous post. The validation of the search results takes us from the watery realm of the hypothesis onto the firm ground of the business case, and it took a lot of people (not me—I just write about it!) a lot of brains, creativity and persistence to get us there.

Google says the answers lie in the individual

Monday, May 28th, 2007

In an opinion piece published by the Financial Times on Friday, Peter Fleischer had this to say:

…the same words can have very different meanings to different people depending on their background and their interests. It is the same idea that is driving Google’s personal search service.

The bulk of his article focuses on the privacy issues of personalized search, which makes sense since Peter is actually Google’s Global Privacy Counsel. But I choose to focus on this perhaps deceptively obvious gem: that identical words can have totally different meanings depending on the user.

That conundrum lies at the core of the predictive search challenge: how can a search engine possibly hope to know what you’re looking for, if what you’re looking for can’t be decisively defined by words?

Google’s answer is, of course, history and demographics. VortexDNA’s approach is core purpose and values—first understand who you are, then use that as a filter. Are there other solutions out there? And do you have any opinions on what might be the best one?