Making personalized search more transparent
At SMX last month, Michael Gray made the following plea to Google:
Be clearer on the SERPs when a result is there because of personalized search and not a normal result. If they’re that much better then why not highlight them?
Good question! That’s what we do, after all. If you’ve just stumbled upon this blog, you may not know that mywebDNA is a Firefox plug-in that circles the two Google results most relevant to the user. Here’s what it looks like:
There are, however, instances when someone might be logged into our system and we don’t circle relevant results. I certainly can’t speak for Google—I have no idea why they don’t highlight theirs—but I can tell you why we mightn’t.
Our technology was based on a hypothesis. The hypothesis is that our purpose and values can help us find more relevant search results.
This hypothesis came out of the understanding that who we are at our core impacts every other aspect of our lives.
When I say ‘who we are at our core’, I’m referring to purpose and values. As I described in an earlier post, my purpose in life is to be an uplifting presence. Yours might be to take care of animals, or to be financially comfortable, or to leave a legacy.
Because it was based on a hundred years of science, we had a pretty good idea that our hypothesis would hold up when people started to use the technology. But, of course, it would have been pretty shoddy of us to put up a plug-in and just say, ‘Trust us! This will work!’ First of all, we had taken what we had learned to another level, and we had to make sure that our hypothesis would hold up. Second, we had to make sure that our algorithms and technology would hold up—after all, it could have been possible for the hypothesis to be accurate and not the technology.
So we went through a validation phase. During this phase, we measured how likely people were to click on a mywebDNA-recommended link versus one that the technology didn’t recommend. This validation allowed us to quantify how much mywebDNA could increase the relevance of search results.
The thing is, though, that if our mywebDNA plug-in had just gone along, merrily circling results, the chances are people would have been more likely to click on the circled links. I mean, they just look more special. In order for our validation to hold water, we had to be able to show that we weren’t creating a self-fulfilling prophecy.
So some of the time we didn’t circle the link, and we measured whether users were more likely to click on a link that mywebDNA thought was relevant even if they didn’t know mywebDNA thought it was relevant.
We found that they were, which meant that mywebDNA was accurately able to predict whether a search result would be relevant to a particular user.
I agree with Michael Gray that it makes sense for personalized search to be more transparent. Wouldn’t you want to know that this result is the one that you personally are likely to be interested in? Especially if you’ve found the technology to be accurate at serving up relevant content?
Would you rather know if you’re seeing personalized results? Or do you think it doesn’t much matter one way or the other?
July 13th, 2007 at 11:10 pm
A visible tick to help learn which results are personalized would aid both my comfort level and my ability to become more proficient. Learning which results are derived from which cached preferences would be enlightening too.
I’m often wondering if searching for A-1 will over-populate or skew subsequent results whenever my interest changes toward A-2.
At this stage of the art, personalization may provide little. Until its effects are more well-known, I’m as nervous about omitting results than I am about collecting appropriate hits.
mywebDNA is visible and judicious. This might deliver more than crunching my search history.
July 17th, 2007 at 3:51 am
Brian,
You raise a good point about the fear of omitting results, and I know you’re not alone. I think most of us have that apprehension because we don’t know what’s out there, and we’re afraid that we’ll miss something great if we limit ourselves to what we already know. I certainly don’t want to be limited to experiences I’ve already had—much of the joy in my life comes from discovery and curiosity.
I suspect the key lies in giving the users control over how adventurous they want the experience to be. Want only results that are relevant to you? Tick a box. Want results that your opposite would see? Tick another one. Want to see what an anonymous user would get? Tick again.
Thank you also for your kind words about mywebDNA. For readers who aren’t familiar with it, two aspects of our software are particularly relevant to this conversation:
1) The plug-in doesn’t change your search results at all. It merely indicates which two you’re likely to find most relevant out of the ten that appear on the page. If you want to click on non-relevant results, hey, that’s your problem
2) mywebDNA is predictive yet non-historical. That means you don’t end up getting the same result over and over because you’ve visited that site a handful of times. Rather than limiting your experience, your purpose and values can be used to open new worlds for you.
I invite others with opinions on personalization to join the conversation—for? Against? Neutral? Thanks for participating.