I was recently asked to speak at the Econsultancy FODM event, specifically on the future of search, in a section of the day that was structured as a series of seven lightning talks, all limited to seven minutes exactly.

I looked at what’s on the horizon post-2012, namely ‘Serendipitous Search’.

As any search professional knows, search engines are not often particularly forthcoming with information.

Even working for a search engine, the complex nature of the products and algorithms; combined with the obligations that come with the data protection role of data owner, mean that there are hard walls between teams which govern data access.

In order to deliver a tangible presentation (in seven minutes) I decided to look specifically at one Google product that has been of particular interest to me for some time and imagine how more recent developments could feed into that product; namely Serendipitous search.

Serendipitous search

I first became aware of this working concept in December 2009, having read a very interesting Telegraph article about Marissa Mayers vision of an “omnivorous Google”. (Wise change of working name methinks).

At this stage the concept is described as having the ability to present information to users before they know what they want to search for. I later wrote about the possible roadmap for omnivorous search, including Google Instant on SEO-Chicks, in September last year.

So this is a concept that has been in the public domain, and communicated to media over time via Marissa Mayer. Most recently, at the TechCrunch disrupt event of May this year, Mayer, speaking to Jason Kincaid thinks we will see this realised “inside of a two year horizon”.

Whilst of course we’re not going to get full concept descriptions at this stage of development, it’s significant that Mayer moved to be VP of Location and Local Services and user location certainly seems to be a significant dependency for such a product.

In addition mobile is often mentioned as the delivery mechanism, with social and connectivity data as integral and participation opt-in.

What might be required to realise such a product?

To imagine what Serendipity might look like or how it could be realised, I focused mainly on Google product development and wider developing technologies.

Google Product Development

Google has been pretty open about the significance and potential of the Caffeine infrastructure upgrade and the developmental strides this can facilitate, in terms of much more data, fresher data, returned far quicker; and interestingly new dimensions to data not previously considered.

Vanessa Fox, for Search Engine Land quotes Google’s Head of Webspam thus:

What’s exciting about Caffeine is that it allows easier annotation of the information stored with documents, and subsequently can unlock the potential of better ranking in the future with those additional signals.

Since Caffeine we’ve seen significant front-end product developments such as Google Instant, which serves results on-the-fly, before the user has finished typing a whole query.

Opinion on this product seems to vary from useful, to mildly annoying; however I’d argue that the additional feedback data that Instant provides, must be essential to informing (or tuning) more sophisticated propensity modelling, than can be inferred from search history and CTR data.

Following Instant, we’ve seen a number of social and quasi social search integrations such as Social Circle and Google +1. 

Both products require the user to have a Google Profile and are more effective the greater the number of primary and secondary connections I connect to my profile (or Google contacts).

In the case of Social Circle, I am shown results from connections in other social networks such as Twitter or Linkedin, which are connected to me using XFN rel attributes, which describe a pages relationship to me.

On my Google Profile page, my social network profiles use the rel=me attribute, and on those social networks people connected to me generally use the rel=contact attribute. If you would like to know more about social graph and implications for search I’d recommend this article.

So, we now have much more sophisticated preference data, adding Google Instant to personalised search capabilities, plus the capability to add additional dimensions to data thanks to Caffeine.

Isn’t it therefore conceivable that propensity models could be improved using social connectivity data?

I.e. If my “contacts” who have entered the term “world” into the search box have a higher propensity to go on to click the instant suggestion “[world] of warcraft”, as opposed to “[world] of leather” then it may be more useful to display that instant option to me should I begin to search for “world”,…

Of course such propensity modelling would require a much broader adoption of Google accounts, and greater numbers of connections to such accounts than currently. However, I’d argue that Google+ is the first stage in bringing true incentive for general users to do so.

Developing technologies

I’ve used the term “developing” as opposed to “emerging” technologies, because of course, smart phone devices, location services on smart phones, and technologies like Augmented Reality have been with us for some time; however there would still need to be greater development and most importantly wider general adoption of such devices and opt-in location based services to add another dimension to a serendipitous product.

In the aforementioned TechCrunch interview Mayer speaks about Google product development in location based services particularly in terms of Google Latitude and improvement to maps; when combined with the check-in facility on Google places one can imagine how adoption of such takes care of opt-in requirements and circumvents inaccuracies with IP as locator.

Imagining a future, I find myself in an unfamiliar city and search each morning for coffee. Might it not be conceivable that by day three (for arguments sake) before I get a chance to search, “serendipity” sends me an alert to a coffee shop that is right nearby where I am? (Ascertained by recent check-in, or recent check-in plus last GPS?).

Considering my contacts that might be opted-in to the same services; might it not be conceivable that I could select to be notified when contacts of mine are nearby? If any of my contacts have the same discovery settings, it would also be possible to receive an alert telling me that my contact is having coffee in a place nearby and suggest I join them!

Of course all of the above is imagined steps, in considering what data would be required, and what additional dimensions to such data would be required (and at what scale) to make a concept like serendipitous search come to life; however it seems to me that so many of the more significant developments of late could very well be part of the roadmap to that product.