After 24 months of development, Tweetmeme CEO Nick Halstead has today unveiled DataSift, a platform from which to manage and filter large amounts of unstructured social data.

The second company to spin out of Halstead’s MediaSift parent group (Tweetmeme being the first), the company describes itself as a scalable solution for managing large volumes of social information from a variety of sources – as well as a ‘sophisticated resyndicator’ of real-time Twitter.

It is one of only two companies in the world with a licence to access Twitter’s full ‘firehose’, meaning that users can search for posts using the metadata contained in a tweet. This makes DataSift’s search functionality far more powerful than many of its competitors.

DataSift allows users to define a ‘stream’ by entering keywords or locations, which is then immediately populated with data in real-time as posts from Twitter and blogs appear around the web.

Halstead said that in starting DataSift he wanted to disrupt the way data is sold.

I believe that democratising data access is going to fundamentally change the market, and bring a whole new wave of innovation in which companies can concentrate on building product rather than re-inventing data aggregation.

The Guardian has already trialled the service, saying on DataSift’s website that it had used the service to capture tweets around the phone hacking story.

In four days we captured 500,000 tweets, which we featured in a series of online interactives, driving tens of thousands of page views on The process of extracting the precise information we wanted from real-time social media could not have been easier.

DataSift has partnered with management solutions business Endeca, business analytics experts Datameer, real time analytics firm Hstreaming and trends company Trenspottr. Customers can purchase their data from DataSift, and then use an API key on each partner’s site to access services.

Halstead is joined on the DataSift board by Stewart Townsend, previously manager of Sun’s Startup Essentials programme for Europe, and Rob Bailey, former business development lead at Yahoo!, who takes the role of CEO.