{{ searchResult.published_at | date:'d MMMM yyyy' }}

Loading ...
Loading ...

Enter a search term such as “mobile analytics” or browse our content using the filters above.

No_results

That’s not only a poor Scrabble score but we also couldn’t find any results matching “”.
Check your spelling or try broadening your search.

Logo_distressed

Sorry about this, there is a problem with our search at the moment.
Please try again later.

I’ve been experimenting with data APIs for a couple of years now. I’m not a professional developer but it’s a lot of fun to get stuck in to see what cool content and tools can be produced.  

Lately I’ve been experimenting with kimono which promises to turn websites into structured APIs from your browser in seconds.

It aims to removes the intimidating technical boundaries previously needed to develop your own API.

Sound interesting? I certainly think so!

If you’re new to the concept of APIs then I’d recommend reading this guide from Zapier.

Using kimono I’m going to take you through creating your own API which will gather the price of a PlayStation 4 from the following retailers:

  • Amazon
  • Argos
  • Tesco

Once developed you could use this API to:

  • Add your own competitor price comparison widget to individual product pages if price is your strong point (business).
  • Get product price email alerts (consumer or business).

The good news is that kimono already has quite a few of these tools built in so you’ll be able to do something useful with the data pretty quickly.

Creating the feed (step by step guide)

  • Create a free kimono account (it’s currently in beta so will remain free for the time being). Drag the ‘kimonify’ icon to your browser bookmarks bar. The page also contains an interactive tutorial if you’d like more information.
  • We’ll now need to visit the individual product pages of the retailers mentioned earlier to capture the data to be captured by the API. Let’s start with Argos. While you’re on this page click the newly created ‘kimonify’ bookmark. This will open the kimono bar at the top of the web page.

  • Hover over elements of the product page and they’ll be surrounded by a yellow box. Select the price text. The yellow box will remain static around this element and indicates what data will be extracted from the page.
  • Review the page and click on the small cross of any unwanted yellow boxes. We only want the product price for this example.

  • Click on the ‘Data Model View’ on the right hand side of the kimono bar.

  • Rename the selected ‘collection1’ to ‘price’. Please ensure the ‘price’ name is kept the same when repeating the process for other retailers. Rename ‘property1’ to the name of the retailer (Argos in this case).

  • Click ‘Done’ on the right hand side of the kimono bar. Give your API a name and set the crawl rate (the frequency at which the API will capture the data from the page). This ranges from real time to monthly. Keep the rate consistent across all the APIs for this job.

  • Click ‘Create API’ to finish. Repeat the process for the remaining retailers.

We’ve now got the product price collection APIs in place for each of the retailers. Using a new feature of kimono we’re going to combine these into one super, all ruling API.

  • Within your kimono account click on ‘Combine APIs’ link in sub navigation beneath your name.

  • Give this new API a name. I’ve called it ‘Playstation 4 Price Comparison’.
  • Tick the three APIs that you have recently created and click ‘Next’

The API has now been created. You can now develop something using the endpoint directly. The screenshot below is an example of the JSON feed:

For non-developers you could also create content using the newly created API with one of their in-built solutions. We’ll go through these below.

  • Email alerts can be set up to send you an alert whenever the data in your API updates (dependent on your entered update frequency).
  • Creating a KimonoApp allows you to display your data in a basic app for your phone.
  • A KimonoBlock is a customisable data widget which can be embedded on a web page. This option is ideal as a quick price comparison widget for product pages.
  • Creating a Webhook is slightly more advanced. Once the data in your API changes you can choose to post it to a custom file on your server from which you could save it to a database for example.

There are certainly other similar tools which are worth a look. Import.io is one such example (yet to try personally).

There’s significant potential in APIs for research, content development and improving user experiences. Tools such as the ones mentioned are opening this up to a wider audience. Use this new found power wisely!

Matthew Redford

Published 13 May, 2014 by Matthew Redford

Matthew Redford is Digital Marketing Executive at Extreme Creations and a contributor to Econsultancy. You can follow Matthew on Google Plus.

2 more posts from this author

Comments (11)

Comment
No-profile-pic
Save or Cancel
Pete Austin

Pete Austin, CTO at Fresh Relevance

Any chance of a comment on the following legal issue?

Go to Amazon, scroll down to the bottom of the page, click on "Conditions of Use and Sale" and read, "You may not extract and/or re-utilise parts of the content of any Amazon Service without our express written consent."

I am not a lawyer, but at first sight it seems that your example may be extracting and re-utilising parts of the content of an Amazon Service - specifically the price from the Web page.

Is there an argument that these terms and conditions don't apply in the circumstances of the example?

about 2 years ago

Matthew Redford

Matthew Redford, Digital Marketing Executive at Extreme Creations Ltd

Hi Pete

Thanks for commenting. You've certainly made a good point and is something that needs to be considered beforehand.

Whilst not directly relevant Kimono does promote responsible scraping in this note to webmasters: http://www.kimonolabs.com/webmasters

about 2 years ago

Avatar-blank-50x50

Barry Rutter

I was involved in Odds comparison development in a start up back in the late nineties and early naughties. This is the old spidering scraping system with a front end which was superseded by XML parsers and bookies creating their own feeds and the main reason for this is in fast moving markets scraping can never be real time where XML is far more efficient.

The other reason why we steered away from this technology is as the other commentator states we were advised legally at the time it was potentially breach of copyright and unless the party you were scraping had something to gain from it or was in agreement if you built business around it you could be spending a long time in litigation. Now I don't know if this has been tested in court in the same way as the meltwater case was for news clippings and maybe someone can advise but it will be interesting to see how it pans out

about 2 years ago

Simon Lamble

Simon Lamble, Director at Blossom Associates

Having worked in in the industry, I would suggest that if you take this approach without prior approval from the website concerned, you should expect to receive a cease and desist letter.

Their IP, which certainly includes the pricing information, is something that most sites protect very seriously.

about 2 years ago

Avatar-blank-50x50

Jonathan

As someone who has run a price comparison site for a few years now, I have never once received any notice from any of the retailers I have displayed content from, this includes large stores (Amazon etc.), some high-street chains and smaller independents. On the contrary, I've actually had stores contacting me asking to get their sites included on my listings.

I have had a couple rejections when applying for affiliation deals with larger super-stores, however none of these have requested their content to be removed from my site, they simply did not want to pay for referrals.

Besides, the vast majority of these sites already provide this data in the form of a products feed via affiliate networks.

You could also argue the case of fair use. Google, Wikipedia and so on certainly gets by on these terms, it never asks for direct permission to index and display information from other websites.

Ultimately, so long as you respond and comply to any take-down requests, I can't see any problem and most retailers now see price comparison sites as necessary evils that they must work with.

about 2 years ago

Mahnoor Malik

Mahnoor Malik, Contributor at AdClout

I have read this article and trust me you have mentioned everything really well.
(http://bizclout.com)

over 1 year ago

Avatar-blank-50x50

Nikola Andelic, CEO at www.friibee.com

Hi Jonathan, I would very much like to have a chat with you regarding comparison website and affiliated links. How can I reach you ? Many thanks and best regards

over 1 year ago

Avatar-blank-50x50

Harshal Sardeshpande, Software Test Engineer at Trideq Agro Enterprises

Nice Post Sir, by the way... may I know once I used this tool. How can I Know, shown price on my website that links to other e-commerce sites are updated daily on here,Whether weekly or bi-weekly?

over 1 year ago

Avatar-blank-50x50

Stephen Roberts, slave at Bee Connected

Just the sort of thing I'm looking for, but have to ask how do you intergrate it into your website so people can use it?

11 months ago

Avatar-blank-50x50

Robyn Mac, Internet Marketing at PayLessDeal

Amazing post.... i have simple question... is it working properly or not. Actually, this is not an easy job.... i think... How many online stores feed you used in this project?

9 months ago

Avatar-blank-50x50

Anam Ahmed, jobs at job

That's quite easy to setup but how effective it is in terms of making money.

3 months ago

Comment
No-profile-pic
Save or Cancel
Daily_pulse_signup_wide

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Daily Pulse newsletter. Each weekday, you ll receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.