Founder / Director / Co-founder at easyBacklog / Aqueduct / Econsultancy
29 June 2009 12:12pm
I have recently put a proposal together for a client to make use of Amazon Turk for processing a large volume of time sensitive tasks that require intelligent processing beyond what is economically viable for a computer to perform. Our client is quite keen to progress with the idea but is understandably concerned with how it works in regards to timeliness, quality control, man power involved in managing a community of users.
I was wondering if anyone on Econsultancy has experience using Amazon Turk, and if so, was it productive? What issues have you encountered and would you recommend it?
BTW. I notice that as a UK based company we are not allowed to sign up as a Requester (submit HITs). There is however an intermediary service called Hit Builder which allows you as a UK company to submit requests to Amazon Turk. Again, has anyone had any experience with them?
Web PR Consultant at Clickthrough Marketing
29 June 2009 14:58pm
Mechanical Turk is a great tool for getting those jobs done that require the human touch.
I can only talk about it from a worker perspective, in that I am one of the Turks! As far as quality control goes, there are fairly rigid measures in place to ensure that you get the right people for the tasks. It all depends, it seems to me, on how you set up your Hits, and how you choose those who submit. For instance, you could create your own 'test' to ensure that those who take on the task of doing your hits are pre-qualified.
As far as timeliness goes, deadlines are strict and as a Turk you cannot afford to miss a deadline because it affects your approval rating if you do. However, whether you will be able to get all your hits accepted within the client's timeframe, I cannot answer. I know that for some of us when we find hits we like doing, we will work consistently on those hits until the job is done, but it is all dependent on the pay and how time-consuming a hit is for the money.
eg the best paid are usually audio transcription and those who focus on that are usually kitted out with all the necessary equipment to make transcribing fast and accurate.
I would imagine that managing the users would be made simpler by finding those that are willing to pre-qualify (thereby getting rid of anyone who has just dropped in to find some pocket money) and keeping your community small.
Just a few thoughts
Free market research on digital marketing
Daily Pulse: award winning newsletter
It takes 30 seconds to register