Zazzle Shop

Screen printing

Wednesday, January 21, 2009

Crowd Sourcing the World

Found in translation: Participants in a new program called txteagle will be able to earn small amounts of money using their phones to complete simple tasks, such as translating words from one language to another.
Credit: txteagle

As smart as computers may seem, they can't match humans in certain tasks: describing the contents of an image, rating the quality of Web search results, or transcribing and translating text from another language, to name a few. Tapping into human expertise to tackle problems that computers struggle with is also a growing business: Google lets customers refine its search results, and Amazon uses a system called the Mechanical Turk to off-load all manner of simple tasks to humans around the world; people will work on these tasks, even for pennies.

Now Nathan Eagle, a research fellow at the Santa Fe Institute, in New Mexico, is launching a project similar to Amazon's Mechanical Turk but that distributes tasks via cell phones. The goal of his project, called txteagle, is to leverage an underused work force in some of the poorest parts of the world.

Eagle says that distributing questions to participants in such developing countries via text messages or audio clips could make certain tasks more economical, such as the translation of documents into other languages, or rating the local relevance of search results. It could also provide a welcome source of income for those involved.

"We're trying to . . . tap into a group of people to complete these tasks who haven't been tapped before," says Eagle. "And we're using mobile phones, which have a high penetration rate. More people are mobile-phone subscribers in developing countries than in the developing world, so we can get a user base of billions of people."

The Finnish cell-phone company Nokia is a partner in the project, and Eagle says that it provides a good example of a Western company that could benefit from txteagle workers. Eagle explains that Nokia is interested in "software localization," or translating its software for specific regions of a country. "In Kenya, there are over 60 unique, fundamentally different languages," he says. "You're lucky to get a phone with a Swahili interface, but even that might be somebody's third language. Nokia would love to have phones for everyone's mother tongues, but it has no idea how to translate words like 'address book' into all of these languages."

Another application is the transcription of audio recordings: a user would listen to a short clip, write it down by hand, and then copy it into an SMS reply. Eagle's studies have shown that this task can be completed in less than two minutes, and he believes that a proficient user could earn about $3 an hour doing the work, which would be 60 percent cheaper than today's transcription rates.

Users would be paid either in credit to their mobile accounts or in cash, as facilitated by a service called mPesa, which allows people to send and receive currency via cell phones and use their phones to claim money at mPesa agents and post offices, says Eagle.

One technical issue that he has considered is quality control. Eagle says that he and his colleagues are developing machine-learning algorithms that can determine the accuracy of different workers' responses. Essentially, identical tasks are off-loaded to a number of workers, and if a high percentage of those come back with a particular response, then it can be assumed to be the most accurate, within a certain level of statistical confidence. Also, if a person consistently responds correctly, then the system deems her more trustworthy, providing her with more tasks, and allowing her to make more money. But Eagle admits that there are still some kinks in the system that need to be ironed out, especially for the translation and transcription tasks, whose accuracy can be somewhat subjective.

Txteagle will use a reputation system similar to one developed by a San Francisco startup called Dolores Labs that works with Amazon's Mechanical Turk. CEO Lukas Biewald says that such a system is a powerful tool for txteagle. "You don't have to make assumptions about who can do your work and who can't," he says. "It allows you to take much more risk with the people doing the job," without sacrificing overall accuracy.

Sharon Chiarella, vice president of Amazon's Mechanical Turk project, says that bringing crowd sourcing to developing nations could be a good idea. "One of the things that's powerful about this space is the promise of leveraging a worldwide workforce," she says.

But Chiarella adds that there will be some limitations. The most widely available cell phones, for instance, are generally only able to send and receive text and voice messages. This makes crowd sourcing more complex tasks, such as tagging images, impossible. "The cell-phone screen size somewhat limits the tasks that can be viable via the phone," she says. "But I think that as cell phones continue to evolve, some of those issues will go away."

Eagle agrees but says that his goal is to start small and see if the model works well enough to expand. He hopes to receive grant money that will allow txteagle to roll out the service in Rwanda, Kenya, Bolivia, and the Dominican Republic within the next year.