Newbie Pitfalls

Things can go wrong, and they often do. For new Turkers, you have not yet established a routine or strategies, and you are still getting a feel for Turking. Here’s a few pitfalls to look out for, and some solutions to avoid falling into them!

(This is a long post. Click here for a summary.)

I can’t find HITs I’m qualified for.

There are two ways you might not be qualified to work on HITs: missing a qualification test and not having good enough stats. Look through the all HITs list, and take note of the HIT groups that are grayed out. Click on the HIT title to expand the group details, and you can see why you aren’t qualified. If after reviewing the HIT you think it’s a HIT you might like to try, get qualified. (HINT: get qualified for Unspun HITs right away.) In addition a Requester can set thresholds for Worker accepted/rejected percentages. Which leads to pitfall #2:

My rejection rate is ruined!

When you start Turking, try to do as many easy, mostly-guaranteed-to-approve HITs as you can. Which ones? It depends on what HITs are available, however you should try to snap up the following: Take our “5 Second Poll”, Unspun HITs, Powerset HITs, and the music album HITs. This way you will have a “buffer zone” if you get a few rejected HITs in the future.

The group of HITs I did were mostly rejected!

There are several ways this can happen.

1. There’s a possibility that you didn’t understand the HIT or what the Requester required, and you performed the HIT incorrectly. In this case, the best you can do is to get Feedback from the Requester, and improve upon your mistakes. Requesters occasionally leave a note in the Feedback field on the daily Status page, or you can try emailing them through the linked name in the list on your daily status page. (To do this, Go to the Status page, find the HIT you’ve performed, and click on the Requester’s name. This brings you to the “Contact Requester” form.)
2. Some HIT groups are just known to have high rejection rates. Do your due diligence, and research the Requester and HIT type on the Turker Nation forum. The following have relatively high rejection rates even for experienced Turkers: Amazon “Are these items different,” ContentSpooling, Steven Research “Web Classification,” . It’s wise to avoid these until you have enough approved HITs to be a buffer.
3. Some Requesters will Reject HITs outright without cause. This can be an accident. Sometimes new Requesters haven’t sorted out their evaluation algorithms. Sometimes the Requester is unscrupulous and wants to take your submitted work and run. Either way, the folks on the Turker Nation forum will quickly compare notes and work out what happened.

The best advice I can give any worker is to only try a few HITs for any HIT group you have no experience with. When testing out a new HIT or Requester, only do as many HITs as you can comfortably afford to get Rejected.

I can’t figure out what work I’ve submitted.

Once you submit a HIT, there is no way to go back and look at the HIT again, nor can you see what answer you gave. In the list of HITs submitted in the daily status details (click on the date in the status page), all you see are the Requester name, HIT title, status, and Feedback. Hidden in the URL of the link to the Requester is the HIT ID. The only way to match up your work to the HIT ID is to keep your own log including the HIT ID and the work you did. This may or may not be worth your time… only you can decide.

Help! My HITs haven’t paid yet!

Relax! Just because Amazon tells you:

Doesn’t mean “shortly”= today. Take a look at my Requesters page to get a sense of roughly how long it takes for Requesters to approve HITs. Some Requesters set their HIT groups to auto-approve after a set amount of time. (Unspun’s “What’s the best website” and “Find Amazon.com product identifiers” are set at 5 days.) Even if the Requester never gets around to evaluating your HIT, Amazon has a policy to auto-approve any un-evaluated HITs after 30 days.

I’m not making much money.

Most jobs on mTurk pay pennies. To make money, you need to do lots of HITs. Experience is key here. Experience will increase your throughput (number of HITs per hour). Experience will tell you how much reward per HIT is right for you. That said, mTurk is not for everyone.

Here’s a summarized short list to getting started without too many pitfalls:

• Get qualified.
• Build up your accepted stats, keeping rejected stats low.
• Get familiar with Requesters and their HITs, and do research.
• If you are trying a group of HITs for the first time, only do as many as you can afford to have rejected.
• Be patient, and keep at it for a little while… you might just get addicted!