GeorgeTag script update #4

We've released version 1.2.2 of GeorgeTag Image Tagging Improved.

Here's the latest version is just altered to work with Georgetag's new handle, TagCow.

Thanks for your patience!

Accept/Skip Keys

ver. 1.0.0

last updated: 05/31/08

This script has been downloaded times.

Description:

Adds hot keys to HIT preview pages allowing you to Accept and Skip HITs by pressing "A" or "S" respectively. (The keys can be changed if desired - see below.)

Instructions:

You must have Firefox, with the Greasemonkey extension installed.

Keyboard shortcuts can be changed via the "Change skip/accept hot keys" menu option. The two selections should be separated by a space. You can choose any letter, number or include ctrl+ or alt+.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

CAPTCHA Focus

ver. 1.0.0

last updated: 05/11/08

This script has been downloaded times.

Description:

Automatically focuses the cursor in a CAPTCHA text box whenever a CAPTCHA is present. Also, switches auto-accept back on if you had it switched on before the CAPTCHA appeared.

Instructions:

You must have Firefox, with the Greasemonkey extension installed. There are no options to set.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Are These Items Different Improved

ver. 1.0.1

last updated: 04/08/08

Update 04/08/08 (ver. 1.0.1):Bug fixed. When you use keyboard shortcut to make your selection with the auto-submit off, you can now hit 'enter' to submit the hit.

This script has been downloaded times.

Description:

Provides general cleanup and streamlining to the "Are these items different" HITs. Allows keyboard shortcut for radio buttons and provides option to auto-submit.

Screenshot of HIT with script installed.

On the standard Items HITs, the two items and their descriptions are arranged vertically with the radio buttons at the bottom. This requires a fair amount of scrolling for this quick-decision HIT. This script streamlines the HIT for quicker comparison and submission.

The Layout: The radio buttons are moved to the top of the HIT, and the items are placed side-by-side.

The blue box that describes the HIT has been streamlined to be as short as we could make it. (We removed the qualification information, and put the rest of the text on one line.)

The instructions are hidden using the show/hide scriptlet. Amazon has 6-8 different versions of the instructions, so it will take viewing quite a few HITs before you get all the instructions accepted into the routine. Once you've reviewed the instructions, click "Don't show me about this change again."

View of the instructions warning.

Selecting radio buttons: User can choose the radio button with a keyboard shortcut. The default shortcuts are: "s" selects the "These items are exact duplicates" button, and the "x" selects "These items are different." radio button. You can customize the keyboard shortcuts under the "User Script Commands" menu, using "Change hot keys." To change the hot keys, separate your two selections by a space. The first selection is the "exact duplicates", and the second is the "different." You can choose letters, numbers, and include ctrl and alt with them by adding a "+." So, for example, if you would like y for same and n for different, you type: "y n" Or, if you'd like ctrl+y and alt+n, you type:"ctrl+y alt+n". If you input an incorrect combination, the input dialog box will reappear.

Auto-submit option: The ultimate in speed. User can choose to have the HIT automatically submitted upon pressing "s" or "x", or the user-defined keyboard shortcuts. Note: this is an advanced feature. You must be very confident in your selection abilities to use this feature successfully. When I first gave it a try, I made several errors before getting used to the selection process.

To turn the auto-submit on, right-click on the Greasemonkey icon, then choose the "User Script Commands" menu option, and choose "Toggle auto submit." The state of the auto-submit toggle is shown at the top of the HIT.

Note: When the auto-submit is on, manually selecting the radio button with a mouse click does not automatically submit the HIT.

Other Notes:

If you're really looking to speed up your Turking, open two tabs in one window with active Items HITs running in each window, with the "Automatically accept next HIT" selected. While you are working on one HIT, the other is loading. With this script and the auto-submit on, you should be able to type, e.g.: x, ctrl+PageDown, x, ctrl+PageDown, s, ctrl+PageDown, etc. Both ctrl+tab and ctrl+PageDown move to the next tab in Firefox. Lickety split!

Instructions:

You must have Firefox, with the Greasemonkey extension installed.

User can choose to have selection automatically submitted via the "User Script Commands", choosing "Toggle auto submit." The state of the auto submit is stated at the top of the HIT.

Keyboard shortcuts can be changed via the "Change hot keys" menu option. The two selections should be separated by a space. You can choose any letter, number or include ctrl+ or alt+. Your new selection will be printed immediately after the radio button.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Show/Hide Scriptlet

The show/hide scriptlet is a small piece of script that we use in many HIT-specific Greasemonkey scripts. Because it is embedded, it is not a script that can be downloaded independently.

Description:

This scriptlet hides the instructions in a HIT. The instructions can be viewed by clicking on "show instructions." If the instructions change, a warning will appear, saying "Instructions appear to have changed. Click link above to view instructions." After the user views the changed instructions, they can click "Don't warn me about this change again" to accept the new set of instructions. After clicking this link, the changed-instructions warning won't appear again, unless the instructions change another time.



Other notes:

We don't think instructions should ever be completely removed from a HIT. A worker needs the ability to review the instructions as necessary. However, after doing hundreds, if not thousands of HITs, a worker does not need to scroll through identical instructions hit after hit.

On first install, the user will receive the changed-instructions warning, and will have to click "Don't warn me about this change again."

GeorgeTag script update #3

We've released version 1.2 of GeorgeTag Image Tagging Improved.

Here's the latest improvement:

You can customize the keyboard shortcut! Don't like the Alt+w, or it doesn't work for you? Choose ctrl+k, or alt+z, or even ctrl+alt+q. Pretty much whatever combination you find most convenient.

From the User Script Commands menu in Greasemonkey, choose "Image Tagging change skip key." In the dialog box, you can type in your choice. The format is like the examples above... you can type ctrl or alt or both, then the "+" key, then a letter or number. (Any character that requires the shift key won't work, but capital letters are ok.) If you input an invalid choice, the dialog box will reappear. The new choice appears in the header of the HIT, and the default remains alt+w.

Hopefully this improvement helps those of you who have non-English set-ups, or even the left-handed folks.

Open HITs Warning

ver. 1.0

last updated: 04/06/08

This script has been downloaded times.

Description:
Colors the "HITs" tab at the top of each mTurk page a discreet but noticeable shade of red when you have a HIT assigned to you. This provides a visual notification to help you avoid letting HITs expire.

Screenshot of mTurk with the script installed when a HIT is assigned.

This script checks whether you have an active HITs assigned to you (i.e. HITs which you've accepted but have not yet completed). If you do have active HITs assigned, it changes the color of the "HITs" tab at the top of the mTurk page to a discreet shade of red to provide a visual notification that you have work that needs completing. The tab will appear red no matter which mTurk page you are viewing.

It's all too easy to forget that you've accepted a HIT and allow it to expire as a result, hurting your qualifications score. With this script installed it will be much harder to forget!

Other Notes:
The tab color will only update when a page is refreshed or a new page opened - so if you have two mTurk windows open and accept a HIT in one, the "HITs" tab in the other won't turn red unless you refresh or open a new page in that window. This script will not warn you about HITs that have already expired (it's too late to do anything about them anyway!).

Instructions:
You must have Firefox, with the Greasemonkey extension installed. There are no options to set.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Requesters: Getting High Quality Results, Part 2

As a follow-up to my original post, Requesters: Getting High Quality Results, there are several more strategies Requesters can use to keep their quality high. You might consider these to be advanced strategies.

9. Use groups of HITs to "grade" workers:
   Rather than using a short qualification test, Requesters can use HITs to grade workers. To do this, you can run several batches of HITs at various times of the day, and analyze the returned data carefully. Then, assign the top workers a qualification value, and only allow workers above that threshold to work on your future HIT groups. Amazon Media Content recently tried this method for their advanced HITs. The downside of this grading method is that you only catch those workers who happened to find your HITs when you posted them. In addition, you might not have a large enough group of workers to draw on, and over time you can have dwindling numbers of workers completing your HITs. To rectify these problems, you will occasionally have to post open HITs to re-grade workers.
   
10. Use sliding grade scale:
    This is like using a dynamic qualification test. After setting a grade (either via a qualification test, or via an assigned value), Requesters can assign qualification points for every approved response, and take away points for every rejection. This way a worker is motivated to give the highest-quality responses. CastingWords transcription HITs work this way. Don't forget to tell workers this is your methodology!
    
11. Include/exclude countries
    Besides qualifying workers based on their approval/rejection stats, Requesters can restrict or allow workers based on their location. Using a locale qualification is a blunt instrument, but sometimes it's the only instrument available.
    
    At least one Requester has previously mentioned that upon analysis, they noticed poor quality work returned from particular countries, perhaps due to language issues. Unfortunately, there is no Amazon "system qualification" for language. When a worker looks at their locale qualification, it states:

    The Location Qualification represents the location you specified with your mailing address. Some HITs may only be available to residents of particular countries, states, provinces or cities.

    Unfortunately, there is no "OR" operator when you list locale codes. This means that, as stated in the AWS mTurk Requester documentation, you cannot, for example, allow workers from US or GB or AU or NZ or CA. The best you can do is to exclude workers from a list of countries. What's unfortunate yet again, is that Amazon restricts the number of qualifications for a HIT group to a maximum of 10. (See this document and search for "QualificationRequirement".)
    
    Establishing HITs based on the location of the worker is fraught with political problems. Workers frequently complain on the message boards at Turker Nation when a Requester only allows workers from a particular country (especially the US). Using a locale qualification to effectively impose a language qualification will inevitably unfairly block some works who speak the language fluently. Be aware that some workers will be annoyed by this, but you might not have any other way to apply a fluency test.
    

As a Requester, you are well within your rights to include and exclude workers who don't meet your criteria to help you achieve the highest quality results. Nowhere in the terms of service does it state that you have to be "fair" in excluding workers. In fact, Amazon won't arbitrate in any Requester-Worker disputes, as stated in the Participation Agreement.

However, the more restrictions and complicated hoops you place in your process, the less likely a worker will work on those HITs. You should also be prepared to respond to many inquiries any time you exclude any group of people. If it appears that you aren't being fair, you could also get a bad reputation, which will scare off other workers.

Although you may have to test out different strategies, when implemented correctly these 11 strategies can help Requesters achieve a high quality return on investment. Workers who achieve the "elite" qualifications take pride in their status, and might well put your HIT groups at the top of their to-do list.

GeorgeTag: New hits script warning

A new type of hit has been posted by georgetag, people matching... the script we've been using won't work on these new ones. They used the same title & instructions, though, so we're going to have to find some creative way to tell these hits apart.

If you are doing these people-matching hits, it might be best to turn off the georgetag greasemonkey script, because we don't know yet if it has adverse effects.

To turn off the script, right click on the greasemonkey icon. If you have a georgetag hit open, you will find the script listed with a green check mark next to it. Select the name (left-click) and it should be listed with a red x next to it. Alternatively you can go into the manage user scripts dialog and un-check the "enable" box for this script.

GeorgeTag script update #2

We have the latest and greatest version of the Georgetag script for you, ver. 1.1. This version corrects two bugs and adds two great new features (no more scrolling, and minimized clicking!):

1. Bug fix 1: If you happened to have first clicked in the "Enter ALL text found in image" text field before selecting a radio button, then selected the "no text" radio button, the box turned red and warned you that you needed to type at least one character. Now once "no text" is selected, the box does not give this warning.


2. Bug fix 2: If you had already selected "no text" then changed your mind and wanted to add text, previously you had to manually erase the "no text." Now, if you click "Contains Text", it will clear the text box, but only if it previously said "no text."


3. New Feature 1: Refocusing of text boxes when radio buttons are checked. When you click the "Contains Text" radio button, the cursor is automatically placed in the "Enter ALL text found in image" field. When you select "no text", the cursor is automatically placed in the "tags" text box. No more clicking around! Select and start typing! (Don't forget, the TAB will jump to the next selection as well.)


4. New Feature 2: Click ALT+w to jump to the next photo. (No more scrolling!) The ALT+w jump iterates through the photos, and once you get to the end, it will put you back at the beginning. Note: the first ALT+w takes you to the second image (since the first was already in view). However, if you forget and scroll the page yourself, the script will only jump to what it thinks is the next image. For example, say you do the first two images using ALT+w, then keep going to the 4th by manually scrolling, the next ALT+w will take you to the 3rd image. It's hard to describe, so you will have to play around with the feature.
   

If you can't wait to get your hands on the latest, greatest version 1.1, head over to the GeorgeTag Image Tagging Improved post, and install the new version. (You shouldn't need to un-install the old script, Greasemonkey should overwrite the old one.)

GeorgeTag script update

Georgetag has uploaded new Image Tagging HITs, with changed title and instructions. This caused ver. 1.0 of our script to break.

We have uploaded ver. 1.0.1 to the blog.

Visit GeorgeTag Image Tagging Improved for the latest version of the script. You shouldn't need to un-install the old script; Greasemonkey should overwrite it for you.

Requesters: Getting High Quality Results

Sounds like TagCow has had some great publicity and response to their enterprise! This means more work, but a few snags as well. Georgetag posted some questions to Turker Nation. These are problems that most Requesters will encounter, but particularly those with high-volume HITs. Although my response is particularly about the Image Tagging set, the tactics outlined below are general enough to be used by any Requester.

Quote:

Garbage tags (intentional
and unintentional):
We filter out meaningless words (the, it, an, a, with) but we are getting some totally irrelevant tags
Vulgarity (jokesters are hijacking the program)"
It looks like MTurk is doing some filtering and we are doing some filtering as well. (Can anyone confirm that?)
...
Incomplete taggings:
We have gotten some images tagged with "boy" where there is more that could obviously be said about the photo, like "boy", "playing", "trains"

There are several things you can do to keep the quality high. Here is a list of tactics implemented by other Requesters on mTurk. I'm certainly not suggesting you use all these ideas, but one or two might work well.

1. Have a good, representative list of examples, and a good description.
   I know folks over at Turker Nation have already mentioned this, but your description is very vague. We aren't certain if you want us to make a list of everything we see in the picture, or just keep it as simple as possible. A separate webpage with lots of examples will go very far. We can then emulate these examples.


2. Warn workers what response will be rejected, and what behavior will get them banned.
   A clear (but not overly-dramatic) warning might just be enough to scare off Roboform-type workers.


3. Require a minimum approval rating for workers.
   Some recent HITs by Amazon's Media Content, and the information extraction group had the approval rating greater than 90%. (Smart Travel Media also has this threshold.) This sounds about right to me. After doing 40k HITs, mine is 99.8%. In the forums, even those who do HITs with high rejections (like the items HITs) still seem to have above 90%. This will prevent some workers who are continually trying to "game" the system.


4. Offer a bonus for high-quality work.
   Georgetag is already offering up a volume bonus based on approvals, which is excellent! Few requesters do this, and more should. Once you get a good verification workflow established, you could track workers' responses and reward those that have submitted the highest quality and volume. Money talks.


5. Include "gotchas."
   Include some pictures that should have an obvious response, like a flower, bird, etc., and particularly ones with a word to include. Then you can start to weed out or ban workers who miss these images. The "are these items different" has a qualification set up and a large set of gotchas. When you miss one, your qualification goes down by 200 points. Basically, after getting 3 wrong, you get timed out for some length of time.


6. Ban very bad workers.
   And ban them quickly. If you haven't already implemented it, check if a single worker is giving you the same response (or few responses) over and over again. I wouldn't be surprised if workers are bypassing the "Enter ALL text found in image" step. Luckily this can be automated. And don't forget to give them some rejections if they replicate their responses beyond some acceptable level.


7. Set up verification HITs.
   After getting all the tags for a given image, you could then create a HIT where the worker verifies that the tags are relevant, and could let you know if any are vulgar or meaningless. An image with a row of checkboxes would be ideal, where you select any tags that are bad, and a comment field to let you know about anything unusual. Hopefully then you can get a great set of tags and you can identify the bad workers using another method.


8. Use a qualification test.
   This one might be a pain to grade, but you could have 5 images that the worker has to successfully tag before being able to do your HITs. Some qualifications even have a quiz about the purpose of the HIT and whether an example is appropriate or not. The quiz-style could be automatically graded.

Implementing 1, 2, and 3 is dead-easy. Making a nice page with a list of good and bad examples will go a long way to fixing some of your problems. I think some workers might be inadvertently giving you poor tags due to lack of understanding.

Politically, using any of the tactics 2-8 can be slightly tricky. (Item 1 should be done by all Requesters. Don't forget, the "description" field cannot be seen by workers once they are in the HIT.) You don't want to scare off your best workers, nor stop people from trying your HITs. Don't be too threatening, or too strict. Workers get very upset if they feel wrongly slighted, and will happily share with all on the Turker Nation forums.

To end on a positive note:
You'll find that most workers really do want to give you exactly the high-quality response that you require. When paid well, we are eager to perfect our responses, and love having discussions and feedback. Continue a good dialog, and you will have a group of willing, quality workers in no time!

GeorgeTag Image Tagging Improved

ver. 1.2.2

last updated: 06/25/08

Update 06/25/08 (ver 1.2.2): Fixed to conform to Georgetag's new name TagCow.

Update 4/8/08 (ver. 1.2): Added capability for user-defined skip image keyboard shortcut, through a "User Script Commands" Greasemonkey option.

Update 4/7/08 (ver. 1.1.2): Fixed to work with new Georgetag group, titled "Image Tagging - Describe what you see. Earn a volume BONUS! Click here to see how."

Update 4/4/08 (ver. 1.1): Fixes 2 bugs and adds 2 features. Click here to see new features.

Update 4/3/08 (ver. 1.0.1): Requester changed the instructions and title of the HIT. New version works with both the old and the new instructions.

This script has been downloaded times.

Description:

Does 5 things to the "Image Tagging" HITs by georgetag:

1. Places the 3 input fields next to each image (rather than below)
   
2. When "Does Not Contain Text" radio button is selected, the "text found in image" field is automatically filled in with "no text"
   
3. All images can be scaled based on a user-supplied percentage, to optimize the layout on your screen
   
4. Once text/no text radio button is selected, cursor is placed in appropriate text box.
   
5. Scroll through images using a ALT+w, or a user-defined keyboard shortcut.

Screenshot of HIT with script installed. Click on image for full-size version.

The requester georgetag, has uploaded a massive dump of image tagging HITs. They are offering a bonus based on the number of approved HITs, so we wanted to simply speed up the throughput of working on these HITs.

This script increases the efficiency of the Turker workflow by rearranging the layout and removing the reduncancy of the "no text" radio button/text field.

1. Rearranging the layout of the hit: Currently in these HITs, each image and its associated 3 input fields are are aligned vertically. We have created a 2x5 table, so that the 3 fields for a given image appear to the right of the image. This arrangement not only reduces the amount of scrolling, but it also allows you to see the tag text field and the image together on the screen.
   
2. Auto-filling of "no text": These Image Tagging HITs have a redundancy input, which is a time-waster for workers. Even when you click the radio button for "Does Not Contain Text", you must also fill out the second text field with "no text." This script automatically fills in the "text found in image" field with "no text" when the "Does Not Contain Text" radio button is clicked.
   
3. Resizing of Images:If you don't like the standard size of the images, you can resize all the images by a given percentage. I find that 75% makes the height of the image the same as the input boxes, but you might have a different preference. To set the scale, right click on the Greasemonkey face in the bottom toolbar, Select "User Script Commands", you will see a text box pop out that says "Image Tagging image size (currently 100%)". If you click that text, a pop-up window appears, and you can enter the image scale as a percentage. For example, to scale all images at 50%, enter "50" in this box. The image scale will take effect the next HIT you view.
   
4. New Feature 1: Refocusing of text boxes when radio buttons are checked. When you click the "Contains Text" radio button, the cursor is automatically placed in the "Enter ALL text found in image" field. When you select "no text", the cursor is automatically placed in the "tags" text box. No more clicking around! Select and start typing! (Don't forget, the TAB will jump to the next selection as well.)
   
5. New Feature 2: Click ALT+w to jump to the next photo. (No more scrolling!) The ALT+w jump iterates through the photos, and once you get to the end, it will put you back at the beginning. Note: the first ALT+w takes you to the second image (since the first was already in view). However, if you forget and scroll the page yourself, the script will only jump to what it thinks is the next image. For example, say you do the first two images using ALT+w, then keep going to the 4th by manually scrolling, the next ALT+w will take you to the 3rd image. It's hard to describe, so you will have to play around with the feature.
   
6. User-defined skip image keyboard shortcut. You can change the keyboard shortcut to be a more convenient combination. See Instructions below for directions.

Other important notes:

The Image Zoom add-on is a helpful extension for these HITs. If you set a small image scale, you can use Image Zoom (on right-click) to zoom in and out of each image for inspection.

Please note that this script was made on-the-fly, and will cease working if the Requester changes the HIT. In addition, we have only tested the script on Firefox ver. 2.0.0.13 and Greasemonkey 0.7.20080121.0. If you find bugs, please let us know, and we'll try to sort out what's going on, time permitting.

So you are aware, we have already had HITs approved by the requester for which we used this script.

Instructions:

You must have Firefox, with the Greasemonkey extension installed.

Image scale can be set via the Greasemonkey "User Script Commands." The image scale defaults at 100%.

You can change the skip image keyboard shortcut. From the User Script Commands menu in Greasemonkey, choose "Image Tagging change skip key." In the dialog box, you can type in your choice. The format is "ctrl" or "alt" or both ("ctrl+alt"), then the "+" key, then a letter or number. (Any character that requires the shift key won't work, but capital letters are ok.) Examples are: ctrl+k, or alt+z, or ctrl+alt+q. If you input an invalid choice, the dialog box will reappear. The new choice appears in the header of the HIT, and the default remains alt+w.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

mTurk is for the dogs

Oh dear! The latest post at the AWS blog finally reveals in print how Amazon feels about the Workers on mTurk. (Something we have known all along! Where's that Worker mTurk API?) The guys in the development section have developed a DCI - The Dog Computer Interface.

They train the office dog, Rufus, to do mTurk hits. And what would a dog do with his reward? They are using dog biscuits instead of the Amazon Payments system, clearly violating the Terms of Service. No wonder our rejection stats have been down the tubes on the items HITs! I am so furious!

mTurk is for poor, starving Workers, period. I highly doubt the ultra-cute Rhodesian Ridgeback would have the troubles I have begging for food. I can't even get strangers to scratch behind my ears! Leave those penny bear HITs for me.

...and their DCI post was published on April 1. As is this one. ;-)

Although, the funniest jokes are ones that have a hint of truth in them.

Restore Old Links

ver. 1.0

last updated: 03/29/08

This script has been downloaded times.

Description:

Place the "Dashboard", "Your Account" and "FAQ" links on all mTurk pages - these links used to be there but Amazon removed them!

On the standard mTurk HITs pages in the upper right corner your name is shown and next to it is a link to "Sign Out." Previously, there were other links in this same area, which took you to your "Dashboard", "Your Account" and the mTurk FAQ. Amazon removed these links during a re-design of the pages on March 3, 2008. This script puts them back!

When Amazon redesigned the site, they also changed the layout of the images in the top toolbar. This means that when you resize the window narrower, the new links in the script do not overlap the tabs like they used to. If you Turk in a narrow window, the links may be out-of-view on the right side of the page.

Instructions:

You must have Firefox, with the Greasemonkey extension installed. There are no options to set.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Add Current Balance To Dashboard

ver. 1.0

last updated: 03/29/08

This script has been downloaded times.

Description:

Show the Amazon Mechanical Turk Current Balance on the Dashboard as an element below "Total Earnings."

This script solves a common problem when Turking -- your Current Balance and Total Earnings are shown on different pages.

On the standard mTurk Dashboard, the values under the "Rewards You Have Earned" are accumulated for the lifetime of your mTurk account. The actual amount of money available to you, the Current Balance, is shown only on the Account Settings page. This script adds a row to the Total Earnings table, below "Total Earnings," which includes the "Current Balance" with a link to the "Account Settings" page, and the value of the Current Balance.

When loading the Dashboard, there might be a slight delay for the row to appear.

Instructions:

You must have Firefox, with the Greasemonkey extension installed. There are no options to set.

---

Disclaimer:
Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Greasemonkey Scripts

Total downloads of all Turk Berserk scripts.

Please use and enjoy our Greasemonkey scripts we have written specifically for Turking. All we ask is that if you would like to share the script, please direct people here to Turk Berserk. Also, please do not upload any of these scripts to any third party sites, or distribute any modified version of our scripts.


Disclaimer

How Safe Is Greasemonkey Anyway?

How to install Greasemonkey scripts.

Our philosophy about Greasemonkey scripts.


Scripts available

---

mTurk Functionality Scripts:

Add Current Balance to Dashboard: Show the Amazon Mechanical Turk Current Balance on the Dashboard as an element below "Total Earnings."

Restore Old Links: Place the "Dashboard", "Your Account" and "FAQ" links on all mTurk pages - these links used to be there but Amazon removed them!

Open HITs Warning: Colors the "HITs" tab red at the top of each mTurk page to provide a visual notification of open HITs and to help avoid letting HITs expire.

CAPTCHA Focus: Automatically moves the cursor to the CAPTCHA text box whenever a CAPTCHA appears. Also switches "auto-accept" back on after you enter the CAPTCHA if it was switched on before.

HIT-specific Scripts:

GeorgeTag Image Tagging Improved: Optimizes the Image Tagging HITs by georgetag.

Are These Items Different Improved: Streamlines the Are These Items Different HITs to increase throughput.

How Safe Is Greasemonkey Anyway?

So we've convinced you that using Greasemonkey scripts is a great way to improve your Turking experience and increase your HIT completion rate. If you're a savvy computer user, before you go and install one of scripts your first question should be "Is it safe to do this?". Downloading and installing third party software to your computer is something that you should always be cautious of! So, in this post I want to briefly run through some of Greasemonkey's safety features.

Before we get to that though, we want to make it clear that we will never include any kind of malicious code in our scripts. We will never purposely try to adversely affect your Turking, make permanent changes to your computer or try to spy on private information. (In fact, Greasemonkey provides no way for anyone to do those last two.) But, since you shouldn't just believe us - read on!!

Some Facts About How GreaseMonkey Works

• GreaseMonkey scripts only work on those webpages listed in the header of each script. We limit our scripts to working only on the pages where they're needed (typically just certain sets of pages within www.mturk.com and sometimes pages needed for specific HITs, such as www.amazon.com/s for Amazon searches), as recommended. When extra data (e.g. an image) is needed by one of our scripts, we encode the image into the script itself, rather than having it be fetched from some third party server. This not only speeds things up (you don't have to wait for the third party server to respond), but avoids the risk of other parties tracking your online activity.


• GreaseMonkey (at least after v0.3.5) cannot access your local files. So, a GreaseMonkey script can't read any information stored on your computer, it can only work with webpages that you open. (The small exception to this is that GreaseMonkey allows scripts to store settings on your computer - for example a script might store your preferred number of results to show on a search page so that this is remembered from one session to the next. Each script can only access its own settings - nothing else.)


• GreaseMonkey scripts are, by construction, open source. That means that, for any script that you install or have installed you can view the Javascript source code for the script. (There's an option to do this when you install a new script, after that you can access the script source through the "Manage User Scripts" dialog.) So, if you're suspicious of a script and you know Javascript you can look through the source and check that it's not trying to do anything of dubious intent. If you don't know Javascript, find a friend who does, buy them a coffee (Javascript...... java...... coffee......) and have them read through the source code for you.

Articles on Greasemonkey Security

As with most things, you can find a lot of discussion online about GreaseMonkey and security issues. Here are a few useful links:

• GreaseSpot: The official GreaseMonkey blog has discussion of all things GreaseMonkey, including security updates. Subscribe to their feed!


• Entry on GreaseMonkey Security over the at the GreaseSpot Wiki.


• An article on Peter Laird's blog discussing the Greasemonkey security model.

OK, and now just in case something does go wrong..............

Disclaimer

Our scripts are provided "as-is". We always aim to provide a well-tested and useful script that aids in your Turking and causes no adverse effects. Given the huge variety of configurations on which our scripts might be used we can never guarantee that something won't go wrong. We take no responsibility for any inconvenience, increased rejection rate, blocking by a requester, loss of income or damage or any other problem that use of our scripts might cause. We recommend that you only use HIT-specific scripts on HITs that you're very familiar with. When you use HIT-specific scripts, treat it as if you were starting a new type of HIT with a new Requester - try doing a few, then wait to be sure that they're getting accepted.

Greasemonkey How-To

We've put together some simple instructions on how to get our Greasemonkey scripts for mTurk installed and working for you. Follow the steps below and you'll have Turk Berserk scripts improving your mTurk experience in a matter of minutes!

Step 1: Get Firefox

Greasemonkey is an Add-On for the Firefox web-browser, so you'll need to use Firefox (at least for Turking) in order to make use of our scripts. If you already have Firefox, skip this step and go straight to Step 2. If you don't have Firefox, it's easy to install. Just click on the "Get Firefox" button in our sidebar (we get a small fee from Google when you click through this link - it costs you nothing but helps keep us motivated to write cool new Greasemonkey scripts!) and click on the blue install button. Or you can go directly to the Firefox page (but then we get no referral fee.....) and click on the big green download button. Follow the instructions and you'll have Firefox installed!

Download Firefox

Step 2: Get Greasemonkey

Already got Greasemonkey? Then skip to Step 3. Otherwise....

Open the Firefox browser and go to the Greasemonkey add-on page. Click on the green "Add to Firefox" button and a small window will open asking you to confirm that you want to install Greasemonkey. Click "OK" and Greasemonkey will be installed. You'll need to restart Firefox for Greasemonkey to activate.

Install Greasemonkey

Step 3: Install a Script

Head on over to our Scripts page and find a script that you'd like to use - then follow the link to that script's own page. At the top of the page you'll find an "Install Now" link. Click on it, and Greasemonkey will open a window asking you to confirm that you want to install the script. Click OK and the script will be installed.

Step 4: Turk with the power of Turk Berserk scripts

You're all set! Head over to mTurk and you should see the scripts you've installed start to work their magic. We're happy to hear your thoughts on how to improve our scripts, or other features you'd like to see, and if you find any bugs, let us know - we'll try to fix them!

Working with GreaseMonkey

GreaseMonkey is designed to "just work" - so you should be able to install scripts and know that they'll work their magic as required. But, there are a few useful things you should know about GreaseMonkey:

Disabling GreaseMonkey: You can completely disable GreaseMonkey by just clicking on the small monkey-face icon in the tool-bar at the bottom of the browser window. The face will become sad when GreaseMonkey is disabled. To re-enable, just click again. You'll need to reload a page after disabling/re-enabling GreaseMonkey for the change to take effect.

Which Scripts Are Active?: To see which scripts are active on the current page, right click on the monkey-face icon. At the top of the menu you'll see a list of all scripts that are active on this page. (If no scripts are active on the current page you'll see "No scripts installed!" instead).

Disabling a Specific Script: If you need to disable a script that's active on the current page, right click on the monkey-face icon. Scripts that are enabled are listed in the GreaseMonkey menu with a green check mark. To disable a script, just select it from the menu. Disabled scripts show up in the menu with a red "X". You can re-enable a script by clicking on it in the Greasemonkey menu again. If you need to disable a script that isn't active on the current page, use the Manage dialog (see below).

The Manage Dialog: Right click on the monkey-face icon and select "Manage User Scripts...". This will open the Manage dialog (see below). On the left, you'll see a list of all of the scripts that you have installed. You can enable or disable any script by clicking on it and then using the "Enabled" check-box in the lower left corner. When you click on a script in this dialog, the boxes on the right show you which web pages this script will work on, and those that it will ignore (by default all pages are ignored). You can also use this dialog to remove scripts - just select the script and click the "Uninstall" button. Finally, if you want to look at the inner workings of a script click on "Edit" and you'll see the Javascript code that does all of the work. You can even edit the script if you want!

The Manage Dialog

Uninstalling GreaseMonkey: If you want to totally remove GreaseMonkey and all of its scripts open the "Tools" menu from Firefox's menu bar and select "Add-ons". Search down the list of add-ons for GreaseMonkey, click on it and then click the "Uninstall" button. You'll need to restart Firefox for the change to take effect.

Where To Find More Help

There's a lot of Greasemonkey help available online. Here are some of the most useful references that we've found:

• Monkey manual: the official guide to Greasemonkey
• Manage dialog: how to use Greasemonkey's "Manage" dialog to manage your scripts
• userscripts.org: a place to find other Greasemonkey scripts
• Scripts section at Turker Nation: Turker community discussion on using scripts with mTurk

mTurk + Greasemonkey = Better, Faster Turking!

If you Turk regularly, and if you read the forums over at Turker Nation, you'll be familar with the thought "mTurk is great - but it would be even better if.....". Whether it's showing your current balance on the Dashboard, or changing the layout of a HIT so that you can complete it with less scrolling and fewer mouse clicks (more money, faster!) there's an long list of improvements that we'd all like to see.

Here at Turk Berserk, we're big fans of Greasemonkey - an extension for the Firefox web browser which lets you change the way websites look and function to your own liking. In the early days of mTurk there used to be lots of Greasemonkey scripts. However, those scripts have either never been maintained or they are for obsolete HITs. The old scripts just don't function any more. It's time to resurrect the joy of Turking with Greasemonkey! We've been using Greasemonkey for some time to improve our own mTurk experience and we want to share the joy with you! So read on, and we'll explain what this Greasemonkey thing is all about.....

What is Greasemonkey?

Greasemonkey is an Add On for the Firefox web browser. On its own, it does nothing - its power comes from small "scripts" (Javascript for the geeks; a bunch of computer instructions that you don't have to worry about for the non-geeks). It works in the background every time you load a web page and allows those scripts to change the appearance and function of that page. All those changes are made after the web page is loaded, on your own computer.

A Greasemonkey script could do something as simple as changing the colors of the web page (to remove that shade of purple that you just can't stand to look at), or something as complicated as reorganizing information in the page, adding buttons and grabbing information from other pages and displaying it. Scripts only work on the pages they're told to - you don't want it trying to put your mTurk balance on cnn.com right....

This is really useful for mTurk! For example, lots of people want the Current Balance (i.e. the amount of money that you actually have available to spend) shown on the Dashbord, which currently just shows your total earnings. There are two ways to get this:

1. E-mail Amazon and beg them to add Current Balance to the Dashboard......

2. Write a short Greasemonkey script that does it automatically every time you view your Dashboard.

Even better, you can install a Greasemonkey script from right here at Turk Berserk that will do exactly this! Interested? At the end of this post we'll explain how to get this Greasemonkey script installed (it will only take a few minutes...). But first, a word about our philosophy.

Our Script Philosophy

We want to make it very clear that the scripts that we make available on Turk Berserk will never allow you to cheat at mTurk. We don't create "bots" that try to complete HITs automatically.

Instead, our scripts are designed to either improve your Turking experience (for example, by giving you the extra information you need to keep track of your work) or to allow you to complete HITs more easily (maybe by reorganizing the layout of a HIT so that it takes fewer mouse clicks to complete). You still have to do the real work of the HIT though!

Amazon is quite explicit about allowing disallowing scripts that complete a HIT for you. In their Conditions of Use they state:

3.b. "(i) you will not use robots, scripts or other automated methods to complete the Services."

The important word here is "complete" - the script must not do the work of the HIT for you, but a script which merely allows you to do the work of the HIT more efficiently is OK.

And, from the horse's mouth: In December, 2005, Jeff Barr (a developer on the AWS side) posted on the official Amazon Web Services Blog

A few Workers are trying to run "bots", which you can think of as mindless HIT robots. While we do recognize that some high performance Workers use a variety of scripts to simplify and increase the efficiency of the Mechanical Turk workflow, we do need to point out that the use of fully automated bots runs counter to the Mechanical Turk Participation Agreement.

We (and apparently Amazon too) don't consider using scripts of this type to be "cheating" - although there has been debate about this in the past and a clearer position on this issue from Amazon would be very welcome. We will gradually be making all of our scripts publicly available - so the playing field remains level (if one of our scripts allows you to complete HITs faster, then it allows everyone to complete them faster). The only thing we ask is that you don't publish our scripts on other websites -- please direct folks here to Turk Berserk.

The bottom line: We want to improve your Turking experience and make mTurk faster and more efficient. That can only help to get more requesters using mTurk and help us all make more money!

OK, Sounds Good. So How Do I Get Your Scripts?

Now that we've convinced you that using Greasemonkey is the way to Turking Zen you'll want to head on over to our Greasemonkey How-To to learn how to get Greasemonkey running. You should also check out our post, How Safe Is Greasemonkey Anyway? (If you already know all about Greasemonkey and have it installed, you can skip the How-To and surf right on over to our Scripts page.)

Newbie Pitfalls

Things can go wrong, and they often do. For new Turkers, you have not yet established a routine or strategies, and you are still getting a feel for Turking. Here’s a few pitfalls to look out for, and some solutions to avoid falling into them!

(This is a long post. Click here for a summary.)

I can’t find HITs I’m qualified for.

There are two ways you might not be qualified to work on HITs: missing a qualification test and not having good enough stats. Look through the all HITs list, and take note of the HIT groups that are grayed out. Click on the HIT title to expand the group details, and you can see why you aren’t qualified. If after reviewing the HIT you think it’s a HIT you might like to try, get qualified. (HINT: get qualified for Unspun HITs right away.) In addition a Requester can set thresholds for Worker accepted/rejected percentages. Which leads to pitfall #2:

My rejection rate is ruined!

When you start Turking, try to do as many easy, mostly-guaranteed-to-approve HITs as you can. Which ones? It depends on what HITs are available, however you should try to snap up the following: Take our “5 Second Poll”, Unspun HITs, Powerset HITs, and the music album HITs. This way you will have a “buffer zone” if you get a few rejected HITs in the future.

The group of HITs I did were mostly rejected!

There are several ways this can happen.

1. There’s a possibility that you didn’t understand the HIT or what the Requester required, and you performed the HIT incorrectly. In this case, the best you can do is to get Feedback from the Requester, and improve upon your mistakes. Requesters occasionally leave a note in the Feedback field on the daily Status page, or you can try emailing them through the linked name in the list on your daily status page. (To do this, Go to the Status page, find the HIT you’ve performed, and click on the Requester’s name. This brings you to the “Contact Requester” form.)
2. Some HIT groups are just known to have high rejection rates. Do your due diligence, and research the Requester and HIT type on the Turker Nation forum. The following have relatively high rejection rates even for experienced Turkers: Amazon “Are these items different,” ContentSpooling, Steven Research “Web Classification,” . It’s wise to avoid these until you have enough approved HITs to be a buffer.
3. Some Requesters will Reject HITs outright without cause. This can be an accident. Sometimes new Requesters haven’t sorted out their evaluation algorithms. Sometimes the Requester is unscrupulous and wants to take your submitted work and run. Either way, the folks on the Turker Nation forum will quickly compare notes and work out what happened.

The best advice I can give any worker is to only try a few HITs for any HIT group you have no experience with. When testing out a new HIT or Requester, only do as many HITs as you can comfortably afford to get Rejected.

I can’t figure out what work I’ve submitted.

Once you submit a HIT, there is no way to go back and look at the HIT again, nor can you see what answer you gave. In the list of HITs submitted in the daily status details (click on the date in the status page), all you see are the Requester name, HIT title, status, and Feedback. Hidden in the URL of the link to the Requester is the HIT ID. The only way to match up your work to the HIT ID is to keep your own log including the HIT ID and the work you did. This may or may not be worth your time… only you can decide.

Help! My HITs haven’t paid yet!

Relax! Just because Amazon tells you:

Doesn’t mean “shortly”= today. Take a look at my Requesters page to get a sense of roughly how long it takes for Requesters to approve HITs. Some Requesters set their HIT groups to auto-approve after a set amount of time. (Unspun’s “What’s the best website” and “Find Amazon.com product identifiers” are set at 5 days.) Even if the Requester never gets around to evaluating your HIT, Amazon has a policy to auto-approve any un-evaluated HITs after 30 days.

I’m not making much money.

Most jobs on mTurk pay pennies. To make money, you need to do lots of HITs. Experience is key here. Experience will increase your throughput (number of HITs per hour). Experience will tell you how much reward per HIT is right for you. That said, mTurk is not for everyone.

Here’s a summarized short list to getting started without too many pitfalls:

• Get qualified.
• Build up your accepted stats, keeping rejected stats low.
• Get familiar with Requesters and their HITs, and do research.
• If you are trying a group of HITs for the first time, only do as many as you can afford to have rejected.
• Be patient, and keep at it for a little while… you might just get addicted!

Resources Beyond mTurk

There are several places beyond mTurk that may help you in your Turking quests:

1. Turker Nation: A mature, well-established forum for Workers.
2. Mechanical Turk Sandbox: The official test & development site. Everything looks and acts as if it’s the regular mTurk, but you don’t get paid for doing or posting HITs. It’s like monopoly money. It will prompt you to create an account on the Sandbox if you want to play around. However, you just use your Amazon login yet again.
3. Amazon Web Services Developer Connection -Mechanical Turk Forum: The official forum for mTurk Requesters. Very technical discussions about how to post HITs, the mTurk APIs, and other Requester coding issues. Amazon responds to technical questions here very quickly. Also, Amazon posts update notifications here. Useful for workers to get an understanding of what Requesters go through.
4. Turkers Forum: A new forum created at the end of February 2007. Doesn’t have too many discussion threads yet, but it’s worth keeping an eye on. Appropriately enough, this Forum advertised and started populating threads via a HIT on mTurk.

Don’t forget as well, that many HITs on mTurk are through companies whose business models rely on mTurk. These Requesters might have more resources and information on their home pages.

If you’re thinking of posting your own HITs, these resources are specifically for the Requester side:

1. AWS Developer Forum: (mentioned above)
2. Command Line Tools: Open source toolkit to make writing HITs much easier. Download through svn.
3. mTurk API Tools in Other Languages: PHP, Perl and Java.
   
4. HIT-Builder.com: Hit-Builder offers more features than the standard mTurk requester interface. You can also hire them to help develop your HITs.
5. von Kempelen: Company offers HIT designing and posting services.
6. Dolores Labs: Offers HIT data collection services. Has many examples of test HITs.
   

If you know of any other useful Turking resources, post a comment, and if it’s worthy, we’ll add it to our list!

Glossary

Like all popular pastimes, mTurk has its own lingo. Here’s a little guide.

As new topics come up, I will add more terms to the glossary. If you have any to add, feel free to let me know!

To open hyperlinked terms in a new tab, hold down the CTRL key and left-click the name.)

URL for Google Spreadsheet: http://spreadsheets.google.com/pub?key=pxpTLP5OADPhuuDGV0UDiFg

The six categories of HITs

There are numerous Requesters on mTurk. However, only a few types of HITs appear over and over again.

I have categorized HITs into 6 categories:

1. Decision: In these HITs, the worker is asked to make a judgment call or to take a poll. Decision HITs usually contain radio buttons or a check list and are quick to perform. Examples of decision HITs are: Amazon's Are these items different, Powerset's Evaluate Search Results, and Content Review's Review User Submitted Images.


2. Research: For research HITs, the worker must search for information, generally on the web. The response is typically a URL, copied reference material, or data. Examples of research HITs are Amazon's NowNow Research Questions, Unspun's Find a URL/Amazon product identifier, and ClayValet's Find a product group of HITs.


3. Image Tagging: Workers interact with the picture in some way. Usually the worker is marking a set of specific features in the image. The two most common Requesters in this sub-type are Geospatial Vision (marking road features) and True Yardage (marking features on golf courses).
   


4. Transcription: Workers are asked to transcribe text from an audio or video file. HITs that ask workers to transcribe text from an image are also included.
   


5. Create: Generally a more involved HIT, these HITs require the worker to create some original content. There are many types of creation HITs. Some ask you to write an article or rewrite a sentence (e.g. ContentSpooling, Paul Pullen), create trivia questions (e.g. UQsoft), or draw something (e.g. draw).
   


6. Traffic Generator: These HITs usually are trying to generate traffic to their website. They might ask the worker to click through some links. Sometimes they require the worker to comment on a blog. Other traffic generator HITs want workers to post links back to their own websites.

These six categories of HITs encompass just about every HIT you encounter on mTurk. I'm sure there are some oddballs out there as well that don't fit these five. I just can't think of any!

You might argue that a few HITs straddle more than one category. For example, rewriting sentences generally means the Requester is using workers for Search Engine Optimization (such as ContentSpooling). Although this has the end result of traffic generation, the actual work performed is mostly a creative process for the worker. Likewise, you could argue that asking a worker to generate a list of tags for an image is a creative process -- as is crafting a response for NowNow questions. However, the former is mostly an image-based process with minimal effort and the latter takes tremendous amounts of research.

In the list of Requesters and HITs, the categories are based on the major type of work a Turker is asked to perform.

Requesters

Here is a list of frequent Requesters, the types of HITs they offer, and roughly how quickly they pay when your HIT is accepted. I will update this list from time-to-time to reflect the current Requesters at mTurk.

note: This is not meant to be a comprehensive list! But it’s a good place to start.

URL for Google Spreadsheet:
http://spreadsheets.google.com/pub?key=pxpTLP5OADPhLyybn0fY_Ag

If you click on a Requester name, you will be taken (within the iframe) to a search page for all HITs currently listed by that Requester ID. To open a new tab with that search, hold down the CTRL button before clicking. Not all Requesters will have HITs listed.

Any cell with "??" indicates missing data. If you have any of the missing information, contact me and I will update the spreadsheet.

The categories listed in the "Type" column are based on the 6 categories of HITs, described here. In addition, the time to payment is an estimate based on my experience. Beware that Requesters may change their time to payment.

Getting Started: A Tour Around mTurk

Here’s a quick primer on how to get started working on mTurk. I will go into details of each of the sections in future blog posts.

Don’t forget to read the official mTurk FAQ on getting started.

Get an Account

Start at the mTurk Introduction page. To work on mTurk, you will need to get an mTurk account. If you already have an account at Amazon.com (i.e. if you’ve purchased products from the US-based Amazon) you can just sign on to mTurk using your existing email address and password. If you don’t already have an Amazon.com account, you will be prompted to create one. You will be prompted to sign in or create an account if you click on the Sign In in the upper right hand corner, if you try to click on Your Account, or if you try to accept a HIT.

Let’s go on a quick tour:

Next, you should explore mTurk. There are three tabs at the top of all mTurk pages: Your Account, HITs, and Qualifications.

Your Account

Under the Your Account tab, you can keep track of your HITs. The Dashboard is the homepage for your stats: summaries of how many HITs you’ve done, cash earned, reject rates, etc. The Status page gives you 45 days of details of your HITs. The Account Settings is the technical details page. It has your account balance and links to the detailed transactions (the actual money transfers from the Requester’s accounts to your own). This is also where you transfer your earned money out of mTurk.

HITs

The HITs tab is where you find HITs to work on. You can look at All HITs available on mTurk, or just the HITs you are qualified to do (HITs Available To You). In addition, if you have any open HITs (you’ve accepted them, but haven’t submitted or returned them), you can find those open HITs on the HITs Assigned To You page.

Qualifications

The Qualifications tab lets you manage and explore qualifications on mTurk. The Qualifications Assigned To You pages gives you a list of all Qualifications that you have earned or have been assigned. Everyone has the HIT abandonment rate (%), HIT approval rate (%), HIT rejection rate (%), HIT submission rate (%), HIT return rate (%), and Location Qualifications. The All Qualifications page lets you browse through a list of all Qualifications you can request or take a test to receive. Finally the Pending Qualifications page shows you the Qualification tests you have taken, but have yet to be graded by the Requester.

Find a HIT

In future posts, I’ll go into the details of how to navigate each of the pages mentioned above. However, I’m sure you’re itching to find a HIT to try!

Go back to the HITs tab, and click on HITs Available To You. You’ll see a list of all the groups of HITs you are qualified for on mTurk. On the upper left, you can sort the HITs in various ways. When you click on the title of the HIT group, the box will expand to show you the description of the HIT, keywords and qualifications. When you click on the “View a HIT in this group”, you are taken to a preview page for the HIT. Read the description, look over the HIT and decide if it’s something you’d like to try. If so, do the HIT! Note that there is a timer in the upper left corner of the HIT page that lets you know how long you’ve been working on the HIT and the total Alloted Time.

Get Approved, Get Paid

Getting your HITs approved by the Requester is one of the biggest variables on mTurk. Some Requesters have a script that verifies your work, and approves or rejects your HIT accordingly. (The Amazon Requester Inc. “Are These Items Different” are this way.) Some Requesters will pay you automatically after a certain number of days. (Amazon Unspun HITs pay automatically after 5-6 days.) And some Requesters have to approve or reject your HIT by hand. It can take anywhere from a few minutes to 2 weeks for the vast majority of HITs to be evaluated and paid, so be patient. Even if a Requester never gets around to evaluating your HIT, Amazon has a policy to auto-approve your hit after 30 days.

Once your HIT is approved, you see the value in the Total Earnings on your Dashboard increase. This amount is not the amount of cash you can transfer out to your bank or Amazon.com balance. Your current balance is quoted on the Account Settings page. There is a delay from the HIT being approved (value on the Dashboard) and the money transfering from the Requester's account to your own. You can see details of these transfers through the View your account balance and transaction history link on the Account Settings page.

Transfer money

Now that your HIT has been approved, you’ve gotten paid, I bet you are eager to get your hands on your earnings! You have several options here, mostly depending on where you are in the world.

If you are in the US, you can choose to either transfer your earnings to a US-based bank account or you can transfer your money to your Amazon.com balance. As of March 18, your earnings are stored in the Amazon Payments system. You can use your earnings to pay for goods wherever Amazon Payments are accepted. In order to transfer your money to your bank account, you will need your account number and routing number from your bank. Both of these options are available from the Account Settings page.

If you are in India, you can have your earnings sent you by check, made out in Rupees. Details of how to do this can be found on the FAQ.

For the rest of the world, things are much more difficult. Unless you have a US-based bank account, your only option is to have your earnings transferred to your Amazon.com balance, then find Sellers on Amazon who will ship products to your country. What’s important here, is that you can NOT transfer your earnings to Amazon.co.uk or Amazon.de, etc. Nor can international Turkers use the Amazon Payments system. The independent Turker Nation forum has a nice thread of Sellers who will ship internationally. Make sure to go to the last page to see the most recent update.

And that’s it for our quick tour! You will learn so much more by getting your hands dirty trying HITs and exploring the mTurk page.

Don’t forget to ready my summary on Pitfalls!

Welcome to Turk Berserk

Thanks for visiting the Turk Berserk blog — a place for all things related to Amazon's Mechanical Turk!

I have been contemplating writing a blog about mTurk for quite some time. There aren’t very many comprehensive resources for workers. I hope to cull together lots of information out there, and present it to you in a useful package.

I’m also hoping that in sharing my experiences on mTurk, I can enrich the experiences of the whole mTurk community — whether you are a Worker, a Requester or just curious.

In the way of introductions… I am Dr. Berserk, I have been Turking since August 2007. My original goal when I started Turking was to earn $100 by Christmas to help buy Christmas presents for my nieces. I hit the $100 mark in just a few weeks. Turking can be very addicting!

Why do I Turk? The extra money is very nice. It’s a great way to get a little cash to splurge on gifts for myself or my husband, Mr. Berserk. Also, I am fascinated in all the different ways Requesters use mTurk. I usually try to do as many new types of HITs as I can, while saving the majority of my mTurk energies for my favorite HITs.

A big part of our Turk Berserk experiment will be to write and release Greasemonkey scripts related to mTurk. We hope they will make your Turking more efficient and productive by cleaning up the HITs, providing extra resources, and add functionality to the mTurk website. Watch this blog for more information!