Self-Service
Crowdsourcing at your fingertips
Sentiment Analysis
Sentiment analysis projects generally asks contributors to determine the sentiment of an object (text analysis, image analysis, audio analysis, video analysis, etc.)
Best practices:
- Use Gold Standard Data to ensure quality - Gold tests contributors as they progress through the job, removing anyone who can't perform up to your standards.
- Make sure your Gold units account for all potentially correct responses. Otherwise, you may punish quality contributors.
- Use gold to train contributors around edge cases as well as your more common cases. Try and make your Gold as evenly distributed as possible across options as possible.
- When considering which geographic locations to run your jobs, think about cultural barriers that could cause conflicts.
- Choose your form language carefully - Clarity is important, even if your job is simple. Examples are highly recommended.
- Use our Ratings tag or Radios tag to capture input.
- Make sure your job is yielding adequate results before running a large job. Tests are highly recommended.
- Be sure to skim through the options available to you in the Advanced Options on the Edit page before running.
An example job:
Note that this form contains logic. As seen on the page's first set of questions, the question labeled "Rate the Sentiment of this Post" is displayed only after the contributor selects "Yes" on the previous question.
The code:
CML:
<p>
<b><a target="_blank" href="{{author-uri}}">{{author-name}}</a>:</b>
{{title}}
</p>
<cml:radios label="Is this tweet related to CrowdFlower" name="related" class="" validates="required" instructions="Please read each tweet carefully" gold="true">
<cml:radio label="Yes"></cml:radio>
<cml:radio label="No"></cml:radio>
</cml:radios>
<cml:radios label="Rate the Sentiment of this Post" class="" only-if="related:[0]" name="sentiment" instructions="Remember, we're looking for the Sentiment with respect to CrowdFlower" validates="required" gold="true">
<cml:radio label="Positive"></cml:radio>
<cml:radio label="Neutral"></cml:radio>
<cml:radio label="Negative"></cml:radio>
</cml:radios>
More to come.
