Self-Service
Crowdsourcing at your fingertips
Annotation
Annotation projects typically require contributors to add notes, explanations, or comments to an object (e.g., image tagging, video tagging, etc.).
Best practices:
- Tagging can be difficult. A two job workflow may be necessary to verify a tag is good.
- Use Gold Standard Data to ensure quality - Gold tests contributors as they progress through the job, removing anyone who can't perform up to your standards.
- Make sure your Gold units account for all potentially correct responses. Otherwise, you may punish quality contributors.
- Use gold to train contributors around edge cases as well as your more common cases. Try and make your Gold as evenly distributed as possible across options as possible.
- When considering which geographic locations to run your jobs, think about cultural barriers that could cause conflicts.
- Validate input whenever possible using one of CF's validators, or a custom regex validator.
- Choose your form language carefully - Clarity is important, even if your job is simple. Examples are highly recommended.
- Make sure your job is yielding adequate results before running a large job. Tests are highly recommended.
- Be sure to skim through the options available to you in the Advanced Options on the Edit page before running.
More to come.
