Quality Assurance

Gold Standard Units

Gold units are questions to which we know the correct answers that our CrowdContributors must complete to qualify to participate in projects with client data. We also seed Gold units into client data in order to weed out spammers and underperformers. CrowdFlower is the first company to use Gold in the enterprise context.

Programmatic Gold

The Programmatic Gold feature allows us to automate Gold creation and ensure fresh Gold data is always present as volume increases.

Judgment Redundancy

Redundancy, or having multiple CrowdContributors review each data point, leads to more predictably accurate results than relying on the work of a single reviewer.

Variable Judgments

The Variable Judgment feature allows us to automatically assign more Contributors to difficult tasks in order to meet accuracy benchmarks and eliminate unnecessary judgments on easier tasks. This feature ensures cost and time efficiency.

Workflow Management

Complex Workflows

The Complex Workflow feature allows us to take a complicated problem and break it down into discrete, small tasks to be sent to the crowd. This ensures accuracy and scalability for complicated data problems like Business Listing Verification. See the following blog post: CrowdFlower Takes on Yelp

Contributor Profiles

This feature tracks historical contributor performance, allowing us (if necessary) to target the workers who have demonstrated ability in certain applications.

Dynamic Training Automation

Our technology trains CrowdContributors in real-time with automated prompts, reducing overhead.

Statistically Backed Quality Control Checks

Our platform is able to identify an underperforming project in real-time, allowing us to optimize accordingly.

Easy to Integrate API

Our API allows you to send and receive data seamlessly.