Aggregation
CML allows you to specify how you would like CrowdFlower to choose the best answer when multiple judgments are received for a single unit. All parent CML elements accept the aggregation attribute. Below is an overview of the different types of aggregation we offer and the tags that use this type of aggregation by default.
- agg
- (checkbox, radios, checkboxes, select)
- This outputs the answer with the highest agreement weighted by worker trust (confidence). A numerical confidence value between 0 & 1 is also returned.
- all
- (text, textarea)
- All responses will be returned in the aggregate response.
- avg
- (rating)
- A numeric average will be calculated from the responses. Variance will also be returned.
- agg_x
- Returns the top x responses for the given field. This is useful when users can select multiple checkboxes for a field. You may only want the top three responses ranked by confidence eg.
agg_3. - cagg_x
- Returns all response with a confidence greater than the number specified. x can be replaced with any number between 0 & 1.
