19 datasets found

A crowd-sourced frame disambiguation corpus with ambiguity.

The dataset used in the paper is a crowdsourcing task assignment dataset. It is a binary classification dataset with a sparse structure.

We consider 6 publicly available data sets, each consisting of labels provided by workers and the ground truth.

We consider the problem of accurately estimating the reliability of workers based on noisy labels they provide, which is a fundamental question in crowdsourcing.

Crowdsourcing dataset for airway annotations in chest CT images

The dataset used in the paper is a crowdsourcing dataset with 200 workers and 10 tasks. Each worker has a set of skills and each task has a set of required skills. The dataset...

The Dialog ESP Game is a crowdsourcing mechanism to acquire quality entity labels.

A POAPS program for finding-fix-verify

A POAPS program for iterative improvement on descriptions for images

A POAPS program for labeling that manages uncertainty without exposing it to the user

A programming language with a POMDP inside

The dataset is a social behaviour-phenotyping data-set for caged mice, obtained from the Medical Research Council, Harwell Institute, Oxfordshire (MRC Harwell) as documented in...

Reputation Agent is a tool that detects unfair reviews on gig markets and prompts fairer assessments by presenting micro-information about gig workers’ conditions and policies.

The Good Judgment Open (GJO) dataset contains 1770 datapoints (698 'forecasts' and 1072 'comments') posted by 242 anonymised users with a range of expertise.

Breast cancer dataset where mammograms have been labeled independently by three doctors. Ground-truth has been obtained through a biopsy, not available to the algorithm nor the...

The dataset consists of synthetic, phonetically proximate options which emulate post-editing scenarios of varying difficulty for five languages: Arabic, German, Hindi, Russian,...

The dataset used in the paper to evaluate the performance of different information elicitation mechanisms.

A crowdsourced dialogue dataset for emergency response tasks, using a Wizard-of-Oz approach.

A Deep Reinforcement Learning framework for task arrangement in crowdsourcing platforms.