Consiglio Nazionale delle Ricerche

Tipo di prodottoArticolo in rivista
TitoloThe Importance of Worker Reputation Information in Microtask-Based Crowd Work Systems
Anno di pubblicazione2016
Formato-
Autore/iAlberto Tarable; Alessandro Nordio; Emilio Leonardi; Marco Ajmone Marsan
Affiliazioni autoriCNR-IEIIT, Politecnico di Torino-DET
Autori CNR e affiliazioni
  • ALESSANDRO NORDIO
  • ALBERTO TARABLE
Lingua/e
  • inglese
AbstractThis paper presents the first systematic investigation of the potential performance gains for crowd work systems, deriving from available information at the requester about individual worker reputation. In particular, we first formalize the optimal task assignment problem when workers' reputation estimates are available, as the maximization of a monotone (submodular) function subject to Matroid constraints. Then, being the optimal problem NP-hard, we propose a simple but efficient greedy heuristic task allocation algorithm. We also propose a simple "maximum a-posteriori" decision rule and a decision algorithm based on message passing. Finally, we test and compare different solutions, showing that system performance can greatly benefit from information about workers' reputation. Our main findings are that: i) even largely inaccurate estimates of workers' reputation can be effectively exploited in the task assignment to greatly improve system performance; ii) the performance of the maximum a-posteriori decision rule quickly degrades as worker reputation estimates become inaccurate; iii) when workers' reputation estimates are significantly inaccurate, the best performance can be obtained by combining our proposed task assignment algorithm with the message-passing decision algorithm.
Lingua abstractinglese
Altro abstract-
Lingua altro abstract-
Pagine da-
Pagine a-
Pagine totali-
RivistaIEEE transactions on parallel and distributed systems (Print)
Attiva dal 1990
Editore: Institute of Electrical and Electronics Engineers, - New York, NY
Paese di pubblicazione: Stati Uniti d'America
Lingua: inglese
ISSN: 1045-9219
Titolo chiave: IEEE transactions on parallel and distributed systems (Print)
Titolo proprio: IEEE transactions on parallel and distributed systems (Print)
Titolo abbreviato: IEEE trans. parallel distrib. syst. (Print)
Titoli alternativi:
  • Institute of Electrical and Electronics Engineers transactions on parallel and distributed systems (Print)
  • Parallel and distributed systems (Print)
Numero volume della rivista-
Fascicolo della rivista-
DOI-
Verificato da referee-
Stato della pubblicazionePublished version
Indicizzazione (in banche dati controllate)-
Parole chiaveReliability, Crowdsourcing, Resource management, Maximum a posteriori estimation, System performance, Systematics, Throughput
Link (URL, URI)-
Titolo parallelo-
Data di accettazione-
Note/Altre informazioni-
Strutture CNR
  • IEIIT — Istituto di elettronica e di ingegneria dell'informazione e delle telecomunicazioni
Moduli CNR-
Progetti Europei-
Allegati