| 000 | 03188nlm1a2200397 4500 | ||
|---|---|---|---|
| 001 | 664077 | ||
| 005 | 20231030041917.0 | ||
| 035 | _a(RuTPU)RU\TPU\network\35261 | ||
| 090 | _a664077 | ||
| 100 | _a20210326a2019 k y0engy50 ba | ||
| 101 | 0 | _aeng | |
| 102 | _aGB | ||
| 135 | _adrcn ---uucaa | ||
| 181 | 0 | _ai | |
| 182 | 0 | _ab | |
| 200 | 1 |
_aReliability of crowdsourcing as a method for collecting emotions labels on pictures _fO. E. Korovina, G. M. A. Baez, F. Casati |
|
| 203 |
_aText _celectronic |
||
| 300 | _aTitle screen | ||
| 320 | _a[References: 15 tit.] | ||
| 330 | _aObjective In this paper we study if and under what conditions crowdsourcing can be used as a reliable method for collecting high-quality emotion labels on pictures. To this end, we run a set of crowdsourcing experiments on the widely used IAPS dataset, using the Self-Assessment Manikin (SAM) emotion collection instrument, in order to rate pictures on valence, arousal and dominance, and explore the consistency of crowdsourced results across multiple runs (reliability) and the level of agreement with the gold labels (quality). In doing so, we explored the impact of targeting populations of different level of reputation (and cost) and collecting varying numbers of ratings per picture. Results The results tell us that crowdsourcing can be a reliable method, reaching excellent levels of reliability and agreement with only 3 ratings per picture for valence and 8 per arousal, with only marginal difference between target populations. Results for dominance were very poor, echoing previous studies on the data collection instrument used. We also observed that specific types of content generate diverging opinions in participants (leading to higher variability or multimodal distributions), which remain consistent across pictures of the same theme. These can inform the data collection and exploitation of crowdsourced emotion datasets. | ||
| 461 | _tBMC Research Notes | ||
| 463 |
_tVol. 12 _v[715, 6 p.] _d2019 |
||
| 610 | 1 | _aэлектронный ресурс | |
| 610 | 1 | _aтруды учёных ТПУ | |
| 610 | 1 | _acrowdsourcing emotions | |
| 610 | 1 | _aempirical study | |
| 610 | 1 | _arating behavior | |
| 610 | 1 | _areliability | |
| 610 | 1 | _aкраудсорсинг | |
| 700 | 1 |
_aKorovina _bO. E. _gOlga Evgenjevna |
|
| 701 | 1 |
_aBaez _bG. M. A. _gGonzalez Markos Antonio |
|
| 701 | 1 |
_aCasati _bF. _cItalian economist and Professor at the University of Trento (Italy) _cProfessor of Tomsk Polytechnic University, candidate of technical Sciences _f1971- _gFabio _2stltpush _3(RuTPU)RU\TPU\pers\39820 |
|
| 712 | 0 | 2 |
_aНациональный исследовательский Томский политехнический университет _bИнститут социально-гуманитарных технологий _bКафедра экономики _bМеждународная научно-образовательная лаборатория технологий улучшения благополучия пожилых людей _h7381 _2stltpush _3(RuTPU)RU\TPU\col\21257 |
| 801 | 2 |
_aRU _b63413507 _c20210326 _gRCR |
|
| 856 | 4 | _uhttps://doi.org/10.1186/s13104-019-4764-4 | |
| 942 | _cCF | ||