Benjamin Timmermans  Benjamin Timmermans photo         

contact information

University Programs Netherlands Lead
Amsterdam, The Netherlands


Professional Associations

Professional Associations:  The Netherlands AI Coalition


Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics
Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, Benjamin Timmermans


VisMet and the crowd
Marianna Bolognesi, Benjamin Timmermans, Lora Aroyo
Metaphor and Metonymy in the Digital Age: Theory and methods for building repositories of figurative language8, John Benjamins Publishing Company, 2019

Crowdsourcing Inclusivity: Dealing with Diversity of Opinions, Perspectives and Ambiguity in Annotated Data
Lora Aroyo, Anca Dumitrache, Oana Inel, Zoltan Szlavik, Benjamin Timmermans, Chris Welty
Companion Proceedings of The 2019 World Wide Web Conference on , pp. 1294-1295
Abstract   crowdsourcing, social media, cultural heritage, ibm, multitude, ambiguity, data science, dissenting opinion, computer science, ground truth


Expert Disagreement in Sequential Labeling: A Case Study on Adjudication in Medical Time Series Analysis (short paper).
Agathe Balayn, Panagiotis Mavridis, Alessandro Bozzon, Benjamin Timmermans, Zoltan Szlavik
SAD/CrowdBias@ HCOMP, pp. 67--71, 2018

CrowdTruth 2.0: quality metrics for crowdsourcing with disagreement
Anca Dumitrache, Oana Inel, Lora Aroyo, Benjamin Timmermans, Chris Welty
arXiv preprint arXiv:1808.06080, 2018

Empirical methodology for crowdsourcing ground truth
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Carlos Ortiz, Robert-Jan Sips, Lora Aroyo, Chris Welty
arXiv preprint arXiv:1809.08888, 2018

Characterising and Mitigating Aggregation-Bias in Crowdsourced Toxicity Annotations
Agathe Balayn, Panagiotis Mavridis, Alessandro Bozzon, Benjamin Timmermans, Zoltan Szlavik
1st Workshop on Subjectivity, Ambiguity and Disagreement in Crowdsourcing, and 1st Workshop on Disentangling the Relation Between Crowdsourcing and Bias Management, 67-71, 2018
Abstract   crowdsourcing, majority rule, machine learning, computer science, artificial intelligence, data subject


Catch Them If You Can: Malicious Behavior Simulation in Deep Question Answering
Nikita Galinkin, Zoltan Szlavik, Lora Aroyo and Benjamin Timmermans
Benelux Conference on Artificial Intelligence 2017

ECrowd: Enterprise Crowdsourcing for Training Cognitive Systems using the Workforce
Benjamin Timmermans, Zolt\'an Szl\'avik, Manfred Overmeen, Alessandro Bozzon
29th Benelux Conference on Artificial Intelligence November 8--9, 2017, Groningen, pp. 396

Computational controversy
Benjamin Timmermans, Tobias Kuhn, Kaspar Beelen, Lora Aroyo
International Conference on Social Informatics, pp. 288--300, 2017

Crowdsourcing Ambiguity-Aware Ground Truth
Collective Intelligence, 2017

Controcurator: Understanding controversy using collective intelligence
Bram Timmermans, Lora Aroyo, T Kuhn, Kaspar Beelen, Evangelos Kanoulas, Bob van de Velde, Gerben van Eerten
Collective Intelligence, 2017

ControCurator: Human-Machine Framework For Identifying Controversy
Benjamin Timmermans, Kaspar Beelen, Lora Aroyo, Evangelos Kanoulas, Tobias Kuhn, Bob van de Velde, Gerben van Eerten
ICT Open, 2017


Capturing a Collective Intelligence for Cognitive Computing using CrowdTruth
Collective Intelligence, 2016

Crowdsourcing ground truth data for analysing brainstem tumors in children
Benjamin Timmermans, Zolt\'an Szl\'avik, Robert-Jan Sips
Belgium Netherlands Artificial Intelligence Conference (BNAIC), 2016

The vu sound corpus: Adding more fine-grained annotations to the freesound database
Emiel van Miltenburg, Benjamin Timmermans, Lora Aroyo
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pp. 2124--2130, 2016

CrowdTruth: Methodology for Gathering Annotated Data by Harnessing Disagreement in Crowdsourcing
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Lora Aroyo, Robert-Jan Sips, Chris Welty
ICT Open, 2016

Exploiting Disagreement Through Open-Ended Tasks for Capturing Interpretation Spaces
Benjamin Timmermans
Proceedings of the 13th International Conference on The Semantic Web. Latest Advances and New Domains - Volume 9678, pp. 873-882, 2016
Abstract   semantic web, semantic gap, semantic interpretation, metadata, crowdsourcing, information retrieval, world wide web, computer science


CrowdTruth: Machine-Human Computation Framework for Harnessing Disagreement in Gathering Annotated Data
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Lora Aroyo, Robert-Jan Sips
PDF hosted at the Radboud Repository of the Radboud University Nijmegen, 15, 2015

Provenance-driven representation of crowdsourcing data for efficient data analysis
Carlos Martinez-Ortiz, Lora Aroyo, Oana Inel, Stavros Champilomatis, Anca Dumitrache, Benjamin Timmermans
2015 IEEE 11th International Conference on e-Science, pp. 300--303

Crowdsourcing ground truth for Question Answering using CrowdTruth
Benjamin Timmermans, Lora Aroyo, Chris Welty
Proceedings of the ACM Web Science Conference on , 2015
Abstract   crowdsourcing, question answering, semantic interpretation, ground truth, use case, information retrieval, data science, annotation, majority rule, computer science, multitude