際際滷

際際滷Share a Scribd company logo
Problem of majority voting
Problem of majority voting
Problem of majority voting
Problem of majority voting
Problem of majority voting
Problem of majority voting
Problem of majority voting
Problem of majority voting
According to the majority rule, with three raters, 100% of the
comments have at least 66.7% agreement (2/3). Our data
shows that when the same comments are evaluated by five
raters, 40.1% (n=117) of the comments have at least 66.7%
agreement. With seven raters, 43.5% (n=127) have at least
66.7% agreement, and for nine raters, the value is 44.9%
(n=131).
0
10
20
30
40
50
60
70
80
90
100
Three Five Seven Nine
Percentage
At least 66.7% agreement
Problem of majority voting
Problem of majority voting
Problem of majority voting
Thanks!
Dr. Joni Salminen
jsalminen@hbku.edu.qa
Qatar Computing Research
Institute

More Related Content

More from Joni Salminen (20)

Determining Online Brand Reputation with Machine Learning from Social Media M...
Determining Online Brand Reputation with Machine Learning from Social Media M...Determining Online Brand Reputation with Machine Learning from Social Media M...
Determining Online Brand Reputation with Machine Learning from Social Media M...
Joni Salminen
Is More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Is More Better?: Impact of Multiple Photos on Perception of Persona ProfilesIs More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Is More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Joni Salminen
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Joni Salminen
OSS-EBM: Open Source Software Entrepreneurial Business Modelling
OSS-EBM: Open Source Software Entrepreneurial Business ModellingOSS-EBM: Open Source Software Entrepreneurial Business Modelling
OSS-EBM: Open Source Software Entrepreneurial Business Modelling
Joni Salminen
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Joni Salminen
Tips for Scale Development: Evaluating Automatic Personas
Tips for Scale Development: Evaluating Automatic PersonasTips for Scale Development: Evaluating Automatic Personas
Tips for Scale Development: Evaluating Automatic Personas
Joni Salminen
Big Data, Small Personas: Research Agenda for Automatic Persona Generation
Big Data, Small Personas: Research Agenda for Automatic Persona GenerationBig Data, Small Personas: Research Agenda for Automatic Persona Generation
Big Data, Small Personas: Research Agenda for Automatic Persona Generation
Joni Salminen
Why do startups avoid difficult problems?
Why do startups avoid difficult problems?Why do startups avoid difficult problems?
Why do startups avoid difficult problems?
Joni Salminen
Social Espionage: Drawing Benefit from Competitors Social Media Presence
Social Espionage: Drawing Benefit from Competitors Social Media PresenceSocial Espionage: Drawing Benefit from Competitors Social Media Presence
Social Espionage: Drawing Benefit from Competitors Social Media Presence
Joni Salminen
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Social Media Marketing (Digital Marketing '15 @ Oulu University)
Social Media Marketing (Digital Marketing '15 @ Oulu University)Social Media Marketing (Digital Marketing '15 @ Oulu University)
Social Media Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Search Engine Marketing (Digital Marketing '15 @ Oulu University)Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Web Analytics (Digital Marketing '15 @ Oulu University)
Web Analytics (Digital Marketing '15 @ Oulu University)Web Analytics (Digital Marketing '15 @ Oulu University)
Web Analytics (Digital Marketing '15 @ Oulu University)
Joni Salminen
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Joni Salminen
Who does what in marketing? Toward an understanding of marketermachine inter...
Who does what in marketing? Toward an understanding of marketermachine inter...Who does what in marketing? Toward an understanding of marketermachine inter...
Who does what in marketing? Toward an understanding of marketermachine inter...
Joni Salminen
Social media marketing for researchers
Social media marketing for researchersSocial media marketing for researchers
Social media marketing for researchers
Joni Salminen
Text analysis: An introduction to methods and techniques
Text analysis: An introduction to methods and techniquesText analysis: An introduction to methods and techniques
Text analysis: An introduction to methods and techniques
Joni Salminen
How has the internet transformed marketing
How has the internet transformed marketingHow has the internet transformed marketing
How has the internet transformed marketing
Joni Salminen
Konversio-optimointi (2016)
Konversio-optimointi (2016)Konversio-optimointi (2016)
Konversio-optimointi (2016)
Joni Salminen
Determining Online Brand Reputation with Machine Learning from Social Media M...
Determining Online Brand Reputation with Machine Learning from Social Media M...Determining Online Brand Reputation with Machine Learning from Social Media M...
Determining Online Brand Reputation with Machine Learning from Social Media M...
Joni Salminen
Is More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Is More Better?: Impact of Multiple Photos on Perception of Persona ProfilesIs More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Is More Better?: Impact of Multiple Photos on Perception of Persona Profiles
Joni Salminen
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for...
Joni Salminen
OSS-EBM: Open Source Software Entrepreneurial Business Modelling
OSS-EBM: Open Source Software Entrepreneurial Business ModellingOSS-EBM: Open Source Software Entrepreneurial Business Modelling
OSS-EBM: Open Source Software Entrepreneurial Business Modelling
Joni Salminen
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Gender effect on e-commerce sales of experience gifts: Preliminary empirical ...
Joni Salminen
Tips for Scale Development: Evaluating Automatic Personas
Tips for Scale Development: Evaluating Automatic PersonasTips for Scale Development: Evaluating Automatic Personas
Tips for Scale Development: Evaluating Automatic Personas
Joni Salminen
Big Data, Small Personas: Research Agenda for Automatic Persona Generation
Big Data, Small Personas: Research Agenda for Automatic Persona GenerationBig Data, Small Personas: Research Agenda for Automatic Persona Generation
Big Data, Small Personas: Research Agenda for Automatic Persona Generation
Joni Salminen
Why do startups avoid difficult problems?
Why do startups avoid difficult problems?Why do startups avoid difficult problems?
Why do startups avoid difficult problems?
Joni Salminen
Social Espionage: Drawing Benefit from Competitors Social Media Presence
Social Espionage: Drawing Benefit from Competitors Social Media PresenceSocial Espionage: Drawing Benefit from Competitors Social Media Presence
Social Espionage: Drawing Benefit from Competitors Social Media Presence
Joni Salminen
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Strategic Digital Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Social Media Marketing (Digital Marketing '15 @ Oulu University)
Social Media Marketing (Digital Marketing '15 @ Oulu University)Social Media Marketing (Digital Marketing '15 @ Oulu University)
Social Media Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Search Engine Marketing (Digital Marketing '15 @ Oulu University)Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Search Engine Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Introduction to Digital Marketing (Digital Marketing '15 @ Oulu University)
Joni Salminen
Web Analytics (Digital Marketing '15 @ Oulu University)
Web Analytics (Digital Marketing '15 @ Oulu University)Web Analytics (Digital Marketing '15 @ Oulu University)
Web Analytics (Digital Marketing '15 @ Oulu University)
Joni Salminen
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Generating Cultural Personas From Social Data - A Perspective of Middle Easte...
Joni Salminen
Who does what in marketing? Toward an understanding of marketermachine inter...
Who does what in marketing? Toward an understanding of marketermachine inter...Who does what in marketing? Toward an understanding of marketermachine inter...
Who does what in marketing? Toward an understanding of marketermachine inter...
Joni Salminen
Social media marketing for researchers
Social media marketing for researchersSocial media marketing for researchers
Social media marketing for researchers
Joni Salminen
Text analysis: An introduction to methods and techniques
Text analysis: An introduction to methods and techniquesText analysis: An introduction to methods and techniques
Text analysis: An introduction to methods and techniques
Joni Salminen
How has the internet transformed marketing
How has the internet transformed marketingHow has the internet transformed marketing
How has the internet transformed marketing
Joni Salminen
Konversio-optimointi (2016)
Konversio-optimointi (2016)Konversio-optimointi (2016)
Konversio-optimointi (2016)
Joni Salminen

Recently uploaded (20)

Different Strategies in Scientific Publishing
Different Strategies in Scientific PublishingDifferent Strategies in Scientific Publishing
Different Strategies in Scientific Publishing
Carlos Baquero
Explore the Museum of the Future in Dubai
Explore the Museum of the Future in DubaiExplore the Museum of the Future in Dubai
Explore the Museum of the Future in Dubai
Damac Cavalli Estates
EVOLUTION OF EUKARYOTIC CELLS AND ENDOSYMBIOSIS.pdf
EVOLUTION OF EUKARYOTIC CELLS AND ENDOSYMBIOSIS.pdfEVOLUTION OF EUKARYOTIC CELLS AND ENDOSYMBIOSIS.pdf
EVOLUTION OF EUKARYOTIC CELLS AND ENDOSYMBIOSIS.pdf
Nistarini College, Purulia (W.B) India
Introduction of Cutaneous Nerves by M.Sundari Singh
Introduction of Cutaneous Nerves  by M.Sundari SinghIntroduction of Cutaneous Nerves  by M.Sundari Singh
Introduction of Cutaneous Nerves by M.Sundari Singh
meharsundarisingh
Importance and Essentials and Necessities of Cell and Molecular Biology
Importance and Essentials and Necessities of Cell and Molecular BiologyImportance and Essentials and Necessities of Cell and Molecular Biology
Importance and Essentials and Necessities of Cell and Molecular Biology
johnfreeguydoe
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills MN
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Anne Barkley
basic tissuse oral epithelium the classifications and subunits
basic tissuse oral epithelium the classifications and subunitsbasic tissuse oral epithelium the classifications and subunits
basic tissuse oral epithelium the classifications and subunits
jemimahrachel1299
Mixing small molecules and macromolecules in the world of informatics
Mixing small molecules and macromolecules in the world of informaticsMixing small molecules and macromolecules in the world of informatics
Mixing small molecules and macromolecules in the world of informatics
Alex Clark
Sciences of Europe No 161 (2025)
Sciences of Europe No 161 (2025)Sciences of Europe No 161 (2025)
Sciences of Europe No 161 (2025)
Sciences of Europe
Vaccines: types, preparations, efficacies and recent developments.pptx
Vaccines: types, preparations, efficacies and recent developments.pptxVaccines: types, preparations, efficacies and recent developments.pptx
Vaccines: types, preparations, efficacies and recent developments.pptx
krishna moorthy
TYPES OF WAVES.pptx can be use for science 10
TYPES OF WAVES.pptx can be use for science 10TYPES OF WAVES.pptx can be use for science 10
TYPES OF WAVES.pptx can be use for science 10
BaltazarRosales2
Membrane polymers for specific applications.pptx
Membrane polymers for specific applications.pptxMembrane polymers for specific applications.pptx
Membrane polymers for specific applications.pptx
JinnJinnkiJaddu
Fetal Circulation - General anatomy by M.Suvarna
Fetal Circulation - General anatomy by  M.SuvarnaFetal Circulation - General anatomy by  M.Suvarna
Fetal Circulation - General anatomy by M.Suvarna
suvarnamattukoyya9
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Anne Barkley
Global change and effects on infectious disease
Global change and effects on infectious diseaseGlobal change and effects on infectious disease
Global change and effects on infectious disease
MichaelMahonPhD
The Sheena Bora Murder Case - Forensic Science.pptx
The Sheena Bora Murder Case - Forensic Science.pptxThe Sheena Bora Murder Case - Forensic Science.pptx
The Sheena Bora Murder Case - Forensic Science.pptx
SahilShah140721
COMPLEXATION AND PROTEIN BINDING. B.pharm
COMPLEXATION  AND PROTEIN BINDING. B.pharmCOMPLEXATION  AND PROTEIN BINDING. B.pharm
COMPLEXATION AND PROTEIN BINDING. B.pharm
kalyanshanam6
pre conception counselling Lectures.pptx
pre conception counselling Lectures.pptxpre conception counselling Lectures.pptx
pre conception counselling Lectures.pptx
drvarunapathak
GNU Linux - Introduction and Administration.
GNU Linux - Introduction and Administration.GNU Linux - Introduction and Administration.
GNU Linux - Introduction and Administration.
Xavier de Pedro
Different Strategies in Scientific Publishing
Different Strategies in Scientific PublishingDifferent Strategies in Scientific Publishing
Different Strategies in Scientific Publishing
Carlos Baquero
Explore the Museum of the Future in Dubai
Explore the Museum of the Future in DubaiExplore the Museum of the Future in Dubai
Explore the Museum of the Future in Dubai
Damac Cavalli Estates
Introduction of Cutaneous Nerves by M.Sundari Singh
Introduction of Cutaneous Nerves  by M.Sundari SinghIntroduction of Cutaneous Nerves  by M.Sundari Singh
Introduction of Cutaneous Nerves by M.Sundari Singh
meharsundarisingh
Importance and Essentials and Necessities of Cell and Molecular Biology
Importance and Essentials and Necessities of Cell and Molecular BiologyImportance and Essentials and Necessities of Cell and Molecular Biology
Importance and Essentials and Necessities of Cell and Molecular Biology
johnfreeguydoe
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills of Minnesota Explores Thermal Energy Concepts Breakthroughs in ...
Travis Hills MN
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Igniting Progress: Outcomes from the FLARE workshop and three challenges for ...
Anne Barkley
basic tissuse oral epithelium the classifications and subunits
basic tissuse oral epithelium the classifications and subunitsbasic tissuse oral epithelium the classifications and subunits
basic tissuse oral epithelium the classifications and subunits
jemimahrachel1299
Mixing small molecules and macromolecules in the world of informatics
Mixing small molecules and macromolecules in the world of informaticsMixing small molecules and macromolecules in the world of informatics
Mixing small molecules and macromolecules in the world of informatics
Alex Clark
Sciences of Europe No 161 (2025)
Sciences of Europe No 161 (2025)Sciences of Europe No 161 (2025)
Sciences of Europe No 161 (2025)
Sciences of Europe
Vaccines: types, preparations, efficacies and recent developments.pptx
Vaccines: types, preparations, efficacies and recent developments.pptxVaccines: types, preparations, efficacies and recent developments.pptx
Vaccines: types, preparations, efficacies and recent developments.pptx
krishna moorthy
TYPES OF WAVES.pptx can be use for science 10
TYPES OF WAVES.pptx can be use for science 10TYPES OF WAVES.pptx can be use for science 10
TYPES OF WAVES.pptx can be use for science 10
BaltazarRosales2
Membrane polymers for specific applications.pptx
Membrane polymers for specific applications.pptxMembrane polymers for specific applications.pptx
Membrane polymers for specific applications.pptx
JinnJinnkiJaddu
Fetal Circulation - General anatomy by M.Suvarna
Fetal Circulation - General anatomy by  M.SuvarnaFetal Circulation - General anatomy by  M.Suvarna
Fetal Circulation - General anatomy by M.Suvarna
suvarnamattukoyya9
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Patagonian dust, Agulhas Current, and Antarctic ice-rafted debris contributio...
Anne Barkley
Global change and effects on infectious disease
Global change and effects on infectious diseaseGlobal change and effects on infectious disease
Global change and effects on infectious disease
MichaelMahonPhD
The Sheena Bora Murder Case - Forensic Science.pptx
The Sheena Bora Murder Case - Forensic Science.pptxThe Sheena Bora Murder Case - Forensic Science.pptx
The Sheena Bora Murder Case - Forensic Science.pptx
SahilShah140721
COMPLEXATION AND PROTEIN BINDING. B.pharm
COMPLEXATION  AND PROTEIN BINDING. B.pharmCOMPLEXATION  AND PROTEIN BINDING. B.pharm
COMPLEXATION AND PROTEIN BINDING. B.pharm
kalyanshanam6
pre conception counselling Lectures.pptx
pre conception counselling Lectures.pptxpre conception counselling Lectures.pptx
pre conception counselling Lectures.pptx
drvarunapathak
GNU Linux - Introduction and Administration.
GNU Linux - Introduction and Administration.GNU Linux - Introduction and Administration.
GNU Linux - Introduction and Administration.
Xavier de Pedro

Problem of majority voting

  • 9. According to the majority rule, with three raters, 100% of the comments have at least 66.7% agreement (2/3). Our data shows that when the same comments are evaluated by five raters, 40.1% (n=117) of the comments have at least 66.7% agreement. With seven raters, 43.5% (n=127) have at least 66.7% agreement, and for nine raters, the value is 44.9% (n=131). 0 10 20 30 40 50 60 70 80 90 100 Three Five Seven Nine Percentage At least 66.7% agreement