Luis von Ahn discusses CAPTCHAs and the ESP Game, which are methods to utilize human brainpower to solve problems computers currently cannot. CAPTCHAs test whether a user is human by having them read distorted text, while the ESP Game labels images by having online partners try to describe the same image without communicating. Both work by crowdsourcing many small tasks. The ESP Game in particular has labeled over 1.5 million images with 15,000 players. The goal is to eventually label all images on the web to improve search, accessibility, and computer vision.
27. CAPTCHA SWEAT SHOPS
SPAM COMPANIES HIRE HUMANS TO
SOLVE CAPTCHAS ALL DAY LONG
$2.50 PER HOUR FOR EACH HUMAN
720 CAPTCHAS PER HOUR PER HUMAN
1/3 CENT PER ACCOUNT
35. USING HUMANS
THE ESP GAME COULD LABEL ALL
IMAGES ON THE WEB IN 30 DAYS!
THE ESP GAME
CLEVERLY
36. TWO-PLAYER ONLINE GAME
PARTNERS DONT KNOW EACH OTHER
AND CANT COMMUNICATE
OBJECT OF THE GAME:
TYPE THE SAME WORD
THE ONLY THING IN COMMON IS
AN IMAGE
THE ESP GAME
37. PLAYER 1 PLAYER 2
GUESSING: CAR GUESSING: BOY
GUESSING: CAR
SUCCESS!
YOU AGREE ON CAR
SUCCESS!
YOU AGREE ON CAR
GUESSING: KID
GUESSING: HAT
THE ESP GAME
38. 息 2004 Carnegie Mellon University, all rights reserved. Patent Pending.
39. 4.1 MILLION LABELS WITH 23,000 PLAYERS
THE ESP GAME IS FUN
THERE ARE MANY PEOPLE THAT PLAY
OVER 20 HOURS A WEEK
40. LABELING THE ENTIRE WEB
INDIVIDUAL GAMES IN YAHOO! AND MSN
AVERAGE OVER 5,000 PLAYERS AT A TIME
5000 PEOPLE PLAYING SIMULTANEOUSLY CAN
LABEL ALL IMAGES ON GOOGLE IN 30 DAYS!
41. A FEW MILLION LABELS
CAN IMPROVE IMAGE SEARCH
CAN BE USED TO IMPROVE
COMPUTER VISION
42. SINGLE PLAYER GAME
A SINGLE PERSON CAN PLAY WITH PRE-
RECORDED ACTIONS AS THEIR PARTNER
WHEN 2 PEOPLE PLAY, WE
RECORD EVERY ACTION
WITH TIMING INFORMATION
WE EMULATE PARTNER
BY PLAYING PRE-
RECORDED MOVES(0:12) CAR
(0:15) HAT
(0:21) KID
(0:08) BOY
(0:23) CAR
NOTICE THAT THIS
DOESNT STOP THE
LABELING PROCESS!
44. WE GIVE PLAYERS TEST IMAGES FOR WHICH
WE KNOW ALL THE COMMON LABELS:
WE ONLY STORE A PLAYERS GUESSES
IF THEY SUCCESSFULLY LABEL THE
TEST IMAGES
WHAT ABOUT CHEATING?
49. THE ESP GAME GIVES ITS PLAYERS A
WEIRD AND BEAUTIFUL SENSE OF
ANONYMOUS INTIMACY. ON THE ONE HAND,
YOU HAVE NO IDEA WHO YOUR PARTNER
IS. ON THE OTHER HAND, THE TWO OF YOU
ARE BRINGING YOUR MINDS TOGETHER IN
A WAY THAT LOVERS WOULD ENVY.
THE ESP GAME GIVES ITS PLAYERS A
WEIRD AND BEAUTIFUL SENSE OF
ANONYMOUS INTIMACY. ON THE ONE HAND,
YOU HAVE NO IDEA WHO YOUR PARTNER
IS.
THE ESP GAME GIVES ITS PLAYERS A
WEIRD AND BEAUTIFUL SENSE OF
ANONYMOUS INTIMACY.
50. STRANGELY ADDICTIVE
ITS SO MUCH FUN TRYNG TO GUESS
WHAT OTHERS THINK. YOU HAVE TO
STEP OUTSIDE OF YOURSELF TO
MATCH
ITS FAST-PACED
HELPS ME LEARN ENGLISH
51. BLOGS
IN OTHER CROWNING ACHEIVEMENTS,
I HAVEN'T PLAYED IN AT LEAST A WEEK
(PROBABLY LONGER), AND I AM STILL ON THE
TOP TEN LIST OF PLAYERS ON THE ESP GAME.
NUMBER EIGHT IF I REMEMBER CORRECTLY.
GOD, MY LIFE IS SAD.
52. THERE ARE MANY FASCINATING THINGS
ABOUT THE ESP GAME:
HOW IT USES A GAME STRUCTURE TO
BUILD AN INDEX OF IMAGES ON THE WEB
WHAT WORDS BECOME TABOO FROM
FREQUENT USE
HOW INTERESTING SOME OF THE IMAGES
PULLED RANDOMLY FROM THE WEB ARE
BUT BY FAR THE MOST INTRIGUING ASPECT
OF THE GAME IS HOW OFTEN YOUR RANDOM,
UNKNOWN PARTNER IS A COMPLETE IDIOT.
54. LOCATING OBJECTS IN IMAGES
THE ESP GAME TELLS US IF AN IMAGE
CONTAINS A SPECIFIC OBJECT, BUT DOESNT
SAY WHERE IN THE IMAGE THE OBJECT IS
SUCH INFORMATION WOULD BE EXTREMELY
USEFUL FOR COMPUTER VISION RESEARCH
55. PAINTBALL GAME
PLAYERS SHOOT AT OBJECTS ON THE IMAGE
SHOOT THE:
CAR
WE GIVE POINTS AND CHECK ACCURACY
BY GIVING PLAYERS IMAGES FOR WHICH
WE ALREADY KNOW WHERE THE OBJECT IS
56. X% OF IMAGES (100-X)% OF IMAGES
PAINTBALL GAME
DONT
KNOW
60. MEDIUM-TERM GOAL
USE THE ESP GAME, ALONG WITH THE PAINT-
BALL GAME AND THE REVEALING GAME TO
CREATE THE MOST ACCURATE COMPUTER
VISION ALGORITHMS TO RECOGNIZE A WIDE
VARIETY OF OBJECT CATEGORIES
62. X OR NOT
CAN BE USED TO CLASSIFY ALL IMAGES ON
THE WEB AS HAVING PROPERTY X OR NOT
WE GIVE PEOPLE
40% OF IMAGES WITH PROPERTY X
40% OF IMAGES WITHOUT PROPERTY X
20% OF UNKNOWN IMAGES
IF PEOPLE LIKE IMAGES WITH PROPERTY X,
WE CAN OFFER AN INCENTIVE: THE MORE
ACCURATE A PLAYER IS, THE MORE IMAGES
WITH PROPERTY X THEY GET
$$$
63. THE POWER OF HUMAN CYCLES
OPEN
PROBLEM
CAPTCHA
GAME
65. OUR WORK ON CAPTCHA HAS
APPEARED IN OVER 100
NEWSPAPERS AROUND THE WORLD
67. THE ESP GAME HAS
APPEARED IN OVER 50
NEWSPAPERS AROUND THE
WORLD
69. Name That Image
Computers excel at sifting information, but they have trouble distinguishing a picture
of a tree from one of a turtle.
So researchers at Carnegie Mellon University knew they needed human volunteers
to successfully label millions of unmarked images on the Web. Luis von Ahn, a
graduate student in computer science, had the inspired idea of turning the laborious
process into a two-player contest called the ESP Game (espgame.org).
Each time you play, you are randomly paired with another anonymous player. You
can't communicate with your partner, although you both see the same image. The
goal is to guess the descriptive word your partner is typing for the image. Once you
both type the same word, you see a new image.
Since October the game has attracted 15,000 registered players, and they have
created more than 1.5 million labels (a label is a single descriptive word; most
photos require multiple labels). The site draws from a database of 200,000 Web
images; another 200,000 are to be added soon. Mr. von Ahn said that properly
labeling Web images would allow for more efficient image searching, improve the
screen readers used by the visually impaired and help users block inappropriate
images.
73. SLIDE DESIGN
LUIS VON AHN
CAPTCHA CREW
LUIS VON AHN
MANUEL BLUM
NICHOLAS HOPPER
JOHN LANGFORD
THE ESP GAME CREW
LUIS VON AHN
LAURA DABBISH
SPEAKERS ENTERTAINMENT
PROVIDED BY
HOMESTARRUNNER.COM
SPECIAL THANKS TO
ADITYA AKELLA
JOSH BENALOH
LENORE BLUM
MANUEL BLUM
ANDREW BORTZ
SHUCHI CHAWLA
LAURA DABBISH
BILL GATES
BRIGHTEN GODFREY
NICK HOPPER
SCOTT HUDSON
TAKEO KANADE
STEVEN KATZ
JOHN LANGFORD
JITENDRA MALIK
WARDROBE PROVIDED BY
THE GAP
HAIRSTYLES
ARNAUD SOME-FRENCH-LAST-NAME
THIS WORK HAS BEEN
AND WILL BE PARTIALLY
SUPPORTED BY
MANUEL BLUM
THE ALADDIN CENTER
MICROSOFT RESEARCH
MY MOM
YOUR TAX DOLLARS
MORE SPECIAL THANKS
UDI MANBER
MONI NAOR
LENORE RAMM
CHUCK ROSENBERG
RONI ROSENFELD
DAVID STORK
LATANYA SWEENEY
SHINGO UCHIHASHI
MANUELA VELOSO