Analyzing an Interactive Chatbot and its Impact on Academic Reference Services
Download as PPTX, PDF
1 like144 views
This document analyzes ANTswers, an interactive chatbot at the University of California, Irvine library that has been used by over 2,786 patrons since 2014. It contains statistics on usage of the chatbot and an analysis of the language patrons use when interacting with it. The analysis found that patrons most commonly use interrogative sentences to ask questions, and 17% include opening or closing phrases when chatting. The document also outlines research questions about how patron usage varies by time of day and what types of questions are most commonly asked.
Analyzing an Interactive Chatbot and its Impact on Academic Reference Services
- 1. Analyzing an Interactive Chatbot and its Impact on Academic Reference Services DANIELLE KANE UNIVERSITY OF CALIFORNIA, IRVINE ACRL 2019 CONTRIBUTED PAPER
- 2. About ANTswers ANTswers went live in 2014 Contains 128 AIML files 2,786 patron transcripts with over 10,000 questions asked Provides consistent answers 24/7 No personal information is tracked ANTswers is anonymous Modeled after Peter the Anteater (UCI Mascot) Loves everything UCI related There are a lot of responses about ants! Some patrons just want to chat ANTswers gets asked out often
- 8. Usage Statistics 2014 - 2018
- 16. Research Questions 1. What language structure do patrons use when using the ANTswers chatbot 2. Do people use this service at different hours of the day? 3. Is the assumption accurate that this service point attracts directional, printing / equipment, library holdings questions, and / or library policy questions
- 18. Opening Phrase Greetings are a common courtesy Patrons used and opening phrase 17% of the time Hi and Hello were used the most often Some patrons want to create a relationship (or pretend to create a relationship) even with a computer program
- 19. Showing Interest Showing interest in ANTswers , patrons attempt to build a connection 248 transcripts included the patron asking questions about ANTswers Patrons liked to ask if ANTswers would go out with them
- 20. Type of Question Declarative statements that tend to end with a period, they tend to lack emotion Interrogative - used to ask questions or make requests, usually end with a question marks Exclamatory forceful version of a declarative sentence, conveys excitement or emotion Imperative give a command or impart instructions, make a request
- 21. Closing Phrase 142 transcripts (17%) used a closing phrase Thank you / thanks was used 5% of the time
- 24. Research Questions 1. What language structure do patrons use when using the ANTswers chatbot 2. Do people use this service at different hours of the day? 3. Is the assumption accurate that this service point attracts directional, printing / equipment, library holdings questions, and / or library policy questions
- 25. ThankYou Danielle Kane University of California, Irvine Digital Scholarship Services Data Management & Curation Librarian https://orcid.org/0000-0002-4086-0203 kaned@uci.edu https://program-o.com https://scholarship.org/uc/uci_libs_antswers https://github.com/UCI-Libraries/ANTswers https://dash.lib.uci.edu/stash/dataset/doi:10.72080/D1Po75
Editor's Notes
- Remember to mention that the stats are so small for summer 18 because of ANTswers going down for 3 weeks to switch over for our catalog antpac to our discover layer library search.
- Out of 2,786 antswers transcripts an opening phrase such as hello, hey, hi, etc were used 460 times or 17%. There are a small number of patrons who just want to chat and they tend to treat ANTswers as if the chatbot is human.
- Of the 248 transcripts 48 (19%) asking How are you 24 (10%) wanted to know what the chatbots name is 23 (9%) wanted to know what it does or what he was 33 (13) asked if the chatbot was human, a robot, or an anteater
- One of the difficulties is that patrons dont tend to use punctuation when talking to ANTswers
- Second layer was used to evaluate user needs: Coded manually and I used the UCI libraries structure to create the structure More than one topic can be asked during a conversation so there were a total of 3,536 total requests Services Find About Library Subject related questions About uci
- 1. Interrogative - used to ask questions or make requests, usually end with a question marks Imperative give a command or impart instructions, make a request 2. While ANTswers is available 24/7 its highest amount of use is during regular working hours. With 1 and 2 PM being the hours with the highest use. ANTswers is used all hours of the day 3. borrowing, vpn, finding books very few patrons asked what could be considered in-depth research questions Library supported Solr search ANTswers data was utilized in a tagging project to improve search capabilities Hours on library homepage Further research: Sentiment analysis Softeners Inappropriate questions in comparison to other chat reference services Do this type of comparison with the reference desk and questionpoint chat