The document summarizes research on evaluating the user experience of recommender systems. It presents hypotheses about how personalized recommendations versus random recommendations affect user perception, choice satisfaction, and feedback behavior. An experiment tested the hypotheses using a video recommender system and found that personalized recommendations increased perceived quality and choice satisfaction, which in turn increased feedback intentions. Privacy concerns decreased feedback intentions while trust in technology reduced privacy concerns. The summarizes lessons learned and discusses areas for future work such as confirming results in other systems and incorporating additional influences.
The document discusses techniques for sentiment analysis and opinion mining from social media, including defining the concepts, describing business interests and software packages for analyzing sentiment, and explaining how sentiment analysis can be done at the document, sentence, and entity level through automated classification of text. It also provides examples of sentiment analysis at the document level as a text classification problem using supervised machine learning algorithms.
The document discusses human centered software design (HCSD) and its benefits. It promotes incorporating human-centered design (HCD) methods into traditional software engineering processes. These methods include interviews, personas, scenarios, storyboards and user testing. When done effectively through iterative design and testing with users, HCSD can lead to increased traffic, sales, user happiness and productivity. The document uses examples from various companies and from a student project at UC Irvine to show how HCSD works in practice.
Inspectability and Control in Social RecommendersBart Knijnenburg
油
1. The study examined how providing users with inspectability and control over recommendations in a social recommender system impacts user experience.
2. The results showed that giving users inspectability through a full graph interface increased understandability and perceived control compared to a list interface. It also improved users' recognition of known recommendations.
3. Allowing users to control recommendations at the item level led to higher novelty through fewer known recommendations, while control at the friend level increased accuracy.
4. Overall, the findings suggest that social recommenders should provide users with inspectability and control through a simple interface to improve the user experience.
This document presents research on profiling Facebook users' privacy behaviors. A survey was conducted with over 300 Facebook users to understand their use of various privacy features and settings. Statistical analysis identified 14 distinct privacy behavior factors. Further analysis classified users into 6 privacy behavior profiles, ranging from "Privacy Maximizers" to "Privacy Minimalists". The profiles differed in their use of features like friend lists, untagging posts, and restricting profile access. The research aims to better understand how to personalize privacy tools based on users' tendencies.
Tutorial on Conducting User Experiments in Recommender SystemsBart Knijnenburg
油
The document provides an introduction to user experiments for evaluating recommender systems. It discusses developing a theoretical framework for user-centric evaluation with four key aspects: 1) measuring how system algorithms and interactions influence user experience and behavior, 2) considering subjective user perceptions and experiences in addition to objective behaviors, 3) accounting for personal and situational characteristics that may impact results, and 4) linking objective system aspects to subjective experience to understand how system aspects affect user experience. The goal is to scientifically evaluate recommender systems from the user perspective using this comprehensive framework.
Understanding choice overload in recommender systemsdirkheld
油
Even though people are attracted by large, high quality rec- ommendation sets, psychological research on choice overload shows that choosing an item from recommendation sets con- taining many attractive items can be a very difficult task. A web-based user experiment using a matrix factorization algorithm applied to the MovieLens dataset was used to investigate the effect of recommendation set size (5 or 20 items) and set quality (low or high) on perceived variety, recommendation set attractiveness, choice difficulty and sat- isfaction with the chosen item. The results show that larger sets containing only good items do not necessarily result in higher choice satisfaction compared to smaller sets, as the increased recommendation set attractiveness is counteracted by the increased difficulty of choosing from these sets. These findings were supported by behavioral measurements reveal- ing intensified information search and increased acquisition times for these large attractive sets. Important implications of these findings for the design of recommender system user interfaces will be discussed.
Helping Users with Information Disclosure Decisions: Potential for Adaptation...Bart Knijnenburg
油
The document describes an experiment that tested different types of justifications for personal information disclosure requests from mobile apps. The experiment tested different justification types (no justification, usefulness for the user, number of others disclosing, usefulness for others, explanation), disclosure request order (context data first vs demographics first), and measured their impact on disclosure rates, perceived value of disclosure, perceived privacy threat, trust in the company, and satisfaction with the system. The results showed that no justification led to the highest disclosure rates, and justifications were perceived as generally helpful except for number of others. The justification of usefulness for others led to higher perceived privacy threat and lower trust in the company.
The document discusses both the promises and perils of big data. It outlines how big data can enable powerful personalized recommendations through techniques like matrix factorization but also how overfitting and a lack of domain knowledge can limit solutions. It emphasizes the need for user experiments to evaluate recommendations and the importance of balancing privacy concerns with personalization through transparency and adaptive defaults.
Privacy in Mobile Personalized Systems - The Effect of Disclosure JustificationsBart Knijnenburg
油
Paper Presentation at the Workshop on Usable Privacy & Security for Mobile Devices (U-PriSM) at the Symposium On Usable Privacy and Security (SOUPS) 2012
Paper can be found here: http://appanalysis.org/u-prism/soups12_mobile-final11.pdf
Full journal paper (under review): http://bit.ly/TiiSprivacy
Information Disclosure Profiles for Segmentation and RecommendationBart Knijnenburg
油
The document discusses moving beyond a one-size-fits-all approach to privacy by developing privacy profiles based on different tendencies to disclose types of information. These profiles can be used to provide tailored privacy recommendations and defaults by predicting a user's disclosure behaviors based on their profile, type of information, and recipient. The goal is to support individual privacy preferences while reducing the complexity of privacy controls.
Simplifying Privacy Decisions: Towards Interactive and Adaptive SolutionsBart Knijnenburg
油
The document discusses approaches to simplifying privacy decisions through interactive and adaptive solutions. It first examines how transparency and control approaches have limitations due to bounded rationality, information overload, and choice overload. It then discusses privacy nudging and persuasion approaches using defaults, justifications, and framing to influence decisions. However, these approaches can also reduce user satisfaction and autonomy. The document proposes an adaptive privacy procedure to provide contextualized nudges based on a dynamic understanding of user concerns.
Explaining the User Experience of Recommender Systems with User ExperimentsBart Knijnenburg
油
A talk I gave at the Netflix offices on July 2nd, 2012.
Please do not use any of the slides or their contents without my explicit permission (bart@usabart.nl for inquiries).
Counteracting the negative effect of form auto-completion on the privacy calc...Bart Knijnenburg
油
This document discusses how form auto-completion tools can negatively impact users' privacy calculus by making it too easy to disclose information without weighing risks and benefits. The researchers propose two new tools - Remove FormFiller and Add FormFiller - that allow users to manually remove or add filled fields, hypothesizing this will reinstate the privacy calculus. They conducted an experiment where participants used an auto-completion tool on forms for different websites (a blog, job site, health insurer). Results showed perceived risk was lower and relevance higher when the type of information matched the website purpose, supporting the role of purpose-specificity in disclosure decisions.
Preference-based Location Sharing: Are More Privacy Options Really Better?Bart Knijnenburg
油
1. The document examines how adding and removing location sharing options affects user preferences. It studies four options: nothing, city, city block, and exact location.
2. The researchers hypothesize that removing the "city" option will either cause users to choose more private options proportionally, or to shift more towards the more revealing "block" option, depending on how close "city" is perceived to the other options.
3. A user study found that when "city" was removed, the share of users choosing "block" increased significantly, suggesting "city" is perceived closer to "block" than "nothing." Adding an "exact" option caused proportional increases across options, suggesting equal perceived distances.
University of Costa Rica
LM-1351
Rosa Mar鱈a Rodr鱈guez P.
A64857
The document defines cohabitation as two people living together without being married and discusses it as an alternative to marriage for couples. It notes advantages like over half of first marriages now being preceded by cohabitation compared to 50 years ago. The document also discusses points of view against cohabitation, citing research that links low religious participation to higher rates of cohabitation and lower rates of subsequent marriage. It briefly mentions cohabitation among gay couples and provides some statistics on views of cohabitation's morality and rates of previous marriage among cohabiting couples.
Caching for Performance Masterclass: The In-Memory DatastoreScyllaDB
油
Understanding where in-memory data stores help most and where teams get into trouble.
- Where in the stack to cache
- Memcached as a tool
- Modern cache primitives
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIASrivaanchi Nathan
油
This business intelligence report, "The Big Ten Biopharmaceutical MNCs: Global Capability Centers in India", provides an in-depth analysis of the operations and contributions of the Global Capability Centers (GCCs) of ten leading biopharmaceutical multinational corporations in India. The report covers AstraZeneca, Bayer, Bristol Myers Squibb, GlaxoSmithKline (GSK), Novartis, Sanofi, Roche, Pfizer, Novo Nordisk, and Eli Lilly. In this report each company's GCC is profiled with details on location, workforce size, investment, and the strategic roles these centers play in global business operations, research and development, and information technology and digital innovation.
UiPath Document Understanding - Generative AI and Active learning capabilitiesDianaGray10
油
This session focus on Generative AI features and Active learning modern experience with Document understanding.
Topics Covered:
Overview of Document Understanding
How Generative Annotation works?
What is Generative Classification?
How to use Generative Extraction activities?
What is Generative Validation?
How Active learning modern experience accelerate model training?
Q/A
If you have any questions or feedback, please refer to the "Women in Automation 2025" dedicated Forum thread. You can find there extra details and updates.
Data-Driven Public Safety: Reliable Data When Every Second CountsSafe Software
油
When every second counts, you need access to data you can trust. In this webinar, well explore how FME empowers public safety services to streamline their operations and safeguard communities. This session will showcase workflow examples that public safety teams leverage every day.
Well cover real-world use cases and demo workflows, including:
Automating Police Traffic Stop Compliance: Learn how the City of Fremont meets traffic stop data standards by automating QA/QC processes, generating error reports saving over 2,800 hours annually on manual tasks.
Anonymizing Crime Data: Discover how cities protect citizen privacy while enabling transparent and trustworthy open data sharing.
Next Gen 9-1-1 Integration: Explore how Santa Clara County supports the transition to digital emergency response systems for faster, more accurate dispatching, including automated schema mapping for address standardization.
Extreme Heat Alerts: See how FME supports disaster risk management by automating the delivery of extreme heat alerts for proactive emergency response.
Our goal is to provide practical workflows and actionable steps you can implement right away. Plus, well provide quick steps to find more information about our public safety subscription for Police, Fire Departments, EMS, HAZMAT teams, and more.
Whether youre in a call center, on the ground, or managing operations, this webinar is crafted to help you leverage data to make informed, timely decisions that matter most.
5 Best Agentic AI Frameworks for 2025.pdfSoluLab1231
油
AI chatbots use generative AI to develop answers from a single interaction. When someone asks a question, the chatbot responds using a natural language process (NLP). Agentic AI, the next wave of artificial intelligence, goes beyond this by solving complicated multistep problems on its way by using advanced reasoning and iterative planning. Additionally, it is expected to improve operations and productivity across all sectors.
Predictive vs. Preventive Maintenance Which One is Right for Your FactoryDiagsense ltd
油
Efficient maintenance is the backbone of any manufacturing operation. It ensures that machinery runs smoothly, minimizes downtime and optimizes overall productivity. Earlier, factories have relied on preventive maintenance but with advancements in technology, Manufacturing PdM Solutions is gaining traction. The question iswhich one is the right fit for your factory? Lets break it down.
Not a Kubernetes fan? The state of PaaS in 2025Anthony Dahanne
油
Kubernetes won the containers orchestration war. But has it made deploying your apps easier?
Let's explore some of Kubernetes extensive app developer tooling, but mainly what the PaaS space looks like in 2025; 18 years after Heroku made it popular.
Is Heroku still around? What about Cloud Foundry?
And what are those new comers (fly.io, railway, porter.sh, etc.) worth?
Did the Cloud giants replace them all?
Computational Photography: How Technology is Changing Way We Capture the WorldHusseinMalikMammadli
油
Computational Photography (Computer Vision/Image): How Technology is Changing the Way We Capture the World
He巽 d端端nm端s端n端zm端, m端asir smartfonlar v kameralar nec bu qdr g旦zl g旦r端nt端lr yarad脹r? Bunun sirri Computational Fotoqrafiyas脹nda(Computer Vision/Imaging) gizlidirkillri 巽km v emal etm 端sulumuzu tkmilldirn, komp端ter elmi il fotoqrafiyan脹n inqilabi birlmsi.
Webinar: LF Energy GEISA: Addressing edge interoperability at the meterDanBrown980551
油
This webinar will introduce the Grid Edge Security and Interoperability Alliance, or GEISA, an effort within LF Energy to address application interoperability at the very edge of the utility network: meters and other distribution automation devices. Over the last decade platform manufacturers have introduced the ability to run applications on electricity meters and other edge devices. Unfortunately, while many of these efforts have been built on Linux, they havent been interoperable. APIs and execution environment have varied from one manufacturer to the next making it impossible for utilities to obtain applications that they can run across a fleet of different devices. For utilities that want to minimize their supply chain risk by obtaining equipment from multiple suppliers, they are forced to run and maintain multiple separate management systems. Applications available for one device may need to be ported to run on another, or they may not be available at all.
GEISA addresses this by creating a vendor neutral specification for utility edge computing environments. This webinar will discuss why GEISA is important to utilities, the specific issues GEISA will solve and the new opportunities it creates for utilities, platform vendors, and application vendors.
The document discusses both the promises and perils of big data. It outlines how big data can enable powerful personalized recommendations through techniques like matrix factorization but also how overfitting and a lack of domain knowledge can limit solutions. It emphasizes the need for user experiments to evaluate recommendations and the importance of balancing privacy concerns with personalization through transparency and adaptive defaults.
Privacy in Mobile Personalized Systems - The Effect of Disclosure JustificationsBart Knijnenburg
油
Paper Presentation at the Workshop on Usable Privacy & Security for Mobile Devices (U-PriSM) at the Symposium On Usable Privacy and Security (SOUPS) 2012
Paper can be found here: http://appanalysis.org/u-prism/soups12_mobile-final11.pdf
Full journal paper (under review): http://bit.ly/TiiSprivacy
Information Disclosure Profiles for Segmentation and RecommendationBart Knijnenburg
油
The document discusses moving beyond a one-size-fits-all approach to privacy by developing privacy profiles based on different tendencies to disclose types of information. These profiles can be used to provide tailored privacy recommendations and defaults by predicting a user's disclosure behaviors based on their profile, type of information, and recipient. The goal is to support individual privacy preferences while reducing the complexity of privacy controls.
Simplifying Privacy Decisions: Towards Interactive and Adaptive SolutionsBart Knijnenburg
油
The document discusses approaches to simplifying privacy decisions through interactive and adaptive solutions. It first examines how transparency and control approaches have limitations due to bounded rationality, information overload, and choice overload. It then discusses privacy nudging and persuasion approaches using defaults, justifications, and framing to influence decisions. However, these approaches can also reduce user satisfaction and autonomy. The document proposes an adaptive privacy procedure to provide contextualized nudges based on a dynamic understanding of user concerns.
Explaining the User Experience of Recommender Systems with User ExperimentsBart Knijnenburg
油
A talk I gave at the Netflix offices on July 2nd, 2012.
Please do not use any of the slides or their contents without my explicit permission (bart@usabart.nl for inquiries).
Counteracting the negative effect of form auto-completion on the privacy calc...Bart Knijnenburg
油
This document discusses how form auto-completion tools can negatively impact users' privacy calculus by making it too easy to disclose information without weighing risks and benefits. The researchers propose two new tools - Remove FormFiller and Add FormFiller - that allow users to manually remove or add filled fields, hypothesizing this will reinstate the privacy calculus. They conducted an experiment where participants used an auto-completion tool on forms for different websites (a blog, job site, health insurer). Results showed perceived risk was lower and relevance higher when the type of information matched the website purpose, supporting the role of purpose-specificity in disclosure decisions.
Preference-based Location Sharing: Are More Privacy Options Really Better?Bart Knijnenburg
油
1. The document examines how adding and removing location sharing options affects user preferences. It studies four options: nothing, city, city block, and exact location.
2. The researchers hypothesize that removing the "city" option will either cause users to choose more private options proportionally, or to shift more towards the more revealing "block" option, depending on how close "city" is perceived to the other options.
3. A user study found that when "city" was removed, the share of users choosing "block" increased significantly, suggesting "city" is perceived closer to "block" than "nothing." Adding an "exact" option caused proportional increases across options, suggesting equal perceived distances.
University of Costa Rica
LM-1351
Rosa Mar鱈a Rodr鱈guez P.
A64857
The document defines cohabitation as two people living together without being married and discusses it as an alternative to marriage for couples. It notes advantages like over half of first marriages now being preceded by cohabitation compared to 50 years ago. The document also discusses points of view against cohabitation, citing research that links low religious participation to higher rates of cohabitation and lower rates of subsequent marriage. It briefly mentions cohabitation among gay couples and provides some statistics on views of cohabitation's morality and rates of previous marriage among cohabiting couples.
Caching for Performance Masterclass: The In-Memory DatastoreScyllaDB
油
Understanding where in-memory data stores help most and where teams get into trouble.
- Where in the stack to cache
- Memcached as a tool
- Modern cache primitives
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIASrivaanchi Nathan
油
This business intelligence report, "The Big Ten Biopharmaceutical MNCs: Global Capability Centers in India", provides an in-depth analysis of the operations and contributions of the Global Capability Centers (GCCs) of ten leading biopharmaceutical multinational corporations in India. The report covers AstraZeneca, Bayer, Bristol Myers Squibb, GlaxoSmithKline (GSK), Novartis, Sanofi, Roche, Pfizer, Novo Nordisk, and Eli Lilly. In this report each company's GCC is profiled with details on location, workforce size, investment, and the strategic roles these centers play in global business operations, research and development, and information technology and digital innovation.
UiPath Document Understanding - Generative AI and Active learning capabilitiesDianaGray10
油
This session focus on Generative AI features and Active learning modern experience with Document understanding.
Topics Covered:
Overview of Document Understanding
How Generative Annotation works?
What is Generative Classification?
How to use Generative Extraction activities?
What is Generative Validation?
How Active learning modern experience accelerate model training?
Q/A
If you have any questions or feedback, please refer to the "Women in Automation 2025" dedicated Forum thread. You can find there extra details and updates.
Data-Driven Public Safety: Reliable Data When Every Second CountsSafe Software
油
When every second counts, you need access to data you can trust. In this webinar, well explore how FME empowers public safety services to streamline their operations and safeguard communities. This session will showcase workflow examples that public safety teams leverage every day.
Well cover real-world use cases and demo workflows, including:
Automating Police Traffic Stop Compliance: Learn how the City of Fremont meets traffic stop data standards by automating QA/QC processes, generating error reports saving over 2,800 hours annually on manual tasks.
Anonymizing Crime Data: Discover how cities protect citizen privacy while enabling transparent and trustworthy open data sharing.
Next Gen 9-1-1 Integration: Explore how Santa Clara County supports the transition to digital emergency response systems for faster, more accurate dispatching, including automated schema mapping for address standardization.
Extreme Heat Alerts: See how FME supports disaster risk management by automating the delivery of extreme heat alerts for proactive emergency response.
Our goal is to provide practical workflows and actionable steps you can implement right away. Plus, well provide quick steps to find more information about our public safety subscription for Police, Fire Departments, EMS, HAZMAT teams, and more.
Whether youre in a call center, on the ground, or managing operations, this webinar is crafted to help you leverage data to make informed, timely decisions that matter most.
5 Best Agentic AI Frameworks for 2025.pdfSoluLab1231
油
AI chatbots use generative AI to develop answers from a single interaction. When someone asks a question, the chatbot responds using a natural language process (NLP). Agentic AI, the next wave of artificial intelligence, goes beyond this by solving complicated multistep problems on its way by using advanced reasoning and iterative planning. Additionally, it is expected to improve operations and productivity across all sectors.
Predictive vs. Preventive Maintenance Which One is Right for Your FactoryDiagsense ltd
油
Efficient maintenance is the backbone of any manufacturing operation. It ensures that machinery runs smoothly, minimizes downtime and optimizes overall productivity. Earlier, factories have relied on preventive maintenance but with advancements in technology, Manufacturing PdM Solutions is gaining traction. The question iswhich one is the right fit for your factory? Lets break it down.
Not a Kubernetes fan? The state of PaaS in 2025Anthony Dahanne
油
Kubernetes won the containers orchestration war. But has it made deploying your apps easier?
Let's explore some of Kubernetes extensive app developer tooling, but mainly what the PaaS space looks like in 2025; 18 years after Heroku made it popular.
Is Heroku still around? What about Cloud Foundry?
And what are those new comers (fly.io, railway, porter.sh, etc.) worth?
Did the Cloud giants replace them all?
Computational Photography: How Technology is Changing Way We Capture the WorldHusseinMalikMammadli
油
Computational Photography (Computer Vision/Image): How Technology is Changing the Way We Capture the World
He巽 d端端nm端s端n端zm端, m端asir smartfonlar v kameralar nec bu qdr g旦zl g旦r端nt端lr yarad脹r? Bunun sirri Computational Fotoqrafiyas脹nda(Computer Vision/Imaging) gizlidirkillri 巽km v emal etm 端sulumuzu tkmilldirn, komp端ter elmi il fotoqrafiyan脹n inqilabi birlmsi.
Webinar: LF Energy GEISA: Addressing edge interoperability at the meterDanBrown980551
油
This webinar will introduce the Grid Edge Security and Interoperability Alliance, or GEISA, an effort within LF Energy to address application interoperability at the very edge of the utility network: meters and other distribution automation devices. Over the last decade platform manufacturers have introduced the ability to run applications on electricity meters and other edge devices. Unfortunately, while many of these efforts have been built on Linux, they havent been interoperable. APIs and execution environment have varied from one manufacturer to the next making it impossible for utilities to obtain applications that they can run across a fleet of different devices. For utilities that want to minimize their supply chain risk by obtaining equipment from multiple suppliers, they are forced to run and maintain multiple separate management systems. Applications available for one device may need to be ported to run on another, or they may not be available at all.
GEISA addresses this by creating a vendor neutral specification for utility edge computing environments. This webinar will discuss why GEISA is important to utilities, the specific issues GEISA will solve and the new opportunities it creates for utilities, platform vendors, and application vendors.
GDG Cloud Southlake #40: Brandon Stokes: How to Build a Great ProductJames Anderson
油
How to Build a Great Product
Being a tech entrepreneur is about providing a remarkable product or service that serves the needs of its customers better, faster, and cheaper than anything else. The goal is to "make something people want" which we call, product market fit.
But how do we get there? We'll explore the process of taking an idea to product market fit (PMF), how you know you have true PMF, and how your product strategies differ pre-PMF from post-PMF.
Brandon is a 3x founder, 1x exit, ex-banker & corporate strategist, car dealership owner, and alumnus of Techstars & Y Combinator. He enjoys building products and services that impact people for the better.
Brandon has had 3 different careers (banking, corporate finance & strategy, technology) in 7 different industries; Investment Banking, CPG, Media & Entertainment, Telecommunications, Consumer application, Automotive, & Fintech/Insuretech.
He's an idea to revenue leader and entrepreneur that helps organizations build products and processes, hire talent, test & iterate quickly, collect feedback, and grow in unregulated and heavily regulated industries.
UiPath Automation Developer Associate Training Series 2025 - Session 1DianaGray10
油
Welcome to UiPath Automation Developer Associate Training Series 2025 - Session 1.
In this session, we will cover the following topics:
Introduction to RPA & UiPath Studio
Overview of RPA and its applications
Introduction to UiPath Studio
Variables & Data Types
Control Flows
You are requested to finish the following self-paced training for this session:
Variables, Constants and Arguments in Studio 2 modules - 1h 30m - https://academy.uipath.com/courses/variables-constants-and-arguments-in-studio
Control Flow in Studio 2 modules - 2h 15m - https:/academy.uipath.com/courses/control-flow-in-studio
鏝 For any questions you may have, please use the dedicated Forum thread. You can tag the hosts and mentors directly and they will reply as soon as possible.
How to teach M365 Copilot and M365 Copilot Chat prompting to your colleagues. Presented at the Advanced Learning Institute's "Internal Communications Strategies with M365" event on February 27, 2025. Intended audience: Internal Communicators, User Adoption Specialists, IT.
5 Must-Use AI Tools to Supercharge Your Productivity!
AI is changing the game! From research to creativity and coding, here are 5 powerful AI tools you should try.
NotebookLM
NotebookLM Your AI Research Assistant
Organizes & summarizes notes
Generates insights from multiple sources
Ideal for students, researchers & writers
Boost your productivity with smarter note-taking!
Napkin.ai
ィ Napkin.ai The Creativity Booster
Connects and organizes ideas
Perfect for writers, designers & entrepreneurs
Acts as your AI-powered brainstorming partner
Unleash your creativity effortlessly!
DeepSeek
DeepSeek Smarter AI Search
Delivers deeper & more precise search results
Analyzes large datasets for better insights
Ideal for professionals & researchers
Find what you needfaster & smarter!
ChatGPT
ChatGPT Your AI Chat Assistant
Answers questions, writes content & assists in coding
Helps businesses with customer support
Boosts learning & productivity
From content to codingChatGPT does it all!
Devin AI
Devin AI AI for Coders
Writes, debugs & optimizes code
Assists developers at all skill levels
Makes coding faster & more efficient
Let AI be your coding partner!
AI is transforming the way we work!
Transcript: AI in publishing: Your questions answered - Tech Forum 2025BookNet Canada
油
George Walkley, a publishing veteran and leading authority on AI applications, joins us for a follow-up to his presentation "Applying AI to publishing: A balanced and ethical approach". George gives a brief overview of developments since that presentation and answers attendees' pressing questions about AIs impact and potential applications in the book industry.
Link to recording and presentation slides: https://bnctechforum.ca/sessions/ai-in-publishing-your-questions-answered/
Presented by BookNet Canada on February 20, 2025 with support from the Department of Canadian Heritage.
4. Recommender systems
Recommend items to users
based on their stated preferences
(e.g. books, movies, laptops)
Users indicate preferences
by rating presented items
(e.g. from one to 鍖ve stars)
Predict the users rating value of new items...
then present items with the highest predicted rating
6. Two premises
Premise 1 | Users want to receive
recommendations
Do recommendations have any effect on the user experience at all?
Compare a system with vs. without recommendations
Premise 2 | Users will provide preference
feedback
Without feedback, no recommendations
What causes - and inhibits - them to do this?
Analyze users feedback behavior and intentions
8. Effect of
Premise 1 | Users want to receive
recommendations
Users are able to notice differences in prediction
accuracy
But... higher accuracy can lead to lower usefulness of
recommendations
Distinction between perception and evaluation
of recommendation quality
9. Constructs and
Perception
Perceived recommendation
quality
User experience
Evaluation Personalized vs.
random
H2a + Choice
satisfaction
Choice satisfaction H1 + Perceived recom-
Perceived system effectiveness mendation quality
Perceived system
H2b + e ectiveness
Questionnaires and
process data
10. Feedback
Premise 2 | Users will provide preference
feedback
Satisfaction increases feedback intentions
However, only a minority is willing to give up personal information
in return for a personalized experience (Teltzrow & Kobsa)
Privacy decreases feedback intentions
However, most people are usually or always comfortable disclosing
personal taste preferences (Ackerman et al.)
11. Constructs and
Feedback
Willingness to provide feedback
User experience
H3a
Privacy Choice
satisfaction
System-speci鍖c privacy
concerns +
Perceived system Intention to
Trust in technology e ectiveness H3b + provide feedback
Process data General trust
in technology
H4 System-speci鍖c
privacy concerns
H5
Actual feedback behavior
12. A model of user
User experience
Personalized vs.
random
H2a + Choice H3a
satisfaction
H1 + Perceived recom-
mendation quality +
Perceived system Intention to
H2b + e ectiveness H3b + provide feedback
General trust H4 System-speci鍖c H5
in technology privacy concerns
13. Experiment
Test
with actual recommender system
Two versions of the Personalized vs.
User experience
random
H2a + Choice H3a
system: H1 + Perceived recom-
satisfaction
mendation quality +
One that provides personalized Perceived system Intention to
+
recommendations H2b + e ectiveness H3b provide feedback
One that provides random clips General trust H4 System-speci鍖c H5
as recommendations in technology privacy concerns
15. Setup
Online experiment
Conducted by EMIC in Germany,
September and October, 2009
Two slightly modi鍖ed versions
of
the MSN ClipClub system
43 participants
25 in the random and 18 in the
personalized condition
65% male, all German
Average age of 31 (SD = 9.45)
16. System
Microsoft ClipClub
Lifestyle & entertainment video
clips
Changes
Recommendations section
highlighted
Pre-experimental instruction
Rating probe
No rating for 鍖ve minutes: ask
user to rate the current item
17. Employed algorithm
Vector Space Model Engine
Use the tags associated to a clip to create a vector of each clip
Create a tag vector for the subset of clips rated by the user
Recommends clips with a tag vector similar to the created tag vector
Older ratings are logarithmically discounted, as are older items
18. Experimental procedure
Each participant:
entered demographic details
was shown an instruction on how to use the system
used the system freely for at least 30 minutes
completed the questionnaires
entered an email address for the raffle
Rating items
Users could perpetually rate items and inspect recommendations in
any given order
Rating probe: at least 6 ratings unless ignored
19. Questionnaires
40 statements Choice satisfaction
9 items, e.g. The videos I chose 鍖tted my
Agree or disagree on a 5-point preference
scale
General trust in technology
Factor Analysis in two batches 4 items, e.g. Im less con鍖dent when I use
technology, reverse-coded
System-speci鍖c privacy concern
6 factors 5 items, e.g. I feel con鍖dent that ClipClub
Recommendation set quality respects my privacy
7 items, e.g. The recommended videos 鍖tted Intention to rate items
my preference
5 items, e.g. I like to give feedback on the
System effectiveness items Im watching
6 items, e.g. The recommender is useless,
reverse-coded
20. Process data
All clicks were logged
In order to link subjective metrics to observable behavior
Process data measures
Total viewing-time
Number of clicked clips
Number of completed clips
Number of self-initiated ratings
Number of canceled rating requests
22. Path model results
Personalized vs.
.572 (.125)*** Choice .346 (.125)**
random
H2a satisfaction H3a
.696 (.276)* Perceived recom-
H1 mendation quality
Perceived system Intention to
.515 (.135)*** e ectiveness .296 (.123)* provide feedback
H2b H3b
General trust -.268 (.156)1 System-speci鍖c -.255 (.113)*
in technology H4 privacy concerns H5
23. Effect of
Personalized vs.
.572 (.125)*** Choice
random
Users notice .696 (.276)*
H2a satisfaction
Perceived recom-
personalization H1 mendation quality
Perceived system
Personalized recommendations .515 (.135)*** e ectiveness
increase perceived H2b
recommendation quality (H1)
Users browse less, but
Users like better watch more
Number of clips watched
recommendations entirely is higher in the
Higher perceived quality personalized condition
increases choice satisfaction Number of clicked clips and
(H2a) and system effectiveness total viewing time are negatively
(H2b) correlated with system
24. Feedback
Choice .346 (.125)**
Better experience satisfaction H3a
increases feedback
Perceived system Intention to
Choice satisfaction and system e ectiveness .296 (.123)* provide feedback
effectiveness increase feedback H3b
intentions (H3a,b)
General trust -.268 (.156)1 System-speci鍖c -.255 (.113)*
in technology H4 privacy concerns H5
Privacy decreases Effect of trust in
feedback technology
Users with a higher system- Privacy concerns increase when
speci鍖c privacy concern have a users have a lower trust in
lower feedback intention (H5) technology (H4).
25. Intention-behavior gap
Number of canceled rating probes
Signi鍖cantly lower in the personalized condition
Negatively correlated with intention to provide feedback
Total number of provided ratings
Not signi鍖cantly correlated with users intention to provide feedback
26. To summarize...
Personalized vs.
.572 (.125)*** Choice .346 (.125)**
random
H2a satisfaction H3a
.696 (.276)* Perceived recom-
H1 mendation quality
Perceived system Intention to
.515 (.135)*** e ectiveness .296 (.123)* provide feedback
H2b H3b
General trust -.268 (.156)1 System-speci鍖c -.255 (.113)*
in technology H4 privacy concerns H5
28. Remaining questions
True for all recommender systems?
Results should be con鍖rmed in several other systems and with a
higher number and a more diverse range of participants
Other in鍖uences?
Incorporate other aspects to get a more detailed understanding of
the mechanisms underlying the user-recommender interaction
Other algorithms?
Test differences between algorithms that only moderately differ in
accuracy
30. Field trails
Full-scale test of the framework
Four different partners, three different countries
Trials are conducted over a longer time-period
Each compares at least three systems (mainly different algorithms)
Questionnaires and process data
Core of evaluation is the same
Algorithm -> perceived recommendation quality -> system
effectiveness
Each partner adds measures of personal interest
31. Want more?
RecSys10 workshop
User-Centric Evaluation of Recommender
attending
Systems and their Interfaces (UCERSTI)
Barcelona, September 26-30
I am
Line-up:
7 paper presentations !"#$%&'
2 keynotes (Francisco Martin, Pearl Pu)
Panel discussion with 5 prominent researchers
1st internation
al workshop on
User-Centric E
valuation of
Recommender
Systems
and Their Inte
rfaces
Editor's Notes
#3: First I want to thank my co-authors and sponsor
#5: Your typical recommender system works like this:
#6: Right now, researchers seem to focus on the algorithmic performance. They believe that better algorithms lead to a better experience. Is that really true?
#7: It can only be true under two assumptions:
1. users want to get personalized recommendations, and 2. they will provide enough feedback to make this possible
In order to answer these questions, we need to evaluate the user experience, not the algorithm!
#9: What existing evidence do we have?
Increased recommendation accuracy is noticeable, but doesn’t always lead to a better UX
McNee et al.: algorithm with best predictions was rated least helpful
Torres et al.: algorithm with lowest accuracy resulted in highest satisfaction
Ziegler et al.: diversifying recommendation set resulted in lower accuracy but a more positive evaluation
#10: Let’s say we have two systems, one with personalized recommendations, and one without:
Perception tests whether we are able to notice the difference
Evaluation tests whether this increases our satisfaction with the system and, ultimately, our choices
These are measures by questionnaires, but we can also look at process data:
Effective systems may show decreased browsing and overall viewing time
In better systems, users will watch more clips from beginning to end
#11: The more beneficial it seems to be, the more feedback users will provide (Spiekermann et al.; Brodie Karat & Karat; Kobsa & Teltzrow)
Minority = Between 40 and 50% in an overview of privacy surveys
Privacy concerns reduce users’ willingness to disclose personal information (Metzger et al.; Teltzrow & Kobsa)
Most people = 80% of the respondents of a detailed survey
Users’ actual feedback behavior may be different from their intentions (Spiekermann et al.)
#12: So now we look at why users provide preference information
We already know choice satisfaction and perceived system effectiveness, and we hypothesize that a better experience increase the intention to provide feedback
However, privacy concerns may reduce feedback intention, and privacy concerns may be higher for those who don’t trust technology in general
Process data:
Due to the intention-behavior gap actual feedback may only be moderately correlated to feedback intentions
#13: So let’s review the hypotheses (laser-point):
Personalized recommendations should have a perceivably higher quality
This should in turn increase the user experience of the system and the outcome (choices)
A better experience in turn increases their intention to provide feedback
However...
#14: Tip: use two conditions to control the causal relations and to single out the effect
Also: log behavioral data and triangulate this with the constructs
#17: Content and system are in German
To explain the rating feature and its effect on recommendations
Opening recommendations before rating any items showed a similar explanation
Pps were allowed to close this pop-up without rating
After rating, participants were transported to the recommendations
#18: (the length of the vector depends on the impact the tags have)
(in terms of cosine similarity)
#19: Allowing ample opportunity to let their feedback behavior be influenced by their user experience
Unless they ignored the rating-probe
The median number of ratings per user was 15
#20: Tip for UX researchers: you cannot measure UX concepts with a single question. Measurement is far more robust if you construct a scale based on several questions
Exploratory Factor Analysis validates the intended conceptual structure
Finally, test the model with path analysis (mediation on steroids)
#23: The model has a good fit, with a non-significant χ2 of 13.210 (df = 13, p = .4317), a CFI of .996 and an RMSEA between 0 and 0.153 (90% confidence interval)
#30: We’ve been developing a framework for this type of research, and validated it in several field trials -->
#31: E.g. Advertisement (MS): Less clips clicked (fewer ads started) but maybe a higher retention (more ads full watched)?
Watch out for our future papers!
#32: Advantages of fitting a model: steps in between reduce variability!