際際滷

際際滷Share a Scribd company logo
Physical and Online Card Sorts:
A Practical Overview and Case Study
Bob Thomas
Katelyn Thompson
May 28, 2008
Outline
 Scope, Goals, and Objectives
 Current Navigational Hierarchy
 Methodology
 Card Sorts
 Analysis of Results
 Recommendations
 Next Steps
Scope, Objectives, and Goals
 Mission
    Our mission is to improve teaching and learning by providing
     customized assessment products and educational services.
     (http://measuredprogress.org/aboutus/index.html)

 Scope
    This project is focused only on Insite, the companys intranet web
     site for employees.

 Objectives
    To make it easier for Measured Progress employees to find
     information on the intranet, i.e., to lead with the need.

 Goals
    Redesign Insite, focusing on the navigation and organization
    Make the site user friendly
    Make the site attractive to employees
Home                                           HR Policies and Procedures




The goal is to unite the separate Insite and HR Policies and Procedures sites, each
created with different software applications.
Participants commented that that they would like to use Search and Site Map
features. These were linked only from the home page.
InService

                                                                   InForm




Youd think the [Accident and Injury Report] form is under InForm.
InService > HR > Policy Guide > General Administration > Operational Policies > Safety > Accident Reporting and Investigation
Plan > [Text Scroll Down Page]  Accident Report Form (7 clicks)

 A person could bleed to death before the form is even located!
Current Navigational Hierarchy
                   Insite is broad and shallow
                   HR Policies & Procedures is narrow and deep




Insite Job Request Form:                         Policies and Procedures Safety Form:
InForm > Job Request Forms > Form (2 clicks)     InService > HR > Policy Guide > General Administration >
                                                 Operational Policies > Safety > Accident Reporting and
                                                 Investigation Plan > [Text]  Accident Report Form (7 clicks)
Card Sorts: What
 Participants sort a series of cards, each labeled
  with a piece of content or functionality, into
  groups that make sense to them
  - Maurer & Warfel (2004), Card sorting: a definitive guide

 Closed card sorts
    Participants sort cards into predefined bins

 Open card sorts
    Participants sort cards and define bins
Card Sorts: Why
 Card sorting is excellent for situations where
  you want the users mental model to drive the
  information architecture of the product.
  - Courage & Baxter (2005), Understanding Your Users


 Open sorts are used for discovery. Closed sorts
  are used for validation.
  - Rosenfeld & Morville (2006), Information Architecture for the
  World Wide Web
Card Sorts: When
 Beginning of site creation
 Beginning of a site redesign
 Middle of a site redesign/creation
Card Sorts: Who
 Regarding users
   Recruit users familiar with the lingo
   But be cautious in a redesign with users who are
    familiar with the website or application
Card Sorts: Where
 Lab
   Moderated
   Physical or online

 Remote
   Moderated or un-moderated
   Restricted to online
Card Sorts: How
 Physical
   Participants may be apt to put more
    thought into sorting

 Online
   Cheaper
   Higher number of participants
   Eliminates data entry
Card Sorts: How
 Tools
   EZCalc
   MindCanvas
   OptimalSort
   SynCaps
   WebSort
Tools: Mind Canvas
Card Sorts: How
 Analysis
   More information to sift through with open sorts
   Dendrograms to see common groupings
Methodology
 Create and run an open card sort with 15 MP
  employees, including follow-up interviews
 Analyze quantitative and qualitative data
 Make recommendations for a new navigational
  hierarchy and structure
Open Card Sort
                 #    Card name
                  1   Corporate Graphics
                  2   Floor Plans
                  3   Facilities and Locations
                  4   Contacts
                  5   Employee Directory
                  6   Corporate Travel
                  7   Glossary and Acronyms
                  8   Mission and Philosophy
                  9   Safety
                 10   Reimbursement
                 11   Payroll
                 12   Purchasing and Requisitions
                 13   Workplace Policies
                 14   Hiring and Employment
                 15   Benefits
                 16   Work Schedules
                 17   Personnel Records
                 18   Shipping and Distribution
                 19   Printing and Copying
                 20   Helpdesk
                 21   Forms
                 22   Computer and Networks
                 23   Corporate Marketing and Branding
                 24   Information Technology
                 25   Phone and Conferencing
                 26   Contracts
                 27   Wellness Programs
                 28   Diversity
                 29   In/Out Board
                 30   News and Announcements
                 31   Photo Gallery
                 32   Corporate Events
                 33   Department Sites
                 34   For New Employees
                 35   Workplace Conduct
                 36   Recognition and Accomplishments
Participant Demographics
                  All 15 participants had experience using Insite.
                  We had participants from 9 different departments.
                  We had 4 men and 11 women.
P#    Gender Department             Used Insite?   Hours Per Week   Web Browser for Insite?   What Do You Do on a Computer?
                                                    0-5      6-10        IE        Firefox Word processing Emailing Web Browsing
  1     F    Testing Services            Y           x                    x                       x          x           x
  2     F    Human Resources             Y           x                    x                       x                      x
  3     M    Client Services             Y           x                    x                       x          x           x
  4     F    Human Resources             Y                    x           x                       x          x           x
  5     F    Testing Services            Y           x                    x                       x          x           x
  6     F    Office of Technology        Y           x                                x           x          x           x
  7     F    Finance                     Y           x                    x                       x          x           x
  8     M    Testing Services            Y           x                    x                       x          x           x
  9     F    Client Services             Y           x                    x                       x          x           x
 10     F    Operational Services        Y           x                    x                       x          x           x
 11     M    Scoring                     Y           x                    x                       x          x           x
 12     F    Process Coordination        Y           x                    x                       x          x           x
 13     F    Marketing                   Y           x                    x                       x          x           x
 14     F    Client Services             Y           x                    x                       x          x           x
 15     M    Marketing                   Y           x                    x                       x          x           x
Sample Open Card Sort Results:
Major Navigation Categories
WHAT I WANT TO USE
 Home Page
 Contracts
 Forms
WHAT I WANT TO REFER TO
 Department Sites
 Employee Directory
 Working at Measured Progress
 Helpdesk
Analysis of Results

 We began to see trends after the
 first day of testing (5 participants)
 Table 2. Main categories among first five participants.
Analysis of Results
 After all 15 card sorts were completed,
  standardized the main categories.
 Gave categories with similar names or
  concepts a consistent name and combined
  groups where participants used the same
  basic concept but a slightly different label
  - Maurer (2007), Instructions for Use: Card Sort Analysis
  Spreadsheet
Analysis of Results
 We started with 106 original categories from all 15
  participants, and applied consistent naming
  conventions.
  Table 3. Standardized categories derived from original categories.
Analysis of Results
 Second, combined similar standardized categories
  and arrived at a total of 13 standardized categories.
  Table 4. Final list of 13 standardized categories.




                                                       0.67
Physical and Online Card Sorts: A Practical Overview and Case Study
Analysis of Results
Table 5. Mapping the dendrogram groupings to primary and
secondary navigation areas.
Analysis of Results
 Mental image the user forms to understand how
  software works and how to operate it
  - Arnowitz et al (2007), Effective Prototyping for Software Makers

    In the case of Insite, many participants in our card study
     could not express a mental model of the website beyond the
     use of In headings
    But many did express their mental model of the ideal
     Measured Progress intranet. One participant noted:
        Theres stuff there that people need to do their jobs.
         They need a reference library for working at the
         company a big bookshelf of benefits and other
         [information].
Analysis of Results
 Participants wanted easier access to information
  using primary, secondary, and third-level navigation

 Participants wanted a consistent navigational
  structure, i.e., one that didnt change.

 Participants wanted HR Policies and Procedures
  broken into two separate categories:
    HR Forms (such as Corporate Travel, Payroll, Requisitions)
    HR Policies (such as Diversity, Safety, Workplace Conduct)
Analysis of Results
 Use a navigation hierarchy of medium depth
  and breadth
   Use a primary navigation hierarchy of no more than 8 or 9
    categories
   Go 3 or 4 levels deep

 Put Search at the top of every page
 Put Site Map link on every page
Recommendations
Recommendations
Next Steps
   Develop wire frames of all eight primary navigational categories.
   Present the wire frames to the design team.
   Develop paper prototypes of the Insite Home page and one other
    subsection, such as HR.

   Test the usability of the paper prototypes with eight participants,
    using task-based scenarios. For example: You are going on a business
    trip for Measured Progress. Using the web site, how would you
    complete the pre-approval process for your trip?
   Develop electronic versions of the Insite Home page and one other
    subsection, such as HR.
   Test the usability of the electronic versions with eight participants,
    using task-based scenarios, as discussed above.

   Redesign Insite based on results from the usability test.
Thank You

 Questions?

More Related Content

Similar to Physical and Online Card Sorts: A Practical Overview and Case Study (20)

Scaling Training Data for AI Applications
Scaling Training Data for AI ApplicationsScaling Training Data for AI Applications
Scaling Training Data for AI Applications
Applause
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Innovation Enterprise
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Julia Grosman
Using alfresco share as a corporate intranet
Using alfresco share as a corporate intranetUsing alfresco share as a corporate intranet
Using alfresco share as a corporate intranet
Alfresco Software
ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF-related data: the new frontier of individualised, predictive healthcare ]...ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF Education
Acctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva: expertise in Business Intelligence, Data Warehousing, Data GovernanceAcctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva Ltd.
1.11 Data and Performance Simplified
1.11 Data and Performance Simplified1.11 Data and Performance Simplified
1.11 Data and Performance Simplified
National Alliance to End Homelessness
6.6 Family and Youth Program Measurement Simplified
6.6 Family and Youth Program Measurement Simplified6.6 Family and Youth Program Measurement Simplified
6.6 Family and Youth Program Measurement Simplified
National Alliance to End Homelessness
Service Design in Experience Design
Service Design in Experience DesignService Design in Experience Design
Service Design in Experience Design
Paul Kahn
Digital Workplace Case Studies (Intranet)
Digital Workplace Case Studies (Intranet)Digital Workplace Case Studies (Intranet)
Digital Workplace Case Studies (Intranet)
Prescient Digital Media
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and ExcellenceTesting in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
TechWell
2.6 Expert Forum: Data and Performance Simplified
2.6 Expert Forum: Data and Performance Simplified2.6 Expert Forum: Data and Performance Simplified
2.6 Expert Forum: Data and Performance Simplified
National Alliance to End Homelessness
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
Kaitlan Chu
Software Project Management Presentation Final
Software Project Management Presentation FinalSoftware Project Management Presentation Final
Software Project Management Presentation Final
Minhas Kamal
Dashlane Triple Track : la recherche de la bonne organisation - Agile en Se...
Dashlane Triple Track :  la recherche de la bonne organisation - Agile en Se...Dashlane Triple Track :  la recherche de la bonne organisation - Agile en Se...
Dashlane Triple Track : la recherche de la bonne organisation - Agile en Se...
Agile En Seine
Think future technologies corporate presentation (public)
Think future technologies  corporate presentation (public)Think future technologies  corporate presentation (public)
Think future technologies corporate presentation (public)
Tft Us
1.8 Data and Performance Simplified (De Jong)
1.8 Data and Performance Simplified (De Jong)1.8 Data and Performance Simplified (De Jong)
1.8 Data and Performance Simplified (De Jong)
National Alliance to End Homelessness
Logramme coverage_matrix__by dr malik khalid mehmood ph_d
Logramme  coverage_matrix__by dr malik khalid mehmood ph_dLogramme  coverage_matrix__by dr malik khalid mehmood ph_d
Logramme coverage_matrix__by dr malik khalid mehmood ph_d
Malik Khalid Mehmood
Intranet ROI: Measuring the Dollar Value of Intranets
Intranet ROI: Measuring the Dollar Value of Intranets Intranet ROI: Measuring the Dollar Value of Intranets
Intranet ROI: Measuring the Dollar Value of Intranets
Prescient Digital Media
Introduction to Neo4j
Introduction to Neo4jIntroduction to Neo4j
Introduction to Neo4j
Neo4j
Scaling Training Data for AI Applications
Scaling Training Data for AI ApplicationsScaling Training Data for AI Applications
Scaling Training Data for AI Applications
Applause
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Actionable Analytics - Solving Real World Problems With Big Data, Xerox Innov...
Innovation Enterprise
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Bob Selfridge - Identify, Collect, and Act Upon Customer Interactions; Rinse,...
Julia Grosman
Using alfresco share as a corporate intranet
Using alfresco share as a corporate intranetUsing alfresco share as a corporate intranet
Using alfresco share as a corporate intranet
Alfresco Software
ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF-related data: the new frontier of individualised, predictive healthcare ]...ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF-related data: the new frontier of individualised, predictive healthcare ]...
ICF Education
Acctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva: expertise in Business Intelligence, Data Warehousing, Data GovernanceAcctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva: expertise in Business Intelligence, Data Warehousing, Data Governance
Acctiva Ltd.
Service Design in Experience Design
Service Design in Experience DesignService Design in Experience Design
Service Design in Experience Design
Paul Kahn
Digital Workplace Case Studies (Intranet)
Digital Workplace Case Studies (Intranet)Digital Workplace Case Studies (Intranet)
Digital Workplace Case Studies (Intranet)
Prescient Digital Media
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and ExcellenceTesting in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
TechWell
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
New Methods to Analyze Large Open Card Sort: UX Study Designed to Maximize Co...
Kaitlan Chu
Software Project Management Presentation Final
Software Project Management Presentation FinalSoftware Project Management Presentation Final
Software Project Management Presentation Final
Minhas Kamal
Dashlane Triple Track : la recherche de la bonne organisation - Agile en Se...
Dashlane Triple Track :  la recherche de la bonne organisation - Agile en Se...Dashlane Triple Track :  la recherche de la bonne organisation - Agile en Se...
Dashlane Triple Track : la recherche de la bonne organisation - Agile en Se...
Agile En Seine
Think future technologies corporate presentation (public)
Think future technologies  corporate presentation (public)Think future technologies  corporate presentation (public)
Think future technologies corporate presentation (public)
Tft Us
Logramme coverage_matrix__by dr malik khalid mehmood ph_d
Logramme  coverage_matrix__by dr malik khalid mehmood ph_dLogramme  coverage_matrix__by dr malik khalid mehmood ph_d
Logramme coverage_matrix__by dr malik khalid mehmood ph_d
Malik Khalid Mehmood
Intranet ROI: Measuring the Dollar Value of Intranets
Intranet ROI: Measuring the Dollar Value of Intranets Intranet ROI: Measuring the Dollar Value of Intranets
Intranet ROI: Measuring the Dollar Value of Intranets
Prescient Digital Media
Introduction to Neo4j
Introduction to Neo4jIntroduction to Neo4j
Introduction to Neo4j
Neo4j

Recently uploaded (20)

Not a Kubernetes fan? The state of PaaS in 2025
Not a Kubernetes fan? The state of PaaS in 2025Not a Kubernetes fan? The state of PaaS in 2025
Not a Kubernetes fan? The state of PaaS in 2025
Anthony Dahanne
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
The Constructor's Digital Transformation Playbook: Reducing Risk With Technology
The Constructor's Digital Transformation Playbook: Reducing Risk With TechnologyThe Constructor's Digital Transformation Playbook: Reducing Risk With Technology
The Constructor's Digital Transformation Playbook: Reducing Risk With Technology
Aggregage
DealBook of Ukraine: 2025 edition | AVentures Capital
DealBook of Ukraine: 2025 edition | AVentures CapitalDealBook of Ukraine: 2025 edition | AVentures Capital
DealBook of Ukraine: 2025 edition | AVentures Capital
Yevgen Sysoyev
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Earley Information Science
UiPath Automation Developer Associate Training Series 2025 - Session 1
UiPath Automation Developer Associate Training Series 2025 - Session 1UiPath Automation Developer Associate Training Series 2025 - Session 1
UiPath Automation Developer Associate Training Series 2025 - Session 1
DianaGray10
Benchmark Testing Demystified: Your Roadmap to Peak Performance
Benchmark Testing Demystified: Your Roadmap to Peak PerformanceBenchmark Testing Demystified: Your Roadmap to Peak Performance
Benchmark Testing Demystified: Your Roadmap to Peak Performance
Shubham Joshi
Supercharge Your Career with UiPath Certifications
Supercharge Your Career with UiPath CertificationsSupercharge Your Career with UiPath Certifications
Supercharge Your Career with UiPath Certifications
DianaGray10
Build with AI on Google Cloud Session #3
Build with AI on Google Cloud Session #3Build with AI on Google Cloud Session #3
Build with AI on Google Cloud Session #3
Margaret Maynard-Reid
Caching for Performance Masterclass: Caching Strategies
Caching for Performance Masterclass: Caching StrategiesCaching for Performance Masterclass: Caching Strategies
Caching for Performance Masterclass: Caching Strategies
ScyllaDB
What is FinTech A Complete Guide to Financial Technology.pdf
What is FinTech A Complete Guide to Financial Technology.pdfWhat is FinTech A Complete Guide to Financial Technology.pdf
What is FinTech A Complete Guide to Financial Technology.pdf
Yodaplus Technologies Private Limited
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIATHE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
Srivaanchi Nathan
Bedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Bedrock Data Automation (Preview): Simplifying Unstructured Data ProcessingBedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Bedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Zilliz
10 FinTech Solutions Every Business Should Know!.pdf
10 FinTech Solutions Every Business Should Know!.pdf10 FinTech Solutions Every Business Should Know!.pdf
10 FinTech Solutions Every Business Should Know!.pdf
Yodaplus Technologies Private Limited
5 Must-Use AI Tools to Supercharge Your Productivity
5 Must-Use AI Tools to Supercharge Your Productivity5 Must-Use AI Tools to Supercharge Your Productivity
5 Must-Use AI Tools to Supercharge Your Productivity
cryptouniversityoffi
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Chris Wahl
Caching for Performance Masterclass: The In-Memory Datastore
Caching for Performance Masterclass: The In-Memory DatastoreCaching for Performance Masterclass: The In-Memory Datastore
Caching for Performance Masterclass: The In-Memory Datastore
ScyllaDB
Transcript: AI in publishing: Your questions answered - Tech Forum 2025
Transcript: AI in publishing: Your questions answered - Tech Forum 2025Transcript: AI in publishing: Your questions answered - Tech Forum 2025
Transcript: AI in publishing: Your questions answered - Tech Forum 2025
BookNet Canada
Leadership u automatizaciji: RPA prie iz prakse!
Leadership u automatizaciji: RPA prie iz prakse!Leadership u automatizaciji: RPA prie iz prakse!
Leadership u automatizaciji: RPA prie iz prakse!
UiPathCommunity
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use CasesMastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Sanjay Willie
Not a Kubernetes fan? The state of PaaS in 2025
Not a Kubernetes fan? The state of PaaS in 2025Not a Kubernetes fan? The state of PaaS in 2025
Not a Kubernetes fan? The state of PaaS in 2025
Anthony Dahanne
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
The Constructor's Digital Transformation Playbook: Reducing Risk With Technology
The Constructor's Digital Transformation Playbook: Reducing Risk With TechnologyThe Constructor's Digital Transformation Playbook: Reducing Risk With Technology
The Constructor's Digital Transformation Playbook: Reducing Risk With Technology
Aggregage
DealBook of Ukraine: 2025 edition | AVentures Capital
DealBook of Ukraine: 2025 edition | AVentures CapitalDealBook of Ukraine: 2025 edition | AVentures Capital
DealBook of Ukraine: 2025 edition | AVentures Capital
Yevgen Sysoyev
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Revolutionizing Field Service: How LLMs Are Powering Smarter Knowledge Access...
Earley Information Science
UiPath Automation Developer Associate Training Series 2025 - Session 1
UiPath Automation Developer Associate Training Series 2025 - Session 1UiPath Automation Developer Associate Training Series 2025 - Session 1
UiPath Automation Developer Associate Training Series 2025 - Session 1
DianaGray10
Benchmark Testing Demystified: Your Roadmap to Peak Performance
Benchmark Testing Demystified: Your Roadmap to Peak PerformanceBenchmark Testing Demystified: Your Roadmap to Peak Performance
Benchmark Testing Demystified: Your Roadmap to Peak Performance
Shubham Joshi
Supercharge Your Career with UiPath Certifications
Supercharge Your Career with UiPath CertificationsSupercharge Your Career with UiPath Certifications
Supercharge Your Career with UiPath Certifications
DianaGray10
Build with AI on Google Cloud Session #3
Build with AI on Google Cloud Session #3Build with AI on Google Cloud Session #3
Build with AI on Google Cloud Session #3
Margaret Maynard-Reid
Caching for Performance Masterclass: Caching Strategies
Caching for Performance Masterclass: Caching StrategiesCaching for Performance Masterclass: Caching Strategies
Caching for Performance Masterclass: Caching Strategies
ScyllaDB
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIATHE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
THE BIG TEN BIOPHARMACEUTICAL MNCs: GLOBAL CAPABILITY CENTERS IN INDIA
Srivaanchi Nathan
Bedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Bedrock Data Automation (Preview): Simplifying Unstructured Data ProcessingBedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Bedrock Data Automation (Preview): Simplifying Unstructured Data Processing
Zilliz
5 Must-Use AI Tools to Supercharge Your Productivity
5 Must-Use AI Tools to Supercharge Your Productivity5 Must-Use AI Tools to Supercharge Your Productivity
5 Must-Use AI Tools to Supercharge Your Productivity
cryptouniversityoffi
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Getting Started with AWS - Enterprise Landing Zone for Terraform Learning & D...
Chris Wahl
Caching for Performance Masterclass: The In-Memory Datastore
Caching for Performance Masterclass: The In-Memory DatastoreCaching for Performance Masterclass: The In-Memory Datastore
Caching for Performance Masterclass: The In-Memory Datastore
ScyllaDB
Transcript: AI in publishing: Your questions answered - Tech Forum 2025
Transcript: AI in publishing: Your questions answered - Tech Forum 2025Transcript: AI in publishing: Your questions answered - Tech Forum 2025
Transcript: AI in publishing: Your questions answered - Tech Forum 2025
BookNet Canada
Leadership u automatizaciji: RPA prie iz prakse!
Leadership u automatizaciji: RPA prie iz prakse!Leadership u automatizaciji: RPA prie iz prakse!
Leadership u automatizaciji: RPA prie iz prakse!
UiPathCommunity
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use CasesMastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Mastering ChatGPT & LLMs for Practical Applications: Tips, Tricks, and Use Cases
Sanjay Willie

Physical and Online Card Sorts: A Practical Overview and Case Study

  • 1. Physical and Online Card Sorts: A Practical Overview and Case Study Bob Thomas Katelyn Thompson May 28, 2008
  • 2. Outline Scope, Goals, and Objectives Current Navigational Hierarchy Methodology Card Sorts Analysis of Results Recommendations Next Steps
  • 3. Scope, Objectives, and Goals Mission Our mission is to improve teaching and learning by providing customized assessment products and educational services. (http://measuredprogress.org/aboutus/index.html) Scope This project is focused only on Insite, the companys intranet web site for employees. Objectives To make it easier for Measured Progress employees to find information on the intranet, i.e., to lead with the need. Goals Redesign Insite, focusing on the navigation and organization Make the site user friendly Make the site attractive to employees
  • 4. Home HR Policies and Procedures The goal is to unite the separate Insite and HR Policies and Procedures sites, each created with different software applications. Participants commented that that they would like to use Search and Site Map features. These were linked only from the home page.
  • 5. InService InForm Youd think the [Accident and Injury Report] form is under InForm. InService > HR > Policy Guide > General Administration > Operational Policies > Safety > Accident Reporting and Investigation Plan > [Text Scroll Down Page] Accident Report Form (7 clicks) A person could bleed to death before the form is even located!
  • 6. Current Navigational Hierarchy Insite is broad and shallow HR Policies & Procedures is narrow and deep Insite Job Request Form: Policies and Procedures Safety Form: InForm > Job Request Forms > Form (2 clicks) InService > HR > Policy Guide > General Administration > Operational Policies > Safety > Accident Reporting and Investigation Plan > [Text] Accident Report Form (7 clicks)
  • 7. Card Sorts: What Participants sort a series of cards, each labeled with a piece of content or functionality, into groups that make sense to them - Maurer & Warfel (2004), Card sorting: a definitive guide Closed card sorts Participants sort cards into predefined bins Open card sorts Participants sort cards and define bins
  • 8. Card Sorts: Why Card sorting is excellent for situations where you want the users mental model to drive the information architecture of the product. - Courage & Baxter (2005), Understanding Your Users Open sorts are used for discovery. Closed sorts are used for validation. - Rosenfeld & Morville (2006), Information Architecture for the World Wide Web
  • 9. Card Sorts: When Beginning of site creation Beginning of a site redesign Middle of a site redesign/creation
  • 10. Card Sorts: Who Regarding users Recruit users familiar with the lingo But be cautious in a redesign with users who are familiar with the website or application
  • 11. Card Sorts: Where Lab Moderated Physical or online Remote Moderated or un-moderated Restricted to online
  • 12. Card Sorts: How Physical Participants may be apt to put more thought into sorting Online Cheaper Higher number of participants Eliminates data entry
  • 13. Card Sorts: How Tools EZCalc MindCanvas OptimalSort SynCaps WebSort
  • 15. Card Sorts: How Analysis More information to sift through with open sorts Dendrograms to see common groupings
  • 16. Methodology Create and run an open card sort with 15 MP employees, including follow-up interviews Analyze quantitative and qualitative data Make recommendations for a new navigational hierarchy and structure
  • 17. Open Card Sort # Card name 1 Corporate Graphics 2 Floor Plans 3 Facilities and Locations 4 Contacts 5 Employee Directory 6 Corporate Travel 7 Glossary and Acronyms 8 Mission and Philosophy 9 Safety 10 Reimbursement 11 Payroll 12 Purchasing and Requisitions 13 Workplace Policies 14 Hiring and Employment 15 Benefits 16 Work Schedules 17 Personnel Records 18 Shipping and Distribution 19 Printing and Copying 20 Helpdesk 21 Forms 22 Computer and Networks 23 Corporate Marketing and Branding 24 Information Technology 25 Phone and Conferencing 26 Contracts 27 Wellness Programs 28 Diversity 29 In/Out Board 30 News and Announcements 31 Photo Gallery 32 Corporate Events 33 Department Sites 34 For New Employees 35 Workplace Conduct 36 Recognition and Accomplishments
  • 18. Participant Demographics All 15 participants had experience using Insite. We had participants from 9 different departments. We had 4 men and 11 women. P# Gender Department Used Insite? Hours Per Week Web Browser for Insite? What Do You Do on a Computer? 0-5 6-10 IE Firefox Word processing Emailing Web Browsing 1 F Testing Services Y x x x x x 2 F Human Resources Y x x x x 3 M Client Services Y x x x x x 4 F Human Resources Y x x x x x 5 F Testing Services Y x x x x x 6 F Office of Technology Y x x x x x 7 F Finance Y x x x x x 8 M Testing Services Y x x x x x 9 F Client Services Y x x x x x 10 F Operational Services Y x x x x x 11 M Scoring Y x x x x x 12 F Process Coordination Y x x x x x 13 F Marketing Y x x x x x 14 F Client Services Y x x x x x 15 M Marketing Y x x x x x
  • 19. Sample Open Card Sort Results: Major Navigation Categories WHAT I WANT TO USE Home Page Contracts Forms WHAT I WANT TO REFER TO Department Sites Employee Directory Working at Measured Progress Helpdesk
  • 20. Analysis of Results We began to see trends after the first day of testing (5 participants) Table 2. Main categories among first five participants.
  • 21. Analysis of Results After all 15 card sorts were completed, standardized the main categories. Gave categories with similar names or concepts a consistent name and combined groups where participants used the same basic concept but a slightly different label - Maurer (2007), Instructions for Use: Card Sort Analysis Spreadsheet
  • 22. Analysis of Results We started with 106 original categories from all 15 participants, and applied consistent naming conventions. Table 3. Standardized categories derived from original categories.
  • 23. Analysis of Results Second, combined similar standardized categories and arrived at a total of 13 standardized categories. Table 4. Final list of 13 standardized categories. 0.67
  • 25. Analysis of Results Table 5. Mapping the dendrogram groupings to primary and secondary navigation areas.
  • 26. Analysis of Results Mental image the user forms to understand how software works and how to operate it - Arnowitz et al (2007), Effective Prototyping for Software Makers In the case of Insite, many participants in our card study could not express a mental model of the website beyond the use of In headings But many did express their mental model of the ideal Measured Progress intranet. One participant noted: Theres stuff there that people need to do their jobs. They need a reference library for working at the company a big bookshelf of benefits and other [information].
  • 27. Analysis of Results Participants wanted easier access to information using primary, secondary, and third-level navigation Participants wanted a consistent navigational structure, i.e., one that didnt change. Participants wanted HR Policies and Procedures broken into two separate categories: HR Forms (such as Corporate Travel, Payroll, Requisitions) HR Policies (such as Diversity, Safety, Workplace Conduct)
  • 28. Analysis of Results Use a navigation hierarchy of medium depth and breadth Use a primary navigation hierarchy of no more than 8 or 9 categories Go 3 or 4 levels deep Put Search at the top of every page Put Site Map link on every page
  • 31. Next Steps Develop wire frames of all eight primary navigational categories. Present the wire frames to the design team. Develop paper prototypes of the Insite Home page and one other subsection, such as HR. Test the usability of the paper prototypes with eight participants, using task-based scenarios. For example: You are going on a business trip for Measured Progress. Using the web site, how would you complete the pre-approval process for your trip? Develop electronic versions of the Insite Home page and one other subsection, such as HR. Test the usability of the electronic versions with eight participants, using task-based scenarios, as discussed above. Redesign Insite based on results from the usability test.

Editor's Notes

  • #8: Talk about each one, show a demo of one or two