際際滷

際際滷Share a Scribd company logo

The importance of quality in item generation; a prerequisite for Rasch analysis

Download as PPTX, PDF
Stephen McKenna
Stephen McKenna

The quality of a patient reported outcome scale depends on: a coherent and valid measurement model, quality item generation, and a simple response format. After these criteria are met, we can think about fit to the Rasch model.

1 of 26
Downloaded 18 times
Stephen McKenna & James Twiss Galen Research Ltd, Manchester, UK Email: smckenna@galen-research.com Quality in item generation; a prerequisite for Rasch analysis
Creating a scale Lets create a new scale .. We need to write a lot of items and then apply Rasch analysis
Introduction But what will it really measure? How relevant will the items be to respondents? The value of Rasch analysis is in perfecting the scale- not in generating its content
Random item generation Impact of MS Impact of MS QoL QoL Func Satis Symp
Targeted item generation based on model of construct Impact of MS QoL
Impairment Loss or abnormality of psychological, physiological or anatomical structure or function Equates to symptoms Disturbances at level of organ Fatigue, pain, dizziness, depression, sleep problems
Disability (activity limitation) Any restriction or lack of ability to perform an activity in the manner or within the range considered normal for a human being Equates to functioning or functional status E.g. restricted mobility, problems dressing & bathing, social restrictions, problems showing affection HRQL measures (such as SF-36) commonly assess functioning in addition to impairment
What is quality of life? Not symptoms, activity limitations, utility or clinical outcome Uniquely patient-based Holistic overall impact of disease and its treatment on the patient Unidimensional
Human life quality is dependent upon the satisfaction of certain basic needs - lack of disease, mobility, adequate nutrition and shelter. Sir Thomas More (1478-1535)
Item generation Appropriate source of items not existing measures not the literature not test-developers not clinicians in most cases relevant patients
Item generation Qualitative unstructured interviews focus groups do not work well do not start with a list of issues only constructs let the interviewee guide the interview as much as possible audio-record and produce transcripts invest a great deal of time analysing transcripts
Item reduction Ensure items assess the required construct. Remove items that: are poorly worded express more than one idea appear difficult to translate are personal to one interviewee will not apply to all respondents; sex, employment etc are likely to produce DIF; interest in shopping, attending gyms etc are specific to one national culture are facts or will not change: I worry that my health will get worse
PRIMUS Patient Reported Indices of Multiple Sclerosis (PRIMUS) consists of 3 scales: Symptoms (impairment) Activity limitations Quality of life
Sample items from the PRIMUS QoL scale I feel I am a burden to others I have little freedom to do what I want to do I have to push myself to do things I avoid physical intimacy I feel as if I have nothing to offer anyone
Rogues gallery SF-36 Does your health now limit you in vigorous activities, such as running, lifting heavy objects, participating in strenuous sports? that rules out 95% of the British! Disliked by older respondents General Health - How TRUE or FALSE are these statements for you? I seem to get sick a little easier than other people I am as healthy as anybody I know I expect my health to get worse My health is excellent Are these outcomes? MCS and PCS
Rogues gallery EQ-5D 5 items adequate coverage? Mobility I have no problems in walking about I have some problems in walking about I am confined to bed There seems to be a gap here! Anxiety/depression I am not anxious or depressed I am moderately anxious or depressed I am extremely anxious or depressed Are anxiety and depression the same constructs? What if I am anxious and depressed? What if I am moderately anxious and severely depressed?
Rogues gallery - WHOQOL Do you get the kind of support from others that you need? Are you able to accept your bodily appearance? How available to you is the information that you need? How satisfied are you with your access to health services? How satisfied are you with your transport? How safe do you feel in your daily life? Not health outcome as intended but static measure of QoL? The few relevant items are also poor: How satisfied are you with your sex life? How often do you have negative feelings, such as blue mood, despair, anxiety, depression?
Rogues gallery - DLQI Over the last week, how much has your skin interfered with you going shopping or looking after your home or garden? Over the last week, how much has your skin created problems with your partner or any of your close friends or relatives? 8/10 items can be answered not relevant (and scored no problem) Measure used to decide who should receive biologic treatment for psoriasis
DLQI (not at all, a little, a lot, very much)
DLQI: DIF by gender Influences the clothes you can wear Expectedvalue Person locations (logits) Females Males
Hypomanic Personality Scale: DIF by gender I find it easy to get people sexually interested in me
FIS: DIF by age I have to rely more on others to help me or do things for me
CES-D: DIF by country I was happy
DLQI: DIF by disease Prevents working or studying Person locations (logits) Expectedvalue Psoriasis Atopic Dermatitis
Conclusions Quality of patient reported outcome scale depends on: Coherent and valid measurement model Quality item generation Simple response format Then we can think about fit to Rasch model
Great - now we can make an excellent scale with the help of Rasch!

More Related Content

The importance of quality in item generation; a prerequisite for Rasch analysis

  • 1. Stephen McKenna & James Twiss Galen Research Ltd, Manchester, UK Email: smckenna@galen-research.com Quality in item generation; a prerequisite for Rasch analysis
  • 2. Creating a scale Lets create a new scale .. We need to write a lot of items and then apply Rasch analysis
  • 3. Introduction But what will it really measure? How relevant will the items be to respondents? The value of Rasch analysis is in perfecting the scale- not in generating its content
  • 4. Random item generation Impact of MS Impact of MS QoL QoL Func Satis Symp
  • 5. Targeted item generation based on model of construct Impact of MS QoL
  • 6. Impairment Loss or abnormality of psychological, physiological or anatomical structure or function Equates to symptoms Disturbances at level of organ Fatigue, pain, dizziness, depression, sleep problems
  • 7. Disability (activity limitation) Any restriction or lack of ability to perform an activity in the manner or within the range considered normal for a human being Equates to functioning or functional status E.g. restricted mobility, problems dressing & bathing, social restrictions, problems showing affection HRQL measures (such as SF-36) commonly assess functioning in addition to impairment
  • 8. What is quality of life? Not symptoms, activity limitations, utility or clinical outcome Uniquely patient-based Holistic overall impact of disease and its treatment on the patient Unidimensional
  • 9. Human life quality is dependent upon the satisfaction of certain basic needs - lack of disease, mobility, adequate nutrition and shelter. Sir Thomas More (1478-1535)
  • 10. Item generation Appropriate source of items not existing measures not the literature not test-developers not clinicians in most cases relevant patients
  • 11. Item generation Qualitative unstructured interviews focus groups do not work well do not start with a list of issues only constructs let the interviewee guide the interview as much as possible audio-record and produce transcripts invest a great deal of time analysing transcripts
  • 12. Item reduction Ensure items assess the required construct. Remove items that: are poorly worded express more than one idea appear difficult to translate are personal to one interviewee will not apply to all respondents; sex, employment etc are likely to produce DIF; interest in shopping, attending gyms etc are specific to one national culture are facts or will not change: I worry that my health will get worse
  • 13. PRIMUS Patient Reported Indices of Multiple Sclerosis (PRIMUS) consists of 3 scales: Symptoms (impairment) Activity limitations Quality of life
  • 14. Sample items from the PRIMUS QoL scale I feel I am a burden to others I have little freedom to do what I want to do I have to push myself to do things I avoid physical intimacy I feel as if I have nothing to offer anyone
  • 15. Rogues gallery SF-36 Does your health now limit you in vigorous activities, such as running, lifting heavy objects, participating in strenuous sports? that rules out 95% of the British! Disliked by older respondents General Health - How TRUE or FALSE are these statements for you? I seem to get sick a little easier than other people I am as healthy as anybody I know I expect my health to get worse My health is excellent Are these outcomes? MCS and PCS
  • 16. Rogues gallery EQ-5D 5 items adequate coverage? Mobility I have no problems in walking about I have some problems in walking about I am confined to bed There seems to be a gap here! Anxiety/depression I am not anxious or depressed I am moderately anxious or depressed I am extremely anxious or depressed Are anxiety and depression the same constructs? What if I am anxious and depressed? What if I am moderately anxious and severely depressed?
  • 17. Rogues gallery - WHOQOL Do you get the kind of support from others that you need? Are you able to accept your bodily appearance? How available to you is the information that you need? How satisfied are you with your access to health services? How satisfied are you with your transport? How safe do you feel in your daily life? Not health outcome as intended but static measure of QoL? The few relevant items are also poor: How satisfied are you with your sex life? How often do you have negative feelings, such as blue mood, despair, anxiety, depression?
  • 18. Rogues gallery - DLQI Over the last week, how much has your skin interfered with you going shopping or looking after your home or garden? Over the last week, how much has your skin created problems with your partner or any of your close friends or relatives? 8/10 items can be answered not relevant (and scored no problem) Measure used to decide who should receive biologic treatment for psoriasis
  • 19. DLQI (not at all, a little, a lot, very much)
  • 20. DLQI: DIF by gender Influences the clothes you can wear Expectedvalue Person locations (logits) Females Males
  • 21. Hypomanic Personality Scale: DIF by gender I find it easy to get people sexually interested in me
  • 22. FIS: DIF by age I have to rely more on others to help me or do things for me
  • 23. CES-D: DIF by country I was happy
  • 24. DLQI: DIF by disease Prevents working or studying Person locations (logits) Expectedvalue Psoriasis Atopic Dermatitis
  • 25. Conclusions Quality of patient reported outcome scale depends on: Coherent and valid measurement model Quality item generation Simple response format Then we can think about fit to Rasch model
  • 26. Great - now we can make an excellent scale with the help of Rasch!