狠狠撸

Experimental Design
How to organise a successful research project
Research Methods Lecture
Bruce Lynn
Director, MSc School of human Health and Performance
Nov 07

Experimental Design
How to organise a successful research project
1. The initial idea
2. Reading the literature
3. Planning
4. Doing
5. Writing

Generating the initial idea
? Something you always wanted to investigate
? Look at past projects and shamelessly borrow
ideas
? Read around a subject of interest looking for
loose ends
? Talk to one of the labs associated with the MSc
Programmes

Reading the literature
Essential
Why we require a literature review before the real work
commences
Make sure project has originality
Find out about the methods used in the field of interest

Planning
The key to a successful research project
? Start early
? Decide what measures to use
? Who are the test population?
? What is being compared to what?
? Statistical issues

What measure(s) to use?
? Questionnaires.
– Find suitable validated instruments.
– Try developing your own?
? Physiological measures
– Explore availability of equipment
– Investigate suitable analytical software
? Example
– Investigating heart rate variability.
? Find suitable heart rate monitoring equipment, e.g. Polar
? Find software package to analyse beat by beat data
? Decide which HRV parameter(s) are appropriate

Who are the test population?
? Biological investigations, e.g. animal studies, or studies of
tissue properties, or cadaver anatomy studies
? Healthy volunteer studies
? Patient studies
? All are regulated re ethics. In particular patient studies need
Central (NHS) ethical clearance (NRES)
? Recruitment for human studies
– Posters
– Letters or e-mails
– Will subjects be paid – if so who funds this?
– Subject information sheets
– Will enough subjects be available within the time period
available for the project?

What is being compared with what?
Experimental designs (1)
Example
A double blind random controlled trial (RCT)
Prospective – subjects are checked out in advance of intervention
Controlled. 2 (or more) groups run in parallel. One receiving an
ineffective treatment that will appear equivalent to the real
treatment, thus controlling for any placebo effect. This is often
the hardest part of any design.
Randomisation. Must not select subjects. More info in a minute.
Subject blinds. Can be impossible. E.g. testing health education
literature.
Observer blinds. Nearly always possible. Need an assistant to
organise this. May not be necessary if outcome measure is
very objective, e.g. blood count.
Dropout rates. Must be monitored. Differential dropout from
control or treatment group can introduce bias.

What is being compared with what?
Experimental designs
Other designs
Crossover trials. Control group subsequently receives treatment
and vice versa. Each subject own control – good feature, but
carry-over of effects is a problem.
Non-experimental designs – surveys
Longitudinal. Following same subjects for period of time.
Cross-sectional. Looking at sample of population at one time
point.
Retrospective. Looking back at health records of patients.

More on randomisation
So pure random sequences can cause problems. Need to remove long runs. Also
need all numbers to appear within any given sequence the same number of times
– for this use random permutations.
Often useful to ensure same number of treatment (T) and control (non-treatment, NT)
within blocks of subjects. So every 10, make sure you have equal T/NT even in a
design for 30. Means if you have to stop short, still have roughly equal T and NT
Often need to ensure balance by randomising separately for male/female, old/young.
severe/less severe etc as appropriate. Make sure you have extra random
allocations in sub-groups if not sure of numbers, e.g. for M/F
Some random numbers from
a published table
03 47 43 73 86 36 96 47 36 61
97 74 24 67 62 42 81 14 57 20
16 76 62 27 66 56 50 26 71 07
12 56 85 99 26 96 96 68 27 31
55 59 56 35 64 38 54 82 46 22
If you were using odd for
treatment and even for control,
then note long runs of up to 8
all the same. When n is small,
these occasional long runs can
upset the balance of the study

Statistical issues – how many subjects
Key statistical issue in the planning phase is whether enough subjects
can be studied to get a clear result.
Involves the question of statistical power.
Need to know 4 things
? The criterion for rejecting the null hypothesis, normally 5%
? The criterion for missing a possible result, normally 20% (NB
100-?, 80% is the “Power” of the study)
? The standard deviation of the outcome measure. Or if using
counted data or another non-continuous measure, then the
appropriate error measure
d The extent of change to be detected.
For a comparison of 2 groups with same n, formulae for required n is
n = 16?2
/d2

Statistical issues – how many subjects
Example (From Statistics at Square One, http://www.bmj.com/collections/statsbk/13.dtl)
BP Trial, comparing 2 groups. Want to find 5mm Hg change if it occurs; standard
deviation in population is 10mm Hg.
n = 16*10*10/5*5 = 64
If we decide only a 10mm Hg change is of significance to us, then
n = 16*10*10/10*10 = 16
So sample size needed depends very much on criterion we set.
Obviously, the smaller the standard deviation in the measure/population, the
smaller n will be. How do we find out s.d. in advance of the study!! Can find
literature values. Or do a small pilot study.
Also, if we set a tougher criterion for rejecting the null hypothesis, e.g. 1%, then n
rises.
A more detailed discussion of this issue, and the formulae for other types of
design and measure, is in Kirkwood, Essentials of Medical Statistics, Chapt 26

Sources of error
1. Systematic – leading to bias
2. Random
Bias is the bigger problem. Can have apparently clear result that is
wrong. Bias may be caused by inadequate randomisation, or not
ensuring good match between T and NT samples, or inadequate
technique (e.g. not maintaining calibration of equipment during study)
Random error will not lead to false positives – but will of course lead
to more false negatives. Important to spend time optimising methods.
Also to consider using only a relatively homogeneous population, e.g.
restrict to young males.

Doing the study
The fun bit!
? Usually need to run initial trials to learn methods, establish
reliability etc
? Then plan the main data collection. Try to keep things
standardised
? If unexpected observations occur, then DO FOLLOW
THEM UP
? Once the main trend is established, if you have time run
more subjects comparing with a different/tougher control
condition
? Back up data. Keep back ups in a separate place from the
main data storage. Hard disc crashes do occur!!

Sorting out and analysing the data
? Excel is useful.
? Be sure to graph data before you analyse.
? Visualising the data usually indicates the real trends –
your eyes are good at statistics!
? Look at average trends; normalise data where
appropriate.
? For most regression, anova or non-parametric tests
you will need a statistical package. SPSS is on the
cluster machines but is rather user unfriendly, so
expect to spend some time finding your way around.

Resources
Statistics at square one, last chapter. Good introduction to
statistical issues. http://www.bmj.com/collections/statsbk/
Kirkwood, B.R. Essentials of Medical Statistics, Chapts 21-26
Bland, M. An Introduction to Medical Statistics, [Chapt 15,
Reliability measures; Chapt 18, smaple size]
Field, A. Discovering Statistics using SPSS [Detailed on
regression; good explanation of SPSS computer package]
Mackenzie, A. Mathematics and Statistics for Life Scientists, Bios
Instant Notes, 2005 [mostly maths]
Random permutations:
http://calculators.stat.ucla.edu/perm.php
http://www.webcalculator.co.uk/statistics/rpermute3.htm
NRES (National Research Ethics Service)
[http://www.nres.npsa.nhs.uk/ ]

狠狠撸

Experimental_Design FGFG FGFGFGFF BFHFF.ppt

More Related Content

Similar to Experimental_Design FGFG FGFGFGFF BFHFF.ppt (20)

Recently uploaded (20)

Experimental_Design FGFG FGFGFGFF BFHFF.ppt