Sections of the protocol
Title
The title should express the question(s) that the study is designed to answer,
and briefly outline the essential method. For example, if the trial is to be
randomised, then this word might appear somewhere in the title or sub-title. The
title chosen now may be used for the final report.
Preliminary comments
The research team is named, and the roles of the investigators, clinicians,
research nurses, evaluators, and statistical advisers are set out. Usually, this
is the opportunity to establish authorship of the final report, not only whose
names are included in the list of authors, but also in which order.
Grant-awarding bodies will need to know that the individuals responsible for
different roles have the necessary skills and resources, and may ask for
Curricula Vitae. This is also the place to define precisely which organisation
or individual has overall responsibility for the research, and to describe the
setting for various parts of the project. Any sponsor should also be disclosed.
Introduction or background
The aim of this section is to leave readers in no doubt about the need for this
research. By the end of the introduction they should be able to conclude that
the researcher has done a thorough literature search, has read all relevant
papers about previous research, and is fully briefed about both the condition
and the proposed treatment to be investigated. This section then concludes with
a clear statement of the study objectives.
The Introduction should be as succinct as possible. Its usual
structure consists, first, of introducing the medical condition, commenting on
its frequency and its clinical or economic importance, and describing the
current management, perhaps pointing out where conventional treatment is
inadequate. Second, it should explain the possible role of acupuncture in
treating the condition, or perhaps similar conditions; in designing a controlled
trial, there needs to be a reasonable expectation that acupuncture will help
patients, but not enough evidence to conclude that acupuncture is definitely
effective for the condition. This is usually expressed as there being sufficient
equipoise in clinical opinion.
Indication that a complete literature search has been done is
necessary in the introduction, and essential references should be quoted to
support each step of the argument. In Denmark, there is an even more demanding
condition for ethical approval: the investigator should have first carried out a
full systematic review on the topic! (2).
Finally, the text of the Introduction should be appropriate to
act as the introduction to the final report or paper. There is a tendency for
authors to use the introduction to display their extensive knowledge about the
subject, but this is largely counterproductive. Readers need to learn the
essential arguments, logically and clearly set out, without extraneous and
baseless supposition.
Aim or research question
The Introduction will end in a general statement of the study objectives, e.g. "to
determine whether acupuncture is effective in rehabilitation after stroke".
The next section defines precisely the specific objective(s) of the study. A
clearly defined, single main question is the most important requirement for
successful clinical trials. This can be expressed as either a question or an
hypothesis (preferred in the USA). Thus, for example: "Is standard
manual acupuncture superior to sham acupuncture in improving the quality of life
after ischaemic stroke?" is equivalent to the hypothesis "Acupuncture
is superior to sham acupuncture in improving... ". The aim is sometimes
stated as a null hypothesis, for example "Acupuncture is not superior to
sham acupuncture...": because this is what inferential statistics
actually test, although this can just add confusion.
The researcher should be clear whether the objective of the
study is to test the efficacy or the effectiveness of acupuncture. Efficacy is
the extent to which acupuncture produces a beneficial result under ideal
conditions, whereas effectiveness is a measure of the success of acupuncture
when carried out in an average clinical environment. Efficacy studies have much
tighter inclusion criteria, whereas effectiveness studies have more clinically
relevant outcome measures.
Secondary questions may be asked in the trial, but should
remain subordinate to the main question. However, they must be defined with just
as much rigour and precision.
Design
A single brief sentence makes it clear to the reader what you are planning - and
will make it instantly obvious whether the design is right! Most rigorous
studies follow randomised subjects forward over time, and may have various
levels of blinding, in a design with 2 or more parallel arms, expressed from the
following options:
- Prospective, retrospective (theoretically)
- Randomised, non-randomised
- Blinding of: subject, evaluator, experimenter
- Control group: sham, no treatment, comparative treatment
- Number of parallel arms
Subjects and methods
Recruitment
One common method of recruiting participants is from successive patients
presenting with the particular condition in a clinic. Precise details should be
given on how the participants are to be approached, and if it is by letter, the
ethics committee will want to see it. Recruitment from media or other publicity
may result in more chronic and resistant cases, or more complex psychological
problems, than average. Include essential details for the screening process that
will be applied, e.g. adults aged 18-65 years of either sex who have had the
condition for at least 6 months.
Sample size
Calculation of sample size is an essential component of research design that is
all too often omitted. The purpose of this is to recruit enough subjects to get
valid results but not more than is needed to produce an answer: so as not to
waste the time, money and effort of participants and investigator. The required
sample size depends on four variables:
alpha, the threshold for significance (which is typically set to 0.05)
beta, the probability of missing an important difference or making a Type II
error (typically, p=0.20, which is equivalent to 80% power)
delta, the minimum difference in the primary outcome measure between groups that
you wish to detect as statistically significant (considering clinical
significance too, if appropriate)
SD,the estimated standard deviation of the primary outcome measure, which can be
gained from a pilot study or from previous publications.
Altman gives an excellent description of a simple method (3).
Inclusion and exclusion criteria
These criteria are applied to make sure that each participant is appropriate and
the group as a whole is homogeneous. The inclusion criteria will usually be a
more precise statement of the medical status, such as the type and severity of a
headache. Importantly, it is wise to include informed consent as one of
the inclusion criteria, to give it due prominence.
Exclusion criteria may include certain medical conditions
(diagnosis of cancer or use of particular drugs, for example), contraindications
to acupuncture (such as refusal to be needled, or major bleeding disorder),
previous acupuncture treatment (for a trial that involves participant-blinding),
and anything that will interfere with measuring the effect of the therapy,
including lack of necessary skills (such as the ability to understand
instructions on how to complete a questionnaire), the likelihood of getting
better without the treatment (such as those with very mild stroke), and the use
of other simultaneous treatments.
Information sheet
Once these criteria have been applied, full verbal and written information about
the trial is given, questions are then invited, and answers given. This 3-part
procedure is essential for informed consent. The information sheet and consent
form are an integral part of the protocol, as an appendix. Many LRECs give clear
advice on what should be in an information sheet, and many also produce a
standardised consent form, When consent has been given, the patient becomes a participant
and is formally enrolled in the study, receiving a unique study number.
Withdrawal criteria
Data from participants who simply drop out of the study must still be included
in an intention-to-treat analysis, discussed below. Those who withdraw
for genuine reasons can be excluded, provided the rules for withdrawal have been
defined and are applied without bias. For example, "Participants
developing serious intercurrent illness may be withdrawn from the study after
assessment by a physician not involved in the study and blinded to the
participant’s group allocation"! Group allocation
If the participants are to be divided into groups, specify the allocation
method. This should usually be by randomisation. Include information on how
randomisation is achieved, and how the allocation will be concealed at the time
when the decision is taken to include the participant in the study. It is
possible that participants need to be stratified by prognostic factor before
randomisation.
Intervention and control procedures
Acupuncture treatment
Define precisely what points will be used, how they will be stimulated and for
how long with what sort of needles. If the treatment is to be individualised,
state the criteria for choosing the points. This information needs to be
comprehensive so that it can be repeated by other researchers and clinicians,
otherwise there is little purpose in doing the research.
Control intervention
The control group may receive nothing, an active therapy, or a sham form of
acupuncture, depending on the research question. For a discussion of the
possible controls that have been used in acupuncture trials, see Filshie and
Cummings (4). State full details of the control intervention so that it, too,
can be replicated.
Schedule of interventions
State the frequency and number of treatment and control sessions. lt is usual to
specify what proportion of the total number each participant must complete to
satisfy the protocol, and whether gaps in treatment are allowed, e.g. for
holidays.
Blinding of participants and evaluators
If the design involves blinding, state how this is to be achieved and maintained
and, moreover, state how it will be checked at the end of the study. If a sham
control is used, consider testing whether participants have been able to tell
that it is not genuine acupuncture. This will involve asking standard questions
of both the active and the control groups, since they should be managed in an
identical way as far as possible.
Patient evaluation
First, specify what outcome measures will be used, and then consider the methods
and schedule for collecting the data.
Outcome measures
This should be a single measure, the primary outcome measure, on which
the overall success or failure of the treatment is assessed. Objective measures,
such as blood pressure, are ideal for efficacy studies where the aim is to
determine whether acupuncture has an effect on the condition at all. When
studying the effectiveness in real life, however, it may be more appropriate to
use a measure that is more relevant subjectively to the participants, such as
pain or quality of life. lt is essential to use a primary outcome measure that
has been validated, and to quote the reference.
There are other measures, known as secondary outcome
measures, which support or amplify the information from the primary measure
or answer secondary questions. For example, if the primary outcome measure in a
study of stroke is the Activities of Daily Living, secondary measures could
include measures of spasticity, timed lo-metre walk, dysphagia, and Hospital
Anxiety and Depression scale. Purpose-designed measures may also be used here,
and may throw useful light on other aspect of the treatment effects that the
researcher is interested in.
Data collection
Baseline data will be required from all participants in order to describe and
compare the make-up of the experimental and control groups. Baseline data should
include usual demographic details plus any information that may affect the
outcome in individual participants. For example, in a study of smoking
cessation, patients who live alone have less success in stopping, so this risk
factor needs to be documented. The actual data forms should be added to the
protocol as an appendix, including the standard forms for recognised measures
and those that have been developed specifically for the study. Follow-up data
may be of crucial importance and the schedule for collecting it must be
specified.
Economic evaluation
It has been recommended that economic evaluation should take place alongside
clinical evaluation in all clinical trials, so that some evaluation of the
cost-effectiveness of the therapy may be obtained. If measurement of costs is to
be included, these need to be predefined in the same way as other outcome
measures.
Adverse reactions
Every study should make the collection of adverse reactions a routine.
Participants should be asked specifically about adverse events.
Confidentiality
This is a good point to describe how personal information is to be kept
confidential. This might involve training staff, keeping paper records securely
locked up, and removing personal details before entering data on computers. The
organisation’s Data Protection Officer may need to be notified, if
appropriate.
Ethics
The protocol may contain a statement of any particular ethical problems which
are anticipated (for example, obtaining consent from patients who have had a
stroke affecting the speech area), any sources of ethical advice, and the
relevant LREC. Ethics committees are primarily concerned with the interests of
the participants.
Statistical methods
Missing data
The method of dealing with missing data from participants who have dropped out
should be specified before the study begins, to avoid the accusation of biasing
the result. There are different ways of doing this: one conservative method
involves substituting baseline data, which means assuming there has been no
change. A more liberal method allows the substitution of an average of all the
other participants’ results.
Analysis
Analysis usually involves statistical tests, and the choice of test depends on
the answers to two questions: What kind of data will be collected? and What
is the goal of the study? Different types of data include measurement from
Gaussian population, rank, score, or measurement from non-Gaussian population,
binomial or dichotomous data, and survival time. Examples of a goal are to
compare two paired groups, or to compare three or more unpaired groups. Precise
statistical methods are beyond the scope of this article, but Motulsky gives a
helpful table (5). It is usual to include the words intention-to-treat
analysis and to include all data including that from those who dropped out
of the study.
Miscellaneous
Costs of study
Under this section, one should consider refunding travel costs and other
out-of-pocket expenses for participants, in addition to costs of therapists,
evaluators and (possibly) statistical advice and analysis, together with any
equipment that may be required. Grant-awarding bodies will look for accurate
estimates of all costs and clear justification for them. Ethics committees will
look for any hint that the study may be biased to satisfy a sponsor, and ensure
that the sponsor will not have the right to decline publication of the report.
Time schedule and feasibility
The proposed time schedule should be outlined, marking milestones such as ethics
approval, preparation of forms, staff training, patient enrolment, data
collection and data entry, analysis, and writing the report. Attention may need
to be paid to precisely how the data forms are to be processed and the data
extracted. This section involves also assessing the feasibility of achieving
each step within. the allotted time.
Conclusion
A protocol is an essential starting point for all ethical research. Research in
acupuncture has generally been of low standards, which means it has been open to
bias, error and confounders. It is imperative to conduct good quality research
in order to treat patients with the greatest possible benefit and least possible
harm.
Acknowledgement
The authors wish to acknowledge the help of Prof Ernst in the preparation of
this article.
Adrian White MA BM BCh
Jongbae Park MKM KMD
Department of Complementary Medicine
School of Postgraduate Medicine
and Health Sciences
University of Exeter, 25 Victoria Park Road
Exeter EX2 4NT (UK)
Email: a.r.white@ex.ac.uk
References
- Committee for Proprietary Medicinal Products (CPMP) Working Party on
Efficacy of Medicinal Products. Good clinical practice for trials on
medicinal products in the European Community: EEC notes for guidance. Pharmacology
& Toxicology 1990; 67: 361-72.
- Editorial. Denmark takes a lead on research ethics. British Medical
Journal 1998; 316: 1189.
- Altman DG. Practical Statistics for Medical Research. 1st ed.
London: Chapman and Hall; 1991. p.455-60.
- Filshie J, Cummings TM. Western medical acupuncture. In: Ernst E, White
AR, editors. Acupuncture: A Scientific Appraisal. Oxford: Butterworth-Heinemann;
1999.
- Motulsky H. Intuitive Biostatistics. Oxford: Oxford University Press 1995;
p.298.
