ClinicalTrials.gg
Sign In
CompletedNCT03498677Updated Mar 22, 2021

Protocol Lab for Online Trial Delphi and Question Effects

An interventional study of presentation order and alternative presentation order in Behavior, Adaptive and Decision Analysis, sponsored by ThinkWell. Completed at 1 site in United Kingdom. Open to participants aged 18 Years and older, including healthy volunteers. Per ClinicalTrials.gov, last updated 2021-03-22.

Sponsored by ThinkWell · Not applicable, Interventional, and Other

Phase
Not applicable
Study type
Interventional
Enrollment
385
Allocation
Randomized
Ages
18 Years and older
Sex
All
01

Study summary

This study nested as a randomized controlled trial within an online Delphi and will assess the impact of using different question section groupings to provide aggregate feedback to participants in an online Delphi study.

Read the detailed description

A similar assessment has been done during the development of a core outcome for critical care trials set with a nested study examining the impact of question order on prioritization of outcomes. Research reports different stakeholders groups may differ in how they value or prioritize research questions and outcomes and it is recommended that each group should be adequately represented.

The Protocol Lab for Online Trials-Delphi (PLOT-D) study will use an online Delphi combined with participatory action research to inform the development of a multi-use protocol template for writing protocols for self-recruited online trials of interventional self-management. The Protocol lab will use the Delphi findings, along with earlier research to redesign a series of protocols for online randomized trials with the aim of providing support for citizens to work alongside researchers to build participatory health trials online.

02

Conditions studied

  • Behavior, Adaptive
  • Decision Analysis

Browse trials for

Keywords

  • online trial
  • decision analysis
03

In context

Behavior

1,573 studies on the registry are indexed under Behavior; 313 are open to participants now.

This study's enrollment of 385 is above the median of 120 across 1,197 interventional studies indexed under Behavior.

Browse Behavior studies →

Lead sponsor

ThinkWell is the lead sponsor of 9 studies on the registry; none are open to participants now.

Counted across the registry records on this site, refreshed daily.

04

Who can participate

Ages eligible
18 Years and older
Sexes eligible
All
Accepts healthy volunteers
Yes

Go through the criteria one by one — yes, no or not sure — and take the list to your doctor or the study team. Kept in this browser only.

Inclusion criteria

  • Protocol lab for online trials Delphi (PLOT-D) participants

Exclusion criteria

Exclusion Criteria:

  • None if inclusion criteria are met
05

Study design

Phase
Not applicable
Primary purpose
Other
Allocation
Randomized
Intervention model
Parallel assignment
Masking
None (open label)
Enrollment
385 participants (actual)

Study arms

  • Active comparator
    Method of presentation one

    Behavioral: presentation order

  • Active comparator
    Method of presentation two

    Behavioral: alternative presentation order

Interventions

  • Behavioralpresentation order

    section presentation one

  • Behavioralalternative presentation order

    section presentation two

06

What researchers measure

Primary outcomes

  1. Context effects of question order

    measure to what extent question section order changes context response between conditions

    Time frame: Up to 20 weeks or study completion

Secondary outcomes

  1. Question section presentation order

    response differences amongst stakeholder groups randomized to 1 of 2 presentation order conditions

    Time frame: Up to 20 weeks or study completion

  2. Retention

    difference in number of items retained at the end of the first Delphi round by randomized assignment to condition

    Time frame: Up to 20 weeks or study completion

07

Study locations

1 site
  • ThinkWell
    Oxford, Oxfordshire OX4 4DN, United Kingdom
08

References and documents

Publications

  • Brookes ST, Chalmers KA, Avery KNL, Coulman K, Blazeby JM; ROMIO study group. Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial. Trials. 2018 Jan 25;19(1):66. doi: 10.1186/s13063-017-2405-6. PubMed 29370827 ↗
  • Potter S, Brookes ST, Holcombe C, Ward JA, Blazeby JM. Exploring methods the for selection and integration of stakeholder views in the development of core outcome sets: a case study in reconstructive breast surgery. Trials. 2016 Sep 23;17(1):463. doi: 10.1186/s13063-016-1591-y. PubMed 27664072 ↗
  • Snow R, Crocker JC, Crowe S. Missed opportunities for impact in patient and carer involvement: a mixed methods case study of research priority setting. Res Involv Engagem. 2015 Aug 4;1:7. doi: 10.1186/s40900-015-0007-6. eCollection 2015. PubMed 29062496 ↗
  • Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011 Jan 25;8(1):e1000393. doi: 10.1371/journal.pmed.1000393. PubMed 21283604 ↗
  • Fletcher AJ, Marchildon GP. Using the Delphi Method for Qualitative, Participatory Action Research in Health Leadership. International Journal of Qualitative Methods 2014;13:1-18
  • Brice A, Price A, Burls A. Creating a database of internet-based clinical trials to support a public-led research programme: A descriptive analysis. Digit Health. 2015 Nov 20;1:2055207615617854. doi: 10.1177/2055207615617854. eCollection 2015 Jan-Dec. PubMed 29942546 ↗
  • Bagley HJ, Short H, Harman NL, Hickey HR, Gamble CL, Woolfall K, Young B, Williamson PR. A patient and public involvement (PPI) toolkit for meaningful and flexible involvement in clinical trials - a work in progress. Res Involv Engagem. 2016 Apr 27;2:15. doi: 10.1186/s40900-016-0029-8. eCollection 2016. PubMed 29062516 ↗
  • Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, Burls A. Patient and public involvement in the design of clinical trials: An overview of systematic reviews. J Eval Clin Pract. 2018 Feb;24(1):240-253. doi: 10.1111/jep.12805. Epub 2017 Oct 27. PubMed 29076631 ↗
  • Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991

Individual participant data

Plan to share: Yes — Aggregated de-identified data will be available on reasonable request

Supporting information: Study protocol, Informed consent form

Time frame: Post-study, five years

Access criteria: Contact study investigator

09

Timeline

Tracking since Sep 25, 2026
ElapsedEstimatedEstimated, not yet reachedSimilar studies: half by the dot; the band is the middle halfHalf of similar studies reached it by the dot; the band is the middle half, from a quarter to three quarters

Ran from May 2018 to primary completion in Apr 2019, 11 months.

Of 741 non-industry interventional studies with no phase under Behavior, half reached primary completion within 14 months of starting (the median), counting those still running; a quarter within 5 and three quarters within 32. This study's 11 months is below that.

Counting every sponsor, half of 757 studies reached it within 13 months; a quarter within 5 and three quarters within 32.

How these figures are worked out, and which studies are left out of them.

Registered
Apr 1, 2018
First posted
Apr 17, 2018
StartActual
May 29, 2018
Primary completionActual
Apr 30, 2019
CompletionActual
Oct 30, 2019
Last update
Mar 22, 2021
What these dates mean
Start.
When the first participant enrolled; while estimated, when the study expects to open to recruitment.
Primary completion.
When the last participant was examined or treated to collect data for the study's main question, its primary outcome. A study that stopped early gives the date its data collection stopped.
Completion.
When the last participant's last visit took place, for every outcome, secondary ones and side effects included: the end of the study. Often later than primary completion, sometimes the same day.
Actual and Est.
Actual means the registry says the date has happened. Est. (estimated) is a date the sponsor plans or expects. It can still move, and the sponsor is expected to change it to actual once it has passed.
No changes since tracking began. The registry record was last updated on Mar 22, 2021, before this site started recording changes on Sep 25, 2026. Its history is on ClinicalTrials.gov ↗
10

Registry details

Key details

Study ID
NCT03498677
Lead sponsor
ThinkWell
Responsible party
Sponsor
First posted
Apr 17, 2018
Start date
May 29, 2018
Primary completion
Apr 30, 2019
Completion
Oct 30, 2019
Last update
Mar 22, 2021

Study contacts

Amy Price, PhD
principal investigator · University of Oxford

Oversight

Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Mar 2021. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Other studies in this condition

Behavior →

Other studies from this sponsor

ThinkWell

Start the discussion