What was prespecified?

The PsiDeR trial published in Nature Medicine randomised 60 people with treatment-resistant major depression 1:1 at a single NHS site. Its primary outcomes were feasibility: recruitment rate, retention rate and estimation of variance on the Montgomery-Åsberg scale. Of 75 people screened, 62 were eligible, 60 were randomised, and 59 of the 60 completed the scale at every follow-up visit.[1]

This is a well-run feasibility trial. Retention is high, the flow from screening to analysis is clear, and the scale was collected almost without gaps. In a feasibility trial the efficacy numbers are produced to size the larger study that follows, and the paper presents them that way.[1]

Where the blind broke

The authors report that blinding did not hold: everyone who received psilocybin, and 70 per cent of those on placebo, correctly guessed their allocation. The comparison was made against an inactive placebo. The authors' own assessment points the same way; they write that expectancy effects likely contributed substantially to the observed differences.[1]

When blinding falls away entirely in the active arm, the adjusted difference of 12.92 points at week 6 carries drug effect and expectancy effect together, and the design is not suited to separating them. The conclusion that follows need not be that the trial was run badly. With a substance whose subjective effects are unmistakable, blinding breaks in every trial; the loss may therefore be a limit on how much any psilocybin trial built against an inactive placebo can establish.[1]

Who the estimate describes

The authors report two further limits of scope: at the NHS site, Black, African, Caribbean or Black British participants were recruited below target, and the sample was highly educated with limited occupational diversity. That is the population the estimate describes. Response at week 3 was 43 per cent in the psilocybin arm against 3 per cent on placebo, and remission 40 per cent against 3 per cent.[1]

The test is simple. If psilocybin in treatment-resistant depression is trialled against an active comparator, or in a design that measures expectancy as a prespecified variable, the reported adjusted difference will come in below 12.92 points. The signal to watch is a published adjusted difference from such a trial. Until that number arrives we hold a hypothesis, and it stays one until it repeats.[1]