adaptive_designs_th

1/40 Adaptive and group-sequential methods for clinical trials – Part II Group-sequential and combination test methods Thomas Hamborg t.hamborg@warwick.ac.uk Statistics & Epidemiology, Division of Health Sciences, Warwick Medical School 10 January 2012 Table of Contents 1. Introduction 2. Group sequential methods 2.1 General Setup 2.2 Classical Designs 2.3 Alpha-spending functions 2.4 Recursive numerical integration 3. Combination test methods 3.1 Introduction 3.2 Combination functions 3.3 Inverse normal combination function 4. Conditional error method 5. Summary 2/40 Introduction — Adaptive Designs I Use interim analyses to assess gradually accumulating data I Adapt design for remainder of trial Definition by CHMP(2007) A study design is called adaptive if statistical methodology allows the modification of a design element (e.g. sample-size, randomisation ratio, number of treatment arms) at an interim analysis with full control of the type I error. 3/40 Introduction — Adaptive Designs Focus on methods for confirmatory trials: I Sample size re-estimation based on nuisance parameter estimates I Sample size re-estimation based on efficacy estimates (including ’self-designing trials’) I Selection or modification of hypotheses tested including subgroup or treatment selection I Early stopping for futility I Early stopping for positive results 3/40 Introduction — Adaptive Designs Focus on methods for confirmatory trials: I Sample size re-estimation based on nuisance parameter estimates I Sample size re-estimation based on efficacy estimates (including ’self-designing trials’) I Selection or modification of hypotheses tested including subgroup or treatment selection I Early stopping for futility I Early stopping for positive results 4/40 Introduction — Adaptive Designs Statistical challenge: Inflation of type I error rate must be avoided! Sources of potential type I error inflation I Repeated hypothesis testing with early rejection/acceptance of null hypotheses I Adaptation of design/analysis features with combination of data across stages I Multiple hypothesis testing 4/40 Introduction — Adaptive Designs Statistical challenge: Inflation of type I error rate must be avoided! Sources of potential type I error inflation I Repeated hypothesis testing with early rejection/acceptance of null hypotheses I Adaptation of design/analysis features with combination of data across stages I Multiple hypothesis testing 5/40 Introduction — How bad can the naı¨ve approach be? Two arm randomised (phase III) clinical trial Compare test statistic for accumulated data at interim to fixed design critical value I Significant at 5% level⇒ stop trial I Not significant⇒ continue with trial # Looks 2 3 4 10 20 50 ∞ Equally spaced .083 .107 .126 .193 .248 .319 1 Worst case .098 .143 .185 .401 .642 .923 1 Table: Type I error rate by number and timing of interim analyses for two-sided test at α = 0.05. 6/40 Group sequential methods — Table of Contents 1. Introduction 2. Group sequential methods 3. Combination test methods 4. Conditional error method 5. Summary 7/40 Group sequential methods — General Setup General Setup Repeated hypothesis testing: Interim analyses to compare treatments E and C Conduct tests so that overall type I error rate does not exceed pre-specified type I error rate α Wish to I Allow for early stopping I Do not allow other adaptation 8/40 Group sequential methods — General Setup Notation θ Scalar parameter of interest (measure of superiority of E over C) Test at jth interim analysis expressed in terms of I Sj – efficient score statistic I Ij – (observed) Fisher’s information Test statistics approximately satisfying S ∼ N(θI, I) exist for each response type 9/40 Group sequential methods — General Setup Stopping boundaries Test H0 : θ = 0 At jth interim analysis Sj ≥ uj: stop and reject H0 Sj ≤ lj: stop and reject H0 (two-sided test) or stop and accept H0 (one-sided test) Stop at the Jth look if not before {u1, . . . , uJ} and {l1, . . . , lJ} form stopping boundaries {(l1, u1), . . . , (lJ, uJ)} form continuation region 10/40 Group sequential methods — Classical Designs Repeated significance tests Idea: “Raise the bar” Modify critical values and define stopping boundaries in terms of critical values E.g. for upper boundary of two-sided test determine uj, j = 1, . . . , J so that: Pr (∪Jj=1Sj ≥ uj) = α/2 11/40 Group sequential methods — Classical Designs Two classical designs are by Pocock (1977) and O’Brien and Fleming (1979): I Pocock: reject H0 if | Sj |≥ uj = uP √ k I OBF: reject H0 if | Sj |≥ uj = uOBF √ K Pocock boundary corresponds to constant nominal p-value Nominal OBF p-value is increasing with increasing j Both designs require finding a single constant (uP, uOBF)⇒ Recursive numerical integration by Armitage, McPherson, Rowe (1969) 12/40 Group sequential methods — Classical Designs Stopping boundaries two-sided test I 13/40 Group sequential methods — Classical Designs Stopping boundaries two-sided test II 14/40 Group sequential methods — Classical Designs Stopping boundaries one-sided test 15/40 Group sequential methods — Classical Designs I Many more group sequential tests exist with different sequences of critical values, e.g. Wang-Tsiatis family uj = uWT(j/J)∆ , 0 ≤ ∆ ≤ 0.5 I All ensure that overall type I error is equal to pre-specified value if test statistics have the canonical form I But all these designs require fixed number and spacing of interim looks specified at the design stage I type I errorX power ? 16/40 Group sequential methods — Alpha-spending functions Spending functions - Idea Instead of specifying number and timing of looks, specify spending function α∗ telling how much α to use by information time Ij/IJ ∈ (0, 1) I Permits changing the number and spacing of the interim looks without affecting type I error. I Specify error spending function at design stage I Re-calculate boundaries if actual interim monitoring schedule is altered 17/40 Group sequential methods — Alpha-spending functions Spending function approach Specify increasing functions 0 = α∗U(0) ≤ α∗U(1) ≤ . . . ≤ α∗U(J) = α/2 0 = α∗L(0) ≤ α∗L(1) ≤ . . . ≤ α∗L(J) = α/2 (two-sided) or 0 = α∗L(0) ≤ α∗L(1) ≤ . . . ≤ α∗L(J) = 1− α/2 (one-sided) Find (l1, u1), . . . , (lJ, uJ) such that Pr(stop on upper boundary by jthlook | H0) = α∗U(j) Pr(stop on lower boundary by jthlook | H0) = α∗L(j) for j = 1, . . . , J. 18/40 Group sequential methods — Alpha-spending functions Spending function approach Consider upper boundary only. From previous slide follows Pr(Sj > uj, Sl < ul, l = 1, . . . , j− 1) = α∗U(j)− α∗U(j− 1) ⇒ Solve recursively for u1, . . . , uJ under H0. Type I error is preserved because α∗U(1)+[α ∗ U(2)−α∗U(1)]+. . .+[α∗U(J)−α∗U(J−1)] = α∗U(J) = α 19/40 Group sequential methods — Alpha-spending functions Examples Flexible spending function families exist which can also approximate group sequential boundaries: I Pocock type: α∗(j) = α log{1 + (e− 1)tj} I OBF type: α∗(j) = 2{1− Φ(Z1−α/2/√tj)} I linear: α∗(j) = αtj I Hwang, Shih, DeCani: α∗(j) = α { 1−exp(−γtj) 1−exp(−γ) } for tj = Ij/IJ 20/40 Group sequential methods — Alpha-spending functions Alpha-spending illustration 20/40 Group sequential methods — Alpha-spending functions Alpha-spending illustration 21/40 Group sequential methods — Alpha-spending functions Final comments I Timing of looks flexible but assumed to be non-data driven (not based on observed S1 . . . Sj) I In practise impact on power and type I error is modest (Lan, DeMets (1989)) because power depends little on number of looks I Asymmetric upper and lower boundaries can be chosen 22/40 Group sequential methods — Recursive numerical integration Stopping boundary calculation Obtain boundary by consideration of density of S at each look First interim analysis: S1 ∼ N(0, I1) (under H0) f1(z1) = 1√ I1 φ ( z1√ I1 ) Score statistics for variety of data types can be shown to have canonical form Canonical form implies independent increment structure: Sj − Sj−1 ∼ N(θ(Ij − Ij−1), Ij − Ij−1) ind. of S1, . . . , Sj−1 Stopping boundary calculation Second interim analysis: I Can stop at first look (S1 /∈ (l1, u1)) I Or second look (S2 is sum of S1 ∈ (l1, u1) and increment S2 − S1 ) f2(z2) = u1∫ l1 1√ I2 − I1 φ ( z2 − z1√ I2 − I1 ) f1(z1)dz1. S I u1 l1 Stopping boundary calculation Second interim analysis: I Can stop at first look (S1 /∈ (l1, u1)) I Or second look (S2 is sum of S1 ∈ (l1, u1) and increment S2 − S1 ) f2(z2) = u1∫ l1 1√ I2 − I1 φ ( z2 − z1√ I2 − I1 ) f1(z1)dz1. S I u1 l1 u2 l2 Stopping boundary calculation Whole boundary (and density of final S) can be found recursively: fj(zj) = uj−1∫ lj−1 1√ Ij − Ij−1 φ ( zj − zj−1√ Ij − Ij−1 ) fj−1(zj−1)dz1. S I u1 l1 u2 l2 u3 l3 Same technique can be used for calculation of maximum sample size and repeated significance testing 25/40 Combination test methods — Table of Contents 1. Introduction 2. Group sequential methods 3. Combination test methods 4. Conditional error method 5. Summary 26/40 Combination test methods — Introduction The basic idea Combining evidence from different stages by aggregating stage-wise (not cumulative!) test statistics in pre-defined way Under null hypothesis distribution of these statistics is known⇒ type I error rate can be controlled Consider two-stage design for illustration below 27/40 Combination test methods — Introduction Example: Fisher’s product Combine evidence from two stages via the product of independent (one-sided) p-values from two stages p1 × p2 Under H0 without early stopping: ln(p1 × p2)−1/2 ∼ χ24 Or in general reject H0 if − ln(p1 . . . pJ) > χ22J(1− α/2) 28/40 Combination test methods — Introduction Fisher’s product with early stopping Let α1 < α early rejection boundary and β1 > α early acceptance boundary, 0 ≤ α1 < α < β1 ≤ 1 Bauer and Ko¨hne (1994) idea: overall type I error rate is retained for second stage critical value Cα2 given by Cα2 = α− α1 ln(β1)− ln(α1) I Can find Cα2 for pre-specified α1, β1 I Or find α1,Cα2 jointly subject to constraint 29/40 Combination test methods — Combination functions Combination function Base decision on stage-wise p-values assumed i.i.d. U[0, 1] (strictly only p-clud) Proceed to 2nd stage if α1 < p1 ≤ β1 Base 2nd stage decision on a combination function C(p1, p2) Reject if C(p1, p2) ≤ c so that α1 + β1∫ α1 1∫ 0 1{C(x, y) ≤ c} dx dy = α 30/40 Combination test methods — Combination functions Examples Popular combination functions: I Fisher’s product C(p1, p2) = p1 × p2 I Weighted inverse normal (WIN) C(p1, p2) = 1− Φ ( ω1Φ −1 (1− p1) + ω2Φ−1 (1− p2) ) with ω1, ω2 pre-planned such that ω21 + ω 2 2 = 1 I Proschan & Hunsberger method C(p1, p2) = 1− Φ (( Φ−1 (1− p1) )2 + ( max[0,Φ−1 (1− p2)] )2) Combination function illustration 1 10 p1 p2 c c reject H0 fail to reject H0 area = 1− α Figure: Rejection regions for Fisher’s product (red) and inverse normal (blue) combination function 32/40 Combination test methods — Inverse normal combination function Inverse normal method – details WIN combination function has special relation to group sequential designs Assume one-sided p-values, then Yj = Wj Φ−1 (1− pj) ∼ N(0,W2j ) Now put W2j = (Ij − Ij−1), so that Yj = √ (Ij − Ij−1)Φ−1 ( Φ ( (Sj − Sj−1) / √ Ij − Ij−1 )) = (Sj − Sj−1) 33/40 Combination test methods — Inverse normal combination function Inverse normal method – details It follows that Sj = Y1 + . . .+ Yj = Sj and Ij = W21 + . . .+ W 2 j = Ij Inverse normal approach and group sequential approach are identical in this case! I Critical values α1, β1, c can be computed with standard group sequential software, e.g. α1 = 1− Φ ( l1/ √ I1 ) I Combination function weight link: ω2j = (Ij − Ij−1)/IJ I Extends to more than two stages 34/40 Conditional error method — Table of Contents 1. Introduction 2. Group sequential methods 3. Combination test methods 4. Conditional error method 5. Summary 35/40 Conditional error method — The basic idea Let Pr(reject H0 | p1) = A(p1) conditional error function (given observed stage 1 p-value) Early acceptance⇒ A = 0 Early rejection⇒ A = 1 Reject 2nd stage if p2 ≤ A(p1) Control the conditional type I error: Mueller and Scha¨fer (2004) showed that overall type I error is preserved for any data-dependent change in trial if conditional type I error is preserved, i.e. EH0(A) ≤ α ⇒ Design determined by observed statistic 36/40 Conditional error method — Combination function link Conditional error function corresponding to combination test given by A(p1) =  1 p1 ≤ α1 maxx∈[0,1]{x | C(p1, x) ≤ c} α1 < p1 ≤ β1 0 p1 > β1 General case 36/40 Conditional error method — Combination function link Conditional error function corresponding to combination test given by A(p1) =  1 p1 ≤ α1 c/p1 α1 < p1 ≤ β1 0 p1 > β1 Fisher’s product 36/40 Conditional error method — Combination function link Conditional error function corresponding to combination test given by A(p1) =  1 p1 ≤ α1 1− Φ ( Φ−1(1−c)−ω1Φ−1(1−p1) ω2 ) α1 < p1 ≤ β1 0 p1 > β1 Weighted inverse normal 37/40 Summary — Table of Contents 1. Introduction 2. Group sequential methods 3. Combination test methods 4. Conditional error method 5. Summary 38/40 Summary — Pros and Cons of the two approaches Kelly et al. (2005) compared Fisher’s combination, inverse normal and group sequential method I For two-stage design with unaltered interim the continuation regions coincide and the power is very similar I For more stages power is slightly less for Fisher’s combination method I Little reason to prefer one approach to another (if no design modification allowed) Using combination test method in unplanned manner typically 10–25 % less efficient than group sequential test (Jennison and Turnbull (2003)) 39/40 Summary — Summary I Group sequential methods generally regarded as well understood I Some research questions remain (optimal designs, overrunning, use of secondary endpoint information) I Combination test methods somewhat less well understood and potentially less efficient but offer additional flexibility I Approaches not discussed: Continuous boundaries (e.g. triangular test), repeated confidence intervals 40/40 Summary — Books I Jennison and Turnbull (2000) Group sequential methods with applications to clinical trials. Chapman and Hall/CRC. I Proschan, Lan and Wittes (2006) Statistical monitoring of clinical trials. Springer. I Chang (2007) Adaptive design theory and implementation using SAS and R. Chapman and Hall/CRC. I Pong and Chow (2010) Handbook of Adaptive Designs in Pharmaceutical and Clinical Development. Chapman and Hall/CRC. Introduction Group sequential methods General Setup Classical Designs Alpha-spending functions Recursive numerical integration Combination test methods Introduction Combination functions Inverse normal combination function Conditional error method Summary