How to Choose Between Simon Designs: A Practical Guide for Phase II Clinical Trials

Why Multiple Designs Exist

Selecting a Simon's two-stage design is often more difficult than calculating one. Most Simon design calculators generate two recommended designs—the Optimal design and the Minimax design. Some also provide a range of admissible designs between these extremes.

So which design should you actually choose? The answer is clear: there is no universally best Simon design. The appropriate choice depends on the scientific, ethical, and operational priorities of your trial.

A Simon's two-stage design must satisfy two statistical requirements: Type I error (α) and Type II error (β), or equivalently statistical power. Many combinations of stage 1 sample size (n₁), early stopping boundary (r₁), total sample size (N), and final success boundary (r) can satisfy these requirements.

Although these designs provide similar statistical operating characteristics, they differ in expected sample size, maximum sample size, probability of early stopping, and operational practicality. Rather than searching for a single “best” design, the goal is to choose the design whose trade-offs best fit your study.

The Two Classical Designs

The Optimal Design minimizes the expected sample size under the null hypothesis (p₀). In practice, ineffective treatments are more likely to stop early, reducing the average number of patients enrolled across many trials.

Advantages include the lowest expected sample size, reduced average patient exposure to ineffective treatments, and lower average trial cost. The disadvantages are that it often has a larger maximum sample size and may require more resources if the treatment ultimately proves promising.

The Minimax Design minimizes the maximum total sample size. Rather than optimizing average enrollment, it limits the largest trial you may have to conduct.

Advantages include the smallest maximum sample size, easier budgeting and resource planning, and a lower worst-case recruitment burden. The disadvantage is that it usually enrolls more patients on average when the treatment is ineffective.

Optimal designs are attractive when minimizing average patient exposure or trial cost is the primary objective. Minimax designs are often preferred when recruitment is challenging or resources are constrained.

Sometimes There Is No Trade-off

Not every trial presents a difficult decision. For some combinations of p₀, p₁, α, and β, the Optimal and Minimax designs are identical.

When this happens, the optimization criteria happen to identify the same integer solution, so no trade-off is required. The design team can proceed with the shared candidate while documenting the target error rates and power.

What Are Admissible Designs?

Simon later showed that many additional designs lie between the Optimal and Minimax extremes. These are known as admissible designs.

Each admissible design represents a different balance between expected sample size and maximum sample size. Rather than optimizing only one objective, admissible designs offer practical compromises between competing goals.

For many real-world trials, an admissible design may better reflect clinical priorities than either classical extreme.

For background on the Pareto frontier and how admissible designs are identified, see the article on admissible Simon designs.

A Simple Way to Think About It

Imagine choosing a car. One model offers excellent fuel economy but modest acceleration. Another delivers powerful acceleration but consumes more fuel. Neither car is objectively better. The right choice depends on what matters most to you.

Simon designs work the same way: every design represents a different compromise between competing objectives.

Which Metrics Should You Compare?

When evaluating candidate designs, compare more than just the total sample size. Considering these metrics together provides a much clearer picture than focusing on any single number.

PET Is More Than a Statistical Metric

The Probability of Early Termination (PET) has important ethical implications. A design with a higher PET is more likely to stop an ineffective treatment after the first stage, reducing unnecessary patient exposure and allowing resources to be redirected toward more promising therapies.

Two designs may have similar expected sample sizes but different PET values. If minimizing exposure to ineffective treatments is a major priority, PET may deserve greater weight in your decision.

PET should still be considered alongside the practical requirements of an interim analysis: timely outcome assessment, database review, statistical analysis, and a formal decision before recruitment continues.

A Practical Approach to Choosing Among Admissible Designs

When faced with several admissible designs, a structured approach can simplify the decision.

Operational Considerations Matter

Early stopping can reduce patient enrollment, but interim analyses also introduce additional operational work.

Depending on the study, an interim analysis may require timely outcome assessment, database review, statistical analysis, and a formal decision before recruitment continues.

These practical considerations may influence design selection just as much as statistical efficiency.

Avoid a Common Mistake

Many investigators automatically choose the Optimal Design simply because its name suggests superiority. This is a misconception.

“Optimal” means optimal for one specific mathematical objective—minimizing expected sample size under the null hypothesis. Likewise, the Minimax Design is optimal only for minimizing the maximum sample size.

Neither is universally better. The best design is the one that best aligns with your trial's scientific goals, ethical priorities, and operational constraints.

A Practical Decision Framework

Use the following framework to connect your trial priority to a candidate design.

How BioStatHub Helps

BioStatHub presents the Optimal, Minimax, and all admissible designs in a single interface, allowing you to explore the complete set of feasible solutions rather than focusing on only one recommended design.

Choosing a Simon design is much easier when you can compare candidate designs directly rather than evaluating them one at a time. BioStatHub allows you to select multiple designs for side-by-side comparison, including the Optimal, Minimax, and any admissible designs that interest you.

The comparison table presents key operating characteristics—including Expected Sample Size (ESS), Maximum Sample Size (N), Probability of Early Termination (PET), Stage 1 sample size (n₁), decision boundaries, and other design characteristics—for all selected designs in a single view.

Power Chart: Compare how the probability of declaring a treatment promising changes across a range of true response rates. Viewing multiple designs on the same chart makes it easy to understand where their operating characteristics differ, particularly around the null response rate (p₀) and the target response rate (p₁).

Expected Sample Size (ESS) Chart: Visualize how the expected number of enrolled patients changes with the true response rate for each design. This reveals the trade-offs between patient exposure and trial efficiency across the entire range of plausible treatment effects—not just under the null hypothesis.

These visualizations often reveal differences that are difficult to appreciate from summary statistics alone. Two designs may have nearly identical expected sample sizes at p₀, yet behave quite differently when the true response rate lies between p₀ and p₁. Comparing complete operating characteristic curves provides a much richer understanding of each design’s performance.

Rather than asking, “Which design is optimal?”, BioStatHub encourages the more meaningful question: “Which design best aligns with the scientific, ethical, and operational priorities of this trial?”

For a walkthrough of generating and comparing designs, see the BioStatHub tutorial.

Conclusion

Choosing a Simon design is ultimately about balancing competing objectives. The Optimal Design minimizes average enrollment. The Minimax Design limits the largest possible trial. Admissible designs offer meaningful compromises between these two extremes.

There is no universally best Simon design. There is only the design that best matches the goals of your trial.

Rather than relying on a calculator’s default recommendation, compare candidate designs using expected sample size, maximum sample size, probability of early stopping, Stage 1 feasibility, and operational considerations. Whenever possible, examine multiple designs side by side and explore their complete operating characteristics through Power and Expected Sample Size curves.

By understanding the trade-offs—and visualizing how candidate designs behave across the full range of treatment effects—you can make a transparent, defensible decision that balances statistical rigor, patient welfare, and operational feasibility.

Ultimately, a Simon design calculator should do more than generate numbers; it should help investigators understand the consequences of each design choice. The best design is not the one with the most impressive mathematical label, but the one that best serves the scientific objectives and practical realities of your Phase II clinical trial.

Frequently Asked Questions