Understanding the Power Function of Simon's Two-Stage Design

The basic setup

In a Simon two-stage Phase II clinical trial, power is often reported as a single number, such as 80% or 90% power at a specified alternative response probability. That number is important, but it tells only part of the story.

The power function provides a more complete picture. It shows how the probability of successfully rejecting the null hypothesis changes as the true response probability varies.

For Simon's two-stage designs, this is particularly useful because the trial can stop early for futility after Stage 1. The resulting power therefore depends on the complete two-stage decision process.

Let p denote the true response probability. The hypotheses for a typical single-arm Phase II trial are:

Here, p₀ is the response probability considered unacceptable. The design is usually constructed using a desirable response probability p₁, together with a Type I error α and target power 1 − β.

For example, p₀ = 0.15, p₁ = 0.35, α = 0.05, and 1 − β = 0.90. The design should control the probability of incorrectly rejecting H₀ when the treatment is ineffective, while providing the required probability of rejection when the true response probability is p₁.

What Is the Power Function?

Let R denote the event that the trial ultimately rejects the null hypothesis. For a particular true response probability p, define the power function as:

Thus, the power function is the probability that the trial will reject the null hypothesis for each possible value of the true response probability p.

The usual reported power at p₁ is simply one point on this function: π(p₁) = P(reject H₀ | p = p₁).

For example, if p₁ = 0.35 and the design has 90% power, then π(0.35) = 0.90. This does not mean that the treatment has a 90% probability of having a response probability of 35%. It means that, if the true response probability is 35%, the trial has a 90% probability of rejecting the null hypothesis.

The Power Function of a Simon's Two-Stage Design

Let n₁ be the Stage 1 sample size, n₂ the Stage 2 sample size, n = n₁ + n₂ the maximum total sample size, r₁ the Stage 1 stopping boundary, and r the final rejection boundary.

Let X₁ denote the number of responses observed in Stage 1. Under a true response probability p, X₁ follows a Binomial(n₁, p) distribution. Suppose the trial stops for futility when X₁ ≤ r₁, and therefore continues to Stage 2 only when X₁ > r₁.

Let X₂ denote the number of responses observed in Stage 2. Conditional on p, X₂ follows a Binomial(n₂, p) distribution. At the end of Stage 2, suppose the null hypothesis is rejected when the total number of responses satisfies X₁ + X₂ ≥ r + 1.

The power function is then:

For every value of p, this calculation determines the probability that the trial does not stop for futility at Stage 1 and ultimately reaches the final response boundary. Repeating the calculation over a range of p values produces the power curve.

Why the Power Function Is a Curve

The formula above gives one probability for one particular value of p. π(0.20) is the probability of rejecting H₀ when the true response probability is 20%; π(0.30) is the probability when it is 30%; and π(0.40) is the probability when it is 40%.

Evaluating π(p) over a range of values creates the power curve. Conceptually, p maps to π(p). For a typical design, power is low when p is near the null value and increases as the true response probability becomes more favorable.

The Relationship between p₀, p₁, and Power

The two response probabilities used to construct the design are especially important points on the power curve.

At the null boundary p = p₀, the design aims to control the Type I error. At the desirable alternative p = p₁, the design aims to achieve the target power.

With p₀ = 0.15, p₁ = 0.35, α = 0.05, and 1 − β = 0.90, the design aims for π(0.15) ≤ 0.05 and π(0.35) ≥ 0.90. These requirements constrain the design, but they do not completely describe its behavior. The power curve shows what happens at response probabilities between, below, and above these reference values.

What Happens between p₀ and p₁?

Consider p = 0.25. This is higher than the unacceptable response probability of 0.15 but lower than the desirable response probability of 0.35.

The probability of rejecting the null will generally be somewhere between the low rejection probability near p₀ and the target power at p₁. A design does not suddenly become powerful only when the true response probability reaches p₁. Instead, the probability of success generally changes continuously as p changes.

The power curve makes this behavior visible and helps investigators understand what the design says about response probabilities that fall between the two planning assumptions.

What Happens above p₁?

If p > p₁, the probability of rejecting H₀ will generally be higher than the target power at p₁. For example, if π(0.35) = 0.90, power at p = 0.40 will typically be higher.

As the true response probability becomes increasingly favorable, the probability of successfully rejecting the null will generally approach 1. The power curve therefore provides information about a much wider range of possible treatment effects than the single value π(p₁).

Power and Early Stopping

The two-stage structure means that power should not be confused with the probability of reaching Stage 2. The probability of continuing after Stage 1 is P(X₁ > r₁ | p). Power is instead the probability of ultimately rejecting H₀.

These are different quantities. A trial may have a high probability of reaching Stage 2 but still fail to meet the final rejection criterion. Conversely, an ineffective treatment may have a high probability of being stopped early.

The power function accounts for the entire two-stage process and therefore reflects the combined effect of the Stage 1 stopping rule and the final decision rule.

Why Power Curves Are Useful when Comparing Designs

Several Simon's two-stage designs may satisfy the same nominal statistical requirements. Two candidate designs might both satisfy π(p₀) ≤ α and π(p₁) ≥ 1 − β, while their power functions differ for response probabilities between p₀ and p₁ or above p₁.

When comparing candidate designs, investigators can consider Type I error, power at p₁, the power function, Stage 1 sample size, maximum sample size, expected sample size, and probability of early stopping together.

The power curve is one part of a broader comparison of operating characteristics. It can reveal differences that are not obvious from a table of sample sizes and target power alone.

Visualizing Power in BioStatHub

A mathematical formula gives the exact definition of the power function, but a graph often makes its behavior easier to understand.

In BioStatHub, the power curve of the final selected Simon two-stage design is displayed directly in the user interface. The same power curve is also included in generated reports.

The chart shows the relationship between the assumed true response probability p and the probability of rejecting the null hypothesis, π(p) = P(reject H₀ | p). This lets investigators see the behavior of the final design across a range of possible true response probabilities rather than relying only on the reported power at p₁.

Comparing Power Curves in BioStatHub

The power function becomes particularly valuable during design comparison. When multiple candidate Simon two-stage designs are compared in BioStatHub, their power curves are displayed together on the same chart.

This makes it easier to ask whether candidate designs have similar power around p₁, how quickly power increases above p₀, whether meaningful differences appear between p₀ and p₁, and how the designs behave when p is greater than p₁.

The purpose is not necessarily to identify a single design with the best power curve. Instead, the visualization provides another way to understand the trade-offs between admissible designs. A design with a sample-size advantage, for example, can be considered alongside its power behavior rather than evaluated in isolation.

Power Curves as Part of Design Selection

The power function fits naturally into the broader process of selecting a Simon two-stage design. First, candidate designs are generated that satisfy the required statistical constraints. Next, their operating characteristics can be compared. The power curves provide a visual representation of one of those characteristics.

The overall process can be thought of as Generate → Compare → Decide → Document. During Generate, valid candidate designs are identified. During Compare, investigators can examine sample size, expected sample size, early stopping behavior, and power curves. During Decide, these trade-offs can be considered together when selecting the design that best fits the study. During Document, the power curve of the final design can be included alongside the other design information.

In BioStatHub, the power chart is therefore part of the workflow for understanding and evaluating the design.

Power Is Not the Probability That the Treatment Works

There is one important statistical distinction to keep in mind. Suppose π(0.35) = 0.90. This means P(reject H₀ | p = 0.35) = 0.90.

It does not mean P(p = 0.35 | observed data) = 0.90. The first is a frequentist operating characteristic. It describes how often the trial would reject the null hypothesis if the true response probability were 35%, under repeated hypothetical repetitions of the trial.

The second is a probability statement about the unknown parameter after observing data and belongs to a different inferential framework. The power function should therefore be understood as a description of the operating behavior of the trial design under different assumed true response probabilities.

Takeaway

The power of a Simon two-stage design is often summarized by a single value, such as 80% or 90% at p₁. But the more complete object is the power function π(p) = P(reject H₀ | p).

For a two-stage design with early stopping, the power function incorporates both the Stage 1 stopping rule and the final decision rule. Evaluating it across a range of p values produces the power curve.

Looking at the complete curve provides information that cannot be obtained from the power at p₁ alone. In BioStatHub, the power curve of the final design is displayed in the UI and included in reports. When multiple candidate designs are compared, their power curves are displayed together, allowing investigators to see how power changes across designs and possible true response probabilities.

Frequently Asked Questions