For real: a thorough look at numeric attributes in subgroup discovery

Subgroup discovery (SD) is an exploratory pattern mining paradigm that comes into its own when dealing with large real-world data, which typically involves many attributes, of a mixture of data types. Essential is the ability to deal with numeric attributes, whether they concern the target (a regression setting) or the description attributes (by which subgroups are identified). Various specific algorithms have been proposed in the literature for both cases, but a systematic review of the available options is missing. This paper presents a generic framework that can be instantiated in various ways in order to create different strategies for dealing with numeric data. The bulk of the work in this paper describes an experimental comparison of a considerable range of numeric strategies in SD, where these strategies are organised according to four central dimensions. These experiments are furthermore repeated for both the classification task (target is nominal) and regression task (target is numeric), and the strategies are compared based on the quality of the top subgroup, and the quality and redundancy of the top-k result set. Results of three search strategies are compared: traditional beam search, complete search, and a variant of diverse subgroup set discovery called cover-based subgroup selection. Although there are various subtleties in the outcome of the experiments, the following general conclusions can be drawn: it is often best to determine numeric thresholds dynamically (locally), in a fine-grained manner, with binary splits, while considering multiple candidate thresholds per attribute.

Futility for subgroup analyses in the adaptive signature design

Subgroup analyses in clinical trials are becoming increasingly important. In cancer research, more and more targeted therapies are explored from which probably only a portion of the whole population will benefit. An adaptive design for subgroup selection with identification of a subgroup, the adaptive signature design, was proposed in the literature. Unfortunately, measuring and validating the variables defining the subgroup (i.e. biomarkers) can be extremely expensive. For this reason, we propose an extension of this design where subgroup analysis is not performed when the overall results suggest that it is unlikely to achieve statistical significance in the subgroup. Avoiding measuring and validating expensive biomarkers in this case can save resources that could be used on more promising research.

Abstract WP25: A Novel Adaptive Trial Design Increases the Power of Endovascular Stroke Studies

Introduction: It is difficult to choose trial enrollment criteria that will yield a robust treatment effect. To address this problem, we developed a novel trial design that restricts enrollment criteria to the patient subgroup most likely to show benefit, if an interim analysis indicates futility in the overall sample. Future recruitment, and the population in which the primary hypothesis is tested, is limited to the selected subgroup. Hypothesis: A design with adaptive subgroup selection increases the power of endovascular stroke studies. Methods: We ran simulations to compare the power of the adaptive design with that of a traditional design. Trial parameters were: type I error 0.025, type II error 0.1, analysis after 450, 675 and 900 patients (interim and final analyses in IMS III). Outcome data were based on 90 day mRS scores observed in IMS III among patients with a vessel occlusion on baseline CTA (n=289). Subgroups were defined a priori according to vessel occlusion (ICA ± distal occlusion vs M1 vs M2-4), onset-to-randomization time (early vs late), and treatment allocation (IA+IV vs IV alone). The treatment effect in the overall cohort was a mean mRS improvement of 0.15 (2.41 for IV+IA vs 2.56 for IV alone; SD 1.45). The subgroup treatment effects were: early ICA = 0.54, late ICA = 0.60, early M1 = 0.33, late M1 = 0.07, early M2-4 = -0.66, and late M2-4 = -0.35. Results: The traditional design showed a treatment benefit in 31% of simulations. The adaptive design showed benefit in 91%, failed to show benefit after enrollment of the maximum sample in 1%, and stopped early for futility in 8% of simulations. The adaptive trial stopped early for benefit in 84% of simulations. Due to early stopping, the mean number of patients randomized is 590±140 with the adaptive design vs 900 with a traditional design. Of the adaptive trial simulations that showed benefit, 91% occur after subgroup selection. The subgroup selected most often (31% of all simulations) includes early and late ICA patients. Conclusions: A trial with adaptive subgroup selection can efficiently test the effect of endovascular stroke treatment. Simulations suggest that with this design, IMS III would have 91% power and would typically stop early after interim analysis shows benefit in a patient subgroup.