scholarly journals Conformal inference of counterfactuals and individual treatment effects

2021 ◽  
Author(s):  
Lihua Lei ◽  
Emmanuel J. Candès
Keyword(s):  
Treatment Effects ◽  
Individual Treatment

An evaluation of estimating and indexing methods to simplify the determination of management treatment effects on raspberry yields

2003 ◽  
Vol 83 (1) ◽  
pp. 141-147 ◽  
Author(s):  
C. G. Kowalenko
Keyword(s):  
Treatment Effects ◽  
Correlation Coefficients ◽  
Crop Management ◽  
Rubus Idaeus ◽  
Individual Treatment ◽  
Index Method ◽  
Significant Treatment ◽  
Nutrient Effects ◽  
Biomass N ◽  
Index Methods

The effectiveness of using several proposals to estimate or index yield and size of raspberries as an alternative to picking berries as they ripen was examined in two field plot trials over two seasons at two locations in south coast British Columbia. The evaluation included examination of general correlations of the proposed estimate and index values with fresh picked yield, comparison of the significant nutrient and inter-row management treatment effects on proposed method values with effects on fresh picked yield values, influence of individual cane variability to distinguish significant treatment effects, and the effect of N on plant components used to derive the estimate and index method. Correlation coefficients for all yield estimate and index method values with fresh picked yields were generally good. Crop management treatment effects determined by the estimate and index values, however, were not the same as determined by harvesting the berries as they ripened. This showed that the estimate and index method values were biased relative to picked yield. Cane-to-cane variability within individual treatment plots was sufficiently large that differences between treatments had to be greater than 10 to 15% to be significant at P < 0.05 when five canes were randomly sampled for index component measurements to represent the plants in the plot. The five canes sampled for each plot were 5 to 10% of all the floricanes in the plots of this study. The concentration and biomass N measurements that were possible on the floricane components that were sampled for the index methods showed that management treatments of the two trials of the study could have influenced berry development, and hence contributed to the bias of the estimate and index method values relative to fresh picked yield. Although the estimate and index methods were generally quite well correlated with fresh picked yield, caution is advised when they are used directly as alternatives to fresh picking to evaluate crop management treatment effects on berry yield. Further knowledge about the physiological changes that occur during berry ripening may provide opportunities to improve the estimate and index measurements. Key words: Raspberry, Rubus idaeus L., yield estimate, yield index, nutrient effects, nitrogen effects

scholarly journals Individual Treatment Effects of Budget Balance Rules

2020 ◽  
Vol 20 (274) ◽  
Author(s):  
Francesca Caselli ◽  
Daniel Stoehlker ◽  
Philippe Wingender
Keyword(s):  
Treatment Effects ◽  
National Level ◽  
Specific Treatment ◽  
Average Treatment Effect ◽  
Fiscal Rules ◽  
Individual Treatment ◽  
Significant Heterogeneity ◽  
Budget Balance ◽  
The Government ◽  
Country Specific

This paper investigates the heterogenous effects of budget balance rules on fiscal policy in a large sample of countries. To derive country-specific treatment effects of fiscal rules and conduct inference, we use a Synthetic Difference-in-Differences Method. Our results indicate that countries with a budget balance rule improve their fiscal balance on average by around 3 percent after its introduction. However, our results also illustrate the importance of going beyond the average treatment effect, as it masks significant heterogeneity in the country-specific impact of the rule. We find that countries that would have had large deficits in the absence of the fiscal rule exhibit positive treatment effects, thus reducing their budget deficits. On the other hand, countries with budget surpluses respond to fiscal rules by reducing their budget surplus and moving closer to the numerical target of the rule. Our results also suggest that rules’ design matters: a small overall number of fiscal rules, and the presence of a monitoring process outside the government, especially at the supra-national level, improve significantly the effectiveness of the rules.

scholarly journals A permutation test for assessing the presence of individual differences in treatment effects

2021 ◽  
pp. 096228022110336
Author(s):  
Chi Chang ◽  
Thomas Jaki ◽  
Muhammad Saad Sadiq ◽  
Alena Kuhlemeier ◽  
Daniel Feaster ◽  
...  
Keyword(s):  
Random Forests ◽  
Error Control ◽  
Treatment Effects ◽  
Type I Error ◽  
Permutation Test ◽  
Individual Treatment ◽  
Significant Heterogeneity ◽  
Type I ◽  
Data Set ◽  
Show Evidence

An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

scholarly journals Individualized treatment effects with censored data via fully nonparametric Bayesian accelerated failure time models

Biostatistics ◽  
2018 ◽  
Vol 21 (1) ◽  
pp. 50-68 ◽  
Author(s):  
Nicholas C Henderson ◽  
Thomas A Louis ◽  
Gary L Rosner ◽  
Ravi Varadhan
Keyword(s):  
Treatment Effect ◽  
Treatment Effects ◽  
Failure Time ◽  
Enzyme Inhibitor ◽  
Model Specification ◽  
Accelerated Failure Time Model ◽  
Tuning Parameter ◽  
Individual Treatment ◽  
Accelerated Failure Time ◽  
Accelerated Failure

Summary Individuals often respond differently to identical treatments, and characterizing such variability in treatment response is an important aim in the practice of personalized medicine. In this article, we describe a nonparametric accelerated failure time model that can be used to analyze heterogeneous treatment effects (HTE) when patient outcomes are time-to-event. By utilizing Bayesian additive regression trees and a mean-constrained Dirichlet process mixture model, our approach offers a flexible model for the regression function while placing few restrictions on the baseline hazard. Our nonparametric method leads to natural estimates of individual treatment effect and has the flexibility to address many major goals of HTE assessment. Moreover, our method requires little user input in terms of model specification for treatment covariate interactions or for tuning parameter selection. Our procedure shows strong predictive performance while also exhibiting good frequentist properties in terms of parameter coverage and mitigation of spurious findings of HTE. We illustrate the merits of our proposed approach with a detailed analysis of two large clinical trials (N = 6769) for the prevention and treatment of congestive heart failure using an angiotensin-converting enzyme inhibitor. The analysis revealed considerable evidence for the presence of HTE in both trials as demonstrated by substantial estimated variation in treatment effect and by high proportions of patients exhibiting strong evidence of having treatment effects which differ from the overall treatment effect.

scholarly journals Higher treatment effect after total knee arthroplasty is associated with higher patient satisfaction

2020 ◽  
Author(s):  
Jörg Lützner ◽  
Franziska Beyer ◽  
Klaus-Peter Günther ◽  
Jörg Huber
Keyword(s):  
Total Knee Arthroplasty ◽  
Patient Satisfaction ◽  
Knee Arthroplasty ◽  
Treatment Effect ◽  
Treatment Effects ◽  
Oxford Knee Score ◽  
Individual Treatment ◽  
Total Knee ◽  
The Mean ◽  
One Year

Abstract Purpose The aim of this study was to investigate what influence the treatment effect after total knee arthroplasty (TKA) had on patient satisfaction. Methods Prospective registry data of a University-based arthroplasty centre were used. 582 patients with unilateral bicondylar TKA were analyzed. Treatment effect (TE) was deduced from Oxford Knee Score (OKS) before and one year after surgery. Positive values correspond to improved symptoms (maximum 1.0 reflect no symptoms at all) and negative values correspond to deterioration of symptoms. Satisfaction on a visual-analogue scale from 0 to 10 and the willingness to undergo TKA surgery again was assessed one year after surgery. Results The mean OKS improved from 22.1 before to 36.7 one year after TKA. Treatment effects ranged from 1.0 to –0.62 with a mean TE of 0.56. Taking an individual treatment effect of 0.2 as a cut-off between responder and non-responder, a total of 85.8% would be classified as responder after TKA. The mean satisfaction score with the TKA was 8.1. There was a significant correlation between the individual treatment effect and satisfaction after TKA (p < 0.001). The majority of patients (84.5%) would undergo surgery again. Patients not willing to undergo surgery again or those uncertain about this had lower satisfaction scores, a lower treatment effect and were more often female compared to patients who would undergo surgery again. Conclusion Higher individual treatment effects resulted in higher patient satisfaction and willingness to undergo surgery again. However, some patients with a relatively low treatment effect were highly satisfied, which indicates the need for both information. Level of evidence II.

scholarly journals Estimating individual treatment effects by gradient boosting trees

2019 ◽  
Vol 38 (26) ◽  
pp. 5146-5159 ◽  
Author(s):  
Shonosuke Sugasawa ◽  
Hisashi Noma
Keyword(s):  
Treatment Effects ◽  
Gradient Boosting ◽  
Individual Treatment

scholarly journals Estimating the effect of adjuvant chemo-therapy for colon-cancer using registry data: a method comparison and validation

2020 ◽  
Author(s):  
Lingjie Shen ◽  
Erik Visser ◽  
Hans de Wilt ◽  
Henk Verheul ◽  
Felice van Erning ◽  
...  
Keyword(s):  
Colon Cancer ◽  
Observational Data ◽  
Treatment Effect ◽  
Gold Standard ◽  
Large Scale ◽  
Treatment Effects ◽  
Direct Method ◽  
Registry Data ◽  
Individual Treatment ◽  
Simple Models

Abstract Background: Although randomized controlled trials (RCT) are the gold standard to estimate treatment effects, they are often criticized in terms of generalizability. Observational data might alleviate this problem by being readily available in large quantities. However, observational data are potentially confounded. In this methodological study we use a large-scale RCT as the gold standard to examine the performance of various statistical methods to control for potential confounding in observational data. Methods: In this paper we compare three types of methods that allow researchers to correct for such potential confounding: direct methods, inverse probability weighting (IPW) methods and doubly robust (DR) methods. We uniquely compare estimates obtained from the population-wide Netherlands Cancer Registry (NCR) on colon cancer (n=52621) with estimates obtained from a large-scale RCT. As the RCT differs from the observational data both in its sampling mechanism and in its treatment assignment mechanism, we first resample the NCR data to reflect the distribution in RCT data. Next, we correct for potential confounding using three alternative types of methods and consequentially evaluate their estimates to those obtained in the RCT. Results: We find that while all estimators qualitatively approximate to findings in the RCT, methods that can flexibly model the response (i.e., direct estimation and DR estimation) performed consistently superior to the inverse propensity score method. Subgroup analysis indicates that relatively simple models allow us to properly estimate the treatment effect. However, these simple models do not properly identify heterogeneous treatment effects in stage2 colon cancer. Careful sensitivity analysis using more flexible models demonstrates both the uncertainty and the potential heterogeneous treatment effect in stage2 cancer and provides robust estimation of treatment effect in stage3 cancer. Conclusions: Our results suggest that both the direct method and the DR method, when executed with care, can be used to reliably estimate treatment effects based on registry data. This methodological validation opens the door to more extensive use of registry data for the estimation of (individual) treatment effects. Additionally, our identification of potentially meaningful subgroups of stage2 colon cancer patients who, based on our analysis seem to benefit from chemotherapy, should be further explored.

scholarly journals Applying machine learning to predict real-world individual treatment effects: insights from a virtual patient cohort

2019 ◽  
Vol 26 (10) ◽  
pp. 977-988 ◽  
Author(s):  
Gang Fang ◽  
Izabela E Annis ◽  
Jennifer Elston-Lafata ◽  
Samuel Cykert
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Treatment Effects ◽  
Model Performance ◽  
Virtual Patient ◽  
Gradient Boosting ◽  
Individual Treatment ◽  
Patient Cohort ◽  
Individual Outcome ◽  
The Impact

Abstract Objective We aimed to investigate bias in applying machine learning to predict real-world individual treatment effects. Materials and Methods Using a virtual patient cohort, we simulated real-world healthcare data and applied random forest and gradient boosting classifiers to develop prediction models. Treatment effect was estimated as the difference between the predicted outcomes of a treatment and a control. We evaluated the impact of predictors (ie, treatment predictors [X1], confounders [X2], treatment effects modifiers [X3], and other outcome risk factors [X4]) with known effects on treatment and outcome using real-world data, and outcome imbalance on predicting individual outcome. Using counterfactuals, we evaluated percentage of patients with biased predicted individual treatment effects Results The X4 had relatively more impact on model performance than X2 and X3 did. No effects were observed from X1. Moderate-to-severe outcome imbalance had a significantly negative impact on model performance, particularly among subgroups in which an outcome occurred. Bias in predicting individual treatment effects was significant and persisted even when the models had a 100% accuracy in predicting health outcome. Discussion Inadequate inclusion of the X2, X3, and X4 and moderate-to-severe outcome imbalance may affect model performance in predicting individual outcome and subsequently bias in predicting individual treatment effects. Machine learning models with all features and high performance for predicting individual outcome still yielded biased individual treatment effects. Conclusions Direct application of machine learning might not adequately address bias in predicting individual treatment effects. Further method development is needed to advance machine learning to support individualized treatment selection.

Sign in / Sign up

Export Citation Format

Share Document