Randomized Controlled Trials (RCTs) are considered as the gold standard for treatment effect assessment in biomedical studies. However, their recruitment criteria are often narrow and their sample sizes are often underpowered for the guidance of personalized treatment decisions. On the other hand, observational studies recruit diverse population and large sample sizes, but are prone to various biases. To safely leverage the strengths of observational studies, we study the problem of falsification, where RCTs are used to validate causal effect estimates learned from observational data. We show that, given data from both an RCT and an observational study, assumptions on the internal and external validity of the observational study imply a set of testable Conditional Moment Restrictions (CMRs) on the difference of causal contrast signals between the RCT and the observational study. A kernel-based test on the set of CMRs can then be constructed, with an easy-to-compute test statistic that has closed-form asymptotic distributions. We demonstrate that our approach has adequate type I error and power superior to other falsification methods using semi-synthetic and real-world datasets. We also show that our approach allows practitioners to visualize the subgroups in the population that lead to falsification of the observational study, which facilitates further investigation on the cause of validity violation and decision on future evidence synthesis.

Please click here for participating the talk online.