Estimating the statistical power to detect set size effects in contralateral delay activity
Published in Psychophysiology, 2021
Recommended citation: Ngiam, W. X. Q., Adam, K. C., Quirk, C., Vogel, E. K., & Awh, E. (2021). Estimating the statistical power to detect set‐size effects in contralateral delay activity. Psychophysiology, 58(5), e13791. https://doi.org/10.1111/psyp.13791
Abstract
The contralateral delay activity (CDA) is an event-related potential component commonly used to examine the online processes of visual working memory. Here, we provide a robust analysis of the statistical power that is needed to achieve reliable and reproducible results with the CDA. Using two very large EEG datasets that examined the contrast between CDA amplitude with set sizes 2 and 6 items (Unsworth et al., 2015) and set sizes 2 and 4 items (Hakim et al., 2019), we present a subsampling analysis that estimates the statistical power achieved with varying numbers of subjects and trials based on the proportion of significant tests in 10,000 iterations. We also generated simulated data using Bayesian multilevel modelling to estimate power beyond the bounds of the original datasets. The number of trials and subjects required depends critically on the effect size. Detecting the presence of the CDA – a reliable difference between contralateral and ipsilateral electrodes during the memory period – required only 30-50 clean trials with a sample of 25 subjects to achieve approximately 80% statistical power. However, for detecting a difference in CDA amplitude between two set sizes, a substantially larger number of trials and subjects was required; approximately 400 clean trials with 25 subjects to achieve 80% power. Thus, to achieve robust tests of how CDA activity differs across conditions, it is essential to be mindful of the estimated effect size. We recommend researchers designing experiments to detect set size differences in the CDA collect substantially more trials per subject.