APriot: A computer program for Monte-Carlo simulations of cumulated error probabilities in ANOVA
When conducting a psychological experiment it often is tempting to intermediately inspect data and then to decide whether to add further participants or to end the study. But we all know that intermediately inspecting statistical data is a bad research practice since there is an accumulation of alpha.
Interstingly, in medical research, intermediately inspecting the data of a running experiment is a widely spread practice. This is comprehensible since ethical reasons often forbid a new drug to be tested for a longer time than necessary. The test procedure is called "group-sequential testing" and is based on the idea that alpha can be adjusted so that the cumulated alpha error probability for multiple inspections does not exceed a predefined value, for example, .05.
In psychology, sequential testing is widely unknown. One reason may be that the sequential tests known from medical research focus on statistical tests based on the normal distribution. This is suitable for testing the effectiveness of a new drug; there is one group of participants receiving the new drug and a control group receiving the older drug. Within classical statistics, a t-test would be conducted. With the number of participants being not too small, the t-distribution approximates the normal distribution and thus, a group-sequential test can be applied to correct alpha for multiple inspections of the data.
In psychological research, the analysis of variance (ANOVA) is very common since it allows for testing more complex experimental designs with multiple variables and interactions between those variables. APriot can conduct Monte-Carlo simulations of the ANOVA and simulate the effects of intermediately inspecting data. APriot can simulate all effects of the ANOVA including repeated measures and interactions of any order.
I have tried to make APriot as easy to use as possible. There is no need for the user to enter complex simulation parameters. Instead, the user enters the data of an earlier study and APriot computes all parameters needed for conducting a simulation.
I hope that APriot will help you to detect effects in a more economical way without the shortcoming of an inflated alpha error probability.
Screenshots (click to enlarge)
APriot is distributed with a user manual in which all steps necessary to conduct a simulation are explained. You can select the manual within APriot by choosing '? -> APriot user manual'.
Alternatively, you can download the manual here.
If you use APriot for your research, I would appreciate your including the following reference to APriot in the papers in which you publish your results:
Lang, A.-G. (in press). Is intermediately inspecting statistical data necessarily a bad research practice? The Quantitative Methods for Psychology. Download PDF
APriot is compatible with Windows 7 - 10. There are 32 and 64 bits versions.
Download and install
- APriot is free for everyone. Commercial distribution is strictly prohibited.
- APriot is distributed from this website. If you wish to distribute APriot in some other way, then you need to seek permission from the author. Please send me an e-mail in which you specify how and for what purpose you intend to distribute APriot.
- You may use screenshots of APriot without asking for permission.
- Considerable effort has been put into program development and evaluation, but there is no warranty whatsoever.
If you find a problem with APriot please send a bug report to Albert-Georg Lang.