Evaluating prediction markets for internal control applications

Abstract

This study estimates the suitability of prediction markets (as instruments of internal control) by analyzing their event and progress sensitivity based on comprehensive experimental data. The underlying experiment was designed using expected average grades and closely observing students' (rank and file) and teachers' (leadership) behaviors. First, a kick-off study asked both parties about the factors that may influence the outcome of the teaching process. The resulting data served as the basis for a detailed tracking survey of four topic-specific indicators. Each week of the experiment, both students and teachers were asked to report on these indicators for 12 courses, thus producing two sets of 48 time-series data. Concurrently, 12 prediction markets were set up, in which the participants could buy and sell their forecasts 24 hours a day, seven days a week. The market utilized just one type of variable (i.e., the final average grade of each course) and the best traders could earn special commendations. The students were also encouraged to keep a trading journal that included the motives for their purchases and sales. For some courses, fake information was fed to the students in order to understand how it would influence the survey indicators, the prediction market prices, and their sensitivity to misinformation. Finally, this study reveals the results of these two extremely different control mechanisms (i.e., periodic detailed surveys vs. voluntary single-number prediction markets) and reports on how they performed and how fast their indications were adapted.