Problems Associated with the Assessment of Experiential Learning Using the Multiple Choice Test

Abstract

"The multiple choice test medium was used in a before- and after test format to assess the degree of learning from a business simulation game. During the test quarter, an experimental group participated in a multinational game and heard three 2-hour lectures on international business topics. The control group played a domestic business game and heard the same lectures. Consequently, the test data attempt to compare the amount of international business knowledge acquired under these two test conditions: simulation plus lecture and lecture only. The before-and-after difference between groups was not statistically significant as measured by the number of correct responses, yet side by side open- ended questions recorded substantial between-group differences. To explain this apparent weakness in the multiple choice test, an in-depth analysis of the test’s validity was undertaken. The analysis, using multivariate techniques, shows strong evidence of learning in spite of the lack of statistical significance. This leads to doubt concerning the suitability of the highly structured multiple choice format for experiential learning assessment and, further, indicated that care should be taken in hypothesis testing using parametric statistics. "