In the last decades, the digitalization of educational content, the integration of computers in different educational settings and the opportunity to connect knowledge and people via the Internet has led to fundamental changes in the way we gather, process, and evaluate information. Also, more and more tablet PCs or notebooks are used in schools and—in comparison to traditional sources of information such as text books—the Internet seems to be more appealing, versatile, and accessible. Technology-based assessment can be defined by two research perspectives: On the one hand, researchers are concerned to transfer already existing measurement instruments to digital devices that are comparable to the original. On the other hand, researchers use interactive tasks and video material to enrich the assessment or make it more efficient. My research in this area is aligned along this dichotomy of comparability of test data gathered on different devices vs. enrichment of the assessment. For example, colleagues and I try to answer fundamental questions about declarative knowledge with a smartphone App: How many dimensions of knowledge have to be distinguished (e.g., humanities vs. natural sciences)? Does knowledge differentiate with age? Is it possible to predict cheating behavior based on auxiliary data such as reaction times and page defocusing events?
Measurement of cognitive abilities & educational competencies
In addition to a good theoretical foundation, efficient and meaningful instruments are necessary to describe and predict educational competencies and general cognitive performance. Together with other researchers I developed a computer-based test for the diagnosis of dyscalculia in elementary school (TeDDy-PC) and the Berlin test of fluid and crystallized intelligence (BEFKI) in secondary school. Although these measures assess rather different competencies at different points in the educational biography, their methodological approaches are similar: A theory-based definition and operationalization of the construct is followed by an iterative process of item development and data-based revisions. The aim is to develop psychometrically sound measurement instruments which are a prerequisite for identifying variables that influence and facilitate learning. For example, we examined in the German National Assessment Study in Mathematics and Science 2012 the influence of instruction time on science achievement taking into account student characteristics (socioeconomic status, migration background, etc.) and school form differences (academic vs. non-academic track schools). Our analyses showed that simply more time in school does not necessarily correlate with better performance in the natural sciences.
Educational and survey research offer special methodical and diagnostic challenges. From a methodological perspective, complex multiple matrix test designs, hierarchical data structures and missing data points partially require their own instruments to ensure the validity of findings and the comparability of performances on different levels of aggregation. Therefore, my current publications deal with psychometric research questions. One example is the application of so-called locally weighted structural equation models as a new method to model covariance structures in dependence of a continuous context variable. These models turned out to be especially useful in describing the development of skills and abilities, because age can be handled as continuous variable without artificial categorization into different age groups. Another example, is the development of psychometrically sound and efficient short scales that are especially needed in large-scale assessment. Thus, we compared the quality and efficiency of different strategies to construct short scales and demonstrated that so-called meta-heuristics outperform traditional strategies of item selection. Such meta-heuristics include, for example, Ant Colony Optimization algorithms which mimic the foraging behavior of ants, and can also be applied to psychological settings due to their great flexibility.