Technology-based assessment

technologyIn the last decades, the digitalization of educational content, the integration of computers in different educational settings and the opportunity to connect knowledge and people via the Internet has led to fundamental changes in the way we gather, process, and evaluate information. Also, more and more tablet PCs or notebooks are used in schools and—in comparison to traditional sources of information such as text books—the Internet seems to be more appealing, versatile, and accessible. Technology-based assessment can be defined by two research perspectives: On the one hand, researchers are concerned to transfer already existing measurement instruments to digital devices that are comparable to the original. On the other hand, researchers use interactive tasks and video material to enrich the assessment or make it more efficient. My research in this area is aligned along this dichotomy of comparability of test data gathered on different devices vs. enrichment of the assessment. For example, colleagues and I try to answer fundamental questions about declarative knowledge with a smartphone App: How many dimensions of knowledge have to be distinguished (e.g., humanities vs. natural sciences)? Does knowledge differentiate with age? Is it possible to predict cheating behavior based on auxiliary data such as reaction times and page defocusing events?

Selection of relevant publications

  • Steger, D., Schroeders, U., & Wilhelm, O. (2019). On the dimensionality of crystallized intelligence: A smartphone-based assessment. Intelligence, 72, 76–85.
  • Steger, D., Schroeders, U., & Gnambs, T. (in press). A meta-analysis of test scores in proctored and unproctored ability assessment. European Journal of Psychological Assessment.
  • Moehring, A., Schroeders, U., Leichtmann, B., & Wilhelm, O. (2016). Ecological momentary assessment of digital literacy: Influence of fluid and crystallized intelligence, domain-specific knowledge, and computer usage. Intelligence, 59, 170–180.
  • Schroeders, U., Bucholtz, N., Formazin, M., & Wilhelm, O. (2013). Modality specificity of comprehension abilities in the sciences. European Journal of Psychological Assessment, 29, 3–11.

Measurement of cognitive abilities & educational competencies

strukturIn addition to a good theoretical foundation, efficient and meaningful instruments are necessary to describe and predict educational competencies and general cognitive performance. Together with other researchers I developed a computer-based test for the diagnosis of dyscalculia in elementary school (TeDDy-PC) and the Berlin test of fluid and crystallized intelligence (BEFKI) in secondary school. Although these measures assess rather different competencies at different points in the educational biography, their methodological approaches are similar: A theory-based definition and operationalization of the construct is followed by an iterative process of item development and data-based revisions. The aim is to develop psychometrically sound measurement instruments which are a prerequisite for identifying variables that influence and facilitate learning. For example, we examined in the German National Assessment Study in Mathematics and Science 2012 the influence of instruction time on science achievement taking into account student characteristics (socioeconomic status, migration background, etc.) and school form differences (academic vs. non-academic track schools). Our analyses showed that simply more time in school does not necessarily correlate with better performance in the natural sciences.

Selection of relevant publications

  • Schroeders, U., Schipolowski, S., Zettler, I., Golle, J., & Wilhelm, O. (2016). Do the smart get smarter? Development of fluid and crystallized intelligence in 3rd Grade. Intelligence, 59, 84-95.
  • Schroeders, U., Schipolowski, S., & Wilhelm, O. (2015). Age-related changes in the mean and covariance structure of fluid and crystallized intelligence in childhood and adolescence. Intelligence, 48, 15–29.
  • Wilhelm, O., Schroeders, U., & Schipolowski, S. (2014). Berliner Test zur Erfassung fluider und kristalliner Intelligenz für die 8. bis 10. Jahrgangsstufe [Berlin Test of fluid and crystallized intelligence for grades 8-10]. Göttingen: Hogrefe.
  • Schroeders, U., Siegle, T., Weirich, S., & Pant, H. A. (2013). Der Einfluss von Kontext-und Schülermerkmalen auf die naturwissenschaftlichen Kompetenzen. In H. A. Pant, P. Stanat, U. Schroeders, A. Roppelt, T. Siegle, & C. Pöhlmann (Eds.), IQB-Ländervergleich 2012. Mathematische und naturwissenschaftliche Kompetenzen am Ende der Sekundarstufe I (pp. 331–346). Münster: Waxmann.


psychometrieEducational and survey research offer special methodical and diagnostic challenges. From a methodological perspective, complex multiple matrix test designs, hierarchical data structures and missing data points partially require their own instruments to ensure the validity of findings and the comparability of performances on different levels of aggregation. Therefore, my current publications deal with psychometric research questions. One example is the application of so-called locally weighted structural equation models as a new method to model covariance structures in dependence of a continuous context variable. These models turned out to be especially useful in describing the development of skills and abilities, because age can be handled as continuous variable without artificial categorization into different age groups. Another example, is the development of psychometrically sound and efficient short scales that are especially needed in large-scale assessment. Thus, we compared the quality and efficiency of different strategies to construct short scales and demonstrated that so-called meta-heuristics outperform traditional strategies of item selection. Such meta-heuristics include, for example, Ant Colony Optimization algorithms which mimic the foraging behavior of ants, and can also be applied to psychological settings due to their great flexibility.

Selection of relevant publications

  • Olaru, G., Schroeders, U., Hartung, J., & Wilhelm, O. (2019). Ant colony optimization and local weighted structural equation modeling. A tutorial on novel item and person sampling procedures for personality research. European Journal of Personality, 33, 400-419.
  • Olaru, G., Schroeders, U., Wilhelm, O., & Ostendorf, F. (2018). A confirmatory examination of age-associated personality differences: Deriving age-related measurement invariant solutions using Ant Colony Optimization. Journal of Personality, 86, 1037–1049.
  • Schroeders, U., Wilhelm, O., & Olaru, G. (2016). Meta-heuristics in short scale construction: Ant Colony Optimization and Genetic Algorithm. PLOS ONE, 11: e0167110.
  • Schroeders, U., Wilhelm, O., & Olaru, G. (2016). The influence of item sampling on sex differences in knowledge tests. Intelligence, 58, 22–32.
  • Schroeders, U., Schipolowski, S., & Wilhelm, O. (2015). Age-related changes in the mean and covariance structure of fluid and crystallized intelligence in childhood and adolescence. Intelligence, 48, 15–29.