Tagged "crystallized intelligence"

Technology-Based Assessment

In the last decades, the digitalization of educational content, the integration of computers in different educational settings and the opportunity to connect knowledge and people via the Internet has led to fundamental changes in the way we gather, process, and evaluate information. Also, more and more tablet PCs or notebooks are used in schools and—in comparison to traditional sources of information such as text books—the Internet seems to be more appealing, versatile, and accessible. Technology-based assessment has been concerned with questions of comparability of test scores across test media, transferring already existing measurement instruments to digital devices. Nowadays, researchers are more interested in enriching the assessment by using interactive tasks and video material or make the testing more efficient using digital behavior traces.

A 120 item gc test

This is a 120 item measure of crystallized intelligence (gc), more precisely, declarative knowledge. Based on previous findings concerning the dimensionality of gc (Steger et al., 2019), we sampled items from four broad knowledge areas - humanities, life sciences, natural sciences, and social sciences. Each knowledge area contained three domains with ten items each, resulting in a total of 120 items. Items were selected to have a wide range of difficulty and to broadly and deeply cover the content domain. Items are available in German and English. The development of the inital item pool is detailed in Steger et al. (2019). We used and described the 120 item gc measure in two recent publications (Schroeders et al., 2021; Watrin et al., 2021). The items can be found in the accompanying OSF project.

This is the third post in a series on a paper — “Age-related nuances in knowledge assessment” — we recently published in Intelligence. The first post reflected on how knowledge is organized, the second post dealt with psychometric issues. This post is going to be more mathematical (yes, there will be some formulae) and it will be a cautionary note on the use of machine learning algorithms. Machine learning algorithms have positively influenced research in various scientific disciplines such as astrophysics, genetics, or medicine. Also, subdisciplines in psychology such as personality science (e.g., Stachl et al., 2020) or clinical research (Cearns et al., 2019) are adapting the new statistical tools. However, as pointed out in my research statement, every new method initially bears the risk of applying new techniques without the necessary background knowledge. I mainly blame statistical and methodological courses in psychology studies for this. We really have to teach math, stats, and methods more rigorously in university teaching, especially in structured PhD programs.

This is the second post in a series on a recent paper entitled “Age-related nuances in knowledge assessment” that we wrote with Luc Watrin and Oliver Wilhelm. The first post dealt with the way how we conceptualize the organization of knowledge in a hierarchy in a multidimensional knowledge space. The second post reflects on the way we measure or model knowledge. In textbooks knowledge assessments have a special standing, because they can be modeled both from a reflective and a formative perspective.

We published a new paper entitled “Age-related nuances in knowledge assessment” in Intelligence. I really like this paper because it deals with on the way we assess, model, and understand knowledge. And, btw, it employs machine learning methods. Thus, both in terms of content and methodology it hopefully sets a stage for future research avenues that are promising to follow up on. I would like to cover some of the key findings in a series of blog posts. This first post deals with knowledge at different levels of granularity, how they relate to age, and the recurring finding that item sampling plays an important role in test compilation.