Age-related nuances in knowledge assessment - A hierarchy of knowledge

We published a new paper entitled “Age-related nuances in knowledge assessment” in Intelligence. I really like this paper because it deals with on the way we assess, model, and understand knowledge. And, btw, it employs machine learning methods. Thus, both in terms of content and methodology it hopefully sets a stage for future research avenues that are promising to follow up on. I would like to cover some of the key findings in a series of blog posts. This first post deals with knowledge at different levels of granularity, how they relate to age, and the recurring finding that item sampling plays an important role in test compilation.

In personality research, the idea of a hierarchical order is prevalent across different theoretical considerations: Both the Five Factor Model (e.g., Costa & McCrae, 1992) and the HEXACO model (Ashton & Lee, 2007) describe on the highest level broad trait domains. Below the traits are different facets (e.g., Neuroticism encompasses the facets Anxiety, Vulnerability, Depression). Recently, a more fine-grained level below the facets has been surmised, which have been labeled personality “nuances” (McCrae, 2015). In most operationalizations the nuances are identical to the personality items (Mõttus et al., 2017; Seeboth & Mõttus, 2018).

We have transferred these ideas about the structure of personality to the field of intelligence research, more precisely, knowledge assessment by proposing the following levels of a knowledge hierarchy: At the apex declarative knowledge, which is according to Cattell’s definition the most important part of crystallized intelligence (gc, Cattell, 1987; Schipolowski et al., 2014). At subordinate levels are broad knowledge areas (e.g., humanities, natural sciences, social sciences), knowledge domains (e.g., chemistry, medicine, art), and – at the lowest level – nodes or nuances. Of course, this hierarchy is intended as a heuristic representing larger, more distinct units on an otherwise dimensional continuum. One can easily think of meso-levels within this hierarchy such as subdomains below the domain level (e.g., organic, inorganic, and physical chemistry) or disciplines below the subdomain level (e.g., polymers, aromatic compounds, amines as components of organic chemistry).

One of the research questions was at which level of the knowledge hierarchy age differences are most pronounced. Thus, we used knowledge at the different levels to predict age. Choosing chronological age as the dependent variable may seem odd, given causality would usually be conceptualize to “flow” in the opposite direction. However, a prediction model is not necessarily a causal model (see also Mõttus & Rozgonjuk, 2019, for a same line of reasoning), and can be seen as a mere statistical abstraction. Thus, age is a proxy for events that are associated with knowledge acquisition. The results are unequivocal: Using elastic net regressions we found the majority of age variance to be located at the item level (R² = .54), in comparison to the domain level (R² = .28), the area level (R² = .24), or the top-level gc (R² = .02). Apparently, declarative knowledge measures have substantial item uniqueness, which conveys important age-related information.

This study demonstrates that item sampling in knowledge assessment is essential and should not be neglected (see also Schroeders et al., 2016). While the assumption of interchangeable items might hold for fluid intelligence tests (e.g., Matrices), it does make a difference which items are compiled to a knowledge scale. We conclude that, unfortunately, such construct-relevant variance at the item level is not only understudied, it is also largely overlooked when considering aggregate knowledge scores. Thus, as associations that are specific to a particular level in the hierarchy (e.g., items) should not be generalized to aggregate levels (e.g., domains), without taking appropriate account of item sampling issues (cf. Mõttus, 2016).

tl;dr: Knowledge nuances AKA items do matter.

References

#crystallized intelligence #item sampling #nuances #psychometrics #machine learning