Age-related nuances in knowledge assessment - A hierarchy of knowledge
We published a new paper entitled “Age-related nuances in knowledge assessment” in Intelligence. I really like this paper because it deals with on the way we assess, model, and understand knowledge. And, btw, it employs machine learning methods. Thus, both in terms of content and methodology it hopefully sets a stage for future research avenues that are promising to follow up on. I would like to cover some of the key findings in a series of blog posts. This first post deals with knowledge at different levels of granularity, how they relate to age, and the recurring finding that item sampling plays an important role in test compilation.
In personality research, the idea of a hierarchical order is prevalent across different theoretical considerations: Both the Five Factor Model (e.g., Costa & McCrae, 1992) and the HEXACO model (Ashton & Lee, 2007) describe on the highest level broad trait domains. Below the traits are different facets (e.g., Neuroticism encompasses the facets Anxiety, Vulnerability, Depression). Recently, a more fine-grained level below the facets has been surmised, which have been labeled personality “nuances” (McCrae, 2015). In most operationalizations the nuances are identical to the personality items (Mõttus et al., 2017; Seeboth & Mõttus, 2018).
We have transferred these ideas about the structure of personality to the field of intelligence research, more precisely, knowledge assessment by proposing the following levels of a knowledge hierarchy: At the apex declarative knowledge, which is according to Cattell’s definition the most important part of crystallized intelligence (gc, Cattell, 1987; Schipolowski et al., 2014). At subordinate levels are broad knowledge areas (e.g., humanities, natural sciences, social sciences), knowledge domains (e.g., chemistry, medicine, art), and – at the lowest level – nodes or nuances. Of course, this hierarchy is intended as a heuristic representing larger, more distinct units on an otherwise dimensional continuum. One can easily think of meso-levels within this hierarchy such as subdomains below the domain level (e.g., organic, inorganic, and physical chemistry) or disciplines below the subdomain level (e.g., polymers, aromatic compounds, amines as components of organic chemistry).
One of the research questions was at which level of the knowledge hierarchy age differences are most pronounced. Thus, we used knowledge at the different levels to predict age. Choosing chronological age as the dependent variable may seem odd, given causality would usually be conceptualize to “flow” in the opposite direction. However, a prediction model is not necessarily a causal model (see also Mõttus & Rozgonjuk, 2019, for a same line of reasoning), and can be seen as a mere statistical abstraction. Thus, age is a proxy for events that are associated with knowledge acquisition. The results are unequivocal: Using elastic net regressions we found the majority of age variance to be located at the item level (R² = .54), in comparison to the domain level (R² = .28), the area level (R² = .24), or the top-level gc (R² = .02). Apparently, declarative knowledge measures have substantial item uniqueness, which conveys important age-related information.
This study demonstrates that item sampling in knowledge assessment is essential and should not be neglected (see also Schroeders et al., 2016). While the assumption of interchangeable items might hold for fluid intelligence tests (e.g., Matrices), it does make a difference which items are compiled to a knowledge scale. We conclude that, unfortunately, such construct-relevant variance at the item level is not only understudied, it is also largely overlooked when considering aggregate knowledge scores. Thus, as associations that are specific to a particular level in the hierarchy (e.g., items) should not be generalized to aggregate levels (e.g., domains), without taking appropriate account of item sampling issues (cf. Mõttus, 2016).
tl;dr: Knowledge nuances AKA items do matter.
- Ashton, M. C., & Lee, K. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11(2), 150–166. https://doi.org/10.1177/1088868306294907.
- Cattell, R. B. (1987). Intelligence: Its structure, growth and action. North Holland.
- Costa, P. T., & McCrae, R. R. (1992). The revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI) professional manual. Psychological Assessment Resources.
- McCrae, R. R. (2015). A more nuanced view of reliability: Specificity in the trait hierarchy. Personality and Social Psychology Review, 19(2), 97–112. https://doi.org/10.1177/1088868314541857
- Mõttus, R. (2016). Towards more rigorous personality trait-outcome research: Personality traits and outcomes. European Journal of Personality, 30(4), 292–303. https://doi.org/10.1002/per.2041
- Mõttus, R., & Rozgonjuk, D. (2019). Development is in the details: Age differences in the Big Five domains, facets, and nuances. Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000276
- Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100
- Schroeders, U., Watrin, L., & Wilhelm, O. (2021). Age-related nuances in knowledge assessment. Intelligence, 85. https://doi.org/10.31234/osf.io/c6bwh
- Schroeders, U., Wilhelm, O., & Olaru, G. (2016). The influence of item sampling on sex differences in knowledge tests. Intelligence, 58(3), 22–32. https://doi.org/10.1016/j.intell.2016.06.003
- Seeboth, A., & Mõttus, R. (2018). Successful explanations start with accurate descriptions: Questionnaire items as personality markers for more accurate predictions: items as personality markers. European Journal of Personality, 32(3), 186–201. https://doi.org/10.1002/per.2147
- Schipolowski, S., Wilhelm, O., & Schroeders, U. (2014). On the nature of crystallized intelligence: The relationship between verbal ability and factual knowledge. Intelligence, 46, 156–168. https://doi.org/10.1016/j.intell.2014.05.014