Towards a competence-based polytomous knowledge structure theory

The aim of the present article is to lay out the foundations of an axiomatic theory of attribute maps. These last can be regarded as an “upgrade” of skill maps (Düntsch & Gediga, 1995) to the case of polytomous items and dichotomous skills. The theory developed here can be regarded as an extension of the so-called competence-based knowledge structure theory (CbKST; Doignon, 1994, Düntsch and Gediga, 1995, Falmagne et al., 1990, Gediga and Düntsch, 2002, Heller, Augustin, et al., 2013, Heller et al., 2015, Heller, Ünlü, and Albert, 2013, Korossy, 1997, Korossy, 1999).

When it was initially formulated in 1985, knowledge space theory (KST; Doignon and Falmagne, 1985, Doignon and Falmagne, 1999, Falmagne and Doignon, 2011, Falmagne et al., 1990) was aimed at non-numerically assessing the knowledge of individuals through a set of dichotomous items. In this framework, the knowledge state of an individual was intended as the specific set of items about a certain knowledge domain that an individual masters. Both a deterministic and a probabilistic framework were developed to this aim. The former was used to establish the properties of specific kinds of collections of knowledge states, called knowledge structures, and to provide methods for building such structures. Different properties of knowledge structures (e.g., closure under set union of the collection of states) were used to define different kinds of structures (e.g., knowledge spaces), each of which presented specific interpretations from the learning perspective (e.g., the mastering of an item can be achieved through different learning paths). From a probabilistic point of view, the basic local independence model (BLIM; Falmagne and Doignon, 1988a, Falmagne and Doignon, 1988b) was developed to provide the deterministic part of the theory with a probabilistic reference model. Parameters of this model are, for each item, lucky guess and careless error probabilities, and the probability of each knowledge state in the population. The deterministic and probabilistic components of KST lead to the development and implementation of adaptive assessment algorithms used to discover the knowledge state of individuals. Some examples of the practical implementation of these algorithms are the ALEKS (www.aleks.com, see e.g., Falmagne, Albert, Doble, Eppstein, & Hu, 2013) and the StatKnowlab (Anselmi et al., 2021, de Chiusole et al., 2020) platforms.

The study of the cognitive processes or abilities implied in items’ solution was not considered in the first phase of KST development. Later on, starting from the beginning of the 90s, the study and formal definition of the connections between the skills needed to solve the items and the observed answering behavior of individuals lead to the first definition of the CbKST. This cognitive development of the theory introduced the distinction between the knowledge state at the performance level, intended as the set of items mastered by an individual, and the competence state, intended as the set of skills that the individual must possess to solve a set of items. The mapping that assigns to each item the set of skills needed to solve it is the skill map. In CbKST, the link between the skills and the solution of each item is usually modeled with the conjunctive model, the disjunctive model, or the competency model. In the conjunctive model all the skills assigned to an item must be possessed by an individual to solve it; in the disjunctive model each one of the skills assigned to the item is sufficient for an individual to solve it; finally, in the competency model an item can be solved using one of different sets of skills. Starting from the skill map and via one of the listed models it is possible to delineate the knowledge states existing in the population. The methods of construction of a skill map include both theory-driven (Düntsch & Gediga, 1995) and data-driven (de Chiusole et al., 2020, de Chiusole et al., 2017, Spoto et al., 2016) procedures.

What remained unchanged between KST and CbKST is the dichotomous nature of both items and skills: each item solution is dichotomously classified as “right” or “wrong”, each skill can be possessed or not by an individual, and each skill can be needed or not to solve an item.

Dichotomous items are well suited for the aim of assessing knowledge. Nonetheless, the possibility of extending KST beyond the dichotomous case has been studied since the paper by Schrepp (1997). Polytomous items offer the possibility of not only enriching the assessment of knowledge, but also extending the KST approach to other fields, such as psychological, personality, and attitudes assessment. Despite the potential importance of this extension, after the work by Schrepp many years passed before some further proposals were advanced in this direction.

The first contribution was proposed by Bartl and Belohlavek (2011), and further developed by Zhou, Li, Wang, and Sun (2022). Both these contributions referred to the application of fuzzy set theory to the definition of knowledge structures in which the items had a non dichotomous answering format.

The other contributions were more adherent to the classical KST approach and to the work by Schrepp. Heller (2021) proposed a polytomous extension of the deterministic part of the theory and of the structure construction methods by showing how the procedures adopted in dichotomous KST can be generalized to the polytomous case. Along the lines defined by Schrepp (1997), and in even more strict connection to Schrepp’s work than Heller, the polytomous extension of the deterministic components of KST was proposed by Stefanutti, Anselmi, de Chiusole, and Spoto (2020). This last work led also to the generalization of the probabilistic framework of KST by means of the polytomous local independence model (PoLIM; Stefanutti, de Chiusole, Anselmi, & Spoto, 2020), a model that addresses items with more than two answer alternatives.

Moreover, Stefanutti, de Chiusole, Gondan, and Maurer (2020) proposed both a deterministic and a probabilistic framework for the polytomous extension of KST aimed at modeling misconceptions. This extension appears to be particularly useful when knowledge is assessed, since it allows to model the different kinds of “wrong” answers and relate them to particular bugs in the learning process of the individual.

In sum, while the polytomous extension of KST has been addressed from both the deterministic and probabilistic perspectives (even if many points, such as model identifiability, remain to be studied yet), the polytomous extension of CbKST has not been sufficiently explored yet. The investigation in this field appears of great relevance for future applications of KST to many different research fields.

In order to contribute to the progress in the study of the delineation of a polytomous structure from a skill map, the present article aims at detecting and formalizing a set of axioms and conditions that ensure the logical coherence and the compatibility between the skill assignment defined by the skill map and the possibly delineated polytomous structure.

The article is structured as follows. Section 2 provides the necessary backgrounds about CbKST and a polytomous extension of KST. The main contribution of the article is found in Section 3, where an extension of CbKST to polytomous items is developed on the basis of a set of four axioms (necessary and sufficient conditions) for consistently deriving a polytomous performance structure from a set of potentially non-independent attributes by a conjunctive model. Special compatibility conditions arise between the “attribute map” (an extension of the notion of a skill map to polytomous items) and the underlying structure on the set of attributes, which are illustrated and discussed in the section. A variety of examples of application of the theory are offered in Section 4. Various cases are covered in the section, including a “quasi-nominal case”, a “partial order case”, and a “linear order case”. Section 5 provides some final remarks and closes the article.

留言 (0)

沒有登入
gif