Volume 112, February 2023, 102740Author links open overlay panelAbstract

The basic local independence model (BLIM) is the standard probabilistic model in knowledge structure theory. It assumes that the probability of a correct response to a problem is constant for all individuals that master the problem, and accordingly, for all individuals that do not master it, irrespective of the mastering of other problems. Recently published data on the problem correct rate as inferred from a response-based assessment of the mastering of the problem seem to contradict this assumption. The analysis presented in this paper, however, reveals that deviations from constancy in the observed direction are to be expected under the BLIM. They are mainly due to the inaccuracy inherent in any response-based assessment. The implications of these results for the empirical validation of the BLIM are discussed.

Introduction

Knowledge structures provide a highly flexible set-theoretic framework for representing the organization of knowledge elements in a domain. Although knowledge structures were first introduced by Doignon and Falmagne (1985) within a purely deterministic approach, this perspective was broadened later on by developing a probabilistic framework (Falmagne and Doignon, 1988a, Falmagne and Doignon, 1988b). The so-called basic local independence model (BLIM; Doignon & Falmagne, 1999) forms the standard probabilistic model in knowledge structure theory (KST). The BLIM makes strong assumptions that help limiting the number of parameters and keep it tractable, even in large scale applications (Falmagne et al., 2006). In particular, it assumes that the probability of a correct response to a problem is constant for all individuals that master the problem, and accordingly, for all individuals that do not master it, irrespective of the mastering of other problems. These assumptions have recently been put under scrutiny, and Doble et al. (2019) and Cosyn et al. (2021) present data that seem to provide evidence against them. Their results are reviewed after introducing the relevant KST notions. The subsequent sections then consider various scenarios of the response-based assessment of the mastering of the problems, and in each case derive the problem correct rate predicted by the BLIM as the data generating model. This analysis reveals that deviations from constancy in the observed direction are to be expected under the BLIM. It shows that at least some of the apparent evidence against the BLIM is due to the inaccuracy inherent in any response-based assessment. Finally, the paper outlines a route to validating the BLIM assumptions within the framework of a generalized local independence model.

Section snippetsBasic notions

Doignon and Falmagne (1985) define a knowledge structure as a pair (Q,K) in which Q is a nonempty set (assumed to be finite throughout the paper), and K is a family of subsets of Q, containing at least Q and the empty set 0̸. The set Q is called the domain of the knowledge structure. Its elements are referred to as problems or items and the subsets in the family K are labeled (knowledge) states. A knowledge state represents the subset of problems in the considered domain that an individual

Empirical evidence on the problem correct rate

As already mentioned above, no rationale is provided for the assumption that the correct rate of problem q is constant for all individuals that master the problem, and accordingly, for all individuals that do not master it. Cosyn et al. (2021) refer to the first situation as an in-state problem, and to the second situation as an out-of-state problem. Intuitively, however, a careless error on an in-state problem may be more likely to occur if an individual does not know much beyond what is

Assessment-based correct rate

In order to derive theoretical predictions of the problem correct rate when drawing upon a response-based assessment, let us assume a BLIM defined on a learning space K on the domain Q with probability distribution π on K, and response probabilities βq,ηq for all q∈Q. This amounts to the problem correct rate given the true state K∈K as specified in (4), which means that ρ(q∣K)=1−βq for in-state problems q irrespective of the layer relative to K, and ρ(q∣K)=ηq for out-of-state problems q, again

Conclusions

The present study is motivated by recently published data on the extra problem correct rate plotted as a function of the layer of the problem relative to the assessed state (Cosyn et al., 2021, Doble et al., 2019). These data (illustrated schematically in Fig. 2) seem to contradict the constancy of the correct rate for all in-state problems and out-of-state problems, respectively, as assumed in the basic local independence model (BLIM). The previous sections consider various scenarios of the

Acknowledgments

I am grateful to Alice Maurer for her valuable comments on a previous version of this paper.

References (18)

There are more references available in the full text version of this article.

View full text

© 2022 Elsevier Inc. All rights reserved.