Reading
-------

For the last class, I'd like you to read some material on the class of
psychometric models known as "cognitive diagnosis models" (CDMs) or
sometimes "diagnostic classification models (DCMs; same thing).



** Miscellaneous background:

These models are basically two-layer Bayes nets with hidden nodes,
that allow us to enumerate the KCs necessary to solve particular
tasks, and allow us to make inferences about whether individuals do or
don't possess those KCs based on their patterns of performance on
specific sets of tasks.  (We can also make inferences about how good
or bad the tasks are at eliciting information about the KCs, which is
also a useful psychometric activity.)

The models are closely related to knowledge tracing models as
discussed in 

  Corbett, A. T., Anderson, J. R. and O'Brien, A. T. (1995). Student
  modeling in the ACT programming tutor. Chapter 2 in Nichols, P. D.,
  Chipman, S. F. and Brennan, R. L. (eds.) (1995).  Cognitively
  diagnostic assessment. Hillsdale, NJ: Lawrence Erlbaum Associates.

CDMs are distinct from KT of the type described in this article mainly
in that they allow for making inferences abour more than one skill at
a time, from individual task performance. (of course several task
performances must be looked at together to reduce the SE of the
inference, as usual [more tasks --> more reliability!]).

A paper that discusses these ideas fairly carefully, before they
caught on in psychometrics, is

  VanLehn, K., Niu, Z., Siler, S. and Gertner, A. (1998). Student
  modeling from conventional test data: a Bayesian approach without
  priors. pp. 434--443 in Goetl, B. et al. (Eds.) (1998).  Proceedings
  of the Intelligent Tutoring Systems Fourth International Conference,
  ITS 98.  Berlin, Hiedelberg: Springer-Verlag.



** What I actually want you to read:

Unfortunately I'm not aware of a single good, short discussion of
CDMs.

I'm going to suggest you look at several sources.  Pdf's are in the
"Reading" subdirectory for this lecture, unless otherwise indicated. 

* One that I like is Junker-sijtsma-apm-2001.pdf.  But it really only
  makes sense to read up through p. 266, for an introduction.

* de la Torre-dina-est-115-30-jebs.pdf is also pretty good, up through
  about p. 118.  It focuses on the particular model, the DINA model,
  that we will consider in class.

* For a broader view, rupp-and-templin-review-article.pdf is good,
  though rather pedantic.  This paper was published with several
  discussions (pro and con), and the links are all here:
  http://www.tandfonline.com/toc/hmes20/6/4 





Data Analysis
-------------

There is a nice R package (CDM) for analyzing data with the DINA
model.  Unfortunately Stampfer's data is not coded in a way that makes
DINA analysis very interesting.

So instead we will look at some of the data from Koedinger & Nathan's
(2004) DFA study of story problems.  

The following stuff is in the data subdirectory for today's lecture:

DFA1&2-hao.csv             -- the data!
cdm.r                      -- the R code!
Koedinger-Nathan-LS04.pdf  -- the article!

I think you have looked at Ken's article in class already but if not
please skim it.  Also, please have a look at, and try, the code in
cdm.r.  It's actually kind of neat.

See you in class,

-BJ