Amanda.  Section 5.

Summary:  

You fit IPF on a 2x3 contingency table, with and without structural zeros.  The structural zeros has a good fit and so you deduce that the independence assumption is valid for this data.  When removing the structural zero, you find that the independence assumption does not hold and so witness choice depends on whether the target is present in the lineup.

In the following section you show that your method is at least just as good as ROC by showing that it shows the 4 qualities of the ROC graph.

You run a test on the data and show that for model selection you find a set of interaction terms.

In the following section, you try to show that your method still works even if 'confidence' means different things to different witnesses.

In 5.4, you show that your method allows for different assumptions of filler IDs.  The graphical model does not change.


General Comments:

To begin, I think you are a good writer, and your sections stand well on their own.  It would be helpful to include more 'flow' between your sections.

For instance, I realize that the different subsections of section 5 highlight good features of your model, especially in comparison to ROC, but I am not sure quite how they fit together as a whole.  Perhaps a summary paragraph at the beginning of the results section would be useful to guide the reader through the many facets of the results.  Something like "Log Linear model does these things just as well ROC and does these things even better."

The G^2 statistic.  I skimmed through sections 3 and 4 and saw that although the G^2 statistic is introduced in 4, it is still confusing.  What do the degrees of freedom correspond to or the numbers for the G^2 values mean?  Explanation and intuition behind G^2 would be greatly appreciated.  Similarly, I have no idea what an unconditional edge test is or what it does.  Some context behind these tools would be useful for clarity.

Figures and Tables:

If you can, try to put the tables in-line.  It is confusing to have to flip to the tables and back to the text.  In the captions, you give chi square and G^2 values, but these numbers mean nothing to a reader.  Longer captions for your graphics would be appreciated along with legends.  For instance in Figure 4, I do not know what the blue or green curve is or how the G^2 distribution shows that it is the superior test.  I like the graphical model image!  This is a very helpful figure.

'Don't' use contractions :)

Specific comments:

Paragraph 3, Section 5.1.  Answer the question right away as it can get lost in the details, and then support it with the details.  It is also unclear to me why the independence assumption no longer holds.  The G^2 value is too high?  Is 120 the cutoff value?  

Paragraph 3, Section 5.1.  "This suggests that witness," does "this" mean the invalidity of the independence assumption?

Section 5.2 Paragraph 1 and 2.  I would move "Thus...the same information" after the first sentence in the first paragraph.  Otherwise, it sounds like the section is going to only talk about ROC curves.  There is a,b,c, and d which seem to refer to 1,2,3, and 4.  Make sure these are the same.

Section 5.2 Numbers at the top of the page.  The numbers are burdensome to memory.  Perhaps make acronym names instead of numbers.  (WC, TS, LC, ECL).

Explain the graphical model at the end of this section saying, "the useful variables in predicting witness choice are ..."

Section 5.3 beginning.  Again I would say how Log linear analysis compares to the ROC at the beginning.  I think the idea is that you look at worst and best case scenarios and show that your method is superior than ROC either way.

5.3 P.16  I don't like how you begin two paragraphs in a row with "we."  I would emphasize the subject matter in one of these and use passive voice such as" A range of distributions were tested.."

"We also tested assumed distributions that are both left-skewed and right-skewed."  I'm not exactly sure what you mean here.

5.4  This section seems clear.  Nice job!