I do understand convergent validity which the last post refers to as triangulation. Usually we use triangulation when we cannot directly measure what we are trying to find, so we measure from three other points for the intersection.

Convergent validity is the issue here. Convergent validity refers to the fact that the same result can be obtained by using different methods, that is, the results all converge. But the independent methods first have to be valid in and of themselves.

My issue with this analysis of the security troll is that the independent methods are (1) not valid in and of themselves and (2) not independent in the sense that the multiplication rule can apply.

On (1), see the analyst's post in which he refers to the controversy around his methods; there is controversy because these methods have never been validated or, in the case of Koppel et al's work on gender, the validation showed how inaccurate the methods are, far too low for instance to be taken seriously in a court of law. And when there are attempts at validating the unvalidated methods, again the results are so low that they cannot be taken seriously as reliable forensics.

On (2), note that the analyst made a mistake in thinking that he should multiply the error rate instead of the accuracy rate when he applied the multiplication rule. Another error in using the multiplication rule is one that the analyst himself mentions--the variables he uses are not independent of each other because punctuation can correlate with sentence length, etc. So the reason that two wrongs don't make a right is that we multiply accuracy rates, not error rates.

It would really behoove anyone interested in this field to actually read the research literature by which I mean reports of experimental results because as Chaski 2005 shows there is a patent-pending method which works, but it does not rely on variables grounded in linguistic theory. (Chaski, Who's at the Keyboard? International Journal of Digital Evidence, Spring 2005.)

I am willing to work with scholars in this field and currently maintain research relationships with other scientists in Canada, Switzerland, Greece, England, Ireland and the US. I applaud the analyst's interest but I recommend that he engage with other researchers so that he can rapidly get out the thicket he has stumbled in through ignorance of the field.

[ reply ]

Link to this comment: http://www.securityfocus.com/comments/articles/11419/34147#34147