, SecurityFocus 2006-10-20
Story continued from Page 1
Moreover, Krawetz's forensics analysis does not stand up well, said Carole Chaski, a forensic linguist and principal researcher at the Institute for Linguistic Evidence, Inc. Chaski makes a living out of researching ways of identifying authors and has analyzed suicide notes and threatening e-mail in criminal cases to determine authorship.
She stressed that techniques such as measuring vocabulary, spelling errors and grammar errors are not good methods of identifying a document's author.
"This whitepaper floats some of these erroneous ideas which have already been shown by research, which the whitepaper is apparently unaware of, to be fairly unreliable for authorship analysis," Chaski said in an e-mail interview.
Chaski also took to task the paper's assertion that, if the errors in the methods were consistent in either being right or wrong, then the accuracy of the identification is improved.
"There is an old expression 'two wrongs do not make a right' which applies here," Chaski said. "If one method is only 60 percent accurate and it is combined with another method--for the nonce let us assume an independent method-- which is only 60 percent accurate, than the combined method has an accuracy rate of 36 percent, far below chance. One would be better off guessing than combining unreliable methods."
Security consultant Krawetz started researching the identity of n3td3v after another Full Disclosure subscriber challenged him to use linguistic forensics--a topic on which Krawetz presented at this year's Black Hat Security Briefings--to uncover the person's identity. He started with the hypothesis that n3td3v and Gobbles were the same and used methods of gender determination as well as analyzing lexicon, word frequency, punctuation frequency and preferred sentence length to create a profile of the two groups.
The analysis seemed to indicate that n3td3v had three members and Gobbles had three or four main members. The characteristics of three members of each group seemed to match up.
Krawetz acknowledges that some aspects of his analysis could have significant error. For example, this is the first time that he attempted to analyze documents to match up three different authors, a research problem that ILE's Chaski believes to be untenable.
Krawetz hopes that, if his analysis is shown to be in error, it will ultimately benefit his research.
"Very few people are using these techniques just because they are new technologies," he said. "On the other hand, if someone comes out and shows me I'm wrong, it could help me tune my system."