Majestic Documents Proven Frauds, page 1

Just to add a little fuel to this, I have done some linguistic analysis work myself. I published a piece of software called “Readability” in the MS-DOS days that measured the grade-level of passages against nine different formulas. It was published by Micro Power & Light out of Dallas, Texas. It sold for over a decade and on the whole probably bought me a new car. I've also published on the subject in peer reviewed journals, such as the 'Journal of Reading.' On the off-chance you’re interested in some of the specifics, this might apply to the Majestic paper analysis.

A computer can do nothing but count and make comparisons, therefore when you use a computer in linguistic analysis you are immediately constrained by the ‘tyranny of the machine.’ You must conform to what a computer can do. ‘Readability’ is the concept that some text is easier to read than others. If you are writing for a fourth-grade level audience you don’t want the text to be at a college level. You measure the difficulty by counting things like the number of words in a sentence, the number of three-syllable (and therefore more difficult) words, or you bounce the written words off a giant list of words and count how many you found. You then use rather simple linear regression equations to slice all the data up and come out with a score. Some formulas are plotted on a graph, e.g. Fry, some formulas give you amusingly precise answers, e.g. Grade 4.32, and some formulas result in a stated range, such as ‘Grade 4-5,’ e.g. Dale-Chall. Some of these formulas are more ‘scientific’ than others in the way the formulas are constructed. Generally speaking, the ones which offer a range are probably more valuable than ones that insist on a specific grade.

One thing is certainly true, and that is that they disagree. They DO tend to show a general trend, but some things can throw the formulas off pretty readily. For example, the Dale Chall word list was originally of 3,000 words known to the majority of fourth graders. It was developed in the forties and had the word ‘schoolmaster,’ but not the word ‘television.’ It has since been updated, but it was used well into the eighties and was one of the more reputable formulas at hand.

I’ll bet you dollars to donuts that this program used to analyze the Majestic papers has at least some readability formulas as part of the analysis. They could easily add the use of punctuation and other parameters. For example, the use of a semi-colon with writers in English is rare. If you compared two samples of writing from ‘different’ people and found the same use of semi-colons, this would be a point in favor of ‘same authorship,’ but only a point. It may be they are the same age and were subjected to the same textbooks. If you found the equivalent use of favorite phrases, such as ‘adamantly reject,’ for example, that would be another point in favor of ‘same authorship.’ Pokey Oats tried to use this method to show that Ghost Raven’s admission of a hoax was written by Springer. It wasn’t. That shows how tricky and dangerous it is to make conclusions built on this kind of evidence. If you find the same mis-use of words consistently between documents, that’s another flag. For example, people using ‘it’s’ when they mean ‘its’ or ‘they’re’ in place of ‘their.’ If you found a consistent form of grammar, such as using a plural verb after a conditional statement (correct, but rare these days. It’s ‘If I were,’ not ‘If I was.’), that would be a point in favor. None of these are a smoking gun, rather maybe smoking pellets in a shotgun shell. If ALL those pellets come out smoking, you’ve (probably) got yourself a match.

Now, the way to validate a formula is through statistics, but that begs the question of the underlying assumptions, which we don’t know. (An aside: the way I started this paragraph types this as my text. I frequently use it.) In order to even be considered valid they would have to say something like, ‘There is a 95% chance that these two samples were written by the same person.” The question is, what led them to that conclusion and how valid is it? This is particularly true when you must layer the governmentese idiolect over the top of a person’s normal mode of expression. Most people would not choose to write like a government memo if they had the freedom not to. It is stylistically strict. If these assumptions are locked away in proprietary software, we have to take their word for it. We’re still flying blind here, putting up a straw man to knock down. It’s just that they put us in the barn in the first place. I think it’s pretty shaky stuff. If I were a lawyer in court with their expert testimony to contend with, I’d tear into this issue with a chain saw.

[edit on 7/24/2007 by schuyler]