A scientometric study estimating the percentage of fabricated experimental data in biomedical scientific literature somewhere at 5-10 %
An awkward biochemical procedure of graphical data presentation has created a unique trap for unscrupulous researchers. "Absent-minded" fabrication
of certain type of this procedure's output very often results in an "impossible" picture that physically can not be based on any real data. Cases
seem to be sufficiently frequent to make statistics for quantitative measuring scientific fraud.
The problem of estimating the percentage of fraudulent research in science is actually a political one. Perhaps, like estimating the spread of
stimulators' abuse in professional sport, or counting the number of bribe-takers in Moscow governmvent. Everywhere we may find an horde of
hypocritical bureaucrats-ideologists always ready to argue that cases of fraud within their jurisdiction are negligibly small. In the field of
estimating scientific fraud the situation seems to be the most absurd. The prevailing bureaucratic opinion is that only from 0.004 to 0.1 percent of
scientific publications are fraudulent. I think it is meaningless to say that these numbers are wrong. The real problem (IMHO) is to understand
whether this number is closer to 1%, to 10% or to 99%. This my work tries to shift an estimate closer to the right on this scale, yet it is too far
from absolute clarity in this question.
Scatchard plot analysis is an extremely popular procedure in many fields of biochemistry, immunology, pharmacology, etc. The meaning of this procedure
is that the saturation isotherm (i.e. the amount of substrate (ligand) Bound to some binder is measured as a function of added (or Total) amount of
this substrate - see left panel of the figure) is redrawn on a very strange coordinate plane with Bound on X-axe and Bound/Free on Y-axe (where
Free=Total-Bound) - see the graph in the center of the figure.
If this is the first time you hear about Scatchard plots, you should have here the right feeling that it is rather difficult to understand how the
experimental data should look like on such coordinate plane. Indeed, this graphical transformation (actually, it has some meaning) converts a plain
and clear presentation of experimental data into something lying absolutely beyond understanding of most biomedical researchers. One funny consequence
is that fabrication of trustworthy Scatchard plot seems to be also a too difficult job for them.
The picture above illustrates a dreadful pitfall in the process of fabrication.
Left panel represents the typical scenario of two "parallel" binding experiments, i.e. two almost identical experiments giving pairs of experimental
points with the same Total concentration and different Bound concn. Of course, on the usual coordinate plane it results in pairs of experimental
points lying precisely one under another.
On the Scatchard plot both X- and Y-axes are linear functions of Bound signal, so, (perhaps with some effort) you may understand that these pairs
should lie approximately on the line drawn through the start of coordinates as it is shown at the central panel of the figure (Free concentration
usually may be considered equivalent to Total).
Now let's imagine that you are trying to fabricate a nice-looking Scatchard plot for such experiment directly, without fabricating data on normal
coordinate plot first. Obviously, if you are not too cautious, almost certainly you will draw pairs of points lying one under another (right panel of
In my view, this peculiarity in the Scatchard plot is quite sufficient to make accusation of data fabrication. It is almost impossible to imagine real
dataset that may correspond to such plot, or at least its receiving requires enormous and absolutely meaningless efforts. I also do not see what sort
of "honest" error may lead to this appearance.
Yet, a pair of official opinions (just simplest bureaucratic figures of speech from Nature journal and ORI) do not share my point of view at least
Of course, this is not the most important type of scientific fraud, it may be called an 'absent-minded' data falsification; but it may have
interesting applications. After I have found the case of this folly in an article in Nature, I clearly understood that it is really a very widespread
type of error and it may provide unique database for quantitative measuring scientific fraud.
Indeed, Scatchard plot analysis is an enormously popular procedure in biomedical sciences; citation index shows over 1000 citations of original
Scatchard's article every year. About 3-4 times more authors use his plot without citing anything. So, stupid browsing scientific journals and
counting falsified plots may resolve two problems:
1. It makes possible to estimate directly the percentage of fabricated experimental data in biomedical sci. literature simply by dividing the number
of falsified plots (i.e. like right panel of fig.1) by the number of correctly drawn plots of parallel experiments (i.e. looking like the central
panel of fig.1).
2. The Scatchard's article is dated by 1949, so it may also be possible to estimate the trend of changing this percentage during last 30 years in
order to check the popular opinion that quality of science seriously deteriorated itoday in comparison with heydays in sixties.
Apparently, the most convenient way to collect statistics for this study is browsing the Journal of Biol. Chemistry - yearly subscription of this
journal contains usually about 200 articles displaying various sorts of Scatchard plots. So, survey of this journal may be quite sufficient for the
first goal listed above, though, I guess, estimating the historical trend of fraud may require more thorough study.
So far, I have succeeded to browse the subscription for this journal for 1992. Take look, if interested, at technical data. The result is that I have
found 32 correctly drawn "parallel" Scatchard plots and two fabricated plots.
It is not quite correct, but I also may add the mentioned above case in Nature estimating that the number of Scatchard plots I have seen accidentally
since I started this project is less than that contained in a half-year subscription for J.Biol.Chem., .
So, the "initial estimate" based on three cases is that the percentage of fabricated plots is somewhere at 5-10%.
This number may be bigger if there are significant additional number of correctly fabricated Scatchard plots. Yet, I don't think there may be such
cases in reality; in my view correct falsification requires from the author much more wits than honest conducting the whole experiment, and,
therefore, I don't see why he should use data falsification.
This estimation does not mean that only 5-10% of biomedical researchers fabricate data. Obviously, every research paper usually contains data from
several different types of experiments such as Scatchard plot analysis. So, the percentage of articles with fabricated pictures at least for one of
them should be several times more - that is about 20-30%. Then, every researcher participate in writing more than one article during his lifetime.
Therefore, the conclusion is that:
MOST LABORATORY RESEARCHERS IN BIOMEDICINE FABRICATE DATA FROM TIME TO TIME.
Dealing with Scatchard plots, I have found at least two other simalar indicators of data fabrication. Unfortunately, they can't serve for numerical
estimation; but being substantially more widespread, they certainly support impression that my conclusion above is correct.
For more detailed report read my article Absurd trivial errors in Scatchard plot analysis; in brief, the first other indicator is that the scattering
of datapoints around trend line on most Scatchard plots does not comply the usual property of physical measurement that small signals are measured
with bigger relative error (CV). Instead, they seem to stress the common notion of a "nice looking curve". It is not a conclusive indicator, but, as
I wrote, it just supports an impression.
The other indicator is again a funny one. There is another type of binding experiment called inhibition (or displacement) experiment resulting in,
roughly speaking, the same curve but turned upside down. Scatchard transformation can not be applied to such curve directly (like floppy disk can not
be read if it is inserted upside down). Or if an obvious modification was used, it physically can't result in a "nice looking curve". Nevertheless
very numerous papers present really nice Scatchard plots derived from inhibition curves. I think it is impossible and all these plots were fabricated,
but I am not sure. Several times I tried to receive explanations from authors about method of deriving their Scatchard plots, but, of course, there
were no responses. So, again, this indicator just supports my impression.