The 1884 Weight Experiment Randomized with Playing Cards

Charles Sanders Peirce (1839–1914), photographed in 1891. Image: Napoleon Sarony; original held by the New York Public Library; source: Wikimedia Commons; license: public domain.

In October 1884, Peirce and his student Jastrow reported a set of weight-perception experiments to the US National Academy of Sciences: the experimenter used the color of cards from a shuffled pack to decide whether the pressure would be “first increased, then decreased” or “first decreased, then increased,” while the subject made judgments from behind a screen. Even when the subjects believed they had no confidence at all, their judgments were still clearly better than guessing. The experiment is often regarded as one of the earliest to use an explicit randomization scheme, but what it randomized was the order of trials, not group assignment.

The Dispute over the Difference Threshold, and the Apparatus

From the mid-19th century onward, the psychophysics of G. T. Fechner assumed the existence of a “difference threshold” (Unterschiedsschwelle): when the difference between two stimuli falls below a certain proportion, people can no longer tell them apart. Peirce, who had long studied observational error at the US Coast Survey, held that the errors made in comparing two sensations arise from a great many tiny chance disturbances along the path of nerve conduction and should follow the error curve (the normal distribution); in that case, there would be no threshold. The two views yield different testable predictions. If a threshold exists, then for differences below it, right and wrong judgments should each make up half in the long run. If the theory of errors holds, then no matter how small the difference, the proportion of correct answers should be above half, and that proportion can be calculated in advance from the theory of errors.

According to Hacking's research, the experiments were carried out between 10 December 1883 and 7 April 1884. Jastrow, then Peirce's student at Johns Hopkins University, recalled in 1914 that the problem, the apparatus, the method and the way of arriving at the results were all supplied by Peirce, and that the apparatus was set up in his own room. Both men served as subjects, each making 150 judgments in a session lasting about two hours.

The apparatus was adapted from a Fairbanks postal scale. One end of the beam carried a small block fitted with a rubber cap, which pressed up from below against a finger of the subject's left hand; with a 1-kilogram weight on the scale pan, the finger bore a pressure roughly equal to one quarter of that weight. A separate small pan holding the “difference weight” could be lowered on or lifted off by a fine rubber band, increasing or decreasing the pressure without any sound or vibration. The difference weights were 60, 30 and 15 parts per thousand of 1 kilogram; later on, Jastrow switched to 20, 10 and 5 parts per thousand. In each trial, the subject had to judge whether the pressure had “increased then decreased” or “decreased then increased,” and was not allowed to answer “can't tell.” At the same time, the subject reported confidence on a scale from 0 to 3: 0 meant no inclination at all, and 3 meant the strongest assurance one could have about a sensation of this kind.

Ordering by Playing Cards and Concealment by a Screen

The order in each trial was determined by playing cards. Each set of 25 trials used a pack of 25 cards, 12 red and 13 black, or the reverse, so that for a given difference weight, the 50 trials of one session had exactly 25 red cards and 25 black. Before each set, the pack was shuffled thoroughly, and the experimenter used the colors of the cards, turned up one after another, to decide whether each trial would be increase-then-decrease or decrease-then-increase. After the subject answered, the experimenter laid the card face down or face up depending on whether the answer was right or wrong, and, according to the confidence level, set it so that the corner bearing 0 to 3 punched holes pointed in a fixed direction. This meant no separate notes were needed and the pace was not slowed; the results were recorded all together once a set was finished.

A postal scale made by the Fairbanks company, in the collection of the Smithsonian National Postal Museum (date of manufacture unverified). Peirce and Jastrow modified a postal scale of the same brand; this image is for reference only and is not the scale used in the experiment. Image: Smithsonian National Postal Museum; license: CC0 1.0.

The reason given in the paper was that this prevented the subject from making any psychological guess about which change the experimenter “might be likely to choose.” The authors also wrote about a side effect of the random sequence: long runs of the same kind of change, occurring by chance, would confuse the subject. They believed, however, that if long runs were deliberately excluded and the subject knew this, the resulting disturbance would be even greater. Hacking points out that the problem of long runs remained contested later, when Fisher promoted randomization, and that Peirce's use of two small packs with opposite numbers of red and black cards had in effect already limited the imbalance between red and black within a session.

As for concealment, a screen hid all of the experimenter's movements, and the paper states that they took ample precautions against “unconsciously received hints.” It should be noted that this blinded only the subject. The experimenter could see the card colors and knew exactly how to add or remove weight each time, so he was not blind. The role of randomization was to take “which order to choose” out of the experimenter's hands, so that even if he fell into some rhythm, consciously or unconsciously, it could not affect the sequence of stimuli. Hacking's summary of the experiment is exactly this: weights were presented to subjects who could not see the manipulation, and randomization was used to guard against unconscious suggestion.

Accuracy of Low-Confidence Judgments

The paper compared the number of errors in each set with the expected value calculated from the theory of errors, and concluded that, as far as could conveniently be traced, the error counts all conformed to the probability curve, thereby “destroying every presumption in favor of a difference threshold.” Even more striking is the confidence record in footnote 4 of the original:

Subject and difference weights	Trials	Overall error rate	Share with confidence recorded as 0	Error rate at confidence 0
Peirce, 15/30/60 parts per thousand	1,125	29%	63%	38%
Jastrow, 15/30/60 parts per thousand	1,975	23%	57%	30%
Jastrow, 5/10/20 parts per thousand	1,675	32%	93%	34%

The last row is the most telling: when the difference weight was only 5 to 20 parts per thousand, Jastrow marked more than nine in ten of his judgments as “no inclination at all,” yet about two-thirds of those judgments were still correct. On this basis, the authors argued that confidence reflects a “secondary sensation” of the difference between the two sensations. This secondary sensation likewise has no threshold, but subjects often overlook it, so much so that confidence was still recorded as 0 even when the rate of correct answers reached three-fifths. They also found that average confidence roughly followed m = c log(p/(1−p)), where p is the probability of a correct answer. As cited by Hacking, Stigler considered this possibly the earliest attempt to elicit subjective probability experimentally.

The final paragraph of the paper pushed the conclusion further: people may gain knowledge of other people's minds through faint sensations of which they are not conscious, and “women's insight” and certain “telepathic” phenomena might perhaps be explained in this way. This paragraph is the authors' speculation; the experiment itself did not test it.

Relation to Fisher's Randomization, and Limitations

Calling it “one of the earliest randomized experiments” rests mainly on research in the history of science. As cited by Hacking, Stigler wrote in 1978 that this was the first experiment he knew of to be conducted according to a “precise, mathematically sound randomization scheme.” Hacking sums up its design as three innovations: deliberate randomization, not allowing subjects to answer “can't tell,” and confidence ratings from 0 to 3. By way of contrast, in Fechner's earlier weight-lifting experiments, Fechner himself served as the subject and knew in advance which box was heavier. Hacking also notes that the French physiologist Charles Richet independently used random card drawing in telepathy tests at about the same time, so “earliest” should be understood as among the earliest known.

The practice was not handed down all the way to Fisher. Hacking found that the later psychophysics literature almost never mentioned randomization again. Fullerton and Cattell's 1892 study says only that trials were conducted in an “irregular order”; when Jastrow later gave a systematic account of the method, he did not mention randomization either; and the debate over Peirce's method centered on “not allowing ‘can't tell’ answers.” Fisher first publicly argued for the central role of randomization in 1926 in “The Arrangement of Field Experiments,” and his 1935 The Design of Experiments made it textbook material. His reasons belonged to his own system of statistical inference: randomization gives error estimation and significance testing an objective basis and can improve the efficiency of experiments. Hacking states explicitly that he does not believe early psychical research had any important “influence” on Fisher; he points out only that the concern about chance long runs in the “lady tasting tea” example of The Design of Experiments had already been raised by Peirce.

Judged by the standards of modern design, the experiment has several limitations. First, it randomized the order of trials within the same subject; it did not randomly assign subjects to treatment and control groups. Dehue 1997 argues that the random group design took shape gradually in educational psychology during the 20s of the 20th century, and that McCall appears to have been the first to propose forming groups at random. Second, there were only two subjects, who were the authors themselves, and the experimenter was not blind. Third, the balanced packs themselves left a clue: a footnote in the paper admits that Jastrow's observational errors in the last two stages were abnormally small, partly because he unconsciously kept track of how many times each of the two changes had appeared in a set and answered accordingly. Fourth, practice effects made Jastrow's errors fall steadily, so the authors could only divide his data into four stages and compare them separately; and in the early experiments of December 1883 and January 1884, the procedure had not yet been perfected. Finally, the paper describes its data analysis in less detail than its data collection. Its inference relied mainly on comparing error counts with the theoretical values of the error curve, and the role of randomization in inference was not explicitly argued in the way Fisher would later argue it.

References

1. Peirce, C. S., & Jastrow, J. (1885). On Small Differences of Sensation. Memoirs of the National Academy of Sciences, 3(1), 73–83 (read 17 October 1884). PhilArchive scan; Full text at Classics in the History of Psychology

2. Hacking, I. (1988). Telepathy: Origins of Randomization in Experimental Design. Isis, 79(3), 427–451. doi:10.1086/354775

3. Stigler, S. M. (1978). Mathematical Statistics in the Early States. The Annals of Statistics, 6(2), 239–265. doi:10.1214/aos/1176344123

4. Dehue, T. (1997). Deception, Efficiency, and Random Groups: Psychology and the Gradual Origination of the Random Group Design. Isis, 88(4), 653–673. doi:10.1086/383850

5. The Collected Papers of Charles Sanders Peirce, Vol. 7, editors' notes (including Jastrow's 1914 recollection). Electronic edition

6. Research Methods · Rituals can be imitated; understanding cannot be faked

7. Compiled and published by Scholay, the one-stop academic platform. If any content infringes your rights, please let us know and it will be taken down.

8. Also published on EnterScholar, a community for scholars in the age of intelligence