No Category Size Bias in Probability Judgments (#202)

Return to View Chart

How to Cite this Report

APA Style

Hannah Perfecto, Leif D. Nelson. No Category Size Bias in Probability Judgments. (2014, October 01). Retrieved 22:44, February 22, 2018 from

MLA Style

"No Category Size Bias in Probability Judgments" Hannah Perfecto, Leif D. Nelson. 01 Oct 2014 16:50 22 Feb 2018, 22:44 <>

MHRA Style

'No Category Size Bias in Probability Judgments', Hannah Perfecto, Leif D. Nelson, , 01 October 2014 16:50 <> [accessed 22 February 2018]

Chicago Style

"No Category Size Bias in Probability Judgments", Hannah Perfecto, Leif D. Nelson, , (accessed February 22, 2018)


No Category Size Bias in Probability Judgments [Internet]. Hannah Perfecto, Leif D. Nelson; 2014 Oct 01, 16:50 [cited 2018 Feb 22]. Available from:

Reference to Original Report of Finding Isaac, M. S., & Brough, A. R. (2014). Judging a Part by the Size of Its Whole: The Category Size Bias in Probability Judgments. Journal of Consumer Research, 41(2), 310-325.
Title No Category Size Bias in Probability Judgments
If the original article contained multiple experiments, which one did you attempt to replicate? e.g., you might respond 'Study 1' or 'Experiment 4'. Study 1
Link to PDF of Original ReportView Article
Brief Statement of Original Result Participants believed that a specific ball was more likely to be drawn from an urn if the ball came from a large category (e.g. this ball, along with the majority were grey) versus a small category (e.g. this ball was one of the few greys).
Type of Replication Attempted Highly Direct Replication
Result Type Failure to Replicate
Difference? Same Direction, .475
Number of Subjects 271
Number of Subjects in Original Study 223
Year in which Replication Attempt was Made 2014
Name of Investigators (Real Names Required) Hannah Perfecto, Leif D. Nelson
Detailed Description of Method/Results The original authors generously provided their materials in an appendix, which we employed in our replication (see below for one exception, in the elicitation of the estimate). See the included PDF for said materials.

We failed to replicate the original authors' effect of category size, with probability estimates for drawing ball #8 not differing significantly whether ball #8 came from a large (M=9.9%) or small (M=8.8%) category, t(242)=.715, p=.475.

We also included two additional cells to test whether the original phrasing of the question could have led participants to misconstrue the question as asking about the probability of drawing *any* grey ball, which could cause their effect.

We accomplished this by asking half of all participants first the probability of the (small or large) category and second, on a separate page, the probability of the specific case. This order was reversed for the other half of participants, such that the first question asked was a direct replication of the original study.

Our goal was to attenuate the original effect through this added factor of phrasing. However, since the original effect failed to emerge, we could not do so, F(1, 511)=.63, p=.428.

The full write-up of this 2x2 design is included as a PDF. Any parties interested in the raw data may contact H.P.

Figure 1 plots both the original study's means as well as our replication's. Figure 2 shows that our replication, however, was inconclusive, based on sample and effect sizes (Simonsohn, 2013).
Any Known Methodological Differences
(between original and present study)?
We did modify the elicitation of the dependent variable to minimize misunderstanding: they asked participants to report their probabilities as a percentage, where as we ask our participants to report their probabilities as the number of times the ball is drawn out of 100 draws. Beyond this, we do not know of any methodological differences between the two studies.
Email of Investigator
Name of individuals who
actually carried out the project
H.P. posted the study online to collect subjects. H.P. and L.D.N. analyzed the data.
Location of ProjectHaas School of Business, UC Berkeley
Characteristics of Subjects
(subject pool, paid, etc.)
Adults tested through internet
Amazon Mechanical Turk
Where did these subjects reside?United States
Was this a Class Project?No
Further Details of Results as pdf PDF

Additional Comments
Email of Original Investigator
Quantitive Information Using the analysis suggested by Simonsohn (2013 - ), which incorporates sample size and effect size, our replication was inconclusive.
I have complied with ethical standards for experimentation on human beings and, if necessary, have obtained appropriate permission from an Institutional Review Board or other oversight group.
TAG: Attention TAG: JDM TAG: Language TAG: Learning TAG: Memory TAG: Perception TAG: Performance TAG: Problem Solving TAG: Social Cognition TAG: Social Psychology TAG: Thinking

Are you posting an unpublished replication attempt that you conducted yourself, or noting a published replication attempt?

Post Unpublished
Post Published