Free shipping on all U.S. orders · Tracking on every package

Research Literacy

Small Studies, Big Claims: Sample Size, Statistical Power and Reporting Quality

Much of the peptide literature rests on small studies. This guide explains, from well-known methodology papers, what statistical power is, why low power both misses real effects and inflates the ones it finds, which features of a research field make findings less reliable, and how the ARRIVE 2.0 reporting guidelines help you judge an animal study.

Last reviewed October 28, 2026 · Peptide Privé Research Team · 8 min read

Sample size and statistical power research guide cover

Many research peptides have a literature made up of a handful of studies, often from one or two groups, often in cells or small animal cohorts. A striking result from a small study can travel far: into reviews, supplier pages and forums, usually without the caveats. This guide explains why small studies deserve particular care, using three widely cited methodology papers and one reporting guideline.

It is a companion to Dish, Mouse or Human?, which looks at what kind of model a study uses. This guide looks at how big and how well reported the study is.

What Statistical Power Means

Statistical power is the probability that a study will detect an effect of a given size, if that effect really exists. It depends mainly on:

  • Sample size: more independent units (animals, cultures, participants) give more power.
  • Effect size: large effects are easier to detect than small ones.
  • Variability: noisy measurements need larger samples.
  • The significance threshold: a stricter threshold needs more data.

A study with 80% power will, on average, detect a real effect of the assumed size four times out of five. A study with 20% power will miss it four times out of five.

Low Power Does More Damage Than You Might Think

The obvious problem with a small, underpowered study is that it can miss a real effect. Button and colleagues, in a 2013 analysis in Nature Reviews Neuroscience, pointed out a less obvious problem: low power also reduces the likelihood that a statistically significant result reflects a true effect.

They estimated that the median statistical power of studies in the neurosciences was between about 8% and 31%. The consequences they describe include:

  • Overestimated effect sizes. When a small study happens to find a significant result, it usually does so because the observed effect was, by chance, larger than the true one. Button and colleagues call this effect-size inflation the "winner's curse."
  • Low reproducibility. Follow-up studies tend to find smaller effects, or none.
  • Ethical and practical waste. Unreliable research uses animals, time and money without producing dependable knowledge.

These findings come from neuroscience, but the statistics are general. Any field built on small studies faces the same problems.

What this means when reading a peptide paper

If a small study reports a large, significant effect, the most likely true effect is smaller than reported, and it may be zero. A single small positive study is a reason to look for replication, not a settled finding.

When Are Research Findings Less Likely to Be True?

In a 2005 essay in PLOS Medicine, "Why Most Published Research Findings Are False," John Ioannidis set out a model of how often a statistically significant finding is actually true. From it he drew six corollaries. In his words:

  1. The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.
  2. The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.
  3. The greater the number and the lesser the selection of tested relationships, the less likely the findings are to be true.
  4. The greater the flexibility in designs, definitions, outcomes, and analytical modes, the less likely the findings are to be true.
  5. The greater the financial and other interests and prejudices in a field, the less likely the findings are to be true.
  6. The hotter a scientific field (with more scientific teams involved), the less likely the findings are to be true.

Several of these describe niche peptide research well. Studies are small. Many outcomes are measured. Designs vary from study to study. Some research is done by groups with commercial interests in the compound. And popular compounds attract many groups chasing quick results. None of this means a given finding is false. It means the prior chance that it is true is lower, and the evidence needed to be confident is higher.

Reporting Quality: The ARRIVE 2.0 Guidelines

Even a well-designed study is hard to judge if it is poorly reported. For animal research, the most widely used reporting standard is ARRIVE 2.0 (Animal Research: Reporting of In Vivo Experiments), published by Percie du Sert and colleagues in PLOS Biology in 2020.

ARRIVE 2.0 organises its items into two sets:

  • The ARRIVE Essential 10 is the basic minimum to include in a manuscript. Without it, reviewers and readers cannot confidently assess the reliability of the findings. It covers the study design, the sample size, measures to reduce subjective bias, outcome measures, statistical methods, the animals, experimental procedures and results.
  • The Recommended Set adds context: the ethical statement, declarations of interest, protocol registration and data access, along with more detailed methodology.

The ARRIVE website explains that the guidelines are also meant for planning: they offer explanations and advice on experimental design, minimising bias, sample size and statistical analysis.

Using ARRIVE as a reader

You do not need to score a paper formally. A quick pass through the Essential 10 catches most problems:

  • Sample size: How many animals per group, and was the number justified, for example by a power calculation? Is the experimental unit clear (animal, litter, cage, or culture well)?
  • Bias reduction: Were animals randomly allocated to groups? Were the people assessing outcomes blinded to group?
  • Inclusion and exclusion: Were any animals or data points excluded, and why?
  • Outcome measures: Was a primary outcome defined in advance, or were many outcomes measured and the significant ones highlighted?
  • Statistics: Are the methods described well enough to reproduce? Are exact values and measures of variability reported?
  • Interests: Does the paper declare conflicts of interest?

A paper missing several of these is not necessarily wrong, but you cannot tell whether it is right.

Red Flags in Small Studies

  • Very large effects from very small groups, such as a few animals per group.
  • Many outcomes, few corrections: dozens of measures with only the significant ones discussed.
  • No randomisation or blinding mentioned.
  • The experimental unit is unclear: for example, many measurements from a few animals treated as independent samples.
  • No replication: a single study, or several studies from the same group with no independent confirmation.
  • Conclusions that outrun the data: a cell or animal result described as if it applied to people.
  • Undeclared or commercial interests.

Green Flags

  • A justified sample size, ideally with a power calculation.
  • Randomisation and blinding described.
  • A pre-specified primary outcome, or a registered protocol.
  • Effect sizes with confidence intervals, not just p-values.
  • Independent replication by another group.
  • Data availability.
  • Cautious conclusions that stay within the model studied.

Putting It Together

When you meet a striking claim about a research peptide, ask four questions:

  1. How big was the study? Small studies inflate effects.
  2. How was it reported? Use the ARRIVE Essential 10 as a checklist.
  3. Has it been replicated independently?
  4. Has anything happened to the paper since? See When a Paper Is Retracted.

For examples of compounds whose literature has needed this kind of careful reading, see DSIP: A Peptide Named for an Effect It Never Pinned Down and Dihexa: Reading a Compound Record When Related Papers Are Retracted.

FAQ

Is a small study worthless? No. Small studies are often how a field starts. They generate hypotheses. The problem is treating them as confirmation.

Does a very low p-value fix a small sample? No. A small sample still tends to overestimate effect sizes, and a single study with a low p-value can still fail to replicate.

Does ARRIVE apply to cell studies? ARRIVE is written for in vivo animal research. Its core ideas (sample size justification, randomisation, blinding, clear statistics) are good questions for any experiment.

Sources

  • Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365-376. https://www.nature.com/articles/nrn3475
  • Ioannidis JPA. Why most published research findings are false. PLOS Med. 2005;2(8):e124. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
  • Percie du Sert N, Hurst V, Ahluwalia A, et al. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. PLOS Biol. 2020;18(7):e3000410. https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000410
  • ARRIVE Guidelines. The ARRIVE guidelines 2.0: https://arriveguidelines.org/arrive-guidelines

Every Peptide Privé batch is tested by an independent lab.

Contact our team with documentation questions.

For in-vitro laboratory research only. Not for human or animal use.

Research Literacy

Cosmetic Signal Peptides: Reading the Argireline and KTTKS Literature

A reference for laboratory teams on two short cosmetic-ingredient peptides: acetyl hexapeptide-8 (marketed as Argireline) and the collagen-derived pentapeptide KTTKS. It covers nomenclature, how they were first described, what penetration and analytical studies measured, and why concentrations across papers differ.

5 min read

Research Literacy

Dihexa: Reading a Compound's Record When Related Papers Are Retracted

A literacy case study for laboratory teams: dihexa, a modified angiotensin IV-derived compound, has a thin primary literature in which a founding paper carries an expression of concern and two related papers from the same group were retracted in 2025. This guide shows how to trace and weigh such a record.

5 min read