Skip to content

Be Skeptical of Asset Management Research: summary and key takeaways

This short essay argues that career and fee incentives push both academic and asset management researchers toward p-hacking and overfitted backtests. The author says the problem is somewhat milder in practice because of performance fees and reputation, but investors should still be skeptical of both.

Harvey, C.R. · 2021 · SSRNSummary updated 20 November 2026
Published2021Working paper
Asset focusAsset-agnosticResearch methodology
TypeResearchQuant Crises & Methods
Reading time4 minthis summary

Be Skeptical of Asset Management Research at a glance

The question
Do economic incentives distort finance research in the same way that sponsor conflicts distort research in other fields, and is practitioner research affected as much as academic work?
Dataset & period
No new dataset or sample period. The paper is a position essay (dated August 16, 2021) that draws on the author's earlier work, including over 400 published factors and about 90% of journal articles reporting supportive evidence.
Method
The paper is a narrative argument, not an empirical test. It walks through how publication and promotion incentives encourage data mining, gives simple examples such as testing 100 variables, and contrasts academic incentives with those of asset managers. It ends with a list of questions investors can ask.

Key findings of Be Skeptical of Asset Management Research

  • About 90% of finance journal articles provide evidence supporting the hypothesis they test, and over 400 factors have been published in top journals.
  • Typical p-hacking choices include picking the best variables, choosing sample start dates, dropping episodes such as the global financial crisis or COVID-19, and altering outlier rules or estimation methods.
  • If 100 variables are tried, about five would look significant purely by chance, so reporting a single winner is not credible.
  • The author concludes that likely half of the empirical findings in finance are false, because of incentives and the rare use of multiple-testing corrections.
  • Asset management is judged less exposed than academia, since performance fees reward repeatable results and poor live performance costs reputation and investors.
  • Rewarding researcher A (whose idea worked) over equally capable researcher B (whose idea failed) can create a culture that pushes both toward data mining.

Robustness, as reported in the paper

  • The paper reports no new tests. It cites prior work on repeatable performance, multiple-testing methods and ETF performance chasing as support for its claims.
  • It notes that peer review adds some trust but does not remove the need for skepticism.

Caveats and limits

  • The paper is an opinion essay with no new data, so the claim that likely half of findings are false is asserted rather than tested here.
  • The view that asset managers face weaker p-hacking incentives is argued from incentives, not measured (inferred).
  • The author concedes that a lot of low-quality research exists in asset management too.

Items marked (inferred) are our reading of the design, not statements by the authors.

Why it matters

The paper gives practitioners a short checklist for judging a new strategy pitch, including asking whether all tried variables are recorded and whether an economic rationale existed before the result. It frames backtest evidence as dependent on the research culture that produced it.

Questions about this paper

Why does the author say finance research findings can be unreliable?

Publication and promotion incentives reward positive results, which encourages data mining and p-hacking. Statistical corrections for the number of variables tried are rarely applied.

Is asset management research less biased than academic research?

The author thinks the problem is somewhat less severe there, because performance fees and reputation penalize overfitted strategies. He still says investors should be skeptical.

What questions should investors ask about a new strategy?

Ask whether the firm keeps a record of all variables tried, whether there is a solid economic foundation, and whether specific alternatives were tested and left unreported.

Where can I read the original paper?

The original is at its publisher: https://doi.org/10.2139/ssrn.3906277. We link to the paper and do not host the PDF.

Follow new quant research

Download MacroFinance market data as CSV or Excel, or pull it by API, and test the ideas in these papers yourself. New summaries are added as we review them.

Create a free account
Summaries are written by MacroFinance from the published papers, in our own words, for information only. They are not investment advice and may omit detail. Papers are not hosted here: each page links to the original at its publisher.