SKIP TO CONTENT

Where Data-Driven Decision-Making Can Go Wrong

Docubyte

Summary.   

When considering internal data or the results of a study, often business leaders either take the evidence presented as gospel or dismiss it altogether. Both approaches are misguided. What leaders need to do instead is conduct rigorous discussions that assess any findings and whether they apply to the situation in question. Such conversations should explore the internal validity of any analysis (whether it accurately answers the question) as well as its external validity (the extent to which results can be generalized from one context to another). To avoid missteps, you need to separate causation from correlation and control for confounding factors. You should examine the sample size and setting of the research and the period over which it was conducted. You must ensure that you’re measuring an outcome that really matters instead of one that is simply easy to measure. And you need to look for—or undertake—other research that might confirm or contradict the evidence. By employing a systematic approach to the collection and interpretation of information, you can more effectively reap the benefits of the ever-increasing mountain of external and internal data and make better decisions.

Let’s say you’re leading a meeting about the hourly pay of your company’s warehouse employees. For several years it has automatically been increased by small amounts to keep up with inflation. Citing a study of a large company that found that higher pay improved productivity so much that it boosted profits, someone on your team advocates for a different approach: a substantial raise of $2 an hour for all workers in the warehouse. What would you do?

A version of this article appeared in the September–October 2024 issue of Harvard Business Review.

Partner Center