Putting Things into Perspective: Montana’s Method to Monitor Wolves

By Bob Crabtree and Adrian Treves

Effective wildlife management and sound science should work together because both depend on accurate predictions of their outcomes. The methods they use, whether an experiment, a management decision, or a statistical model, must be evaluated to determine how good their predicted outcomes are. Scientific testing of such methods is as necessary, if not more so, than testing a hypothesis because methods are used to test hypotheses and make decisions. The failure to do so is akin to administering a new, low-cost vaccine without first conducting independent testing. Yet the state of Montana would like us to believe that the management method they used to estimate wolf abundance is accurate because it has been peer-reviewed in a scientific journal. This is misleading for multiple reasons, any one of which should preclude its use. These reasons, at three levels, are:

1. Peer review provides an initial test as well as a method to catch and correct errors. Yet, it has been criticized by many for years as a flawed process that can only be verified by post-publication review by the global community of scientific experts in the field. Reviewers are time-stressed volunteers and are not necessarily specialized or fully qualified to review a particular submitted paper. In other words, each scientific paper and its peer reviewers may be wrong; only time and scrutiny by qualified experts from a diverse community of scientists will reveal if the single paper’s findings are accurate and reliable. Crabtree et al. (2025) conducted such a review and applied accepted scientific standards to evaluate a published method for determining wolf abundance, which is used to manage wolves in Montana. The state of Montana has still not responded seriously or scientifically to this and other post-publication reviews by scientists. Until the state responds with scientific evidence, its claims remain unsubstantiated. Possibly even more troublesome than their ignorance of constructive criticism, a key to advancing science, is the testing and evidence presented by Crabtree et al. (2025) that indicates the State of Montana is using unscientific methods to estimate wolf abundance, which we further summarize here.

2. The post-publication review and critique by Crabtree et al. (2025) strikes at the heart of the State of Montana’s claim that their estimate of wolf abundance is scientifically accurate. At a higher level, there are two main ways to ensure the accuracy of scientific work. First, the input data used must be verified within a valid sampling design and coherently linked to the predictive model. Then, the model's predicted outcomes must also be validated by comparison with real-life consequences. This is not possible with the State of Montana’s method for estimating wolf abundance, called iPOM, because it has a flawed sampling design that neither uses nor collects landscape observations or measurements to verify the input data and confirm its predictions. An additional approach is to use a statistical model that predicts a quantity that can not be counted or observed directly. When scientists use such models for elusive species like wolves, they must verify and test every step in their methods transparently–a hallmark of science–and reject the step or method if false (this is the principle of falsifiability, another hallmark of science). At this level, the State of Montana’s iPOM model fails to meet fundamental scientific standards for data verification, model validation, transparency, and falsifiability.

3. Evaluation at the level of the statistical models in iPOM evokes another essential standard of science: the overriding concept of bias, which is the difference between the predicted outcome and the truth. Because actual bias is rarely known, as in estimating wolf abundance, biologists are left with one option: test the assumptions underlying their models. Assumption testing ensures valid, unbiased results and provides the path to improvement. Ignoring it produces false or misleading conclusions. The iPOM method used by the State of Montana is so severely biased and unreliable that it cannot accurately predict wolf abundance, let alone detect changes over time. This is due to many problems, but first and foremost is that they simply ignored the identification and testing of assumptions. When tested, Crabtree et al. (2025) found violations of iPOM’s key assumptions and also found (1) mathematical and statistical analysis errors, (2) circular logic, and (3) the unverified use of ad-hoc variables. These serious problems are further compounded by the incoherence of iPOM’s cobbled-together models (each with its own long list of unmet assumptions), which, in turn, leads to further bias. We also found they excluded important known factors that directly affect wolf populations and instead chose to include factors that don’t change over time. This resulted in iPOM producing unrealistic, constant outputs unrelated to changing annual conditions. Lastly, because they did not provide a sufficient description of their methods and, upon request, did not provide input data and analysis results, iPOM cannot be reproduced (another principle of science). 

In summary, the State of Montana would like everyone to believe it knows what it is doing and should not be questioned. But sound science respects no authority. Instead, science respects transparent descriptions of coherent methods that can be repeated, testing of assumptions, and validation of results, thereby raising confidence in the model’s predictions. Instead, the State of Montana dispensed with accepted methods for wolf counting, eliminated the field monitoring of marked wolves needed to correct for bias, and ignored core scientific standards for evaluation and improvement. Cutting expenses does not justify abandoning science, especially for species of high economic and ecological importance, such as wolves. These problems, along other changes and ignored errors, resulted in the worst-case possible scenario for conservationists, managers, and wolves themselves: The State of Montana claims iPOM produces unbiased and precise estimates of abundance, when in fact iPOM severely overestimates wolf abundance and has an actual variance (reported as a confidential interval) that is even more biased than wolf abundance to the point one cannot detect a change, even if the wolf population falls below 150 individuals.

Read the full publication here:

Misleading Bias in iPOM_Crabtree Conner Treves_2025.pdf

Download PDF • 1.02MB