Wolf Management Needs Unbiased, Best Available Science

PREFACE:  Why the testing of assumptions is the key to reliable, unbiased scientific methods that get at the truth.

The coupling of best management practices with sound, unbiased science for wildlife conservation should go hand in hand because both measure success by their ability to predict outcomes. In fact, you can answer the question, “How good is your science?”—whether experiments, management decisions, or statistical models—by evaluating the bias of the outcome, which is the difference between the predicted outcome and the truth. Unfortunately, the actual measure of a prediction's bias, such as when estimating wolf abundance with a statistical model, is rarely, if ever, known. Hence, to conduct quality science while getting at the truth, biologists and managers are left with one choice: test the assumptions of the method or model to evaluate bias. Assumption testing is a prerequisite for a statistical model to produce valid inferences and reliable outcomes. Failing to meet assumptions leads to incorrect conclusions, misleading results, biased estimates, false positives or negatives, and inaccurate model representation of the data. And this is why our recent peer-reviewed publication evaluating a new technique, called iPOM, for estimating wolf abundance in Montana primarily focuses on the biological and statistical testing of assumptions. Without the identification and testing of assumptions, you simply can’t get to the truth. And without knowing how far you are from the truth (bias), you are flying blind.

Wolf Management Needs Unbiased, Best Available Science

It’s common sense and best business practice for company owners and managers to track their inventories over time to sustain profits and avoid debt and foreclosure. Similarly, fish and wildlife agencies are mandated to manage, track, and account for species populations over time using the best available science to maintain healthy populations held in the “public’s trust.” This is accomplished by reducing risks to sustain their abundance and strengthen their role in maintaining a healthy ecosystem, which we humans also rely on.

Like a business, the economic importance (costs and benefits) of wolves is high, and their ecological importance is even higher. However, their abundance is exceedingly challenging to track and account for compared to a business’s accounting methods. This is because carnivores are elusive, secretive, exist at low densities, display high mobility over large landscapes, and often exhibit aversive responses to humans and capture methods. Wolves are even more challenging because of their dynamic pack structure, territoriality, and complex social behavior. Like most vertebrate species, wolves cannot be counted in their entirety, so biologists sample the population using statistical methods and models. This is no easy job, and it requires managers and biologists to develop and apply the best available science and methodologies commensurate with their economic and ecological importance.

Introduction and Background

To explore and explain this challenging situation and to summarize the results of our recently published peer-reviewed technical paper about a new method, called iPOM, used to estimate wolf abundance, it is essential to understand the broad concept of bias (see Preface), which dominates as the primary scientific standard of a method's reliability, effectiveness, and successful application. If this essay explains the overriding concept of bias sufficiently, you will understand and capture the majority of the findings in our technical, peer-reviewed publication. However, when writing a technical paper, the authors often take certain fundamental concepts for granted, which leaves non-technical readers in the dark. In response, this essay (and preface) will provide a clearer explanation of what bias is really all about, why its evaluation is required for science to proceed, and, unfortunately, why it might be ignored. Bias is also the most crucial attribute used to assess the quality of an estimate (e.g., abundance and its confidence interval) because you simply can’t manage a population without reliably knowing how many you have. Because abundance is the primary criterion used for management and conservation decision-making, you do not want, for example, to enact increased killing of wolves based on a method that produces a biased abundance estimate that could likely be double the number they actually are (overestimation). These and other reasons are why we must evaluate, test, and verify the method used to determine abundance, especially when the technique is newly developed and applied annually to inform wolf conservation and management decisions.

We provide a graphical illustration in Figure 1 to elucidate the broad concept and meaning of bias. It depicts the two attributes (bias and variance) that determine the quality of a statistical method or model, such as iPOM. Another reason bias is an overriding concept is that, in addition to the bias of the point estimate of, say, 1000 wolves (a red dot in Figure 1), the variance can also be biased because it is estimated directly from the sample data. In this example, 1000 wolves in Montana might have a variance (reported as a confidence interval) of ±100 wolves (e.g., the spread of red dots in Figure 1A). This means that a population decline from an unbiased estimate of 1000 down to, say, 700 wolves can be reliably detected if this variance is correct. However, suppose the variance estimate is large enough that the confidence interval covers zero (1000 wolves ± 1200). In that case, one cannot detect any change, placing the wolf population in a high-risk situation because it could fall to, say, 150 or fewer without being detected.

Figure 1. Four scenario outcomes for the two independent attributes used to assess the quality of an estimation or prediction method. Bias is the distance between the red dots and the target’s center (true value); precision, measured by the variance, is the spread of the red dots. (A) is unbiased and precise; (B) is unbiased and imprecise; (C) is biased and precise; and (D) is biased and imprecise. They are graphically analogous to shooting a rifle at a target, where the shot taken is the abundance value (the point estimate). If the shooter’s grip is shaky, then the variance (spread of shots) is higher (B, D) than with a steady hand (A, C). Bias is where the sighting of the rifle (curved barrel, bad scope) is “off” and shots are consistently off-center (C, D) versus on-center or unbiased (A, B)

Now here is the catch-22 for bias. Because the bias of the point estimate of abundance cannot be calculated from the sample data (unlike the variance), one can only assess its bias by testing the method's assumptions. This is not only critically important to strong inference (science’s goal) but also illuminates the path to improvement by reducing bias and correcting errors included in a particular scientific methodology. That’s why we say it’s all about assumptions, assumptions, assumptions, just like the repetitive chant on the importance of location in real estate and audience in marketing. Fortunately, assumptions can be tested biologically and/or statistically, which is the responsibility of the developer and user of a statistical model, such as with iPOM, if users, like agencies, want to conduct science and provide the best available scientific methods that yield valid, reliable, trusted, and reproducible results. One can also perform simulation testing by varying specific features of iPOM to provide insight into the direction and magnitude of a particular bias. However, before any such simulations, you first have to have a reason to conduct them, that is, testing an assumption to see if it is violated, which indicates a high likelihood that bias exists. Next, one needs the actual data sets (e.g., hunter surveys, sampling design specifics, center (centroids) of wolf territories, radio-tracking results, and iPOM analysis results) to compare them with the simulation results. If not, then the simulation results are untethered from reality and futile. It is important to note that any simulations preceded by assumption testing are primarily the responsibility of both developers and users, particularly when a new method is being used and when developers withhold the necessary data.

Evaluating bias by testing assumptions accounted for the majority of the content of our recent publication that constructively critiqued iPOM. We also assessed other features of iPOM related to bias, which can cripple scientific inference if left unaddressed. They include the sampling design, model coherence, reproducibility, data verification, model validation, and the measurement of uncertainty–iPOM’s estimate of variance used to report a confidence interval. We also examined iPOM for circular logic errors, errors of omission (e.g., exclusion bias), and commission errors in their mathematical and statistical calculations. In our evaluation–a critique with recommendations for improvement on all issues–we evaluated and assessed these features and more.

Misleading biases in iPOM

In science, management, and decision-making, assumptions are everywhere. As such, we identified, evaluated, and tested over 20 of iPOM’s most essential assumptions–both implicit and explicit–in their three main models, each with covariate submodels. These covariate models are very important because, if constructed properly, they can correct for bias when detected. For example, their main model that estimates the area occupied (AO) by all individual wolf territories in MT assumes that the population of individual wolves and their territories are both closed and stable. If these explicit assumptions are violated, then abundance will be overestimated (positive bias), and a covariate model should then be applied each year. In any study, there are also critical implicit assumptions, for example, (1) choose a representative portion of the population to sample, and (2) sample those individuals in a way that represents their spatial movements if you use a spatial model. Implicit assumptions guide your decisions about how you design the study, the model(s) you choose, how you sample each year, how you conduct the analysis, and how you report the data. Another critically crucial implicit assumption is that the variance estimate includes all significant components of variance. Suppose one uses the correct formula for calculating variance (and the CI) given the chosen variables, called covariates. In that case, the outcome would properly reflect the contribution of the included data. Still, it would be biased if an investigator omitted variables that are essential components known to affect the phenomenon (e.g., changes in wolf distribution). This assumption, known as “exclusion bias,” would lead to underreporting bias in the variance (and the confidence interval). It would also weaken inference (which is what science is all about), mislead biologists and managers, and put the wolf population at risk.

Before diving into the basic findings of our evaluation of iPOM’s models and submodels, we made three initial observations: (1) a variety of structural problems starting with iPOM’s non-traditional use of spatial, rather than demographic models of abundance, (2) ignoring previous criticisms of iPOM’s approach, and (3) a complete failure to test assumptions. First, we found that iPOM substituted two spatial models, each of which estimates area, for the traditional demographic approach of counting or sampling the wolf population to estimate the number of packs. We then demonstrate in our paper that these substituted spatial models are ill-suited to capture mechanistic changes in abundance, such as mortality, unnecessarily add complexity, and introduce substantially more unrealistic assumptions, which were found to be violated and led to severe overestimation of abundance. iPOM’s area-to-number conversion (using territorial regions to determine the number of packs) comes with a whole host of assumptions, several of which are sensitive, which means if violated, abundance will be severely biased. Second, we corroborated the findings of other scientists who provided constructive critiques of iPOM. They indicated, among many issues, that spatial models used to monitor population dynamics are highly sensitive to the area covered by a mobile species. Intuitively, biologists should expect that the sensitive and restrictive assumptions of spatial models will be violated, especially given common biological knowledge of wolves, their social and highly mobile behavior, and how they should be sampled. Third, and perhaps the most devastating of iPOM’s problems, is the complete failure of iPOM’s developers to test the numerous assumptions, both explicit and implicit, contained in iPOM’s many models and submodels. Several of these assumptions are known to lead to substantial overestimation of abundance if violated. This last point demonstrates the unscientific nature of iPOM–it excludes or ignores the results and recommendations of dozens of previous and recent studies. We conclude that iPOM, for this reason alone, should not be used until iPOM’s assumptions are tested and results validated.

Instead, iPOM developers combined, rather than integrated, three main models: area-occupied (by wolf territories), territory-size, and pack-size. In the first, they decided to modify a spatial model, built initially to estimate a species' distribution, to delineate the spatial location and area (km²) included in each wolf territory across the state of Montana every year during the late fall hunting season, using surveys of untrained elk and deer hunters' recollections of where, when, and what they saw. Then, iPOM uses the second territory-size model to simulate the average territory size (km2) of wolves across Montana each year, without using any annual empirical data from Montana’s wolves and landscapes. Then, to determine the number of wolf packs, iPOM sums the area occupied by each assumed unique and singular territorial area (also assumed to be stable spatially and demographically–no shifts or mortalities) and divides it by the average annual territory size from a simulation model. In the last significant step, iPOM multiplies the number of assumed wolf packs each year by the output value of an outdated pack-size model that, again, does not incorporate annual empirical wolf data or dynamic landscape conditions that affect their movements and territory distribution during the late fall survey season. As an example calculation, suppose there were 60,000 km² of total area, summed from each wolf pack's non-overlapping territories across Montana, and that the average territory size was simulated at 600 km². Then 60,000 divided by 600 would equal 100 wolf packs in Montana. 100 packs multiplied by a pack size of say, 5, would result in an abundance of 500 wolves.

We found, as in other critiques, a serious problem with the iPOM's structural process, which we refer to as model incoherence. They claim their main models are integrated, even though they are actually linear combinations of models that are not independent of one another (an implicit assumption violation). In addition, because they are structured incoherently, they produce nearly constant values each year since 2012, despite many changes that occurred afterward, especially the increased human-caused mortality due to liberalized hunting regulations enacted in 2021. These problems are further exacerbated by the exclusion of the realistic factors, called covariates, known to affect wolf distribution and abundance during late–fall wolf surveys, including snowpack, deer and elk distributions, and forage availability. This departure from reality is surprising because decades of research clearly show that ungulate (deer and elk) prey populations, for example, are highly predictive of wolf abundance. These factors are also dynamic–changing seasonally and annually–as any hunter or biologist would know. Thus, iPOM’s exclusion of these known mechanistic factors renders it incapable of responding to management and natural impacts, such as climate and other landscape changes. Excluding dynamic (changing) environmental covariates compounds the existing bias in abundance estimates because iPOM already excludes annual mortality and dispersal information, which, in turn, are affected by those factors. iPOM also ingests constant values from its pack-size and territory-size models, which exclude annual factors of known change. iPOM also fails to account for the bias introduced by the direct and indirect impacts of hunters on wolf occurrence and distribution. These are the same hunters who kill and affect wolf distribution just before and during the late-fall hunter surveys, thereby violating the sensitive assumption of demographic and geographic closure that iPOM’s main model requires. In particular (and most troublesome), MTFWP cancelled a program to maintain a sufficient sample of radio-marked wolves from pack individuals, which would have enabled testing and correction of many of the biases we identified.

Previous research clearly indicates how sensitive iPOM’s spatial models are to the many problems, even minor ones, inherent in iPOM’s sampling methods, which aim to sample the area represented by observations of two or more wolves assumed to belong to and reside within a single, stable wolf territory. We noted that two previous critiques of iPOM are being ignored by its developers. The first was a technical paper written by an eminent wolf biologist at Montana State University (see WWP Wolf final), and the second appeared as a scientific opinion in the Bozeman Daily Chronicle, entitled “Guest column: Wolf management plan should be informed by science.” The authors of the latter were (1) the lead wolf biologist at Yellowstone National Park for 27 years, and (2) the senior-most wolf specialist for Montana, Fish, Wildlife, and Parks (MTMTFWP), who was a coauthor of the iPOM paper that we evaluated. They concluded that iPOM should not be used to determine wolf abundance and that alternative methods, such as genetic capture-recapture, should be applied instead. Recently, a federal court ruled in 2025 that iPOM produces uncertain and unreliable results, based on largely independent and qualitative grounds. Despite these criticisms, we proceeded to investigate and objectively evaluate iPOM. Yet, after we completed our evaluation of iPOM, we did corroborate the findings of the technical paper and the expert scientific opinion listed above.

From a high-elevation perspective, iPOM and its developers ignored the main lesson learned during the development and testing of methods for estimating wildlife abundance. For mobile, difficult-to-observe mammals, there are traditional, tried-and-true methods (as well as their recent modifications) that all rely on an adequate sample of individually marked animals or groups (e.g., a pack) to correct for the many known and recurring biases. In particular, these difficult-to-sample and mobile vertebrate species, such as wolves, require marking (radio collar, DNA, eartag, leg band) because raw, unmarked counts are often biased and misleading due to missing individuals that go undetected or, more commonly, duplicate observations of the same animal or group. Thus, marking individuals, whether done actively (capture and radio-collaring) or passively (DNA or natural unique markings), is used to identify and test for bias and, then, to correct for biases that, if ignored, lead to under- or overestimation of abundance. 

In iPOM, we found bias in nearly every tested assumption, some seemingly minor and others major, resulting in abundance being overestimated rather than underestimated. Because the presence of a wolf pack is easily detected by multiple signs (scats, observations, howling, kill sites, tracks, etc.), there is a low probability, if any, of what’s called a false-negative (a wolf pack is present but not detected). Instead, the common problems for wolves and most wildlife species are false-positive errors (detecting and reporting something present when it is absent). And the most insidious false-positive error that leads to severe overestimation of abundance is the multiple-counting of wolf packs and their territories among and within the sampled 600 km² grid cell ‘patches’ in iPOM. For example, highly mobile wolves from the same territorial pack can be counted multiple times in more than one grid cell during different weeks of the 5-week late-fall survey period. Moreover, a detection of a territorial pack in iPOM is defined as a group of two or more wolves observed and later reported by untrained hunters. This means that an average pack of eight, which often travels in smaller groups of two or more, can be counted multiple times (see Figure 2 below). Also, untrained hunters likely report errors in where, when, and what they saw during phone surveys after their unplanned, inadvertent sampling of supposed wolves. Such biases stemming from how iPOM uses hunter telephone surveys could have been avoided if iPOM had a sampling approach designed to prevent them, but unfortunately, it did not. We further explain these and other fundamental problems below.

Figure 2. A simplified schematic illustrating some of the problems with iPOM that cause overestimation of abundance. Four territorial wolf packs of undistinguishable individuals (no marks). Territorial boundaries are in red with the breeding adults in the center. The average size of a wolf territory is the same as the area of iPOM’s grid cell, 600 km2. As highly mobile unmarked wolves move during the late fall 5-week hunting season, untrained hunters can observe and record two or more wolves (or coyotes) of the same pack as belonging to two or more packs in more than one grid cell during any of the 5 weeks. Note that each of the three packs (solid lines) includes individuals that overlap along the boundaries in two, three, or four of iPOM’s grid cells. This could result in the recording of nine 600 km2 grid cells for only three packs. It could also cause wolf specialists, using indirect “signs” (howls, scats, kills, tracks) from one unmarked wolf pack, to record two or more territorial wolf packs where there is only one. The one pack with a dashed boundary depicts a “pack dissolution” in which a hunter killed an alpha breeder, and the remaining pack members dispersed to other grid cells, where they might be mistaken for another pack, yet no pack exists.

Biases found in iPOM’s models

We identified and evaluated iPOM’s biases in (1) their data input model, (2) each of their three main process models and their respective submodels, called covariate models, and (3) their variance model used to report a confidence interval. In addition to these evaluations of bias, we also found numerous other problems that limit valid inference and add to iPOM’s unreliability: mathematical and statistical analysis errors, the use of ad-hoc variables, model incoherence, a lack of data verification, circular logic errors, a lack of model validation, and the inability to reproduce iPOM.

Like any data model, iPOM’s is populated from a sampling process. These sample data are ingested into iPOM’s three main process models to determine the values of its unknown variables. The problems we found with this data model process are included in a largely non-technical section 4 entitled “Input data and sampling design” of our technical publication. We discovered that iPOM essentially lacks a sampling design because it did not include the criteria commonly used to develop and plan a proper sampling design: selecting appropriate spatial and temporal scales, incorporating information relevant to the time and place, aligning sampling units and efforts with observational and ecological processes, considering sample size and effect size, and accounting for inherent environmental heterogeneity of highly mobile wolves traversing complex landscapes/habitats. Second, their data model is incoherent with the process models because annual empirical data samples from Montana’s wolf population were not collected and thus were not used in their three process models (area occupied, territory size, pack size) used to predict abundance each year. Despite this breach of scientific inference, iPOM, however, uses two empirical inputs annually for determine the area occupied by wolf territories: (1) phone surveys of hunters’ recollections of where, when, and what they observed, and (2) the hand-marked centers of individual wolf territories made by wolf specialists based on indirect sign they found (howls, scats, kills, tracks, and talking with locals). Both of these empirical data sources are indirect and subjective and were not verified. Further, why would one sample deer hunters’ inadvertent recollections of supposed wolf observations to make inferences about wolf populations, instead of sampling wolves directly? Also, how can one determine the center (centroid) of an individual wolf territory when no data are collected on the movements of individuals within a known territorial pack (e.g., tracking radio-collared wolves)? 

We then identified and tested the sensitive and vital assumptions of the first main process model (with covariate submodels) called area occupied (AO). It estimates the area occupied by individual wolf territories across Montana using a dynamic occupancy model that corrects for false-positive errors and allows closure violations between years but not during the 5-week late-fall survey period. We tested sensitive assumptions and identified violations across three major types of biases. Two of these, closure violation and false-positive errors, were violated in multiple ways, and both result in severe overestimation bias, a phenomenon repeatedly cautioned against in the scientific literature. The remaining bias type, called resolution bias (the sampled ‘patch’ grid size was too large), is due to observation problems of highly mobile wolves moving in adjacent grid cells, or what some scientists call the “edge effect” (Figure 2 above). We simulated the effect of grid size and found that iPOM needs to sample at smaller grid cell sizes to better correct for resolution bias. Our simulation findings were consistent with those for wolves in Wisconsin and with the developers of occupancy modeling (used for ‘area occupied’ in iPOM).

The three leading causes or ‘culprits’ that cause assumption violation and drive the three major biases that inflate (overestimate) the area occupied by wolf territories, which subsequently overestimates wolf abundance, are: (1) substantial mortality of wolves and the susequent dissolution of territories just before and during the late-fall hunter survey, (2) three ways iPOM double- or triple-counts wolf packs because it uses untrained hunters and subjectively determinating centroids–delineating territories where none exist, and (3) deficient and static covariate models used to potentially correct for bias. The closure assumption means that both pack animals and the territories they occupy are demographically and geographically stable during the survey period. This means no movements outside of their territories, no mortality of individual pack animals, and no dissolution of territories, which happens about one-third of the time when a breeding adult is killed. When a pack dissolves, dispersal of the remaining pack members further amplifies assumption violation and over-estimation bias because they can be counted as pack observations and additional territories (false positives) as they move across multiple grid cells (Figure 2 above).

Movement (violation of geographic closure) mixed with misidentifcation of wolves further adds to false-positive errors because untrained hunters might mistakenly reported (1) two adult wolves for coyote pairs, (2) large wolf pups traveling outside of their natal territories in late fall excursions, (3) a small group of non-territorial wolves (left the territory), and (4) packs and/or subgroups of packs that are counted two or more times (double- and triple-counting) due to their mobility during the ‘closed’ 5-week late-fall survey season. The critical and sensitive closure assumption means that territories are spatially stable (with no major shifts or dissolution), that wolves are not killed, and that pack animals traveling in groups of two or more are observed within their territories. We hope that future development and improvements to iPOM do not continue to ignore testing assumptions, especially critical and sensitive ones that, if violated, lead to severe overestimation of abundance. We further hope they take the many recommendations we have made for improvement.

One assumption violation was surprisingly sensitive: no errors (e.g., false-positives) in their “centroid” model, which uses a 'confirmed' sample of known territories and is used in their area-occupied model to estimate the area of all territories in Montana. Several scholarly publications by top quantitative biologists demonstrate that errors as small as 1% to 10% in the ‘confirmed’ singular territories cause severe over-estimation. We do not see how MTFWP can confirm or determine known and verified individual territories after they stopped radio-collaring a sufficient sample of wolves to satisfy this critical confirmation step, which is provided by a second independent method. Instead, MTFWP relies on indirect signs (howls, scats, tracks, kills) collected over an unspecified period and then claims to delineate unique wolf territories (different from adjacent territories) with ‘hand-marked’ territorial centroids. This collected “sign” from any number of wolves belonging to multiple pack individuals or non-pack individuals does not constitute a set of confirmed or verified individual territorial packs. Not surprisingly, we demonstrate that they double- or even triple-counted wolf territories. This is also a circular logic error that occurs when one makes a statement or conclusion that is synonymous with a premise but provides no evidence to support it.

For their other two main models, territory size and pack size, we found numerous assumption violations and analysis errors that, for both models, directly cause over-estimation of wolf abundance. We refer the reader to those sections in our publications for more details. Similarly, each of the main models used covariate models that attempted to correct for bias. In some cases, they were eliminated. Furthermore, in all three main models, the covariate models were deficient and/or (1) excluded known causal variables, and (2) included fixed covariates that resulted in constant output when dynamic variables (that change each year) were eliminated. This renders their covariate models, built to correct for biases like false-positives, insufficient, or incapable of doing so. For example, they excluded causal factors (assumption violation) known to account for wolf distribution and movements in late fall (e.g., snowpack, prey availability, and forage for prey). These biases are covered in more detail in our publication, along with the other problems we listed above.

The precise amount, or magnitude, of overestimation bias could not be precisely determined because, upon request, iPOM developers did not provide the data or analysis results needed. Our testing, however, demonstrates with certainty that iPOM overestimates wolf abundance, but one cannot say by how much. We do suspect it is overestimated by around double (2x) the actual number. We were, however, able to determine the magnitude of the bias in iPOM’s variance, reported as a confidence interval. We found a severe bias of at least 8x (750%), indicating that iPOM cannot detect abundance changes. Again, this means that if iPOM is used, it leaves biologists, managers, and conservationists in the worst-case misleading situation (see Figure 1), because FWS claims unbiased (accurate) and precise estimates, when in fact, wolf abundance is overestimated and imprecise.

We found circular logic errors in each of the main models. And it is essential to remember that circular logic is a fallacy that is antithetical to science. It can also be harmful because it systematically undermines truthful, reliable reasoning and decision-making. They distort how conclusions are reached, hide weak or unsupported claims, and make persuasion appear more substantial than the underlying evidence. Providing evidence to support a premise is similar to testing an assumption with proof, and both are implicitly assumed to be integral to advancing science. Even the claim that iPOM produces accurate (unbiased) abundance estimates can be considered circular logic.

Implications 

Many of the biases in iPOM’s models can be corrected using marking techniques; otherwise, overestimation biases will persist, including multiple counting, false-positive detections, and multiple sources of closure violations. Human-caused mortality just before and during the survey amplifies the main, sensitive biases. For example, in the early years of iPOM, a sufficient number of individuals were radio-collared and tracked to confirm (a strict assumption) that it belonged to a unique and separate territorial area from those adjacent territories. If not, packs that typically travel in smaller subgroups within and beyond territorial boundaries are easily double- or triple-counted, leading to a severe overestimation bias. With individual markings, one could correct for the violation of the mortality assumption during the survey. This sensitivity to these types of false-positive errors is severe and has been documented several times in the scientific literature by leading quantitative ecologists. Remember also that testing assumptions and correcting for bias is also how we, as a society, recognize false claims, including errors of commission and omission, and combat misinformation and disinformation campaigns.

The implications of the biases we identified are even more severe because of how MTFWP uses iPOM in wolf management decisions in Montana. iPOM estimates of wolf abundances set hunting quotas, guide decisions to maintain wolf populations at a given level, and are used to list and delist wolves under the Endangered Species Act (ESA). In addition, the estimated wolf abundance from iPOM is used to project or forecast the impacts of factors such as hunting, trapping, poaching, climate change, disturbance, and habitat alteration on wolf populations. Without a reliable (unbiased and low-variance) estimate of wolf abundance each year, the MTFWP is flying blind, as the projection models they use start with an artificially high value, which means wolves can decline to a critically low population level without being detected. If the population size of wolves (abundance) is unknown or highly uncertain because the method to determine abundance is unreliable or invalid, then the result of any model that uses it would simply be an exercise in futility. Such a situation is the worst-case scenario for managers and decision-makers. This misleading set of biases poses a high risk to wolves because MTFWP claims that iPOM produces an accurate (unbiased) and precise (low variance and tight confidence interval) estimate, when in fact, we found that iPOM overestimates the number of wolves with high variance (uncertain and imprecise) to the point that it cannot detect a change — all this without assumption testing, data verification, and validation.

This situation with iPOM can negatively affect more than just wolves, because responsible, stewardly agencies consider both the ecological and economic value of managed species in maintaining a healthy environment. For gray wolves, the best available science indicates that they have high value and a relatively small negative (but economically important) impact on livestock. In fact, local economies benefit from wolves in numerous ways, and they play a crucial role in maintaining healthy ecosystems that humans also rely on. Direct benefits include the suppression of overabundant elk, deer, and coyote populations which cause damage to livestock and crops, maintaining healthy game populations by culling out the weak and suppressing disease from spilling over from wildlife and humans, restoring vegetation which helps water quality songbirds and insect pollinators, making roads safer by reducing deer-vehicle collisions, and generation of cash and jobs (80+ million dollar economy around Yellowstone). Naturally formed larger packs, those that are not killed, enhance these benefits. Overall, wolves play a disproportionately large role, being at the top of the food chain–not even a grizzly or a polar bear can outcompete a pack of wolves. Even small changes in the relatively few species at the apex have significant effects on species and processes further down the food chain, all the way to grass and forb production.

We understand the need to reduce expenses, even though wolves are of high economic importance to the general public, to ranchers who lose livestock, and to the ecosystems and quality habitats that Montanans rely upon. However, in achieving one of its goals–to reduce expenses–iPOM ensured the failure of its primary goal: to provide accurate, reliable estimates of wolf abundance. This is tantamount to ignoring a valuable opportunity cost in economics. And that’s why we recommend a method commensurate with the economic and ecological importance of wolves, and why we highly recommend using a marking technique, such as DNA, in a process known as genetic mark-recapture. It foregoes the high expense of physically capturing, marking, and releasing individual wolves and instead identifies unique DNA marks in their scats, which are readily “captured” by collecting them on the ground.

If you choose to read this essay and the URL-linked technical publication, please keep in mind that, regardless of their goals, the most troublesome aspect of iPOM has been their unwillingness, so far, to address and acknowledge problems with iPOM and work together to solve them, particularly in testing assumptions both biologically and statistically. This testing takes little, if any, additional expense. From the time of their 2022 paper introducing iPOM to today, we have yet to see an approach grounded in the foundational tenets of science–the ability to make valid inferences and gain reliable knowledge. iPOM developers and users are capable of producing this, mainly if they focus on model coherency: a designed harmonious flow of information and testing through these steps (1) adopt an unbiased sampling design to collect empirical data from the wolf population sampling units every year, (2) build a verifiable data model for input, (3) choose the correct mechanistic variables and covariates in their models, (4) proper ingestion of annual sample data into models with data-hungry variables, (5) make annual predictions, (6) report the proper variance and confidence intervals, (7) validate their models, and (8) based on assumption testing make annual adaptive adjustments to reduce bias further. Thus, at every step of this process, assumption testing is vital and the only way to identify and reduce bias, thereby gaining reliable knowledge to manage wolves effectively and make valid predictions. We hope to see the developers of iPOM acknowledge or address the problems we have identified at each of these steps, particularly by ceasing their unwillingness to identify and test the dozens of assumptions underlying iPOM's complex models and submodels that we examined. There is a famous quote by Nobel Laureate Max Planck: “Science advances one funeral at a time,” which means that resistance to change is the antithesis of scientific progress.

Read the full publication here:

Misleading Bias in iPOM_Crabtree Conner Treves_2025.pdf

Download PDF • 1.02MB