Individual-Based Model use in Marine Policy

ABSTRACT

Individual-based models (IBMs) are increasingly used in marine conservation research, making this is an ideal time to assess IBM use in marine policy. IBMs can contribute important information to marine management and policy questions, as they offer complex methods of understanding ecosystems and animal behaviour, by allowing for heterogeneity in both individuals and environments. A review of 108 international peer-review publications utilizing marine IBMs was conducted using Web of Science (WoS). It was determined that 55% of the WoS articles claimed that the IBMs were relevant or important to marine conservation policy or management. A relevant English-language policy document was located for 83% of the IBMs, but only 32% were cited, while 85% of the same policy documents cited a different, non-IBM, modelling method. A separate survey of 175 policy documents from the Government of Canada was conducted. Of the 60 that contained citations, zero documents cited an IBM, while 75% cited a different modelling method. Of 407 webpages reviewed from the National Oceanic and Atmospheric Administration, the New Zealand Department of Conservation, and the UK Government website, only 4% referenced IBMs. This research demonstrates that, despite claims of usefulness by researchers, IBMs are not used to inform policy, while other model methods are commonly cited. Modellers should not assume that their model will inherently be useful for policy and should instead ensure that they are: 1) addressing a policy need; and 2) making the information accessible to policymakers by crafting a communication plan and/or joining a relevant boundary organization.

KEYWORDS:

• Marine
• policy
• individual-based Model
• simulation
• conservation

Introduction

Environmental policies are improved with the input of scientists (Meyer et al., 2010). Research has shown that policymakers feel that scientists should take a more active role in policymaking, beyond simply reporting results (Steel et al. 2004; Akerlof 2022). Researchers can act as “knowledge brokers”, though this requires a balance between maintaining scientific credibility while also contributing to the policymaking process (Nelson and Vucetich 2009; Turnhout et al. 2013). However, surveys of scientists who aim to conduct applied research, or research conducted for a specific application (i.e. conservation), show that researchers rarely present to governmental/agency meetings, nor do they regularly work directly with conservation managers (Thornhill 2014; Akerlof 2022).

Models are necessary and effective tools for understanding present and future environments and should be utilized along with other forms of research in conservation policy. Complex systems require complex adaptative modelling, which allows for the interactions of multiple phenomenon (Squazzoni and Boero 2010; Bruce and Gershenson 2015). Individual-based models, which simulate how micro level behaviour in a system creates macro level behaviours, are particularly useful for understanding complex ecological systems (Squazzoni and Boero 2010). When models are focused on conservation and management, in particular of threatened, endangered, or exploited species, it is prudent that models be developed with policy implications in mind. This research focuses on individual-based models (IBMs) and marine conservation policy and management. As IBMs are becoming more prevalent in the field of marine conservation, this is an ideal time to assess use in policy and assess communication and implementation strategies.

The term “model” refers to many different methods of artificially representing the world, from simple sketches, to complex, multi-level computer models. While many assume models are inherently mathematical, models are also used in informal, often implicit ways to explain the world. It is usually only scientific models that are documented, with assumptions and biases made explicit (Epstein et al. 2014). Models are also used as diverse tools, not always for prediction but for understanding as well (Epstein et al. 2014). Models are tools of both scientific research and policy, as they can be used to answer scientific questions and/or to test the impact(s) of potential policy decisions (Paolisso et al. 2015). They are also used to inform many different issues, ranging from economics to public health to fisheries (Railsback and Grimm 2019).

When scientific publications assert the usefulness of a model for ecological management, there is not always evidence that the model has been utilized in policy, or if the impact of a model on policy development has been documented (either in a published paper or elsewhere, such as a government report). Methods of communicating the model results to relevant policymakers are rarely described in peer-reviewed publications. Furthermore, strategies for model implementation and/or information for policymakers is virtually never present in these publications or model documentation.

Modelling has been used in ecological conservation for decades, with varying levels of success (Béland & Howlett, 2016). Traditionally, these models were equation based (i.e. Gordon 1954). Individual-based modelling (IBM)¹ is a type of modelling that allows for bottom-up modelling, with populations that include individuals that are heterogenous, with complex, individual life cycles, and describe population changes in number of individuals, rather than density (Uchmański and Grimm 1996). IBMs also allow for the consideration of how individuals interact with and affect, or are affected by, their environment (and are often, but not always, spatially explicit) (Uchmański and Grimm 1996; Grimm and Railsback 2005). With the rapid increase in computing power, it is likely that IBMs will continue to grow in complexity and applicability.

Because of their nature, IBMs may be able to contribute important information to conservation management and policy questions, as IBMs offer more complex methods of understanding ecosystems, animal behaviour, and resource and population dynamics, by allowing for heterogeneity in both individuals and environments (while traditional models often assume uniformity or the “average of” individuals or environments) (Uchmański and Grimm 1996). This allows for a stronger understanding of patterns at higher population levels (Squazzoni and Boero 2010). However, the degree to which IBMs are actually used in conservation policy is unclear, as is the extent of IBM availability to address policy-relevant questions. The focus of this paper is to shed some light on these questions, with a particular emphasis on marine conservation. Here, marine is defined as relating to marine conservation and management, including, but not limited to, species and habitat conservation/management and anthropogenic impacts to species and habitats (i.e. fishing, climate change).

Overview of model use in policy

The definition of “model use in policy” can be broad, including simply communicating model results to the target audience, training users, and one-off or routine use of the model for policy development and analysis (Kolkman et al. 2016). This paper will focus on IBMs used for policy development or management. While there are many examples of traditional mathematical models being used in environmental policy, there is not currently an exhaustive list of models used in policy (Kolkman et al. 2016), making a comprehensive assessment difficult (Chang et al. 2021). Still, there are many documented examples of environmental models informing policy, such as Regional Air Pollution INformation and Simulation (RAINS) model (Tuinstra et al. 2002), Climate Options for Long Term (COOL) Project (Tuinstra et al. 2002), and Physical Habitat Simulation System (PHABISM) (Cartwright et al. 2016).

While traditional mathematical and ecosystem models are well accepted in conservation and policy development, IBMs are less frequently utilized, despite their potential for stronger, more dynamic, conservation impacts (Codling 2008; Islam and Jørgensen 2017; Ortiz and Jordán 2021). IBMs can be spatially explicit and include realistic, complex elements, such as bounded knowledge (incomplete and different levels of access to information by different actors), something that may be important for the conservation of migratory species (Hare & Deadman, 2004).

One area conservation could learn from is the public health sector, where various types of models have long been utilized in public policy (Pullin and Knight 2003). IBMs are commonly used in human health (Milne et al. 2008; Gojovic et al. 2009; McBryde et al. 2020; Lorig et al. 2021), and epidemiological IBMs have been applied successfully to disease outbreaks in animal populations (Eisinger et al. 2005; Railsback and Grimm 2011). They have also been applied to human behaviour with regard to climate change (IPCC 2022).

IBMs are becoming more popular in fisheries research (García-Asorey et al. 2011; Childress 2014). As of 2021, InSTREAM (individual-based Stream TRout Environmental Assessment Model) has been used for 22 years and applied to 50 sites in multiple countries (Railsback et al. 2021). IBMs have allowed researchers to understand fish behaviour, with potential impacts for conservation policy (Railsback & Harvey, 2002), and have allowed researchers to understand anthropogenic impacts on fish more holistically (Grimm and Railsback 2005).

Purpose

In this research, we looked at marine IBMs and assessed evidence of their use in marine policy. Marine IBMs were focused on due to the common use of various model methods (including IBMs) in fisheries science, among other marine-related topics, as well as the applicability of IBMs to marine research, due to the complex, migratory nature of many endangered, threatened, and exploited species. The limited focus on marine research also allowed for a more manageable data set that is more in line with the authors’ expertise, as IBMs with policy relevancy span across all disciplines. While model use in policy for human health issues, like epidemiology, is well documented, it is less well documented in marine science, making this research a novel contribution to marine science and policy literature.

Furthermore, focusing on marine policy allowed us to find policy documents from a larger source pool, as many marine organisms cross international boundaries, meaning multiple countries could publish policy reports on the same species or marine conservation/management issue. This was especially vital for policy documents, as not all countries have searchable or accessible online databases of policy information.

The purpose of this research is to 1) assess the number of marine IBMs that are published in scientific publications and claim relevancy to policy and 2) assess the rate of individual-based model use in marine conservation policy. Use is being assessed by citations or references to IBMs in policy documents. This limits the source pool for policy documents, as many policy documents do not contain citations and a significant amount of scientific information is shared in informal meetings. However, this method was chosen as it is a documentable and quantifiable method of analysis.

Methods

This research consisted of three components:

1. Assessing the number of scientific papers that use IBMs to address marine topics and claim the results are relevant for policy, as well as a determination if the scientific papers were cited in any policy documents related to the research topic.

2. Assessing the number of policy documents from the Government of Canada Publication website that cite IBMs and other model methods.

3. Searching generally for any references to IBMs on the following websites: National Oceanic and Atmospheric Administration (NOAA) Fisheries publications page (which includes published research, reports, and outreach materials; https://www.fisheries.noaa.gov/resources/all-publications), NOAA fisheries general website (https://www.fisheries.noaa.gov), the New Zealand Department of Conservation (https://www.doc.govt.nz), and the UK Government website (https://www.gov.uk).

Scientific papers

To assess the number of scientific papers that utilize IBMs and argue that the results are useful for marine conservation policy or management, Web of Science (WoS) was searched. WoS is a global citation database for all indexed journals, which searches the title, abstract, and indexing (i.e. keywords) of its database. The scientific papers assessed were written in English, but the WoS results included research on marine areas in South America, North America, Africa, and Europe.

Two keywords searches for Individual-Based Model, Marine, Ecology, and Individual-Based Model, Marine, Conservation were conducted (WoS search included “or” IBM, ABM, and Agent Based Model; See Appendix A, Appendix Table A1 for the search criteria). Results were narrowed to peer-reviewed publications. Results for all searches were combined and duplicates removed (Table 1).

Table 1. Web of science search results.

Database name	Search terms	Article hits	IBM	Marine IBM
Web of Science	Individual Based Model* Marine Conservation Marine Ecology	621	326	108

Web of Science (WoS) results for “individual based model + Marine Conservation” and “Individual based model + Marine Ecology”. Due to a high number of duplicates, all WoS results were combined and duplicates removed. Searches were narrowed to only include scientific papers that used IBMs in relation to marine issues. WoS indexes the title, abstract, and keywords of each article. *Agent based model, IBM, and ABM were also searched (See Appendix A, Appendix Table A1 for the search criteria).

Papers were chosen for review if they fit the following criteria:

• Utilized an individual-based model in the methods.

• Were related to the marine environment. Papers relating to sea birds, marine mammals such as seals, and anadromous fish were included in this study, as was research on tourism, climate and other anthropogenic impacts on marine species/habitats.

The number of papers that stated that the research had management or policy implications and/or that were beneficial to management or policy were counted. This was assessed by context; the authors did not need to state explicitly that the paper was “beneficial to policy”. For example, a paper that assessed the effectiveness of a fishing quota would be considered as having management or policy implications, without having stated that outright (see Supplementary table S1 for which abstracts qualified, and which did not). Only abstracts were read for this study, under the assumption that if this was a key takeaway of the paper, it would be in the abstract. The WoS search was conducted in January 2022.

For each paper that claimed relevancy for policy, a search was made for a policy document published on the same topic, after the publication date of the scientific paper. A policy document was defined broadly as any document written by or for any governmental agency, reporting information meant to inform marine policy. This could include, but were not limited to, management reports, population estimates of protected or exploited species for the purposes of management, best practices for mitigating anthropogenic impacts, policy recommendations published by or for governmental agencies and technical reports published by or conducted for governmental agencies. Only policy documents with citations were included in this component. For the purpose of this study, sources were counted as a “citation” even if they were informally cited (i.e. a footnote with a simple URL), though the majority were written in a formal reference lists.

Search methods included copy and pasting the entire (APA) citation and the paper title into Google, searching a relevant agency or organization website for policy documents on the topic, and searching Google for terms such as “[marine topic] management”, “[marine topic] conservation” or “[marine topic] plan” in order to find policy and management papers on the same conservation issue referenced by the scientific paper. For example, to find a policy document relevant to research on the impacts of fishing on spiny lobster populations in the Florida Keys, we would search key terms such as “spiny lobster Florida Keys”, “spiny lobster conservation” in Google. When possible, government and agency databases of the country(ies) relevant to the scientific paper were searched. For example, in conjunction with the google search we would search the Florida Fish and Wildlife Conservation Commission website, along with any other relevant databases (i.e. NOAA’s website), for information on spiny lobster.

If a policy document was found, it was then determined if it cited the WoS scientific paper or if it cited a different IBM. While the term “individual-based model” is used here for consistency, all documents were checked for references to either individual or agent-based models. Policy documents were also assessed to determine if they cited any other type of modelling method. This was done by searching for all references to the word “model” within the document and determining the model type. Determining the model type was done by contextual clues within the text (i.e. the model method was stated explicitly) or by checking the model methods of the cited source (the term “individual-based model” did not need to be used explicitly to be identified by this method). Both the text of policy documents and the references list were assessed, however searches of the reference list relied on the term “model” being used in the title of a cited paper. If the specific model method was not included in the title, the paper was reviewed to determine the method. This method was repeated for the terms “simulate” and “simulation”. Searches for policy documents related to the WoS articles were conducted in January and February 2022.

Government of Canada publications

Because of a lack of comprehensive, international databases for policy documents, there was no method of conducting a thorough, systematic search of international policy documents comparable to the WoS search. The Government of Canada has an extensive searchable policy database (https://publications.gc.ca), which consists only of Governmental publications, as opposed to general webpages, employee biographies, or other general governmental information. This was used to conduct a search for policy documents analogous to the WoS search. This was chosen specifically because of the quality and centralized nature of the search database, as well as the fact that Canada has two large coastlines. All documents found in this search were related to Canadian policy, and only those with citations were included.

The Government of Canada Publications database was searched using the terms “individual-based model” and “marine conservation” (Table 3). The terms were entered into the “basic search” which searches the text (title, subtitle, series title, subject terms,² abstract, author, department/agency) of documents. The option to “find variations of search terms” was checked. Searches were sorted by relevance, in descending order. The number of articles with citations was counted. The written text and citations of these policy documents were assessed using the same method as the other policy documents to determine if an IBM had been referenced or cited.

Table 2. Assessing model use in policy.

Marine IBMs that claim policy relevance	Marine IBMs that were published before a relevant policy document	Number of Policy Relevant papers published after Marine IBMs	Number IBMs cited in a policy paper	Number of Policy papers that cite an IBM	Number of Policy papers that cite a different modelling method
59*	49	52	16	25	44

49 of the marine IBMs were published before a relevant policy document was published (some IBMs address the same species or conservation issue and would have been relevant to the same policy document). Nine policy documents cited two or more IBMs (including an IBM identified by the WoS search) and four policy documents cited an IBM not identified in the WoS search.

*Supplementary table S1 shows which line in the abstract demonstrated policy relevance.

Table 3. Search of Government of Canada’s publication database.

Database	Search term	Total Article hits	Articles Assessed	Articles with citations	Articles that cite IBM	Articles that cite other models
Government of Canada Publications	Marine Conservation	480	150*	54	0	39
	Individual Based Model	25	25	6	0	6
Total	-	505	175	60	0	45

Only documents with citations were assessed. Out of a total of 175 policy documents, 60 contained citations. Out of those 60, 0% cited an IBM and 75% cited a different modeling method. *Of 480 results, 150 (30%) were assessed..

Searching individual based model in the Canadian Government database returned 25 results, only six of which were marine relevant policy documents with citations. A search for marine conservation in the Canadian Government database resulted in 480 results. Due to the low results for individual based model, only the first 15 pages of search results for marine conservation were assessed (a total of 150 documents, or 30%). The search of the Government of Canada database was conducted in February 2022.

Other websites

More general searches for mentions of IBMs in governmental websites were conducted in February and March 2022 to supplement the geographic limits of the Canadian Publication search. This more ad hoc method, which is not meant to be directly comparable to the previously described searches, included websites, employee biographies, and other more generalized information on government websites that may include information about IBM use in policy. Due to the diversity of publication types found, documents and webpages in this search were not required to have citations but were assessed solely for any mention of IBMs. This included workshop listings, employee publication lists, informational pages for the public, and press releases, along with formal policy and management documents.

The NOAA general website and New Zealand Department of Conservation did not have a separate search page or advanced search that allowed for narrowing results to policy publications (as opposed to webpages, event announcements, employee biographies, and other non-formal media). While the NOAA publication page and UK databases allowed for narrowing by document type, both contained small sample sizes for most searches, and so were combined with the general websites in data analysis.

UK searches were run by narrowing results to “Topic: Environment”, “Subtopic: Marine” and searches were conducted by further narrowing to “Content type: Policy documents and Consultations” via the menu options provided in the search (Table 4). This was done to reduce search results to more manageable sizes and to exclude non-marine pages, as the UK government website indexes pages from all UK agencies. Narrowing the search criteria ensured only relevant pages were indexed. The NOAA publication search was not narrowed in any way, as it is already a marine focused agency.

Table 4. Search results from Government websites.

Database	Search term	Article hits	Articles that mention IBMs
NOAA Fisheries	Individual Based Model	166	3
	“Individual Based Model”	6	6
	Agent Based Model	18	2
	“Agent Based Model”	3	3
NOAA Fisheries Publication	Model	45	0
	Individual Based Model	0	0
	Agent Based Model	0	0
New Zealand Department of Conservation	Individual Based Model	60	0
	“Individual Based Model”	0	0
	Agent based model	13	0
	“Agent Based model”	0	0
UK Government Publications	Individual Based Model	22	0
Search narrowed to “Topic: Environment”, “Subtopic: Marine” And “Content type: Policy documents and Consultations”	Agent based model	18	1
UK Government Publications	Model	55	0
Search narrowed to “Topic: Environment”, “Subtopic: Marine”	“Individual Based Model”	0	0
	“Agent based model”	1	1
Total	-	407	16

A general review of five government websites from the United States, UK, and New Zealand. Webpages were assessed for formal or informal mentions of IBMs. These included formal citations, workshop advertisements, employee biographies and other general informational pages. Four percent of all searches returned a reference to an IBM. Quotation marks indicate when searches included quotes around the phrase. Duplicate results were not removed from the total..

The four websites were chosen because they a: had a high-quality search function, b: had a high volume of centralized documents/webpages, and c: are all from nations that have a marine coastline and for which the marine environment is economically important. For all websites, the search terms Model, Individual Based Model and Agent based Model were used. When appropriate, searches with and without quotation marks were conducted in order to narrow results. For all sites but NOAA, no difference in search results was found when Individual based Model or Individual-Based Model; Agent-based Model or Agent based Model was used. When searching NOAA databases, no dash was used as it returned a higher number of results. For example, agent-based model returned three results (The same results as when using quotation marks), while agent based model returned 18 results.

Results

Scientific papers

The WoS search produced 621 scientific papers in total. Of those, 185 were not an IBM and 328 were IBMs that were not marine-related (Table 1). These were dropped from the study. A total of 108 marine IBMs were found, 55% (59 papers) of which claimed to have policy relevancy (Table 1, Table 2, Supplementary table S1). A total of 52 relevant policy documents were located that were relevant to 49 (of the 59) scientific papers (Table 2). Note that multiple policy documents and scientific papers were published on the same conservation topic (i.e. there were multiple scientific publications that used IBMs to address sea turtle conservation, all of which were potentially relevant to the same sea turtle management reports). Policy documents that were relevant to more than one scientific paper are listed with each scientific paper but are marked as duplicates in Supplementary table S1. Duplicate policy papers were only counted once for the policy document total.

Of the 49 scientific papers that claimed policy relevance and had a relevant policy document, only 16 (32%) were cited in a policy document. Forty-four of the policy documents (85%) cited a different modelling method³ (species distribution models, traditional mathematical population models, and non-IBM ecosystem models and climate models were common) but did not cite the WoS marine IBM nor a different IBM. Eight policy documents cited a scientific IBM that did not appear on the WoS Search. Five of those eight policy documents cited both the WoS IBM and an additional IBM. Four policy documents cited a single IBM, but one that was not identified by the WoS search.

Government of Canada publications

The earliest policy document identified in the search was published in 1990, with the majority being published post-2000. Of the first 150 results, 60 contained citations. 54 of those 60 were a marine related policy document. Out of a total of 60 papers with citations, none referenced IBMs, but 45 (75%) cited other methods of modelling⁴ (Table 3).

Other websites

NOAA fisheries

A search in the NOAA Fisheries website for “model” returned 1540 articles. Due to the high volume, these results were not analysed. Individual based model returned 166 results, only three of which referred to an IBM (Table 4). Many results used the word “individual” to refer to “individual age class” or “individual stock assessment” or similar phrases. This is because a phrase without surrounding quotes will return search results for each word individually. A search for “individual based model” (in quotes) returned six articles, all of which referenced IBMs. The use of quotes means the search will only index pages that use the entire phrase. Agent based model resulted in 18 articles, two of which referred to ABMs, and the same term in quotes resulted in 3 articles, all of which referred to an ABM (Table 4). All three of these were employee information pages (biography and publication list).

A search of the NOAA Fisheries publication page for model returned 45 articles, none of which referred to IBMs. individual based model and agent based model (with no other refinements) returned zero results (Table 4).

New Zealand department of conservation

A search for model in the NZ Department of Conservation website resulted in 389 articles (Due to the high volume of results, individual pages were not assessed). Individual based model resulted in 60 articles, but no reference to an individual-based model was found. “Individual based model” (in quotes) resulted in zero article results. A search for agent based model resulted in 13 articles (none of which referred to ABMs) and the same term in quotes resulted in zero articles (Table 4).

UK government

When search results were narrowed to “Topic: Environment”, “Subtopic: Marine” and “Content type: Policy documents and Consultations”, individual based model returned 22 article results (none of which referred to an IBM), while agent based model returned 18 results, with one reference to an ABM.

When search results were narrowed to only “Topic: Environment”, “Subtopic: Marine”, Model returned 55 article results, none of which referred to an IBM. “Individual based model” returned zero article results, while “gent based model” in quotes returned the same single article found in the non-quotation search (Table 4).

Discussion

A majority (85%) of the marine IBMs identified by the WoS search were not cited in policy documents, despite 55% of those IBMs stating that the findings were relevant to policy or management. Notably, other model methods were in high use and often referenced/cited directly, as shown by the fact that 44% of policy documents relevant to WoS IBM and 45% Government of Canada publications cited a different modelling model. The number of references to IBMs on the general website search was also very low (4%).

This indicates that agencies are using results from non-IBM model methods to inform policy but are either uninterested or unaware of the results from relevant IBMs. One UK document, a 2013 summary of the scientific evidence used to inform UK marine policy (Marine science Ninth Report of Session 2012–13 2013), even contained a review of acceptable model methods but did not mention IBMs. It is largely unclear why each policy document included the model methods (or other scientific evidence) that they did, nor is it clear what evidence may have been reviewed, but ultimately left unused by policymakers. Future research on policymakers may shed light on this decision-making process.

This research demonstrates an effort by governments to be transparent with regard to sources of information for policy development. A common complaint among scientists is that it is difficult to overcome the “evidence-policy gap”, and ensure that evidence, produced by scientists, is used by policymakers to inform policy. This application gap is well documented with regard to modelling (Mcintosh et al. 2008; Syme et al. 2011; Kolkman et al. 2016; Chang et al. 2021), but it is not clear that this research demonstrates that. A substantial percentage of the policy documents surveyed contained citations, specifying the source of the scientific information used to inform policy. However, this research also demonstrates the challenge of finding a coherent list of policy documents. Few governments have databases of policy and technical documents, searchable by topic, document type, or agency, something the Government of Canada provides. If other agencies adopted these database methods, it would increase transparency between government policy and the public.

It is important to note that policy is developed through more than just “scientific evidence”, as policymakers have more diverse external pressures (pressure from donors, voters, political parties, etc.) and/or different goals than researchers, who may also have their own policy goals (e.g. sustainable resource use) (Cash et al. 2003; Cairney and Oliver 2017; Cockrell et al. 2018; McConnell and Hart 2019), a fact demonstrated in marine policy development (Koehler and Lowther 2022). It is also not desirable to have policy formed solely on the basis of a model’s output, but to include a model as one piece of scientific evidence (Süsser et al. 2021). This was observed in the policy documents assessed in this study, which always contained multiple research citations. Many policymakers are not able to access up-to-date scientific research (often published behind a paywall or written in confusing jargon), meaning they cannot use it to inform management decisions, regardless of their desire to do so (Pullin and Knight 2005). Policymakers rarely have enough time to keep up to date on various topics and may instead rely on scientists sharing the results of their research, though scientists are not always skilled at communicating to policymakers, nor do they always actively try to or have the time to do so (Akerlof et al. 2018; N.; Rose and Parsons 2015; Thornhill 2014).

Some of the WoS IBMs may not have been keyed into the policy needs of conservation managers or policymakers, an issue documented in previous research on model use in policy. For example, Lorig et al. (2021) noted a discrepancy between what policymakers needed and what many COVID-19 simulation models tested (Lorig et al. 2021). Collaboration with policymakers, which has been documented to result in considerable environmental policy success (Tuinstra 2022), is not without risks, however. A review of energy modelling use in policy found that, while collaboration with modellers and policymakers could produce more policy-relevant models, there was also evidence that policymakers influenced model development and/or pushed for familiar modelling methods to support existing policy aims (Süsser et al. 2021). Kolkman et al. (2016) found that a combination of a model’s characteristics (i.e. complexity) and organizational factors, such as a lack of model “advocates” or the reputation of a particular modeller can impact policymakers decision to implement or use output from a model (Kolkman et al. 2016), which can contribute to the bias towards more established methods of modelling (Süsser et al. 2021). There can also be a bias towards well-established modelling tools because governmental agencies are more likely to turn to familiar modelling teams, despite political incentive to diversify methods (Turnpenny et al. 2009; Süsser et al. 2021). This may be an explanation for why there were few IBMs cited, as the general website searches, which included employee biographies and publication lists, did not result in a high rate of individuals listing expertise in IBMs. This bias can unnecessarily restrict modelling methods, limiting innovation.

Freshwater conservation provides examples of successful and long-term IBM use for policy. Railsback et al. (2009) created InSTREAM in response to the limitations of PHABISM (a non-IBM model). In collaboration with the United States Department of Agriculture (USDA), they have published a detailed user guide for not only how to set up the model, but how to apply it to management problems (Railsback et al., 2009). MORPH (an IBM most often applied to, but not limited to, coastal birds), a model documented in this study but not found cited in policy documents, has a more technical and less clear user-guide. The Bournemouth University Individual Ecology website, which manages MORPH, provides a vast list of scientific resources related to MORPH and IBMs, but few resources for non-experts in modelling and none for those without a science background (MORPH n.d.). While the model is downloadable to any potential user for free, a key step to accessibility (Cartwright et al. 2016), the website does not include any documentation for best practices for communication. That being said, the website does document case studies, many of which cite government-based funding sources (Case Studies n.d.), indicating a potential connection to policy development not documentable by the methods of this study. The case studies, which can be sorted by location and conservation topic, contain a standardized method of very brief “recommendations from modeling”, a simple but effective communication method that could be adopted by other researchers. However, if researchers are not actively advocating for their model, it is not clear how relevant conservation managers will know if specific model methods or results can be useful for their needs, especially if they are unaware of their modelling options to begin with.

It does seem that researchers can advocate for their own models. One policy document located while assessing the WoS IBMs (Williams et al. 2017) actually described the use and updating of an IBM, while the rest of the policy documents assessed simply cited a paper that used an individual-based model. This is likely due to the fact that the original lead author of the IBM (Hall et al. 2006) was also a co-author on the Williams et al. (2017) policy document.

There is also evidence of public pressure for more complex modelling. One UK policy document contained public comment, noting the main complaint was that the cited model was “too simplistic” (Marine Strategy Framework Directive Consultation: UK initial assessment and proposals for Good Environmental Status 2012). More complex models, such as IBMs, may be an avenue for governments to respond to public pressure.⁵ Notably, this public pressure only exists due to transparency of evidence used to inform policy. The issue of choosing model methods grows more complex when there are competing models of the same issue, sometimes with conflicting results (Mika and Newman 2010) or when the “black box” nature of a model reduces perceived credibility (Cash et al. 2003). The policy documents reviewed did not indicate if there was competition between model methods or if one method is considered more credible than another. Further research should be conducted to understand how or why certain methods are chosen for policy development.

There also may be up-and-coming agency interest in IBMs. One search result from NOAA was a schedule for a previously held modelling workshop that included a section on IBMs, indicating interest in IBMs even if these have not become more prominent within the agency yet. As previous research showed a bias towards “in house” or previously used model methods (Turnpenny et al. 2009; Süsser et al. 2021), this may be evidence a gradual shift towards IBM use at NOAA, presumably as a result of expanding expertise.

Communication

Communication is always a challenge when it comes to model use in policy development (Lorig et al. 2021). Communicating model assumptions, methods, and results can pose particular challenges to those working with IBMs, due to policymakers’ lack of familiarity with IBMs (or modelling in general). If a policymaker does not interpret the model output the way the modellers intended, there may be undetected misunderstandings, potentially negatively impacting policy (Syme et al. 2011). Therefore, models need to be communicated clearly, accurately, and transparently (Mcintosh et al. 2008; Kolkman et al. 2016; Lorig et al. 2021). While it is not always possible to predict the future policy landscape, modellers should thoughtfully consider how their models can be used and who the potential users of their model or model results could be.

Modelers who wish to conduct applied research should familiarize themselves with the policy landscape of their research topic and reflect on their role in the political arena, as it is policymakers, not researchers, who hold the most power when it comes to model use in policy (Süsser et al. 2021). Research has shown that understanding of policy theory can help researchers make progress advocating for their chosen issue (Weiner 2011). However, most scientists are overwhelmingly ignorant of communication research or policy theory (Hayes et al. 2008). Researchers are often ill-equipped to participate in science communication, though ample research has been published on the topic (Pullin and Knight 2003; Stamatakis et al. 2010; Wilhelm‐Rechmann and Cowling 2011; Fuller et al. 2014; Rose 2015). Modelers can also utilize boundary organizations, which assist interactions between scientists and policymakers, something relatively few researchers use (Lemos et al. 2014; Suhay and Cloyd 2018; Akerlof 2022). A majority of American Association for the Advancement of Science members, an organization that works to connect scientists with policymakers, report having communicated their research with policymakers (Suhay and Cloyd 2018), indicating the usefulness of such organizations. Institutions consistently undervalue science communication, but if institutional support for science communication increased, it may positively impact the evidence-policy interface.

Unlike the TRAnsparent and Comprehensive Ecological modelling (TRACE) and Overview, Design concepts and Details (ODD) documents (Grimm et al. 2006), which provide clear guidance on communicating models to other researchers, there is no broadly used or accepted procedure for communicating IBMs (or other complex models) to stakeholders or other non-experts (Cartwright et al. 2016). This suggests that while researchers are communicating with each other in increasingly uniform ways, they are not communicating models to policymakers and the broader public in a consistent or systemic way. TRACE and ODD do not require any information about communication strategies, making it difficult to assess if or how modellers advocated for their model results to be used in policy. The lack of any systemic framework or guidance means it is not even standard practice for researchers to consider communication with non-experts.

Some researchers have argued that communication frameworks similar to public health can be used by conservationists when trying to advocate for policy (Pullin and Knight 2003). Those involved with medical research have advocated that scientists use “stories” to communicate scientific results (Stamatakis et al. 2010), though conservation researchers argue that story-telling is insufficient, and that influencing policy also requires that research be “framed” in a politically-salient context (Mcintosh et al. 2008; Syme et al. 2011; Rose 2015; Kolkman et al. 2016). In other words, researchers should take advantage of relevant policy windows. Other researchers have argued that better model descriptions (i.e. ODD) are required to improve model use in policy, and that standardized, transparent communication methods are needed (Lorig et al. 2021). However, this assumes the target audience is familiar with and comfortable reading a model description, something a policymaker may not be, especially considering that many policymakers have large workloads and are unable to interpret scientific writing (Akerlof et al. 2018). Grimm et al. (2020) provided guidance for policymakers, to help them assess if a models’ output should be used for decision-making.

In answer to the communication challenge, Cartwright et al. (2016) present a framework for communicating complex ecological models to non-experts, while Fischhoff and Davis (2014) have provided detailed guidance for scientists in how to assess and communicate uncertainty in research. Others have argued that uncertainty can be viewed by researchers as an opportunity to collaborate with stakeholders, allowing researchers to take advantage of local and shared knowledge, rather than as a weakness of the model (Paolisso et al. 2015). Researchers have also highlighted the importance of understanding the mental and cultural frameworks of stakeholders, which may impact understanding and trust in models (Paolisso et al. 2015). There exists ample literature on management strategy evaluation (MSE), which can assist with model implementation, evaluation, and communication between managers and policymakers (Holland 2010; Kaplan et al. 2021).

While providing communication guidelines specific to models is outside of the scope of this paper, modellers should consider the most recent communication research to ensure successful application of their research. Further research could also evaluate the time between the completion of scientific research, the publication of peer-reviewed research, and the publication of policy documents. There have been recent efforts to improve communication between the scientific community and policymakers, the results of which may not yet be documentable.

Limitations of this study

One set of limitations relates to the WoS search. First, there were a significant number of false positives, i.e. search results that did not actually include marine IBMs. While the search identified a large number of marine IBMs, 52% of the search results contained non-marine IBMs. Second, we can identify at least some false negatives, i.e. missing search results pointing to marine IBMs. We found eight policy documents that contained an IBM not identified by the WoS Search. Of those eight, four should have appeared in the WoS search, as WoS indexes the journal they were published in. Why they did not appear in the search results is unclear. The other four were instances where original research that utilized IBMs were described in the policy document itself. It is likely that there were more false negatives that we did not identify. The WoS indexes titles, abstracts, and keywords, so if the authors did not include any of the search terms (Appendix Table A1) in their title, abstract, or keywords, then their article would not appear in the WoS search. Further research could expand the use of keywords to include more generalized terms, such as “model” and “simulation”; however, doing so would almost certainly identify a very large number of false positives. At the same time, it would allow for a comparison of the rate of IBM using in marine science compared to other, more established model methods, which we did not attempt to do here.

We also recognize that this research is neither comprehensive nor representative of policy documents worldwide. It is difficult to find a comprehensive list of policy documents and many policy documents do not list citations, making it impossible to assess what, if any, evidence informed the policy, a challenge other researchers have noted (Chang et al. 2021). It is also not clear if this research documents evidence of absence, or simply absence of evidence of IBM use in policy. Evidence is often used in policy in a way that is difficult, if not impossible, to document, such as in committee meetings or in documents that lack citations. However, considering the number of policy documents that cited other model methods, the results seem to indicate evidence of absence. Notably, some policy documents included original research using IBMs, such as European Commission, Directorate-General for Maritime Affairs and Fisheries et al. (2020), which provided technical guidance resulting from new research conducted by the authors.

Some potential limitations related to the search for references to IBMs in policy documents include language barriers, word choice, and issues with how the policy documents themselves reflected model use. With respect to language, English is predominately the language in scientific publications, while conservation policy is conducted in the language of the nation. Several WoS IBMs were written on topics that impacted non-English speaking countries, making it difficult to search for policy documents using an English language search approach. Therefore, this research should not be considered representative of non-English marine policy. Future research should include scientific research and policy documents in languages other than English.

The limitation related to word choice is similar to that we noted about the WoS search. While many papers written on models will use the word “model” or “simulation” in the title (making them easily identifiable in the references list of policy documents), this is not always the case. Any scientific paper that used models but did not contain the terms “model” or “simulation” in the title would not have been counted in a review of the policy documents’ reference lists.

Finally, if a policy document utilized information derived from scientific evidence, such as estimated population sizes, they may not have cited the source or noted that this information came from a model. This could potentially result in undercounting the use of both IBMs and/or other modelling methods. It is also not clear how the UK, NOAA, and New Zealand websites index their searches. For example, they may not include meta tags, or may only search the title of a page, not the entire text of a webpage. The search engine used may have been a limiting factor in the use of these websites. We should also consider that the quality of some research, which we did not assess, may be a factor in the low rate of use for policy, especially in areas where IBMs are still new and unverified.

Conclusion

This research demonstrates that IBMs are not frequently used to inform marine policy, while other model methods are. This study also highlights the challenge of tracking research use in policy development, as the majority of policy documents did not cite sources, making it impossible to determine what, if any, scientific research informed the conservation policy. However, of the policy documents with citations, a majority did cite other model methods. It is not clear how scientific evidence (in this case, modelling methods) is found and why some evidence is utilized when others are not. While this likely varies between departments, agencies, and countries, this demonstrates a challenge for researchers who wish for their model results to be used in the development of “evidence-based policy”. More research is needed to understand the decision-making process when it comes to evidence selection and inclusion in reports and/or policy development.

Modelers who work with IBMs and wish to develop applicable research should not assume that their model results will be useful for policy and should instead ensure that they are 1) explicitly addressing a policy need and 2) making the information accessible to policymakers, via crafting a communication and/or implementation plan for policymakers or by joining a relevant boundary organization.

Highlights

1. A large percentage of international peer-reviewed scientific research that used individual-based models (IBMs) for marine-focused research claimed that the results of the IBMs were relevant or important to marine conservation policy or management. However, the IBMs were rarely cited in policy documents on the same marine topic.

2. IBMs were cited in international marine policy documents at a significantly lower rate than other model methods.

3. Researchers who use IBMs should take proactive steps to communicate their research to relevant policymakers.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

SUPPLEMENTARY MATERIAL

Supplemental data for this article can be accessed online at https://doi.org/10.1080/1943815X.2023.2271550

Notes

1. Also referred to as agent-based modelling (ABM). The term IBM is most commonly used in ecological research, while ABM is more common in social science literature. There is no fundamental difference between the two. In this paper, IBM will be used consistently throughout for clarity.

2. Key words identified in the description of the document.

3. No policy document used “simulate” or “simulation” in place of the term “modelling” but 19 (36%) used both terms interchangeably or in combination (i.e. “model simulation”).

4. No policy document used “simulate” or “simulation” in place of the term “modelling” but 11 (18%) used both terms interchangeably or in combination (i.e. “model simulation”).

5. It should be noted that IBMs can require a high amount of data to parametrize, which may be a potential explanation for their limited use, despite public pressure within the last decade.

Appendix

Table A1. Web of science search criteria.

Keywords	Web of Science Search Criteria
Marine Conservation	((TS=(“individual based model”) OR TS=(“IBM”) OR TS= (“Agent based model”) OR TS= (“ABM”) AND TS=(Marine)) AND TS=(conservation)) NOT (DT==(“BIOGRAPHICAL ITEM” OR “DATA PAPER” OR “REPRINT” OR “RETRACTED PUBLICATION” OR “POETRY” OR “BIBLIOGRAPHY” OR “ART EXHIBIT REVIEW” OR “DANCE PERFORMANCE REVIEW” OR “FILM REVIEW” OR “MEETING ABSTRACT” OR “RETRACTION” OR “SOFTWARE REVIEW” OR “EXPRESSION OF CONCERN” OR “CORRECTION” OR “LETTER” OR “NEWS ITEM” OR “BOOK REVIEW” OR “EDITORIAL MATERIAL”))
Marine Ecology	((TS=(“individual based model”) OR TS=(“IBM”) OR TS= (“Agent based model”) OR TS= (“ABM”) AND TS=(Marine)) AND TS=(ecology)) NOT (DT==(“BIOGRAPHICAL ITEM” OR “DATA PAPER” OR “REPRINT” OR “RETRACTED PUBLICATION” OR “POETRY” OR “BIBLIOGRAPHY” OR “ART EXHIBIT REVIEW” OR “DANCE PERFORMANCE REVIEW” OR “FILM REVIEW” OR “MEETING ABSTRACT” OR “RETRACTION” OR “SOFTWARE REVIEW” OR “EXPRESSION OF CONCERN” OR “CORRECTION” OR “LETTER” OR “NEWS ITEM” OR “BOOK REVIEW” OR “EDITORIAL MATERIAL”))

Two searches were conducted for “Individual based model” and “marine conservation” and “Individual based model” and “marine ecology”. Web of Science allows for searches to include alterative terms (“or” in the table), so IBM, agent based model, and ABM were searched alongside individual based model. The results of both searches were combined..

See CSV file for Supplementary table S1.