The following 10 rules constitute an informal and general best practice guide to scientific simulation modelling and appear roughly in the order in which issues might pop up in the course of a modelling project. They focus on conceptual issues (Rules 1 to 3), general technical and coding issues (Rules 4 to 7), and analysis and presentation of modelling results (Rules 8 to 10).

So, if you shouldn’t model the system you know in complete detail, what should you model instead? This depends to a great extent on the type of model that you have decided to make (Rule 1). Assuming that some mechanistic insight is the aim, based on knowledge of the system and precise formulation of the research question, a good strategy is to identify putative interactions or mechanisms that could underlie the behaviour or phenomenon under investigation and model those mechanisms. In other words, it is sensible to propose an engine that could be driving the phenomenon of interest and then make characterising and investigating that engine the central focus of the model. Anything in the system irrelevant to that engine is irrelevant to the model. Part of the art of modelling is identifying putative mechanisms and ensuring that the logical thread between mechanism and behaviour is unbroken.

Perhaps this is an obvious rule—cience and rigour usually go hand in hand. Successful modelling requires considerable attention to mathematical and computational detail. It is vital that all concepts and all code used in a simulation model are clear to the author and rigorously tested—from their construction to their downstream implications (see Rule 6). Using convenient mathematical or statistical concepts with little or no knowledge about their inner workings will likely lead to their misuse and maybe to serious fundamental errors. Similarly, it can be dangerous to use, for example, software packages or useful-looking chunks of code as untested black boxes (but see Rule 6). The assumptions of the methods implemented in the code, and the implementation itself, must be clear and consistent with the assumptions and aims of your model. Alongside the mathematical and technical details of each part of the model, careful consideration should be given to the order in which these parts or processes are executed and what effect that ordering (or scheduling) has on the output.

All of this is to say that modelling decisions need careful thought and, as with empirical data analysis, designing a model that describes your system in adequate (but not excessive) detail, and deriving robust inferences from it, can take significant time. This is time which needs to be budgeted for in any simulation modelling project from the outset.

The aim of this rule is to help you to avoid a surprisingly common and deeply frustrating situation: creating a monster simulation with so many moving parts that it just can’t be analysed meaningfully—a situation where you could say to yourself something like “I’ve finished coding this simulation, but how on earth can I analyse so many interactions and so many outputs?” Reaching this point probably means that something has gone wrong in the implementation of Rules 1 or 2—or that the model has been coded without a good plan for its analysis. So, perhaps you know the question you want to answer, you know or have guessed the relevant aspects of the system, but you didn’t plan how the model can answer the question. In general, it is important to have an idea of the analysis steps necessary to answer aspects of the central question. These plans are flexible and develop throughout a project, of course, but an initial plan is crucial.

Your exact plan for analysis will depend on your aim and the kind of model you have constructed. For example, if you are interested in unravelling mechanistic relationships, you might use simulated data to explore the workings of the model and to analyse exactly how different parameters or processes affect the outcome. To do this successfully, the number of parameters and processes should be manageable (see Rule 2), and parameter explorations should be principled (see Rule 8).

Or, if you are interested in fitting your model to observed data, you may need to consider exactly how the model output maps to existing and future data and identify appropriate statistical techniques to robustly test between models and estimate parameters. Here, it will be important to be aware of the limitations and assumptions of the statistical techniques in addition to those of the model itself (see Rule 3).

You will need to revisit your code, sometimes many months after the analysis has been done and the paper has been written. This can either be a difficult exercise for your future self or a simple matter of glancing at your structured, documented, and organised code. Of course, it is best to aim for the latter. This means that code needs to be well structured and well commented so that you are able to easily reconstruct what has been done. To this end, this rule advises a number of good coding practices that are particularly important for simulation models, which can become large and unwieldy projects very quickly. All of these practices will benefit the readers of your model as much as they benefit your future self.

First, the names of variables, functions, and parameters should be intuitive, consistent, and immediately intelligible. This makes the code more readable and minimises unnecessary comments. Similarly, adopting consistent formatting conventions (such as indentation and other whitespace), much like punctuation in a sentence, improves code readability. Commenting should be done with both your future self and your future readers in mind. For larger-scale multiresearcher and multiyear projects, more detailed documentation should be created and developed in tandem with the code.

As a project grows, it becomes difficult to keep track of the interactions and dependencies between the different parts of the code. This complexity can be made more manageable by giving your code a well-defined and modular structure, meaning that sections of code should be divided into logically related units and the interfaces between these units of code should be well defined. Different programming languages offer different tools to achieve this, and the following advice applies equally to all of them. We’ll use the concept of functions as an example because this is the most fundamental and most commonly used. The aim here is to identify units of self-contained logic and create functions that encapsulate this logic. Functions should have a single clear purpose so, if you find you’re writing a function which performs multiple conceptual actions, consider breaking it up into 2 or more functions. Encapsulating logic in this way creates higher-order building blocks that can be reused and avoid code duplication. This reduces the opportunity for errors, not only because there will be less code to begin with, but also because each logical unit can be easily tested individually.

In fact, it is good practice to test functions as you write them—this helps to avoid bugs in “low-level” functions that often manifest as difficult to diagnose problems downstream. The ubiquitously useful coding mantra “fail early and fail loudly” posits that it is better to raise an error as soon as it could feasibly be detected. So, for example, it is best to have functions check that their inputs are valid rather than allowing them to blindly compute and pass nonsensical output downstream.

Despite taking these steps to manage the complexity of your project, you will write bugs. Everyone does and it is unavoidable. But, if you have written modular, well-structured code, it will be much easier to test the code, find the bugs, and squash them.

With a similar aim in mind, it is now common practice and explicitly required by some journals that model code is published alongside the paper it generates. This has a number of important benefits. It facilitates open and reproducible science. It also benefits you as a modeller. Bugs and coding issues can be identified by reviewers and readers who go through the code. This can be an important source of error checking.

Finally, but equally importantly, you will need to test that the (now hopefully bug free) code actually does what it is supposed to in terms of the model’s dynamics. The full code, and each constituent part, should be put through its paces using simple test cases for which you have either good intuitions or solid analytical expectations to which the model outcomes should conform.

Practical code doesn’t have to be perfect, fully optimised, or even beautifully written, but it is fit for purpose, well tested, and clearly documented so that others (and you) can have confidence in it (see Rule 5). Practical coding also means using your coding time efficiently. Investing some time in learning to use the debugging tools available to your programming language will quickly pay for itself in time saved identifying and fixing bugs more quickly. There are a great many well-tested and already optimised scientific libraries/packages, and there is usually nothing to be gained by reimplementing functionality that already exists. Making use of good quality libraries will speed up development and reduce your opportunities to write bugs (see Rule 5), although it is critical to ensure that the methods used are appropriate for your context (see the note about the dangers of “black boxes” in Rule 3).

Fit-for-purpose code runs in a reasonable amount of time. Many parameter sweeps with many repetitions of stochastic simulations will be necessary to generate robust, publishable results (see Rule 7), so it’s very likely that you’ll need to perform some optimisation to make your model run more quickly. There is some discussion about the best time and way to make code efficient. Conventional wisdom often attributed to the programming pioneer Donald Knuth states that early optimisation is problematic when a lot of time might be wasted making the wrong parts of the code faster. This issue of timing does not change the fact that, for simulation models, writing efficient code is important, but it does mean that making the right parts of the code efficient is key. A common approach is to write relatively quick “prototype” code that emphasises correctness, good structure, and readability and then identify potential bottlenecks by using profiling tools to see how much time is spent running each part of your code. When it comes to actually implementing optimisations, understanding the intricacies—and potentially underexploited capabilities—of your particular programming language helps enormously. For example, large speed increases can be gained in R, Python, and MATLAB by vectorising code that relies heavily on loops, but this may not be the case, for instance, in C++ under the same circumstances. In general, developing a good understanding of the strengths and weaknesses of the data structures provided by your programming language allows you to make informed decisions about how to best temporarily store, permanently save, and manipulate information.

Once your model is constructed, it may be tempting to think that you are on the home stretch. As with model construction, though, running a simulation is fraught with difficult decisions that require robust justifications. How exactly will the model be run? How will it be initialised? How long should it run for? How many iterations are enough? Unfortunately, usually, there are no hard and fast rules to help with any of those questions; the answers depend on the model, and it is up to the modeller to decide and to demonstrate that the decisions lead to robust results.

Initialising a model is not always straightforward. For many models, it is best to randomly set initial states or parameter values—perhaps drawn from a prior distribution. Alternatively, it may be possible to start from some deterministic point, potentially informed by empirical observations. Either way, you need to be sure that the outcome of the model is not obscured by the initial conditions. This might require, for example, a sufficiently long burn-in time, such that the model reaches a state where meaningful comparisons between outcomes can be made—this could be at an equilibrium point or some other predefined state that is appropriate for the specific model (e.g., when a critical minimum threshold is reached). Additionally, a full characterisation of the system may involve a thorough sweep of initial conditions.

Simulation models often have many moving parts (but hopefully not too many; see Rule 2) and probably a number of stochastic elements. So, in order to comprehensively explore the space of possible outcomes, then, the model needs to be run a large number of times to fully capture the effects of the model parameters as well as stochasticity (another good reason for efficient model code; see Rule 6). What “large” means here depends on the model and only the modeller can decide and demonstrate that the space of possible outcomes has been meaningfully explored. When there is a stochastic element to the simulation, single runs are largely uninformative—understanding and quantifying the means and distribution of outcomes is what’s important (Rules 9 and 10).

It might be tempting to see reporting this kind of model failure as decreasing the value of the work, but the opposite is true. Reporting detailed information on where the model breaks down hugely increases its value and the confidence that readers can have that the inferences you draw are not just the result of cherry picking parameter values or arbitrarily reducing the parameter space post hoc.

This follows from and builds on the previous rule. Rule 8 says that substantial effort should be dedicated to finding the limits of a simulation model’s technical utility. This rule suggests that this search for the model’s limits extends to its conceptual robustness, too. For example, if the model can only give one result—even if it is correct as far as the aims of the project are concerned—the model is not meaningful and is likely to be uninformative when confronted with data. There are limits to the usefulness of any model and understanding what you can accurately and confidently infer from the model and what is out of reach is the basis of good interpretation. It is the author’s responsibility to ensure that this information is clearly communicated to the reader and that the limits of inference, given the assumptions, are carefully explained. This is, at the end of the day, a matter of honesty in interpretation and honesty in reporting.

Models live or die on their explicit and implicit assumptions and these need to be made unambiguously clear. Listing explicit assumptions, although sometimes overlooked and often difficult, is easier than the more daunting task of examining all implicit assumptions built into the model and their impact on the dynamic. Once this is done, any model interpretation must be tightly bound to those assumptions. The literature is full of examples of slippage where assumptions were carefully explained in a methods section but, later, no longer constrain interpretation. This can easily lead to overinterpretation, i.e., results are interpreted outside the range of explored and reported parameters, and conclusions are drawn that are not justified.

Clearly summarising model outputs or simulated data involves showing outcomes across the chosen parameter space—where the range or limits of this is robustly justified (see Rule 8)—and for multiple, rather than single runs (see Rule 7). To summarise multiple runs of a simulation, it is often advisable to go beyond just mean values—understanding the range of possible outcomes is itself informative. It is often highly instructive to use a range of metrics to describe these outcomes or even show the raw distributions themselves which can offer insights and suggest interpretations that are obscured by summary statistics. It is only in this way that you can discover anomalies like bimodal results or a large spread of outcomes that defy interpretation.

Communication of models and modelling results can be a difficult balance. If the work is interdisciplinary in the sense that it will be read by both fellow modellers and nonmodellers, it is important to bear in mind the level of technical expertise of all readers. You should aim to provide details in a form that is understandable and digestible to those who, for example, might want to reproduce and extend the model itself and to those who wish to make use of its insights but are not modellers themselves. In other words, some effort should be made to enable readers of all backgrounds to follow (and critique) the logic and implementation of the model as far as possible.

Clear and informative figures play a huge role here in communicating both the results from, and limits to, a model. Figures are often the way that readers will first encounter the model and its results, and the more useful, readable information they contain, the more your readers can assess about the utility of the model at a glance.