Calibration

Disease models typically require contextualization to a relevant setting of interest prior to addressing “what-if” scenario questions. The process of tuning model input parameters so that model outputs match observed data is known as calibration. There are many approaches to model calibration, ranging from manual tuning to fully Bayesian methods.

For many applications, we have found that an optimization-based approach is sufficient. Such methods avoid the tedious process of manual tuning and are less computationally expensive than fully Bayesian methods. One such optimization-based approach is the Optuna library, which is a Bayesian hyperparameter optimization framework. Optuna is designed for tuning hyperparameters of machine learning models, but it can also be used to calibrate disease models.

Calibration libraries often treat the disease model as a black box, where the input parameters are the “hyperparameters” to be tuned. The calibration process is often iterative and requires a combination of expert knowledge and computational tools. The optimization algorithm iteratively chooses new parameter values to evaluate, and the model is run with these values to generate outputs. The outputs are compared to observed data, and a loss function is calculated to quantify the difference between the model outputs and the observed data. The optimization algorithm then uses this loss function to update its search strategy and choose new parameter values to evaluate. This process continues until the algorithm converges to a set of parameter values that minimize the loss function.

While many optimization algorithms are available, Starsim has a built-in interface to the Optuna library, which we will demonstrate in this guide. We will use a simple Susceptible-Infected-Recovered (SIR) model as an example. We will tune three input parameters, the infectivity parameter, beta, the initial prevalence parameter, init_prev, and the Poisson-distributed degree distribution parameter, n_contacts. We will calibrate the model using a beta-binomial likelihood function so as to match prevalence at three distinct time points.

We begin with a few imports and default settings:

##%% Imports and settings
import sciris as sc
import starsim as ss
import numpy as np
import pandas as pd
import matplotlib.dates as mdates

n_agents = 2e3
debug = False # If true, will run in serial

The calibration class will require a base Sim object. This sim will later be modified according to parameters selected by the optimization engine. The following function creates the base Sim object.

def make_sim():
    sir = ss.SIR(
        beta = ss.peryear(0.075),
        init_prev = ss.bernoulli(0.02),
    )
    random = ss.RandomNet(n_contacts=ss.poisson(4))

    sim = ss.Sim(
        n_agents = n_agents,
        start = ss.date('2020-01-01'),
        stop = ss.date('2020-02-12'),
        dt = ss.days(1),
        diseases = sir,
        networks = random,
        verbose = 0,
    )

    return sim

Now let’s define the calibration parameters. These are the inputs that Optuna will be able to modify. Here, we define three such parameters, beta, init_prev, and n_contacts.

Each parameter entry should have range defined by low and high as well as a guess values. The guess value is not used by Optuna, rather only for a check after calibration completes to see if the new parameters are better than the guess values.

You’ll notice there are a few other parameters that can be specified. For example, the data type of the parameter appears in suggest_type. Possible values are listed in the Optuna documentation, and include suggest_float for float values and suggest_int for integer types.

To make things easier for the search algorithm, it’s helpful to indicate how outputs are expected to change with inputs. For example, increasing beta from 0.01 to 0.02 should double disease transmission, but increasing from 0.11 to 0.12 will have a small effect. Thus, we indicate that this parameter should be calibrated with log=True.

# Define the calibration parameters
calib_pars = dict(
    beta = dict(low=0.01, high=0.30, guess=0.15, suggest_type='suggest_float', log=True), # Note the log scale
    init_prev = dict(low=0.01, high=0.05, guess=0.15), # Default type is suggest_float, no need to re-specify
    n_contacts = dict(low=2, high=10, guess=3, suggest_type='suggest_int'), # Suggest int just for this demo
)

The optimization engine iteratively chooses input parameters to simulate. Those parameters are passed into the following build_sim function as a dictionary of calib_pars along with the base sim and any other key word arguments. The calib_pars will be as above, but importantly will have an additional key named value containing the value selected by Optuna.

When modifying a sim, it is important to realize that the simulation has not been initialized yet. Nonetheless, the configuration is available for modification at sim.pars, as demonstrated in the function below for the SIR example.

def build_sim(sim, calib_pars, n_reps=1, **kwargs):
    """
    Modify the base simulation by applying calib_pars. The result can be a
    single simulation or multiple simulations if n_reps>1. Note that here we are
    simply building the simulation by modifying the base sim. Running the sims
    and extracting results will be done by the calibration function.
    """

    sir = sim.pars.diseases # There is only one disease in this simulation and it is a SIR
    net = sim.pars.networks # There is only one network in this simulation and it is a RandomNet

    for k, pars in calib_pars.items(): # Loop over the calibration parameters
        if k == 'rand_seed':
            sim.pars.rand_seed = v
            continue

        # Each item in calib_pars is a dictionary with keys like 'low', 'high',
        # 'guess', 'suggest_type', and importantly 'value'. The 'value' key is
        # the one we want to use as that's the one selected by the algorithm
        v = pars['value']
        if k == 'beta':
            sir.pars.beta = ss.peryear(v)
        elif k == 'init_prev':
            sir.pars.init_prev = ss.bernoulli(v)
        elif k == 'n_contacts':
            net.pars.n_contacts = ss.poisson(v)
        else:
            raise NotImplementedError(f'Parameter {k} not recognized')

    # If just one simulation per parameter set, return the single simulation
    if n_reps == 1:
        return sim

    # But if you'd like to run multiple simulations with the same parameters, we return a MultiSim instead
    # Note that each simulation will have a different random seed, you can set specific seeds if you like
    # Also note that parallel=False and debug=True are important to avoid issues with parallelism in the calibration
    # Advanced: If running multiple reps, you can choose if/how they are combined using the "combine_reps" argument to each CalibComponent, introduced below.
    ms = ss.MultiSim(sim, iterpars=dict(rand_seed=np.random.randint(0, 1e6, n_reps)), initialize=True, debug=True, parallel=False)
    return ms

The Starsim framework has been integrated with the Optuna hyperparameter optimization algorithm to facilitate calibration through the Calibration class. Recall that an optimization-based approach to calibration minimizes a function of the input parameters. This function is key to achieving an acceptable calibration.

There are two ways to describe the goodness-of-fit function for the Calibration. The first method is to directly provide a function that the algorithm will call. The eval_fn will be passed each completed sim after running, and is expected to return a float representing the mismatch (lower is better as the optimization algorithm is configured to minimize). Data can be passed into the eval_fn via eval_kwargs.

As an alternative to directly specifying the evaluation function, you can use CalibComponents. Each component includes real data, for example from a survey, that is compared against simulation data from the model. Several components can be used at the same time, for example one for disease prevalence and another for treatment coverage. Each component computes a likelihood of the data given the input parameters, as assessed via simulation. Components are combined assuming independence.

The base class for a component is called CalibComponent, which you can use to define your own likelihood. However, we have provided components for several key likelihood functions including BetaBinomial, Binomial, DirichletMultinomial, GammaPoisson, and Normal. The Normal component is most like a traditional squared error. Each component takes in a name and a weight, which is used when combining log likelihoods.

Importantly, each component takes in the calibration target, the real data that was observed, in an argument called expected. This argument should be a Pandas Dataframe with one row per time point and columns that will depend on the specific component type. For example, the Binomial component requires columns of n (trials) and x (successes).

The components also handle extracting data from each simulation using the extract_fn argument. The value of this argument should be a function that takes in a simulation and returns a Pandas DataFrame. The specifics of the columns will depend a bit on the type of component (e.g. BetaBinomial is different from Normal), but often looks like a simulated version of expected. We will see examples below.

We’ll also see how to use the conform argument, the purpose of which is to temporally align the simulation output to the real data. This argument works along with the extract_fn to produce the final simulation outputs that are used in the likelihood function. The conformer is a function that takes in the expected data you provided and the actual simulation result the comes out of the extract_fn. The conformers we have built in are as follows:

• step_containing: Conform by simply choosing the simulated timestep that contains the time indicated in the real data (expected)
• prevalent: Interpolate the simulated timepoints to estimate the values that would have occurred at each real timepoint
• incident: While the two methods above capture the state of the model at a particular point in time (stocks), this component allows you to capture the behavior of the model over time (flows). Instead of just giving one time value, t, you’ll provide a second time value as well called t1. This conformer will add up events occurring between the two time points.

Let’s make a Binomial component, as might be used to calibrate disease prevalence.

prevalence = ss.Normal(
    name = 'Disease prevalence',
    conform = 'prevalent',

    expected = pd.DataFrame({
        'x': [0.13, 0.16, 0.06],    # Prevalence of infection
    }, index=pd.Index([ss.date(d) for d in ['2020-01-12', '2020-01-25', '2020-02-02']], name='t')), # On these dates
    
    extract_fn = lambda sim: pd.DataFrame({
        'x': sim.results.sir.prevalence,
    }, index=pd.Index(sim.results.timevec, name='t')),

    # You can specify the variance as well, but it's optional (max likelihood estimates will be used if not provided)
    # This could be a single float or an array with the same shape as the expected values
    sigma2 = 0.05, # e.g. (num_replicates/sigma2_model + 1/sigma2_data)^-1
    #sigma2 = np.array([0.05, 0.25, 0.01])
)

Finally, we can bring all the pieces together. We make a single base simulation and create an instance of a Starsim Calibration object. This object requires a few arguments, like the calib_pars and sim. We also pass in the function that modifies the base sim, here our build_sim function. No additional build_kw are required in this example.

We also pass in a list of components. Instead of using this “component-based” system, a user could simply provide an eval_fn, which takes in a completed sim an any eval_kwargs and returns a “mismatch” score to be minimized.

We can also specify the total number of trials to run, the number of parallel works, and a few other parameters.

sc.heading('Beginning calibration')

# Make the sim and data
sim = make_sim()

# Make the calibration
calib = ss.Calibration(
    calib_pars = calib_pars,
    sim = sim,
    build_fn = build_sim,
    build_kw = dict(n_reps=3), # Run 3 replicates for each parameter set
    reseed = True, # If true, a different random seed will be provided to each configuration
    components = [prevalence],
    total_trials = 100,
    n_workers = None, # None indicates to use all available CPUs
    die = True,
    debug = debug, # Run in serial if True
)

# Perform the calibration
sc.printcyan('\nPeforming calibration...')
calib.calibrate();

—————————————————————

Beginning calibration

—————————————————————





Peforming calibration...

Removed existing calibration file starsim_calibration.db

sqlite:///starsim_calibration.db

[I 2025-08-04 18:05:41,057] A new study created in RDB with name: starsim_calibration
[I 2025-08-04 18:05:42,755] Trial 1 finished with value: -0.44981093690565604 and parameters: {'beta': 0.07553254901838954, 'init_prev': 0.010623494926858133, 'n_contacts': 10, 'rand_seed': 433446}. Best is trial 1 with value: -0.44981093690565604.
[I 2025-08-04 18:05:42,761] Trial 3 finished with value: -0.449572325794545 and parameters: {'beta': 0.01961207740877714, 'init_prev': 0.01012261691719703, 'n_contacts': 8, 'rand_seed': 180549}. Best is trial 1 with value: -0.44981093690565604.
[I 2025-08-04 18:05:42,778] Trial 4 finished with value: -0.5066014924612117 and parameters: {'beta': 0.058658631402975434, 'init_prev': 0.042534023778888305, 'n_contacts': 3, 'rand_seed': 724195}. Best is trial 4 with value: -0.5066014924612117.
[I 2025-08-04 18:05:42,995] Trial 5 finished with value: -0.47252010357232277 and parameters: {'beta': 0.14532343237046544, 'init_prev': 0.020894275400617297, 'n_contacts': 4, 'rand_seed': 855145}. Best is trial 4 with value: -0.5066014924612117.
[I 2025-08-04 18:05:43,038] Trial 9 finished with value: -0.4788145480167672 and parameters: {'beta': 0.012898853458389823, 'init_prev': 0.02490264521778353, 'n_contacts': 10, 'rand_seed': 18040}. Best is trial 4 with value: -0.5066014924612117.
[I 2025-08-04 18:05:43,186] Trial 15 finished with value: -0.47738454801676716 and parameters: {'beta': 0.012105785028107737, 'init_prev': 0.023531045033863034, 'n_contacts': 8, 'rand_seed': 309556}. Best is trial 4 with value: -0.5066014924612117.
[I 2025-08-04 18:05:43,196] Trial 14 finished with value: -0.5101623257945449 and parameters: {'beta': 0.012920769255954593, 'init_prev': 0.044341142361296534, 'n_contacts': 6, 'rand_seed': 382646}. Best is trial 14 with value: -0.5101623257945449.
[I 2025-08-04 18:05:43,268] Trial 0 finished with value: -0.49035121468343384 and parameters: {'beta': 0.01885821140391564, 'init_prev': 0.03115300759327229, 'n_contacts': 9, 'rand_seed': 255576}. Best is trial 14 with value: -0.5101623257945449.
[I 2025-08-04 18:05:43,306] Trial 17 finished with value: -0.4607020480167672 and parameters: {'beta': 0.021503779057190493, 'init_prev': 0.01593263874981397, 'n_contacts': 10, 'rand_seed': 606508}. Best is trial 14 with value: -0.5101623257945449.
[I 2025-08-04 18:05:43,328] Trial 12 finished with value: -0.4629187146834339 and parameters: {'beta': 0.23149023956723366, 'init_prev': 0.01637406805329071, 'n_contacts': 5, 'rand_seed': 373703}. Best is trial 14 with value: -0.5101623257945449.
[I 2025-08-04 18:05:43,365] Trial 6 finished with value: -0.5109581591278782 and parameters: {'beta': 0.022787990007422064, 'init_prev': 0.04461016314827858, 'n_contacts': 7, 'rand_seed': 820817}. Best is trial 6 with value: -0.5109581591278782.
[I 2025-08-04 18:05:43,454] Trial 10 finished with value: -0.4937517702389894 and parameters: {'beta': 0.05608624538868571, 'init_prev': 0.03372998002387857, 'n_contacts': 3, 'rand_seed': 731006}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,456] Trial 7 finished with value: -0.4592303813501005 and parameters: {'beta': 0.27014934555622566, 'init_prev': 0.014551256749120865, 'n_contacts': 3, 'rand_seed': 529264}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,462] Trial 2 finished with value: -0.46576454801676725 and parameters: {'beta': 0.11740703374041636, 'init_prev': 0.01801381497891888, 'n_contacts': 6, 'rand_seed': 791255}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,477] Trial 8 finished with value: -0.5169209369056561 and parameters: {'beta': 0.01816702263490502, 'init_prev': 0.049784007865639836, 'n_contacts': 4, 'rand_seed': 575437}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,483] Trial 16 finished with value: -0.4929392702389894 and parameters: {'beta': 0.019154157712861667, 'init_prev': 0.03335062254891218, 'n_contacts': 3, 'rand_seed': 344204}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,534] Trial 11 finished with value: -0.5136698257945449 and parameters: {'beta': 0.026076247410548755, 'init_prev': 0.04660187584597949, 'n_contacts': 6, 'rand_seed': 522922}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,703] Trial 13 finished with value: -0.46390399246121167 and parameters: {'beta': 0.28416832894326555, 'init_prev': 0.01673353634506193, 'n_contacts': 5, 'rand_seed': 694512}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,704] Trial 22 finished with value: -0.4475501035723227 and parameters: {'beta': 0.08978885373494497, 'init_prev': 0.01086742309237589, 'n_contacts': 6, 'rand_seed': 799893}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,704] Trial 20 finished with value: -0.5051389924612116 and parameters: {'beta': 0.03773333295301447, 'init_prev': 0.044429039170489695, 'n_contacts': 10, 'rand_seed': 687540}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,754] Trial 18 finished with value: -0.4581117702389894 and parameters: {'beta': 0.08383449618356444, 'init_prev': 0.013869845871404878, 'n_contacts': 8, 'rand_seed': 338634}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,770] Trial 23 finished with value: -0.49230538135010055 and parameters: {'beta': 0.08961280509108403, 'init_prev': 0.033753042102942374, 'n_contacts': 9, 'rand_seed': 829446}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,833] Trial 19 finished with value: -0.4949959369056562 and parameters: {'beta': 0.1359578871775199, 'init_prev': 0.03208320467025896, 'n_contacts': 8, 'rand_seed': 167916}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:43,901] Trial 25 finished with value: -0.46740593690565607 and parameters: {'beta': 0.106121274847802, 'init_prev': 0.020949380166629415, 'n_contacts': 6, 'rand_seed': 965652}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,027] Trial 27 finished with value: -0.5131653813501005 and parameters: {'beta': 0.0342878140580294, 'init_prev': 0.049850734265389086, 'n_contacts': 6, 'rand_seed': 995857}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,059] Trial 26 finished with value: -0.45178593690565605 and parameters: {'beta': 0.01279053585550109, 'init_prev': 0.013592654796255635, 'n_contacts': 6, 'rand_seed': 717333}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,106] Trial 24 finished with value: -0.47987399246121165 and parameters: {'beta': 0.06278699924289378, 'init_prev': 0.028563181735690592, 'n_contacts': 10, 'rand_seed': 191570}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,148] Trial 21 finished with value: -0.45886343690565606 and parameters: {'beta': 0.06825369231990483, 'init_prev': 0.01769583168076677, 'n_contacts': 4, 'rand_seed': 59961}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,154] Trial 28 finished with value: -0.5118014924612116 and parameters: {'beta': 0.03466177558077176, 'init_prev': 0.04920693843565434, 'n_contacts': 7, 'rand_seed': 941636}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,248] Trial 32 finished with value: -0.5101656591278784 and parameters: {'beta': 0.03327097169309982, 'init_prev': 0.04804772639738838, 'n_contacts': 5, 'rand_seed': 571871}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,288] Trial 33 finished with value: -0.5125323257945449 and parameters: {'beta': 0.030235748071597665, 'init_prev': 0.049400211707656246, 'n_contacts': 5, 'rand_seed': 992949}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,347] Trial 29 finished with value: -0.513472325794545 and parameters: {'beta': 0.033229074556924545, 'init_prev': 0.04980244299817063, 'n_contacts': 5, 'rand_seed': 950607}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,370] Trial 30 finished with value: -0.513472325794545 and parameters: {'beta': 0.033186258070507534, 'init_prev': 0.049726091250762536, 'n_contacts': 5, 'rand_seed': 961535}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,391] Trial 34 finished with value: -0.5125323257945449 and parameters: {'beta': 0.034043886999593834, 'init_prev': 0.049546386064057875, 'n_contacts': 5, 'rand_seed': 996838}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,394] Trial 31 finished with value: -0.5125323257945449 and parameters: {'beta': 0.03453233058233213, 'init_prev': 0.04931432727235206, 'n_contacts': 5, 'rand_seed': 980271}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,668] Trial 36 finished with value: -0.516113436905656 and parameters: {'beta': 0.033538234233236604, 'init_prev': 0.04931149246991297, 'n_contacts': 7, 'rand_seed': 582183}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,669] Trial 35 finished with value: -0.513472325794545 and parameters: {'beta': 0.03614871132902712, 'init_prev': 0.049849525015371786, 'n_contacts': 5, 'rand_seed': 987517}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,671] Trial 38 finished with value: -0.5120312146834338 and parameters: {'beta': 0.0321235646269469, 'init_prev': 0.04948538248904738, 'n_contacts': 7, 'rand_seed': 917373}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,854] Trial 37 finished with value: -0.5167942702389895 and parameters: {'beta': 0.03292793181636787, 'init_prev': 0.04983891178816257, 'n_contacts': 7, 'rand_seed': 951373}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,875] Trial 40 finished with value: -0.5162356591278783 and parameters: {'beta': 0.03380411681570785, 'init_prev': 0.0496489170897043, 'n_contacts': 7, 'rand_seed': 929719}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,932] Trial 42 finished with value: -0.5153814924612116 and parameters: {'beta': 0.03665879944411343, 'init_prev': 0.04940364624705149, 'n_contacts': 5, 'rand_seed': 527347}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:44,967] Trial 45 finished with value: -0.5143459369056561 and parameters: {'beta': 0.03212594446036035, 'init_prev': 0.04881209512841181, 'n_contacts': 2, 'rand_seed': 998316}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,005] Trial 44 finished with value: -0.513472325794545 and parameters: {'beta': 0.036642963518545214, 'init_prev': 0.04987392954477107, 'n_contacts': 5, 'rand_seed': 467665}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,023] Trial 46 finished with value: -0.5020770480167671 and parameters: {'beta': 0.028668561674763372, 'init_prev': 0.03986318305366871, 'n_contacts': 4, 'rand_seed': 469789}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,023] Trial 43 finished with value: -0.5160639924612116 and parameters: {'beta': 0.03840932892187582, 'init_prev': 0.04984452300795703, 'n_contacts': 5, 'rand_seed': 504479}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,027] Trial 48 finished with value: -0.5018384369056561 and parameters: {'beta': 0.028080648426744865, 'init_prev': 0.03955932069792161, 'n_contacts': 4, 'rand_seed': 459295}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,042] Trial 39 finished with value: -0.5120312146834338 and parameters: {'beta': 0.03419298000717285, 'init_prev': 0.04947451542881073, 'n_contacts': 7, 'rand_seed': 993686}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,276] Trial 49 finished with value: -0.5015134369056562 and parameters: {'beta': 0.043203093509745445, 'init_prev': 0.03895746336340112, 'n_contacts': 2, 'rand_seed': 890512}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,278] Trial 47 finished with value: -0.49824510357232277 and parameters: {'beta': 0.027113723043322907, 'init_prev': 0.03696169859693052, 'n_contacts': 4, 'rand_seed': 466077}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,319] Trial 41 finished with value: -0.5114142702389894 and parameters: {'beta': 0.031330975922492926, 'init_prev': 0.049404793303803096, 'n_contacts': 2, 'rand_seed': 548641}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,360] Trial 54 finished with value: -0.5015134369056562 and parameters: {'beta': 0.045453834397924046, 'init_prev': 0.0391854377591971, 'n_contacts': 2, 'rand_seed': 456366}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,360] Trial 53 finished with value: -0.5024945480167672 and parameters: {'beta': 0.044066936175276364, 'init_prev': 0.03982280801451209, 'n_contacts': 2, 'rand_seed': 467526}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,374] Trial 50 finished with value: -0.5022564924612117 and parameters: {'beta': 0.04765352253906807, 'init_prev': 0.039636538106174055, 'n_contacts': 2, 'rand_seed': 436058}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,387] Trial 52 finished with value: -0.5012570480167672 and parameters: {'beta': 0.04664323062776957, 'init_prev': 0.039284915576735344, 'n_contacts': 4, 'rand_seed': 464197}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,476] Trial 56 finished with value: -0.4965392702389894 and parameters: {'beta': 0.015922765970832812, 'init_prev': 0.039567390025823726, 'n_contacts': 2, 'rand_seed': 453104}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,480] Trial 51 finished with value: -0.5030217702389894 and parameters: {'beta': 0.04698004870233414, 'init_prev': 0.040586701956071634, 'n_contacts': 2, 'rand_seed': 898004}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,640] Trial 58 finished with value: -0.497064825794545 and parameters: {'beta': 0.015067936196999, 'init_prev': 0.03969364504392992, 'n_contacts': 7, 'rand_seed': 456335}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,643] Trial 55 finished with value: -0.5019945480167672 and parameters: {'beta': 0.04525716048006801, 'init_prev': 0.03943902565142538, 'n_contacts': 2, 'rand_seed': 464528}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,660] Trial 59 finished with value: -0.4990012146834339 and parameters: {'beta': 0.043148013625057074, 'init_prev': 0.04027217896118974, 'n_contacts': 7, 'rand_seed': 445321}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,778] Trial 57 finished with value: -0.5020926035723227 and parameters: {'beta': 0.025367089436645587, 'init_prev': 0.039699038969874124, 'n_contacts': 2, 'rand_seed': 424082}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,933] Trial 61 finished with value: -0.5064903813501006 and parameters: {'beta': 0.04651831002333319, 'init_prev': 0.04591100455303464, 'n_contacts': 9, 'rand_seed': 628583}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:45,962] Trial 64 finished with value: -0.506969825794545 and parameters: {'beta': 0.044447515468434125, 'init_prev': 0.04658979040952751, 'n_contacts': 9, 'rand_seed': 630370}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,003] Trial 65 finished with value: -0.5107937146834339 and parameters: {'beta': 0.044466699849850175, 'init_prev': 0.04615062544332026, 'n_contacts': 9, 'rand_seed': 895636}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,169] Trial 60 finished with value: -0.4997870480167672 and parameters: {'beta': 0.048947418436240175, 'init_prev': 0.040614216416346986, 'n_contacts': 7, 'rand_seed': 883221}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,180] Trial 62 finished with value: -0.5067339924612116 and parameters: {'beta': 0.04690989789857649, 'init_prev': 0.046325172191867214, 'n_contacts': 9, 'rand_seed': 881683}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,287] Trial 63 finished with value: -0.506969825794545 and parameters: {'beta': 0.04575299194180441, 'init_prev': 0.04646848305505414, 'n_contacts': 9, 'rand_seed': 636272}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,294] Trial 68 finished with value: -0.5067070480167671 and parameters: {'beta': 0.023082885597055843, 'init_prev': 0.04648668683213909, 'n_contacts': 7, 'rand_seed': 623896}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,294] Trial 71 finished with value: -0.506299825794545 and parameters: {'beta': 0.015658244116536945, 'init_prev': 0.046041845295603304, 'n_contacts': 7, 'rand_seed': 654136}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,317] Trial 66 finished with value: -0.5107217702389895 and parameters: {'beta': 0.04806514898752032, 'init_prev': 0.04609895952437706, 'n_contacts': 7, 'rand_seed': 645557}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,318] Trial 69 finished with value: -0.5110506591278784 and parameters: {'beta': 0.016863518502375524, 'init_prev': 0.046545288050022456, 'n_contacts': 7, 'rand_seed': 633093}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,391] Trial 74 finished with value: -0.506601214683434 and parameters: {'beta': 0.02366099462998718, 'init_prev': 0.04663192885029669, 'n_contacts': 8, 'rand_seed': 621674}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,391] Trial 70 finished with value: -0.5067070480167671 and parameters: {'beta': 0.022766641974117954, 'init_prev': 0.046787890512729545, 'n_contacts': 7, 'rand_seed': 610725}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,434] Trial 73 finished with value: -0.5063626035723228 and parameters: {'beta': 0.0239302612100303, 'init_prev': 0.04619166532034476, 'n_contacts': 8, 'rand_seed': 638092}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,484] Trial 67 finished with value: -0.5067070480167671 and parameters: {'beta': 0.025393730019332917, 'init_prev': 0.04645329368411897, 'n_contacts': 7, 'rand_seed': 636063}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,636] Trial 78 finished with value: -0.5063867702389895 and parameters: {'beta': 0.02164691638772139, 'init_prev': 0.04651234952675082, 'n_contacts': 8, 'rand_seed': 633171}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,638] Trial 72 finished with value: -0.506601214683434 and parameters: {'beta': 0.02248244365769078, 'init_prev': 0.04674535658315714, 'n_contacts': 8, 'rand_seed': 658007}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,706] Trial 75 finished with value: -0.5060367702389894 and parameters: {'beta': 0.025203624338242327, 'init_prev': 0.04636693471598282, 'n_contacts': 8, 'rand_seed': 645448}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,727] Trial 77 finished with value: -0.5061178813501006 and parameters: {'beta': 0.023820323224988862, 'init_prev': 0.04595781145863685, 'n_contacts': 8, 'rand_seed': 626485}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,909] Trial 79 finished with value: -0.5063626035723228 and parameters: {'beta': 0.022375964113718904, 'init_prev': 0.04628280958689703, 'n_contacts': 8, 'rand_seed': 642470}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,952] Trial 80 finished with value: -0.5007878813501005 and parameters: {'beta': 0.020942140102709648, 'init_prev': 0.04259620426399234, 'n_contacts': 3, 'rand_seed': 852863}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,962] Trial 84 finished with value: -0.5014887146834339 and parameters: {'beta': 0.05588044198778258, 'init_prev': 0.04311739770433148, 'n_contacts': 3, 'rand_seed': 761349}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,967] Trial 82 finished with value: -0.5020314924612116 and parameters: {'beta': 0.021242520502909864, 'init_prev': 0.04325258061097305, 'n_contacts': 3, 'rand_seed': 765047}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:46,974] Trial 76 finished with value: -0.5057892702389895 and parameters: {'beta': 0.02398712260225186, 'init_prev': 0.04632821052105929, 'n_contacts': 8, 'rand_seed': 637704}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,036] Trial 87 finished with value: -0.5026953813501005 and parameters: {'beta': 0.020781428859643947, 'init_prev': 0.04378966352203858, 'n_contacts': 6, 'rand_seed': 513210}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,086] Trial 81 finished with value: -0.5010492702389895 and parameters: {'beta': 0.010112454583116152, 'init_prev': 0.04347152929106646, 'n_contacts': 3, 'rand_seed': 510157}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,110] Trial 83 finished with value: -0.5007878813501005 and parameters: {'beta': 0.016699349683781426, 'init_prev': 0.042620222011509785, 'n_contacts': 3, 'rand_seed': 770322}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,275] Trial 85 finished with value: -0.5014887146834339 and parameters: {'beta': 0.05642784796107949, 'init_prev': 0.04320290294931737, 'n_contacts': 3, 'rand_seed': 583691}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,329] Trial 91 finished with value: -0.5008356591278784 and parameters: {'beta': 0.01891357185864384, 'init_prev': 0.042877678709097325, 'n_contacts': 6, 'rand_seed': 518215}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,330] Trial 93 finished with value: -0.5090831591278784 and parameters: {'beta': 0.02040291185283774, 'init_prev': 0.048130053761924585, 'n_contacts': 6, 'rand_seed': 510950}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,333] Trial 92 finished with value: -0.501352325794545 and parameters: {'beta': 0.020766896854564537, 'init_prev': 0.04317639515613413, 'n_contacts': 6, 'rand_seed': 511287}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,362] Trial 94 finished with value: -0.5015314924612118 and parameters: {'beta': 0.039978109585131094, 'init_prev': 0.04353543432214712, 'n_contacts': 6, 'rand_seed': 755640}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,400] Trial 89 finished with value: -0.5083589924612117 and parameters: {'beta': 0.020164639581070502, 'init_prev': 0.047811570162748826, 'n_contacts': 6, 'rand_seed': 761359}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,413] Trial 88 finished with value: -0.5008356591278784 and parameters: {'beta': 0.01854670886902243, 'init_prev': 0.04269767069130883, 'n_contacts': 6, 'rand_seed': 507352}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,414] Trial 86 finished with value: -0.5010959369056561 and parameters: {'beta': 0.0188480066723251, 'init_prev': 0.04312260789467864, 'n_contacts': 6, 'rand_seed': 517685}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,465] Trial 90 finished with value: -0.5024456591278783 and parameters: {'beta': 0.010132680027743635, 'init_prev': 0.04344516125606053, 'n_contacts': 6, 'rand_seed': 516281}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,516] Trial 96 finished with value: -0.5094837146834338 and parameters: {'beta': 0.03996873944813089, 'init_prev': 0.048381504196801525, 'n_contacts': 6, 'rand_seed': 506412}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,546] Trial 95 finished with value: -0.5010959369056561 and parameters: {'beta': 0.01061826801251261, 'init_prev': 0.043157659316225394, 'n_contacts': 6, 'rand_seed': 501674}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,620] Trial 97 finished with value: -0.5088401035723229 and parameters: {'beta': 0.01810262965285105, 'init_prev': 0.04795272153337613, 'n_contacts': 6, 'rand_seed': 511721}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,626] Trial 98 finished with value: -0.5094837146834338 and parameters: {'beta': 0.03981280044715889, 'init_prev': 0.04833163447249708, 'n_contacts': 6, 'rand_seed': 569291}. Best is trial 8 with value: -0.5169209369056561.
[I 2025-08-04 18:05:47,864] Trial 99 finished with value: -0.5094414924612116 and parameters: {'beta': 0.03868351077682247, 'init_prev': 0.048267625410711654, 'n_contacts': 5, 'rand_seed': 953739}. Best is trial 8 with value: -0.5169209369056561.

Making results structure...
Processed 100 trials; 0 failed
Best pars: {'beta': 0.01816702263490502, 'init_prev': 0.049784007865639836, 'n_contacts': 4, 'rand_seed': 575437}
Removed existing calibration file starsim_calibration.db

Let’s look at the best parameters that were found. Note that the rand_seed was selected at random, but the other parameters are meaningful.

calib.best_pars

{'beta': 0.01816702263490502,
 'init_prev': 0.049784007865639836,
 'n_contacts': 4,
 'rand_seed': 575437}

Once the calibration is complete, we can compare the guess values to the best values found by calling check_fit.

# Confirm - Note the comparison is here configured over n_reps=15 replicates
sc.printcyan('\nConfirming fit...')

# Increase replicates to 15 for more representative results when running check_fit
calib.build_kw['n_reps'] = 15

calib.check_fit(do_plot=False)

Confirming fit...



Checking fit...

Fit with original pars: -0.5454663813501005

Fit with best-fit pars: -0.5167228813501004

✗ Calibration did not improve fit as the objective got worse (-0.5454663813501005 --> -0.5167228813501004), but this sometimes happens stochastically and is not necessarily an error

False

After calling check_fit, we can plot the results. This first plot shows the Normal likelihood distributions from each of the 15 simulations we did in check_fit as the colored lines. The vertical dashed line is located at the real (expected) data. Top row is the “guess” values and the bottom row is the new “best” parameters. We want the vertical dashed line to cross the Gaussians at high points, representing high likelihood.

calib.plot();

Another way to plot the results is via bootstrapping. Here we repeatedly choose 15 from the n_reps=15 simulations (with replacement), compute the average (or sum for some components), and repeatedly calculate the mean. We then plot the distribution of means, and hope it lands near the vertical dashed lines representing the real data.

calib.plot(bootstrap=True); # Pass bootstrap=True to produce this plot

We can view some plots of the final fitted results. Whereas the two plots above were from the check_fit, running both “guess” and “best” parameters, here we make make new simulations to visualize the results.

g = calib.plot_final(); # Run the model for build_kw['n_reps'] = 15 replicates

calib

<starsim.calibration.Calibration at 0x77eed52ead20>
[<class 'starsim.calibration.Calibration'>, <class 'sciris.sc_printing.prettyobj'>]
————————————————————————————————————————————————————————————————————————
Methods:
  _eval_fit()             make_study()            run_sim()               
  _sample_from_trial()    parse_study()           run_trial()             
  build_fn()              plot()                  run_workers()           
  calibrate()             plot_final()            to_df()                 
  check_fit()             plot_optuna()           to_json()               
  eval_fn()               remove_db()             worker()                
————————————————————————————————————————————————————————————————————————
 after_fits: -0.5167228813501004
 after_msim: MultiSim(n_sims: 15; base: Sim(n=2000;
             2020.01.01—2020.02.12; not ini [...]
before_fits: -0.5454663813501005
before_msim: MultiSim(n_sims: 15; base: Sim(n=2000;
             2020.01.01—2020.02.12; not ini [...]
  best_pars: {'beta': 0.01816702263490502, 'init_prev':
             0.049784007865639836, 'n_c [...]
   build_fn: <function build_sim at 0x77eed5010860>
   build_kw: {'n_reps': 15}
 calib_pars: {'beta': {'low': 0.01, 'high': 0.3, 'guess': 0.15,
             'suggest_type': 's [...]
 calibrated: True
 components: [Calibration component with name Disease prevalence]
         df:     index  mismatch      beta  init_prev  n_contacts
             rand_seed
             8     [...]
        die: True
    elapsed: 7.303730487823486
    eval_fn: <bound method Calibration._eval_fit of
             <starsim.calibration.Calibrati [...]
    eval_kw: {}
      label: None
   prune_fn: None
     reseed: True
   run_args: #0. 'n_trials':    5
             #1. 'n_workers':   20
             #2. 'debug':       False
             # [...]
        sim: Sim(n=2000; 2020.01.01—2020.02.12; not initialized)
      study: <optuna.study.study.Study object at 0x77eed3d0b770>
 study_data: #0. 'index':      [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
             12, 13, 14,  [...]
    verbose: True
————————————————————————————————————————————————————————————————————————

Optuna has lots of diagnostic plots that we can explore. Possible plots include:

• plot_contour
• plot_edf
• plot_hypervolume_history
• plot_intermediate_values
• plot_optimization_history
• plot_parallel_coordinate
• plot_param_importances
• plot_pareto_front
• plot_rank
• plot_slice
• plot_terminator_improvement
• plot_timeline

Here are some examples:

calib.plot_optuna('plot_optimization_history'); # Plot the optimization history

/home/cliffk/idm/starsim/starsim/calibration.py:433: ExperimentalWarning: plot_optimization_history is experimental (supported from v2.2.0). The interface can change in the future.
  fig = getattr(vis, method)(self.study)

calib.plot_optuna('plot_contour');

/home/cliffk/idm/starsim/starsim/calibration.py:433: ExperimentalWarning: plot_contour is experimental (supported from v2.2.0). The interface can change in the future.
  fig = getattr(vis, method)(self.study)
[W 2025-08-04 18:05:58,665] Output figures of this Matplotlib-based `plot_contour` function would be different from those of the Plotly-based `plot_contour`.

calib.plot_optuna('plot_param_importances');

Could not run plot_param_importances: Tried to import 'sklearn' but failed. Please make sure that the package is installed correctly to use this feature. Actual error: No module named 'sklearn'.

/home/cliffk/idm/starsim/starsim/calibration.py:433: ExperimentalWarning: plot_param_importances is experimental (supported from v2.2.0). The interface can change in the future.
  fig = getattr(vis, method)(self.study)

If you choose not to use components, you can always create your own mismatch function, as in the following example:

my_data = (ss.date('2020-01-12'), 0.13)

def eval(sim, expected):
    # Compute the squared error at one point in time.
    # expected will contain my_data in this example due to eval_kw
    date, p = expected
    if not isinstance(sim, ss.MultiSim):
        sim = ss.MultiSim(sims=[sim])

    ret = 0
    for s in sim.sims:
        ind = np.searchsorted(s.results.timevec, date, side='left')
        prev = s.results.sir.prevalence[ind]
        ret += (prev - p)**2
    return ret


# Define the calibration parameters
calib_pars = dict(
    beta = dict(low=0.01, high=0.30, guess=0.15, suggest_type='suggest_float', log=True),
)

# Make the sim and data
sim = make_sim()

# Make the calibration
calib = ss.Calibration(
    calib_pars = calib_pars,
    sim = sim,
    build_fn = build_sim,
    build_kw = dict(n_reps=2), # Two reps per point
    reseed = True,
    eval_fn = eval, # Will call my_function(msim, eval_kwargs)
    eval_kw = dict(expected=my_data), # Will call eval(sim, **eval_kw)
    total_trials = 20,
    n_workers = None, # None indicates to use all available CPUs
    die = True,
    debug = debug,
)

# Perform the calibration
sc.printcyan('\nPeforming calibration...')
calib.calibrate()

# Check
calib.check_fit()

[I 2025-08-04 18:06:00,518] A new study created in RDB with name: starsim_calibration

Peforming calibration...

Removed existing calibration file starsim_calibration.db

sqlite:///starsim_calibration.db

[I 2025-08-04 18:06:01,051] Trial 1 finished with value: 0.024646500000000002 and parameters: {'beta': 0.10812104554755586, 'rand_seed': 457085}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,165] Trial 6 finished with value: 0.024646500000000002 and parameters: {'beta': 0.04272537526112574, 'rand_seed': 296821}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,173] Trial 5 finished with value: 0.024646500000000002 and parameters: {'beta': 0.10369447330355346, 'rand_seed': 365326}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,208] Trial 4 finished with value: 0.024646500000000002 and parameters: {'beta': 0.019254887946383973, 'rand_seed': 385376}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,214] Trial 2 finished with value: 0.024646500000000002 and parameters: {'beta': 0.01350572406574447, 'rand_seed': 314951}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,297] Trial 3 finished with value: 0.024646500000000002 and parameters: {'beta': 0.06764637766228687, 'rand_seed': 401985}. Best is trial 1 with value: 0.024646500000000002.
[I 2025-08-04 18:06:01,402] Trial 11 finished with value: 0.02431325 and parameters: {'beta': 0.2528275433685965, 'rand_seed': 687263}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,411] Trial 16 finished with value: 0.024646500000000002 and parameters: {'beta': 0.05351495410795384, 'rand_seed': 921098}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,411] Trial 18 finished with value: 0.02431325 and parameters: {'beta': 0.21032836388449866, 'rand_seed': 660102}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,416] Trial 10 finished with value: 0.024646500000000002 and parameters: {'beta': 0.014400108380040829, 'rand_seed': 401159}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,425] Trial 19 finished with value: 0.024646500000000002 and parameters: {'beta': 0.09603932462994463, 'rand_seed': 135104}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,438] Trial 13 finished with value: 0.024646500000000002 and parameters: {'beta': 0.025122279640374723, 'rand_seed': 939981}. Best is trial 11 with value: 0.02431325.
[I 2025-08-04 18:06:01,454] Trial 9 finished with value: 0.024202 and parameters: {'beta': 0.2826094882235986, 'rand_seed': 529560}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,459] Trial 17 finished with value: 0.024646500000000002 and parameters: {'beta': 0.10525677575114477, 'rand_seed': 416325}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,474] Trial 14 finished with value: 0.02431325 and parameters: {'beta': 0.19551294607769445, 'rand_seed': 501594}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,491] Trial 7 finished with value: 0.02453425 and parameters: {'beta': 0.1558078609373016, 'rand_seed': 553213}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,505] Trial 15 finished with value: 0.024646500000000002 and parameters: {'beta': 0.05295489053032275, 'rand_seed': 957211}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,505] Trial 12 finished with value: 0.024646500000000002 and parameters: {'beta': 0.011810378879347867, 'rand_seed': 293875}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,506] Trial 8 finished with value: 0.024646500000000002 and parameters: {'beta': 0.030139367611309073, 'rand_seed': 591577}. Best is trial 9 with value: 0.024202.
[I 2025-08-04 18:06:01,518] Trial 0 finished with value: 0.024646500000000002 and parameters: {'beta': 0.018485014725483816, 'rand_seed': 976184}. Best is trial 9 with value: 0.024202.

Making results structure...

Processed 20 trials; 0 failed

Best pars: {'beta': 0.2826094882235986, 'rand_seed': 529560}

Removed existing calibration file starsim_calibration.db



Checking fit...

Fit with original pars: 0.024646500000000002

Fit with best-fit pars: 0.024021

✓ Calibration improved fit 0.024646500000000002 --> 0.024021

True

For more, take a look at test_calibration.py in the tests directory.