Deployment

Since Starsim is implemented in pure Python, it can easily be deployed on the cloud. Here we describe some different approaches for doing this.

Virtual machine

One of the most common approaches is to run Starsim on a single large virtual machine (VM). By default, ss.MultiSim (and ss.parallel()) will use all available cores. If your script already makes use of these, then you don’t need to make any more changes:

import sciris as sc
import starsim as ss

base_pars = sc.objdict(
    n_agents = 10e3,
    diseases = sc.objdict(
        type = 'sis',
        beta = 0.1,
    ),
    networks = 'random',
    rand_seed = 1,
    verbose = False,
)

# Generate sims in serial
sims = sc.autolist() # Can also just use []
for i in range(10):
    pars = base_pars.copy()
    pars.diseases.beta *= sc.perturb()
    pars.rand_seed = i
    sim = ss.Sim(pars)
    sims += sim

# Run in parallel
msim = ss.parallel(sims)
msim.plot(legend=False)

Exception ignored in: <_io.BytesIO object at 0x7f61d855f150>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 705, in _check_dist_arg
    @classmethod
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d8378220>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 705, in _check_dist_arg
    @classmethod
BufferError: Existing exports of data: object cannot be re-sized

Figure(768x576)

Note that this example uses sc.objdict() rather than dict() – either work, but it means you can use pars.diseases.beta rather than pars['diseases']['beta']. You could also create full Starsim objects (e.g. diseases = ss.SIS() and then modify pars.diseases.pars.beta).

In some cases, creating the sim is itself a time-consuming step (especially if hundreds or thousands are being generated). In this case, you can write a make_sim() function and parallelize that too:

def make_sim(i, pars):
    pars.diseases.beta *= sc.perturb() # Don't need to copy pars since implicitly copied via the pickling process
    pars.rand_seed = i
    sim = ss.Sim(pars)
    return sim

sims = sc.parallelize(make_sim, range(10), pars=base_pars, serial=False)
msim = ss.parallel(sims)
msim.plot(legend=False)

Exception ignored in: <_io.BytesIO object at 0x7f61d7f2aca0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/sciris/sc_odict.py", line 1208, in __getattribute__
    def __getattribute__(self, attr):
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f2b3d0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/sciris/sc_odict.py", line 1208, in __getattribute__
    def __getattribute__(self, attr):
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61f051b010>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/sciris/sc_odict.py", line 1208, in __getattribute__
    def __getattribute__(self, attr):
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: Exception ignored in: <_io.BytesIO object at 0x7f61d7f2a700><_io.BytesIO object at 0x7f61d7f2a700>

Exception ignored in: Traceback (most recent call last):
  File "<frozen abc>", line 117, in __instancecheck__
<_io.BytesIO object at 0x7f61d7f2a700>BufferError
Traceback (most recent call last):
Traceback (most recent call last):
:   File "<frozen abc>", line 117, in __instancecheck__
  File "<frozen abc>", line 117, in __instancecheck__
BufferErrorExisting exports of data: object cannot be re-sizedBufferError: Exception ignored in: : 
Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f2a700>Existing exports of data: object cannot be re-sizedException ignored in: 


<_io.BytesIO object at 0x7f61d7f40720>
Exception ignored in: Traceback (most recent call last):
Exception ignored in: <_io.BytesIO object at 0x7f61d7f40720>Traceback (most recent call last):

  File "<frozen abc>", line 117, in __instancecheck__
  File "<frozen abc>", line 117, in __instancecheck__
<_io.BytesIO object at 0x7f61d7f40720>BufferErrorTraceback (most recent call last):

BufferError:   File "<frozen abc>", line 117, in __instancecheck__
BufferError: Traceback (most recent call last):
Existing exports of data: object cannot be re-sized:   File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sized
Existing exports of data: object cannot be re-sized
BufferErrorException ignored in: : Exception ignored in: 
Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f40ae0><_io.BytesIO object at 0x7f61d7f40720>Exception ignored in: 


Exception ignored in: <_io.BytesIO object at 0x7f61d7f40ae0>Traceback (most recent call last):
Traceback (most recent call last):
<_io.BytesIO object at 0x7f61d7f40ae0>  File "<frozen abc>", line 117, in __instancecheck__


  File "<frozen abc>", line 117, in __instancecheck__
BufferErrorTraceback (most recent call last):
Traceback (most recent call last):
BufferError: :   File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sizedExisting exports of data: object cannot be re-sized  File "<frozen abc>", line 117, in __instancecheck__
BufferError
: BufferError
Exception ignored in: : Existing exports of data: object cannot be re-sizedException ignored in: <_io.BytesIO object at 0x7f61d7f41030>Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f40ae0>



Traceback (most recent call last):
Exception ignored in: Traceback (most recent call last):
Exception ignored in: <_io.BytesIO object at 0x7f61d7f41030>  File "<frozen abc>", line 117, in __instancecheck__
<_io.BytesIO object at 0x7f61d7f41030>  File "<frozen abc>", line 117, in __instancecheck__

BufferErrorBufferError
: Traceback (most recent call last):
: Traceback (most recent call last):
Existing exports of data: object cannot be re-sized  File "<frozen abc>", line 117, in __instancecheck__
  File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sizedBufferErrorBufferError: 

Exception ignored in: Existing exports of data: object cannot be re-sizedException ignored in: : Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f41580>
<_io.BytesIO object at 0x7f61d7f41030>

Exception ignored in: 
Traceback (most recent call last):
<_io.BytesIO object at 0x7f61d7f41580>Exception ignored in: Traceback (most recent call last):
  File "<frozen abc>", line 117, in __instancecheck__
<_io.BytesIO object at 0x7f61d7f41580>
BufferError  File "<frozen abc>", line 117, in __instancecheck__
: 
Traceback (most recent call last):
BufferErrorExisting exports of data: object cannot be re-sized  File "<frozen abc>", line 117, in __instancecheck__
Traceback (most recent call last):
: 
BufferErrorException ignored in: :   File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f41ad0>Existing exports of data: object cannot be re-sized


BufferErrorException ignored in: Exception ignored in: <_io.BytesIO object at 0x7f61d7f41ad0><_io.BytesIO object at 0x7f61d7f41580>

: Traceback (most recent call last):
Traceback (most recent call last):
  File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sizedTraceback (most recent call last):
BufferError  File "<frozen abc>", line 117, in __instancecheck__
  File "<frozen abc>", line 117, in __instancecheck__

: BufferErrorExisting exports of data: object cannot be re-sizedException ignored in: 
<_io.BytesIO object at 0x7f61d7f41ad0>
BufferErrorTraceback (most recent call last):
: : Existing exports of data: object cannot be re-sized  File "<frozen abc>", line 117, in __instancecheck__
Existing exports of data: object cannot be re-sizedBufferError

: Exception ignored in: Existing exports of data: object cannot be re-sized<_io.BytesIO object at 0x7f61d7f41ad0>

Traceback (most recent call last):
  File "<frozen abc>", line 117, in __instancecheck__
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f2bba0>
Traceback (most recent call last):
Exception ignored in: Exception ignored in:   File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/copy.py", line 248, in _reconstruct
<_io.BytesIO object at 0x7f61d7f2bba0><_io.BytesIO object at 0x7f61d7f2bba0>

Traceback (most recent call last):
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/copy.py", line 152, in deepcopy
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/copy.py", line 152, in deepcopy
            rv = reductor(4)def _reconstruct(x, memo, func, args,rv = reductor(4)


BufferErrorBufferError: BufferError: Existing exports of data: object cannot be re-sized: Existing exports of data: object cannot be re-sized

Existing exports of data: object cannot be re-sizedException ignored in: 
<_io.BytesIO object at 0x7f61d7f2bba0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/copy.py", line 248, in _reconstruct
    def _reconstruct(x, memo, func, args,
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f2a700>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f40720>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f40ae0>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f41030>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f41580>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f41ad0>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 261, in _rebuild
    out = unpickling_func(*args)
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d7f2bba0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/dill/_dill.py", line 436, in find_class
    def find_class(self, module, name):
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f6200307f60>
Traceback (most recent call last):
  File "/home/runner/work/starsim/starsim/starsim/time.py", line 873, in __new__
    def __new__(cls, value=None, base=None, **kwargs):
BufferError: Existing exports of data: object cannot be re-sized
Exception ignored in: <_io.BytesIO object at 0x7f61d855fc90>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.7/x64/lib/python3.13/site-packages/matplotlib/_api/__init__.py", line 86, in check_isinstance
    for k, v in kwargs.items():
BufferError: Existing exports of data: object cannot be re-sized

Figure(768x576)

Note that parallelizing the build process pickles and unpickles the sims, which can be an expensive operation. make_sim() functions can often get quite complicated, so it’s often good software engineering practice to separate them out anyway. You can use the serial=True argument of Sciris’ sc.parallelize() function (which is what ss.parallel() calls under the hood) in order to run in serial, to see if it’s the same speed or faster.

While the traditional way to run on a VM is via SSH and terminal, it is also possible to run remotely via VS Code (and Cursor etc.), PyCharm, or Spyder. You can also run a Jupyter server on the VM and access it that way (we like The Littlest JupyterHub).

Dask and Coiled

Adapting the examples above, we can fairly easily make Starsim simulations run using other popular tools such as Dask and Joblib. Here’s a Dask example:

import dask
import dask.distributed as dd
import numpy as np
import starsim as ss


def run_sim(index, beta):
    """ Run a standard simulation """
    label = f'Sim {index}, beta={beta:n}'
    sis = ss.SIS(beta=beta)
    sim = ss.Sim(label=label, networks='random', diseases=sis, rand_seed=index, verbose=False)
    sim.run()
    sim.shrink() # Remove People and other states to make pickling faster
    return sim


if __name__ == '__main__':

    # Run settings
    n = 8
    n_workers = 4
    betas = 0.1*np.sort(np.random.random(n))

    # Create and queue the Dask jobs
    client = dd.Client(n_workers=n_workers)
    queued = []
    for i,beta in enumerate(betas):
        run = dask.delayed(run_sim)(i, beta)
        queued.append(run)

    # Run and process the simulations
    sims = list(dask.compute(*queued))
    msim = ss.MultiSim(sims)
    msim.plot()

Figure(768x576)

Coiled, which is a paid service by Dask that allows auto-scaling across clusters, has a similar syntax:

import sciris as sc
import starsim as ss
import coiled
import dask.distributed as dd

# Parameters
n_workers = 50
n = 1000

def run_sim(seed):
    sim = ss.Sim(n_agents=100e3, dur=100, diseases='sis', networks='random', rand_seed=seed)
    sim.run().shrink()
    return sim

# Set up cluster
cluster = coiled.Cluster(n_workers=n_workers, workspace="<your_coiled_workspace>")
client = cluster.get_client()

# Set up futures
futures = []
for seed in range(n):
    future = client.submit(run_sim, seed)
    futures.append(future)

# Run
sims = client.gather(futures)

# Plot
msim = ss.MultiSim(sims)
msim.plot()

(Note: You will need a Coiled subscription to run this example.)

Interactive dashboards

Another common desire is to make interactive dashboards. There are many ways to do this, including Shiny for Python, Voila, and Panel, but the simplest is probably Streamlit:

import streamlit as st
import starsim as ss

def run_sim(beta, n_agents):
    sis = ss.SIS(beta=beta)
    sim = ss.Sim(
        n_agents = n_agents,
        diseases = sis,
        networks = 'random',
    )
    sim.run()
    sim.label = f'Beta={beta:n} • Agents={n_agents:,} • Time={sim.timer.total:0.1f} s'
    return sim

# Create the Streamlit interface
st.title('SIS Dashboard')
beta = st.slider('Transmission rate (beta)', 0.0, 1.0, 0.1)
n_agents = st.slider('Number of agents', 1_000, 100_000, 10_000)

# Run simulation and plot results
sim = run_sim(beta, n_agents)
fig = sim.diseases.sis.plot()
fig.suptitle(sim.label)
st.pyplot(fig)

This example is saved in this folder as streamlit.py, and (after pip install streamlit) can be run with streamlit run streamlit.py. This should give something like this:

Streamlit example