samples
Create a class for storing a large number of simulations.
Hierarchy - result: parameters, seed, beta - samples: collection of results with same parameters but different seeds - dataset: collection of samples (with different parameters)
Classes
| Name | Description |
|---|---|
| Dataset | Store the results and provide options for filtering |
| Samples | Stores CSV outputs and summary dataframes |
Dataset
samples.Dataset(folder=None, results=None, *args, **kwargs)Store the results and provide options for filtering
Args:
folder (str/path): path to a folder containing `.zip` Samples files
results (odict): pre-constructed dict of Samples objects (used internally by filter())Attributes
| Name | Description |
|---|---|
| ids | Return dictionary of parameters across results |
Methods
| Name | Description |
|---|---|
| filter | Return results matching particular ids |
| get | Retrieve a single result from a filter operation; raises if 0 or >1 match. |
filter
samples.Dataset.filter(**kwargs)Return results matching particular ids kwargs: key-value pairs, that should be present in Sample.id, to filter results on Can specify a list/set of values to match multiple items
get
samples.Dataset.get(**kwargs)Retrieve a single result from a filter operation; raises if 0 or >1 match.
Example: res = dataset.get(scenario='foo') instead of dataset.filter(scenario='foo')[0]
Samples
samples.Samples(fname, memory_buffer=True, preload=False)Stores CSV outputs and summary dataframes
To construct, use Samples.new(). To read an existing one, use Samples(fname). The sample files are just ZIP archives with plain text CSV and TXT files so they can be easily accessed externally as well.
Attributes
| Name | Description |
|---|---|
| columns | Alias summary dataframe columns |
| id | Return a dictionary with the identifiers and associated values |
| identifier | Return tuple identifier for this run |
| index | Alias summary dataframe index |
| seeds | Return array of all seeds |
Methods
| Name | Description |
|---|---|
| apply | Apply/map function to every dataframe |
| copy | Shallow copy - shared cache, copied summary |
| get | Retrieve dataframe and summary row |
| items | Iterate over seeds and dataframes |
| new | Create a new Samples instance |
| preload | Load all dataframes into cache |
apply
samples.Samples.apply(fcn, *args, **kwargs)Apply/map function to every dataframe
The function will be applied to every individual dataframe in the collection.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| fcn | A function to apply. It should take in a dataframe | required | |
| args | Additional arguments for fcn |
() |
|
| kwargs | Additional arguments for fcn |
{} |
Returns: A list with the output of fcn
copy
samples.Samples.copy()Shallow copy - shared cache, copied summary
This allows efficient filtering of seeds within runs by removing rows from the copy’s summary, while not reloading or duplicating any of the dataframes in memory
get
samples.Samples.get(seed)Retrieve dataframe and summary row
Use Samples[seed] to read only the dataframe. Use Samples.get(seed) to read both the dataframe and summary row
items
samples.Samples.items()Iterate over seeds and dataframes
Example usage
res = Samples(…) for seed, (row, df) in res.items(): …
Tuple with
| Name | Type | Description |
|---|---|---|
| - seed | ||
| - Samples.get(seed) i.e. a tuple with - the summary dataframe row for the requested seed - the corresponding CSV output for that run |
new
samples.Samples.new(
outputs,
identifiers=None,
folder='.',
fname=None,
verbose=True,
)Create a new Samples instance
Pass in outputs (list of tuples (df:pd.DataFrame, summary_row:dict)) to wrap them in a Samples instance.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| outputs | A list of tuples (df:pd.DataFrame, summary_row:dict) where the summary row as an entry ‘seed’ for the seed | required | |
| identifiers | A list of columns to use as identifiers. These should appear in the summary dataframe and should have the same value for all samples. This is useful when generating multiple sets of results e.g., for scenarios (optional) | None |
|
| folder | path |
the folder to save the zip file in | '.' |
| fname | str / BytesIO |
filename for the zip file; auto-generated from identifiers if not supplied. If an io.BytesIO instance is passed, the zip file is created in-memory |
None |
| verbose | bool | whether to print the save path (default True) | True |
preload
samples.Samples.preload()Load all dataframes into cache
This is done based on the seeds in self.seeds, therefore if some of the seeds are removed prior to preloading, then those dataframes will not be loaded