leaspy.algo.simulate.simulate module

class SimulationAlgorithm(settings)

Bases: AbstractAlgo

To simulate new data given existing one by learning the individual parameters joined distribution.

You can choose to only learn the distribution of a group of patient. To do so, choose the cofactor(s) and the cofactor(s) state of the wanted patient in the settings. For instance, for an Alzheimer’s disease patient, you can load a genetic cofactor informative of the APOE4 carriers. Choose cofactor [‘genetic’] and cofactor_state [‘APOE4’] to simulate only APOE4 carriers.

Parameters:

settingsAlgorithmSettings

The algorithm settings. They may include the following parameters, described in __Attributes__ section:

noise

bandwidth_method

cofactor

cofactor_state

number_of_subjects

mean_number_of_visits, std_number_of_visits, min_number_of_visits, max_number_of_visits

delay_btw_visits

reparametrized_age_bounds

sources_method

prefix

features_bounds

features_bounds_nb_subjects_factor

Raises:

LeaspyAlgoInputError: If algorithm parameters are of bad type or do not comply to detailed requirements.

Notes

The baseline ages are no more jointly learnt with individual parameters. Instead, we jointly learn the _reparametrized_ baseline ages, together with individual parameters. The baseline ages are then reconstructed from the simulated reparametrized baseline ages and individual parameters.

By definition, the relation between age and reparametrized age is:

$\psi_i (t) = e^{\xi_i} (t - \tau_i) + \bar{\tau}$

with $t$ the real age, $\psi_i (t)$ the reparametrized age, $\xi_i$ the individual log-acceleration parameter, $\tau_i$ the individual time-shift parameter and $\bar{\tau}$ the mean conversion age derived by the model object.

One can restrict the interval of the baseline reparametrized age to be _learnt_ in kernel, by setting bounds in reparametrized_age_bounds. Note that the simulated reparametrized baseline ages are unconstrained and thus could, theoretically (but very unlikely), be out of these prescribed bounds.

Attributes:

name'simulation'

Algorithm’s name.

seedint

Used by numpy.random & torch.random for reproducibility.

algo_parametersdict

Contains the algorithm’s parameters.

bandwidth_methodfloat or str or callable, optional

Bandwidth argument used in scipy.stats.gaussian_kde in order to learn the patients’ distribution.

cofactorlist[str], optional (default = None)

The list of cofactors included used to select the wanted group of patients (ex - [‘genetic’]). All of them must correspond to an existing cofactor in the attribute Data of the input result of the run() method. TODO? should we allow to learn joint distribution of individual parameters and numeric/categorical cofactors (not fixed)?

cofactor_statelist[str], optional (default None)

The cofactors states used to select the wanted group of patients (ex - [‘APOE4’]). There is exactly one state per cofactor in cofactor (same order). It must correspond to an existing cofactor state in the attribute Data of the input result of the run() method. TODO? it could be replaced by methods to easily sub-select individual having certain cofactors PRIOR to running this algorithm + the functionality described just above (included varying cofactors as part of the distribution to estimate).

features_boundsbool or dict[str, (float, float)] (default False)

Specify if the scores of the generated subjects must be bounded. This parameter can express in two way:

bool : the bounds are the maximum and minimum scores observed in the baseline data (TODO: “baseline” instead?).

dict : the user has to set the min and max bounds for every features. For example: {'feature1': (score_min, score_max), 'feature2': (score_min, score_max), ...}

features_bounds_nb_subjects_factorfloat > 1 (default 10)

Only used if features_bounds is not False. The ratio of simulated subjects (> 1) so that there is at least number_of_subjects that comply to features bounds constraint.

mean_number_of_visitsint or float (default 6)

Average number of visits of the simulated patients. Examples - choose 5 => in average, a simulated patient will have 5 visits.

std_number_of_visitsint or float > 0, or None (default 3)

Standard deviation used into the generation of the number of visits per simulated patient. If <= 0 or None: number of visits will be deterministic

min_number_of_visits, max_number_of_visitsint (optional for max)

Minimum (resp. maximum) number of visits. Only used when std_number_of_visits > 0. min_number_of_visits should be >= 1 (default), max_number_of_visits can be None (no limit, default).

delay_btw_visits

Control by how many years consecutive visits of a patient are delayed. Multiple options are possible:

float > 0 : regular spacing between all visits
dictionary : {‘min’: float > 0, ‘mean’: float >= min, ‘std’: float > 0 [, ‘max’: float >= mean]} Specify a Gaussian random spacing (truncated between min, and max if given)
function : n (int >= 1) => 1D numpy.ndarray[float > 0] of length n giving delay between visits (e.g.: 3 => [0.5, 1.5, 1.])

noiseNone or str in {‘model’, ‘inherit_struct’} or DistributionFamily or dict or float or array-like[float]

Wanted noise-model for the generated observations:

Set noise to None will lead to patients follow the model exactly (no noise added).
Set to 'inherit_struct', the noise added will follow the model noise structure and for Gaussian noise it will be computed from reconstruction errors on data & individual parameters provided.
Set noise to 'model', the noise added will follow the model noise structure as well as its parameters.
Set to a valid input for noise_model_factory to get the corresponding noise-model, e.g. set to 'bernoulli', to simulate Bernoulli realizations.
Set a float will add for each feature’s scores a noise of standard deviation the given float (‘gaussian_scalar’ noise).
Set an array-like[float] (1D of length n_features) will add for the feature j a noise of standard deviation noise[j] (‘gaussian_diagonal’ noise).

<!> When you simulate data from an ordinal model, you HAVE to keep the default noise=’inherit_struct’ (default)

(or use ‘model’, which is the same in this case since there are no scaling parameter for ordinal noise)

number_of_subjectsint > 0

Number of subject to simulate.

reparametrized_age_boundstuple[float, float], optional (default None)

Define the minimum and maximum reparametrized ages of subjects included in the kernel estimation. See Notes section. Example: reparametrized_age_bounds = (65, 70)

sources_methodSourcesMethod or str

Source method.

prefixstr

Prefix appended to simulated patients’ identifiers

Methods

`load_parameters`(parameters)	Update the algorithm's parameters by the ones in the given dictionary.
`run`(model, *args[, return_loss])	Main method, run the algorithm.
`run_impl`(model, individual_parameters, data)	Run simulation - learn joined distribution of patients' individual parameters and return a results object containing the simulated individual parameters and the simulated scores.
`set_output_manager`(output_settings)	Set a `FitOutputManager` object for the run of the algorithm

deterministic: bool = False

family: str = 'simulate'

load_parameters(parameters: dict)

Update the algorithm’s parameters by the ones in the given dictionary. The keys in the io which does not belong to the algorithm’s parameters keys are ignored.

Parameters:

parametersdict: Contains the pairs (key, value) of the wanted parameters

Examples

>>> settings = leaspy.io.settings.algorithm_settings.AlgorithmSettings("mcmc_saem")
>>> my_algo = leaspy.algo.fit.tensor_mcmcsaem.TensorMCMCSAEM(settings)
>>> my_algo.algo_parameters
{'n_iter': 10000,
 'n_burn_in_iter': 9000,
 'eps': 0.001,
 'L': 10,
 'sampler_ind': 'Gibbs',
 'sampler_pop': 'Gibbs',
 'annealing': {'do_annealing': False,
  'initial_temperature': 10,
  'n_plateau': 10,
  'n_iter': 200}}
>>> parameters = {'n_iter': 5000, 'n_burn_in_iter': 4000}
>>> my_algo.load_parameters(parameters)
>>> my_algo.algo_parameters
{'n_iter': 5000,
 'n_burn_in_iter': 4000,
 'eps': 0.001,
 'L': 10,
 'sampler_ind': 'Gibbs',
 'sampler_pop': 'Gibbs',
 'annealing': {'do_annealing': False,
  'initial_temperature': 10,
  'n_plateau': 10,
  'n_iter': 200}}

name: str = 'simulation'

output_manager: FitOutputManager | None

run(model: AbstractModel, *args, return_loss: bool = False, **extra_kwargs) → Any

Main method, run the algorithm.

TODO fix proper abstract class method: input depends on algorithm… (esp. simulate != from others…)

Parameters:

modelAbstractModel: The used model.
datasetDataset: Contains all the subjects’ observations with corresponding timepoints, in torch format to speed up computations.
return_lossbool (default False), keyword only: Should the algorithm return main output and optional loss output as a 2-tuple?

Returns:

Depends on algorithm class: TODO change?

See also

AbstractFitAlgo
AbstractPersonalizeAlgo
SimulationAlgorithm

run_impl(model: AbstractModel, individual_parameters: IndividualParameters, data: Data) → Tuple[Result, torch.FloatTensor | None]

Run simulation - learn joined distribution of patients’ individual parameters and return a results object containing the simulated individual parameters and the simulated scores.

<!> The AbstractAlgo.run signature is not respected for simulation algorithm… TODO: respect it… at least use (model, dataset, individual_parameters) signature…

Parameters:

modelAbstractModel: Subclass object of AbstractModel. Model used to compute the population & individual parameters. It contains the population parameters.
individual_parametersIndividualParameters: Object containing the computed individual parameters.
dataData: The data object.

Returns:

Result: Contains the simulated individual parameters & individual scores.

Notes

In simulation_settings, one can specify in the parameters the cofactor & cofactor_state. By doing so, one can simulate based only on the subject for the given cofactor & cofactor’s state.

By default, all the subjects provided are used to estimate the joined distribution.

set_output_manager(output_settings: OutputsSettings) → None

Set a FitOutputManager object for the run of the algorithm

Parameters:

output_settingsOutputsSettings: Contains the logs settings for the computation run (console print periodicity, plot periodicity …)

Examples

>>> from leaspy import AlgorithmSettings
>>> from leaspy.io.settings.outputs_settings import OutputsSettings
>>> from leaspy.algo.fit.tensor_mcmcsaem import TensorMCMCSAEM
>>> algo_settings = AlgorithmSettings("mcmc_saem")
>>> my_algo = TensorMCMCSAEM(algo_settings)
>>> settings = {'path': 'brouillons',
                'console_print_periodicity': 50,
                'plot_periodicity': 100,
                'save_periodicity': 50
                }
>>> my_algo.set_output_manager(OutputsSettings(settings))

class SourcesMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Possible source methods.

full_kde : the sources are also learned with the gaussian kernel density estimation.
normal_sources : the sources are generated as multivariate normal distribution linked with the other individual parameters.

Methods

`capitalize`(/)	Return a capitalized version of the string.
`casefold`(/)	Return a version of the string suitable for caseless comparisons.
`center`(width[, fillchar])	Return a centered string of length width.
`count`(sub[, start[, end]])	Return the number of non-overlapping occurrences of substring sub in string S[start:end].
`encode`(/[, encoding, errors])	Encode the string using the codec registered for encoding.
`endswith`(suffix[, start[, end]])	Return True if S ends with the specified suffix, False otherwise.
`expandtabs`(/[, tabsize])	Return a copy where all tab characters are expanded using spaces.
`find`(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
`format`(args, *kwargs)	Return a formatted version of S, using substitutions from args and kwargs.
`format_map`(mapping)	Return a formatted version of S, using substitutions from mapping.
`index`(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
`isalnum`(/)	Return True if the string is an alpha-numeric string, False otherwise.
`isalpha`(/)	Return True if the string is an alphabetic string, False otherwise.
`isascii`(/)	Return True if all characters in the string are ASCII, False otherwise.
`isdecimal`(/)	Return True if the string is a decimal string, False otherwise.
`isdigit`(/)	Return True if the string is a digit string, False otherwise.
`isidentifier`(/)	Return True if the string is a valid Python identifier, False otherwise.
`islower`(/)	Return True if the string is a lowercase string, False otherwise.
`isnumeric`(/)	Return True if the string is a numeric string, False otherwise.
`isprintable`(/)	Return True if the string is printable, False otherwise.
`isspace`(/)	Return True if the string is a whitespace string, False otherwise.
`istitle`(/)	Return True if the string is a title-cased string, False otherwise.
`isupper`(/)	Return True if the string is an uppercase string, False otherwise.
`join`(iterable, /)	Concatenate any number of strings.
`ljust`(width[, fillchar])	Return a left-justified string of length width.
`lower`(/)	Return a copy of the string converted to lowercase.
`lstrip`([chars])	Return a copy of the string with leading whitespace removed.
`maketrans`(x[, y, z])	Return a translation table usable for str.translate().
`partition`(sep, /)	Partition the string into three parts using the given separator.
`removeprefix`(prefix, /)	Return a str with the given prefix string removed if present.
`removesuffix`(suffix, /)	Return a str with the given suffix string removed if present.
`replace`(old, new[, count])	Return a copy with all occurrences of substring old replaced by new.
`rfind`(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
`rindex`(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
`rjust`(width[, fillchar])	Return a right-justified string of length width.
`rpartition`(sep, /)	Partition the string into three parts using the given separator.
`rsplit`(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
`rstrip`([chars])	Return a copy of the string with trailing whitespace removed.
`split`(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
`splitlines`(/[, keepends])	Return a list of the lines in the string, breaking at line boundaries.
`startswith`(prefix[, start[, end]])	Return True if S starts with the specified prefix, False otherwise.
`strip`([chars])	Return a copy of the string with leading and trailing whitespace removed.
`swapcase`(/)	Convert uppercase characters to lowercase and lowercase characters to uppercase.
`title`(/)	Return a version of the string where each word is titlecased.
`translate`(table, /)	Replace each character in the string using the given translation table.
`upper`(/)	Return a copy of the string converted to uppercase.
`zfill`(width, /)	Pad a numeric string with zeros on the left, to fill a field of the given width.

KDE = 'full_kde'

NORMAL = 'normal_sources'

capitalize(/)

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold(/): Return a version of the string suitable for caseless comparisons.

center(width, fillchar=' ', /)

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count(sub[, start[, end]]) → int: Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

encode(/, encoding='utf-8', errors='strict')

Encode the string using the codec registered for encoding.

encoding: The encoding in which to encode the string.
errors: The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

endswith(suffix[, start[, end]]) → bool: Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

expandtabs(/, tabsize=8)

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find(sub[, start[, end]]) → int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

format(*args, **kwargs) → str: Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).

format_map(mapping) → str: Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).

index(sub[, start[, end]]) → int

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

isalnum(/)

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isalpha(/)

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isascii(/)

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

isdecimal(/)

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit(/)

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isidentifier(/)

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

islower(/)

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isnumeric(/)

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isprintable(/)

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

isspace(/)

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

istitle(/)

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isupper(/)

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

join(iterable, /)

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

ljust(width, fillchar=' ', /)

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower(/): Return a copy of the string converted to lowercase.

lstrip(chars=None, /)

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

static maketrans(x, y=<unrepresentable>, z=<unrepresentable>, /)

Return a translation table usable for str.translate().

If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.

partition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

removeprefix(prefix, /)

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

replace(old, new, count=-1, /)

Return a copy with all occurrences of substring old replaced by new.

count
Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

rfind(sub[, start[, end]]) → int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex(sub[, start[, end]]) → int

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rpartition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

rsplit(/, sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

rstrip(chars=None, /)

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

split(/, sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

splitlines(/, keepends=False)

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

startswith(prefix[, start[, end]]) → bool: Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

strip(chars=None, /)

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase(/): Convert uppercase characters to lowercase and lowercase characters to uppercase.

title(/)

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

translate(table, /)

Replace each character in the string using the given translation table.

table
Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper(/): Return a copy of the string converted to uppercase.

zfill(width, /)

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.