leaspy.io.outputs

Submodules

Classes

`IndividualParameters`	Data container for individual parameters, contains IDs, timepoints and observations values.
`Result`	Result object class.

Package Contents

class IndividualParameters

Data container for individual parameters, contains IDs, timepoints and observations values. Output of the Leaspy.personalize() method, contains the random effects.

There are used as output of the personalization algorithms and as input/output of the simulation algorithm, to provide an initial distribution of individual parameters.

Attributes:

_indiceslist: List of the patient indices
_individual_parametersdict: Individual indices (key) with their corresponding individual parameters {parameter name: parameter value}
_parameters_shapedict: Shape of each individual parameter
_default_saving_typestr: Default extension for saving when none is provided

VALID_IO_EXTENSIONS = ['csv', 'json']

add_individual_parameters(index, individual_parameters)

Add the individual parameter of an individual to the IndividualParameters object

Parameters:

indexstr: Index of the individual
individual_parametersdict: Individual parameters of the individual {name: value:}

Raises:

LeaspyIndividualParamsInputError

If the index is not a string or has already been added
Or if the individual parameters is not a dict.
Or if individual parameters are not self-consistent.

Parameters:

index (leaspy.utils.typing.IDType)
individual_parameters (leaspy.utils.typing.DictParams)

Examples

Add two individual with tau, xi and sources parameters

>>> ip = IndividualParameters()
>>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]})
>>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]})

items(): Get items of dict _individual_parameters.

subset(indices, *, copy=True)

Returns IndividualParameters object with a subset of the initial individuals

Parameters:

indiceslist[ID]: List of strings that corresponds to the indices of the individuals to return
copybool, optional (default True): Should we copy underlying parameters or not?

Returns:

IndividualParameters: An instance of the IndividualParameters object with the selected list of individuals

Raises:

LeaspyIndividualParamsInputError: Raise an error if one of the index is not in the IndividualParameters

Parameters:

indices (Iterable[leaspy.utils.typing.IDType])
copy (bool)

Examples

>>> ip = IndividualParameters()
>>> ip.add_individual_parameters('index-1', {"xi": 0.1, "tau": 70, "sources": [0.1, -0.3]})
>>> ip.add_individual_parameters('index-2', {"xi": 0.2, "tau": 73, "sources": [-0.4, -0.1]})
>>> ip.add_individual_parameters('index-3', {"xi": 0.3, "tau": 58, "sources": [-0.6, 0.2]})
>>> ip_sub = ip.subset(['index-1', 'index-3'])

get_aggregate(parameter, function)

Returns the result of aggregation by function of parameter values across all patients

Parameters:

parameterstr: Name of the parameter
functioncallable: A function operating on iterables and supporting axis keyword, and outputing an iterable supporting the tolist method.

Returns:

list or float (depending on parameter shape): Resulting value of the parameter

Raises:

LeaspyIndividualParamsInputError

If individual parameters are empty,
or if the parameter is not in the IndividualParameters.

Parameters:

parameter (leaspy.utils.typing.ParamType)
function (Callable)

Return type:

list

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_median = ip.get_aggregate("tau", np.median)

get_mean(parameter)

Returns the mean value of a parameter across all patients

Parameters:

parameterstr: Name of the parameter

Returns:

list or float (depending on parameter shape): Mean value of the parameter

Raises:

LeaspyIndividualParamsInputError

If individual parameters are empty,
or if the parameter is not in the IndividualParameters.

Parameters:

parameter (leaspy.utils.typing.ParamType)

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_mean = ip.get_mean("tau")

get_std(parameter)

Returns the standard deviation of a parameter across all patients

Parameters:

parameterstr: Name of the parameter

Returns:

list or float (depending on parameter shape): Standard-deviation value of the parameter

Raises:

LeaspyIndividualParamsInputError

If individual parameters are empty,
or if the parameter is not in the IndividualParameters.

Parameters:

parameter (leaspy.utils.typing.ParamType)

Examples

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> tau_std = ip.get_std("tau")

to_dataframe()

Returns the dataframe of individual parameters

Returns:

pandas.DataFrame: Each row corresponds to one individual. The index corresponds to the individual index (‘ID’). The columns are the names of the parameters.

Return type:

DataFrame

Examples

Convert the individual parameters object into a dataframe

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> ip_df = ip.to_dataframe()

static from_dataframe(df)

Static method that returns an IndividualParameters object from the dataframe

Parameters:

dfpandas.DataFrame: Dataframe of the individual parameters. Each row must correspond to one individual. The index corresponds to the individual index. The columns are the names of the parameters.

Returns:

IndividualParameters

Parameters:

df (DataFrame)

static from_pytorch(indices, dict_pytorch)

Static method that returns an IndividualParameters object from the indices and pytorch dictionary

Parameters:

indiceslist[ID]: List of the patients indices
dict_pytorchdict[parameter:str, torch.Tensor]: Dictionary of the individual parameters

Returns:

IndividualParameters

Raises:

LeaspyIndividualParamsInputError

Parameters:

indices (list[leaspy.utils.typing.IDType])
dict_pytorch (leaspy.utils.typing.DictParamsTorch)

Examples

>>> indices = ['index-1', 'index-2', 'index-3']
>>> ip_pytorch = {
>>>    "xi": torch.tensor([[0.1], [0.2], [0.3]], dtype=torch.float32),
>>>    "tau": torch.tensor([[70], [73], [58.]], dtype=torch.float32),
>>>    "sources": torch.tensor([[0.1, -0.3], [-0.4, 0.1], [-0.6, 0.2]], dtype=torch.float32)
>>> }
>>> ip_pytorch = IndividualParameters.from_pytorch(indices, ip_pytorch)

to_pytorch()

Returns the indices and pytorch dictionary of individual parameters

Returns:

indices: list[ID]: List of patient indices
pytorch_dict: dict[parameter:str, torch.Tensor]: Dictionary of the individual parameters {parameter name: pytorch tensor of values across individuals}

Return type:

tuple[list[leaspy.utils.typing.IDType], leaspy.utils.typing.DictParamsTorch]

Examples

Convert the individual parameters object into a dataframe

>>> ip = IndividualParameters.load("path/to/individual_parameters")
>>> indices, ip_pytorch = ip.to_pytorch()

save(path, **kwargs)

Saves the individual parameters (json or csv) at the path location

TODO? save leaspy version as well for retro/future-compatibility issues?

Parameters:

pathstr: Path and file name of the individual parameters. The extension can be json or csv. If no extension, default extension (csv) is used
**kwargs: Additional keyword arguments to pass to either: * pandas.DataFrame.to_csv() * json.dump() depending on saving format requested

Raises:

LeaspyIndividualParamsInputError

If extension not supported for saving
If individual parameters are empty

Parameters:

path (str)

classmethod load(path)

Static method that loads the individual parameters (json or csv) existing at the path location

Parameters:

pathstr: Path and file name of the individual parameters.

Returns:

IndividualParameters: Individual parameters object load from the file

Raises:

LeaspyIndividualParamsInputError: If the provided extension is not csv or not json.

Parameters:

path (str)

Examples

>>> ip = IndividualParameters.load('/path/to/individual_parameters_1.json')
>>> ip2 = IndividualParameters.load('/path/to/individual_parameters_2.csv')

class Result(data, individual_parameters, noise_std=None)

Result object class. Used as logs by personalize algorithms & simulation algorithm.

Parameters:

dataData: Object containing the idx, time-points and observations of the patients
individual_parametersdict [str, torch.Tensor]: Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’
noise_stdfloat or torch.FloatTensor, optional (default None): Desired noise standard deviation level

Attributes:

dataData: Object containing the idx, time-points and observations of the patients.
individual_parametersdict [str, torch.Tensor]: Contains log-acceleration ‘xi’, time-shifts ‘tau’ & ‘sources’ (dictionary of torch.Tensor).
ID_to_idxdict: The keys are the individual ID & the items are their respective ordered position in the data file given by the user. This order remains the same during the computation. Example - in Result.individual_parameters[‘xi’], the first element corresponds to the first patient in ID_to_idx.
noise_stdfloat or torch.FloatTensor: Desired noise standard deviation level.

Parameters:

data (Data)
individual_parameters (leaspy.utils.typing.DictParamsTorch)

data

individual_parameters

ID_to_idx: dict[leaspy.utils.typing.IDType, int]

noise_std = None

get_torch_individual_parameters(ID=None)

Getter function for the individual parameters.

Parameters:

IDstr or list[str], optional (default None): Contains the identifiers of the wanted subject.

Returns:

dict [str, torch.Tensor]: Contains the individual parameters.

Parameters:

ID (Union[leaspy.utils.typing.IDType, list[leaspy.utils.typing.IDType]])

Return type:

leaspy.utils.typing.DictParamsTorch

get_dataframe_individual_parameters(cofactors=None)

Return the dataframe of the individual parameters.

Each row corresponds to a subject. The columns correspond (in this order) to the subjects’ ID, the individual parameters (one column per individual parameter) & the cofactors (one column per cofactor).

Parameters:

cofactorsstr or list[str], optional (default None): Contains the cofactor(s) to join to the logs dataframe.

Returns:

pandas.DataFrame: Contains for each patient his ID & his individual parameters (optional and his cofactors states)

Parameters:

cofactors (Union[str, list[str]])

Return type:

DataFrame

Notes

The cofactors must be present in the leaspy data object stored into the .data attribute of the result instance. See the example.

Examples

Load a longitudinal multivariate dataset & the subjects’ cofactors. Compute the individual parameters for this dataset & get the corresponding dataframe with the genetic APOE cofactor

>>> import pandas as pd
>>> from leaspy.api import Leaspy
>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.io.data import Data
>>> from leaspy.io.logs.visualization import Plotter
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')  # replace with your own path!
>>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv')  # replace with your own path!
>>> print(genes_cofactors.head())
           ID      APOE4
0  sub-HS0102          1
1  sub-HS0112          0
2  sub-HS0113          0
3  sub-HS0114          1
4  sub-HS0115          0

>>> data.load_cofactors(genes_cofactors, ['GENES'])
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> individual_results_df = individual_results.get_dataframe_individual_parameters('GENES')
>>> print(individual_results_df.head())
                   tau        xi  sources_0  sources_1  APOE4
ID
sub-HS0102   70.329201  0.120465   5.969921  -0.245034      1
sub-HS0112   95.156624 -0.692099   1.520273   3.477707      0
sub-HS0113   74.900673 -1.769864  -1.222979   1.665889      0
sub-HS0114   81.792763 -1.003620   1.021321   2.371716      1
sub-HS0115   89.724648 -0.820971  -0.480975   0.741601      0

save_individual_parameters_csv(path, idx=None, cofactors=None, **args)

Save the individual parameters in a csv format.

Parameters:

pathstr: The logs’ path.
idxlist [str], optional (default None): Contain the IDs of the selected subjects. If None, all the subjects are selected.
cofactorsstr or list [str], optional (default None): Contains the cofactor(s) to join to the logs dataframe.
**args: Parameters to pass to pandas.DataFrame.to_csv().

Parameters:

path (str)
idx (list[leaspy.utils.typing.IDType])

Notes

The cofactors must be present in the leaspy data object stored into the data attribute of the result instance. See the example.

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv') # replace with your own path!
>>> genes_cofactors = pd.read_csv('data/genes_cofactors.csv')  # replace with your own path!
>>> data.load_cofactors(genes_cofactors, ['GENES'])
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_csv(output_path, idx, cofactors='GENES')

save_individual_parameters_json(path, idx=None, human_readable=None, **args)

Save the individual parameters in a json format.

Parameters:

pathstr

The logs’ path.

idxlist [str], optional (default None)

Contain the IDs of the selected subjects. If None, all the subjects are selected.

human_readableAny, optional (default None) –> TODO change to bool

Deprecated since version 1.0:

If None (default): save as json file
If not None: call save_individual_parameters_torch().

**args

Arguments to pass to json.dump. Default to: dict(indent=2)

Raises:

NotADirectoryError: if parent directory of path does not exist.

Parameters:

path (str)
idx (list[leaspy.utils.typing.IDType])

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_json(output_path, idx)

save_individual_parameters_torch(path, idx=None, **args)

Save the individual parameters in a torch format.

Parameters:

pathstr: The logs’ path.
idxlist [str], optional (default None): Contain the IDs of the selected subjects. If None, all the subjects are selected.
**args: Arguments to pass to torch.save.

Raises:

NotADirectoryError: if parent directory of path does not exist.

Parameters:

path (str)
idx (list[leaspy.utils.typing.IDType])

Examples

Save the individual parameters of the twenty first subjects.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> output_path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt'
>>> idx = list(individual_results.individual_parameters.keys())[:20]
>>> individual_results.save_individual_parameters_torch(output_path, idx)

classmethod load_individual_parameters_from_csv(path, *, verbose=True, **kwargs)

Load individual parameters from a csv.

Parameters:

pathstr: The file’s path. The csv file musts contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].
verbosebool (default True): Whether to have verbose output or not
**kwargs: Parameters to pass to pandas.read_csv().

Returns:

dict [str, torch.Tensor]: A dictionary of torch.tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.csv'
>>> individual_parameters = Result.load_individual_parameters_from_csv(path)

static load_individual_parameters_from_dataframe(df)

Load individual parameters from a pandas.DataFrame.

Parameters:

dfpandas.DataFrame: Must contain two columns named ‘tau’ and ‘xi’. If the individual parameters come from a multivariate model, it must also contain the columns ‘sources_i’ for i in [0, …, n_sources].

Returns:

dict [str, torch.Tensor]: A dictionary of torch.tensor which contains the individual parameters.

Parameters:

df (DataFrame)

static load_individual_parameters_from_json(path, *, verbose=True, **kwargs)

Load individual parameters from a json file.

Deprecated : also load torch files.

Parameters:

pathstr: The file’s path.
verbosebool (default True): Whether to have verbose output or not
**kwargs: Parameters to pass to json.load().

Returns:

dict [str, torch.Tensor]: A dictionary of torch.Tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> individual_parameters = Result.load_individual_parameters_from_json(path)

static load_individual_parameters_from_torch(path, *, verbose=True, **kwargs)

Load individual parameters from a torch file.

Parameters:

pathstr: The file’s path.
verbosebool (default True): Whether to have verbose output or not
**kwargs: Parameters to pass to torch.load().

Returns:

dict [str, torch.Tensor]: A dictionary of torch.Tensor which contains the individual parameters.

Parameters:

path (str)

Examples

Load an individual parameters dictionary from a saved file.

>>> from leaspy.io.outputs import Result
>>> path = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.pt'
>>> individual_parameters = Result.load_individual_parameters_from_torch(path)

classmethod load_individual_parameters(path_or_df, **kwargs)

Load individual parameters from a pandas.DataFrame, a csv, a json file or a torch file.

Parameters:

path_or_dfstr or pandas.DataFrame: The file’s path or a DataFrame containing the individual parameters.
**kwargs: Keyword-arguments to be passed to the corresponding load function.

Returns:

dict [str, torch.Tensor]: A dictionary of torch.tensor which contains the individual parameters.

Raises:

FileNotFoundError: if path is invalid

classmethod load_result(data, individual_parameters, *, cofactors=None, **kwargs)

Load a Result class object from two file - one for the individual data & one for the individual parameters.

Parameters:

datastr or pandas.DataFrame or Data: The file’s path or a DataFrame containing the features’ scores.
individual_parametersstr or pandas.DataFrame: The file’s path or a DataFrame containing the individual parameters.
cofactorsstr or pandas.DataFrame, optional (default None): The file’s path or a DataFrame containing the individual cofactors. The ID must be in index! Thus, the shape is (n_subjects, n_cofactors).
**kwargs: Parameters to pass to Result.load_individual_parameters method.

Returns:

Result: A Result class object which contains the individual parameters and the individual data.

Examples

Launch an individual parameters estimation, save it and reload it.

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.io.outputs import Result
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> leaspy_logistic = Leaspy('logistic')
>>> data = Data.from_csv_file('data/my_leaspy_data.csv')
>>> model_settings = AlgorithmSettings('mcmc_saem', seed=0)
>>> personalize_settings = AlgorithmSettings('mode_real', seed=0)
>>> leaspy_logistic.fit(data, model_settings)
>>> individual_results = leaspy_logistic.personalize(data, model_settings)
>>> path_data = 'data/my_leaspy_data.csv'
>>> path_individual_parameters = 'outputs/logistic_seed0-mode_real_seed0-individual_parameter.json'
>>> individual_results.data.to_dataframe().to_csv(path_data)
>>> individual_results.save_individual_parameters_json(path_individual_parameters)
>>> individual_parameters = Result.load_result(path_data, path_individual_parameters)

get_error_distribution_dataframe(model, cofactors=None)

Get signed residual distribution per patient, per sub-score & per visit. Each residual is equal to the modeled data minus the observed data.

Parameters:

modelAbstractModel
cofactorsstr, list [str], optional (default None): Contains the cofactors’ names to be included in the DataFrame. By default, no cofactors are returned. If cofactors == “all”, all the available cofactors are returned.

Returns:

residuals_dataframepandas.DataFrame with index [‘ID’, ‘TIME’]

Examples

Get mean absolute error per feature:

>>> from leaspy.algo import AlgorithmSettings
>>> from leaspy.api import Leaspy
>>> from leaspy.io.data import Data
>>> data = Data.from_csv_file("/my/data/path")
>>> leaspy_logistic = Leaspy('logistic')
>>> settings = AlgorithmSettings("mcmc_saem", seed=0)
>>> leaspy_logistic.calibrate(data, settings)
>>> settings = AlgorithmSettings("mode_real", seed=0)
>>> results = leaspy_logistic.personalize(data, settings)
>>> residuals_dataframe = results.get_error_distribution_dataframe(model)
>>> residuals_dataframe.abs().mean()

static get_cofactor_states(cofactors)

Deprecated since version 1.0.

Given a list of string return the list of unique elements.

Parameters:

cofactorslist[str]: Distribution list of the cofactors.

Returns:

list: Unique occurrences of the input vector.

Parameters:

cofactors (list)

Return type:

list

get_parameter_distribution(parameter, cofactor=None)

Deprecated since version 1.0.

Return the wanted parameter distribution (one distribution per covariate state).

Parameters:

parameterstr: The wanted parameter’s name (ex: ‘xi’, ‘tau’, …). It can also be sources_i to only get the i-th dimension of multivariate sources parameter.
cofactorstr, optional (default None): The wanted cofactor’s name.

Returns:

list[float] or dict[str, Any]

Raises:

LeaspyIndividualParamsInputError: if unsupported individual parameters
LeaspyInputError: if unknown cofactor

Parameters:

parameter (leaspy.utils.typing.ParamType)

Notes

If cofactor is None:

If the parameter is univariate => return a list the parameter’s distribution:
list[float]
If the parameter is multivariate => return a dictionary:
{‘parameter1’: distribution of parameter variable 1, ‘parameter2’: …}

If cofactor is not None:

If the parameter is univariate => return a dictionary:
{‘cofactor1’: parameter distribution such that patient.covariate = covariate1, ‘cofactor2’: …}
If the parameter is multivariate => return a dictionary:
{‘cofactor1’: {‘parameter1’: …, ‘parameter2’: …}, ‘cofactor2’: {…}, …}

get_cofactor_distribution(cofactor)

Deprecated since version 1.0.

Get the list of the cofactor’s distribution.

Parameters:

cofactorstr: Cofactor’s name

Returns:

list: Cofactor’s distribution.

Parameters:

cofactor (str)

get_patient_individual_parameters(idx)

Deprecated since version 1.0.

Get the dictionary of the wanted patient’s individual parameters

Parameters:

idxstr: ID of the wanted patient

Returns:

dict[param_name:str, torch.Tensor]: Patient’s individual parameters

Parameters:

idx (leaspy.utils.typing.IDType)