Benchmarking#
Benchmark#
- class calibrain.benchmark.Benchmark(solver, solver_param_grid, data_param_grid, ERP_config, source_simulator, leadfield_builder, sensor_simulator, uncertainty_estimator, metric_evaluator, random_state=42, logger=None)[source]#
Bases:
object- __init__(solver, solver_param_grid, data_param_grid, ERP_config, source_simulator, leadfield_builder, sensor_simulator, uncertainty_estimator, metric_evaluator, random_state=42, logger=None)[source]#
Initialize the Benchmark class.
Parameters:#
- solvercallable
The solver function (e.g., gamma_map, eloreta).
- solver_param_griddict
Grid of solver hyperparameters (e.g. noise_type, init_gamma).
- data_param_griddict
Grid of data generation hyperparameters.
- ERP_configdict
Configuration for ERP simulation.
- source_simulatorSourceSimulator
Instance of SourceSimulator for generating source data.
- leadfield_builderLeadfieldBuilder
Instance of LeadfieldBuilder for generating leadfields.
- sensor_simulatorSensorSimulator
Instance of SensorSimulator for generating data.
- uncertainty_estimatorUncertaintyEstimator
Instance of UncertaintyEstimator for uncertainty estimation.
- metric_evaluatorMetricEvaluator
Instance of MetricEvaluator for evaluating metrics.
- random_stateint, optional
Random seed for reproducibility.
- loggerlogging.Logger, optional
Logger instance for logging messages.
- create_experiment_directory(base_dir, params, desired_order)[source]#
Create a directory structure for the experiment, with subdirectories for each parameter in a specified order, followed by any remaining parameters.
Parameters: - base_dir (str): Base directory for the experiment. - params (dict): Dictionary of parameters. - desired_order (list): List of parameter keys in the desired order for the directory structure.
Returns: - experiment_dir (str): Path to the experiment directory.