.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_tutorials/07_data_generator.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_tutorials_07_data_generator.py: 07. Data Generation =================== This tutorial explains the high-level ``DataGenerator`` class. It shows how ``DataGenerator`` orchestrates benchmark runs across the upstream workflow: - source simulation; - leadfield retrieval; - sensor simulation; - source estimation; - run-wise experiment summaries returned as a ``DataFrame``. .. GENERATED FROM PYTHON SOURCE LINES 20-37 Scientific role of ``DataGenerator`` ------------------------------------ ``DataGenerator`` is the benchmark-orchestration layer behind CaliBrain's upstream workflow. Unlike the lower-level classes, it does not correspond to a single scientific transformation. Instead, it coordinates repeated runs over matched or varying conditions. Conceptually, its role is: 1. choose a grid of source, sensor, solver, and noise settings; 2. execute one run for each configuration; 3. return run-wise summaries that can later be compared, aggregated, and calibrated. This tutorial uses a minimal synthetic setup so the orchestration logic can be demonstrated directly. .. GENERATED FROM PYTHON SOURCE LINES 37-92 .. code-block:: Python from tempfile import TemporaryDirectory import matplotlib.pyplot as plt import numpy as np from mne.io.constants import FIFF from calibrain import DataGenerator, LeadfieldBuilder, SensorSimulator, SourceSimulator, gamma_map_sflex RANDOM_SEED = 83 tmpdir = TemporaryDirectory() FIG_DIR = tmpdir.name # Build a tiny leadfield fixture # ------------------------------ # # ``DataGenerator`` expects ``LeadfieldBuilder`` to provide leadfields. In this # runnable tutorial, we provide a deterministic fixed-orientation EEG leadfield # through the same high-level builder interface. # # Units: # # - source amplitudes are in ``nAm``; # - source coordinates are represented in ``m``; # - the synthetic EEG leadfield is interpreted as ``µV / nAm``; # - the resulting simulated EEG sensor signals are therefore in ``µV``. rng = np.random.default_rng(RANDOM_SEED) subject = "demo_subject" n_sensors = 16 n_sources = 32 src_coords = rng.normal(scale=0.04, size=(n_sources, 3)) leadfield = rng.normal(scale=0.03, size=(n_sensors, n_sources)) leadfield /= np.maximum( np.linalg.norm(leadfield, axis=0, keepdims=True), np.finfo(float).eps, ) leadfield *= 0.6 q_basis = np.zeros((n_sources, 3, 0), dtype=float) leadfield_dir = TemporaryDirectory() np.savez( f"{leadfield_dir.name}/{subject}_fixed_leadfield.npz", leadfield=leadfield, sensor_kind=FIFF.FIFFV_EEG_CH, sensor_units=FIFF.FIFF_UNIT_V, sensor_unitmult=FIFF.FIFF_UNITM_MU, coil_type=FIFF.FIFFV_COIL_EEG, src_coords=src_coords, Q_basis=q_basis, ) print("leadfield shape:", leadfield.shape) print("source coordinates shape:", src_coords.shape) .. rst-class:: sphx-glr-script-out .. code-block:: none leadfield shape: (16, 32) source coordinates shape: (32, 3) .. GENERATED FROM PYTHON SOURCE LINES 93-104 Define a small benchmark grid ----------------------------- ``DataGenerator`` is configured from three grids: - ``solver_param_grid`` for estimator hyperparameters; - ``data_param_grid`` for source/sensor-generation settings; - ``noise_param_grid`` for workflow noise handling. Here we keep them deliberately small. Two ``alpha_SNR`` values create two matched runs that differ only in signal-to-noise setting. .. GENERATED FROM PYTHON SOURCE LINES 104-150 .. code-block:: Python erp_config = { "tmin": -0.1, "tmax": 0.8, "stim_onset": 0.0, "sfreq": 100, "fmin": 2, "fmax": 8, "amplitude_distribution": { "median": 8.0, "sigma": 0.15, "clip": [2.0, 20.0], }, "random_erp_timing": False, "erp_min_length": 20, } source_simulator = SourceSimulator(ERP_config=erp_config) leadfield_builder = LeadfieldBuilder(leadfield_dir=leadfield_dir.name) sensor_simulator = SensorSimulator() generator = DataGenerator( solver=gamma_map_sflex, solver_param_grid={ "sigma": [0.01], "max_iter": [150], "tol": [1e-7], }, data_param_grid={ "subject": [subject], "nnz": [4], "orientation_type": ["fixed"], "alpha_SNR": [0.5, 0.8], "sensor_white_noise_std": [0.2], }, noise_param_grid={ "noise_type": ["oracle"], }, ERP_config=erp_config, source_simulator=source_simulator, leadfield_builder=leadfield_builder, sensor_simulator=sensor_simulator, save_posterior_stats=False, random_state=RANDOM_SEED, ) .. GENERATED FROM PYTHON SOURCE LINES 151-156 Run the benchmark orchestrator ------------------------------ ``DataGenerator.run`` returns a ``pandas.DataFrame`` with one row per run. Each row summarizes one experiment in the grid. .. GENERATED FROM PYTHON SOURCE LINES 156-166 .. code-block:: Python results = generator.run( nruns=1, fig_path=FIG_DIR, n_jobs=1, ) print("result columns:", list(results.columns)) print(results[["global_run_id", "solver", "noise_type", "alpha_SNR", "nnz"]]) .. rst-class:: sphx-glr-script-out .. code-block:: none 2026-07-08 12:14:22 - INFO - [run: 1/1 | config: 1/2 | total: 1/2] gamma_map_sflex | oracle | 4 NNZ | 0.5 SNR 2026-07-08 12:14:22 - INFO - [run: 1/1 | config: 2/2 | total: 2/2] gamma_map_sflex | oracle | 4 NNZ | 0.8 SNR result columns: ['run_id', 'global_run_id', 'seed', 'solver', 'noise_type', 'max_iter', 'sigma', 'tol', 'alpha_SNR', 'nnz', 'orientation_type', 'sensor_white_noise_std', 'subject', 'sensor_kind', 'coil_type', 'n_sources', 'n_times', 'gamma', 'noise_var', 'active_indices_size'] global_run_id solver noise_type alpha_SNR nnz 0 1 gamma_map_sflex oracle 0.5 4 1 2 gamma_map_sflex oracle 0.8 4 .. GENERATED FROM PYTHON SOURCE LINES 167-174 Visualize one representative generated dataset ---------------------------------------------- ``DataGenerator`` orchestrates the same source, leadfield, and sensor objects used elsewhere in the toolbox. To make that concrete, we visualize one representative fixed-orientation source realization and its corresponding noisy EEG sensor data under the same tutorial settings. .. GENERATED FROM PYTHON SOURCE LINES 174-215 .. code-block:: Python times = np.arange( erp_config["tmin"], erp_config["tmax"], 1.0 / erp_config["sfreq"], ) x_demo, active_demo = source_simulator.simulate( n_sources=n_sources, nnz=4, orientation_type="fixed", seed=RANDOM_SEED, ) y_clean_demo, y_noisy_demo, _, _ = sensor_simulator.simulate( x=x_demo, L=leadfield, alpha_SNR=0.5, sensor_white_noise_std=0.2, seed=RANDOM_SEED, ) fig, axes = plt.subplots(1, 2, figsize=(10.0, 3.6), sharex=True) for src_idx in active_demo: axes[0].plot(times, x_demo[src_idx], lw=1.8, label=f"source {src_idx}") axes[0].set( xlabel="Time (s)", ylabel="Source amplitude (nAm)", title="Representative generated source activity", ) axes[0].grid(True, linestyle="--", alpha=0.35) axes[0].legend(loc="upper right", fontsize=8, ncols=2) for sensor_idx in range(min(6, y_noisy_demo.shape[0])): axes[1].plot(times, y_noisy_demo[sensor_idx], lw=1.2) axes[1].set( xlabel="Time (s)", ylabel="Sensor amplitude (µV)", title="Representative generated noisy EEG data", ) axes[1].grid(True, linestyle="--", alpha=0.35) fig.tight_layout() .. image-sg:: /auto_tutorials/images/sphx_glr_07_data_generator_001.png :alt: Representative generated source activity, Representative generated noisy EEG data :srcset: /auto_tutorials/images/sphx_glr_07_data_generator_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 216-228 What this stage contributes scientifically ------------------------------------------ At this stage, the key output is not a final calibration result. It is a set of repeated, structured runs that can later be compared across conditions. In other words, ``DataGenerator`` prepares the experimental basis for later questions such as: - how does calibration change with ``alpha_SNR``? - how does it change with solver or noise mode? - how stable are posterior summaries across repeated runs? .. GENERATED FROM PYTHON SOURCE LINES 230-245 Summary ------- ``DataGenerator`` is the high-level orchestration class for the upstream benchmark stage. In this tutorial it: - retrieved a leadfield through the standard builder interface; - simulated source and sensor data; - ran ``gamma_map_sflex`` across a small condition grid; - returned one row per run for later comparison. The next workflow stages consume these repeated runs to build uncertainty summaries and calibration analyses. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.295 seconds) .. _sphx_glr_download_auto_tutorials_07_data_generator.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: 07_data_generator.ipynb <07_data_generator.ipynb>` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: 07_data_generator.py <07_data_generator.py>` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: 07_data_generator.zip <07_data_generator.zip>`