Skip to content

Explainer

dicex.Dicex(model, task='regression', target_class=None, perturbation=None, alpha=0.1, scaler='auto', preset='mid', *, verbose='info', log_file=None, **vmf_vns_kwargs)

Directional counterfactual explainer for a fitted model.

Configure the explainer with the model, the uncertainty of the execution and the risk level, call fit on training data to learn the standardization of the features, and call explain (or explain_batch) to obtain the recommended direction of change for a record.

Initialize the high-level DiCEx explainer.

Parameters:

Name Type Description Default
model Any

Fitted model to explain, exposing predict (regression) or predict_proba (classification).

required
task TaskType

"regression" raises the prediction; "classification" raises the probability of target_class.

'regression'
target_class int | None

Index of the class whose probability is raised. Required for classification.

None
perturbation BasePerturbation | None

Law of the random step length T along the direction and of the additive noise. Defaults to GaussianPerturbation(mu=0.1, sigma=0.02). Its parameters are expressed in standardized units (standard deviations of each feature) when a scaler is used, and in the original feature units when scaler=None.

None
alpha float

Risk level of the lower-tail CVaR objective: 1.0 maximizes the expected gain, 0.1 the mean of the worst 10% of outcomes.

0.1
scaler DicexScaler | Literal['auto'] | None

"auto" learns a scaler from fit, or pass a fitted DicexScaler. Use None to search and perturb directly in the original feature units. In all cases, the returned direction is expressed in the original feature space.

'auto'
preset Literal['low', 'mid', 'high']

Computational budget of the optimizer, adapted to the number of features (see dicex.core.presets.PRESETS).

'mid'
verbose bool | str | int

"info" (default), "debug", "none", or a logging level.

'info'
log_file str | Path | None

If provided, also write the log to this file.

None
**vmf_vns_kwargs Any

Options of VmfVns that override the preset, e.g. seed for reproducible results, max_evals or timeout_seconds.

{}
Source code in src/dicex/explainer.py
def __init__(  # noqa: PLR0913
    self,
    model: Any,  # noqa: ANN401
    task: TaskType = "regression",
    target_class: int | None = None,
    perturbation: BasePerturbation | None = None,
    alpha: float = 0.1,
    scaler: DicexScaler | Literal["auto"] | None = "auto",
    preset: Literal["low", "mid", "high"] = "mid",
    *,
    verbose: bool | str | int = "info",
    log_file: str | Path | None = None,
    **vmf_vns_kwargs: Any,  # noqa: ANN401
) -> None:
    """Initialize the high-level DiCEx explainer.

    Args:
        model (Any): Fitted model to explain, exposing ``predict`` (regression) or
            ``predict_proba`` (classification).
        task (TaskType): ``"regression"`` raises the prediction; ``"classification"`` raises the
            probability of ``target_class``.
        target_class (int | None): Index of the class whose probability is raised. Required for
            classification.
        perturbation (BasePerturbation | None): Law of the random step length ``T`` along the
            direction and of the additive noise. Defaults to
            ``GaussianPerturbation(mu=0.1, sigma=0.02)``.
            Its parameters are expressed in standardized units (standard deviations of each
            feature) when a scaler is used, and in the original feature units when ``scaler=None``.
        alpha (float): Risk level of the lower-tail CVaR objective: ``1.0`` maximizes the
            expected gain, ``0.1`` the mean of the worst 10% of outcomes.
        scaler (DicexScaler | Literal["auto"] | None):
            ``"auto"`` learns a scaler from ``fit``, or pass a fitted ``DicexScaler``. Use ``None`` to
            search and perturb directly in the original feature units. In all cases, the returned
            direction is expressed in the original feature space.
        preset (Literal["low", "mid", "high"]): Computational budget of the optimizer, adapted
            to the number of features (see ``dicex.core.presets.PRESETS``).
        verbose (bool | str | int): ``"info"`` (default), ``"debug"``, ``"none"``, or a
            ``logging`` level.
        log_file (str | Path | None): If provided, also write the log to this file.
        **vmf_vns_kwargs (Any): Options of `VmfVns` that override the preset, e.g. ``seed``
            for reproducible results, ``max_evals`` or ``timeout_seconds``.
    """
    if task not in ("regression", "classification"):
        msg = f"Unsupported task '{task}'. Expected 'regression' or 'classification'."
        raise InvalidParameterError(msg)
    if task == "classification" and target_class is None:
        msg = "target_class must be provided for classification tasks."
        raise InvalidParameterError(msg)

    self.original_model = model
    self.task = task
    self.target_class = target_class
    self.alpha = alpha
    self.perturbation = perturbation or GaussianPerturbation(mu=0.1, sigma=0.02)
    self.preset = preset
    self._vmf_vns_kwargs = dict(vmf_vns_kwargs)
    self.verbose = verbose
    self.log_file = log_file
    self._d: int | None = None

    # Configure logging
    if verbose or log_file:
        configure_verbosity(verbose=verbose, log_file=log_file)

    self.model = self._adapt_model(model)

    if scaler == "auto":
        self.scaler: DicexScaler | None = DicexScaler()
        self._is_fitted = False
    elif scaler is None:
        self.scaler = None
        self._is_fitted = True
    elif isinstance(scaler, DicexScaler):  # pyright: ignore[reportUnnecessaryIsInstance]
        self.scaler = scaler
        self._is_fitted = scaler.mean_ is not None and scaler.std_ is not None
    else:
        msg = (
            f"Unsupported scaler of type {type(scaler).__name__!r}. Use scaler='auto' (a DicexScaler is "
            "fitted in `fit`), scaler=None (no scaling), or a dicex.preprocessing.scaling.DicexScaler instance. "
            "To reuse a fitted sklearn StandardScaler `s`, create `d = DicexScaler()` and set "
            "`d.mean_ = s.mean_` and `d.std_ = s.scale_`."
        )
        raise InvalidParameterError(msg)

explain(x0)

Generate a directional counterfactual explanation for a single point.

Parameters:

Name Type Description Default
x0 ndarray

The baseline point to explain, as a 1D array.

required

Returns:

Name Type Description
ExplanationResult ExplanationResult

The optimal direction and metrics.

Raises:

Type Description
NotFittedError

If the explainer requires fitting but fit has not been called.

InvalidParameterError

If x0 is not a 1D array.

Source code in src/dicex/explainer.py
def explain(self, x0: np.ndarray) -> ExplanationResult:  # noqa: PLR0915
    """Generate a directional counterfactual explanation for a single point.

    Args:
        x0 (np.ndarray): The baseline point to explain, as a 1D array.

    Returns:
        ExplanationResult: The optimal direction and metrics.

    Raises:
        NotFittedError: If the explainer requires fitting but `fit` has not been called.
        InvalidParameterError: If `x0` is not a 1D array.
    """
    if not self._is_fitted:
        msg = "Dicex.fit must be called before explain when scaler='auto'."
        raise NotFittedError(msg)

    if x0.ndim != VECTOR_NDIM:
        msg = f"x0 must be a 1D array. Got shape {x0.shape}"
        raise InvalidParameterError(msg)

    if self._d is None:
        self._d = len(x0)

    _logger.info(
        "Explaining point: d=%d, alpha=%.3f, task=%s",
        len(x0),
        self.alpha,
        self.task,
    )

    model_for_optimizer = self.model
    x0_optimizer = x0.astype(float, copy=False)
    metadata: dict[str, Any] = {}
    perturbation_for_optimizer = self.perturbation

    if self.scaler is not None:
        x0_optimizer = self.scaler.transform(x0.reshape(1, -1)).reshape(-1)
        model_for_optimizer = self.scaler.wrap_model(self.model)

    optimizer_params = self._optimizer_params()

    optimizer = VmfVns(
        model=cast("Any", model_for_optimizer),
        x0=x0_optimizer,
        perturbation=cast("BasePerturbation", perturbation_for_optimizer),
        alpha=self.alpha,
        target_class=self.target_class,
        **cast("Any", optimizer_params),
    )
    result_scaled = optimizer.run()

    direction = result_scaled.direction
    if self.scaler is not None:
        direction = self.scaler.transform_direction(result_scaled.direction)
        metadata["direction_scaled"] = result_scaled.direction

    metadata.update(result_scaled.metadata)

    _logger.info(
        "Explanation complete: CVaR=%.6f, %d model evals",
        result_scaled.robust_value,
        result_scaled.metadata.get("n_model_evals", 0),
    )

    # Compute baseline CVaR (c=0) with the same reevaluation budget.
    n_baseline = int(optimizer_params["n_evaluation"])
    baseline_t0 = time.perf_counter()
    env_samples = cast("BasePerturbation", perturbation_for_optimizer).sample_env(n_baseline, len(x0_optimizer))
    pts_baseline = x0_optimizer + env_samples

    if self.target_class is not None:
        cls_model = cast("Any", model_for_optimizer)
        y_baseline = cls_model.predict_proba(pts_baseline)[:, self.target_class]
        y0 = cls_model.predict_proba(x0_optimizer.reshape(1, -1))[0, self.target_class]
    else:
        reg_model = cast("Any", model_for_optimizer)
        y_baseline = reg_model.predict(pts_baseline).flatten()
        y0 = reg_model.predict(x0_optimizer.reshape(1, -1)).flatten()[0]

    z_baseline = y_baseline - float(y0)
    baseline_cvar = float(lower_tail_cvar_empirical(z_baseline, self.alpha))
    metadata["baseline_cvar"] = baseline_cvar
    phase_eval_counts = dict(metadata.get("phase_eval_counts", {}))
    phase_wall_times = dict(metadata.get("phase_wall_times", {}))
    phase_eval_counts["baseline_evaluation"] = phase_eval_counts.get("baseline_evaluation", 0) + n_baseline + 1
    phase_wall_times["baseline_evaluation"] = round(
        phase_wall_times.get("baseline_evaluation", 0.0) + (time.perf_counter() - baseline_t0),
        6,
    )
    metadata["phase_eval_counts"] = phase_eval_counts
    metadata["phase_wall_times"] = phase_wall_times
    metadata.setdefault("n_test", optimizer_params["n_test"])
    metadata.setdefault("n_evaluation", n_baseline)
    metadata.setdefault("n_bootstrap", optimizer_params.get("n_bootstrap", 1000))
    metadata.setdefault("trace_level", optimizer_params.get("trace_level", "none"))

    if result_scaled.robust_value > baseline_cvar:
        _logger.info(
            "Action recommended: Found direction strictly better than inaction (%.4f > %.4f)",
            result_scaled.robust_value,
            baseline_cvar,
        )
    else:
        _logger.info(
            "Inaction recommended: Best direction is not better than absorbing ambient noise (%.4f <= %.4f)",
            result_scaled.robust_value,
            baseline_cvar,
        )

    return ExplanationResult(
        direction=direction,
        robust_value=result_scaled.robust_value,
        alpha=self.alpha,
        metadata=metadata,
    )

explain_batch(x)

Explain each row of a 2D input array independently.

Parameters:

Name Type Description Default
x ndarray

Array of points to explain, of shape (n_samples, n_features).

required

Returns:

Type Description
list[ExplanationResult]

list[ExplanationResult]: A list of explanation results.

Raises:

Type Description
InvalidParameterError

If x is not a 2D array.

Source code in src/dicex/explainer.py
def explain_batch(self, x: np.ndarray) -> list[ExplanationResult]:
    """Explain each row of a 2D input array independently.

    Args:
        x (np.ndarray): Array of points to explain, of shape (n_samples, n_features).

    Returns:
        list[ExplanationResult]: A list of explanation results.

    Raises:
        InvalidParameterError: If `x` is not a 2D array.
    """
    if x.ndim != MATRIX_NDIM:
        msg = f"x must be a 2D array. Got shape {x.shape}"
        raise InvalidParameterError(msg)

    _logger.info("Batch explanation: %d points", len(x))
    return [self.explain(row) for row in x]

fit(x_train)

Fit the internal scaler when scaling is enabled.

Parameters:

Name Type Description Default
x_train ndarray

Training data array of shape (n_samples, n_features).

required

Returns:

Name Type Description
Self Self

The fitted explainer instance.

Raises:

Type Description
InvalidParameterError

If x_train is not a 2D array.

Source code in src/dicex/explainer.py
def fit(self, x_train: np.ndarray) -> Self:
    """Fit the internal scaler when scaling is enabled.

    Args:
        x_train (np.ndarray): Training data array of shape (n_samples, n_features).

    Returns:
        Self: The fitted explainer instance.

    Raises:
        InvalidParameterError: If `x_train` is not a 2D array.
    """
    if x_train.ndim != MATRIX_NDIM:
        msg = f"x_train must be a 2D array. Got shape {x_train.shape}"
        raise InvalidParameterError(msg)

    if self.scaler is not None:
        self.scaler.fit(x_train)
        _logger.info("Scaler fitted on %d samples, d=%d", len(x_train), x_train.shape[1])

    self._d = x_train.shape[1]
    self._is_fitted = True
    return self

dicex.ExplanationResult(direction, robust_value, alpha, metadata=_default_metadata()) dataclass

Result of a DiCEx explanation.

Attributes:

Name Type Description
direction ndarray

Recommended direction of change, a unit vector in the original feature space. The zero vector means that no direction reliably beats not acting.

robust_value float

Lower-tail CVaR, at level alpha, of the gain in the prediction when moving along the direction with a random step.

alpha float

Risk level used.

metadata dict[str, Any]

Details of the search, among them beats_baseline (whether acting beats not acting), baseline_cvar (the value of not acting), direction_scaled (the direction in standardized units, when a scaler is used), n_model_evals, stop_reason and wall_time_seconds.