Cross-Validation API¶

cross_val_score is the high-level dimensionality-selection API. By default, it first calibrates a spectral cutoff and a sampling fraction from the similarity matrix, then evaluates the candidate ranks you provide with restricted entrywise cross-validation. The returned CVResult exposes the selected model rank as cv.model_rank, the fold-level validation scores as cv.fold_scores, and the per-rank averages as cv.rank_scores. Calibration itself is documented in Calibration.

Entries of a similarity matrix are not independent observations: once observed entries exceed the matrix-completion threshold, held-out entries are already determined by the training entries, which biases rank selection toward overly large ranks. cross_val_score therefore fixes a sparse observation pattern before cross-validation, keeping each fold's training set below that threshold. Folds are assigned at the level of unordered off-diagonal pairs, mirrored symmetrically, and treated as missing during fitting.

cross_val_score ¶

cross_val_score(
    similarity_matrix: ndarray,
    ranks: Sequence[int],
    sampling_fraction: float | None = None,
    n_folds: int = 5,
    n_repeats: int = 1,
    random_state: RandomStateLike = 0,
    n_jobs: int | None = -1,
    threads_per_worker: int | str | None = None,
    missing_values: float | None = np.nan,
    srf_kwargs: Mapping[str, object] | None = None,
) -> CVResult

Select the SRF model rank by restricted entrywise cross-validation.

Folds hide symmetric sets of observed off-diagonal entries, which are treated as missing while SRF fits the remaining entries and are then predicted from the fitted embedding. Validation error is the mean squared error on held-out entries, and the rank with the lowest mean error across folds and repeats is selected. Diagonal entries are never held out. When sampling_fraction is None, the hold-out protocol is first calibrated on the data with pysrf.coherence.calibrate_cross_validation.

Parameters:

Name	Type	Description	Default
`similarity_matrix`	`ndarray of shape (n_samples, n_samples)`	Symmetric similarity matrix. Missing values should be marked according to the `missing_values` parameter.	required
`ranks`	`sequence of int`	Candidate model ranks to evaluate. Duplicates are dropped; each rank must be between 1 and n_samples.	required
`sampling_fraction`	`float or None`	Fraction of observed off-diagonal entries each fold trains on, in (0, 1). None calibrates it from the data.	`None`
`n_folds`	`int`	Number of cross-validation folds. Must be at least 2.	`5`
`n_repeats`	`int`	Number of times the fold split is repeated with fresh randomness.	`1`
`random_state`	`(int, Generator, RandomState, SeedSequence or None)`	Controls fold splits and SRF initializations.	`0`
`n_jobs`	`int or None`	Number of parallel workers for fold fits. -1 uses all available cores, None runs a single worker.	`-1`
`threads_per_worker`	`(int, 'auto' or None)`	BLAS threads for each parallel worker. None uses one thread per worker when several run in parallel, 'auto' divides available cores across workers, and an integer is used as given.	`None`
`missing_values`	`float or None`	Sentinel value marking missing entries in the similarity matrix.	`np.nan`
`srf_kwargs`	`mapping or None`	Additional keyword arguments passed to every SRF fit, for example `{"max_outer": 200}`. Must not contain `rank`, `bounds`, `missing_values` or `random_state`, which are set per fit.	`None`

Returns:

Name	Type	Description
`result`	`CVResult`	Selected model rank, fold-level and per-rank validation scores, and calibration details.

Examples:

>>> import numpy as np
>>> from pysrf import SRF, cross_val_score
>>> rng = np.random.default_rng(0)
>>> s = rng.random((100, 100))
>>> s = (s + s.T) / 2
>>> cv = cross_val_score(s, range(1, 21), random_state=42)
>>> model = SRF(rank=cv.model_rank, random_state=42)
>>> w = model.fit_transform(s)

Source code in pysrf/cross_validation.py

def cross_val_score(
    similarity_matrix: np.ndarray,
    ranks: Sequence[int],
    sampling_fraction: float | None = None,
    n_folds: int = 5,
    n_repeats: int = 1,
    random_state: RandomStateLike = 0,
    n_jobs: int | None = -1,
    threads_per_worker: int | str | None = None,
    missing_values: float | None = np.nan,
    srf_kwargs: Mapping[str, object] | None = None,
) -> CVResult:
    """Select the SRF model rank by restricted entrywise cross-validation.

    Folds hide symmetric sets of observed off-diagonal entries, which are
    treated as missing while SRF fits the remaining entries and are then
    predicted from the fitted embedding. Validation error is the mean
    squared error on held-out entries, and the rank with the lowest mean
    error across folds and repeats is selected. Diagonal entries are never
    held out. When ``sampling_fraction`` is None, the hold-out protocol is
    first calibrated on the data with
    ``pysrf.coherence.calibrate_cross_validation``.

    Parameters
    ----------
    similarity_matrix : ndarray of shape (n_samples, n_samples)
        Symmetric similarity matrix. Missing values should be marked
        according to the ``missing_values`` parameter.
    ranks : sequence of int
        Candidate model ranks to evaluate. Duplicates are dropped; each
        rank must be between 1 and n_samples.
    sampling_fraction : float or None, default=None
        Fraction of observed off-diagonal entries each fold trains on,
        in (0, 1). None calibrates it from the data.
    n_folds : int, default=5
        Number of cross-validation folds. Must be at least 2.
    n_repeats : int, default=1
        Number of times the fold split is repeated with fresh randomness.
    random_state : int, Generator, RandomState, SeedSequence or None, default=0
        Controls fold splits and SRF initializations.
    n_jobs : int or None, default=-1
        Number of parallel workers for fold fits. -1 uses all available
        cores, None runs a single worker.
    threads_per_worker : int, 'auto' or None, default=None
        BLAS threads for each parallel worker. None uses one thread per
        worker when several run in parallel, 'auto' divides available
        cores across workers, and an integer is used as given.
    missing_values : float or None, default=np.nan
        Sentinel value marking missing entries in the similarity matrix.
    srf_kwargs : mapping or None, default=None
        Additional keyword arguments passed to every SRF fit, for example
        ``{"max_outer": 200}``. Must not contain ``rank``, ``bounds``,
        ``missing_values`` or ``random_state``, which are set per fit.

    Returns
    -------
    result : CVResult
        Selected model rank, fold-level and per-rank validation scores,
        and calibration details.

    Examples
    --------
    >>> import numpy as np
    >>> from pysrf import SRF, cross_val_score
    >>> rng = np.random.default_rng(0)
    >>> s = rng.random((100, 100))
    >>> s = (s + s.T) / 2
    >>> cv = cross_val_score(s, range(1, 21), random_state=42)
    >>> model = SRF(rank=cv.model_rank, random_state=42)
    >>> w = model.fit_transform(s)
    """
    _check_args(n_folds, n_repeats, n_jobs)
    fit_kwargs = _checked_srf_kwargs(srf_kwargs)

    s = _prepare_similarity(similarity_matrix, missing_values)
    n = s.shape[0]
    ranks = _validate_ranks(ranks, n)
    bounds = _observed_bounds(s)
    sampling_fraction, calibration = _cv_sampling_fraction(
        s, sampling_fraction, random_state, n_jobs
    )

    fold_fraction = get_fold_fraction(sampling_fraction, n_folds, n)
    fold_scores = _cross_validate_ranks(
        s,
        ranks,
        fold_fraction,
        n_folds,
        n_repeats,
        random_state,
        n_jobs,
        threads_per_worker,
        bounds,
        fit_kwargs,
    )
    rank_scores = _summarize(fold_scores)

    return CVResult(
        model_rank=_best_rank(rank_scores),
        fold_scores=fold_scores,
        rank_scores=rank_scores,
        spectral_cutoff=_spectral_cutoff(calibration),
        sampling_fraction=_training_fraction(fold_fraction, n_folds),
        candidate_ranks=ranks,
        calibration=calibration,
    )

CVResult `dataclass` ¶

CVResult(
    model_rank: int,
    fold_scores: DataFrame,
    rank_scores: DataFrame,
    spectral_cutoff: int | None,
    sampling_fraction: float,
    candidate_ranks: tuple[int, ...],
    calibration: CVCalibration | None = None,
)

Rank-selection result returned by cross_val_score.

Attributes:

Name	Type	Description
`model_rank`	`int`	Selected rank: the candidate with the lowest mean validation MSE.
`fold_scores`	`DataFrame`	One row per fit with columns `repeat`, `fold`, `candidate_rank` and `val_mse`.
`rank_scores`	`DataFrame`	Per-rank summary with columns `candidate_rank`, `val_mse_mean`, `val_mse_std`, `n_fold_scores` and `val_mse_sem`.
`spectral_cutoff`	`int or None`	Spectral cutoff estimated during calibration. None when an explicit `sampling_fraction` skipped calibration.
`sampling_fraction`	`float`	Realized fraction of observed off-diagonal entries each fold trained on.
`candidate_ranks`	`tuple of int`	Ranks that were evaluated.
`calibration`	`CVCalibration or None`	Full calibration result. None when an explicit `sampling_fraction` skipped calibration.

Cross-Validation API¶

cross_val_score ¶

CVResult dataclass ¶

CVResult `dataclass` ¶