Random forest

Task	Classification
Method key (analysis_type)	random_forest
Prediction on new data	Yes — see Prediction
Library	sklearn.ensemble.RandomForestClassifier

When to use

• The relation between colour and class is non-linear or involves interactions between channels.
• You want to know which features matter: the feature importance plot is available only for this method.
• Features have different ranges and you do not want to scale them: trees do not need scaling.

If the classes are strongly imbalanced, see also Balanced Random Forest.

How it works

A random forest is a set of decision trees. Each tree is grown on a bootstrap sample of the training rows (rows drawn with replacement), and at every split it considers only a random subset of $\sqrt{p}$ of the $p$ features. The trees are grown until the leaves are pure. The forest predicts the class with the highest average probability over all trees:

$$\hat{P}(k \mid x) = \frac{1}{T} \sum_{t=1}^{T} \hat{P}_t(k \mid x) .$$

The two sources of randomness make the trees different from each other. Averaging many different, overfitted trees gives a model that generalises much better than a single tree.

Feature importance is the mean decrease in impurity (Gini): for every feature, the total decrease of the Gini impurity over all splits that use this feature, averaged over the trees and normalised to sum to 1.

Parameters

Defaults as set by the analysis dialog. The model runs with random_state=42 and the scikit-learn defaults for everything else: criterion='gini', max_features='sqrt', bootstrap=True. For more settings (criterion, max features, leaf sizes), use Balanced Random Forest.

Parameter	UI label	Default	Notes
n_estimators	Number of Trees	100	More trees give a more stable result and take longer; accuracy rarely improves beyond a few hundred.
max_depth	Max Depth	empty (unlimited)	Limit the depth to make the trees simpler and less overfitted.
feature_selection	Enable Feature Selection	off	See Feature selection.
average_replicates	Average technical replicates	off	See Averaging replicates.

Preprocessing

No scaling: decision trees split on thresholds of single features, so the result does not depend on the scale of the features.

Before the model: replicate averaging and feature selection, if they are on. See Order of steps.

Results and metrics

The common classification metrics are described in Metrics. For binary problems, AUC uses the average tree probability of the positive class.

Random-forest-specific fields:

Field	Meaning
feature_importances	Mean decrease in Gini impurity per feature (final model), sums to 1
prediction_probabilities	Class probabilities for every sample (final model)
model_info	n_estimators and max_depth actually used

tip

Training metrics of a random forest (the final model on its own training data) are usually close to 100 %: every tree has memorised its bootstrap sample. Judge the model by the cross-validated metrics only.

Visualizations

• Feature importance
• Confusion matrix and ROC curve
• Permutation plot
• Time series and heatmap of the input data

Source code

run_random_forest_analysis in chrometrica/analysis/analysis.py. The imports, the model in the CV loop and the final model:

analysis.py · run_random_forest_analysis() · lines 2109–2301

def run_random_forest_analysis(X, y, parameters, cv_method, cv_folds):
    """Run Random Forest analysis with optional feature selection"""
    from sklearn.ensemble import RandomForestClassifier
    from sklearn.model_selection import StratifiedKFold, GroupKFold, KFold
    from sklearn.metrics import classification_report, confusion_matrix, accuracy_score, roc_auc_score
    import numpy as np
    # ...
    for train_idx, test_idx in cv.split(X_processed, y, groups=groups if cv_method == 'group' else None):
        X_train, X_test = X_processed[train_idx], X_processed[test_idx]
        y_train, y_test = y[train_idx], y[test_idx]

        rf_cv = RandomForestClassifier(random_state=42, n_estimators=n_estimators, max_depth=max_depth)
        rf_cv.fit(X_train, y_train)
        y_pred = rf_cv.predict(X_test)
    # ...
    # Fit final model
    rf = RandomForestClassifier(random_state=42, n_estimators=n_estimators, max_depth=max_depth)
    rf.fit(X_processed, y)
    y_pred_full = rf.predict(X_processed)
    y_pred_proba = rf.predict_proba(X_processed)

Full source of run_random_forest_analysis()

analysis.py · run_random_forest_analysis() · lines 2109–2427

def run_random_forest_analysis(X, y, parameters, cv_method, cv_folds):
    """Run Random Forest analysis with optional feature selection"""
    from sklearn.ensemble import RandomForestClassifier
    from sklearn.model_selection import StratifiedKFold, GroupKFold, KFold
    from sklearn.metrics import classification_report, confusion_matrix, accuracy_score, roc_auc_score
    import numpy as np

    # ---------- Helper to sanitize JSON ----------
    def sanitize_for_json(obj):
        if isinstance(obj, dict):
            return {k: sanitize_for_json(v) for k, v in obj.items()}
        elif isinstance(obj, list):
            return [sanitize_for_json(v) for v in obj]
        elif isinstance(obj, float) and np.isnan(obj):
            return None
        elif isinstance(obj, (np.float32, np.float64)) and np.isnan(obj):
            return None
        elif isinstance(obj, np.ndarray):
            return sanitize_for_json(obj.tolist())
        else:
            return obj

    if parameters.get('average_replicates', False):
        groups = parameters.get('groups')
        if groups is None:
            raise ValueError("'groups' must be provided when averaging replicates is enabled.")
        X, y, new_groups = average_replicates(X, y, groups, method='mean')
        parameters['groups'] = new_groups

    # Get overall classes for consistent confusion matrix dimensions
    overall_classes = np.unique(y)
    overall_classes_list = overall_classes.tolist()
    n_overall_classes = len(overall_classes_list)
    is_binary = (n_overall_classes == 2)

    # Number of trees and depth from the analysis dialog
    n_estimators = int(parameters.get('n_estimators', 100))
    max_depth = _optional_int(parameters.get('max_depth'))

    # Check for feature selection
    feature_selection_params = get_feature_selection_params(parameters)
    if feature_selection_params:
        X_processed, selected_features, feature_scores = select_features(
            X, y, feature_selection_params, 'classification'
        )
    else:
        X_processed = X
        selected_features = list(range(X.shape[1]))
        feature_scores = None

    # --- AUC initialisation (only used if binary) ---
    cv_auc_scores = []       # AUC per fold
    cv_scores_all = []       # decision scores / probabilities for all test samples

    # Enhanced cross-validation with metrics
    cv_scores = []
    cv_predictions = []
    cv_true_labels = []
    cv_class_reports = []
    cv_confusion_matrices = []
    cv_sensitivity_scores = []
    cv_specificity_scores = []

    if cv_method == 'stratified':
        cv = StratifiedKFold(n_splits=cv_folds, shuffle=True, random_state=42)
    elif cv_method == 'group':
        groups = parameters.get('groups')
        if groups is None:
            raise ValueError("Groups parameter required for group cross-validation")
        cv = GroupKFold(n_splits=cv_folds)
    else:
        cv = KFold(n_splits=cv_folds, shuffle=True, random_state=42)

    for train_idx, test_idx in cv.split(X_processed, y, groups=groups if cv_method == 'group' else None):
        X_train, X_test = X_processed[train_idx], X_processed[test_idx]
        y_train, y_test = y[train_idx], y[test_idx]

        rf_cv = RandomForestClassifier(random_state=42, n_estimators=n_estimators, max_depth=max_depth)
        rf_cv.fit(X_train, y_train)
        y_pred = rf_cv.predict(X_test)

        # ---------- AUC computation only for binary ----------
        if is_binary:
            # For binary we use predict_proba (positive class probability)
            if hasattr(rf_cv, "predict_proba"):
                y_prob = rf_cv.predict_proba(X_test)
                y_score = y_prob[:, 1]  # positive class probability
            elif hasattr(rf_cv, "decision_function"):
                y_score = rf_cv.decision_function(X_test)
            else:
                y_score = None

            if y_score is not None:
                try:
                    auc_fold = roc_auc_score(y_test, y_score)
                except ValueError:
                    auc_fold = None
                cv_auc_scores.append(auc_fold)
                if isinstance(y_score, np.ndarray):
                    cv_scores_all.extend(y_score.tolist())
                else:
                    cv_scores_all.extend(y_score)
            else:
                cv_auc_scores.append(None)
        else:
            # Multiclass: skip AUC entirely
            cv_auc_scores.append(None)

        # Store fold results
        cv_scores.append(accuracy_score(y_test, y_pred))
        cv_predictions.extend(y_pred)
        cv_true_labels.extend(y_test)

        fold_report = classification_report(y_test, y_pred, output_dict=True, zero_division=0)
        cv_class_reports.append(fold_report)

        cm = confusion_matrix(y_test, y_pred, labels=overall_classes_list)
        cv_confusion_matrices.append(cm)

        if is_binary:
            tn, fp, fn, tp = cm.ravel()
            sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
            specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
            cv_sensitivity_scores.append(sensitivity)
            cv_specificity_scores.append(specificity)
        else:
            cv_sensitivity_scores.append(None)
            cv_specificity_scores.append(None)

    # --- After CV loop: aggregated AUC only for binary ---
    auc_aggregated = None
    if is_binary and cv_scores_all:
        try:
            auc_aggregated = roc_auc_score(cv_true_labels, cv_scores_all)
        except ValueError:
            auc_aggregated = None
        if auc_aggregated is not None and np.isnan(auc_aggregated):
            auc_aggregated = None

    if is_binary:
        valid_auc = [a for a in cv_auc_scores if a is not None]
        cv_auc_mean = np.mean(valid_auc) if valid_auc else None
        cv_auc_std = np.std(valid_auc) if valid_auc else None
    else:
        cv_auc_mean = None
        cv_auc_std = None

    # ---- Continue with existing CV metrics (unchanged) ----
    cv_precision_scores = []
    cv_recall_scores = []
    cv_f1_scores = []

    for report in cv_class_reports:
        if 'macro avg' in report:
            cv_precision_scores.append(report['macro avg']['precision'])
            cv_recall_scores.append(report['macro avg']['recall'])
            cv_f1_scores.append(report['macro avg']['f1-score'])

    cv_confusion_matrix_aggregated = confusion_matrix(cv_true_labels, cv_predictions,
                                                      labels=overall_classes_list).tolist()

    if is_binary:
        cm_agg = np.array(cv_confusion_matrix_aggregated)
        tn, fp, fn, tp = cm_agg.ravel()
        cv_sensitivity_aggregated = tp / (tp + fn) if (tp + fn) > 0 else 0
        cv_specificity_aggregated = tn / (tn + fp) if (tn + fp) > 0 else 0
    else:
        cv_sensitivity_aggregated = None
        cv_specificity_aggregated = None

    if cv_confusion_matrices:
        cv_confusion_matrix_avg, avg_matrix_classes = create_average_confusion_matrix(
            cv_confusion_matrices, [overall_classes_list] * len(cv_confusion_matrices)
        )
        if is_binary:
            cm_avg = np.array(cv_confusion_matrix_avg)
            tn, fp, fn, tp = cm_avg.ravel()
            cv_sensitivity_avg = tp / (tp + fn) if (tp + fn) > 0 else 0
            cv_specificity_avg = tn / (tn + fp) if (tn + fp) > 0 else 0
        else:
            cv_sensitivity_avg = None
            cv_specificity_avg = None
    else:
        cv_confusion_matrix_avg = []
        avg_matrix_classes = []
        cv_sensitivity_avg = None
        cv_specificity_avg = None

    # Fit final model
    rf = RandomForestClassifier(random_state=42, n_estimators=n_estimators, max_depth=max_depth)
    rf.fit(X_processed, y)
    y_pred_full = rf.predict(X_processed)
    y_pred_proba = rf.predict_proba(X_processed)

    final_classes = rf.classes_.tolist()

    if cv_confusion_matrix_avg and avg_matrix_classes != final_classes:
        avg_to_final = {cls: idx for idx, cls in enumerate(avg_matrix_classes)}
        n_classes = len(final_classes)
        reordered_matrix = np.zeros((n_classes, n_classes))
        for i, true_cls in enumerate(final_classes):
            for j, pred_cls in enumerate(final_classes):
                if true_cls in avg_to_final and pred_cls in avg_to_final:
                    orig_i = avg_to_final[true_cls]
                    orig_j = avg_to_final[pred_cls]
                    if (orig_i < len(cv_confusion_matrix_avg) and
                            orig_j < len(cv_confusion_matrix_avg[0])):
                        reordered_matrix[i, j] = cv_confusion_matrix_avg[orig_i][orig_j]
        cv_confusion_matrix_avg = reordered_matrix.tolist()
        avg_matrix_classes = final_classes

    full_confusion_matrix = confusion_matrix(y, y_pred_full, labels=final_classes)
    if is_binary and full_confusion_matrix.shape == (2, 2):
        tn, fp, fn, tp = full_confusion_matrix.ravel()
        full_sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
        full_specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
    else:
        full_sensitivity = None
        full_specificity = None

    validation_message = None
    if is_binary and full_sensitivity is not None and full_specificity is not None:
        avg_sens_spec = (full_sensitivity + full_specificity) / 2
        accuracy = accuracy_score(y, y_pred_full)
        accuracy_diff = abs(accuracy - avg_sens_spec)
        if accuracy_diff > 0.01:
            validation_message = f"Note: Accuracy ({accuracy:.3f}) differs from (sensitivity + specificity)/2 ({avg_sens_spec:.3f}) by {accuracy_diff:.3f}"

    cv_sensitivity_scores_filtered = [s for s in cv_sensitivity_scores if s is not None]
    cv_sensitivity_mean = np.mean(cv_sensitivity_scores_filtered) if cv_sensitivity_scores_filtered else None
    cv_sensitivity_std = np.std(cv_sensitivity_scores_filtered) if cv_sensitivity_scores_filtered else None

    cv_specificity_scores_filtered = [s for s in cv_specificity_scores if s is not None]
    cv_specificity_mean = np.mean(cv_specificity_scores_filtered) if cv_specificity_scores_filtered else None
    cv_specificity_std = np.std(cv_specificity_scores_filtered) if cv_specificity_scores_filtered else None

    # --- Full-data (resubstitution) AUC and accuracy, field parity with run_lda_analysis ---
    full_auc = None
    if is_binary:
        try:
            pos_idx = final_classes.index(overall_classes_list[1])
            full_auc = roc_auc_score(y, y_pred_proba[:, pos_idx])
        except Exception:
            full_auc = None
    full_accuracy = accuracy_score(y, y_pred_full)

    results = {
        'method': 'Random Forest',
        'cv_scores': cv_scores,
        'cv_mean': np.mean(cv_scores) if cv_scores else 0.0,
        'cv_std': np.std(cv_scores) if cv_scores else 0.0,
        'cv_precision': np.mean(cv_precision_scores) if cv_precision_scores else 0.0,
        'cv_precision_std': np.std(cv_precision_scores) if cv_precision_scores else 0.0,
        'cv_recall': np.mean(cv_recall_scores) if cv_recall_scores else 0.0,
        'cv_recall_std': np.std(cv_recall_scores) if cv_recall_scores else 0.0,
        'cv_f1': np.mean(cv_f1_scores) if cv_f1_scores else 0.0,
        'cv_f1_std': np.std(cv_f1_scores) if cv_f1_scores else 0.0,
        # Sensitivity
        'cv_sensitivity': cv_sensitivity_mean,
        'cv_sensitivity_std': cv_sensitivity_std,
        'cv_sensitivity_aggregated': cv_sensitivity_aggregated,
        'cv_sensitivity_avg': cv_sensitivity_avg,
        'cv_sensitivity_scores': cv_sensitivity_scores,
        # Specificity
        'cv_specificity': cv_specificity_mean,
        'cv_specificity_std': cv_specificity_std,
        'cv_specificity_aggregated': cv_specificity_aggregated,
        'cv_specificity_avg': cv_specificity_avg,
        'cv_specificity_scores': cv_specificity_scores,
        # AUC (sanitized; only for binary)
        'cv_auc': sanitize_for_json(cv_auc_mean),
        'cv_auc_std': sanitize_for_json(cv_auc_std),
        'cv_auc_scores': sanitize_for_json(cv_auc_scores),
        'cv_auc_aggregated': sanitize_for_json(auc_aggregated),
        # Full-data AUC / accuracy (+ CV accuracy alias), field parity with LDA
        'auc': sanitize_for_json(full_auc),
        'accuracy': full_accuracy,
        'cv_accuracy': np.mean(cv_scores) if cv_scores else 0.0,
        # Validation
        'is_binary': is_binary,
        'validation_message': validation_message,
        # Confusion matrices
        'cv_confusion_matrix': cv_confusion_matrix_avg,
        'cv_confusion_matrices': [cm.tolist() for cm in cv_confusion_matrices],
        'cv_confusion_matrix_aggregated': cv_confusion_matrix_aggregated,
        'cv_class_reports': cv_class_reports,
        'classification_report': classification_report(y, y_pred_full, output_dict=True, zero_division=0),
        'confusion_matrix': full_confusion_matrix.tolist(),
        'sensitivity': full_sensitivity,
        'specificity': full_specificity,
        'feature_importances': rf.feature_importances_.tolist(),
        'labels': y.tolist(),
        'predictions': y_pred_full.tolist(),
        'prediction_probabilities': y_pred_proba.tolist(),
        'classes': final_classes,
        'model_info': {'n_estimators': n_estimators, 'max_depth': max_depth},
    }

    if feature_selection_params:
        results['feature_selection'] = {
            'selected_features': selected_features,
            'feature_scores': feature_scores,
            'method': feature_selection_params.get('method', 'anova'),
            'k': feature_selection_params.get('k', 'all')
        }

    if is_binary and full_sensitivity is not None and full_specificity is not None:
        results['metric_consistency_check'] = {
            'accuracy': accuracy_score(y, y_pred_full),
            'sensitivity': full_sensitivity,
            'specificity': full_specificity,
            'avg_sens_spec': (full_sensitivity + full_specificity) / 2,
            'difference': abs(accuracy_score(y, y_pred_full) - (full_sensitivity + full_specificity) / 2)
        }

    results = sanitize_for_json(results)

    # Final fitted model for inference on unknown samples (popped in run_analysis)
    return attach_model_bundle(results, rf, selected_features, feature_selection_params)

Limitations and common pitfalls

• Training metrics look perfect. See the tip in Results and metrics.
• Importance is biased towards features with many distinct values, and it is split between correlated features: two nearly identical channels each get about half of the importance they would have alone.
• Importance comes from the training data. It says what the final model uses, not what generalises. Check it together with the CV metrics and a permutation test.

References

• Breiman L. Random forests. Machine Learning, 45, 5–32 (2001). doi:10.1023/A:1010933404324
• scikit-learn user guide: Random forests.