The dominant paradigm in machine learning has long been model-centric. Sambasivan et al.⁸ documented empirically that data quality issues were a primary source of failure in deployed AI systems; yet, data work remained undervalued. Northcutt et al.⁹ demonstrated that confident learning could identify mislabeled samples at scale, revealing non-trivial error rates in widely used benchmarks. Surveys of data-centric methods^10,11 have highlighted that dataset quality improvements often yield larger performance gains than equivalent investments in model complexity.

Koh and Liang¹ showed that the parameter change from upweighting sample i by ε is Δθ ≈ -ε Hθ⁻¹ g_i, composable with the validation gradient to yield a scalar influence score. Tractability relies on CG or LiSSA¹²-based inverse Hessian approximations. TracIn² sums gradient dot products across checkpoints—computationally lighter but sensitive to checkpoint frequency; we ablate checkpoint frequency for TracIn in Section “TracIn Checkpoint Frequency Ablation”. Representer point selection³ provides exact attribution for L2-regularized models via the representer theorem, but is primarily suited to neural network final layers.

Recent work has extended influence estimation in several directions. FastIF and related Hessian-free methods⁵ reduce the cost of the inverse-Hessian-vector solve through low-rank approximations and sketching, making full-model influence tractable for moderately large networks. Incremental influence methods¹³ update influence scores as new data arrive without full recomputation, enabling streaming applications. Several analyses⁷ have examined when the second-order approximation underlying classical influence functions remains reliable in overparameterized networks, finding that reliability degrades with non-convexity and that LOO calibration is essential for empirical validation—as we show in Section “Leave-One-Out Calibration.”

Data maps⁴ track the confidence and variability of model predictions across epochs, identifying easy, hard, and ambiguous samples without Hessian computation. Ambiguous samples were often the most informative for generalization—a finding complementary to influence-based pruning strategies explored here.

Data Shapley¹⁴ defines sample value as the average marginal contribution across all dataset subsets, satisfying axiomatic properties but requiring exponentially many model evaluations. Practical approximations^15,16 improve scalability but remain substantially more expensive than influence function approaches. Our framework can be distinguished from data valuation approaches used within iterative pipelines: Shapley-based methods optimize an axiomatically^17,18 grounded global value function, whereas the data gradient framework optimizes a local, gradient-based influence signal with respect to a fixed validation objective, a distinction that makes the latter more computationally tractable for iterative refinement but less robust to distributional shift. For tree ensembles specifically, leaf-influence methods¹⁹ and Tree SHAP²⁰ provide principled attribution directly from tree structure without gradient approximations; we perform a comparison against these baselines in Section “Baselines.”

Influence functions and TracIn^1,2 are designed for differentiable models. Shapley methods¹⁴ are prohibitively expensive for iterative refinement. Data maps⁴ do not quantify influence on a specific validation objective. Tree-specific methods^19,20 handle non-differentiable architectures but are not integrated into end-to-end pipeline refinement loops. The data gradient framework addresses these gaps by combining the influence function estimator with pipeline integration, an aggregate quality metric, and explicit—if approximate—treatment of tree-based models.

Let D = {(x_i, y_i)}^k_i=1 denote the training dataset, D_val the held-out validation set, and θ ∈ ℝ^R the model parameters minimizing empirical loss L(θ; D) = (1/N) Σ_i ℓ(θ; x_i, y_i), where ℓ denotes the per-sample loss function. The per-sample gradient is g_i = ∇θ ℓ(θ; x_i, y_i) and the Hessian of the training loss is Hθ = (1/N) Σ_j ∇²θ ℓ(θ; x_j, y_j). The validation gradient is v = ∇θ L(θ; D_val). All quantities are evaluated at the converged parameter estimate θ̂ obtained after training. We use θ throughout to denote model parameters (not Θ) and γ to denote the reweighting temperature (not Y).

Following Koh and Liang,¹ the influence of training sample i on the validation loss isIi=−giTS,where S=Hθ−1v

The vector S is computed once per refinement iteration by solving the linear system Hθ S = v and is reused across all N training samples. This factorization reduces the total cost of computing all influence scores from O(N · P²) to O(P² + N · P)—or, with stochastic solvers, to O(JTP + NP)—since the dominant H⁻¹v solve is performed only once. A positive I_i indicates a training sample that, if removed, would increase validation loss (i.e., beneficial); a negative I_i indicates a potentially harmful sample whose removal would decrease validation loss.

For DistilBERT on IMDB, gradients are computed with respect to the classification head parameters only: a linear layer of dimension 768 × 2 with bias, yielding P = 768 × 2 + 2 = 1,538 parameters. The 768-dimensional projection weights account for 1,536 parameters; the 2-dimensional bias vector accounts for the remaining 2. The pre-trained transformer backbone is frozen during fine-tuning and influence computation. Bias parameters are included in the gradient computation.

Head-only influence computation reduces the LOO rank correlation to ρ = 0.76 (vs. ρ = 0.81 for ResNet-18, where the full model is used), reflecting information loss from treating the frozen backbone as a fixed feature extractor. Full-model influence for transformer architectures is computationally challenging due to large P, but methods such as influence sketching⁵ and layer-wise decomposition offer promising directions for improving fidelity without full Hessian computation (Section “DistilBERT Influence Fidelity”).

Three perturbation forms are supported. Sample removal eliminates instances with I_i below the α-th percentile (default α = 10, ablated in Section “DCS, Ablations, and Validation Sensitivity”). Label correction flags samples in the bottom 5th percentile for automated relabeling using confident learning.⁹ Sample reweighting assigns w_i = σ(γ I_i) = 1/(1 + exp(−γ I_i)), where γ = 1.0 (ablated in Section “DCS, Ablations, and Validation Sensitivity”). Reweighting is preferred over removal in fairness-sensitive or small-dataset settings.

Because the validation gradient v = ∇θ L(θ; D_val) is central to all influence scores and is reused across refinement iterations, there is a risk that repeated refinement cycles implicitly overfit to the validation set. We address this risk in four ways. First, validation and test sets are held strictly separate throughout all experiments; influence computation uses D_val and model evaluation uses the held-out test set. Second, we limit refinement to a maximum of three iterations, consistent with the plateau observed in DCS and performance gains. Third, we report sensitivity analyses in Section “Model Performance After Refinement”, showing DCS and test accuracy as functions of refinement iteration, making any divergence between validation-based pruning decisions and test performance visible. Fourth, we report a holdout-of-holdout sanity check: for CIFAR-10 (10% noise), the validation set is split 50/50 into D_val^a (used for influence computation) and D_val^B (used to verify that DCS on D_val^B tracks DCS on D_val^a); the two DCS trajectories differ by at most 0.03 over three iterations, providing no evidence of leakage within the tested range. A full cross-fitting procedure remains an important direction for future work.

The DCS is defined asDCS=1/N′ΣiĨi, Ĩi=Ii/maxj|Ij| where N′ is the number of retained samples and normalization maps scores to [-1, +1]. DCS ranges from -1 (all samples harmful) to +1 (all samples maximally beneficial). Important caveats are as follows: (i) aggressive pruning mechanically increases DCS by removing low-influence samples from the average, regardless of downstream performance, and (ii) in imbalanced datasets, minority-class samples carry disproportionately high influence scores, inflating aggregate DCS. Class-stratified DCS is therefore reported by default in all imbalanced experimental settings (Table 6). Table 6 (Section “DCS, Ablations, and Validation Sensitivity”) provides a compact alignment of DCS deltas with downstream performance changes across α thresholds, confirming that DCS is a diagnostic signal and should not be used as a standalone optimization target.

For image classification, we use CIFAR-10,²² a 10-class dataset (50,000 training/10,000 test images). Synthetic label noise is introduced by randomly flipping 10% and 20% of training labels following Northcutt et al.⁹ For text classification, we use IMDB,²³ with 25,000 training and 25,000 test reviews. For tabular data, we use UCI Adult,²⁴ with 48,842 instances and 14 features (75% negative class imbalance, ~1–2% duplicate rows). All splits: 70/15/15 (train/val/test), fixed seed 42 (Table 2).

CIFAR-10: ResNet-18²⁵ (PyTorch, batch normalization, no pre-training). IMDB: DistilBERT²⁶ fine-tuned with frozen backbone; gradients computed for classification head only (P = 1,538; see Section “DistilBERT: Gradient Scope and Bias Handling”). UCI Adult: logistic regression, Random Forest (500 trees), XGBoost (100 rounds).

Four baselines: Influence Functions¹ (exact CG inversion, no pipeline integration); TracIn² (gradient dot products across five checkpoints at epochs 10, 20, 30, 40, 50; checkpoint frequency ablated in Section “TracIn Checkpoint Frequency Ablation”); Representer Point Selection³ (final layer of each neural model); and Random Pruning (same removal count, uniformly random). For GBT and Random Forest settings on UCI Adult, we additionally compare against leaf-influence¹⁹ and Tree SHAP²⁰ as tree-specific baselines on identical splits. All baselines use identical splits, configurations, and evaluation protocols.

All experiments: five independent random seeds {0, 1, 2, 3, 42}, weights re-initialized, and data reshuffled. Results: mean ± standard deviation. Statistical tests: paired two-sided Wilcoxon signed-rank test (nonparametric, appropriate for n = 5 seeds) with α = 0.05. Effect sizes: Cohen’s d computed from per-seed values. Multiple-comparison correction: Benjamini–Hochberg FDR at q = 0.05 across all dataset–baseline pairs within each model family. Primary metrics: classification accuracy and macro-averaged F1-score on the held-out test set. Secondary metrics: DCS before and after refinement, mislabel detection precision and recall, influence computation time, and peak memory.

LOO validation on stratified 1,000-sample subsets of each dataset. For each sample, the model is retrained from scratch with that sample excluded and the change in validation loss recorded. Calibration metrics: Spearman rank correlation ρ, mean absolute error (MAE), and AUC for harmful-sample detection (where ground truth is the synthetically flipped labels for CIFAR-10 noisy variants). Direct comparison between our pseudo-residual GBT estimator and the leaf-influence baseline¹⁹ uses rank correlation and AUC confidence intervals on identical LOO splits.

Four ablation experiments: (i) removal threshold α ∈ {5, 10, 15, 20} percentile; (ii) LiSSA recursion depth T ∈ {100, 300, 500, 1000}—LOO rank correlation versus computation time; (iii) reweighting temperature γ ∈ {0.5, 1.0, 2.0, 5.0}; and (iv) TracIn checkpoint frequency (1, 2, 5, 10 checkpoints)—mislabel detection precision versus storage cost. All ablation figures are presented in graphical form in Appendix C.

Table 3 reports the Spearman rank correlation, MAE, and AUC for harmful-sample detection against LOO ground truth. Correlations are the highest for logistic regression (ρ = 0.93) and the lowest for Random Forests (ρ = 0.58). For CIFAR-10 with 10% synthetic noise, the data gradient estimator achieves AUC = 0.79 ± 0.03 for detecting flipped labels, compared to AUC = 0.61 ± 0.05 for the proximity-based Random Forest estimator. The GBT pseudo-residual estimator achieves ρ = 0.71 and AUC = 0.73, compared to ρ = 0.74 and AUC = 0.76 for the leaf-influence baseline on identical splits, indicating that the tree-specific method provides modestly better calibration.

Table 5 reports the test accuracy and macro F1-score with effect sizes (Cohen’s d) and 95% confidence intervals derived from per-seed distributions. All significance tests are Wilcoxon signed-rank with Benjamini–Hochberg FDR correction (q = 0.05). Random Pruning produces no consistent improvement, confirming that gains are attributable to sample selection quality rather than dataset size reduction. The data gradient framework matches or outperforms all baselines across settings. Gains for Random Forest are modest and do not survive FDR correction, consistent with the lower LOO calibration reported in Section “Leave-One-Out Calibration.”

LiSSA ablation: LOO rank correlation improves from ρ = 0.61 (T = 100) to ρ = 0.80 (T = 300), plateauing at ρ = 0.81 (T = 500 and T = 1000), while computation time scales linearly; T = 500 is selected as the optimal trade-off. Reweighting: stable across γ ∈ {0.5, 1.0, 2.0}, degrades at γ = 5.0 (binary-like behavior approaching hard removal). DCS and test accuracy track each other through three refinement iterations with no evidence of divergence. The holdout-of-holdout sanity check (Section “Validation Leakage and Overfitting Risk”) confirms no leakage signal within this range (Table 6).

Three main conclusions follow from the experimental results. First, influence-guided refinement produces consistent, statistically significant improvements over unrefined baselines across all differentiable model families, with effect sizes ranging from Cohen’s d = 0.41 to 1.12. Gains are modest under clean label conditions but increase substantially under label noise, where mislabeled sample identification drives performance. This pattern is consistent with Koh and Liang,¹ and Northcutt et al.⁹

Second, the comparison against TracIn,² Representer Point Selection,³ and tree-specific methods^19,20 reveals broadly comparable detection performance, with the primary practical advantage of the data gradient framework lying in pipeline integration. Third, LOO calibration and AUC results confirm that Random Forest influence estimates are substantially less reliable than differentiable-model estimates, and the modest gains for Random Forest do not survive FDR correction.

DCS reliably tracks the direction of performance improvement across refinement iterations in most settings but diverges from test accuracy when pruning is aggressive. The α = 20 ablation (Table 6) makes this visible: DCS continues rising, while test accuracy declines toward baseline. Practitioners should treat DCS as a complementary signal—diagnostic, not an optimization target. Class-stratified DCS is recommended by default in imbalanced settings. When minority classes are highly influential, aggregate DCS will be dominated by minority-class samples even if majority-class quality is poor; stratification makes this visible. The threshold alignment in Table 6 provides actionable guidance: DCS gains above α = 10 no longer correlate with test performance gains, and practitioners should not exceed this threshold without explicit holdout verification.

Computing influence scores only for the classification head (P = 1,538 parameters) reduces LOO rank correlation to ρ = 0.76 compared to ρ = 0.81 for ResNet-18. This gap reflects the information loss from treating the frozen transformer backbone as a fixed feature extractor. Full-model influence for transformer architectures is computationally challenging due to large P, but methods such as influence sketching⁵ and layer-wise decomposition offer promising directions for improving fidelity without full Hessian computation.

The sensitivity analysis confirms that DCS and test accuracy track each other through three refinement iterations with no evidence of divergence. The holdout-of-holdout sanity check (Section “Validation Leakage and Overfitting Risk”) further corroborates this. However, this analysis cannot rule out leakage over more iterations or in settings where validation and test distributions differ. We recommend the three-iteration limit as a practical safeguard. A full cross-fitting procedure—rotating D_val across folds—remains the most important near-term methodological improvement for reducing leakage risk.

Group-wise influence score distributions were computed on UCI Adult (logistic regression) stratified by sex and race. Female-coded samples had mean normalized influence score 0.28 ± 0.04 versus 0.34 ± 0.03 for male-coded samples (Wilcoxon p < 0.01), meaning that a standard α = 10 threshold would remove proportionally more female-coded samples. Group-aware pruning—applying α = 10 separately within each demographic stratum—preserved group representation while maintaining 85.3 ± 0.4% test accuracy (vs. 85.6 ± 0.4% for standard pruning). Reweighting without removal preserved both accuracy (85.4 ± 0.4%) and group representation fully. These results confirm that group-aware strategies are preferable in fairness-sensitive deployment contexts, and reweighting is the recommended default when demographic balance is a constraint.

The influence function estimator assumes model convergence and a locally quadratic loss surface, both imperfect for overparameterized neural networks. For DistilBERT, influence is computed for the head only, limiting fidelity. The iterative refinement loop assumes full retraining feasibility, which does not hold for large foundation models. Experimental validation uses benchmark datasets with synthetic noise; naturalistic noise patterns may yield different calibration. The Random Forest proximity estimator is a heuristic; practitioners requiring high-precision attribution should use Tree SHAP²⁰ or leaf-influence.¹⁹

Incremental influence estimation—updating scores as new data arrive without full recomputation¹³—would enable real-time monitoring in production pipelines. Integration with AutoML could allow influence scores to inform feature selection and curriculum design. For foundation models, influence sketching and distributed Hessian approximation methods warrant investigation. Full-model influence for fine-tuned transformers and cross-fitting-based validation strategies are the most immediately impactful near-term extensions.