SMARS & SCMARS: Accelerating Nonparametric Regression for the Big Data Era

TL;DR

Nonparametric models like Multivariate Adaptive Regression Splines (MARS) are celebrated for their flexibility in modeling nonlinear relationships, but they are notoriously slow on large datasets. This paper proposes SMARS and SCMARS—hybrid algorithms that use a "Mapping" approach to intelligently prune candidate knots. The result? A 3x to 10x speedup in training time with no loss in accuracy, making these tools viable for real-time applications like banking interest rate prediction.

The Bottleneck: The Exhaustive Search Problem

The power of MARS lies in its ability to build models using Basis Functions (BFs): $\hat{f}(\mathbf{x})={\hat{c}}_0+{\sum }_{m=1}^M{\hat{c}}_m{\psi }_m(\mathbf{x})$ However, to find the optimal "knots" (the points where the splines bend), traditional MARS searches through every distinct value of every predictor. In a world of Big Data, this exhaustive search is a computational nightmare. Even the advanced CMARS, which uses Tikhonov regularization to improve robustness, inherits this sluggish forward-selection phase.

The "Mapping" Intuition: Why SMARS Works

The core insight of the authors is that we don't need to check every data point to find a good knot. By using a Mapping Approach (inspired by Self-Organizing Maps), the algorithm first projects the high-dimensional data into a lower-dimensional space of "neurons."

1. Data Mapping: Data points are mapped to representative units.
2. Knot Restriction: Only data points near these representative units are considered as candidate knots.
3. Hybrid Execution: This restricted set is fed into the forward selection, which is then pruned by either MARS's backward elimination or CMARS's Conic Quadratic Programming.

Experimental Showdown

The authors tested these hybrids against standard MARS, CMARS, and versions using uniform knot spacing.

• Speed: In small, medium, and large synthetic datasets, SMARS and SCMARS consistently outperformed the baselines. As sample size ($n$) increased, SCMARS became the most efficient choice, maintaining stability even as complexity grew.
• Accuracy: Despite searching fewer knots, the hybrid methods achieved comparable or superior RMSE and $Adj-R^2$. In many cases, SMARS was actually more accurate because the mapping approach acts as a natural filter against local noise.

Case Study: Banking on Efficiency

A real-world application for a Turkish bank involved predicting daily interest rates for retail deposits.

• The Challenge: Large dimensionality (21 predictors) and the need for frequent model updates.
• The Result: SCMARS produced the most robust and stable models, identifying "account maturity" and "previous interest rates" as the most significant predictors. The efficiency gain (4x faster) allows the bank to retrain models more frequently as market conditions shift.

Critical Insight & Conclusion

While SCMARS typically produces more complex models (more basis functions) than SMARS, its robustness—measured by the standard deviation of performance across folds—is superior. The "takeaway" for data scientists is clear: Exhaustive search is an anti-pattern in Big Data. By introducing a structure-aware pre-processing step (Mapping), we can retain the interpretability and flexibility of regression splines while meeting the latency requirements of modern finance.

Limitations

• Complexity: SCMARS models can be "dense," retaining basis functions with near-zero coefficients. Future work could integrate Bootstrapping to further prune these models.
• Scope: Currently optimized for continuous responses; extension to classification (discrete labels) is still an open area.