Five-Fold Cross-Validation Protocol
Why every Lewsearch accuracy number is out-of-sample, and how we prevent calibration leakage across the benchmark pool.
A common failure mode in synthetic panel validation is in-sample fitting: tuning a system on the same questions used to report accuracy. Lewsearch reports only cross-validated figures.
We use five-fold cross-validation across the full benchmark pool. Calibration parameters are estimated on four folds and evaluated on the fifth. Each question appears exactly once in a held-out evaluation fold.
Survey-date conditioning
Each item is administered to agents as of the midpoint of the source survey's field window, not as of today's date. That temporal discipline prevents leakage from the wrong field period and keeps comparisons honest against published marginals from the original instrument.
Raw vs calibrated
We publish raw and calibrated error side by side. Post-hoc calibration reduces pooled MAE by roughly 28–43% depending on panel, but it cannot fix category mismatch or items where the underlying model prior is far from the population marginal.
Product validation
Customer-facing benchmark tables and live predictions are on lewsearch.com/methodology. Due diligence materials available on request.