This is true even within the narrow context in which the effect was originally demonstrated: A model can produce highly significant effects of interest that nevertheless do not enable an analyst to nontrivially predict outcomes for new observations even when those observations are sampled from the same data-generating process as previously seen observations.
← all excerpts
Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction.
1
—
—