A new article by Adam DeJans Jr. argues that decision systems should be judged closer to the economic outcome they actually produce—not solely by the accuracy of the forecast feeding them.
The central idea is that a forecast describes uncertainty, while a decision policy converts that information into action. When the business ultimately experiences the action, improvements in forecast error metrics are valuable only to the extent that they improve the downstream decision.
The article outlines a practical approach using simulation-based policy optimization: define an operable policy, expose it to plausible uncertain futures, score the resulting decisions economically, and tune the policy against that objective. It also discusses decision regret, operational stability, and why system architecture should be designed backward from the decision.