Adam DeJans Jr describes how to think about decision systems through the lens of sequential decision problems. “Most operational environments are not one-time optimization problems. They are ongoing processes where decisions must be made repeatedly as the state of the world evolves.

Consider a transportation network. At any given moment, there is a current state of the system: trucks are located in different regions, orders are arriving, drivers have hours-of-service constraints, weather conditions are changing, and new information is constantly entering the system. Any decision we make (dispatching a truck, accepting a load, repositioning inventory) changes that state and affects the options available in the future. From a technical perspective, these systems usually revolve around a few core components.” Link

First is the state representation. This is the structured description of the operational world at a given moment. It might include the location of resources, current commitments, outstanding demand, capacity limits, and other constraints that define what actions are feasible.

Second is the decision policy. This is the mechanism used to determine the next action. In practice this might involve mathematical optimization, heuristics, simulation models, reinforcement learning policies, or some hybrid of these approaches. The key point is that the policy maps the current state of the system to a decision.

Third is the state transition. Once a decision is executed, the system evolves. Trucks move, orders are fulfilled, inventory levels change, and new external information arrives. The next decision must then be made based on this updated state.

What makes real-world decision systems challenging is that this loop operates continuously under uncertainty. Demand forecasts are imperfect. Travel times fluctuate. Equipment fails. Competitors react. The system must adapt as new information arrives.