MIT press published an excellent book “Algorithms for decision making” with free download that provides a broad introduction to algorithms for decision making under uncertainty. The book takes an agent based approach when physical entities like humans or robots act based on observations of their environment. The interaction between the agent and the environment follows an observe-act cycle. Link