Question

According to Bellman’s Principle of Optimality, what must be true about the sub-policies of an optimal policy for a multi-stage decision process?

Accepted Answer

According to Bellman’s Principle of Optimality, an optimal policy must consist of optimal sub-policies regardless of the initial state or the initial decision. A policy is a sequence of decisions made across multiple stages of a process. A multi-stage decision process is a sequence of steps where each choice influences the state of the system and the potential future rewards. The principle states that if you have an optimal way to reach a final goal from a starting point, then any portion of that path—starting from an intermediate state and ending at the goal—must also be the optimal way to reach the goal from that intermediate state. If a sub-path were not optimal, you could replace it with a better sub-path to improve the overall total outcome, which would contradict the assumption that the original full policy was optimal. For example, if the fastest driving route from New York to Los Angeles passes through Chicago, the segment of that route from Chicago to Los Angeles must also be the fastest possible way to get from Chicago to Los Angeles. If a faster way existed between Chicago and Los Angeles, you would take it to shorten your total travel time, meaning the original route through Chicago was not truly the optimal path.

Home → All Courses → Science and Mathematics Courses → Decision Science and Optimization → Flashcard

According to Bellman’s Principle of Optimality, what must be true about the sub-policies of an optimal policy for a multi-stage decision process?