A model in which a foraging animal can reproduce if its energy reserves reach a critical level is presented. The animal's reserves are modelled as a finite-state Markov chain. The animal has the choice between foraging options which may have the same mean net gain but differ in their variances. In addition to starvation, animals are subject to death because of predation, bad weather, and so on. We focus on the case where the rate of mortality due to these sources is the same under all options. We investigate policies that maximise the expected lifetime reproductive success. It is found that the optimal value function is concave at low reserves (low-variance action region) and convex at high reserves (high-variance action region). The value function under the low-variance action has also the same shape and the same inflexion point. This result allows us to compute optimal policies just by looking at the low-variance value function. The result is also used to show that increasing the mortality rate increases the high-variance region under the optimal policy. The pattern of risk-sensitive behaviour predicted by this model is in contrast to that predicted by a similar model in which no reproduction occurs and the optimality criterion is to minimise the probability of death (McNamara (1990)).

A risk-sensitive certainty equivalence principle is deduced, expressed in Theorem 1, for a model with linear dynamics and observation rules, Gaussian noise and an exponential-quadratic criterion of the form (2). The senses in which one is now to understand certainty equivalence and the separation principle are discussed.

The conventional linear/quadratic/Gaussian assumptions are modified in that minimisation of the expectation of cost G defined by (2) is replaced by minimisation of the criterion function (5). The scalar –θ is a measure of risk-aversion. It is shown that modified versions of certainty equivalence and the separation theorem still hold, that optimal control is still linear Markov, and state estimate generated by a version of the Kalman filter. There are also various new features, remarked upon in Sections 5 and 7. The paper generalises earlier work of Jacobson.