Power control over Gilbert-Elliot channel with no observable states

A dynamic communication channel is modeled as Markov chain where states describe the quality of channel. One such example is two state Gilbert-Elliot channel. The states of a channel is never observed by transmitter, but success and failure is observed with probability depending on state of channel. The information available to transmitter is the current belief about states and it is updated based on action and observation of a signal. The transmitter want to send a packet over channel with different power control schemes in each slot to maximise long term discounted reward. We formulate this as infinite horizon discounted reward problem. We write a dynamic program, and derive the properties of value function. For a special case, we show that the optimal policy has a single threshold. Further, we present few numerical examples to illustrate this.