:: Volume 7, Issue 2 (3-2011) ::
JSRI 2011, 7(2): 103-120 Back to browse issues page
Taylor Expansion for the Entropy Rate of Hidden Markov Chains
H. Yar 1, Z. Nikooravesh
1- , yari@iust.ac.ir
Abstract:   (3728 Views)

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate for the entropy rate of hidden Markov chain by matrix algebra and its spectral representation. To do so, we use the Taylor expansion, and calculate some estimates for the first and the second terms, for the entropy rate of the hidden Markov process and its binary version, respectively. For small varepsilon (channel's parameter), the entropy rate has o(varepsilon^2), as a maximum error, when it is calculated by the first term of Taylor expansion and it has  o(varepsilon^3) , as a maximum error, when it is calculated by the second term.

Keywords: Entropy rate, hidden Markov process, Taylor approximation, spectral representation
Full-Text [PDF 214 kb]   (949 Downloads)    
Type of Study: Research | Subject: General
Received: 2016/01/10 | Accepted: 2016/01/10 | Published: 2016/01/10



XML   Persian Abstract   Print



Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Volume 7, Issue 2 (3-2011) Back to browse issues page