VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -1notes
5 likes909 views
This document provides formulas and concepts related to information theory and coding. It defines key information theory terms like self-information, entropy, information rate, bits, Hartley's, nats, extremal entropy, source efficiency, and source redundancy. Formulas are given for calculating each of these terms. The document also provides formulas for calculating the average information content of symbols from different states. Finally, it lists some basic log properties that are important in information theory calculations.
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -1notes
- 1. INFORMATION THEORY AND CODING 5th SEM E&C JAYANTHDWIJESH H P M.tech (DECS) Assistant Professor Dept of E&C B.G.S INSTITUTE OF TECHNOLOGY (B.G.S.I.T) B.G Nagara, Nagamangala Tq, Mandya District- 571448
- 2. FORMULAS FOR REFERENCE MODULE 1(INFORMATION THEORY) Amount of information or Self information. = log ( ) or = ヰ ( ) or I ( ) = log ( ) Entropy of source or Average information content of the source. H = 倹 ヰ ( = ) bits/symbol or H = ヰ ( = ) bits/symbol or H(S) = 倹 ヰ ( = ) bits/symbol or H(S) = 倹 ヰ( = ) bits/symbol or H(S) = ヰ ( = ) bits/symbol Information rate or average information rate. = H(S) bits/sec or R= H bits/sec or R=r H bits/sec Bits. = ヰ ( ) bits Hartleys or Decits. = ヰ ( ) Hartleys or Decits Nats or Neper. = ヰ ( ) Nats or Neper. Extremal or Upper bound or Maximum entropy () = ヰ bits/message-symbol or () = ヰ bits/message-symbol. Source efficiency = () () or = () () X % Source redundancy = 1- = (1 - () () ) X % The average information content of the symbols emitted from the i th state. = 倹 ヰ ( 倹 = ) bits/symbol or = 倹 ヰ ( 倹 = ) bits/symbol The average information content of the symbols emitted from the k th state.
- 3. = 倹 ヰ ( 倹 = ) bits/symbol The average information content per symbol in a message of length N. = ( )log ( ) or = ( )log P () = H ( ) The entropy of the second order symbols. = H ( ) where N=2. The entropy of the third order symbols. = H ( ) where N=3. Log properties 1. ヰ = ヰ . ヰ ヰ = ヰ . ヰ = ln (10)