Tham khảo tài liệu 'communication systems engineering episode 1 part 2', kỹ thuật - công nghệ, cơ khí - chế tạo máy phục vụ nhu cầu học tập, nghiên cứu và làm việc hiệu quả | Communication Systems Engineering Lecture 2 Entropy Eytan Modiano Eytan Modiano Slide 1 Information content of a random variable Random variable X - Outcome of a random experiment - Discrete . takes on values from a finite set of possible outcomes PMF P X y Px y How much information is contained in the event X y - Will the sun rise today Revealing the outcome of this experiment provides no information - Will the Celtics win the NBA championship Since this is unlikely revealing yes provides more information than revealing no Events that are less likely contain more information than likely events Eytan Modiano Slide 2 Measure of Information I xi Amount of information revealed by an outcome X xi Desirable properties of I x 1. If P x 1 or P x 0 then I x 0 2. If 0 P x 1 then I x 0 3. If P x P y then I x I y 4. If x and y are independent events then I x y I x I y Above is satisfied by I x Log2 1 P x Base of Log is not critical - Base 2 information measured in bits Eytan Modiano Slide
