Multistate, Multinomial

LA home
Computing
MML
 Glossary
 KL
  M-state
  Normal
  M-state

Also see:  M-state

KL-distance from p1 to p2

E.g. Consider these two distributions over {A, C, G, T}:
  A C G T  
p1 1/4 1/4 1/4 1/4 pr
2 2 2 2 -log pr, bits
p2 1/2 1/4 1/8 1/8 pr
1 2 3 3 -log pr, bits 
               1    1    1    1    1
KL(p1, p2) = - -1 + -0 + -1 + -1 = - bits
               4    4    4    4    4

               1    1    1    1    1
KL(p2, p1) =   -1 + -0 - -1 - -1 = - bits
               2    4    8    8    4
So KL(p1, p2) = KL(p2, p1) in this case -- but not in general.
 
 
Now you calculate the two distances for these two distributions:
  ACGT
p1 1/21/41/81/8
p2 1/81/21/41/8
KL(p1, p2) = ...

KL(p2, p1) = ...
www #ad:
↑ © L. Allison, www.allisons.org/ll/   (or as otherwise indicated).
Created with "vi (Linux)",  charset=iso-8859-1,   fetched Monday, 02-Oct-2023 22:27:06 UTC.
Free: Linux, Ubuntu operating-sys, OpenOffice office-suite, The GIMP ~photoshop, Firefox web-browser, FlashBlock flash on/off.