Lecture 4: Probability and statistics

You should be sure that you are comfortable with the basic rules of probability:

• Conditional probabilities: what does p(a|b) mean?
• Marginalization: p(a) = sum_b p(a,b)
• Chain rule: p(a,b) = p(a) p(b|a) = p(b) p(a|b)
• Iterated chain rule: p(a,b,c,d,...,z) = p(a) p(b|a) p(c|a,b) ... p(z|a,b,c,...,y)
• Bayes' rule: p(a|b) = p(a) p(b|a) / p(b)
• Generalized Bayes' rule: how to do BR on p(a|b,c) or p(a,b|c)
• Expectations: E_p[f(x)] = sum_x p(x) f(x)

You should also be comfortable answering the probability questions from the homework, which we'll work through in class.

You should also be comfortable with some basic statistics:

• Maximum likelihood estimates
• = relative frequencies
• = counting and normalizing
• The difference between
• Structural zeros
• Statistical zeros
• The importance of smoothing