bayesian statistics

[clarkedavida]

i've been learning a lot about bayesian statistics. here's some stuff to add:

• something about the philosophy of it. you're using Data analysis: a Bayesian tutorial, which is a great book
• start with posterior pdf for fit parameters. taylor expand about the point x_0 of maximum probability. leading order term drops off by defn. next term is quadratic, and is a measure of the spread of the pdf about x_0. the true pdf of the fit parameters should be gaussian (according to CLT and Bayes). we want to compute the variance in the standard way, but we can't because the true pdf depends on a true mean and true spread we don't know a priori. this quadratic term in the expansion, however, has no dependence on these true values, since we are evaluating at the "most likely result", which we take to be the result of our fit. approximating our original pdf by exp(quadratic) only leads to the chi2 distribution. using this approximation, calculating the covariance leads to the inverse hessian of the chi2 distribution.
• in n-d, the leading-order term can be expressed as a matrix equation. the eigenvectors of that matrix are the principal axes of an ellipsoid, at least when the matrix equation is a constant (contour plot). the length along each vector is related to the corresponding eigenvalue
• add something in a math section somewhere about general ellipsoids and principal axis theorem? https://www.maa.org/external_archive/joma/Volume8/Kalman/General.html