1. Bayesian methods can be used to approximate the probability of B given A (P(B|A)) by treating it as a function (f(A;w)) of the exemplars A and weights w, based on training data of examples of A and B.
2. A Bayesian approach involves placing prior probabilities on the model parameters (weights) to avoid overfitting, and computing the posterior distribution over the weights given the data.
3. Key aspects of Bayesian methods include choosing a model shape (e.g. linear or sigmoid function), placing independent precision tuners on the weights and basis functions to control complexity, and interpreting the confidence levels of the unknown weights.
1 of 13
Download to read offline
More Related Content
Bayesian His2011
1. Bayesian Methods in Machine
Learning
Mohammad Shazri
mshazri@gmail.com
shazri.shahrir@extolcorp.com
2. Sources
Papers from Michael Tipping
-Bayesian Inference: An introduction to Principles in Machine Learning
Youtube from mathematicalmonk
3. approximate P(B|A)
P(B|A)=f(A;w) .. A is the exemplars and w is
the weights.|given A, what is B?.
B and A usually relates as D= {A_n,B_n} from
n=1 to N.
Prior statements, possible to over-fit.
5. Setup and Model
Setup; it is a supervised learning situation
(1) Models/Basis* shape are normally
distributed
(2) confidence weights/tuneVar are
independent.
(3) there are precision tuners on
basis+weights. Var and alpha
(4) all are known except weights.
6. Usual vs MAP
Usual ; over-fitting
MAP ; solves over-fitting but there is no
interpretation of BR.
______________________________________
Confidence-Level/Interpretation of the
unknown
7. Complexity Control : Regularization
E(w)= E_d(w)+lambda*E_w(w)
E_d is the standard regression model, liner,
sigmoid etc.
E_w is the penalty
Lamda is a hyperparameter