
Learn three guidelines to succeed: ask questions via the Q&A, meet prerequisites, and engage by taking handwritten notes for conceptual lectures and coding along with coding exercises.
Perform a handwritten derivation of the linear regression estimator w_hat for y = w x + epsilon, and show its mean is w with variance sigma^2 over sum x_i^2.
Explore how least squares equals maximum likelihood under normal errors, and how Bayesian map estimation with a normal prior yields shrinkage and a bias-variance trade-off.
Provide detailed feedback through the suggestion box form, sharing your background, course, difficulty, missing topics, and concrete examples to guide future course improvements.
Derive the posterior predictive distribution for multivariate Bayesian linear regression, yielding a normal with mean mu_w dot x_hat and variance sigma^2 plus x_hat transpose sigma_w x_hat.
Prepare code for bayesian linear regression on an advertising dataset and learn to estimate unknown variance with least squares, use the precision matrix, and avoid inversion.
Learn how to succeed in this course by asking questions in the q&a, verifying prerequisites, and implementing theory in code to connect background knowledge with practice.
Note that the Python notebooks aren’t on GitHub, log in, and use the official code link; turn off VPN to receive verification emails and resolve access issues.
Welcome to Bayesian Linear Regression!
I first started this course series on Bayesian Machine Learning many years ago, with a course on A/B Testing. I had always intended to expand the series (there's a lot to cover!) but kept getting pulled in other directions.
Today, I am happy to announce that the Bayesian Machine Learning series is finally back on track!
In the first course, a lot of students asked, "but where is the 'machine learning'?", since they thought of machine learning from the typical supervised/unsupervised parametric model paradigm. The A/B Testing course was never meant to look at such models, but that is exactly what this course is for.
If you've studied machine learning before, then you know that linear regression is the first model everyone learns about. We will approach Bayesian Machine Learning the same way.
Bayesian Linear Regression has many nice properties (easy transition from non-Bayesian Linear Regression, closed-form solutions, etc.). It is best and most efficient "first step" into the world of Bayesian Machine Learning.
Also, let's not forget that Linear Regression (including the Bayesian variety) is simply very practical in the real-world. Bayesian Machine Learning can get very mathematical, so it's easy to lose sight of the big picture - the real-world applications. By exposing yourself to Bayesian ideas slowly, you won't be overwhelmed by the math. You'll always keep the application in mind.
It should be stated however: Bayesian Machine Learning really is very mathematical. If you're looking for a scikit-learn-like experience, Bayesian Machine Learning is definitely too high-level for you. Most of the "work" involves algebraic manipulation. At the same time, if you can tough it out to the end, you will find the results really satisfying, and you will be awed by its elegance.
Sidenote: If you made it through my Linear Regression and A/B Testing courses, then you'll do just fine.
Suggested Prerequisites:
Python coding: if/else, loops, lists, dicts, sets
Numpy and Pandas coding: matrix and vector operations, loading a CSV file
Basic math: calculus, linear algebra, probability
Linear regression
Bayesian Machine Learning: A/B Testing in Python (know about conjugate priors)