The Bayesian Maze: Unlocking the Power of Hierarchical Bayesian Regression (Part 1)

Navigating Multi-Level Regression in Market Mix Modeling

Market mix modelling is the state-of-the-art approach to capture the influence of advertisements on sales, and numerically attribute their effects. Thus, helps businesses to analyze and optimize the spending on certain advertisements.

It is easy to understand what we do in Market Mix Modelling (MMM) but the complexity lies within the concept. Multi-level modelling is one solution to the problems of MMM. In this blog, I will explain Hierarchical Bayesian Regression as a powerful multi-level modelling approach.

Attributing the media based on a regression approach is a simple strategy in a pooled/un-pooled model, But to capture the mixed effect we need to create a mixed-effect capturing model.

Pooled (left) and Un-pooled (Right) Model intuition

Thus, in a single model itself we would be able to capture product level effects and the influence of brand level advertisements on the individual product sales. And this capability is provided by the complex yet interesting hierarchical structure of modelling (primarily where we have individual and group level relation in the products)

Mixed-effect model (Hierarchical Model)

Hierarchical Bayesian Regression

However complex the model may be, breaking it down is the key. Then, I’ll make it clear for all to see.

Let’s begin by deconstructing the term in reverse order, starting with “Regression” then we’ll explore how “Bayesian” fits into the picture, and finally, we’ll delve into the concept of “Hierarchy”.

For a linear/Non-linear model, the degrees of a polynomial fight to their maximum value to fit the curve.

Point Estimate Approach

A linear model of a certain polynomial degree will be able to capture the underlying structure of the data. To find the best optimal value of ω, such that the error between the estimated and actual point is minimum, thus we solve the below error function (eq 1.2), and analytics way to calculate the best value of coefficient (eq 1.3).

eq 1.2 — Sum squared error Function

from vector regression

eq 1.2 Analytical solution of ω in vector regression

Solving the above equation will help us generate a better fit on the curve and thus the regression model, however a higher degree polynomial will simultaneously cause an overfit issue, which needs to be regularized, and hence Ridge regression comes in as a solution which is a regularization term added to the error function (eq 1.4), handling the overfitting problem of the model.

eq 1.4 Sum Squared error function

The persistence of uncertainty in these weights arises from our lack of assurance regarding their certainty. What if we were to treat our targets as a distribution?

Probabilistic Approach

We are considering our target t as a Random variable (distribution of many probable values).

Certain question arises

what distribution does the target follow? → depends Do they follow the same distribution? → Often Do they have different parameters or the same parameters? → may or may not share

Assume: All random variables follow Gaussian Distribution, they are independent of each other and have the same variance.

fig 2.1 Target distribution Graph and corresponding probability distribution

It can clearly be read that our target is a random variable at each value of mu, Normally distributed (Fig 2.1). The question is what values of ω and precision β, will give the maximum probability of these target random variables.

As we know from joint probability → P(A ∩ B) = P(A) × P(B), where A and B are independent events, thus

eq. 2.1

This is what we call Likelihood (chance of something happening). But Products in Machine learning are often not appreciated, the reason being smaller value products make value tend to 0. Hence, we take the log on both sides(derivation), forming Log-Likelihood.

eq 2.2 — Log Likelihood

We see taking log naturally generated the equation in a similar format to the sum squared error function (eq 2.2), which means there is some connection between maximum likelihood and Regression. However, In Machine learning, generally, we go for the Minimization process, and hence we multiply both sides with a minus, calling it a Negative Likelihood Function (eq 2.3).

eq 2.3 — Negative likelihood Function

As we know from linear algebra, to find the minimum, we take the partial derivative. Therefore, to minimize the negative likelihood, we take the partial derivative of the negative likelihood concerning both parameters, ω and β. This process helps us calculate the optimal values for ω and β for the negative likelihood.

[embed]The Bayesian Maze : Unlocking the Power of Hierarchical Bayesian Regression (Part 2) Introduction of Bayes theorem in regressionprateeknigam9.medium.com

Conclusion

In our journey through Bayesian Regression and Market Mix Modelling, we went through the introduction of advertising impact on sales and how are they calculated. By breaking down regression and Bayesian concepts, we are uncovering the solution of Hierarchical Bayesian Regression to the complex challenges of Market Mix Modelling, yet we are left with several doubts, that I would resolve going further.

Unveil these answers and exciting details about regression and Bayes in Part 2. Explore the basics of priors, posteriors, and other concepts to reveal the wonders of hierarchical Bayesian regression (HBR) and predictive distributions.

Subscribe to my blogs to stay updated on new articles and exclusive content. Join me in exploring the full potential of HBR.