Beginner’s Guide on AI / ML — Series — Part 6

Linear Regression Model

Understanding AI Models starting from Middle school Math

Introduction

In our previous blogs ( 5 full blogs 😬), we’ve dived into the basics of AI and ML. We’ve busted the myth that AI is all about robot butlers and sci-fi movies (shoutout to Rajni fans — Chitti anyone?).

We learned that AI and ML aren’t the same thing; they’re two distinct concepts that often work together. Then we ventured into the world of supervised learning, where the model learns from labeled data, kind of like a student getting a homework assignment with all the answers in the back of the book.

Then, the writer got a bit too excited after hearing the latest Nobel Prize news and wrote a blog on that (oops, diversion much?). Then she thought “wait a minute, I was trying to follow a path here.” So she circled back to where we left off and dove into unsupervised learning.

Got lost anywhere in between? Dont worry. Your own Dora the Explorer Map is attached below… Press any of the links to read the blogs !

The Map

1. What on earth is AI -ML

2. Wait, ML and AI are not the same? No, they are not

3. Supervised Learning

4. Unsupervised Learning and Clustering

5. Nobel Prize in Physics 2024 to AI researchers?

(The list just keeps getting bigger and bigger doesn’t it? Makes this writer happy…)

In this blog, we are going to start understanding some actual algorithms and use some very “complex and alien math” (You will see how complex in a few minutes 😂)

We will learn about Linear Regression Model and about the Cost function in this and the next blog (yes, easter eggs left to be collected)

Putting the M in Maths

Okay, let’s start with basic math.

What is a linear equation?

A linear equation is an algebraic equation in which the highest power of the variable(s) is 1. It represents a straight line when plotted on a graph, hence the term “linear.”

Now, let’s bring back your old middle-school memories -

A person buys 7 bananas, and it totally costs 49 rupees. He bought 9 bananas and it cost 63 ruppess. What is the cost of 50 bananas?

Very simple question right?

(Yes, back to the random person who buys 50 bananas. )

source : reddit

Lets take this math problem and break it down into smaller parts. This will help us to understand complex ML algorithms! (yes, middle school math is going to help us build machines which might conquer this world…All the middle schoolers out there , take your math seriously 🤣)

source : canva

You can assume the variable we want to find (the price) as y (the dependent variable) and the number of bananas as x (independent variable).

y = ax+b , where a and b are unknown constants which we need to solve to get the equation which will help us predict the cost of 50 bananas.

Using math (and the last surviving brain cell) we find a = 7 and b = 0.

Our task now, is to develop such a machine which can solve for ‘a’ and ‘b’ on its own when the above problem is given… This is where the linear regression model kicks in.

The goal of linear regression is to establish a mathematical relationship between the independent variables and the dependent variable, such that the dependent variable’s value can be predicted or estimated given new values of the independent variables.

Now some terminologies to clear up (don’t want to make all the blogs to sound like greek and latin do we?)

Terminologies

1. Training Set — Dataset used to train the model. It contains both the input features and the corresponding output labels (for supervised learning), or just input features (for unsupervised learning). The training set is used by the model to learn the patterns, relationships, or structures within the data in order to make predictions or decisions.
2. x (lower case x) — input variable / feature
3. y (lower case y) — output variable/target variable.
4. m — total number of training example
5. (x,y) — indicates a single training example. A particular x gives y ( 7 bananas — 49 rupees)

Supervised Learning

• In supervised learning , the training set contains both features x and targets y. This is given to the learning algorithm. (Read the blog “Supervised Learning” linked above in the intro if you feel slightly lost!)
• There are two ways of solving the above problem, using simple algebra (transposing numbers) or hit and trial method
• Hit and trial method includes putting random values of a and b and checking its accuracy. We repeat the process till we get the right answer. Our machine will use a similar method.
• In the end ,when we give a particular x into the function, it can predict the particular outcome y.

Here we will work out linear regression in one variable. This means a single feature is passed into the function. Also called an univariate linear regression, this is the simplest form of linear regression and helps us understand how a change in one feature affects the outcome.

In univariate linear regression, we are trying to fit a straight line to the data points. The equation of this straight line is:

y=wx+b

(seems familiar huh?)

We collect data consisting of pairs of values (x,y), where x is the feature, y is the corresponding target value, w and b are the weights or the coeffecients. ( read the terminologies given above again if you didn’t understand what these terms mean!) This is what we are going to train outrmachine to predict accurately.

Cost Function

So, when we try to solve a problem, how do we verify if the solution we came up with is right? We take our solution to the problem and send in a test set of values, a set of values in which we know both x and y . If the y we get through our solution matches the actual y, then our answer is right.. If not go again and try again!

This is exactly what our ML model also does. The cost function tells us how well the model is doing and helps it keep getting better.

y = wx + b ( where y is the predicted value through the model)

Our aim here is to find w and b such that y* and y (actual target) is close to eachother.

Cost function takes y* and compares it with y. Error in any given function can be given by *(y- y)** . This tells us how far the prediction is far off from the target.

But wait! This error formula gives an issue.

Let us take this pretty graph.

Let’s calculate the error ( from left to right)

a) Actual point — 4, Point predicted by graph — 3 so error = +1

b) Actual point — 5.2 , Point predicted by graph — 3. 2 (ish) so error = +2

c) Actual point — 3, Point predicted by graph — 4 so error = -1

d) Actual point — 4 , Point predicted by graph — 6 so error = -2

total when added = 0

See what is happening here? You will notice as the signs are being taken into consideration, the summation of errors are cancelling eachother out. This would be misleading as the graph does contain errors but the summation is zero.

How do we deal with this?

This we shall see in the next blog 😎

Stick around, cause if you are thinking modulus is the way to go , then you are wrong!

Conclusion

In this blog, we introduced linear regression, a fundamental statistical method used for predicting a dependent variable based on one or more independent variables. We covered key terminologies ,explaining how they represent the relationship between variables. Additionally, we explored the cost function, which quantifies the error between predicted and actual values. We discussed how the cost function helps optimize the model by minimizing this error. In the end, the writer did something very unfair and left the answer hanging. 😬

Thanks for reading this article 🤩

If you found this article useful — Give it all your 50 Claps👏 to show your encouragement, and help me keep Motivated everyday to write more!

Feel free to follow for more insights.

Let’s stay in touch on LinkedIn — ❤️to keep the conversation going!

See you again next time, have a great day ahead

Keep your eyes peeled for more blogs by MP.T, signing off (cue dramatic bow)🤗