AdaBoost Explained Simply (Part 3: Using)

AdaBoost — Using

This section shows how to use the trees in AdaBoost to make the prediction for a new sample.

Let us say we have six trees and the classification results of a sample are:

Ultimately, the patient is classified as HAS HEART DISEASE, because the total amount 2.7 is the larger sum than 1.23.

Summary:

The ideas behind AdaBoost are summarised as follows:

1. AdaBoost combines a lot of weak learners to make classifications. In our examples, the weak learners are stumps. A stump is a decision tree which only has one split.

2. Some stumps get more say in the classification than others.

3. Each Stump is made by taking previous stump’s mistakes into account. In our example, if we have a weighted Gini Function, then we use it with the sample weights, otherwise, we can use the sample weights to create a new dataset that reflects those weights.

Next: Gradient Boost will be discussed.

This note summarises the Youtube Videos published by Josh Starmer (Youtube Account: StatQuest with Josh Starmer). I would like to say a big thank you to him and his super useful videos!