Support Vector Machine

Optimal hyperplane for linearly separable patterns

Extend to patterns that are not linearly separable by transformations of original data to map into new space using the Kernel function

Support vector

data points that are closest to data point is called support vector it is crucial for deciding the support vector

Hard margin and Soft margin

Hard margin SVMs require a perfect separation of data points with a clear margin, while soft margin SVMs allow for some misclassifications or margin violations, making them more flexible and robust for real-world datasets.

Logistic loss and hinge loss

Logistic loss:

Hinge loss:

🔍 What It Measures:

Hinge loss measures how far a data point is from being confidently correct.

• ✅ Correct and Confident: If the model predicts the right class with a margin ≥ 1, loss = 0.
• ⚠️ Correct but Weak: If it predicts correctly but the confidence (margin) is < 1, there is some loss.
• ❌ Wrong Prediction: If the prediction is wrong, the loss increases linearly with how wrong it is.

Mathematic behind SVM

Assuming all data is at distance larger than 1 from the hyperplane, the following two constraints follow for a training set {(xi ,yi )}

• For support vectors, the inequality becomes an equality; then, since each example’s distance from the

Margin: the distace between the hyperplane which we need to maximize

d=2/||w||

so we need to maximize margin so we have to minimize ||w|| in order to do this we use lagrange method and try to minimize ||w||^2 and then

FURTHER LAGRANGE USED WHICH I DON’T KNOW FOR NOW