Introduction to Machine Learning

Neural Networks for CV and NLP — EP01

The entire discovery in the field of vision and neural networks begin with “Hubel & Wiesel’s Experiment on the Visual Cortex (1959)”.

The scientists recorded the electrical activity of an individual neuron by inserting microelectrodes into the primary visual cortex (V1), by projecting patterns of light and dark.

Hubel & Wiesel’s Experiment on the Visual Cortex (1959); Nobel Prize in 1981 (Nobel Prize in Physiology)

No spikes have been observed for the images, but spikes are seen during the optical slides.

Machine Learning — Field of study that gives computers the ability to learn without being explicitly programmed (by Arthur Samuel in 1959)

Applications of ML

Types of Machine Learning: -

1. Supervised Learning (Regression, Classification, etc.)
2. Unsupervised Learning (Fully Un-supervised, Semi Supervised, Self Supervised, etc.)
3. Reinforcement Learning

SUPERVISED LEARNING: model is trained on labeled dataset allowing model to learn the mapping from i/p to o/p.

(i) Regression — predicts continuous numerical values like temperature or price. Algorithms: Linear Regression, Decision Trees, Random Forests, Gradient Boosting.

(ii) Classification — predicts discrete categories or classes like yes/no, spam/not spam or cat/dog. Algorithms: Logistic Regression, Decision Trees, Random Forests, SVM, KNN, Naive Bayes.

Application of SL

UN-SUPERVISED LEARNING: class labels of the data are not given or unknown

(i) Clustering — it groups similar data poiints into clusters Algorithms: K-Means, Hierarchical Clustering, DBSCAN, Spectral Clustering.

(ii) Association Rule Learning — finds relationship between variables in datasets such as “if-then” rules. Algorithms: Logistic Regression, Decision Trees, Random Forests, SVM, KNN, Naive Bayes.

(iii) Dimensionality Reduction — reduces dataset features while retaining essential information. Algorithms: Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Non-negative Matrix Factorization (NMF), Isometric Mapping, Locally Linear Embedding (LLE).

Applications of USL

SELF — SUPERVISED LEARNING: the model is trained on a dataset without explicit human-annotated labels. Here the supervisory signals are generated from structure/inherent props of data.

Self-Supervised Learning (SSL) — GeeksforGeeks

It has three steps:

1. Pretext task — a problem is set up by system with known answer or answer derivable from data. (e.g., image colorization, predicting next word in sentence).
2. Representation Learning — the model learns useful features/representation from previous step.
3. Fine-tuning — the pre-trained model is fine-tuned on a dataset for specific application and problems (downstream tasks).

e.g., when a model is trained to color grayscale images it learns what, where, shapes and boundaries of those objects which helps in segmentation directly.

REINFORCEMENT LEARNING: an agent learns to make decisions by interacting with an environment in order to maximize the total or cumulative reward over time.

Reinforcement learning — Wikipedia

Agent: the decision-make (e.g., robot, program, model) Environment: the world the agent interacts with State (s): a representation of the current situation Action (a): **a choice the agent can make in a state Reward (r):* feedback signal (+ve/-ve) given after an action*

NOW LET’s DISCUSS SUPERVISED LEARNING IN MORE DETAIL… 🙂

Getting to know common terms and concepts :-

There are multiple scenarios for the label space:

If we take a training dataset (x,y) then x is element of R^d (d-dimensional feature space) which is the input instance & y its label.

A training set is the dataset for training/teaching the ML model by finding optimal/best possible parameters.

OCKHAM’s RAZOR — prefer the simplest hypothesis consistent with data

Why it matters in Supervised Learning?

In SL, we often face a choice between simple and complex models (such as linear regression vs deep neural network). A complex model can fit the training data extremely well, but it may also overfit — memorizing noise instead of learning general patterns. Ockham’s razor reminds us to prefer simpler models resulting in improved performance on unseen data.

In simple words it tells don’t add complexity unless it’s necessary to explain data.

HYPOTHESIS FUNCTION (NPTEL_NeuralNetworkForCVandNLP)

Why squared errors?

Using squared errors makes the math smooth, so the model can learn easily. Bigger mistakes get punished more strongly than small ones. The function is shaped like a bowl, so it’s simple to find the lowest point (best solution).

LOGISTIC REGRESSION — in classification the output ranges from 0 to 1.

In logistic regression, the probability of the occurrence of an event (being apple) is estimated.

LOSS FUNCTION FOR LOGISTIC REGRESSION

The loss function measures how far the prediction is from the true label.

If the true label y=1: Loss = −log⁡(hϕ(x)) Prediction close to 1 → loss is small (good). Prediction close to 0 → loss is very large (bad).

If the true label y=0: Loss = −log⁡(1−hϕ(x)) Prediction close to 0 → loss is small (good). Prediction close to 1 → loss is very large (bad).

Gradient Descent for Logistic Regression: Gradient Descent in Logistic Regression — Learn in Minutes!

**MULTI CLASS CLASSIFICATION**

Multivariate Linear Regression

STOCHASTIC GRADIENT DESCENT vs TYPICAL GRADIENT DESCENT

In SGD, we find out the gradient of the cost function of a single example at each iteration instead of the sum of the gradient of the cost function of all examples. Less expensive than normal GD.

Since only one sample from the dataset is chosen at random for each iteration, the path taken by the algorithm to reach the minima is usually noisier than your typical GD algorithm.

But that doesn’t matter as the path taken by the algo doesn’t matter, as long as we reach the minima and with significantly shorter training time.

PATH OF CONVERGENCE FOR 3 TYPES OF GRADIENT DESCENT