All you need to know about Statistics Fundamental before start learning Machine Learning and AI

Before start learning Data Science, you should learn Statistics because it is the fundamental and here I summarize it for you.

Data

• Data are the output of measurement devices

Data Types

Sample vs Population

Measures of Central Tendency

• Mean: arithmetic mean, average of data
• Median: middle of data
• Mode: most common value

Measures of Dispursion

• Variance: how far each number in the set is from the mean
• Standard deviation: how far individual points in a dataset are dispersed from the mean of that set.

Why Is Standard Deviation Often Used More Than Variance?

Standard deviation is the square root of variance. It is sometimes more useful since taking the square root removes the units from the analysis. This allows for direct comparisons between different things that may have different units or different magnitudes. For instance, to say that increasing X by one unit increases Y by two standard deviations allows you to understand the relationship between X and Y regardless of what units they are expressed in.

Probability

• Probability is a numerical description of how likely an event is to occur or how likely it is that a proposition is true. Probability is a number between 0 and 1, where, roughly speaking, 0 indicates impossibility and 1 indicates certainty.
• The sum of all probabilities in a set must sum to 1

Sources of sampling variability

• Natural variation: Often seen in biology (e.g. height, weight) and physics (e.g. earthquake magnitude, number of stars per galaxy)
• Measurement noise: The sensors are imperfect (e.g. electrical line noise, measuring in μg with a gram-precision scale)
• Complex systems: Measuring some factors while ignoring others (e.g. measuring height while ignoring age)
• Stochasticity (randomness): The universe is a wild and unpredictable place (e.g. photons hitting a camera lens)

Hypothesis Testing

IVs and DVs

• DV: Dependent variable (outcome variable): The variable you are trying to explain
• IV: Independent variable (explanatory variables): The variables that you hope will explain the DV

Example:

• Effects of soil moisture on plant growth
• Effects of time spent on Facebook on irritability
• Relation between money spent on clothes and bar-hopping

Model-fitting in Statistics

A model is an equation that explains some features in a dataset

Residuals should be small, but models should be simple.

A model should be as simple as it can be, and as complicated as it must be.

What is an hypothesis?

A falsifiable claim that requires verification, typically from experimental or observational data, and that allows for predictions about future observations.

• null hypothesis: is the hypothesis that nothing interesting is happening in the data. In statistical analysis, you test the null hypothesis.
• In the research, you specify the “alternative hypothesis” (should be called the “effect hypothesis”

Null hypothesis example

Hₐ: People will buy more widgets after seeing advertisement X compared to advertisement Y

H₀: The advertisement type has no effect on widget purchases

P-Value

• P-value are probabilities. They range from 0 to 1
• Values closer to zero indicate low probability of Hₐ|H₀
• Values closer to one indicate high probability of Hₐ|H₀
• A finding is called “statistically significant” if the test statistic is greater than a threshold. That is, if p(Hₐ) < p(α)
• Threshold is arbitrary; common values are p < 0.05 or p < 0.01

Outlier

Other name: Anomaly, Extreme (deviant) data, Non-representative data, Noise (Why are they dangerous?)

• Not all outliers are equal. Outliers are worse near the “edges” of the data distribution compared to the “middle”

Where do outliers come from?

• Noisy data
• Noisy or faulty equipment
• Human error (e.g. typo)
• Non-cooperative research participant
• Natural variation

How to deal with outliers?

• Startegy 1: Identify outliers and remove them from the data prior to any analyses, Assumption: Outliers are noise or otherwise invalid
• Startegy 2:Leave outliers in and use robust methods that attenuate the negative impact of the outliers on the results, Assumption: Outliers are unusual but valid data

Common Statistical tests

T-test Formula

• tₖ = Difference of means / Standard deviations

T-Test Family

• One-sample, Two dependent (paired) samples. Two independent samples
• Wilcoxon signed-rank test, Signed-rank test
• Mann-Whiteny U test, Mann-Wihitney-Wilcoxon U test, Wilcoxon rank-sum test
• Permutation t-test

Correlation

• The correlation coefficient varies between -1 and +1. -1 means a perfect inverse relationship, 0 means no relationship, and +1 means a perfect positive relationship
• The correlation coefficient isself is a continuous measure of correlation strength. A corresponding p-value must be computed to interpret its statistical significance

ANOVA

• ANalysis Of VAriance
• The goal of an ANOVA is to determine the effects of several categorical independent variables on one numerical dependent variable

ANOVA “ways”

• <number>-way: The number of factors

Examples:

• One-way ANOVA: Determine the influence of day-of-week on iPhone purchases
• Two-ways ANOVA: Determine the influences of day-of-week and gender (male, female) on iPhone purchases

Repeated-measure ANOVA

• rmANOVA: If at least one factor involves multiple measurements from the same individual

Example:

• Research question: Determine effects of snack type on mood
• Expreiment: Volunteers eat chocolate for 2 days, potato chips for 2 days, and ice cream for 2 days (order randomized)
• Factor: Snack type (levels: chocolate, chips, ice cream)

ANONVA F test as a ratio of variances

F = “Explained” variance / “Unexplained” variance

F = Due to factors / Natural variation

• Correct interpretaion of p < 0.05: At least one level (group) is statistically significantly different from at least one other level.
• Determining which groups differ requires data visulization follow-up t-tests

Example:

• Two-way ANOVA: Determine the influences of day-of-week and gender (male, female) on iPhone purchases

Regression

ANOVA vs Regression

• Use an ANOVA when IVs are categorical
• Use a Regression when IVs are continuous

Example of regression

• Question: Can we predict how tall a person is?

h = β₀ + β₁s + β₂p + β₃n

h = height

β₀ = intercept (average height when all other parameters are 0)

s = sex (m/f), dummy-coded

p = parents’ height

n = childhood nutrition

• Reasearch Question: Do sutdent’s grade on a stats exam (y) depend on sleep (s), hours studied (h), and the interaction between them?

y = β₀ + β₁s + β₂h + β₃(sxh) + ∊