The Law of Total Probability states that the sum of the probabilities of all possible outcomes of an experiment is 1. This rule is fundamental to probability theory and provides a way to calculate the probability of complex events by breaking them down into simpler, mutually exclusive events.

Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) are the two types of factor analysis commonly used in data science. EFA is used when the structure of the underlying factors is not known, while CFA is used when the researcher has specific hypotheses about the factor structure.

The Central Limit Theorem is a fundamental concept in probability theory and statistics. The theorem states that, as the size of a sample is increased, the sampling distribution of the mean will be closer to a normal distribution. This happens no matter the shape of the population distribution.

Skewness affects the relationship between the mean, median, and mode. In a positively skewed distribution, the mean is usually greater than the median, which is greater than the mode. In a negatively skewed distribution, the mode is usually greater than the median, which is greater than the mean.

The Central Limit Theorem holds true when the sample size is sufficiently large (usually n > 30), regardless of the shape of the population distribution. This theorem states that if you have a population with mean μ and standard deviation σ and take sufficiently large random samples from the population with replacement, then the distribution of the sample means will be approximately normally distributed.

Effect size measures the magnitude of the difference or the strength of the relationship in the population. A larger effect size means a larger difference or stronger relationship, which in turn increases the statistical power of the test. Power is the probability that the test correctly rejects the null hypothesis when the alternative is true.

A binomial distribution is discrete, meaning it only takes on integer values on a countable range, and it represents the number of successes in a fixed number of independent Bernoulli trials with a given success probability. A normal distribution is continuous, and it is often used as a first approximation to the binomial distribution, when the number of trials is large.

The assumption of linearity in a multiple linear regression model assumes that the relationship between each independent variable and the dependent variable is linear. This implies that the change in the dependent variable due to a one-unit change in the independent variable is constant, regardless of the value of the independent variable.

Kurtosis refers to the sharpness of the peak of a frequency-distribution curve. It measures the tails and sharpness of the distribution. Distributions with large kurtosis exhibit tail data exceeding the tails of the normal distribution.

Factor analysis and PCA differ primarily in what they seek to model. Factor analysis models the shared variance among variables, focusing on the latent or unobservable variables, while PCA models the total variance and aims at reducing the dimensionality.