In Simple Linear Regression, the method of _________ is often used to estimate the coefficients.
- Clustering
- Gradient Descent
- Least Squares
- Neural Networks
The method of least squares is commonly used in Simple Linear Regression to estimate the coefficients by minimizing the sum of squared errors.
Dimensionality reduction can help in mitigating the problem of ___________, which refers to the difficulties of analyzing data in a high-dimensional space.
- multicollinearity
- overfitting
- scaling problems
- the curse of dimensionality
The term "curse of dimensionality" refers to the difficulties that arise when analyzing and organizing data in high-dimensional spaces. Dimensionality reduction can mitigate this problem by reducing the number of dimensions, making the data more manageable.
In a situation where you have limited data, how would you decide between using Cross-Validation or Bootstrapping, and why?
- Always use Bootstrapping
- Always use Cross-Validation
- Choose based on computational resources
- Choose based on the model, the nature of the data, and the analysis objectives
Deciding between Cross-Validation and Bootstrapping when dealing with limited data depends on the model, the nature of the data, and the analysis objectives. Cross-Validation provides robust validation by utilizing all data for both training and validation, while Bootstrapping can offer statistical insights. The decision should be tailored to the specific scenario.
In LDA, the goal is to maximize the ___________ variance and minimize the ___________ variance.
- between-class, within-class
- data, features
- features, data
- within-class, between-class
In LDA, the goal is to "maximize the between-class variance and minimize the within-class variance" to find a decision boundary that separates classes.
How can interaction effects be included in a Multiple Linear Regression model?
- By creating new variables for interactions
- By increasing model complexity
- By reducing variables
- By using more data
Interaction effects can be included by creating new variables that represent the product of two interacting variables, allowing for combined effects to be modeled.
Explain how the Odds Ratio is interpreted in Logistic Regression.
- As a clustering metric
- As a measure of feature importance
- As a measure that quantifies the effect of a one-unit increase in a predictor on the odds of the response
- As a probability measure
The Odds Ratio in Logistic Regression quantifies the effect of a one-unit increase in a predictor variable on the odds of the response variable. An Odds Ratio greater than 1 indicates an increase in the odds, and less than 1 indicates a decrease.
How does Cross-Validation help in reducing overfitting?
- By adding noise to the data
- By allowing a more robust estimate of model performance
- By increasing the dataset size
- By regularizing the loss function
Cross-Validation reduces overfitting by allowing for a more robust estimate of the model's performance. By using different splits of the data, it ensures that the model's validation is not overly reliant on a specific subset, helping to detect if the model is overfitting to the training data.
The Logit function can be defined as the logarithm of the __________ of the probability of the event occurring.
- Difference
- Odds
- Product
- Sum
The Logit function is defined as the logarithm of the odds of the probability of the event occurring.
What is the mathematical criterion that K-Means attempts to minimize, and how does it relate to centroid initialization?
- Maximizing centroid distances to data points
- Maximizing inter-cluster distance
- Minimizing the number of clusters
- Minimizing the sum of squared distances to centroids
K-Means minimizes the sum of squared distances from each point to its assigned centroid. Centroid initialization affects how quickly this criterion is minimized and the quality of the final clusters.
If two attributes in a Decision Tree have the same entropy, the attribute with the __________ Gini Index would generally be preferred.
- Equal
- Higher
- Lower
- Random
If two attributes in a Decision Tree have the same entropy, the attribute with the lower Gini Index would generally be preferred. A lower Gini Index indicates a purer node and would typically result in a better split.