You need to improve the performance of a weak learner. Which boosting algorithm would you select, and why?
- AdaBoost
- Any boosting algorithm will suffice
- Gradient Boosting without considering the loss function
- Random Boosting
AdaBoost is a boosting algorithm designed to improve the performance of weak learners. By adjusting the weights of misclassified instances and focusing on them in subsequent models, AdaBoost iteratively corrects errors and enhances the overall model's performance.
What is the relationship between dimensionality reduction and the curse of dimensionality?
- Depends on the method
- Has no relationship
- Helps in mitigating the curse
- Intensifies the curse
The relationship between dimensionality reduction and the curse of dimensionality is that dimensionality reduction helps in mitigating the curse. The curse of dimensionality refers to the challenges that arise as the number of dimensions increases, and reducing dimensions can alleviate these problems.
__________ is a boosting algorithm that builds trees one at a time, where each tree corrects the errors of the previous one.
- AdaBoost
- Bagging
- Gradient Boosting
- Random Forest
AdaBoost is a boosting algorithm that builds trees sequentially, with each tree correcting the errors of the previous one by giving more weight to the misclassified instances.
What are the underlying assumptions of LDA, and how do they affect the performance?
- Assumes different covariance matrices, normal distribution; affects adaptability
- Assumes equal class sizes; affects bias
- Assumes equal variance, non-normal distribution; affects robustness
- Assumes normal distribution, equal covariance matrices; affects classification accuracy
LDA assumes that the features are normally distributed and that the classes have equal covariance matrices. These assumptions, if met, lead to better "classification accuracy," but if violated, may lead to suboptimal performance.
How is the Logit function related to Logistic Regression?
- It is a type of cost function
- It is an alternative name for Logistic Regression
- It's the inverse of the Sigmoid function and maps probabilities to log-odds
- It's used for multi-class classification
In Logistic Regression, the Logit function is the inverse of the Sigmoid function. It maps probabilities to log-odds and forms the link function in logistic modeling.
One method to mitigate multicollinearity is to apply ___________ regression, which adds a penalty term to the loss function.
- Lasso
- Logistic
- Polynomial
- Ridge
Ridge regression is a technique that can mitigate multicollinearity by adding a penalty term to the loss function. The penalty term helps in reducing the effect of correlated variables, leading to more stable coefficients.
In what situations would it be appropriate to use Logistic Regression with the Logit link function?
- All regression problems
- Binary classification with a nonlinear relationship between predictors
- Binary classification with linear relationship between predictors
- Multi-class classification
Logistic Regression with the Logit link function is particularly suited for binary classification problems where there is a linear relationship between the predictors and the log-odds of the response.
You have two models with similar Accuracy but different Precision and Recall values. How would you decide which model is better for a given application?
- Choose based on the specific application's needs and tolerance for false positives/negatives
- Choose the one with higher Precision
- Choose the one with higher Recall
- nan
When models have similar Accuracy but different Precision and Recall, the choice between them should be based on the specific application's needs. If false positives are more costly, prioritize Precision; if false negatives are more crucial, prioritize Recall.
If the relationship between variables in a dataset is best fit by a curve rather than a line, you might use _________ regression.
- Linear
- Logistic
- Polynomial
- Ridge
If the relationship between variables is best fit by a curve rather than a line, Polynomial regression would be used. It can model nonlinear relationships by including polynomial terms in the equation.
What is dimensionality reduction, and why is it used in machine learning?
- All of the above
- Increasing model accuracy
- Reducing computational complexity
- Reducing number of dimensions
Dimensionality reduction refers to the process of reducing the number of input variables or dimensions in a dataset. It is used to simplify the model and reduce computational complexity, potentially improving model interpretability, but it does not inherently increase model accuracy.
You're comparing two Polynomial Regression models: one with a low degree and one with a high degree. The higher degree model fits the training data perfectly but has poor test performance. How do you interpret this, and what actions would you take?
- Choose the high degree model
- Choose the low degree model or consider regularization
- Ignore test performance
- Increase the degree further
The high degree model is likely overfitting the training data, leading to poor test performance. Choosing the low degree model or applying regularization to the high degree model can improve generalization.
Describe the process of Bootstrapping and its applications in model evaluation.
- Repeated sampling with replacement for bias reduction
- Repeated sampling with replacement for variance reduction
- Repeated sampling with replacement to estimate statistics and evaluate models
- Repeated sampling without replacement for model validation
Bootstrapping involves repeated sampling with replacement to estimate statistics and evaluate models. By creating numerous "bootstrap samples," it allows the calculation of standard errors, confidence intervals, and other statistical properties, even with a small dataset. It's valuable for model evaluation, hypothesis testing, and providing insight into the estimator's distribution.