What is overfitting in the context of machine learning?
- Enhancing generalization
- Fitting the model too closely to the training data
- Fitting the model too loosely to the data
- Reducing model complexity
Overfitting occurs when a model fits the training data too closely, capturing the noise and outliers, making it perform poorly on unseen data.
How would you approach the problem of data leakage during the preprocessing and modeling phase of a Machine Learning project?
- Ignore the problem as it has no impact
- Mix the test and training data for preprocessing
- Split the data before any preprocessing and carefully handle information from the validation/test sets
- Use the same preprocessing techniques on all data regardless of splitting
To prevent data leakage, it's crucial to split the data before any preprocessing, ensuring that information from the validation or test sets doesn't influence the training process. This helps maintain the integrity of the evaluation.
In a multiclass classification problem with imbalanced classes, how would you ensure that your model is not biased towards the majority class?
- Implement resampling techniques and consider using balanced algorithms
- Increase the number of features
- Use only majority class for training
- Use the same algorithm for all classes
Implementing resampling techniques to balance the classes and considering algorithms that handle class imbalance can ensure that the model doesn't become biased towards the majority class.
_________ is a metric that considers both the ability of the classifier to correctly identify positive cases and the ability to correctly identify negative cases.
- AUC
- F1-Score
- Precision
- nan
AUC (Area Under the Curve) considers both the ability of the classifier to identify positive cases (sensitivity) and the ability to identify negative cases (specificity) at various thresholds, providing a comprehensive view.
Imagine you have a dataset where the relationship between the variables is cubic. What type of regression would be appropriate, and why?
- Linear Regression
- Logistic Regression
- Polynomial Regression of degree 3
- Ridge Regression
Since the relationship between the variables is cubic, a Polynomial Regression of degree 3 would be the best fit. It will model the cubic relationship effectively, whereas other types of regression would not capture the cubic nature of the relationship.
How do pruning techniques affect a Decision Tree?
- Decrease Accuracy
- Increase Complexity
- Increase Size
- Reduce Overfitting
Pruning techniques remove branches from the tree to simplify the model and reduce overfitting.
What role does the distance metric play in the K-Nearest Neighbors (KNN) algorithm?
- Assigns classes
- Defines decision boundaries
- Determines clustering
- Measures similarity between points
The distance metric in KNN is used to measure the similarity between points and determine the nearest neighbors.
In a case where both overfitting and underfitting are concerns depending on the chosen algorithm, how would you systematically approach model selection and tuning?
- Increase model complexity
- Reduce model complexity
- Use L1 regularization
- Use grid search with cross-validation
Systematic approach involves the use of techniques like grid search with cross-validation to explore different hyperparameters and model complexities. This ensures that the selected model neither overfits nor underfits the data and generalizes well to unseen data.
You have built a Logistic Regression model, but the link test indicates that the Logit link function may not be appropriate. What could be the issue?
- Incorrect loss function
- Multicollinearity
- Non-linearity between predictors and log-odds
- Overfitting
If the Logit link function is not appropriate, it might indicate that there is a non-linear relationship between the predictors and the log-odds of the response, violating the assumptions of Logistic Regression.
You notice that your KNN model is highly sensitive to outliers. What might be causing this, and how could the choice of K and distance metric help in alleviating this issue?
- Choose a larger K and an appropriate distance metric to mitigate sensitivity
- Choose a small K and ignore outliers
- Focus only on the majority class
- Outliers have no effect
Choosing a larger K and an appropriate distance metric can help mitigate the sensitivity to outliers, as it would reduce the influence of individual data points.