In a scenario where dimensionality reduction is essential but preserving the original features' meaning is also crucial, how would you approach using PCA?
- You would avoid PCA and use another method
- You would carefully interpret the principal components in terms of original features
- You would perform PCA on a subset of the original features
- You would use PCA without considering the original features' meaning
In this scenario, careful interpretation of the principal components in terms of the original features would be the key to preserve their meaning while still benefiting from dimensionality reduction.
What are the challenges in imbalanced classification problems?
- Balanced data
- Equal representation of all classes
- No challenges
- Overfitting to the majority class
Imbalanced classification problems, where the classes are not equally represented, can lead to models that are biased towards the majority class. This can result in poor performance on the minority class, requiring special techniques to address.
What is underfitting, and how does it differ from overfitting?
- Enhancing model complexity; similar to overfitting
- Fitting the model too closely to the training data; same as overfitting
- Fitting the model too loosely to the training data; opposite of overfitting
- Reducing model complexity; similar to overfitting
Underfitting is when a model fits the training data too loosely and fails to capture the underlying pattern, the opposite of overfitting, where the model fits too closely.
In the context of Decision Trees, how can overfitting be controlled using pruning techniques?
- By increasing the number of features
- By increasing the tree complexity
- By reducing the training data
- By reducing the tree complexity
Overfitting in Decision Trees can be controlled using pruning techniques by reducing the tree's complexity. By removing branches that add little predictive power, the model becomes less sensitive to noise in the training data and generalizes better to unseen examples.
In classification, when a model is biased toward predicting one class over another, it is known as a(n) ________ problem.
- Clustering
- Imbalanced classification
- Multiclass classification
- Overfitting
When a model consistently predicts one class over another, particularly when the classes are not equally represented, this is known as an imbalanced classification problem.
The ________ measures the average of the squares of the errors, while the ________ takes the square root of that average in regression analysis.
- MAE, MSE
- MSE, RMSE
- R-Squared, MAE
- RMSE, MAE
The Mean Squared Error (MSE) calculates the average of the squared differences between predicted and actual values, and the Root Mean Squared Error (RMSE) takes the square root of that average. RMSE gives more weight to large errors and is more interpretable as it is in the same unit as the response variable.
You are working with a large dataset, and you want to reduce its dimensionality using PCA. How would you decide the number of principal components to retain, considering the amount of variance explained?
- By always retaining all principal components
- By always selecting the first two components
- By consulting with domain experts
- By retaining components explaining at least a predetermined threshold of variance
The number of principal components to retain can be decided based on a predetermined threshold of variance explained. For example, you may choose to keep components that together explain at least 95% of the total variance.
How does Lasso regression differ from Ridge regression?
- Both use L1 regularization
- Both use L2 regularization
- Lasso uses L1 regularization, Ridge uses L2
- Lasso uses L2 regularization, Ridge uses L1
Lasso (Least Absolute Shrinkage and Selection Operator) regression uses L1 regularization, which can lead to some coefficients being exactly zero, thus performing feature selection. Ridge regression uses L2 regularization, which shrinks the coefficients but doesn't set them to zero. These different regularization techniques define their behavior and application.
The ___________ matrix in LDA represents the spread of the individual classes around their mean.
- between-class scatter
- correlation
- covariance
- within-class scatter
The "within-class scatter" matrix in LDA represents the spread of the individual classes around their mean, capturing the within-class variance.
Explain how cross-validation can be used to mitigate the risk of overfitting.
- By adding noise to the training data
- By increasing model complexity
- By reducing model complexity
- By splitting the data into multiple subsets and training on different combinations
Cross-validation mitigates the risk of overfitting "by splitting the data into multiple subsets and training on different combinations." It ensures that the model is evaluated on unseen data and helps in tuning hyperparameters without relying on the test set.