How are the coefficients of Simple Linear Regression estimated?
- By Maximizing the Variance
- By Minimizing the Sum of the Squares of the Residuals
- Through Classification
- Through Clustering
The coefficients in Simple Linear Regression are estimated by minimizing the sum of the squares of the residuals. This method ensures that the line fits as closely as possible to the observed data.
Ensemble methods like Random Forest and Gradient Boosting are considered powerful tools, but they can lead to __________ if not tuned properly.
- Both Underfitting and Overfitting
- Overfitting
- Underfitting
- nan
Ensemble methods like Random Forest and Gradient Boosting can lead to overfitting if not tuned properly, as they may become too complex and fit the noise in the training data instead of the underlying pattern.
You are given a dataset without clear instructions on what the targets are. How would you proceed to build a predictive model?
- Build a regression model directly
- Consult with domain experts or analyze the data for insights
- Guess the targets
- Ignore the data
Consulting with domain experts or analyzing the data through exploratory data analysis (EDA) can help identify potential targets and correlations within the data. This collaborative and investigative approach ensures that the predictive model is aligned with the underlying patterns and relevant subject matter.
In a scenario where your model is consistently achieving mediocre performance on both training and validation data, what might be the underlying problem, and what would be your approach to fix it?
- Increase complexity
- Overfitting, reduce complexity
- Reduce complexity
- Underfitting, add complexity
The underlying problem might be underfitting, where the model is too simple to capture the underlying patterns. Increasing the model's complexity would likely improve performance on both training and validation data.
In the context of PCA, the ________ are unit vectors that define the directions of maximum variance, whereas the ________ represent the magnitude of variance in those directions.
- Eigenvalues, Eigenvectors
- Eigenvectors, Eigenvalues
- principal components, Eigenvectors
- principal directions, magnitudes
In PCA, the "Eigenvectors" are unit vectors that define the directions of maximum variance in the data, whereas the "Eigenvalues" represent the magnitude of variance in those directions. Together, they form the core mathematical components of PCA.
What does the assumption of linearity imply in Simple Linear Regression?
- Both Variables are Categorized
- Dependent Variable is Linear
- Independent Variable is Linear
- Relationship between Dependent and Independent Variables is Linear
The assumption of linearity implies that the relationship between the dependent and independent variables is linear. A non-linear relationship may lead to biased or inefficient estimates.
What challenges might arise when using Hierarchical Clustering on very large datasets?
- Computationally intensive and requires high memory
- Less accurate and requires more hyperparameters
- Less sensitive to distance metrics and more prone to noise
- Prone to overfitting and less interpretable
Hierarchical Clustering can be computationally intensive and require a lot of memory, especially when dealing with very large datasets. The algorithm has to compute and store a distance matrix, which has a size of O(n^2), where n is the number of data points. This can lead to challenges in computational efficiency and memory usage, making it less suitable for large-scale applications.
Imagine a scenario where you want to assess the stability of a statistical estimator. How would Bootstrapping help in this context?
- By fixing the bias in the estimator
- By increasing the size of the dataset
- By repeating the sampling process with replacement and calculating the variance
- By repeating the sampling process without replacement
Bootstrapping assesses the stability of a statistical estimator by repeating the sampling process with replacement and calculating variance, standard error, or other statistics. By creating numerous "bootstrap samples," it allows insights into the estimator's distribution, thereby providing a measure of its stability and reliability.
Why might pruning be necessary in the construction of a Decision Tree?
- Determine Leaf Nodes
- Increase Complexity
- Increase Size
- Reduce Overfitting
Pruning is necessary to remove unnecessary branches, simplifying the model and reducing the risk of overfitting the training data.
You are using Simple Linear Regression for a time-series dataset, and the residuals show a pattern. What does this imply, and what might be the remedy?
- Autocorrelation Present, Use Time-Series Model
- Model is Perfect
- Multicollinearity, Remove Variables
- Normal Distribution, No Remedy Needed
If residuals show a pattern in a time-series dataset, autocorrelation might be present, violating the independence assumption. A time-series model like ARIMA may be a more suitable approach.