The old-school but still super effective way to predict continuous values
| R-squared | - |
|---|---|
| Adjusted R-squared | - |
| Mean Squared Error (MSE) | - |
| Root Mean Squared Error (RMSE) | - |
| Mean Absolute Error (MAE) | - |
| Feature | Coefficient (β) | Standard Error | t-value | p-value | Significance |
|---|
Fits a straight line (or hyperplane for multiple variables) to the data points.
Predicts continuous values like prices, sales, temperature, etc.
Few parameters to tune, making it beginner-friendly for anyone learning ML.
Can predict based on multiple features (Multiple Linear Regression).
Linear regression models the relationship between a dependent variable (Y) and one or more independent variables (X) using a linear equation:
Where:
The model uses the Least Squares Method to find the line of best fit, minimizing the sum of squared differences between observed and predicted values.
Linear regression is a statistical method that models the relationship between a dependent variable (Y) and one or more independent variables (X) using a linear equation. It's used to predict continuous outcomes and understand relationships between variables.
The simplest form is simple linear regression with one independent variable: Y = β₀ + β₁X + ε. For multiple variables, it's called multiple linear regression: Y = β₀ + β₁X₁ + β₂X₂ + ... + βₙXₙ + ε.
Linear regression is appropriate when:
It's commonly used in economics, finance, biology, epidemiology, and social sciences.
Linear regression makes several key assumptions:
Violations of these assumptions may require model adjustments or different techniques.
R-squared (R²) is a statistical measure that represents the proportion of variance in the dependent variable that's explained by the independent variables in the model.
For example, an R² of 0.80 means 80% of the variance in Y is explained by X. However, a high R² doesn't necessarily mean the model is good - it could be overfit.
Adjusted R² is a modified version that accounts for the number of predictors in the model, preventing artificial inflation of R² when adding more variables.
The key differences are:
| Aspect | Linear Regression | Logistic Regression |
|---|---|---|
| Output | Continuous numeric value | Probability (0 to 1) for classification |
| Use Case | Predicting quantities (price, sales) | Binary classification (yes/no, spam/not spam) |
| Equation | Y = β₀ + β₁X + ε | log(p/(1-p)) = β₀ + β₁X |
| Assumptions | Linear relationship, normal residuals | No need for linear relationship |