Skip to content

15. Regression assumptions and diagnostics

Module items

R Script file code

  • [[Copy the code]] below ➜ Paste into [[RStudio console]] ➜ Hit Enter

    • source(url("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/data/0-packages-data.R")); 
      (function(f="15-regression-assumptions-diagnostics.R"){if(!file.exists(f)){download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/15.-regression-assumptions-diagnostics/15-regression-assumptions-diagnostics.R",f,mode="wb");file.edit(f)}else{download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/15.-regression-assumptions-diagnostics/15-regression-assumptions-diagnostics.R",gsub(".R","-original.R",f),mode="wb");file.edit(gsub(".R","-original.R",f))}})()
      
      • When this R script file opens in a new tab, [[Save R script file|save your previous R script file(s)]], and
        • Close the previous tabs (R Script files), which you can find later in the [[Files tab]].

Lab assignment

Regression analysis assumptions and diagnostics

Sample lab assignment

In this assignment, you will run the exact same codes provided in the R script file without making any changes. Therefore, there’s no sample lab assignment file.

Suggested reading


  • Hernán, Miguel A., John Hsu, and Brian Healy. 2019. “A Second Chance to Get Causal Inference Right: A Classification of Data Science Tasks.” Chance 32(1):42–49. doi:10.1080/09332480.2019.1579578

Learning outcomes

  1. Learn about five important checks in regression analysis
    1. Identify the issues posed by homoscedasticity and heteroscedasticity
    2. Identify the issues posed by multicollinearity
    3. Identify the issues posed by curvilinear relationships
    4. Identify the issues posed by non-normally distributed errors
    5. Identify response categories with fewer than 10% of the cases
  2. Learn the diagnostic tools:
    1. Performance package
    2. Scatterplot matrix
  3. Revise a regression model and compare the results before and after revision

[[Linear regression assumptions]] - Overview

  • Assumption 1: Homoscedasticity
    • The errors our model makes should be about the same size across different predicted values.
      • When the size of the errors changes, we call it heteroscedasticity.
  • Assumption 2: Multicollinearity
    • The factor variables should not be too strongly correlated with each other.
      • When they are, it becomes difficult to separate their relationships with the outcome variable.
  • Assumption 3: Linear relationship
    • There should be a linear relationship between the outcome variable and continuous factor variables.
      • This means that a straight line should describe the relationship reasonably well.
  • Assumption 4: Normal distribution
    • The continuous variables used should display approximately normal distribution;
      • Most people's values should be clustered around the middle, not at the very high or very low ends.
  • Check 5: At least 10% of the cases
    • Enough sample size for each category of the dummy variables;
      • In this class, we check whether the categories have at least 10% of the responses.

Assumption 1: Homoscedasticity

  • [[Homoscedasticity]] means that the errors in our regression model have about the same spread across different predicted values.
    • An error is the difference between an actual value and the value predicted by the model.
    • Linear regression aims to create a straight-line model that best fits the data.
      • Two small scatterplot diagrams compare variance patterns: one shows homoscedasticity with data points evenly spread around a horizontal line across all x-values, and the other shows heteroscedasticity with the spread of points increasing as x-values increase. Arrows and lines illustrate constant versus changing variability across the range of the independent variable.
    • Several reasons may contribute to the [[heteroscedasticity]] issue:
      • Outliers: Extreme values in data can lead to heteroscedasticity.
      • Nonlinear relationships: When the relationship between the independent and dependent variables is nonlinear (i.e., curvilinear), it can lead to heteroscedasticity.
      • Omitted variables: If important variables are omitted from the regression model, they can lead to heteroscedasticity.

        • Addressing the heteroscedasticity

          • For this module, we will try removing some variables and then check the model again. Removing variables does not always solve this issue.

Assumption 2: Multicollinearity

  • [[Multicollinearity]] occurs when two or more factor variables in a regression model are strongly correlated with each other.
    • This makes it difficult to separate their individual relationships with the outcome variable.
      • A diagram uses overlapping circles to illustrate multicollinearity, showing two variables with overlapping areas representing shared variance. The figure contrasts slightly correlated variables with highly correlated variables by increasing overlap between the circles.
    • Several solutions exist for [[multicollinearity]] issue:
      • Removing one of the strongly correlated variables
      • Creating an index variable using strongly correlated variables
      • Centering variables (subtracting the mean value from each observation)
      • Lasso regression (L1 Regularization)

        • Addressing the multicollinearity issue

          • For this module, we will remove some strongly related variables and check the model again.

Assumption 3: Linear relationship

  • The term "[[linearity]]" in linear regression means that a straight line can reasonably describe the relationship between a continuous factor variable and the outcome variable.
    • A [[curvilinear relationship]] between a continuous factor and an outcome variable may not be described well by a straight line.
      • A scatterplot shows a curved, non-linear relationship between variables X and Y, with points forming an inverted U-shaped pattern rather than a straight line. A red X marks the plot to indicate that this curvilinear pattern violates the linearity assumption of linear regression.
    • Several solutions exist for [[curvilinear relationship]] issue:
      • Recoding the variable into categorical
      • Polynomial regression
      • Adding interaction terms
      • Trying a different way to represent the relationship in the model

        • Addressing the curvilinear relationship

          • For this module, we will try removing some variables and check the model again. In types of modeling, curved relationships can also be included in regression.

Assumption 4: Normal distribution

  • In linear regression, the model's errors should have an approximately [[normal distribution]] for the usual statistical tests.
    • Set of three curves shows positively skewed, symmetric, and negatively skewed distributions with the positions of mean, median, and mode marked.
  • For example, the variable below has an nonnormal distribution:

    • 1
      2
      3
      plot_frq(gss$weekswrk,
      type = "bar",
      geom.colors = "#336699")
      
      • Histogram of respondents’ weeks worked last year, with very large peaks at 0 and especially 52 weeks and relatively few responses in between; the mean is marked at about 30 weeks.
    • Several ways of dealing with a [[nonnormal distribution]] include:

      • Recoding the variable into categorical
      • Logarithmic transformation (log(x))
      • Square root transformation (sqrt(x))
      • Inverse transformation (1/x)
      • Adding Polynomial terms (x^2)

      • Addressing the nonnormal distribution

        • For this module, we will address the nonnormal distribution issue by removing the problematic variables from the model.

Check 5: At least 10% of the cases

  • For this class, we use [[10% of the cases]] as a guideline for identifying small response categories.

    • Before creating dummy variables, we check the frequency distributions to see whether some categories have fewer than 10% of the cases.
    • Let's check the frequency table of class variable.

      • We can create dummy variables for each category, but "Upper class" has only 4.13% of valid responses. For this exercise, we will consider combining it with another category.
    • frq(gss$class, out = "v")
      
      • Respondents' subjective class identification (Variable label)

        • value value label frq raw.prc valid.prc cum.prc
          1 Lower class 446 11.19 11.31 11.31
          2 Working class 1587 39.81 40.25 51.56
          3 Middle class 1747 43.83 44.31 95.87
          4 Upper class 163 4.09 4.13 100.00
          5 No class 0 0.00 0.00 100.00
          NA NA 43 1.08 NA NA
    • Several solutions exist for having less than [[10% of the cases]] issue:

      • Removing the variable from the model
      • Collapsing the rare category into an adjacent one (e.g., merging "Upper class" with "Middle class")
      • Dropping cases in the rare category from the sample (use cautiously; may introduce bias)
      • Treating the variable as continuous if it is ordinal and the distribution is otherwise reasonable

GSS example: Predicting social life index score

  • We'll use [[computing]] to create an [[index variable]] for our outcome variable, sociallife_index.

    • flowchart LR
      subgraph C0[Factor variables]
          direction TB
          A[Respondents' socio-economic index score]
          B[Respondents' education in years]
          D[Respondents' personal income]
          F[Respondents' family income]
          G[Respondents' occupational prestige score]
          H[Number of children respondents have]
          I[Level of finding life exciting]
      end
      subgraph O0[Outcome variable - Index]
          E[Social life index score<br><br> The mean of: <br><br>  1: Frequency of social evening with relatives <br><br> 2: Frequency of social evening with neighbors]
      end
      A -.->|Association?| E
      B -.->|Association?| E
      D -.->|Association?| E
      F -.->|Association?| E
      G -.->|Association?| E
      H -.->|Association?| E
      I -.->|Association?| E
  • The first two variables are to create sociallife_index variable.

    • Variable name Variable label Variable type Question wording and response categories
      socrel Frequency of social evening with relatives Ordinal ✅ RECODE How often do you spend a social evening with relatives?

      (1: Almost daily; 2: Once or twice a week; 3: Several times a month; 4: About once a month; 5: Several times a year; 6: About once a year; 7: Never)
      socommun Frequency of social evening with neighbors Ordinal ✅ RECODE How often do you spend a social evening with neighbors?

      (1: Almost daily; 2: Once or twice a week; 3: Several times a month; 4: About once a month; 5: Several times a year; 6: About once a year; 7: Never)
      educ Respondents' education in years Continuous What is the highest year of school you completed?

      (Min: 0, Max: 20)
      coninc Respondents' family income Continuous What is your family income in dollars?

      (Min: $281.5, Max: $139,024.4)
      conrinc Respondents' personal income Continuous What is your income in dollars?

      (Min: $281.5; Max, $123,761.9)
      prestg10 Respondents' occupational prestige score Continuous Respondent's occupational prestige score (calculated)

      (Min: 16, Max: 80)
      childs Number of children respondents have Continuous How many children do you have?

      (Min: 0, Max: 8)
      life Level of finding life exciting Ordinal, RECODE In general, do you find life exciting, pretty routine, or dull?

      (1: Exciting; 2: Routine; 3: Dull)

[[Recoding]] and [[computing]] #code

  • [[Working code]]

    • # Recode sociallife_index (social life index) variables
      
      gss$socrel_reversed <- rec(gss$socrel, rec = 
      "1=7 [almost daily];
      2=6 [once or twice a week]; 
      3=5 [several times a month]; 
      4=4 [about once a month]; 
      5=3 [several times a year]; 
      6=2 [about once a year];
      7=1 [never]", append = FALSE)
      
      gss$socommun_reversed <- rec(gss$socommun, rec = 
      "1=7 [almost daily];
      2=6 [once or twice a week]; 
      3=5 [several times a month]; 
      4=4 [about once a month]; 
      5=3 [several times a year]; 
      6=2 [about once a year];
      7=1 [never]", append = FALSE)
      
      # Compute sociallife_index (social life index) variable
      
      gss$sociallife_index <- structure(rowMeans(
      gss[, c("socrel_reversed", "socommun_reversed")], na.rm = TRUE), 
      label = "Social life index score")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will create three more variables.

[[Dummy variable]] #code

  • [[Working code]]

    • gss$exciting <- 
      ifelse(gss$life == 1, 1, 0,
      label = "Finding life exciting")
      
      gss$routine <- 
      ifelse(gss$life == 2, 1, 0,
      label = "Finding life routine")
      
      gss$dull <- 
      ifelse(gss$life == 3, 1, 0,
      label = "Finding life dull")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will create three more variables.

[[Linear regression]] #code (Model 1)

  • [[Working code]]

    • model1 <- lm(sociallife_index ~ sei10 + educ + conrinc + coninc + prestg10 + childs + exciting + routine, data = gss)
      tab_model(model1, show.std = T, show.ci = F, collapse.se = T)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Linear regression]] #output (Model 1)

  • Social life index score

    • Factors Coeff. std. Coeff. p
      (Intercept) 3.69
      (0.34)
      -0.00
      (0.03)
      <0.001***
      Respondents' socio-economic index score 0.00
      (0.00)
      0.04
      (0.07)
      0.505
      Respondents' education in years -0.04
      (0.02)
      -0.07
      (0.04)
      0.076
      Respondents' personal income 0.00
      (0.00)
      0.00
      (0.05)
      0.950
      Respondents' family income -0.00
      (0.00)
      -0.06
      (0.05)
      0.254
      Respondents' occupational prestige score -0.00
      (0.01)
      -0.03
      (0.06)
      0.618
      Number of children respondents have 0.02
      (0.03)
      0.02
      (0.04)
      0.537
      Finding life exciting 1.02
      (0.22)
      0.36
      (0.08)
      <0.001***
      Finding life routine 0.40
      (0.21)
      0.15
      (0.08)
      0.056
      Observations 794
      R² / R² adjusted 0.062 / 0.052

[[Linear regression]] #interpretation (Model 1)

  • Linear regression interpretation sample

    • First section: The significance levels
      • Finding life exciting has a statistically significant association with social life index score since the p-value is less than 0.05.
      • Respondents' socio-economic index score, respondents' education in years, respondents' personal income, respondents' family income, respondents' occupational prestige score, number of children respondents have, and finding life routine are not statistically significant factors of social life index score since their p-values are greater than 0.05.
    • Second section: The explanation of coefficients
      • Finding life exciting is associated with 1.02 points higher predicted social life index score than finding life dull.
    • Third section: The explanation of standardized coefficients
      • The largest standardized association with social life index score is for finding life exciting (std. Coeff=0.36).
    • Fourth section: The explanation of adjusted R-squared
      • The adjusted R-squared value indicates that the model, which includes socio-economic index score, education in years, personal income, family income, occupational prestige score, number of children, finding life exciting, and finding life routine, accounts for 5.2% of the variation in social life index score.

Assessing the assumptions

[[Performance diagnostic]] #code

  • [[Working code]]

    • check_model(model1)
      
      • This code will help us check the first three assumptions.
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[plots tab]] of RStudio).

[[Performance diagnostic]] #output

  • A multi-panel regression diagnostic figure shows several checks: (1) a residuals versus fitted values plot indicating non-constant spread of residuals, suggesting heteroscedasticity; (2) a variance inflation factor (VIF) plot showing elevated values for some predictors, indicating multicollinearity; and (3) a residuals versus fitted values plot with a curved trend, indicating non-linearity. Other panels display posterior predictive checks, influential observations, and a normal Q–Q plot for residuals.
    • The output shows:
      • Assumption (1) [[Homoscedasticity]] diagnostic
      • Assumption (2) [[Multicollinearity]] diagnostic
      • Assumption (3) [[Linearity]] diagnostic

Assumption 1: Homoscedasticity

  • Homogeneity of variance (Reference line should be flat and horizontal):
    • The plot shows a pattern in the spread of residuals across fitted values, rather than a constant (horizontal) band. The smoothed green line is curved instead of flat, and the variance appears to change (wider/narrower regions).
      • The plot suggests that the spread of errors is not the same across predicted values, so the current model is not homoscedastic.
      • We'll run the following code for more details:

[[Homoscedasticity]] #code (Model 1)

  • [[Working code]]

    • check_heteroscedasticity(model1)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Homoscedasticity]] #output (Model 1)

  • Viewer output for the heteroscedasticity check. A warning states that heteroscedasticity was detected, with p = 0.036.
    • Since the p-value is less than 0.05, this test provides evidence of heteroscedasticity in Model 1.

Assumption 2: Multicollinearity

  • [[VIF]]:
    • The Variance Inflation Factor (VIF) is a measure used to detect the presence and severity of multicollinearity in a regression analysis.
    • Lower VIF values are generally better. A VIF of 1 means no multicollinearity with the other factors.
    • Family income (coninc) and personal income (conrinc) are related concepts. Occupational prestige (prestg10) and socio-economic index score (sei10) are also related.
      • We can use these variables together, but strong relationships between them may make it harder to interpret their separate associations.
        • VIF values are all below ~5, with most predictors in the low-to-moderate range.
  • We'll run the following code for more details:

[[Multicollinearity]] #code (Model 1)

  • [[Working code]]

    • check_collinearity(model1)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Multicollinearity]] #output (Model 1)

  • Term VIF VIF 95% CI adj. VIF Tolerance Tolerance 95% CI
    sei10 3.58 [3.19, 4.03] 1.89 0.28 [0.25, 0.31]
    educ 1.49 [1.37, 1.65] 1.22 0.67 [0.61, 0.73]
    conrinc 2.35 [2.12, 2.63] 1.53 0.43 [0.38, 0.47]
    coninc 2.33 [2.10, 2.61] 1.53 0.43 [0.38, 0.48]
    prestg10 2.81 [2.52, 3.16] 1.68 0.36 [0.32, 0.40]
    childs 1.03 [1.00, 1.44] 1.01 0.98 [0.69, 1.00]
    exciting 4.98 [4.42, 5.65] 2.23 0.20 [0.18, 0.23]
    routine 4.92 [4.36, 5.58] 2.22 0.20 [0.18, 0.23]
    • Several VIF values are higher than 2. The highest are for exciting (4.98) and routine (4.92), so we should pay attention to these two variables.

Assumption 3: Linearity

  • Linearity (Reference line should be flat and horizontal):
    • The residuals show a slight curved pattern rather than random scatter around zero, and the smoothed line is not perfectly flat.
      • The reference line is not flat. It curves downward as predicted values increase.
        • This suggests that the model may not capture some curved relationships very well.
    • To further see this issue, we'll use [[scatterplot graph matrix]].

[[Scatterplot graph matrix]] #code (Model 1)

  • [[Working code]]

    • scatterplot_matrix <- gss[, c("sociallife_index", "sei10", "educ", "conrinc", "coninc", "prestg10", "childs")]
      pairs_panels_pval(scatterplot_matrix, color = "#15616d")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[plots tab]] of RStudio).

[[Scatterplot graph matrix]] #output (Model 1)

  • The scatterplot boxes show curvilinear relationships between sociallife_index and socio-economic varibles, namely "sei10", "educ", "conrinc" and coninc".
    • The highlighted scatterplots suggest curvilinear (nonlinear) relationships rather than straight-line relationships.
    • Specifically, the relationships between sociallife_index and some socioeconomic variables (sei10, educ, conrinc, and coninc) appear to be curved.
      • In other words, the relationship may change at different values of these variables. This suggests that a straight-line model may not describe it well.

Assumption 4: Normal distribution

  • We'll use [[scatterplot graph matrix]] to look at the distributions of the variables. However, the normality assumption in regression concerns the model's errors, not the factor variables themselves.

[[Scatterplot graph matrix]] #code (Model 1)

  • [[Working code]]

    • scatterplot_matrix <- gss[, c("sociallife_index", "sei10", "educ", "conrinc", "coninc", "prestg10", "childs")]
      pairs_panels_pval(scatterplot_matrix, color = "#15616d")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[plots tab]] of RStudio).

[[Scatterplot graph matrix]] #output (Model 1)

  • Scatterplot matrix showing relationships among variables (e.g., social life, education, income, prestige, children), with histograms on the diagonal and correlation coefficients (r) and p-values in the upper panels; most associations are weak, with a few moderate-to-strong positive correlations among socioeconomic variables. Childs variable shows nonnormal distribution
    • Most histograms look approximately normal, but childs is skewed (nonnormal).

Check 5: At least 10% of the cases

  • We created dummy variables for life variable without checking the frequency distribution.

[[Frequency table]] #code

  • [[Working code]]

    • frq(gss$life, out = "v")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Frequency table]] #output

  • Level of finding life exciting

    • value value label frq raw.prc valid.prc cum.prc
      1 Exciting 971 24.36 36.42 36.42
      2 Routine 1519 38.11 56.98 93.40
      3 Dull 176 4.42 6.60 100.00
      NA NA 1320 33.12 NA NA
      • Here the "Dull" category has only 6.6% of valid responses (less than 10%).
        • For this exercise, we will merge it with "Routine".

[[Dummy variable]] #code

  • [[Working code]]

    • 1
      2
      3
      4
      5
      6
      7
      gss$exciting_new <- 
      ifelse(gss$life == 1, 1, 0,
      label = "Finding life exciting")
      
      gss$routine_dull_new <- 
      ifelse(gss$life == 2 | gss$life == 3, 1, 0,
      label = "Finding life routine or dull")
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will create two more variables.

Fixing the model (Model 2)

[[Linear regression]] #code (Model 2)

  • [[Working code]]

    • model2 <- lm(sociallife_index ~ educ + exciting_new, data = gss)
      tab_model(model2, show.std = T, show.ci = F, collapse.se = T)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Linear regression]] #output (Model 2)

  • Social life index score

    • Factors Coeff. std. Coeff. p
      (Intercept) 4.01
      (0.19)
      -0.00
      (0.03)
      <0.001***
      Respondents' education in years -0.04
      (0.01)
      -0.08
      (0.03)
      0.004**
      Finding life exciting 0.60
      (0.08)
      0.20
      (0.03)
      <0.001***
      Observations 1313
      R² / R² adjusted 0.043 / 0.041

[[Linear regression]] #interpretation (Model 2)

  • Linear regression interpretation sample

    • First section: The significance levels
      • Respondents' education in years and finding life exciting have statistically significant associations with social life index score since their p-values are less than 0.05.
    • Second section: The explanation of coefficients
      • One additional year of respondents' education is associated with 0.04 points lower predicted social life index score.
      • Finding life exciting is associated with 0.60 points higher predicted social life index score than finding life routine or dull.
    • Third section: The explanation of standardized coefficients
      • The largest standardized association with social life index score is for finding life exciting (std. Coeff=0.20), followed by respondents' education in years (std. Coeff=-0.08).
    • Fourth section: The explanation of adjusted R-squared
      • The adjusted R-squared value indicates that the model, which includes respondents' education in years and finding life exciting, accounts for 4.1% of the variation in social life index score.

Assessing the assumptions

[[Performance diagnostic]] #code (Model 2)

  • [[Working code]]

    • check_model(model2)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[plots tab]] of RStudio).

[[Performance diagnostic]] #output (Model 2)

  • A multi-panel regression diagnostic figure shows several checks: (1) a residuals versus fitted values plot indicating non-constant spread of residuals, suggesting heteroscedasticity; (2) a variance inflation factor (VIF) plot showing elevated values for some predictors, indicating multicollinearity; and (3) a residuals versus fitted values plot with a curved trend, indicating non-linearity. Other panels display posterior predictive checks, influential observations, and a normal Q–Q plot for residuals.

[[Homoscedasticity]] #code (Model 2)

  • [[Working code]]

    • check_heteroscedasticity(model2)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Homoscedasticity]] #output (Model 2)

  • Heteroscedasticity check - OK: Error variance appears to be homoscedastic (p = 0.202).
    • Since the p-value is greater than 0.05, this test does not find statistically significant evidence of heteroscedasticity in Model 2.

[[Multicollinearity]] #code (Model 2)

  • [[Working code]]

    • check_collinearity(model2)
      
      • [[Find this working code in the R script file]].
        • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Multicollinearity]] #output (Model 2)

  • Term VIF VIF 95% CI adj. VIF Tolerance Tolerance 95% CI
    educ 1.02 [1.00, 1.45] 1.01 0.98 [0.69, 1.00]
    exciting_new 1.02 [1.00, 1.45] 1.01 0.98 [0.69, 1.00]
  • Both VIF values are 1.02, which suggests little multicollinearity in Model 2.