Skip to content

13. Logistic regression basics

Module items

R Script file code

  • [[Copy the code]] below ➜ Paste into [[RStudio console]] ➜ Hit Enter 

    • source(url("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/data/0-packages-data.R")); 
(function(f="13-logistic.R"){if(!file.exists(f)){download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/13.-logistic-regression-basics/13-logistic.R",f,mode="wb");file.edit(f)}else{download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/13.-logistic-regression-basics/13-logistic.R",gsub(".R","-original.R",f),mode="wb");file.edit(gsub(".R","-original.R",f))}})()
      • When this R script file opens in a new tab, [[Save R script file|save your previous R script file(s)]], and
        • Close the previous tabs (R Script files), which you can find later in the [[Files tab]].

Lab assignment

Logistic regression

Sample lab assignment

Sample: Logistic regression

Suggested reading


  • Osborne, Jason W. 2015. “A Conceptual Introduction to Bivariate Logistic Regression.” Pp. 1–18 in Best practices in logistic regression. Los Angeles: Sage.

Learning outcomes

  1. Define logistic regression and identify when it is appropriate (binary outcome variable)
  2. Identify situations in which logistic regression is appropriate
  3. Interpret odd ratios (OR) and standardized odd ratios (Std. OR)
  4. Calculate negative odds ratios to report negative effects in interpretable positive terms
  5. Interpret the Tjur R-squared value

Logistic regression structure

  • [[Logistic regression]] is a regression model where;
    • The [[outcome variable]] is
      • A [[dummy variable]], and thus [[binary]], which can take only two values, "1" and "0", such as,
        • pass (1) / fail (0),
        • win (1) / lose (0),
        • alive (1) / dead (0),
        • healthy (1) / sick (0),
        • return (1) / stay (0),
        • voted (1) / didn’t vote (0).
      • Categorical factor variables can also be represented by dummy variables, as in the previous module.

  • We examine how factor variables are associated with the odds of the outcome coded 1.

    • For example, we can examine whether hours of studying (continuous factor variable; min: 0, max: 7) is associated with the odds of passing an exam (binary outcome variable; 1: passing; 0: failing).

      • flowchart LR
subgraph F["Continuous variable"]
    A[Hours of studying<br><br>min: 0, max: 7]
end

subgraph O["Dummy outcome variable"]
    B[Passing the exam <br><br> 1: passing; 0: failing]
end

A -.->|Association?| B

  • The figure below shows that the odds ratio of passing the exam increases with study hours, using 2 hours of studying as the comparison.

    • At 3 hours, the odds ratio is 4.20.
    • At 4 hours, the odds ratio is 17.64.
    • At 5 hours, the odds ratio is 74.09.
      • This means a student who studies for 5 hours has 74.09 times the odds of passing the exam compared with a student who studies for 2 hours.
        • An exponential curve shows how the odds ratio of passing an exam increases with study hours, using two hours of studying as the comparison.

Logistic regression specifics

  • [[Logistic regression]] table provides:
    • [[Odds ratio]] (OR): Shows how the odds of the outcome coded 1 compare for a one-unit increase in a factor variable, while the other factor variables remain the same.
      • For example, one additional hour of studying is associated with 4.20 times the odds of passing the exam.
      • An odds ratio greater than 1 means higher estimated odds of the outcome.
      • An odds ratio less than 1 means lower estimated odds of the outcome.
      • An odds ratio equal to 1 means no estimated difference in odds. An odds ratio does not directly describe changes in probability.
      • [[Standard error]]: Shows the uncertainty in an estimated coefficient. Smaller standard errors generally mean more precise estimates. Standard errors appear in parentheses below the odds ratios.
    • [[Standardized odds ratio]] (std. OR): An odds ratio rescaled to help compare associations between factor variables measured in different units.
      • This is useful because different variables are measured differently.
        • Age is measured in years.
        • Education is measured in years.
        • Income may be measured in dollars.
        • Occupational prestige may be measured with a score.
    • [[p-value]]: Helps us assess evidence against the starting assumption that a factor variable has no association with the odds of the outcome (an odds ratio of 1) under the model's assumptions.
      • When the p-value is less than 0.05, we call the association statistically significant.
      • When the p-value is greater than 0.05, we do not have enough evidence to conclude that the factor variable has a statistically significant association with the odds of the outcome.
    • [[Tjur R-squared]]: Tjur R-squared tells us how well a logistic regression model distinguishes between the two groups of the outcome variable.
      • A larger value means the model distinguishes between the groups more clearly.
        • We report this difference in percentage points.

Example: Passing the exam

  • Here's what a logistic regression table looks like:

    • Passing the exam

      • Factors Odds Ratios std. OR p
        (Intercept) 0.03
        (0.01)        		 0.90
        (0.03)        		 0.001*
        Drinking coffee before exam 1.08
        (0.12)        		 1.03
        (0.04)        		 0.430
        Hours of studying 4.20
        (0.35)        		 2.40
        (0.12)        		 0.001*
        Having a full-time job 0.45
        (0.08)        		 0.70
        (0.05)        		 0.001*
        Observations 500
        R² Tjur 0.312

        • Which factors are significant and which one is nonsignificant?

          • Hours of studying and having a full-time job have statistically significant associations with the odds of passing since their p-values are less than 0.05.
          • Drinking coffee before the exam is not a statistically significant factor of passing the exam since the p-value is greater than 0.05.

        • Which factor is positive and which one is negative?

          • Hours of studying is associated with higher odds of passing (OR = 4.20).
          • Having a full-time job is associated with lower odds of passing (OR = 0.45).

  • How to interpret an odds ratio greater than 1?

    • One additional hour of studying is associated with 4.20 times the odds of passing the exam.
  • How to interpret an odds ratio below 1?
    • Having a full-time job is associated with 0.45 times the odds of passing the exam compared with not having a full-time job.
      • We see 0.45 odd ratio on the table.
        • When reporting negative odd ratios, we divide 1 by the negative odd ratios and standardized odd ratios.
          • Type “calculator” on Google
          • Divide 1 by the odd ratios and standardized odd ratios
            • 1 / 0.45 = 2.22
            • 1 / 0.70 = 1.43
    • If we included "Not having a full-time job" to the model, instead of `Having a full-time job", we'd have this table:

      • Passing the exam

        • Factors Odds Ratios std. OR p
          (Intercept) 0.03
          (0.01)          		 0.90
          (0.03)          		 0.001*
          Drinking coffee before exam 1.08
          (0.12)          		 1.03
          (0.04)          		 0.430
          Hours of studying 4.20
          (0.35)          		 2.40
          (0.12)          		 0.001*
          Not having a full-time job 2.22
          (0.08)          		 1.43
          (0.05)          		 0.001*
          Observations 500
          R² Tjur 0.312

GSS example: Predicting perceiving as higher class

  • Logistic regression analysis will answer the following questions:
    • Which factor variables have statistically significant associations with the odds of perceiving as higher class?
      • How do the estimated odds differ with one additional year of education or between the included dummy-variable categories and their comparison categories?
      • Which factor variables have the largest standardized odds ratios in this model?
      • How well does the model distinguish between respondents who perceive themselves as higher class and those who perceive themselves as lower class?

Find the variables in Variables in GSS page

  • In this analysis, we examine whether education, sex, and race are associated with the odds of perceiving as higher class.

    • flowchart LR
subgraph C0[Continuous factor variable]
    direction TB
    A[Education]
end

subgraph D0[Dummy factor variables]
    subgraph I0[Sex]
        direction TB
        I1[Being male]
        I2[Being female]
    end

    subgraph M0[Race]
        direction TB
        M1[Being white]
        M2[Being nonwhite]

    end
end

subgraph O0[Dummy outcome variable]
    E[Class<br><br>1: Perceiving as higher class <br><br> 0: Perceiving as lower class]
end

A -.->|Association?| E
I0 -.->|Association?| E
M0 -.->|Association?| E

  • We want to make sure that educ is a continuous variable or usable ordinal variables (Ordinal ✅), and need to see the values of categorical variables for dummy variable codes (both outcome and factor variables).

    • Variable name Variable label Variable type Question wording and response categories
      class Respondents' subjective class identification Ordinal ✅ If you were asked to use one of four names for your social class, which would you say you belong in?

      (1: Lower class; 2: Working class; 3: Middle class; 4: Upper class)
      educ Respondents' education in years Continuous What is the highest year of school you completed?

      (Min: 0, Max: 20)
      sex Respondents' sex Binary What's your sex?

      (1: Male; 2: Female)
      race Respondents' race Nominal What's your race?

      (1: White; 2: Black; 3: Other)

  • First, let's create the dummy variables first.

    • We'll create dummy variables for:
      • The outcome variable, class, and
      • The factor variables: sex and race.

[[Dummy variable]]: Categorical (nominal/ordinal) #code

  • We'll merge "1: Lower class" and "2: Working class", and call it lowerclass;
  • We'll merge "3: Middle class" and "4: Upper class" and call it higherclass.
  • We use higherclass as the outcome variable (1: perceiving as higher class; 0: perceiving as lower class). lowerclass identifies the other outcome category.

  • [[Model code]]

    • 1
2
3
4
5
6
7
      gss$dummyvar1 <- 
ifelse(gss$orig_var == value  | gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

gss$dummyvar2 <- 
ifelse(gss$orig_var == value | gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

  • [[Working code]]

    • 1
2
3
4
5
6
7
      gss$lowerclass <- 
ifelse(gss$class == 1  | gss$class == 2, 1, 0,
label = "Perceiving as lower class")

gss$higherclass <- 
ifelse(gss$class == 3 | gss$class == 4, 1, 0,
label = "Perceiving as higher class")
      • Line 1: We put the new variable name here:
        • lowerclass here ➜ dummyvar1
          • It's better to write something simple and memorable, thus lowerclass.
      • Line 2: We put the original variable that we want to create a dummy variable.
        • class here ➜ orig_var
        • Since we want to merge 1 "Lower class" and 2 "Working class", we put gss$class == 1 | gss$class == 2, 1, 0 here ➜ gss$orig_var == value | gss$orig_var == value, 1, 0
          • The whole code with the last 1, 0 means:
            • "if class is 1 or 2, create a new variable called lowerclass, assign them “1”, and assign the rest “0”".
      • Line 3: We write this new dummy variable's variable label here "Perceiving as lower class"
      • Line 5: We put the new variable name here:
        • higherclass here ➜ dummyvar2
          • It's better to write something simple and memorable, thus higherclass.
      • Line 6: We put the original variable that we want to create a dummy variable.
        • class here ➜ orig_var
        • Since we want to merge 3 "Middle class" and 4 "Upper class", we put gss$class == 3 | gss$class == 4, 1, 0 here ➜ gss$orig_var == value | gss$orig_var == value, 1, 0
          • The whole code with the last 1, 0 means:
            • "if class is 3 or 4, create a new variable called higherclass, assign them “1”, and assign the rest “0”".
      • Line 7: We write this new dummy variable's variable label here "Perceiving as higher class"
      • Creating a dummy variable is in a way recoding a variable, and thus creating a new variable.
        • [[Find this working code in the R script file]].
          • [[Highlighting and running]] this code will create two more variables,lowerclass, and higherclass.

[[Dummy variable]]: Categorical (binary) #code

  • We'll create a dummy variable for male;
  • We'll create a dummy variable for female;
  • We'll include female in the model, and male will be our omitted comparison category.

  • [[Model code]]

    • 1
2
3
4
5
6
7
      gss$dummyvar1 <- 
ifelse(gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

gss$dummyvar2 <- 
ifelse(gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

  • [[Working code]]

    • 1
2
3
4
5
6
7
      gss$male <- 
ifelse(gss$sex == 1, 1, 0,
label = "Being male")

gss$female <- 
ifelse(gss$sex == 2, 1, 0,
label = "Being female")
      • Line 1: We put the new variable name here:
        • male here ➜ dummyvar1
          • It's better to write something simple and memorable value label here, thus male.
      • Line 2: We put the original variable that we want to create a dummy variable.
        • sex here ➜ orig_var
        • since 1 is "male" in GSS dataset, we put 1 here ➜ value
          • The whole code with the last 1, 0 means:
            • "if sex is 1, create a new variable called male, assign them “1”, and assign the rest “0”".
      • Line 3: We write this new dummy variable's variable label here "Being male"
      • Line 5: We put the new variable name here:
        • female here ➜ dummyvar2
          • It's better to write something simple and memorable here, thus female.
      • Line 6: We put the original variable that we want to create a dummy variable.
        • sex here ➜ orig_var
        • since 2 is "female" in GSS dataset, we put 2 here ➜ value
          • The whole code with the last 1, 0 means:
            • "if sex is 2, create a new variable called female, assign them “1”, and assign the rest “0”".
      • Line 7: We write this new dummy variable's variable label here "Being female"
      • Creating a dummy variable is in a way recoding a variable, and thus creating a new variable.
        • [[Find this working code in the R script file]].
          • [[Highlighting and running]] this code will create two more variables,male and female.

[[Dummy variable]]: Categorical (nominal/ordinal) #code

  • We'll keep "1: White" and call it white;
  • We'll merge "2: Black" and "3: Other" and call it nonwhite.
  • We'll include nonwhite in the model, and white will be our omitted comparison category.

  • [[Model code]]

    • 1
2
3
4
5
6
7
      gss$dummyvar1 <- 
ifelse(gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

gss$dummyvar2 <- 
ifelse(gss$orig_var == value | gss$orig_var == value, 1, 0,
label = "Dummy variable's variable label")

  • [[Working code]]

    • 1
2
3
4
5
6
7
      gss$white <- 
ifelse(gss$race == 1, 1, 0,
label = "Being white")

gss$nonwhite <- 
ifelse(gss$race == 2 | gss$race == 3, 1, 0,
label = "Being nonwhite")
      • Line 1: We put the new variable name here:
        • white here ➜ dummyvar1
          • It's better to write something simple and memorable, thus white.
      • Line 2: We put the original variable that we want to create a dummy variable.
        • race here ➜ orig_var
        • since 1 is "white" in GSS dataset, we put 1 here ➜ value
          • The whole code with the last 1, 0 means:
            • "if race is 1, create a new variable called white, assign them “1”, and assign the rest “0”".
      • Line 3: We write this new dummy variable's variable label here "Being white"
      • Line 5: We put the new variable name here:
        • nonwhite here ➜ dummyvar2
          • It's better to write something simple and memorable, thus nonwhite.
      • Line 6: We put the original variable that we want to create a dummy variable.
        • race here ➜ orig_var
        • Since we want to merge 2 "Black" and 3 "Other", we put gss$race == 2 | gss$race == 3, 1, 0 here ➜ gss$orig_var == value | gss$orig_var == value, 1, 0
          • The whole code with the last 1, 0 means:
            • "if race is 2 or 3, create a new variable called nonwhite, assign them “1”, and assign the rest “0”".
      • Line 7: We write this new dummy variable's variable label here "Being nonwhite"
      • Creating a dummy variable is in a way recoding a variable, and thus creating a new variable.
        • [[Find this working code in the R script file]].
          • [[Highlighting and running]] this code will create two more variables,white and nonwhite.

[[Logistic regression]] #code

  • [[Model code]]

    • 1
2
      model1 <- glm(dummy_outcome_here ~ factor1_here + factor2_here + factor3_here, data = gss, family = binomial(link="logit"))
tab_model(model1, show.std = T, show.ci = F, collapse.se = T)

  • [[Working code]]

    • 1
2
      model1 <- glm(higherclass ~ educ + female + nonwhite, data = gss, family = binomial(link="logit"))
tab_model(model1, show.std = T, show.ci = F, collapse.se = T)
      • Line 1: We put higherclass here ➜ dummy_outcome_here; educ here ➜ factor1_here; female here ➜ factor2_here; nonwhite here ➜ factor3_here.
        • Outcome variable first; then, factor variables separated by plus (+). We model the odds of higherclass = 1 and use male and white as the comparison categories.
      • Line 2: Check the first argument: model1. If this is model1, then we should use model1 here
        • This needs to be model1, otherwise this code won't work.
          • [[Find this working code in the R script file]].
            • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Logistic regression]] #output

  • Perceiving as higher class

    • Factors Odds Ratios std. OR p
      (Intercept) 0.04
      (0.01)      		 0.94
      (0.03)      		 0.001***
      Respondents' education in years 1.26
      (0.02)      		 1.96
      (0.08)      		 0.001***
      Being female 0.97
      (0.07)      		 0.99
      (0.03)      		 0.693
      Being nonwhite 0.66
      (0.05)      		 0.82
      (0.03)      		 0.001***
      Observations 3831
      R² Tjur 0.102

[[Logistic regression]] with dummy variables #interpretation

  • Logistic regression with dummy variables interpretation sample

    • First section: The significance levels
      • Respondents' education in years and being nonwhite have statistically significant associations with perceiving as higher class since their p-values are less than 0.05.
      • Being female is not a statistically significant factor of perceiving as higher class since the p-value is greater than 0.05.
    • Second section: The explanation of odds ratios
      • One additional year of respondents' education is associated with 1.26 times higher odds of perceiving as higher class.
      • Being nonwhite is associated with 1.52 times lower odds of perceiving as higher class compared with being white.
    • Third section: The explanation of standardized odds ratios
      • The largest standardized association with the odds of perceiving as higher class is for respondents' education in years (std. OR=1.96), followed by being nonwhite (std. OR=1.22).
    • Fourth section: The explanation of Tjur R-squared
      • The Tjur R-squared value indicates that the model, which includes respondents' education in years, being female, and being nonwhite, distinguishes between the two groups with a 10.2 percentage-point difference in their average predicted probabilities.

  • Logistic regression with dummy variables interpretation template

    • First section: The significance levels
      • [[Variable label]] of significant [[factor variable]] 1, variable label of significant factor variable 2, variable label of significant factor variable 3... have statistically significant associations with variable label of [[outcome variable]] since their p-values are less than 0.05.
      • [If any]: Variable label of nonsignificant factor variable 4 is not a statistically significant factor of variable label of outcome variable since the p-value is greater than 0.05.
    • Second section: The explanation of odds ratios
      • One additional [year/point/unit/score of factor variable 1] of variable label of significant factor variable 1 is associated with [odds ratio] times higher/lower odds of variable label of outcome variable.
      • Variable label of included dummy variable 1 is associated with [odds ratio] times higher/lower odds of variable label of outcome variable compared with variable label of omitted - comparison dummy variable 1.
      • Variable label of included dummy variable 2 is associated with [odds ratio] times higher/lower odds of variable label of outcome variable compared with variable label of omitted - comparison dummy variable 2.
    • Third section: The explanation of standardized odds ratios
      • The largest standardized association with the odds of variable label of outcome variable is for variable label of the strongest factor variable (std. OR=[value]), followed by variable label of the second strongest factor variable (std. OR=[value])...
        • When a standardized odds ratio is less than 1, divide 1 by the standardized odds ratio before comparing the sizes of associations.
    • Fourth section: The explanation of Tjur R-squared
      • The [[Tjur R-squared]] value indicates that the model, which includes variable label of factor variable 1, variable label of factor variable 2, variable label of factor variable 3..., distinguishes between the two groups with a [Tjur R-squared × 100] percentage-point difference in their average predicted probabilities.

  • Interpretation explanation

    • First section: The significance levels
      • We identify the variable labels of factor variables that have statistically significant associations with the odds of the outcome variable.
        • When the p-value is less than 0.05,, we say the factor variable has a statistically significant association with the odds of the outcome.
        • When the p-value is greater than 0.05 (p > 0.05), we say the factor variable is not a statistically significant factor of the outcome.
    • Second section: The explanation of odds ratios
      • We use the Odds Ratios column to explain the associations of statistically significant factor variables with the odds of the outcome coded 1.
        • For a numerical factor variable, we explain the odds ratio for a one-unit increase (year, point, score, etc.).
        • For a dummy variable, we compare the included group with the omitted comparison group.
        • When the odds ratio is greater than 1, use the odds ratio and write times higher odds.
        • When the odds ratio is less than 1, divide 1 by the odds ratio and write times lower odds.
    • Third section: The explanation of standardized odds ratios
      • We use the std. OR column to compare the relative size of the standardized associations with the odds of the outcome.
        • When the standardized odds ratio is less than 1, divide 1 by the standardized odds ratio.
          • Note: We do not include nonsignificant factor variables in this section.
    • Fourth section: The explanation of Tjur R-squared
      • We report Tjur R-squared as a difference in percentage points between the average predicted probabilities of the two groups.
        • We list all factor variables included in the model, including those that are not statistically significant.
        • Note: Tjur R-squared is not the percentage of variation explained by a logistic regression model.