06. Chi-square analysis

Module items¶

R Script file code¶

• [[Copy the code]] below ➜ Paste into [[RStudio console]] ➜ Hit Enter

  • source(url("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/data/0-packages-data.R")); 
    (function(f="06-chisquare.R"){if(!file.exists(f)){download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/06.-chi-square-analysis/06-chisquare.R",f,mode="wb");file.edit(f)}else{download.file("https://raw.githubusercontent.com/ttezcann/ssric-reg/refs/heads/main/docs/assets/attachments/modules/06.-chi-square-analysis/06-chisquare.R",gsub(".R","-original.R",f),mode="wb");file.edit(gsub(".R","-original.R",f))}})()
    

    • When this R script file opens in a new tab, [[Save R script file|save your previous R script file(s)]], and
      • Close the previous tabs (R Script files), which you can find later in the [[Files tab]].

Lab assignment¶

Chi-square

Sample lab assignment¶

Sample: Chi-square

Suggested reading¶

Learning outcomes¶

1. Define the purpose and function of chi-square analysis
2. Identify situations where a chi-square test is the appropriate statistical method
3. Differentiate between factor and outcome variables
4. Learn the p-value to determine the statistical significance of a relationship
5. Learn how to generate and interpret a chi-square analysis table

Chi-square basics¶

• Relationship between preferred pet and respondents’ age group

  • Do you think there is a relationship between preferred pet and respondents’ age group?

  • In other words, do you think the preferred pet is influenced by respondents’ age group?

    • Age group	Cat	Dog	Total
      Younger	207 41.4%	293 58.6%	500 100%
      Older	267 53.4%	233 46.6%	500 100%
      Total	474 47.4%	526 52.6%	1000 100%

      • Show the answer

        • While the table shows differences in preferences between age groups (e.g., younger people prefer dogs more than older people), we CANNOT conclude that the differences are statistically significant.
          • To assess the evidence against independence between age group and preferred pet, we use a statistical test like the chi-square test, assuming its requirements are met.

• The [[chi-square]] test is used to discover if there is a relationship between:

  • Two [[categorical]] variables.
  • The chi-square test can only compare categorical variables.
    • It cannot make comparisons between continuous variables or between categorical and continuous variables.

Functions of variables: Factor variable and outcome variable¶

• A [[variable]]:
  • Depending on the [[functions of the variable]] in the analysis is either:
    • [[Outcome variable]]: the main topic that we investigate. It is the outcome: What is being affected or changed? This is also called the dependent variable.
    • [[Factor variable]]: the variable used to explain or understand the outcome variable (main topic). This is also called the independent variable.
      • A factor variable is assumed to cause a change in the outcome variable.
• If we consider the "relationship between preferred pet and respondents’ age group" table, we need to ask the following question:
  • In this table, what seems to change what?
    • Someone's preferred pet will not change their age group. However, the opposite could be the case: someone's age group may influence their preferred pet. Therefore:
      • Age group is the [[factor variable]],
      • Preferred pet is the [[outcome variable]].
• We need to identify, logically, which variable plays which role, because the codes we use depend on this distinction.
  • Some variables can function as either a [[factor variable]] or an [[outcome variable]]:
    • For example, we could argue that the level of education increases someone’s income. In this case;
      • Education is the factor variable, and
      • Income is the outcome variable.
    • In another study, we could argue the opposite: income increases someone’s level of education. In this case,
      • Income is the factor variable, and
      • Education is the outcome variable.
    • These two research examples are completely different because their factor and outcome variables are defined differently.
• However, some variables cannot realistically be outcome variables:
  • Nothing can change a person’s age.
  • Similarly, variables such as place of birth or ethnicity cannot be outcomes because they are not influenced by other variables in the dataset.
  • These variables can only serve as factor variables because they may influence other outcomes:
    • Age → may influence political attitudes, health status, or technology use.
    • Place of birth → may influence language ability or cultural preferences.
    • Ethnicity → may influence income, occupation, or health outcomes.

Null hypothesis¶

• Before we examine whether two variables are related, we start with an assumption that they are not related in the population.
  • This assumption is called the [[null hypothesis]].
    • For example, if we want to study the relationship between age group and preferred pet, our null hypothesis is that age group and preferred pet are not related in the population.
    • We then use data from our sample to see whether we have evidence against this assumption.
      • This is simply our starting assumption. We do not yet know whether the two variables are related.

Statistical significance and p-value¶

• In this and the following modules, we will use [[statistical significance]] to help us understand whether two variables are related. For example, is age group related to preferred pet?
  • We begin by assuming that the two variables are not related in the population. This assumption is called the [[null hypothesis]].
    • We use the [[p-value]] to assess the evidence against this assumption.
      • A smaller p-value gives us more reason to question our assumption that the two variables are not related.
        • All p-values are between 0 and 1.
          • In this course, when a p-value is less than 0.05, we call the result statistically significant. This provides evidence against our assumption that the variables are not related.
            • We write this as p < 0.05.

How to check statistical significance using the [[p-value]]?¶

• [[Is my p-value less than 0.05?]]
  • To determine [[statistical significance]]
    • We can use this website: www.whichnumberislarger.com.
      • Enter your p-value in the first box and 0.05 in the second box.
        • If your p-value is less than 0.05, the result is statistically significant.
        • If your p-value is 0.05 or greater, the result is not statistically significant under this course's p < 0.05 rule.
• [[Check asterisks]]
  • Another way to determine [[statistical significance]]:
    • Our tables and figures use asterisks (*) to indicate statistical significance.

    • If you see **at least one asterisk (*)**, the result is statistically significant at the 0.05 level.

      • The number of asterisks (*) indicates the significance level

        • (1) * means p < 0.05
        • (2) ** means p < 0.01
        • (3) *** means p < 0.001

[[Chi-square]] specifics¶

• When both [[factor variable]] and [[outcome variable]] are [[categorical]], we conduct chi-square.
  • For example, we could wonder if income group has a statistically significant association with life satisfaction level:
    • Factor variable: Income groups [(1) Low, (2) Medium, (3)High],
    • Outcome variable: Life satisfaction level [(1)Not satisfied, (2) Moderately satisfied, (3) Satisfied],
      • If the p-value that the chi-square test generates is less than 0.05, the test provides evidence of an association between income group and life satisfaction level in the population.

GSS example 1: Significant p-value¶

The relationship between (degree and health)¶

• We wonder if respondents' education degree has a statistically significant association with their perceived personal health quality.

  • flowchart LR
        subgraph F["Factor variable (Categorical)"]
            A[Education degree<br/>0: Less than high school<br/> 1: High school<br/> 2: Associate/junior college<br/> 3: Bachelor's<br/> 4: Graduate]
        end
    
        subgraph O["Outcome variable (Categorical)"]
            B[Health quality<br/>1: Excellent<br/> 2: Very good<br/> 3: Good<br/> 4: Fair<br/> 5: Poor]
        end
    
        A ==>|May be associated with| B

Find the variables in Variables in GSS page¶

1. We want to make sure that degree and health are categorical variables.

2. [[Search]] the variable names, degree and health, in Variables in GSS page.

   • Variable name	Variable label	Variable type	Question wording and response categories
     degree	Respondents' education degree	Ordinal	Do you have less than high school, high school, associate/junior college, bachelor's, or graduate degree? (0: Less than high school; 1: High school; 2: Associate/junior college; 3: Bachelor's; 4: Graduate)
     health	Perceived personal health quality	Ordinal ✅ RECODE	Would you say that in general your health is Excellent, Very good, Good, Fair, or Poor? (1: Excellent; 2: Very Good; 3: Good; 4: Fair; 5: Poor)

[[Chi-square]] #code¶

• [[Model code]]

  • 1	sjt.xtab(gss$factor_here, gss$outcome_here, show.row.prc = TRUE)

• [[Working code]]

  • 1	sjt.xtab(gss$degree, gss$health, show.row.prc = TRUE)

    • Line 1: We will put degree here ➜ factor_here and health here ➜ outcome_here.
      • [[Factor variable]] first, [[outcome variable]] second.
        • [[Find this working code in the R script file]].
          • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Chi-square]] #output¶

• Perceived personal health quality by respondents' education degree

  • Respondents' education degree	Excellent	Very Good	Good	Fair	Total
    Less than high school	39 11.1%	140 40%	134 38.3%	37 10.6%	350 100%
    High school	242 13.4%	955 52.8%	520 28.7%	93 5.1%	1810 100%
    Associate/junior college	55 15.3%	211 58.6%	81 22.5%	13 3.6%	360 100%
    Bachelor's	188 21.9%	504 58.6%	148 17.2%	20 2.3%	860 100%
    Graduate	139 24.1%	338 58.6%	91 15.8%	9 1.6%	577 100%
    Total	663 16.8%	2148 54.3%	974 24.6%	172 4.3%	3957 100%

    • χ² = 201.239, df = 12, Cramer's V = 0.130, ==p<0.000***==

[[Chi-square]] #interpretation significant (p < 0.05)¶

• Significant (p < 0.05) chi-square interpretation sample

  There is a statistically significant association between respondents' education degree and perceived personal health quality since the p-value is less than 0.05.

  We can conclude that perceived personal health quality differs in a statistically significant way across respondents with less than high school, high school, associate/junior college, bachelor's, and graduate degree.

• Significant (p < 0.05) chi-square interpretation template

  There is a statistically significant association between [[variable label]] (of [[factor variable]]) and [[variable label]] (of [[outcome variable]]) since the [[p-value]] is less than 0.05.

  We can conclude that [[variable label]] (of [[outcome variable]]) differs in a statistically significant way across [[value label]] 1, [[value label]] 2, ... (of [[factor variable]]).

• Interpretation explanation

  • The [[p-value]] tells us whether the relationship between the two variables is statistically significant.
    • When the p-value is less than 0.05, we conclude that there is a statistically significant association between the two variables.
    • When the p-value is 0.05 or greater, we do not have enough evidence to conclude that the two variables are related.
  • (1) First sentence: We use the variable labels of the factor variable and outcome variable to describe their association.
    • "There is a statistically significant association between respondents' education degree and perceived personal health quality since the p-value is less than 0.05."
  • (2) Second sentence: We explain what the significant chi-square result means.
    • We list the value labels of the factor variable and compare them on the outcome variable.
      • For example: "We can conclude that perceived personal health quality differs in a statistically significant way across respondents with less than high school, high school, associate/junior college, bachelor's, and graduate degree."

GSS Example 2: Nonsignificant p-value¶

The relationship between (sex and happy)¶

• We wonder if respondents' sex has a statistically significant association with their happiness level.

  • flowchart LR
        subgraph F["Factor variable (Categorical)"]
            A[Sex<br/>1=Male<br/> 2: Female]
        end
    
        subgraph O["Outcome variable (Categorical)"]
            B[Happiness<br/>1=Very happy<br/> 2: Pretty happy<br/> 3: Not too happy]
        end
    
        A ==>|May be associated with| B

Find the variables in Variables in GSS page¶

1. We want to make sure that sex and happy are categorical variables.

2. [[Search]] the variable names, sex and happy, in Variables in GSS page.

   • Variable name	Variable label	Variable type	Question wording and response categories
     sex	Respondents' sex	Binary	What's your sex? (1: Male; 2: Female)
     happy	Happiness level	Ordinal, RECODE	Would you say that you are very happy, pretty happy, or not too happy? (1: Very happy; 2: Pretty happy; 3: Not too happy)

[[Chi-square]] #code¶

• [[Model code]]

  • 1	sjt.xtab(gss$factor_here, gss$outcome_here, show.row.prc = TRUE)

• [[Working code]]

  • 1	sjt.xtab(gss$sex, gss$happy, show.row.prc = TRUE)

    • Line 1: We will put sex here ➜ factor_here and happy here ➜ outcome_here.
      • [[Factor variable]] first, [[outcome variable]] second.
        • [[Find this working code in the R script file]].
          • [[Highlighting and running]] this code will generate the output below (which will appear in the [[viewer tab]] of RStudio).

[[Chi-square]] #output¶

• Happiness level by respondents' sex

  • Respondents' sex	Very happy	Pretty happy	Not too happy	Total
    Male	350 19.8%	1025 57.8%	397 22.4%	1772 100%
    Female	441 20.5%	1264 58.7%	450 20.9%	2155 100%
    Total	791 20.1%	2289 58.3%	847 21.6%	3927 100%

    • χ² = 1.399, df = 2, Cramer's V = 0.019, ==p=0.497==

[[Chi-square]] #interpretation nonsignificant (p > 0.05)¶

• Nonsignificant (p > 0.05) chi-square interpretation sample

  There is no statistically significant association between respondents' sex and happiness level since the p-value is greater than 0.05.

  We conclude that happiness level does not differ in a statistically significant way across males and females.

• Nonsignificant (p > 0.05) chi-square interpretation template

  There is no statistically significant association between [[variable label]] (of [[factor variable]]) and [[variable label]] (of [[outcome variable]]) since the [[p-value]] is greater than 0.05.

  We conclude that [[variable label]] (of [[outcome variable]]) does not differ in a statistically significant way across [[value label]] 1, [[value label]] 2, ... (of [[factor variable]]).

• Interpretation explanation

  • The [[p-value]] tells us whether the association between the two variables is statistically significant.
    • When the p-value is less than 0.05, we have evidence that the two variables are related.
    • When the p-value is greater than 0.05, we do not have enough evidence to conclude that the two variables are related.
  • (1) First sentence: We use the variable labels of the factor variable and outcome variable to describe their association.
    • "There is no statistically significant association between respondents' sex and happiness level since the p-value is greater than 0.05."
  • (2) Second sentence: We explain what the nonsignificant chi-square result means.
    • We list the value labels of the factor variable and explain that the groups do not differ in a statistically significant way on the outcome variable.
      • For example: "We conclude that happiness level does not differ in a statistically significant way across males and females."