GITHUB PAGE
https://ballaghjoshua.github.io/

Final Tutorial: Public Education in Louisiana

--- CMPS-3160 ---
--- Tulane University ---
--- By Josh Ballagh and Ellen Waller ---
--- Fall 2020 ---

Introduction

Last Updated 11/24/2020

Last year, the Louisiana Department of Education released a statewide results one-pager noting positive trends in statistics meant to measure the strength of Louisiana's public school system. From 2012 to 2019, statewide ACT scores and graduation rates have been trending upwards (with the exception of 2019 ACT Scores, which saw a small dip).

The Department of Education states that Louisiana is well on track to meet its 2025 education goals, which are centered around raising the standards for 'A' rated schools. By 2025, such schools will have:

Given the Department's optimism at positive statewide trends in recent years, we ask:

Do certain subsets of students drive statewide trends?

That is, are certain groups, either geographic or demographic, improving more than others? If so, which ones? What variables might correlate with greater improvement?

Seeking to answer this question, we have analyzed data made available by the Louisiana Department of Education, as well as some economic data from the US Department of Agriculture. We will examine School Performance Scores (SPS) and ACT Scores in recent years across each of Louisiana's 64 parishes; we will then examine ACT scores over the years across a few key demographic subgroups. In doing so, we hope to determine whether or not any group has been improving at a greater rate than any other group in recent years.

See the project assignment on the course webpage here: https://nmattei.github.io/cmps3160/projects/FinalTutorial/

Data we will be using for this project:

US Department of Agriculture - data on parish attributes: https://www.louisianabelieves.com/resources/library/data-center

Louisiana Department of Education - data on school metrics and student attributes: https://www.louisianabelieves.com/resources/library/data-center

US Census - data used to map Louisiana using Geopandas: https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html

Table of Contents

I. Setup
    A. Import Libraries
    B. Loading/Tidying Parish Attribute Data
II. SPS Exploration
    A. Loading/Tidying SPS Data
    B. SPS Through the Years
    C. Analyzing SPS By School
III. ACT Exploration
    A. Loading/Tidying ACT Data
    B. ACT Through the Years
    C. ACT Correlation with Parish Characteristics
    D. ACT Distributions
IV. Subgroup Exploration
    A. Loading/Tidying Subgroup Data
    B. Visualizing the Achievement Gap
V. Conclusion
    A. Summary of Findings
    B. Limitations and Further Work

I. Setup

I.A. Import Libraries

In this project, we utilize the following libraries:

I.B. Loading/Tidying Parish Attribute Data

In this section we will create two dataframes for later use in our analysis: 1) parishes_geo, a GeoFrame that maps Louisiana by parish, and 2) ParishAttrbitues, a dataframe that will store data from the USDA and Department of Education on each parish's median household income (2018) and percent of students classified as economically disadvantaged (2020).

Though these datasets are not from the same year, they are the most recent data we could find.

Terminology: "Parish" vs. "School System"

There are 64 parishes in Louisiana, but there are more school systems. Why is this? Well, every parish has its own school system, but not every school system belongs to a parish. Special schools that operate apart from a parish's school system, such as magnet schools like the Louisiana School for Math, Science, and the Arts (LSMSA, of which Josh is an alumni!), are coded as their own school systems in the state's datasets. For the most part, our analysis on a parish-by-parish basis will only consider schools within a parish school system. Though LSMSA is in Natchitoches, it will not appear as part of Natchitoches's data!

Terminology: Economically Disadvantaged Students

As per the Louisiana Board of Elementary and Secondary Education, a student is determined to be “Economically Disadvantaged” if (s)he meets any one of the following criteria:

i. Is eligible for Louisiana’s food assistance program for low-income families.

ii. Is eligible for Louisiana’s disaster food assistance program.

iii. Is eligible for Louisiana’s program for assistance to needy families with children to assist parents in becoming self-sufficient.

iv. Is eligible for Louisiana’s healthcare program for families and individuals with limited financial resources.

v. Is eligible for reduced price meals based on the latest available data.

vi. Is an English Language Learner.

vii. Is identified as homeless or migrant pursuant to the McKinney-Vento Homeless Children and Youth Assistance Act and the Migrant Education Program within the Elementary and Secondary Education Act.

viii. Is incarcerated with the Office of Juvenile Justice or in an adult facility.

ix. Has been placed into custody of the state

Loading the Louisiana GeoFrame
Loading and Tidying the 2020 Student Attributes Data from the Department of Education

Though this dataset contains many attributes, we are primarily interested in the % of Economically Disadvantaged Students in each Parish.

Loading and Tidying 2018 USDA Economic Data

This dataset includes unemployment data by parish for several years, but we are interested in Median Household Income by parish, for which the dataset only includes 2018.

Putting It All Together: the Parish Attributes DataFrame

This dataframe includes all of the data from the 2020 Student Attributes dataset, as well as 2018 Median Household Income by Parish from the USDA. We will use it for our Parish-level analysis of SPS and ACT Scores.

Loading and Tidying School Attribute Data

For what data we are able, we would like to perform more granular analysis - analysis on a school-by-school basis. Here we will load another sheet from the same 2020 School Attributes dataset which we created the students20DF, this time using school as an observational unit instead of parish.

II. School Performance Scores (SPS) Exploration

For the first section of our exploration, we will examine School Performance Scores (SPS). As the Department of Education Explains, SPS are derived from a combination of metrics which differ for elementary, middle, and high schools. For example, for high schools, SPS are dependent on:

We understand that SPS is a function of the Louisiana Department of Education's own standards. Still, we find a reasonable question for the beginning of our analysis to be: by the state's own standards, have any parishes been improving at a greater rate than others? If so, does this improvement have anything to do with median household income or the percentage of the student body that is economically disadvantaged?

II.A. Loading and Tidying

The Louisiana Department of Education makes available data on SPS from 1999 onward. We will load the separate datasets containing data as far back as 2012, then extract the data that we want and warehouse it in a single dataframe that we will use for analysis, grouped and indexed by Parish.

Cleaning 2012 Data
Cleaning 2013-2017 Data
Cleaning 2018-2019 Data
Integrating SPS Data

Now that we have loaded and cleaned up the SPS datasets, let's integrate the data we want into a single dataframe grouped by school system. The SPS column, once grouped by school system, will represent the MEAN SPS for the entire system! So, '2018 SPS' for a given parish will represent that parish's mean SPS in 2018.

II.B. SPS Through the Years

To examine change over time, we will use average rate of change over the period of our analysis, which for this section will be 2015-2019. Rate of change can be computed as, for a given start point and end point a and b of a period of time:

(f(b) - f(a)) / (b-a)

For our purposes, b = 2019, a = 2015, and the function f is a parish's mean SPS.

The parish attributes that will serve as independent variables in this section are Median Household Income (2018) and % of Students Economically Disadvantaged (2020).

Average Rate of Change in Parish Mean SPS as a Function of Parish Attributes

As you can see in the above plots, there is virtually no relationship between Avg. Rate of Change in SPS and either of our variables. This may suggest that wealth of a parish or its student body do not have an effect on how much their overall SPS has changed during this four year period.

However, it is entirely possible that analysis at the parish level is not granular enough to determine a relationship between changes in SPS over time and the % of a given student body that is economically disadvantaged. There exist a range of schools in any given parish, some of which may have significantly greater performance than others, and some of which may have many more economically disadvantaged students than others.

Mean 2019 SPS as a Function of Parish Attributes

Despite a lack of relationships in the above section, it is common wisdom that the wealthier a school district, the better its schools will perform. Here we will examine the relationship between each parish's mean SPS from 2019 and our dependent variables to see if this wisdom holds true for Louisiana.

The above plots show us much stronger relationships than we found before, with a 55.7% positive correlation between median household income and mean SPS and a 77.2% negative correlation between % of economically disadvantaged students and mean SPS. The latter of these two is the strongest relationship we have found so far.

At this point, we can state that:

This means that parishes with wealthier student bodies tend to have a higher SPS. However, this DOES NOT INDICATE that a parish with a wealthier student body has seen greater change in SPS than a parish with more economically disadvantaged students.

II.C. Analyzing SPS By School

It seems that, at a Parish level, there is not a strong relationship between average rate of change in SPS and either of our dependent variables. However, we do have access to school-level data on both SPS and the percentage of each school's student body that is economically disadvantaged. As we stated in section II.B., there may exist a range of school wealth and school performance within a given school system. Due to this fact, our parish-level analysis may have been obscured by a problem of granularity.

Here, we will conduct the most granular analysis possible at a school level to determine whether or not the percentage of economically disadvantaged students in a school's student body has any relationship to how that school's SPS has changed from 2015 to 2019.

SPS Rate of Change by School as a Function of School Attributes

As the plot above confirms, there is virtually no relationship between a school's percentage of economically disadvantaged students and the rate of change in the school's SPS. Our prior analysis was in fact not obscured by aggregating schools by parish; there simply exists no relationship here.

III. ACT Exploration

In this section, we aim to explore how ACT scores have changed over time in the state of Louisiana. We will evaluate them primarily at the parish level, in order to examine whether changes seen at the parish level are reflective of certain attributes that belong to the parish. In doing so, we hope to answer our question: Are the overall changes uniform across parishes? Or are parishes seeing different amounts of change depending on their make-up? Specifically in this section we will compare average rate of change of ACT scores with the Median Household Income of parishes, as well as the percent of economically disadvantaged students that a parish has. In doing this, we attempt to see if a parish that has high percent of economically disadvantaged students is seeing a greater (or lesser) increase in ACT scoring than parishes that have less economically disadvantaged students.

Our reasoning for examining ACT scores is to address problems of granularity. We have decided to examine ACT scores as well as SPS because ACT scores are a subcomponent of SPS. Additionally, school assessment is part of SPS, which is reliant on many qualitative judgments. ACT scores are purely quantitative. We suspect that the relationships between parish attributes and the performance metric in question will be similar for ACT and SPS, but still find the analysis worth conducting in order to confirm our hunch.

III.A. Loading and Tidying

The Louisiana Department of Education makes available data on ACT (https://www.louisianabelieves.com/resources/library/high-school-performance). Each year has a separate dataset, so we will load the separate datasets containing data as far back as 2015, then compile the separate years into a singular dataframe that is grouped and indexed by Parish.

Now that we have read in all the separate years, we will compile them into a singular dataframe:

IIIB. 2019 Mean ACT as a Function of Parish Attributes

We will first investigate how the current ACT scores relate to parish attributes, median household income (2018) and % of economically disadvantaged students (2020).

As with 2019 mean SPS in section II, 2019 mean ACT score has a very strong relationship with both dependent variables here. Notably, the relationship between median household income and mean ACT score is much stronger than the relationship between median household income and mean SPS, with a 75% positive correlation.

Additionally, the relationship between percentage of economically disadvantaged students in a parish and 2019 mean ACT is the strongest we have found so far, with a 86.3% negative correlation. This means that the greater the portion of a parish's students are economically disadvantaged, the lower its test scores will be.

Rate of Change in Mean ACT as a Function of Parish Attributes

From the figures above, we can see that there appears to not be a significant correlation between the change in ACT scores that a parish has experienced and either of our dependent variables. Does this mean that poorer parishes are experiencing the same increase in ACT scores as more wealthy parishes? Possibly, but we can't be sure.

This lack of relationships seems consistent with our findings in our analysis of SPS in section II.B.

III.C. ACT Geographic Analysis

So far, we have considered parishes as discrete units. However, despite having distinct school systems, parishes are not necessarily entirely discrete. Certain industries, groups of people, and cultures are distributed all over the state; their geographies cross parish borders. In order to understand regional trends in ACT scores, we will visualize the strongest relationship we have found so far on maps of Louisiana's parishes.

Above, with the maps we are able to visualize how parishes with low ACT scores have a high percent of economically disadvantaged students. It's important to visualize this in map form, as it allows us to not look at parishes as discrete objects. Instead we can see that regional trends exist that span across multiple parishes. For example, you can see in the top northeast corner of the state a collection of parishes that have low ACT scores and high percentages of economically disadvantaged students. The Southwest corner of the state appears to have higher average ACT scores and fewer economically disadvantaged students by comparison.

Lastly, for extra context, let's take a look at the distribution of ACT scores by school system in each year.

It appears as though the distribution of ACT scores has become increasingly centered as the years progress from 2015 to 2019.

IV. Subgroup Exploration

In this section, we will examine on a parish-by-parish basis the ACT scores of key subgroups: Black or African American students, students with disabilities, and students that are economically disadvantaged. Our reasoning for this is twofold: 1) these subgroups' statewide performance are normally below the statewide mean performance and 2) we are missing data on other subgroups prior to 2018.

Our main method for understanding disparity is by computing the achievement gap within each parish. We define the achievement gap as PARISH MEAN ACT SCORE - SUBGROUP MEAN MEAN SCORE.

Given the subgroup data available, our analysis will cover the period from 2012-2018.

IV.A. Loading and Tidying

Cleaning the Subgroup ACT Data

We'll need to add a year column to our dataframe in order to analyze as a time series. Let's this DF out into dataframes by year, then change the observational unit. Instead of the observational unit being a school district, our observational unit will be a school district in a given year. That means that there will be separate rows for Orleans Parish in 2013, 2014, 2015, etc.

Now let's concatenate them all together in a new dataframe called subgroups_df

IV.B. Visualizing the Achievement Gap

In the above graph we can see that statewide the trends across time of these specific subgroups are mainly reflective of the overall average trend. We do see a large drop following 2012 (which is when the mandate that all students take the ACT, not just students that are planning on attending college was put into place), but beyond that is a fairly steady upwards trend until 2017. We are unsure of what could have taken place between 2017 and 2018 to have made this drop occur.

Now we will move on to visualizing achievement gap over time of a particular subgroup: economically disadvantaged students.

As we can see, some parishes saw their achievement gaps for this subgroup decrease drastically over the four-year period from 2015 to 2018. However, not all parishes see such changes, especially parishes that have an extremely high percentage of economically disadvantaged students. (As a thought experiment - if 80% of students in a parish are economically disadvantaged, the gap between their score and the parish's mean score will be virtually nonexistent unless the remaining 20% of students all score extremely high).

To illustrate how widely parishes' situations vary, look at the tables below. Morehouse Parish has seen a slight decrease in average ACT score, but has made even larger gains in closing the achievement gap for economically disadvantaged students. La Salle Parish's average scores have hardly changed, but its achievement gap for economically disadvantaged students has consistently widened. Vernon Parish (Josh's home parish!) has seen a fairly constant average score and achievement gaps that fluctuate with no real pattern.

V. Conclusion

In our project, we hoped to discover whether the overall upwards trend of school metrics in Louisiana were uniform across the board. If they weren't uniform, then we wanted to know which students (whether it be a particular group of parishes, or demographic groups) were driving the trend.

In our exploration of the data available we found no evidence that any particular subset of Louisiana students is driving recent statewide trends in school performance scores or ACT scores, no matter whether grouped students by Parish or by demographic characteristics. We were however able to look at the relationship between current performance and parish attributes. So while no meaningful conclusions can be drawn on whether or not certain parish attributes dictate a parish's propensity for increase in achievement, we can state that we saw significant correlation between SPS/ACT scores and median household income of a parish, as well as percent of economically disadvantaged students.

Furthermore, looking at subgroups within the state, we saw that trends of certain subgroups reflected the overall state trends. Diving further into our subgroup analysis, we analyzed the achievement gap and saw that there is a possibility of decrease in achievement gaps across the state, but we cannot be sure of this as this analysis is highly sensitive to individual parish conditions.

V.A. Summary of Findings

V.B. Limitations and Further Work