Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities
Abstract
To improve college affordability and graduation rates, universities are increasingly allocating âcompletion grantsâ to students who are nearing the finish line but facing financial challenges. Using an experimental design and common program model across 11 broad-access public universities in ten states, we assessed the impact of a completion grants averaging $1,200 distributed among more than 14,000 students. We find that, despite university expectations that most students were near completion, only two-thirds of students eligible to receive a completion grant graduated within the academic year. Receiving a completion grant did not improve that rate. However, nearly all eligible students (95%) graduated within three years or were still working on their degrees. While completion grants are intended to enhance equity, we do not find evidence that they exerted positive impacts for marginalized groups as designed in this study. Moreover, while there was some program implementation variation across universities, it did not lead to differences in program impact.
Finance and Economics Discussion Series Federal Reserve Board, Washington, D.C. ISSN 1936-2854 (Print) ISSN 2767-3898 (Online) Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities Sara Goldrick-Rab, Christine Baker-Smith, Travis T. York, Kallie Clark, Douglas Webber, and Christel Perkins 2023-047 Please cite this paper as: Goldrick-Rab, Sara, Christine Baker-Smith, Travis T. York, Kallie Clark, Douglas Webber, and Christel Perkins (2023). “Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities,” Finance and Economics Discussion Series 2023-047. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2023.047. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.
Affording Degree Completion Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities Sara Goldrick-Rab, Christine Baker-Smith, Travis T. York, Kallie Clark, Douglas Webber, and Christel Perkins
Affording Degree Completion Abstract To improve college affordability and graduation rates, universities are increasingly allocating “completion grants” to students who are nearing the finish line but facing financial challenges. Using an experimental design and common program model across 11 broad-access public universities in ten states, we assessed the impact of a completion grants averaging $1,200 distributed among more than 14,000 students. We find that, despite university expectations that most students were near completion, only two-thirds of students eligible to receive a completion grant graduated within the academic year. Receiving a completion grant did not improve that rate. However, nearly all eligible students (95%) graduated within three years or were still working on their degrees. While completion grants are intended to enhance equity, we do not find evidence that they exerted positive impacts for marginalized groups as designed in this study. Moreover, while there was some program implementation variation across universities, it did not lead to differences in program impact. Keywords Higher education, affordability, graduation, financial aid, inequality
Affording Degree Completion Acknowledgements This research was supported by the Institute for Education Sciences, U.S. Department of Education, through Grant R305N170020 to Temple University. All research funding support to Board economists (Webber) occurred prior to employment with the Board of Governors. The analysis and conclusions in this paper are those of the author and should not be interpreted as reflecting the views of the Board of Governors or the Federal Reserve System. The opinions expressed are those of the authors and also do not represent views of the Institute or the U.S. Department of Education. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the authors’ affiliated institutions. We thank the Association of Public and Land-grant Universities, the Coalition of Urban Serving Universities and The Hope Center for College, Community, and Justice for research and logistical support, in particular Vanessa Coca, Greg Kienzl, Sarah Magnelia, Eddy V. Conroy, Andrew Paradise, Christal Perkins, Jessica Bennett, Allegra Damari and Andrew Kretz. The authors also are deeply grateful to the university administrators, staff, and students who participated in this study.
Affording Degree Completion With college and universities across the United States enrolling greater numbers and more diverse cohorts of students, America’s postsecondary system is increasingly focused on how to equitably serve students. While the number of degree earners and rates of college completion have steadily increased in the U.S. over the past decade, particularly at four-year institutions (from about 1.24 million graduates in 2001 to about 1.98 million graduates in 2018), disparities in graduation rates persist for low-income and underrepresented racial minority subgroups (NCES, 2020). Moreover, time-to- degree extends well beyond four years across the nation and, as with other outcomes, time-to-degree is longer for many structurally marginalized groups (NCES, 2021). These gaps in opportunity and success have a direct negative impact on students and their families. For example, many students who progress several years into their academic journeys accumulate significant debt but do not complete their degrees due to unmet financial need. As a result, nearly 40% of borrowers are trying to pay off loans without the benefit of a degree (Bernard, 2022). They are disproportionately from low-income households and/or are racial or ethnic minorities. Seeking to increase graduation rates and enhance the odds of debt repayment, postsecondary institutions developed a range of strategies aimed at understanding and meeting the changing affordability needs of their students. These efforts include food and housing programs to addressing struggles with living costs, textbook affordability initiatives, emergency aid, free tuition promise programs, and more. Recognizing that a growing number of students face financial shortfalls late in their academic journeys and struggled to continue in college due to unpaid institutional debts, completion grant programs began to proliferate in the mid-2010s. The theory of change was simple: colleges could use administrative data and account balances
Affording Degree Completion to identify students in financial trouble and add a grant to their financial aid package to reduce that need, hopefully promoting the odds of timely degree completion. At the time completion grants began to be widely adapted there was little evidence that they were effective. A decade later, the evidence remains limited. Most promising are recent statistical analyses of Georgia State University’s program, one of the first and widely considered a leader in the field. Evaluations show that receiving a completion grant is associated with shorter time-to-degree and less debt, but not higher graduation rates (Ascendium, 2019; Gumbel, 2020; Renick, 2016; Rossman et al., 2022). However, the evaluators were unable to adjudicate between two competing explanations: On the one hand, the grants could be boosting students to more positive outcomes. On the other hand, the grants might be awarded to students whose outcomes were already more likely to be positive, even without the grants. To be more concrete, it is possible that completion grants are disproportionately given to students who are more adept at navigating the complex financial aid system, more willing to ask for help, and/or in possession of greater social capital to help them survive despite low incomes. In The Privileged Poor, Anthony Jack describes substantial heterogeneity among lowincome students – many look the same on paper, particularly according to their financial aid or stage in school, but have characteristics that are typically unobservable to researchers, such as burdens or supports relating to family/living/work. Since students from more privileged backgrounds with fewer burdens are also more likely to persist/complete relative to their peers, even without the help of a completion grant, it is entirely possible that any measured benefit outside of a randomized setting is a spurious correlation rather than a causal effect. If this is the
Affording Degree Completion case, the use of completion grants could even increase inequality in student outcomes, contrary to the objectives of the schools which offer them. With limited resources at institutions of higher education, the question of whether completion grants are effective at improving graduation rates and reducing time-to- degree is important. Answering that question requires understanding what the outcomes of grant recipients would be if they didn’t receive completion grant aid. Using a randomized-control trial design, with support from the Institute for Education Sciences and a partnership with the Association of Public and Land-grant Universities (APLU), this study estimated the causal impact of completion grants for students nearing graduation across 11 public, open- and broad-access institutions across 10 states. Specifically, we tested the efficacy of a completion grant program with an average grant value of approximately $1,200 distributed among more than 14,000 students. Students did not apply for the grants; they were awarded based on administrative records indicating that they were advanced in their educational trajectory and had unmet financial need. Like the Georgia State approach, these grants were proactively packaged and were not responsive emergency aid awarded due to a particular crisis identified by students. We estimate both average and heterogeneous impacts on completion and time-to- degree. We find that completion grants in this study did not move the needle in a meaningful way, on average or for specific groups of students. There is also no clear evidence suggesting that program effects varied or promoted equity. Moreover, while there was some program implementation variation across universities, it did not lead to differences in program impact.
Affording Degree Completion While we do not view this as the final word on completion grants (certainly more work is necessary given the variety of ways that completion grants can be both designed and implemented), this is an important result for several reasons. First, as mentioned above, in a non-randomized setting there is significant potential for such grants to be unintentionally given to student more likely to persist/complete on the basis of other factors, thus compounding inequality rather than ameliorating it. Second, even in the absence of this form of statistical bias, financial aid offices are often severely resource-constrained, so an intervention with no positive causal effect may be displacing one which could move the needle on student achievement. Finally, our RCT is notable for its large size relative to typical interventions, and thus its ability to rule out even small effect sizes. Affordability and Degree Completion With the average graduation rate of postsecondary institutions prior to the COVID-19 pandemic hovering at 60% for more than a decade, higher education institutions have been looking for ways to improve this rate, particularly for structurally minoritized students (Causey et al., 2020). One factor influencing degree completion is students’ ability to afford expenses (tuition and otherwise) during college. Many studies show that financial challenges increase stress, distract from learning, increase debt and reduce the odds that students will successfully complete college (e.g., Anderson et al., 2020; Bettinger et al., 2019; Broton et al., 2016; Castleman et al., 2018; Goldrick-Rab et al., 2016). Sometimes college becomes less affordable as students get closer to degree completion. Annual tuition increases and additional fees associated with higher-level courses (a problem particularly salient for students in STEM fields) cause students’ costs to increase (Goldrick-Rab
Affording Degree Completion et al., 2021b; Ma et al., 2020; NCES, 2018). Access to continuing federal financial aid (both grants and loans) depends on making Satisfactory Academic Progress (SAP) and some state aid programs add additional requirements measured through a complex set of metrics. Many students lose aid as a result (Schudde & Scott-Clayton, 2016; Lee, 2021). Living costs, a key component of college prices, rise due to increases in rent, the price of meal plans and health care, and so on. Moreover, since the average student requires more than four years to graduate (NCES 2021), costs of attendance continue to compound at the same time as students’ remaining available aid dollars are depleted (Abdul-Alim, 2016; APLU, 2016; Katsinas et al., 2013). The Evolution of Completion Grants Georgia State University is widely credited with pioneering the approach known as “completion grants.” (Jones, 2022; Lee, 2022a) Its Panther Retention Grant program began in 2011 under the direction of Dr. Timothy Renick who was concerned about students who were leaving college due to what appeared to be relatively small financial obstacles. These challenges are especially common at Georgia’s universities because of the rules associated with the state’s longstanding HOPE grant, which includes performance requirements that often cause students to lose aid. The Panther grant was designed to offset that loss and specifically to help students (disproportionately in their last year of college) with unpaid debts to the university of at least $2,500 or more (Weissman, 2022). Grants were as little as $300 and averaged $900. According to the university, one key aspect of the program is that students do not have to apply for the grant—instead “college administrators use existing records on financial need and degree progress to identify students and notify them they have been chosen for the grant. This allows
Affording Degree Completion students to skip filling out forms with information that colleges already have (Lee, 2022a.)” Georgia State assessed its program and then, impressed by the high graduation rates of grant recipients, expanded the effort (GSU, 2018; Gumbel, 2020; Renick, 2016). By 2022, the university had awarded more than 10,000 completion grants (Weismann, 2022). That year Ithaka S&R reported that grant recipients completed college about a half a term more quickly than non-recipients and incurred less debt. However, graduation rates were unaffected (Rossman et al., 2022). Also, the analyses did not address the potential that positive impacts were due to selection effects (who received the grant) rather than program impact. In November 2022 the University Innovation Alliance (UIA) held an event promoting the use of completion grants: “Dr. Renick summarized the results of a just-completed study of Georgia State undergraduates between 2013 and 2021. In comparing completion and graduation rates for students that never had to stop out for a financial reason versus those that did, it was 70% to 20%. For seniors alone, the difference was 80% to 30%. The truly significant number was revealed by comparing this 50- point gap with students who received a completion grant: [Dr. Renick said] "For an average grant of about a thousand dollars, the graduation rates for those students go right back up to 80%. So it completely fills the outcome gap. These students are not stopping out, because we proactively provide the money to keep them enrolled. Do you know of other uses of institutional aid where for a thousand dollars you
Affording Degree Completion can change the outcomes from a 30% graduation rate to an 80% graduation rate? You invest a little bit in these students, not only do they graduate at much higher rates, but the institution generates more revenue by holding onto these students. It's in effect the fiscally selfish thing to do; at the same time, it's the morally right thing to do (Burns, 2022)." Many in the higher education field were inspired by and learned from Georgia State. A 2016 survey by the National Association of Student Personnel Administrators revealed that almost one-third of institutions offered some type of program aimed to address affordability challenges. Many were focused on “near-completers”—students holding almost enough credits to complete their degree—though the majority were not titled “completion grants” and were informal programs funded with institutional dollars (Kruger et al., 2016). Today, there are completion grant programs funded with state as well as institutional dollars. In 2020 the Georgia governor allocated $5M to the effort, and in 2022 he signed legislation expanding the practice (Lee, 2022b). Philanthropy played a key role as well. In 2015 the Lumina Foundation and the Ascendium Education Group funded a project headed by the APLU and the Coalition of Urban Serving Universities, to create completion grants programs at nine universities. The initiative began with a workshop at APLU’s annual meeting in November 2015 where four universities detailed their programs. Subsequently, APLU opened a request for proposals which provided participating institutions with $50,000 to initiate or scale- up existing programs by 2018. Knowing these institutions would not have the funds to serve all students with unmet need, the program focused on “near completers.” It was at that point that the specific phrase “completion grants” was coined. Grants ranged from $500 to $1,500 and were aimed at currently enrolled
Affording Degree Completion students within 30 credit hours of degree completion with “genuine unmet need and an unpaid university balance.” More than 1,200 grants were distributed and 93% of recipients completed their degree or remained enrolled a year after they were awarded (APLU, 2016). In 2017 The Bill and Melinda Gates Foundation invested approximately $4 million in another project on completion grants run by the UIA. Over three years 11 UIA institutions distributed those funds to nearly 5,000 students (the average grant was $741). At the conclusion of that work, the UIA reported that 83% of recipients completed their degrees or remained enrolled in college (University Innovation Alliance, 2021). The organization also released a playbook to inform broader practice (Ascendium, 2021). Clearly, completion grants have become a popular practice. They are driven by wellfounded concerns with college affordability and an equity lens that recognizes that financial challenges disproportionately harm the graduation prospects of structurally disadvantaged students. Program design seeks to reduce administrative burden on students by not requiring an application, instead aiming to provide students with money and get out of the way. The theory of change is strong and common to many need- based grant programs. On the other hand, there are many reasons why the positive outcomes of completion grant recipients may not be caused by the grants. The focus on awarding money to “near-completers” may introduce a type of survival bias, upwardly biasing outcomes by targeting resources to students who would make it to the finish line without them. The results may also depend on institutional attributes and resources such as advanced student tracking systems or larger-thanaverage numbers of staff focused on student aid and assistance. It is likely that institutions with
Affording Degree Completion higher levels of disadvantaged students or lower levels of federal and state investment, for example, may have different results than similar grant programs due to the administrative demands of such a program (Goldrick-Rab et al., 2021c). It is also possible that targeting completion grants based on need is hindered by a reliance on administrative records and a focus on unpaid balances as a measure of unmet need. Financial aid data comes from complex forms using information that is often years old and frequently reflects parental rather than student resources. A students’ institutional balance may be low even though their challenges with living expenses are substantial. Therefore, many emergency aid programs use an application to help students obtain help for immediate needs and allow those needs to go beyond what is owed to the institution (Goldrick-Rab et al., 2021a). For these reasons, the field needed a rigorous evaluation of completion grants that looked at efficacy across multiple universities. This evaluation considers the following questions: (1) Do completion grants exert an independent positive impact on students’ academic performance or attainment in college? (2) Does that impact vary depending on program implementation? (3) Does that impact vary among demographic subgroups of students? RESEARCH METHODOLOGY Financial aid is delivered to students by programs run by people. How funding reaches students depends on the design and administration of those programs (Goldrick-Rab, 2016). Thus, to optimize the odds that completion grants would effectively support students, prior to
Affording Degree Completion this evaluation we spent a year working with universities to develop a completion grant program model that was like the pioneering one at Georgia State while also appropriately adapted to individual university contexts (Goldrick-Rab et al., 2021c). During that stage we identified commonalities in programming as well as areas of divergence. We then convened the universities to co-construct a consistent program model for evaluation. Setting The 11 universities in this study collectively serve more than a quarter million undergraduates, averaging 25,000 undergraduate students each (Table 1). All share both a research mission and an access mission, with an average admissions rate of 66%, making them “broad” access. More than half of the enrolled students are from minority (non-White) backgrounds, 40% are Pell recipients, and just over half are attending university in their home state. The participating universities are: • Arizona State University • Florida International University • Portland State University • University of Illinois at Chicago • University of Memphis • Florida State University • Indiana University Purdue University Indianapolis • University of North Carolina at Charlotte • University of Colorado Denver
Affording Degree Completion • Rutgers University - Newark • Virginia Commonwealth University All 11 universities have substantial challenges promoting timely degree completion. The average four-year graduation rate is 30%, with an average six-year rate of 56%. This is an affordability challenge, as the average in-state cost of attendance is just over $25,000 a year, and half of the students use federal loans to pay for college. <<<Table 1 about here>>> Intervention Completion grants are intended to help students approaching graduation stay on track by addressing their unmet need. Following the example of the Georgia State model, the programs are targeted, use administrative data rather than an application to identify recipients, and address unmet financial need by offering a meaningful amount of grant funding (averaging around $1,000). Though there was some slight variation in program implementation across the 11 participating universities in this study, all adhered to those basic principles with a focus on five core design elements: money, requirements, eligibility, messaging, and timing (Figure 1). <<<Figure 1 about here>>> Across all institutions, completion grants were distributed without repayment requirements.1 The grant amount varied from $500 to $3,000 with an average of $1,200 and grants were packaged to not exceed the student’s cost of attendance. In a point of divergence from the Georgia State model, the universities agreed that to promote graduation, students should receive completion grants if they had unmet need, regardless of whether that was a balance due to the institution or a shortfall of funding for living expenses.
Affording Degree Completion Targeting the grants to near-completers was challenging, especially since few universities had accurate degree audit practices in place. Instead, they decided to focus the grant on students who had already completed at least 75% of the credits required for degree completion. If the students were on a four-year graduation track, that would be the equivalent of targeting funding to seniors; however, the four-year graduation rate at these universities averaged about 30% (Table 1). Other eligibility requirements included: completion of a Free Application for Federal Student Aid (FAFSA) application; an Expected Family Contribution (EFC) of $10,000 or less (within 200% of Pell Grant eligibility); compliance with satisfactory academic progress (SAP) requirements, as defined by the institution; in-state residency; and a current course load of at least six credits. Students did not have to have maxed out all federal aid eligibility (including loans) to receive a completion grant. Institutions used standardized email messaging to notify students of their award and encouraged, but did not require, that they complete other institution-specific activities.2 For example, several universities had previously required academic advising meetings to receive their completion grant. While these were no longer required, they were still available to all eligible students. Completion grants were awarded for up to an academic year (2018–19, institutions could award both fall and spring or only fall) and then we measured academic outcomes for three years to assess program impact.3 Grants were automatically packaged without an application required, and with just one exception packaging occurred no later than two weeks following the start of term.
Affording Degree Completion Student Sample Each university funded their own completion grants and aimed to support an average of 200 students. In summer 2018 they identified eligible students using administrative records (14,226 in all) and then, with our support, selected participants using a lottery with separate draws for Pell recipients and non-recipients. Across universities, 16% of eligible students were selected to participate in the program. The average award was approximately $1,200 and ranged from $223 to $3,000 (Table 2). <<<Table 2 about here>>> Eligible students certainly appeared to be nearing degree completion at the time they were selected for a completion grant. While they were only required to have completed 75% of their credit requirements to be eligible for a grant, on average they had completed 93% (Table 3). More than four in five students were enrolled full-time, suggesting that they should graduate within the academic year. According to their financial aid package, these students also needed financial support— their average EFC was just over $1,700, with an average family income of about $23,000. Almost half of the students were independent for financial aid purposes and 85% were Pell eligible. They faced unmet need of more than $6,000 and almost sixty percent had accepted federal student loans. Gender and racial/ethnic representation in this sample favors female-identified students (55%), as it does nationally, and is majority-minority: 28% of students are Latinx, 13% are Black or African American, and 12% are Asian. Eligible students were divided into equivalent groups of recipients and non-recipients
Affording Degree Completion prior to the time that grants were awarded. In accordance with What Works Clearinghouse standards, the small baseline differences were adjusted through statistical modeling that would allow us to obtain unbiased estimate of program impact.4 <<<Table 3 about here>>> Implementation Prior to examining the impacts of completion grants on program outcomes, we checked to ensure that students received the grants, which often means more than simply packaging the dollars. To use the funds, some research indicates that students need to know that they received dollars and know where they came from and how they can be used (Goldrick-Rab, 2016). This can be challenging in large bureaucracies like universities which distribute a lot of financial aid. Evidence from a survey fielded to all eligible students shortly after grants were distributed confirmed that grant recipients were more likely to know that they received a grant from their university.5 When asked about the sources of their financial aid, 78% of completion grant recipients said they received a grant from their university, compared to 59% of nonrecipients (p<.001). While this sizable difference is promising, this also means that 22% of students who received a completion grant in their aid package did not recognize that it came from the university. Also, a sizable fraction of eligible students (almost 60%) already received institutional support (Table 4). <<<Table 4 about here>>> Broadly consistent with their assessment of types of financial aid they received, a higher percentage of completion grant recipients reported that they used grants from the university to
Affording Degree Completion pay for college. (However, more students seem to use institutional aid to pay for college than recognized that they received any.) Notably, a lower percentage of completion grant recipients reported that they were using credit cards to pay for college (28.4% vs. 32.9% among nonrecipients, p<.01). That promising result is undermined, however, by evident confusion among students about why they received the grant or what they had to do to retain the funds (Figure 2). While two-thirds correctly understood it was awarded based on need, almost the same fraction thought it had to do with their academic performance, which their comments on the surveys suggested meant their grades. Indeed, 85% thought they had to get good grades to keep the grant, 79% thought they had to stay enrolled full- time, and 29% thought they had to meet with an advisor. In other words, many students perceived the grant support as conditional and not without strings—despite the lack of messaging to them indicating those requirements. <<<Figure 2 about here>>> Also, even though the grants were awarded by universities, they appear to have no discernible influence on students’ connection to their financial aid office or sense of empowerment around financial stability (see Appendix A4). Data and Analytic Approach to Estimating Program Impacts We collected detailed administrative records on students’ demographic characteristics, financial aid package, and academic transcripts to estimate the average program impact. Using Equation 1 we estimated the causal impact of treatment on key academic outcomes using the Intent-to-Treat (ITT) framework. This measures the effect of being randomly assigned to get a completion grant. In this study nearly all students who were supposed to get completion grants
Affording Degree Completion did receive them, so results for a Treatment-on-Treated (TOT) do not substantively differ. (E1) yi = α + β * Treatmenti + Xi + Ii + εi yi represents an outcome for student i, Treatmenti is an indicator variable for whether a student was assigned to the treatment group (i.e., student was awarded a completion grant), Xi is a vector of additional student-level covariates included when not equivalent at baseline,; Ii represents institution-level fixed effects, and εi is a term for student- specific random error. β represents the treatment impact, the average difference in outcome yi for the treatment group relative to the control group. If completion grants are effective, estimates of β are expected to be positive, statistically significant, and substantively important. When analyzing variation in the impacts of completion grants, whether across universities or across subgroups of students, we included an additional covariate or set of covariates to examine subgroup-specific impacts (e.g., Completion Grants x Male). Where outcomes are continuous, we use a linear regression model. Where outcomes are dichotomous, we rely on a logistic framework and report coefficients of these models as log-odds. AVERAGE IMPACTS OF COMPLETION GRANTS ON ACADEMIC OUTCOMES Do completion grant programs at public universities exert an independent positive impact on the odds that students will complete degrees faster or at a higher rate? We turn to the evidence, beginning with a descriptive look at how students fared if they were not offered completion grants. Recall that the primary eligibility criteria for these grants was that students were “nearcompletion” at the time of selection, meaning that they had completed at least 75% of their
Affording Degree Completion credits for a degree. The universities identified numerous students exceeding that criterion, and as a result the average percent of credits completed for a degree was 93% before grants were awarded (Table 3). Since more than 80% of students were enrolled full-time, the universities anticipated that most students—particularly those offered these additional grants—would graduate from college within the same academic year. Instead, just two-thirds graduated in that timeframe (Table 5). It took two academic years following the time the grant was awarded for many students to finish (88.9%). Moreover, nearly all students deemed eligible for the grant graduated or remained enrolled in three years of the time it was awarded (95%). Thus, assessing “near-completion” using a students’ credits alongside their SAP standing does not appear a very accurate proxy of how close they were to the finish line. However, it was useful in identifying a group of students with unmet need who, irrespective of the completion grant, would likely graduate. The completion grants averaging $1,200 did not substantively improve the odds that near-completers would graduate more quickly or at all (Table 5). Students finished approximately 22 additional credits in the year grants were awarded and took an average of 10 additional academic months to graduate. Despite the evident unmet financial need apparent in their administrative records, virtually all students (97%) were retained or graduated by the end of the first academic year, with the grants producing just a one percent higher boost in retention (p<.05) that quickly dissipated in the following years. <<<Table 5 about here>>> We tested whether completion grants improved outcomes for marginalized students, examining subgroups based on gender, race, Pell receipt, or the amount of unmet need, and
Affording Degree Completion found no consistent and compelling pattern of evidence (see Appendix Tables A5-A8 for more detail).6 Similarly, the variation in program implementation across universities (evidenced in Table 2) did not produce meaningful variation in program impact. Table 6 addresses that possibility and displays unadjusted results while concealing specific university names. While there is some indication that one college saw returns from distributing completion grants, further examining of the evidence using adjusted modeling and Hierarchical Linear Modeling did not reveal any clear indication that program attributes mattered. <<<Table 6 about here>>> DISCUSSION By examining an emerging practice in financial aid using rigorous methods and a large sample, this study offers information that should inform decisions made by practitioners and policymakers allocating limited resources. We find no clear evidence that completion grants are promoting graduation or improving equity. Thus, the results raise questions about the growing popularity of completion grant programs. It seems likely that descriptive and anecdotal evidence on completion grants pointing to apparently positive results may confuse selection effects (who is eligible for completion grants) with program effects. For example, in this study nearly all the “near-completers” with unmet need graduated. If a university simply reported their graduation rates without offering information on how they would have fared without the completion grants, it might offer the impression that the grants improved graduation rates. In this large and diverse study, however, they did not. Limitations
Affording Degree Completion The large sample size in this study supports our confidence that the overall pattern of effects for completion grants is null. The main analyses have the statistical power to detect small effect sizes; the null effects are precisely estimated zeroes rather than an imprecise estimate of a true positive causal effect. The subgroup analyses have smaller samples, introducing the possibility that we might be failing to capture some small positive impacts of completion grants for specific types of students. But even if those were identified, the size of the effect would likely be too small to justify the monetary investment. It is possible that the impact estimates are context specific. The participating universities offered a lot of institutional support, financial and otherwise, to students facing financial challenges. That about one in two eligible students in this study said they received institutional aid- even if they did not receive a completion grant- is clear evidence of this supportive context. Results might differ if completion grants were deployed in colleges or universities with fewer existing resources for students. This evaluation began nearly two years prior to the COVID-19 pandemic, which greatly increased students’ financial needs, disrupted their academic plans, and introduced multiple new sources of financial aid supports (albeit temporarily). Grants were awarded in fall 2018 and students had nearly two years to complete college before the pandemic began—there was not a pattern of positive impacts during that time. Thus, it seems unlikely that the effects we estimate were affected by the pandemic, but completion grants may work differently in the wake of the pandemic. Implications This study identified several ways to potentially improve the practice of completion grant
Affording Degree Completion programs where they continue to operate. Universities (and students) clearly need a more accurate assessment of students’ standing as it relates to the timing of degree completion. Credit progress is insufficient, as one in three students in this study did not complete in the near-term (a year). When this evaluation began just one university had a degree audit system capable of accurately and efficiently determining (e.g., not by hand or relying upon student self-reports) whether a given student was within a year of graduation. These administrative hurdles have an influence both on the cost-effectiveness of the completion grant program and its ability to target support to students truly nearing completion. Changing one's major, or simply carrying credits that do not contribute to a specific program's degree requirements, would also cause a credit threshold to mechanically overstate students’ progress towards a degree. Program administrators might consider whether targeting resources to students in good academic standing, who already enjoy more access to federal and state aid, is the best use of resources. SAP standards have a disproportionate impact on marginalized students; even a single difficult semester can cause a hard-working and talented individual to lose their financial aid. It is worth experimenting with whether allocating institutional resources to near-completers who have lost that aid due to SAP would create a bigger difference in their outcomes. Many students in this study did not know why they received a completion grant or what it required. Program administrators should improve the transparency and clarity of communications to students. They might also consider whether assessing students’ needs using administrative records, rather than by engaging the student to determine that need, might be more effective. Emergency aid programs tend to use an application because FAFSA data often fail to identify students with basic needs insecurity—some students appear to have need that they have
Affording Degree Completion already covered through employment, while others have substantial need not revealed through a reporting of their parents’ income. Perhaps universities could open access to completion grants via more than one route, including self-identification of need. This would also likely increase the odds that the students understand the purpose of the program. In a time of tight budgets, growing accountability pressures, and widespread financial insecurity among students, higher education leaders are searching for new solutions. In recent years, completion grants emerged as a practice that motivated widespread adoption; in Georgia it even led to state legislation. In this study, more than 2,000 students across 11 public universities received financial support, collectively representing an expenditure of at least $2,000,000. Will other investments like these yield dividends? This study casts doubt. However, these results do not mean that money does not matter to students or that all financial aid is ineffective. They also do not suggest that adding more program requirements would generate bigger effects. Other studies find that simpler programs are often more effective, and there is emerging promising evidence that emergency aid—a far more nimble and responsive approach to student need—may pay off (Anderson, 2021; Evans et al., 2017). Rather, perhaps the most important lesson from this study is that higher education leaders and policymakers need to carefully pilot and rigorously evaluate programs when allocating resources. In a sector where sorting—into colleges, programs, and degrees—is widespread, it is difficult but essential to test for efficacy in ways that minimize selection bias. Universities may appear to have effective programs by restricting access to those programs to students already likely to succeed. In an age of inequality, resources must be allocated in ways that work against
Affording Degree Completion that bias and focus where they can make the biggest difference. Current financial aid standards— which direct more resources to full- time students, those with higher grades, or those who are farther along in their programs, or assess need only among students who can complete complex forms—may be undermining financial aid’s potential return on investment. Future research should consider the effects of directing financial aid to students who appear to be behind, rather than ahead, and to students who demonstrate direct and immediate need.
Affording Degree Completion REFERENCES Abdul-Alim, J. (2016, May). Juggling act. Diverse Issues in Higher Education, 33, 10–11. Anderson, D. (2021, February). Edquity grantees cross the finish line at Compton College. Edquity. Anderson, D. M., Broton, K. M., Goldrick‐Rab, S., & Kelchen, R. (2020). Experimental evidence on the impacts of need‐based financial aid: Longitudinal assessment of the Wisconsin Scholars Grant. Journal of Policy Analysis and Management, 39(3), 720–739. Ascendium Education Group. (2019). A shaper focus: 2019 education philanthropy report. Ascendium. Ascendium Education Group. (2021, May 13). UIA completion grans playbook shows how a little can go a long way for student success. Association for Public & Land-Grant Universities (APLU). (2016). Completion grants: Lessons from a pilot program. Bernard, T.S. (2022, June 1). They got the debt, but not the degree. The New York Times. Bettinger, E., Gurantz, O., Kawano, L., Sacerdote, B., & Stevens, M. (2019). The long- run impacts of financial aid: Evidence from California's Cal Grant. American Economic Journal: Economic Policy, 11(1), 64–94. Broton, K., Goldrick-Rab, S., & Benson, J. (2016). Working for college: The causal impacts of financial grants on undergraduate employment. Education Evaluation and Policy Analysis, 38(3), 477–494.
Affording Degree Completion Burns, B. (2022, November 3). Completion grants: Innovative Financial aid for today’s students. University Innovation Alliance. Castleman, B. L., & Long, B. T. (2016). Looking beyond enrollment: The causal effect of needbased grants on college access, persistence, and graduation. Journal of Labor Economics, 34(4), 1023–1073. Castleman, B. L., Long, B. T., & Mabel, Z. (2018). Can financial aid help to address the growing need for STEM education? The effects of need‐based grants on the completion of science, technology, engineering, and math courses and degrees. Journal of Policy Analysis and Management, 37(1), 136–166. Causey, J., Huie, F., Lang, R., Ryu, M., & Shapiro, D. (2020, December). Completing college 2020: A national view of student completion rates for 2014 entering cohort (Signature Report 19). National Student Clearinghouse Research Center. Center for Law and Social Policy (CLASP). (2018). When financial aid falls short: New data reveal students face thousands in unmet need. Crutchfield, R., Carpena, A., McColyn, T., & Maguire, J. (2020). The starving student narrative: how normalizing deprivation reinforces basic need insecurity in higher education. Families in Society. Evans, W.N., Kearney, M.S., Perry, B.C., & Sullivan, J.X. (2017). Increasing community college completion rates among low-income students: Evidence from a randomized controlled trial evaluation of a case management intervention. NBER Working Paper
Affording Degree Completion Series. Georgia State University. (2018). Student success programs. Goldrick-Rab, S., Baker-Smith, C., Bettinger, E., Walton, G., Brady, S., Gill, J., & Looker, E. (2022). Connecting community college students to non-tuition supports during the COVID-19 pandemic. The Hope Center for College, Community, and Justice. Goldrick-Rab, S., Hacker, N.L., Kienzl, G., Price, D.V., & Curtis, D. (2021a). When care isn’t enough: Scaling emergency aid during the pandemic. The Hope Center for College, Community, and Justice. Goldrick-Rab, S., Hernandez, D., Coca, V., Williams, T., & Richardson, B. (2020). Houston food scholarship program report. The Hope Center for College, Community, and Justice. Goldrick-Rab, S. (2016). Paying the price: College costs, financial aid, and the betrayal of the American dream. The University of Chicago Press. Goldrick-Rab, S., Kelchen, R., Harris, D. N., & Benson, J. (2016). Reducing income inequality in educational attainment: Experimental evidence on the impact of financial aid on college completion. American Journal of Sociology, 121(6), 1762– 1817. Goldrick-Rab, S., Roksa, J., Bowman, A., Coca, V., Kinsley, P., Baker-Smith, C., Colo, E., & Kienzl, G. (2021b). The price of STEM success: The impact of need-based financial aid on STEM production. The Hope Center for College, Community, and Justice. Goldrick-Rab, S., York, T., Cady, C., & Baker-Smith, C. (2021c). Completion grants: A multi-
Affording Degree Completion method examination of institutional practice. Journal of Student Financial Aid, 50(1), 1– 19. Gumbel, A. (2020). Won’t lose this dream: How an upstart urban university rewrote the rules of a broken system. The New Press. Gurantz, O. (2015). Who loses out? Registration order, course availability, and student behaviors in community college. Journal of Higher Education, 86(4), 524–63. Henry, L. (2020). Experiences of hunger and food insecurity in college. Palgrave Macmillan; Stebleton, M.J., Lee, C.K., & Diamond, K.K. (2020). Understanding the food insecurity experiences of college students: A qualitative inquiry. The Review of Higher Education. Herd, P., & Moynihan, D. P. (2018). Administrative burden: Policymaking by other means. Russell Sage Foundation. Jones, A. (2022, June 27). Completion grants pioneered at Georgia State to expand statewide. Georgia State News Hub. Katsinas, S. G., Davis, J. E., Friedel, J. N., Koh, J. P., & Grant, P. D. (2013). The impact of new Pell Grant restrictions on community colleges: A three state study of Alabama, Arkansas, and Mississippi. Education Policy Center. University of Alabama. Kane, T. (1999). The price of admission: Rethinking how Americans pay for college. Brookings Institution Press. Kruger, K., Parnell, A., & Wesaw, A. (2016). Landscape analysis of emergency aid programs. National Association of Student Personnel Administrators (NASPA).
Affording Degree Completion Lee, J. (2022a, February 23). Completion grants help students persist and graduate. Georgia Budget & Policy Institute. Lee, J. (2021, July 1). Keeping and losing HOPE. Georgia Budget & Policy Institute. Lee, J. (2022b, March 11). Legislation advances to establish need-based grant supporting college completion | Bill analysis: House bill 1435 (LC 50 0377S). Georgia Budget & Policy Institute. Ma, J., Pender, M., & Libassi, C. (2020, October). Trends in college pricing and student aid 2020. College Board. Renick, T. (2019). Georgia State University Complete College Georgia. Georgia State University. Rossman, D., Karon, J., & Alamuddin, R. (2022, March 31). The impacts of emergency microgrants on student success: Evaluation study of Georgia State University’s Panther Retention Grant Program. https://doi.org/10.18665/sr.316611 Schudde, L., & Scott-Clayton, J. (2016, October). Performance standards in need-based student aid. NBER Working Paper No. 22713. Snyder, T.D., de Brey, C., & Dillow, S.A. (2019). Digest of education statistics 2018 (NCES 2020-009). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. Terriquez, V., Gurantz, O., & Gomez, A. (2013). California’s college stopouts: The significance of financial barriers to continuous school enrollment. Pathways to Postsecondary Success, Policy Report, 7. Los Angeles, CA: UC/ACCORD and
Affording Degree Completion PATHWAYS to Postsecondary Success. Terriquez, V. (2015). Dreams delayed: Barriers to degree completion among undocumented community college students. Journal of Ethnic and Migration Studies, 41(8), 1302–23. The Hope Center for College, Community, and Justice. (2021). States leading the way in emergency aid for college students. University Innovation Alliance. (2021). University innovation alliance completion grants playbook. Urban Institute. (2021). Understanding college affordability. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Higher Education General Information Survey (HEGIS). (2020). Degrees and other formal awards conferred surveys, 1976–77 and 1980– 81. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Integrated Postsecondary Education Data System (IPEDS). (2020). Enrollment Survey. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Integrated Postsecondary Education Data System (IPEDS). (2020). Completion Survey. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. (2021). Undergraduate retention and graduation rates. Weissman, S. (2022, April 5). Clearing the ‘final obstacle’ to a degree. Inside Higher Ed.
Affording Degree Completion What Works Clearinghouse. (2020). What Works Clearinghouse standards handbook, version 4.1. Wooldridge, J.M. (2015). Introductory econometrics: A modern approach. Cengage Learning. Woosley, S. (2003). Stop-out or drop-out? An examination of college withdrawals and reenrollments. Journal of College Student Retention: Research, Theory & Practice, 5(3), 293-30
Affording Degree Completion TABLE 1 | INSTITUTIONAL CHARACTERISTICS OF PARTICIPATING UNIVERSITIES Average / Percentage Institutional Characteristics Total undergraduates (N) 25,273 Academics 4-year Bachelor's degree completion rate (%) 30 6-year Bachelor's degree completion rate (%) 56 Admissions rate (%) 66 Non-White (%) 53 In-State (%) 53 Region South (%) 45 Southwest (%) 18 Midwest (%) 18 Pacific Northwest (%) 9 East (%) 9 Urbanicity Rural (%) 0 Suburb (%) 9 Small city (%) 0 Midsize city (%) 27 Large city (%) 63 Financials In-State Tuition and fees ($) 9,429 In-State Cost of attendance ($) 25,666 Pell* (%) 40 Federal student loans* (%) 49 Source | Where * is listed data is from 2015-16 College Navigator website; otherwise data is drawn from fall 2016 Notes | Percentages may not add up to 100% due to rounding error.
Affording Degree Completion TABLE 2 | PROGRAM ELIBILITY, RECEIPT, AND GRANT SIZE BY UNIVERSITY Received Completion Grant Completion Grant Award ($) Percent Total Served Total Institution eligible (%) Served Average Maximum Overall 14,266 16 2,231 1,232 3,000 Arizona State University 2,446 17 410 985 1,000 Florida International University 832 12 100 1,000 1,000 Portland State University 1,369 9 125 1,586 2,000 University of Illinois at Chicago 3,223 5 146 499 5,000 University of Memphis 342 58 198 2,995 3,000 Florida State University 2,082 9 196 1,000 1,000 Indiana University Purdue 1,010 20 200 995 1,000 University Indianapolis University of North Carolina at 1,068 20 210 1,500 1,500 Charlotte University of Colorado Denver 1,099 18 199 1,000 1,000 Rutgers University - Newark 183 28 51 1,500 1,500 Virginia Commonwealth 572 69 396 993 1,000 University Source | Data are obtained from institution records. Notes | Award amount could not exceed student's unmet need, therefore, a small number of students in the treatment group received award amounts of $0 (contact authors for details). Percent who received a completion grant is of total eligible students.
Affording Degree Completion TABLE 3 | BASELINE COMPARISON OF STUDENTS BY TREATMENT STATUS Treatment Status No Completion Completion Effect Size Student Characteristics All Grant Grant Difference Variables Used in Randomization Average percent of degree credits 93 93 93 0.06 completed (%) Enrolled at full-time status (%) 81 81 80 0.03 Took out student loans (%) 58 57 64 0.15 Pell eligible (%) 85 85 85 0.00 Average unmet need ($) 6,183 6,191 6,140 0.01 Average EFC ($) 1,728 1,719 1,776 0.02 Additional Demographics (post-randomization) Race/Ethnicity: Asian (%) 12 12 10 0.10 Race/Ethnicity: Black or African 13 12 17 0.25 American (%) Race/Ethnicity: Latinx (%) 28 29 22 0.22 Race/Ethnicity: White (%) 41 41 43 0.07 Female (%) 55 55 59 0.09 Independent on FAFSA (%) 48 47 51 0.10 Average family income ‘18-‘19 ($) 23,016 22,364 26,531 0.18 Source | Data are obtained from institution’s administrative records.
Affording Degree Completion Notes | N=14,226 (completion grant group=2,226, no completion grant group=12,000). The column for “Difference” includes effect sizes calculated according to What Works Clearinghouse standards. Family income is a combination of student income and parent income as reported on the FAFSA. Percent of degree credits completed refers to the percentage of needed credits to graduate from the university attended already completed at the time of randomization. Percent of students who took out student loans refers to the percent of students who took out any kind of federally reported student loan of any amount. Independent on the FAFSA refers to the percent of students who have been determined to be independent for FAFSA filing purposes, and therefore do not need to report parental income. Nine of the 11 universities have no missing data. University of Illinois at Chicago has two students with missing FAFSA information. Rutgers has no information on the independent status of participants. The remaining missing data are from Florida International University. Missing values on continuous variables have been mean imputed. Frequency of missing are as follows: female (23), took out student loans (81), Pell eligible (81), race and ethnicity (210), independent on FAFSA (85), EFC (83), unmet need (81), and percent of degree credits completed (81). Full-time enrollment and family income have no missing values. See Appendix A-1 for differences within each institution. TABLE 4 | PAYING FOR COLLEGE, BY TREATMENT GROUP Mean by Completion Grant (Treatment) Status No Yes Difference Way I Pay for College (%) Government 92.4 93.41 1.04 Grants from university 70.94 84.09 13.96 *** Credit cards 32.9 28.42 -4.54 *** Employer 15.78 15.35 -0.43 Family/Friends 49.19 52.07 2.88 Loans 72.26 74.2 1.94 Savings 48.22 50.17 1.95 Type of Financial Aid (%) Pell 86.57 86.52 -0.06 Grant-State 55.06 59.15 4.1 * Grant-University 59.5 78.28 19.45 *** Loan-Federal 68.02 70.09 2.08 Loan-College 16.19 16.38 0.19 Loan-Private 8.48 10.52 1.99 * Source | Survey administered to survey sample during winter 2018-19. Notes | *=p<.05, **=p<.01, ***=p<.001. Total survey sample includes all students in study sample who responded to survey (completion grant(n)=1,348; no completion grant(n)=2,575), sample size may vary slightly by survey question. Analyses are comparison between treatment and control groups in survey sample estimated with multivariate logistic regression and
Affording Degree Completion controlling for variables not equivalent at baseline including: student loan status, race, gender, and FAFSA independence. Results presented as predicted probabilities from marginal means. TABLE 5 | ADJUSTED INTENT-TO-TREAT ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES Mean by Completion Grant (Treatment) Status No Yes Impact SE Months / Credits Credits completed 2018-2019 22.3 22.5 0.15 -- 0.21 Months to graduation 10.7 10.7 0.01 -- 0.17 Marginal Percent (%) Coefficient Mean Completed degree: Spring 66.2 67.5 1.37 0.07 0.06 2019 Completed degree: Spring 88.8 89.1 0.22 0.02 0.08 2020 Completed degree: Spring 93.4 94.4 1.07 0.19 0.11 2021 Completed degree or still 97.2 98.2 1.06 0.42 0.19 enrolled: Spring 2019^ Completed degree or still 94.9 95.3 0.45 0.10 0.12 enrolled: Spring 2020 Completed degree or still 95.0 95.9 0.91 0.20 0.13 enrolled: Spring 2021 Source | Data are obtained from college administrative and National Student Clearinghouse records. Notes | *=p<.05, **=p<.01, ***=p<.001; N=14,226 (completion grant group=2,226, no completion grant group=12,000). However, for the model estimating the impact of completion
Affording Degree Completion grants on completed degree or still enrolled outcome in spring 2019, participants at UNCC (n=1,068) were excluded from the analyses as less than 2% of the sample were not enrolled or had completed a degree. Months to graduation is calculated only for students who graduated (n=13,111). Degree completion and continued enrollment are derived from multivariate logistic regression models: impacts are reported in predicted probability by way of marginal means. Cumulative credits, and months to graduation impacts are derived from multivariate linear regression models: impacts are reported in relative units. All models control for college fixed effects, gender, race and ethnicity, dependency status on the FAFSA, family income, percent of credits needed to complete college, and whether student has loans. Nine of the 11 universities have no missing data. University of Illinois at Chicago has two students with missing FAFSA information. Rutgers has no information on the independent status of participants. The remaining missing data are from Florida International University. Missing values have been mean imputed. Frequency of missing are as follows: female (23), took out student loans (81), Pell eligible (81), race and ethnicity (210), independent on FAFSA (85), EFC (83), unmet need (81), and percent of degree credits completed (81). Full-time enrollment and family income have no missing values. TABLE 6 | UNADJUSTED INTENT-TO-TREAT IMPACT ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES, BY INSTITUTION
Affording Degree Completion Source | Data are obtained from college administrative and National Student Clearinghouse records. Notes | *=p<.05, **=p<.01, ***=p<.001. Overall (N)=14,226 (completion grant group=2,226; no completion grant group=12,000). Months to graduation is calculated only for students who graduated. Cumulative credits, and months to graduation impacts are derived from multivariate linear regression models: impacts (i.e., difference between treatment and control groups) are reported in relative units. Degree completion and continued enrollment are derived from multivariate logistic regression models: impacts are reported as the difference in predicted probability between the treatment group and control group by way of marginal means. Overall models include for college fixed effects. Spring 2019 results for Colleges F and H were not available as all students in control or treatment groups were enrolled or completed a degree.
Affording Degree Completion APPENDIX A-1 | BASELINE COMPARISON OF STUDENTS BY TREATMENT STATUS AND INSTITUTION Difference (Hedges / Cox Effect Size) Institution: Student Characteristics A B C D E F G H I J K Variables Used in Randomization Average percent of degree credits completed 0.03 0.77 0.05 0.07 0.12 0.02 0.02 0.04 0.08 0.04 0.12 Enrolled at full-time status 0.06 0.43 0.13 0.05 0.01 0.18 0.08 0.02 0.01 0.79 0.12 % Took out student loans 0.07 0.64 0.04 0.13 0.02 0.11 0.01 0.12 0.03 0.31 0.01 Pell eligible 0.03 0.44 0.18 0.05 0.02 0.03 0.01 0.00 0.03 0.34 0.00 Average unmet need 0.02 0.63 0.05 0.01 0.09 0.02 0.08 0.02 0.10 0.07 0.06 $ Average EFC 0.01 0.55 0.02 0.07 0.09 0.06 0.03 0.07 0.05 0.02 0.00 Average family income 18-19 0.03 0.18 0.01 0.11 0.03 0.10 0.05 0.07 0.12 0.30 0.01 Additional Demographics Asian 0.19 0.37 0.25 0.10 0.50 0.31 0.38 0.19 0.12 0.03 0.17 Black or African American 0.18 0.99 0.59 0.26 0.21 0.02 0.11 0.00 0.04 0.08 0.19 Latinx 0.09 0.78 0.02 0.04 0.12 0.17 0.36 0.06 0.13 0.10 0.00 % White 0.11 0.19 0.00 0.06 0.27 0.04 0.11 0.07 0.03 0.18 0.22 Female 0.02 0.68 0.02 0.10 0.40 0.04 0.03 0.09 0.03 0.16 0.08 Independent on FAFSA 0.07 0.73 0.19 0.04 0.05 0.11 0.02 0.00 0.19 0.31 0.04 Source | Data are obtained from institutional administrative records. Note School A (N=2,446), School B (N=832), School C (N=1,369), School D (N=3,223), School F (N=2,082), School G (N=1,010), School H (N=1,068), School I (N=1,099), School J (N=183), School K (N=572). Differences between treatment and control are reported in effect size. Effect sizes are calculated according to What Works Clearinghouse standards, only for variables on which students were randomized. Family income is a combination of student income and parent income as reported on the FAFSA. Percent of degree credits completed refers to the percentage of needed credits to graduate from the university students attended at the time of randomization. Percent of students who took out students loans refers to the percent of students who took out any kind of federally reported student loan of any amount. Independent on the FAFSA refers to the percent of students who have been determined to be independent for FAFSA filing purposes, and therefore do not need to report parental income. Nine of the eleven universities have no missing data. UIC has two students with missing FAFSA information. The remaining missing data are from Florida International University. Missing values have been mean imputed. Frequency of missing are as follows: Female (19), Took out student loans (81), Pell Eligible (81), Race and Ethnicity (1), Independent on FAFSA (81), EFC (83), Unmet need (81), Percent of Degree Credits
Affording Degree Completion Completed (81). Full-time enrollment and family income have no missing values.
Affording Degree Completion TABLE A-2 | INTENT-TO-TREAT UNADJUSTED ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES Unadjusted No Completion Completion Standard Grant Grant Impact Error Months / Credits Credits completed 2018-2019 22.31 22.35 0.04 0.21 Months to graduation 10.67 10.65 -0.02 0.17 Marginal Percent Mean Coefficient Completed degree: Spring 2019 66.13 67.70 1.59 0.07 0.05 Completed degree: Spring 2020 88.84 89.06 0.23 0.02 0.08 Completed degree: Spring 2021 93.37 94.37 1.06 0.18 0.11 Completed degree or still enrolled: Spring 2019* 97.23 98.13 1.03 0.41 0.19 Completed degree or still enrolled: Spring 2020 94.89 95.30 0.42 0.09 0.12 Completed degree or still enrolled: Spring 2021 95.03 95.88 0.90 0.20 0.13 Source | Data are obtained from institutional administrative records. Notes: N=14,226 (Completion Grant group=2,226, No Completion Grant group=12,000). Months to graduation is calculated only for students who graduated (8,963). Cumulative credits is missing for 542 students. All other outcomes are calculated for all students. Degree completion and continued enrollment are derived from logistic regression models; impacts are reported in percentage points. Cumulative credits, and months to graduation impacts are derived from linear regression models; impacts are reported in relative units. All models include college fixed effects.
Affording Degree Completion TABLE A-3 | ADJUSTED INTENT-TO-TREAT IMPACT ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES, BY INSTITUTION Completed Degree (%) Completed Degree or Still Enrolled (%) School Spring 2019 Spring 2020 Spring 2021 Spring 2019 Spring 2020 Spring 2021 Impact Coeff SE Impact Coeff SE Impact Coeff SE Impact Coeff SE Impact Coeff SE Impact Coeff SE School A 3.28 0.20 0.14 0.91 0.14 0.22 1.95 0.43 0.29 0.78 0.53 0.54 1.38 0.37 0.31 2.08 ~ 0.57 0.34 School B 2.07 0.09 0.23 -0.36 -0.04 0.35 -0.55 -0.09 0.42 1.06 0.63 1.12 2.90 0.81 0.78 -0.27 -0.05 0.47 School C 2.72 0.13 0.21 0.52 0.05 0.28 -0.71 -0.09 0.32 1.80 0.61 0.75 -0.87 -0.11 0.32 -1.43 -0.22 0.34 School D -7.53 ~ -0.36 0.19 -3.24 -0.35 0.25 -0.65 -0.12 0.33 -1.43 -0.48 0.41 -0.59 -0.13 0.36 -0.85 -0.20 0.36 School E 4.59 0.27 0.27 5.63 ~ 0.62 0.37 5.46 0.71 0.48 5.95 ** 1.46 0.58 -0.10 -0.01 0.48 4.88 0.69 0.51 School F 3.70 0.16 0.16 -1.52 -0.19 0.25 1.06 0.29 0.45 n/a n/a n/a 1.01 0.38 0.53 2.37 1.01 0.75 School G 0.99 0.04 0.16 1.88 0.12 0.21 2.15 0.20 0.26 0.92 0.23 0.42 1.41 0.17 0.29 3.04 0.38 0.31 School H -0.33 -0.02 0.18 -0.67 -0.14 0.34 -0.55 -0.20 0.43 n/a n/a n/a -0.87 -0.31 0.41 -0.64 -0.30 0.47 School I -1.08 -0.05 0.16 -1.88 -0.12 0.20 1.01 0.10 0.25 1.15 0.34 0.45 -0.42 -0.06 0.29 0.62 0.09 0.30 School J -5.20 -0.23 0.35 5.19 0.53 0.57 4.89 0.55 0.60 -0.31 -0.10 1.13 1.68 0.31 0.73 2.44 0.34 0.65 School K 7.96 * 0.38 0.20 2.79 0.30 0.30 2.77 ~ 0.73 0.45 -0.39 -0.16 0.63 2.13 0.60 0.45 2.56 ~ 1.02 0.54
Affording Degree Completion Source | Data are obtained from college administrative and National Student Clearinghouse records. Notes | ~=p<.10, *=p<.05, **=p<.01, ***=p<.001. Overall (N)=14,226 (completion grant group=2,226; no completion grant group=12,000). Months to graduation is calculated only for students who graduated. Cumulative credits, and months to graduation impacts are derived from multivariate linear regression models; impacts (i.e., difference between treatment and control groups) are reported in relative units. Degree completion and continued enrollment are derived from multivariate logistic regression models; impacts are reported as the difference in predicted probability between the treatment group and control group by way of marginal means. Overall models include college-level fixed effects and control for characteristics not equivalent at baseline. Institution specific models control for the previously mentioned characteristics, only if they exceeded an effect size of 0.05 at baseline (see Table 3). Nine of the eleven universities have no missing data. School D has two students with missing FAFSA information. The remaining missing data are from School B. Spring 2019 results for Colleges F and H were not available as all students in either treatment or control group where either enrolled or completed a degree.
TABLE A-4 | HETEROGENEITY OF ADJUSTED IMPACTS – PELL STATUS Impact Standard (b) Error p-value Outcome: cred_Y1 (N = 14226) Treatment -0.76 0.50 0.13 Pell (base:no Pell) 1.07 0.24 0.00 Pell X Treatment -0.06 0.55 0.91 Outcome: monthstograd_NSC (N = 13111) Treatment -0.38 0.39 0.33 Pell (base:no Pell) 0.03 0.19 0.87 Pell X Treatment 0.19 0.42 0.66 Outcome: gradt2_NSC (N = 14226) Treatment 1.18 0.15 0.21 Pell (base:no Pell) 0.99 0.06 0.83 Pell X Treatment 0.93 0.13 0.62 Outcome: gradt4_NSC (N = 14226) Treatment 1.19 0.24 0.37 Pell (base:no Pell) 1.17 0.10 0.06 Pell X Treatment 0.85 0.18 0.46 Outcome: gradt6_NSC (N = 14226) Treatment 1.60 0.45 0.10 Pell (base:no Pell) 1.17 0.13 0.14 Pell X Treatment 0.75 0.23 0.34 Outcome: engradT2 (N = 14226) Treatment 1.75 0.85 0.25 Pell (base:no Pell) 1.19 0.20 0.29 Pell X Treatment 0.84 0.43 0.73 Outcome: engradT4 (N = 14226) Treatment 1.68 0.57 0.13 Pell (base:no Pell) 1.16 0.14 0.24 Pell X Treatment 0.64 0.23 0.21 Outcome: engradT6 (N = 14226) Treatment 1.86 0.66 0.08 Pell (base:no Pell) 1.08 0.14 0.53 Pell X Treatment 0.66 0.25 0.27
TABLE A-5 | HETEROGENEITY OF ADJUSTED IMPACTS – NEED Impact Standard (b) Error p-value Outcome: cred_Y1 (N = 14226) Treatment -1.45 0.76 0.06 $1,000-3,000 (base: $0-$999) -0.54 0.33 0.10 $3,001-6,000 (base: $0-$999) -0.38 0.29 0.20 $6,001-9,000 (base: $0-$999) -0.06 0.30 0.83 above $9,000 (base: $0-$999) 0.18 0.30 0.55 $1,000-3,000 X Treatment 1.49 0.92 0.10 $3,001-6,000 X Treatment 1.01 0.84 0.23 $6,001-9,000 X Treatment 0.20 0.86 0.81 above $9,000 X Treatment 0.44 0.86 0.60 Outcome: monthstograd_NSC (N = 13111) Treatment 0.21 0.60 0.72 $1,000-3,000 (base: $0-$999) -0.16 0.23 0.48 $3,001-6,000 (base: $0-$999) 0.21 0.21 0.33 $6,001-9,000 (base: $0-$999) 0.15 0.22 0.49 above $9,000 (base: $0-$999) 0.60 0.22 0.01 $1,000-3,000 X Treatment -0.20 0.71 0.78 $3,001-6,000 X Treatment -0.41 0.66 0.54 $6,001-9,000 X Treatment -0.56 0.67 0.41 above $9,000 X Treatment -0.65 0.67 0.33 Outcome: gradt2_NSC (N = 14226) Treatment 0.86 0.15 0.38 $1,000-3,000 (base: $0-$999) 0.99 0.08 0.88 $3,001-6,000 (base: $0-$999) 0.95 0.07 0.43 $6,001-9,000 (base: $0-$999) 1.02 0.07 0.77 above $9,000 (base: $0-$999) 0.92 0.06 0.20 $1,000-3,000 X Treatment 1.45 0.32 0.09 $3,001-6,000 X Treatment 1.42 0.28 0.08 $6,001-9,000 X Treatment 1.15 0.23 0.47 above $9,000 X Treatment 1.33 0.27 0.15 Outcome: gradt4_NSC (N = 14226) Treatment 0.72 0.17 0.16 $1,000-3,000 (base: $0-$999) 0.95 0.11 0.66 $3,001-6,000 (base: $0-$999) 0.91 0.10 0.36 $6,001-9,000 (base: $0-$999) 1.02 0.11 0.83 1
above $9,000 (base: $0-$999) 0.83 0.09 0.08 $1,000-3,000 X Treatment 1.43 0.44 0.24 $3,001-6,000 X Treatment 1.91 0.54 0.02 $6,001-9,000 X Treatment 1.22 0.34 0.48 above $9,000 X Treatment 1.52 0.42 0.13 Outcome: gradt6_NSC (N = 14226) Treatment 0.78 0.24 0.42 $1,000-3,000 (base: $0-$999) 0.78 0.12 0.09 $3,001-6,000 (base: $0-$999) 0.83 0.11 0.17 $6,001-9,000 (base: $0-$999) 0.94 0.13 0.64 above $9,000 (base: $0-$999) 0.82 0.11 0.14 $1,000-3,000 X Treatment 1.74 0.69 0.16 $3,001-6,000 X Treatment 2.30 0.85 0.02 $6,001-9,000 X Treatment 1.29 0.46 0.48 above $9,000 X Treatment 1.59 0.58 0.20 Outcome: engradT2 (N = 14226) Treatment 3.84 3.92 0.19 $1,000-3,000 (base: $0-$999) 1.02 0.27 0.95 $3,001-6,000 (base: $0-$999) 0.70 0.16 0.11 $6,001-9,000 (base: $0-$999) 0.93 0.22 0.77 above $9,000 (base: $0-$999) 0.56 0.12 0.01 $1,000-3,000 X Treatment 0.29 0.33 0.27 $3,001-6,000 X Treatment 0.71 0.78 0.75 $6,001-9,000 X Treatment 0.27 0.29 0.22 above $9,000 X Treatment 0.35 0.37 0.32 Outcome: engradT4 (N = 14226) Treatment 0.83 0.29 0.59 $1,000-3,000 (base: $0-$999) 0.86 0.15 0.37 $3,001-6,000 (base: $0-$999) 0.85 0.13 0.30 $6,001-9,000 (base: $0-$999) 0.91 0.15 0.55 above $9,000 (base: $0-$999) 0.73 0.11 0.04 $1,000-3,000 X Treatment 1.54 0.71 0.35 $3,001-6,000 X Treatment 1.76 0.73 0.17 $6,001-9,000 X Treatment 1.07 0.44 0.87 above $9,000 X Treatment 1.43 0.59 0.39 Outcome: engradT6 (N = 14226) Treatment 1.24 0.51 0.59 $1,000-3,000 (base: $0-$999) 0.80 0.14 0.19 2
$3,001-6,000 (base: $0-$999) 0.83 0.13 0.22 $6,001-9,000 (base: $0-$999) 0.98 0.16 0.88 above $9,000 (base: $0-$999) 0.89 0.14 0.44 $1,000-3,000 X Treatment 0.97 0.48 0.96 $3,001-6,000 X Treatment 2.01 0.99 0.15 $6,001-9,000 X Treatment 0.76 0.35 0.55 above $9,000 X Treatment 0.84 0.39 0.71 TABLE A-6 | HETEROGENEITY OF ADJUSTMED IMPACTs – RACE Standard Impact Error (b) (se) p-value Outcome: cred_Y1 (N = 14226) Treatment -1.12 0.32 0.00 Latinx (base: White) -1.41 0.20 0.00 Black (base: White) -2.55 0.26 0.00 Asian (base: White) 0.50 0.28 0.08 Another race or unknown (base: White) -0.30 0.41 0.47 Latinx X Treatment 0.92 0.52 0.08 Black X Treatment 0.17 0.56 0.77 Asian X Treatment 0.07 0.70 0.92 Another race or unknown (base: White) 1.08 0.83 0.19 Outcome: monthstograd_NSC (N = 13111) Treatment -0.38 0.22 0.09 Latinx (base: White) 0.22 0.15 0.13 Black (base: White) 0.16 0.21 0.45 Asian (base: White) 0.18 0.20 0.36 Another race or unknown (base: White) -0.24 0.25 0.34 Latinx X Treatment 0.17 0.39 0.67 Black X Treatment 0.44 0.45 0.33 Asian X Treatment -0.38 0.51 0.46 Another race or unknown (base: White) 1.31 0.65 0.04 Outcome: gradt2_NSC (N = 14226) Treatment 1.15 0.09 0.07 Latinx (base: White) 0.90 0.04 0.03 Black (base: White) 0.81 0.05 0.00 Asian (base: White) 0.93 0.06 0.28 Another race or unknown (base: White) 0.94 0.08 0.43 Latinx X Treatment 1.02 0.13 0.87 3
Black X Treatment 0.83 0.12 0.19 Asian X Treatment 1.15 0.21 0.43 Another race or unknown (base: White) 0.70 0.14 0.08 Outcome: gradt4_NSC (N = 14226) Treatment 1.18 0.14 0.16 Latinx (base: White) 0.92 0.07 0.25 Black (base: White) 0.68 0.06 0.00 Asian (base: White) 1.30 0.15 0.02 Another race or unknown (base: White) 0.83 0.10 0.14 Latinx X Treatment 0.84 0.16 0.37 Black X Treatment 0.94 0.19 0.74 Asian X Treatment 0.64 0.18 0.11 Another race or unknown (base: White) 0.68 0.19 0.17 Outcome: gradt6_NSC (N = 14226) Treatment 1.49 0.24 0.01 Latinx (base: White) 0.99 0.09 0.95 Black (base: White) 0.68 0.07 0.00 Asian (base: White) 1.29 0.19 0.08 Another race or unknown (base: White) 0.76 0.11 0.05 Latinx X Treatment 0.66 0.17 0.11 Black X Treatment 0.99 0.27 0.96 Asian X Treatment 0.60 0.22 0.17 Another race or unknown (base: White) 0.64 0.22 0.19 Outcome: engradT2 (N = 14226) Treatment 1.61 0.41 0.06 Latinx (base: White) 1.26 0.19 0.12 Black (base: White) 0.98 0.18 0.89 Asian (base: White) 1.51 0.35 0.08 Another race or unknown (base: White) 0.68 0.14 0.06 Latinx X Treatment 0.80 0.36 0.61 Black X Treatment 0.78 0.35 0.57 Asian X Treatment 0.74 0.50 0.65 Another race or unknown (base: White) 2.03 1.58 0.36 Outcome: engradT4 (N = 14226) Treatment 1.13 0.19 0.46 Latinx (base: White) 1.06 0.11 0.60 Black (base: White) 0.74 0.09 0.02 Asian (base: White) 1.36 0.22 0.06 4
Another race or unknown (base: White) 0.56 0.08 0.00 Latinx X Treatment 0.91 0.26 0.75 Black X Treatment 0.93 0.27 0.80 Asian X Treatment 1.08 0.51 0.87 Another race or unknown (base: White) 1.34 0.54 0.47 Outcome: engradT6 (N = 14226) Treatment 1.58 0.30 0.02 Latinx (base: White) 1.06 0.11 0.62 Black (base: White) 0.67 0.08 0.00 Asian (base: White) 1.28 0.21 0.14 Another race or unknown (base: White) 0.71 0.12 0.04 Latinx X Treatment 0.61 0.18 0.10 Black X Treatment 0.92 0.29 0.78 Asian X Treatment 0.68 0.31 0.40 Another race or unknown (base: White) 0.60 0.24 0.19 5
TABLE A-7 | HETEROGENEITY OF ADJUSTED IMPACTS – GENDER Standard Impact Error (b) (se) p-value Outcome: cred_Y1 (N = 14226) Treatment -1.39 0.33 0.00 Female (base:male) 0.13 0.17 0.43 Female X Treatment 1.03 0.41 0.01 Outcome: monthstograd_NSC (N = 13111) Treatment -0.31 0.25 0.22 Female (base:male) -0.75 0.12 0.00 Female X Treatment 0.14 0.31 0.65 Outcome: gradt2_NSC (N = 14226) Treatment 1.14 0.09 0.08 Female (base:male) 1.32 0.05 0.00 Female X Treatment 0.95 0.10 0.59 Outcome: gradt4_NSC (N = 14226) Treatment 1.00 0.11 0.98 Female (base:male) 1.33 0.08 0.00 Female X Treatment 1.10 0.16 0.53 Outcome: gradt6_NSC (N = 14226) Treatment 1.28 0.18 0.09 Female (base:male) 1.35 0.10 0.00 Female X Treatment 0.96 0.19 0.85 Outcome: engradT2 (N = 14226) Treatment 1.13 0.25 0.58 Female (base:male) 1.10 0.13 0.42 Female X Treatment 1.86 0.64 0.07 Outcome: engradT4 (N = 14226) Treatment 1.19 0.19 0.28 Female (base:male) 1.34 0.12 0.00 Female X Treatment 0.91 0.20 0.66 Outcome: engradT6 (N = 14226) Treatment 1.36 0.23 0.06 Female (base:male) 1.42 0.12 0.00 Female X Treatment 0.89 0.21 0.62 6
7
FIGURES FIGURE 1 Program Theory of Change Problem Approach Outcome Funds Eligibility Disbursement Advanced Grants Fall disbursement standing Hurdles to $500- EFC<10k May distribute in Progress $3,000 spring also degree Meet SAP towards comple�on degree Part �me or Automa�c award more In-state resident 8
Figure 2: Student Perceptions of Completion Grant Requirements and Eligibility Source | Survey administered to survey sample during winter 2018-19. Notes | Sample includes all students in treatment group who responded to survey n=1,384 though each question has a different sample size due to missing data and skip patterns. Academics 64% Unpaid balance 7% Need financial aid 65% Requested 15% Keep good grades 85% Stay enrolled 92 Enroll full time 79% Meet with advisor 29% 9
10
1 One institution conducted the pilot study using a loan approach to the completion funds. They agreed to use grants for the fall 2018 term to be part of the study. However, subsequent site visits suggest they may have reverted to the loan model without notifying researchers at the time. This institution also awarded funds substantially later than any other institution and is excluded from some analyses due to these variations from the common model. 2 See Appendix Figure A-1 for a sample of the messaging used for this study. 3 Two institutions offered the grants in spring 2019 due to attrition in the initially eligible sample which allowed them extra funds to disburse – a new sample was pulled for spring 2019 following the eligibility criteria to receive the remaining funds. One institution required a second randomization like the other two institutions but awarded the grant during fall 2018.Two other institutions elected to distribute aid across the two terms as opposed to providing a lump sum in fall 2018. 4 WWC guidelines require that variables with effect sizes between 0.05 and 0.25 be used as control variables in all statistical analyses (What Works Clearinghouse, 2020). 5 We fielded a survey to a subsample of eligible students (n=6,000 in treatment and control groups) in the baseline year (fall 2018) to capture student experiences with grant receipt. The survey obtained a response rate of 65%. 6 There are some subgroups where one or two academic impacts are evident but these are likely subject to type 1 error, as the analysis makes a large number of comparisons (Wooldridge, 2015). 11
TABLE 1 | INSTITUTIONAL CHARACTERISTICS OF PARTICIPATING UNIVERSITIES Average / Percentage Institutional Characteristics Total undergraduates (N) 25,273 Academics 4-year Bachelor's degree completion rate (%) 30 6-year Bachelor's degree completion rate (%) 56 Admissions rate (%) 66 Non-White (%) 53 In-State (%) 53 Region South (%) 45 Southwest (%) 18 Midwest (%) 18 Pacific Northwest (%) 9 East (%) 9 Urbanicity Rural (%) 0 Suburb (%) 9 Small city (%) 0 Midsize city (%) 27 Large city (%) 63 Financials In-State Tuition and fees ($) 9,429 In-State Cost of attendance ($) 25,666 Pell* (%) 40 Federal student loans* (%) 49 Source | Where * is listed data is from 2015-16 College Navigator website; otherwise data is drawn from fall 2016 Notes | Percentages may not add up to 100% due to rounding error. 12
TABLE 2 | PROGRAM ELIBILITY, RECEIPT, AND GRANT SIZE BY UNIVERSITY Received Completion Grant Completion Grant Award ($) Percent Total Served Total Institution eligible (%) Served Average Maximum Overall 14,266 16 2,231 1,232 3,000 Arizona State University 2,446 17 410 985 1,000 Florida International University 832 12 100 1,000 1,000 Portland State University 1,369 9 125 1,586 2,000 University of Illinois at Chicago 3,223 5 146 499 5,000 University of Memphis 342 58 198 2,995 3,000 Florida State University 2,082 9 196 1,000 1,000 Indiana University Purdue 1,010 20 200 995 1,000 University Indianapolis University of North Carolina at 1,068 20 210 1,500 1,500 Charlotte University of Colorado Denver 1,099 18 199 1,000 1,000 Rutgers University - Newark 183 28 51 1,500 1,500 Virginia Commonwealth 572 69 396 993 1,000 University Source | Data are obtained from institution records. Notes | Award amount could not exceed student's unmet need, therefore, a small number of students in the treatment group received award amounts of $0 (contact authors for details). Percent who received a completion grant is of total eligible students. 13
TABLE 3 | BASELINE COMPARISON OF STUDENTS BY TREATMENT STATUS Treatment Status No Completion Completion Effect Size Student Characteristics All Grant Grant Difference Variables Used in Randomization Average percent of degree credits 93 93 93 0.06 completed (%) Enrolled at full-time status (%) 81 81 80 0.03 Took out student loans (%) 58 57 64 0.15 Pell eligible (%) 85 85 85 0.00 Average unmet need ($) 6,183 6,191 6,140 0.01 Average EFC ($) 1,728 1,719 1,776 0.02 Additional Demographics (post-randomization) Race/Ethnicity: Asian (%) 12 12 10 0.10 Race/Ethnicity: Black or African 13 12 17 0.25 American (%) Race/Ethnicity: Latinx (%) 28 29 22 0.22 Race/Ethnicity: White (%) 41 41 43 0.07 Female (%) 55 55 59 0.09 Independent on FAFSA (%) 48 47 51 0.10 Average family income ‘18-‘19 ($) 23,016 22,364 26,531 0.18 Source | Data are obtained from institution’s administrative records. Notes | N=14,226 (completion grant group=2,226, no completion grant group=12,000). The 14
column for “Difference” includes effect sizes calculated according to What Works Clearinghouse standards. Family income is a combination of student income and parent income as reported on the FAFSA. Percent of degree credits completed refers to the percentage of needed credits to graduate from the university attended already completed at the time of randomization. Percent of students who took out student loans refers to the percent of students who took out any kind of federally reported student loan of any amount. Independent on the FAFSA refers to the percent of students who have been determined to be independent for FAFSA filing purposes, and therefore do not need to report parental income. Nine of the 11 universities have no missing data. University of Illinois at Chicago has two students with missing FAFSA information. Rutgers has no information on the independent status of participants. The remaining missing data are from Florida International University. Missing values on continuous variables have been mean imputed. Frequency of missing are as follows: female (23), took out student loans (81), Pell eligible (81), race and ethnicity (210), independent on FAFSA (85), EFC (83), unmet need (81), and percent of degree credits completed (81). Full-time enrollment and family income have no missing values. See Appendix A-1 for differences within each institution. TABLE 4 | PAYING FOR COLLEGE, BY TREATMENT GROUP Mean by Completion Grant (Treatment) Status No Yes Difference Way I Pay for College (%) Government 92.4 93.41 1.04 Grants from university 70.94 84.09 13.96 *** Credit cards 32.9 28.42 -4.54 *** Employer 15.78 15.35 -0.43 Family/Friends 49.19 52.07 2.88 Loans 72.26 74.2 1.94 Savings 48.22 50.17 1.95 Type of Financial Aid (%) Pell 86.57 86.52 -0.06 Grant-State 55.06 59.15 4.1 * Grant-University 59.5 78.28 19.45 *** Loan-Federal 68.02 70.09 2.08 Loan-College 16.19 16.38 0.19 Loan-Private 8.48 10.52 1.99 * Source | Survey administered to survey sample during winter 2018-19. Notes | *=p<.05, **=p<.01, ***=p<.001. Total survey sample includes all students in study 15
sample who responded to survey (completion grant(n)=1,348; no completion grant(n)=2,575), sample size may vary slightly by survey question. Analyses are comparison between treatment and control groups in survey sample estimated with multivariate logistic regression and controlling for variables not equivalent at baseline including: student loan status, race, gender, and FAFSA independence. Results presented as predicted probabilities from marginal means. TABLE 5 | ADJUSTED INTENT-TO-TREAT ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES Mean by Completion Grant (Treatment) Status No Yes Impact SE Months / Credits Credits completed 2018-2019 22.3 22.5 0.15 -- 0.21 Months to graduation 10.7 10.7 0.01 -- 0.17 Marginal Percent (%) Coefficient Mean Completed degree: Spring 66.2 67.5 1.37 0.07 0.06 2019 Completed degree: Spring 88.8 89.1 0.22 0.02 0.08 2020 Completed degree: Spring 93.4 94.4 1.07 0.19 0.11 2021 Completed degree or still 97.2 98.2 1.06 0.42 0.19 enrolled: Spring 2019^ 16
Completed degree or still 94.9 95.3 0.45 0.10 0.12 enrolled: Spring 2020 Completed degree or still 95.0 95.9 0.91 0.20 0.13 enrolled: Spring 2021 Source | Data are obtained from college administrative and National Student Clearinghouse records. Notes | *=p<.05, **=p<.01, ***=p<.001; N=14,226 (completion grant group=2,226, no completion grant group=12,000). However, for the model estimating the impact of completion grants on completed degree or still enrolled outcome in spring 2019, participants at UNCC (n=1,068) were excluded from the analyses as less than 2% of the sample were not enrolled or had completed a degree. Months to graduation is calculated only for students who graduated (n=13,111). Degree completion and continued enrollment are derived from multivariate logistic regression models: impacts are reported in predicted probability by way of marginal means. Cumulative credits, and months to graduation impacts are derived from multivariate linear regression models: impacts are reported in relative units. All models control for college fixed effects, gender, race and ethnicity, dependency status on the FAFSA, family income, percent of credits needed to complete college, and whether student has loans. Nine of the 11 universities have no missing data. University of Illinois at Chicago has two students with missing FAFSA information. Rutgers has no information on the independent status of participants. The remaining missing data are from Florida International University. Missing values have been mean imputed. Frequency of missing are as follows: female (23), took out student loans (81), Pell eligible (81), race and ethnicity (210), independent on FAFSA (85), EFC (83), unmet need (81), and percent of degree credits completed (81). Full-time enrollment and family income have no missing values. TABLE 6 | UNADJUSTED INTENT-TO-TREAT IMPACT ESTIMATES OF COMPLETION GRANTS ON ACADEMIC OUTCOMES, BY INSTITUTION 17
Source | Data are obtained from college administrative and National Student Clearinghouse records. Notes | *=p<.05, **=p<.01, ***=p<.001. Overall (N)=14,226 (completion grant group=2,226; no completion grant group=12,000). Months to graduation is calculated only for students who graduated. Cumulative credits, and months to graduation impacts are derived from multivariate linear regression models: impacts (i.e., difference between treatment and control groups) are reported in relative units. Degree completion and continued enrollment are derived from multivariate logistic regression models: impacts are reported as the difference in predicted probability between the treatment group and control group by way of marginal means. Overall models include for college fixed effects. Spring 2019 results for Colleges F and H were not available as all students in control or treatment groups were enrolled or completed a degree. 18
Cite this document
Sara Goldrick-Rab, Christine Baker-Smith, Travis T. York, Kallie Clark, Douglas Webber, & and Christel Perkins (2023). Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities (FEDS 2023-047). Board of Governors of the Federal Reserve System, Finance and Economics Discussion Series. https://whenthefedspeaks.com/doc/feds_2023-047
@techreport{wtfs_feds_2023_047,
author = {Sara Goldrick-Rab and Christine Baker-Smith and Travis T. York and Kallie Clark and Douglas Webber and and Christel Perkins},
title = {Affording Degree Completion: An Experimental Study of Completion Grants at Accessible Public Universities},
type = {Finance and Economics Discussion Series},
number = {2023-047},
institution = {Board of Governors of the Federal Reserve System},
year = {2023},
url = {https://whenthefedspeaks.com/doc/feds_2023-047},
abstract = {To improve college affordability and graduation rates, universities are increasingly allocating âcompletion grantsâ to students who are nearing the finish line but facing financial challenges. Using an experimental design and common program model across 11 broad-access public universities in ten states, we assessed the impact of a completion grants averaging $1,200 distributed among more than 14,000 students. We find that, despite university expectations that most students were near completion, only two-thirds of students eligible to receive a completion grant graduated within the academic year. Receiving a completion grant did not improve that rate. However, nearly all eligible students (95%) graduated within three years or were still working on their degrees. While completion grants are intended to enhance equity, we do not find evidence that they exerted positive impacts for marginalized groups as designed in this study. Moreover, while there was some program implementation variation across universities, it did not lead to differences in program impact.},
}