SBAC Results

Why SFUSD's proficiency rates are not as impressive as it claims

The 2022 SBAC scores were released last week. Here’s how SFUSD’s press release positioned it.

In 2022 the San Francisco Unified School District’s (SFUSD) ELA and math proficiency rates continued to be higher than the state, which was also the case in 2016, 2017, 2018, and 2019.

The underlying data shows that SFUSD’s proficiency rate was 54.5% in ELA and 45.6% in Math, higher than the statewide averages of 47.1% in ELA and 33.4% in Math. If we average the two tests, that’s 50% in SFUSD compared with 40.2% statewide. So the claim is correct. The reader is obviously meant to come away believing SFUSD is doing great. Left unstated is the fact that this outperformance is entirely due to the fact that we have far more Asian students than the average district.

A fairer comparison would compare like with like. How do socio-economically disadvantaged (SED) Latino students do in San Francisco compared with other districts? How do Asian students who are not socio-economically disadvantaged (Asian non-SED) students do in San Francisco compared with elsewhere?

Figure 1 shows the proficiency rates in SFUSD compared with the state averages for select demographic groups. The blue discs represent the proficiency rate in SFUSD for each of the demographic groups shown and the orange discs represent the statewide proficiency rate. The proficiency rate shown is the average of the proficiency rates for ELA and Math. The size of the discs represents the demographic group’s share of the total student population. SFUSD’s proficiency rates are higher than the state averages purely because it has far more Asian students than the state average. Latino SED students, who tend to have low proficiency rates, are by far the largest group in California schools. Asian SED students, who tend to have much higher proficiency rates, are the largest group in SFUSD.

Figure 1: Proficiency Rates by Demographic Group

If the blue disc is above the corresponding orange disc that means that students from that demographic group do better in SFUSD than elsewhere in the state. We can see that four groups outperform state averages in SFUSD: White (both SED and non-SED), Asian SED and Two or more races non-SED. Together these account for 41.7% of SFUSD’s students. The other twelve groups¹, comprising 58.3% of students, underperform. A few other points:

• The group that does comparatively worst in San Francisco is that of Filipino SED students. Only 36% are proficient in San Francisco compared with 52% statewide. I am sometimes asked why Filipino exists as a group separate from Asian and the very different proficiency rates provides a good reason.

• Latino and Black students, both SED and non-SED do worse in San Francisco than elsewhere in the state. Only 17% (22% in ELA; 12% in Math) of the biggest group, Latino SED students, are proficient compared with 25% (32% in ELA; 18% in Math) statewide.

• Asian non-SED students are the highest performing group in San Francisco but they actually underperform the state average. Meanwhile, Asian SED students here outperform the state average.

• The biggest outperformance is by White non-SED students. They average 75% proficiency (80% in ELA and 69% in Math), compared with 63% (69% in ELA; 57% in Math) statewide.

I believe the underperformance of Asian non-SED students and the overperformance of White non-SED students can both be explained, at least partially, by parental education which is known to correlate with student achievement. The proportion of White adults with college degrees and postgraduate degrees is higher in San Francisco than in any other county while the proportion of Asian adults with college degrees is lower than in many other counties. I don’t have a pet theory to explain why Filipino SED students do so poorly in SFUSD.

The Achievement Gap

It is a fact that girls do better than boys in school. Ask yourself why that is. Your answer will almost certainly begin “Girls are…” or “Boys are…”. Phrasing the question as why do girls do better than boys invites an answer that generalizes about girls or generalizes about boys. Similarly, when we ask why do Latino and Black students do so much worse than Asian and White students in San Francisco, we invite answers that generalize about those groups. But whereas speculation about gender differences can be done freely, speculation about proficiency differences between racial or ethnic groups is fraught with danger. It should be avoided.

A much better approach would be to observe that Black students, say, do much worse in SFUSD than in other districts and ask why that is. Since we’re comparing Black students with other Black students, it’s almost impossible to produce a race-based explanation. It doesn’t make it any easier to decide on the answer - if the answer was obvious, the district would already have acted on it - but at least the discussion could focus on the right things (such as what the district should do or should stop doing) and there’s less risk of accidentally offending someone.

Proficiency Rates By Demographic Group

In that spirit, let’s look at how various demographic groups do in SFUSD and other districts. In order to have plenty of Bay Area districts in the comparison, we’re going to look at the 200 largest school districts. I’m only going to show a few of the groups and comment on a few of the districts but you can explore the data for all districts and all groups here.

Latino SED students

Figure 2 shows the average proficiency rates of socioeconomically disadvantaged (SED) Latino students².

Districts from Bay Area counties are shown in different colors. I also use purple to make visible three districts that I’ve labeled high-performing districts in previous posts, namely ABC Unified and Long Beach Unified in Los Angeles County and Clovis Unified in Fresno County. The horizontal axis shows the percentage of students in the district who are Latino and SED. The vertical axis shows the proficiency level of those students (I’ve again averaged the proficiency rates in ELA and Math) and the size of the disc represents the number of Latino SED test takers in the district.

• SFUSD is the small red dot in the bottom left. Latino SED students do much better in most other districts than they do in San Francisco. Of the 200 districts shown, San Francisco ranked 187th with only 17.3% of its Latino SED students proficient.

• Bay Area districts in general do poorly. Several were even worse than San Francisco, including San Mateo-Foster City, Oakland, Antioch, and West Contra Costa. (with West Contra Costa being the worst). Our near neighbor, South San Francisco Unified, was at the state average of 25%. Berkeley was best of the Bay Area districts at 33% but it had only 500 Latino SED test takers.

• Looking further afield, many districts do a much better job than SFUSD. Clovis in Fresno County stands out. It had more Latino SED test takers than SFUSD (5509 vs 4479) but 39% of them were proficient (49% in ELA; 30% in Math). Kings Canyon, also in Fresno County, was nearly as good (37% proficient) even though 77% of all its students were Latino SED students. In other words, the level of family poverty is not an excuse for SFUSD’s low scores.

• There's basically no relationship between how Latino a district is (i.e. the share of students who are Latino) and how well those students do on the SBAC tests.

Latino non-SED Students

Latino students who are not socioeconomically disadvantaged (Latino non-SED) do substantially better than Latino students who are disadvantaged. Simply not being disadvantaged raises the statewide average proficiency rate from 25% to 41.5%. The proficiency rate in SFUSD improves from 17.3% for the SED students to 36.5% for the non-SED students. That’s still well below the average but it’s not as egregiously bad as it is for the Latino SED students.

Figure 3: Proficiency Rates of Latino students who are not socioeconomically disadvantaged (SED)

Bay Area districts do a better job with these students. Districts that do much better than the statewide average include not just Palo Alto and Berkeley but also Oakland. Meanwhile, Hayward Unified, Superintendent Wayne’s old district, is one of the worst in the state for these students.

SFUSD likes to benchmark itself against Long Beach because Long Beach is a big urban district whose Latino students do much better than SFUSD’s. But it’s interesting to see that Long Beach’s Latino non-SED students actually do a little worse than the statewide average. Clovis and ABC, on the other hand, do much better than the statewide average.

Asian SED students

Socioeconomically disadvantaged Asian students are the largest single demographic group in SFUSD and they actually do a little better in San Francisco than elsewhere.

Figure 4: Proficiency rates of socioeconomically disadvantaged Asian students

Some observations:

There is enormous variability across Bay Area districts. There are some that do much better than the state average (Cupertino Union, 83%; Milpitas and Castro Valley, both 68%) and some that do much worse (Hayward, 35%; Oakland 45%; Berkeley 50%).

The district with the greatest number of Asian SED students is Elk Grove in Sacramento County and those students do substantially worse than the average. Indeed, other districts in Sacramento County (Sacramento City, Twin Rivers, San Juan) also have sizable numbers of Asian SED students who do badly.

Long Beach, SFUSD’s favorite benchmark, actually does worse than average with its Asian SED students while Clovis, which did so well with its Latino students, only does average with its Asian SED students. ABC, and even giant Los Angeles Unified, continue to do better than SFUSD.

Asian non-SED students

What’s really striking here is that most of the districts that have large concentrations of Asian SED students can be found in the Bay Area and they all do really well with those students.

Figure 5: Proficiency rates of non-disadvantaged Asian students

15% of San Francisco’s students are Asian but not disadvantaged. They do slightly worse in SFUSD than elsewhere in the state (76% vs 82%).

Long Beach is one of the districts where Asian non-SED students do poorly (only 67% average proficiency) and Clovis is also a little below average (but still better than SFUSD). Meanwhile, ABC continues to do very well (89% proficient) and Los Angeles Unified is also a bit better than average (83%)

White non-SED Students

White students who are not socioeconomically disadvantaged comprise 11% of SFUSD’s test takers, making them the fourth largest group. SFUSD does comparatively well with these students (80.2% are proficient in ELA and 69.3% in Math compared with 69.4% and 56.6% statewide). It even does a little bit better than the three high-performing districts (Clovis, Long Beach, ABC) but it does a little bit worse than Oakland (who would have predicted that?), Berkeley, Palo Alto, and Cupertino. In fact, most Bay Area districts also do above average with these students, probably because of the high education levels of White parents in the Bay Area.

Figure 6: Proficiency rates of non-disadvantaged White students

Black SED Students

SFUSD’s Black SED students do particularly poorly with an average of 10.7% proficiency (14.4% in ELA and only 6.9% in Math). Only 2% (18 out of 908) of students were rated in the highest Above Standard category in Math.

The districts with the highest scores for this group tend to have vanishingly small numbers of students (for example, Cupertino had only 31 test takers). But ABC and Clovis combined have more Black SED students than SFUSD and an average proficiency rate of around 35% compared with SFUSD’s 10%. Black students are perfectly capable of doing much much better than they do in SFUSD. Districts that have historically had a Black identify, such as Compton and Inglewood in LA, also have more Black SED students than SFUSD and much higher proficiency rates too.

Figure 7: Proficiency rates for socioeconomically disadvantaged Black students

Summary

As we’ve seen, even when we look at a single demographic group, such as Latino SED students, the proficiency levels vary enormously from district to district. As we’ve also seen, one district may have some groups doing above average and other groups doing below average. Long Beach does much better than the statewide average and SFUSD with Latino SED students but much worse than SFUSD and the statewide average with Asian SED students.

We can now test the claim made at the top, namely that SFUSD’s above average results were entirely due to the fact that we have more Asian students. Suppose the rest of the state had the same demographics as SFUSD (i.e. 19.8% Asian SED, 19.0% Latino SED, 15.2% Asian non-SED etc.) but the performance of each demographic group was unchanged (e.g. the statewide average proficiency for Latino SED students stays at 24.9% and the statewide average proficiency for Asian SED students stays at 55.6%). By taking a weighted average, we can compute that the the statewide average across all students would be 51% proficiency, which is higher than SFUSD’s 50%. In other words, SFUSD is indeed a slightly below average district.

We can extend this to individual districts and calculate which districts would have the highest and lowest overall proficiency rates if they all had SFUSD’s demographics. Figure 8 shows the result

Figure 8: Adjusted Proficiency Rates Assuming SFUSD Demographics

We can see again that SFUSD (labeled “SF” on the chart) sits right in the middle of the range. Long Beach (“LB” on the chart) is right next to SFUSD on the chart. It’s better than SFUSD with some demographic groups but worse with others and those cancel each other out³. SFUSD might be better benchmarking itself against ABC, Clovis, Glendale, and Placentia-Yorba Linda which have roughly the same SED rate but much higher adjusted proficiency rates.

There’s an obvious negative correlation between SED rates and proficiency rates. The highest adjusted proficiency rates can be found in districts, such as Los Alamitos and Palo Alto, that have very few SED students. The lowest adjusted proficiency rates can be found in districts with many SED students. Effectively, everyone does worse when there are more SED students in a district. This may be because teachers adjust the pace of the classes to reflect the abilities of the average student and the average student in a low SED district is better than the average student in a high SED district. Or it may be because there are factors (such as parental education) affecting student achievement which are not captured by the simple adjustment I’ve done. In any case, it definitely means that we should not blindly claim that the highest scoring districts are “the best”.

Next Up

I’ll look at the claims SFUSD made about which schools were high-performing. In the meantime, here’s another link to the vizzes shown above.

1. I left the two Pacific Islander and American Indian groups off the chart because they are the smallest groups and the chart is already a bit busy.

2. Los Angeles Unified actually had 142,000 Latino SED test takers but the chart caps their number at 30,000 in order to enhance the visibility of all the other districts. Otherwise, there’d be one enormous disc, representing LA, and lots of microscopic dots representing all the other districts.

3. This is true only because we’re using SFUSD’s demographics for the comparison. If we were comparing the two districts using Long Beach’s demographics (or the statewide average demographics) as the base, Long Beach would score much higher than SFUSD because it does much better with the group that would be weighted much higher i.e. Latino SED students.