The misrepresentation of Oakland crime data

City Administration used flawed data comparisons to claim that total crime has dropped in 2024

On May 1, Oakland Mayor Sheng Thao claimed a “significant and sustained decrease in overall crime”, and attributed it to her administrative actions. News media—including the Chronicle, CBS News, NBC Bay Area, KRON4, KTVU, and others—parroted these claims without scrutiny, citing seemingly dramatic drops in crime in almost every category. 

But that hopeful story is untrue. We cannot know yet if crime is up or down because the year-to-date crime data has not been tabulated fully by Oakland Police. The Mayor and the news media incorrectly compare partially counted 2024 data to fully counted 2023 data. 

We won’t have a clear view of crime trends until the 2nd half of the year at the earliest, and even that mid-year view is subject to substantial error. Moreover, we cannot know the impact of Administration actions that were effected only two months ago, nor can we know if any potential impacts will be sustained. 

This error in data interpretation might be excused if it was made by a member of the general public. But it is inexcusable when made by the City Administration itself. They are the producer of crime statistics, and they have a responsibility to present the truth to the public without bias. Crime often means the loss of property, loss of livelihoods, loss of health, and loss of life for Oakland residents. We owe it to the victims to get the facts straight. And we owe it to our futures. 

If we do not understand the reality of our streets, our policy and electoral discourse degenerates into mythology, conjecture and ideological squabbling. And we will continue to make poor choices with dire consequences for public safety, for the lives of people, and for the sustainability of our City.

---

2023 was a record setting year for crime in Oakland, and residents’ emotions and expectations regarding crime are intense. Elected officials are scanning the latest Oakland Police Department (OPD) crime reports to see how 2024 is unfolding—any change in crime rate could have significant consequences for electoral outcomes in November.

Such politically charged conditions are ripe for misrepresentation or misinterpretation of information. Many residents' daily experiences suggest crime is not getting better.  Yet the Mayor’s office claims crime has dropped due to her actions.

Unfortunately, the Mayor arrives at these claims using a flawed comparison; she contrasts incomplete 2024 crime tallies to complete 2023 crime tallies. Obviously, this will lead to incorrect conclusions.

For example, she cites the incorrect Chronicle article that says all crimes except for robbery have dropped since 2023. But an “apples to apples” comparison of crime data—using data as it was first reported by OPD in 2024 and 2023—suggests otherwise. Assault, vehicle theft, robbery and violent crime are all elevated in 2024 compared to 2023, while burglary, larceny, rape and homicides are down thus far.

The truth is that we just don’t know yet if crime is up or crime is down relative to 2023—at least not based on crime statistics to date. Data are incompletely tabulated. And history shows that burglaries and total crimes are significantly under-counted by ~20-35% at this point in the year (see second figure below). If 2024 follows the same pattern, the apparent improvements in burglary and total crime will evaporate as data becomes fully tabulated.

Why can’t we assess crime trends this early in the year?

Crime data collected through April of this year (or though April of any year) are inadequate to judge year-on-year crime trends. There are two main reasons why.

First, it takes several weeks to tabulate all the crimes that happened during a given week. That means the current-year statistics reported in the weekly OPD crime reports are always incomplete. For example, the total burglaries reported by OPD during week 17 of 2023 (April 17-23) were 3849.  One year later, after all data had been completely tabulated, the final burglary count was 6026 for that same week. This means the initial crime report under-counted burglaries by 57% relative to the original tally or 3849 crimes.

As the year goes on, OPD year-to-date crime totals will gradually become more accurate. This is because the last few weeks of incomplete data become a smaller and smaller portion of the total crime tally. This time-dependent reduction in error is shown below for violent crime in 2017 and 2018. (Errors are reported as a % of the original reported crime tally.)

Looking across the past six years (figure below), we can see that the initial weekly crime report almost always under-counts the true crime. Violent crime appears to be the most accurate, with under-counting less than 10% by May (week 18), and less than 5% by October. (Errors are reported a % of the original crime tally. And COVID years are excluded as unrepresentative of normal City operations.)

The least accurate counts are burglaries which tend toward 10-20% under-counts in April/May, and in recent years have been under-counted by 20-50%. This suggests that tabulation is fastest for violent crime, because it is prioritized, while non-violent crime like burglary takes longer. 

Given that burglaries are more than 35% of total crime, this also has a significant impact on the under-counting of total crime. Total crime has been under-counted by 20-35% as of April/May in recent years.

Thus the Mayor’s judgement of year-to-date crime is almost certainly an underestimate of true 2024 crime by 20-35%. The Mayor and news media claimed that total crime has fallen 33% year to date—which is consistent with the historical under-counting error. And it is very likely the claimed crime drops would disappear if full data was available. 

The second reason that we cannot yet assess crime trends is because they are not constant throughout the year. In 2023, we had an explosion of crime starting around April. Thus, even if our OPD weekly crime reports were perfectly accurate, we would not know yet if crime rates had sustainably shifted. 

Finally, we note that the under-counting and crime variability issues do not even consider that residents are under-reporting actual crime. It is commonly discussed in Neighborhood Councils, personal communications, social media and surveys that businesses and residents do not even report many crimes because they cannot reach 911 dispatch, or they never get any response from police even if they do reach dispatch. And increasingly, businesses are also avoiding crime reports for fear of losing insurance.¹

Because the magnitude of such under-reporting is unknown, we cannot assess if it has a significant impact on the measured crime data in present or past years. Thus we cannot incorporate our knowledge of under-reporting into the assessment of year-on-year crime trends. We can, however, recognize that under-reporting is a significant blind spot in our understanding of reality—and therefore an obstacle to sound policy making and City administration. It would be worthwhile for the City to recognize this issue formally, and consider how to resolve it.

It is still possible that trends will improve this year, regardless of the counting error, if crime truly does get better. But as noted in the “apples to apples” comparison of partial data from 2023 and 2024, the trends thus far are not encouraging. We can only hope for improvements soon.

How can we understand the root causes of crime fluctuations?

Even if we knew the true City-wide crime trends right now, those data alone are not sufficient to identify the root causes of changes. And short-term City-wide trends should not be used to claim causality of particular administrative actions. 

As decades of rigorous crime research make clear, including Oakland’s own assessment of violence prevention programs, root cause assessment requires establishing matched control groups in carefully designed studies. But such studies take commitment, patience, and humility—particularly because they often do not confirm pre-conceived speculation. 

Most certainly—given the unknowns of actual crime trends and the City’s lack of controlled rigorous assessments—we cannot know that specific City administrative actions taken in the past 2 months ago are delivering reductions in crime as the Mayor claims they are.

So what are policy makers and residents to do?

The first thing we can do is have the discipline to avoid reacting to incomplete data. And more importantly, we can avoid drawing policy conclusions prematurely. That only hurts the lives of residents and the City’s future. 

Second, we can speed up the data verification and tabulation. Publicly available information does not indicate how long it takes OPD to fully tabulate the data. But we do know that timeline can vary depending on the availability of clerical OPD employees to verify and resolve crime reports. OPD is presently under-staffed in civilian employees; for example, their sole data analyst left the department early in 2023, and that position has not yet been filled. Good data requires sufficient analytical resources.

Third, while OPD is working through its backlog of crime reports, they can include a tally of pending reports that have not yet been verified and tabulated. While this number will also change with time as more delayed reports filter in and some reports are rejected or duplicates, it could still provide a clearer view of the present reporting error. This information could help the public and leaders understand the reliability of preliminary crime reports.

Fourth, we can recognize that determining root causes of social problems, and the effectiveness of social policies, is a complex undertaking. We cannot determine root causes by a few data points from a single crime report. We must first deeply consider lessons learned from decades of criminology and policing. And when we introduce operational changes or test novel public safety approaches, we must pair them with rigorously planned impact assessment studies. Such studies also need to be conducted by unconflicted third parties who will not benefit or suffer one way or another from the conclusions.

All of these changes require that we prioritize a long term commitment to truth above any personal or Political aim. If we prioritize truth and transparency, we can find lasting solutions.

Appendix

Table 1: Error in crime counts for week 17, 2023 (April 17-23). Error is the difference between what was originally reported by OPD for a given week compared to the crime tally for that same week reported a year later.

Tags: Policing, Public Safety

1.  See public comments to City Council on 25 September 2023 by Salah Elbakri, Owner of Concise Insurance Services. Cue to 03:15:25 of video: https://oakland.granicus.com/player/clip/5692?meta_id=513617