The Utah Air Quality Reporting Gap (Updated)
As I sit here at my desk in Salt Lake City, a look out the window hints at the problem with our air. A brown haze sits low around the foothills and the high-rises downtown are obscured by a thick muddy soup. During the city’s famous winter inversions, we can see some of the worst air pollution in the nation along with our southern neighbor Provo and other Utah cities.
The Utah Department of Environmental Quality reports on current conditions. Due to the health risks posed by high levels of pollution, these numbers are also reported to the EPA and it is important to get them right. When concentrations reach certain levels — at risk groups like children, seniors, and people with health issues should remain indoors to minimize their risk.
Numbers Problem
There appears to me to be problems with the data UDEQ is reporting.
Let’s start by looking at a current trends graph for Utah county covering December 31 to today.
Can you see what’s wrong here?
There are concentration measurements falling below zero on January 1. How is it possible to have a measurement of particulate density that is a negative number? Density measures how much of something is in a given volume. You can have any number of jelly beans in a jar, but your jar can never have a negative amount of them. In this case, it is clear that UDEQ’s numbers are wildly innacurate.
Viewing the source of that page will show you the actual reported PM2.5 concentrations used in the graph:
Yes, UDEQ is reporting that Utah County had a PM2.5 concentration of -75.6 μg/m3. If the actual PM2.5 concentration was its lowest possible value (zero), then we can conclude that the sensor’s reading is inaccurate by at least 75.6 μg/m3.
This seems like a smoking gun — or maybe I should say it’s a gun emitting negative quantities of smoke.
This inaccuracy is carried through to the numbers the EPA has. Here is the 2014 data for Salt Lake County:
Notice how the scale goes to -10 to accommodate all the numbers reported below zero. If you download the actual data from the EPA website, you will see negative numbers reported there.
Just for fun, let’s compare this to another city, say… Dallas, TX.
You can see almost immediately that the Dallas data does not include any negative numbers, and has a much more realistic spread of measurements. Everything in Salt Lake County hangs around zero. This strikes me as unlikely.
At this point it seems inescapable that we must conclude that UDEQ’s numbers can’t be trusted. This poses a problem, seeing as this is where just about everyone in Utah gets their Air Quality information from and there are potentially serious health risks involved.
Alternative Data
Enter purpleair.org, a grassroots organization dedicated to monitoring air quality in Utah due to the lack of comprehensive (and apparently reliable) data. They’ve deployed their own network of sensors all around Utah to monitor particulate concentrations. Take a look at Utah’s data for Salt Lake County for the last two days:
Now look at purpleair.org’s data for their own sensor nearby:
UDEQ’s numbers top out just over 40, while purpleair.org’s numbers hit 100. UDEQ reports that the last two days were “moderate” while purpleair.org’s measurements put concentrations mostly in the “unhealthy” zone.
Is this an isolated incident?
Here is a map of the Wasatch range. It’s a bit hard to tell the difference, but the round pins are showing purpleair.org’s AQI (a number describing many types of pollution put together) and the square pins shows UDEQ’s AQI number.
In Ogden we have purpleair.org reporting an AQI of 158 while UDEQ reports 76.
In Salt Lake City purpleair.org reports an AQI of 165 while UDEQ reports 80.
In Provo it’s the same — purpleair.org reports 164 while UDEQ reports 74.
UDEQ’s numbers are consistently less than half of what independent sensors are reading. To be fair, note that UDEQ’s readings are delayed up to two hours which may explain some of the discrepancy. The trends posted above show it better.
For your health
This is no small difference. AQI’s of between 51–100 are considered only moderately dangerous and restrictions only apply to groups of very high risk. AQI’s of over 150 are considered unhealthy for all groups of people. You can see the full guidelines for AQI here.
So UDEQ is telling Utah residents and the EPA that conditions are moderate while independent sensors are telling us we’re all the way up in the unhealthy range. So are we breathing clean air or not?
Are we breathing clean air or not?
It is certainly possible that purpleair.org’s sensors are wildly innacurate. They have, however, each been tested for consistency. So if they are inaccurate, they are all exactly the same amount of inaccurate.
The fact that UDEQ is already reporting negative PM2.5 concentrations does not inspire confidence in their number and neither does their habit of completely deleting spikes in PM2.5 concentration from their data.
To be clear, I’m not saying that UDEQ is being intentionally misleading in their reporting (though if one were to venture in that direction it wouldn’t be hard to imagine a motive), and I am certainly not qualified to say one way or the other.
But something is clearly wrong with the data being presented to Utahns and, given its importance and potential health implications, it deserves further investigation by someone more qualified. Then we can relax and breathe easy — or know to stay indoors if we can’t.
I have reached out to UDEQ and the U for their input on this article.
Update 1/5/16: A user on reddit commented on a link to this article with what might give an insight into why Utah’s numbers might be so inaccurate:
I actually do the hiring for DEQ which includes DAQ. Honestly, just trying to hire people who can accurately run and troubleshoot these machines is really difficult. You have to find people with years of experience with complex hardware and software willing to work for way under market. Calibration issues do happen with these machines and DAQ does their best to address them.
I think the article insinuating that they are intentionally manipulating the data is really upsetting. The people who work there do a pretty thankless job for way below market wages because they really believe in what their doing. It’s hard to watch someone who doesn’t seem to be very familiar with the work slander a whole lot of good people.
Edit 1/8/16: Facebook Janet Roubik Kinneberg commented on a link to this story with a response from UDEQ. Here is the unedited response as posted by Kinneberg:
“Thanks for asking, Janet Roubik Kinneberg. DAQ uses continuous air monitors that must to meet strict EPA siting and operational requirements. Because the monitors are recording data minute-by-minute, any reduction in the weight of the particulates in the filter from evaporation (moisture) or volatilization can cause the weight of the particulates in the filter to drop relative to earlier measurements. This results in “negative” particulate mass relative to previous measurements. Because we post data hourly and calculate data every six minutes, these negative numbers are calculated in context and are factored into the 1-hour and 24-hour numbers we use to compare our readings with the national ambient air quality standards (NAAQS). In fact, EPA expects negative numbers to appear in the monitoring data. Purpleair.org use sensors rather than filters to measure particulates. These sensors may or may not be accurate, but most importantly, they are not subject to the same siting requirements (placement can affect accuracy of the readings) and aren’t subject to the same quality control measures required by EPA. We are always interested in providing the most accurate data possible, but do not know enough about the methods and types of sensors in use by purpleair.org to make any determination as to the accuracy of their data. If you’d like to learn more about EPA monitoring requirements, check outhttp://cfpub.epa.gov/oarweb/mkb/faq.cfm. The South Coast Air Quality Management District has an interesting web page about portable sensors. You can find that at http://www.aqmd.gov/…/2014-news-archives/air-quality-sensors. Hope this helps! Thanks again for reaching out to us.”
Update 1/22/16: Adrian Dybwad is the founder and operater of the purpleair.org project. He reached out with some useful information about the accuracy of their sensors:
Found a contributing factor to high readings, but it only accounts for some. For instance, on your graph in the article, the average of 100 might become something like 70 but it is still nearly double the state numbers.
This issue only affects numbers over 40 or so as follows:
Numbers in brackets would be the new values.PM2.5: 199.15µg/m3 (131.95)
PM2.5: 173.10µg/m3 (114.25)
PM2.5: 152.29µg/m3 (100.43)
PM2.5: 120.91µg/m3 (82.55)
PM2.5: 107.68µg/m3 (72.32)
PM2.5: 90.45µg/m3 (62.09)
PM2.5: 75.50µg/m3 (55.75)
PM2.5: 58.70µg/m3 (48.60)
PM2.5: 43.35µg/m3 (39.95)
PM2.5: 37.05µg/m3 (35.75)
PM2.5: 30.85µg/m3 (30.85)
PM2.5: 14.80µg/m3 (14.80)
PM2.5: 10.45µg/m3 (10.45)
PM2.5: 7.45µg/m3 (7.45)
PM2.5: 3.36µg/m3 (3.36)More info to come…
