astro1234
14 hours ago
As a data scientist it is very frustrating to see a consistent lack of error bars on these numbers or any discussion about the magnitude of uncertainty around them by all news reports every time these numbers get released
mrloopex
13 hours ago
This should have nothing to do with data science but basic math literacy. Quantities without context are meaningless. Reporters should know better but if when headlines have to catchy day in day out … shrugs. Stock market reporters are the worst: a 1000 point drop in the Dow meant a lot more when it was 7000 than it does at 40,000.
astro1234
11 hours ago
Oh my god do not trigger me about the news reporting “points” for the DOW or the S&P, I have seriously considered calling news stations about this.
Or not providing inflation adjusted values when referring to historical financial reference points.
I have to go find a happy place now
mjburgess
13 hours ago
If honest, the error bars would be so large as to make drawing conclusions from these numbers seem pointless. Certainly, both jobs growth and jobs loss are within the CI.
What's actually being tracked by people using this data is the delta from previous years -- which, conditioning on the same collection methodology etc -- is more reliable.
The actionable piece of info is sign(delta)
rented_mule
7 hours ago
Something the best data scientist I ever worked with was adamant about... there is almost never an inherent "speed bump" at 0. If error bounds expand on both sides of 0, we have no idea if the true sign is positive or negative. There is no "directional" read in that case. There are some areas where 0 has a special meaning, but that's rare and the cause of that needs to be understood before assuming that's the case. Everywhere else, 0 is just another point on the number line.
> the error bars would be so large as to make drawing conclusions from these numbers seem pointless
That is often the case, and being truly data based requires acknowledging that. We can then act based on other data, or admit that we're falling back to intuition and abandoning data based decision making for now.
europoorean
14 hours ago
As a physicist, I always shed a tear when looking at data without error bars.
As a data scientist/generic programmer now, I don't really see them anywhere. I remember a colleague during my postdoc saying 'don't worry, no error bars in industry' when I asked about regression with ML lol
natsucks
13 hours ago
hah. it's true. no interest in uncertainty except when forced.
Noaidi
13 hours ago
Our fake economy needs even more fake certainty now. They are all hiding from the obvious issue of stagflation because it is the most difficult economic problem to solve. The Fed know this which is why they did nothing with rates.
biophysboy
13 hours ago
The bureau of labor statistics publishes confidence intervals in their reports
jmalicki
13 hours ago
Not in their primary news release. A footnote says it is "generally conducted at the 90% interval", but the individual numbers do not have their actual computed error bars.
Yes, they do publish them, but it is disappointing their primary releases do not have them.
biophysboy
13 hours ago
It’s common in scientific/stats publishing to have the headline state the avg/median, the main text plot the errors, and the supplementary give the explicit numbers. They have a big page about std errors.
jmalicki
12 hours ago
The main text doesn't plot the error bars, and the full data tables in their release don't have error bars either.
You have to really dig into their website to find their std errors.
biophysboy
12 hours ago
bls.gov/ces is the main page for this survey and has a plot.
The error updates annually. Their handbook has a Reliability section where they detail sampling/nonsampling error.
throwaway27448
13 hours ago
NBC wouldn't know how to navigate bls.gov if you put a gun to their head. And it's all downhill from there
biophysboy
13 hours ago
Based on the discussions in this thread, they aren’t alone
throwaway27448
13 hours ago
I don't think most of the people in this thread are journalists
biophysboy
12 hours ago
God help us if they ever become one
astro1234
12 hours ago
Yes! I am aware of this and they are really great. Problem is nobody will ever read that, but they WILL watch Fox News, CNBC, etc. and those outlets never even point out that there are errorbars/uncertainty. Then I hear things like outrage when BLS revises numbers up or down (facepalm).
renegade-otter
14 hours ago
The numbers are as good as they can get with hard manual work that goes into compiling them.
The problem is that small businesses lag by 1-2 months reporting them, so the initial number being reported is almost always irrelevant, as it mostly applies to numbers coming out of larger corporations.
jstanley
14 hours ago
> The numbers are as good as they can get
That doesn't mean they don't warrant error bars.
Error bars aren't there to show that you didn't do a good job collecting the data, they're there to show your level of uncertainty.
vlovich123
14 hours ago
I would say the initial numbers should be embargoed until the number is reliable. People don’t understand error bars and reporting can more easily manipulate the current situation without outright lying.
theptip
13 hours ago
No, just report them with error bars. We know the 95th percentile revision size of the distribution.
Moot point as the people who actually act on the data understand how to roll their own anyway.
jfengel
12 hours ago
Error bars would be misleading, because this isn't a sampling error. These start with actual counts. The more important errors are systematic: the definitions of terms, the reporting mechanisms, etc.
Except that those things are constant from month to month, so deltas largely cancel them out.
The problem isn't in the size of the possible errors, but in understanding what those numbers mean and how they are gathered. That requires far more background reading.
twosdai
13 hours ago
Both? I don't think error bars and waiting to release data until more accurate are mutually exclusive.
vlovich123
13 hours ago
That’s what I meant. There should be error bars to reflect uncertainty of the inherent data, but they should wait until all the data that could change things comes in rather than using projections which always drastically change
jmalicki
13 hours ago
No, they shouldn't. Timely estimates as to the best knowledge of our current situation is extremely important for decision making under uncertainty.
If the error bars are large, that is important information to know.
vlovich123
13 hours ago
What decisions are so important and time sensitive they depend on these numbers right after a month is over? And also we at one point were making decisions without these numbers yet the quality of the decision making isn’t clearly improved.
jmalicki
12 hours ago
"What kind of stocks should my mutual fund invest in?"
"What sort of startups should I fund?"
"Should my business aggressively hire this quarter anticipating growth, or hold out for a possible recession?"
vlovich123
11 hours ago
A few months of extra no data won’t impact the stock investments if no one has that data. Startup funding is similarly not so time sensitive.
As for businesses hiring, they should hire based on their own growth. Using an external signal that itself gets adjusted drastically is similarly problematic; I don’t hire because I think no one else is hiring and the numbers get majorly adjusted up. Similarly I hire a bunch of people and it turns out no one else hired.
No one benefits from faster numbers that are majorly shifted in a few months anyway. You wouldn’t make any different decision if no one had the numbers. The only thing you’re doing differently is when the numbers come out your gambling whether or not you think the true number will be revised in some major way in a few months. That’s not helpful for the economy writ large.
win311fwg
13 hours ago
> If the error bars are large, that is important information to know.
The data is available. If it were important to the causal reader, however, then those casual readers would have already demanded its presence. But they don't care. What difference does it make to Hilary Homemaker? She just wants to get to say "Did you hear about the bad economic news?" when gossiping with her friends.
Those who need to understand the uncertainty will already be looking at more than a news headline anyway.
parineum
13 hours ago
Ah yes, lie to them and withold information because the public is too stupid to understand. Then, when they don't trust you anymore, throw up your hands and say it's not your fault.
jmalicki
13 hours ago
That's true for the establishment survey, but the population survey, where they contact individuals, is what is actually used to calculate these numbers.
It is literally calling/writing 60,000 people and asking "are you employed? did you look for a job? Etc."
"How the government calculates unemployment" [0]
| Does anyone in this household have a business or a farm? | Last week, did you do any work for (either) pay (or profit)? | If the answer to question 1 is "yes" and the answer to question 2 is "no," the next question is: | Last week, did you do any unpaid work in the family business or farm? | For those who reply "no" to both questions 2 and 3, the next key questions used to determine employment status are: | Last week, (in addition to the business) did you have a job, either full or part time? Include any job from which you were temporarily absent. | Last week, were you on layoff from a job? | What was the main reason you were absent from work last week? | For those who respond "yes" to question 5 about being on layoff, the following questions are asked:
amanaplanacanal
13 hours ago
You have to wonder, since they are doing two different surveys trying to measure the same reality, which one more accurately measures the facts on the ground? Asking businesses how many people they are employing, or asking households whether or not their folks are working? The numbers don't really match up between the two.
jmalicki
12 hours ago
This is very common for economic statistics.
GDP too. That's how you can get more accurate information, and you have actual numbers on the discrepencies.
They rely on the household data more because it doesn't have as much systemic error, even though it does have more sampling error. Asking businesses gives you a more complete picture, but it has much more bias.
Hell, this is why even in accounting, you have double-entry bookkeeping, and even there, the numbers don't always match up.
Or "my bank account doesn't match up with my cashflow statement".
It's not that one is better and you should use that, calculating multiple ways is central to accounting so you can detect errors.
jm4
13 hours ago
What makes you think it's small businesses? I would assume most businesses doing any reporting at all are using a payroll system. Mine handles the reporting automatically. The kinds of payroll systems used by small businesses can be set up in a day. If anything, small businesses that make even a small effort to stay on top of things have an easier time than big ones with complicated systems and processes.
pgwhalen
13 hours ago
I think you say “as good as they can get with [the] hard manual work that goes into compiling them” as a way to respect the challenge of the BLS and say they’re doing the best with their resources, but it’s worth pointing out they could be better with more resources.
jmalicki
13 hours ago
DOGE hurt them pretty hard: "The Bureau of Labor Statistics posted a notice on Wednesday stating that it stopped collecting data in three not-so-small cities (Lincoln, Nebraska; Buffalo, New York; and Provo, Utah) and increased “imputations” for certain items"
https://www.cnn.com/2025/06/05/economy/cpi-data-bls-reductio...
cousinbryce
13 hours ago
https://www.bls.gov/jlt/jltreliability.htm Read the docs
astro1234
12 hours ago
Oh I’m well aware the BLS knows what they’re doing and publish error bars, they are great. The problem is news articles, no one ever reads BLS reports but they do watch Fox News, CNBC, etc. and those outlets never ever get into uncertainty.
seanhunter
12 hours ago
The actual source this comes from has a 90% confidence interval in the published number. When people talk about the economy gaining or losing jobs they are talking about the bureau of labor statistics "nonfarm payroll" number. The methodology and a lot more are on the bls website
anigbrowl
11 hours ago
Wait until you see the weekly reports for the Strategic Petroleum reserve. Go ahead, try to highlight that text.
amelius
13 hours ago
Yes they should have said 23.0 * 10^3.
inigyou
13 hours ago
2.30 × 10⁴ or bust
selicos
12 hours ago
The BLS has been gutted. It's surprising anything negative is escaping in these reports.
splatter9859
12 hours ago
Exactly. I'm somewhat surprised someone's give-a-damn wasn't broken enough to do a cursory filter. Not like it isn't plainly obvious in other statistics, however.