Any decent error message is a kind of oracle

29 pointsposted 4 months ago
by bobbiechen

27 Comments

Groxx

3 months ago

For debugging purposes, because having users tell you what error they got is sometimes very useful:

generate a random number (e.g. a uuid), log it with the error, and display that number.

doesn't leak data because it's different every time, but you can uniquely pair it up with what they are seeing.

spockz

3 months ago

Better yet. Just use the trace id for the message. That id is correlated with all log statements already and it will allow you to in one go follow the whole flow of the user without setting up another per-request flow lookup table.

RadiozRadioz

3 months ago

The prerequisite to this is setting up OTEL tracing and having already instrumented everything. Which is not trivial. You can't "just use" it.

It's like you get to the middle of a recipe and they say to "just" add in the meat you've already been marinating for 48 hours.

spockz

3 months ago

Telemetry for me is table stakes. We have been doing forms of tracing (first home grown) since 2012. Strictly speaking only the edge gateway needs to know about it.

Tracing is trivial to add with OTel. It is trivial with Spring Boot (since v3). At least on technical endpoint level. Adding tracing to business logic obviously requires work, but simple propagation is table stakes.

Groxx

3 months ago

you really only need it at the entry point to have a trace ID... but yeah. tracing is hard to add on later, though I'm glad it's becoming more common for newer code.

kmoser

3 months ago

I like to have the error message include the line number that emitted it, e.g. "Unable to log in to your account (error code 123)". This comes in handy especially when you don't want (or are unable) to log messages, and/or several lines emit the same error message. So line 123 might be because no DB connection could be established, while line 456 might be because the connection timed out.

user

3 months ago

[deleted]

ChrisMarshallNY

3 months ago

I have found that having this in the user-facing message is not helpful.

It seems that the parent suggestion is unpopular, but having a technically-available log, with a reference from an entry, seems to be a good compromise. Bit more work for the developer, but less for the user, which is really the goal.

kmoser

3 months ago

> Many “Oopsie woopsie”-style errors are a fallback message that appears in unexpected errors - since the developer doesn’t expect it to happen, it could be dangerous to reveal application context as part of the error.

In my experience most of those "oopsie woopsie" error messages are an attempt to seem less techie and/or lack of desire to provide meaningful information to the user.

The main thing users really want to know: did they mess up, or did the system mess up? And what should they do next: try again? Contact support? That basic information is not only usually lacking, but providing it to the user would not threaten system security.

recursivecaveat

3 months ago

It's kind of sad that every computer user has to learn that when they click a link to "example.com/news/2003-07-14/update/very-long-base64-encoded-string", the very patronizing error message accusing them of making a typo just means that the site deleted the content, but is too embarrassed to admit they do that kind of thing.

ChrisMarshallNY

3 months ago

> So why aren’t these errors better? “Password is incorrect, try again.” or, “No account exists for this email.” Is that so hard?

I can tell you exactly why I don't do this, for my app.

I don't want to indicate which of the fields is an issue.

Most folks use Sign up with Apple, though, which obviates this.

The best error message is to avoid the error; either by effective design, or by good affordances.

But this is what WFM. YMMV.

user

3 months ago

[deleted]

saagarjha

3 months ago

> I don't want to indicate which of the fields is an issue.

Why not?

ChrisMarshallNY

3 months ago

Narrows down the possibilities, for a hacker by 50%.

saagarjha

3 months ago

Ideally you have enough entropy that you can spend a factor of two on making your software more pleasant to use

ChrisMarshallNY

3 months ago

That’s why I use Sign up with Apple.

Fair bit of work.

The nature of the target demographic demands that I don’t cut any corners, with security.

But I’m also a big proponent of usability, so I would agree, for some applications.

sippeangelo

3 months ago

The answer should be that it's a privacy leak! Do you allow random actors to brute force your login?

ChrisMarshallNY

3 months ago

It's not Fort Knox, mainly because I refuse to take much information from users, so even if baddies get in, they won't get much they can use.

Admins can't log into the frontend (as admins), so hackers can't deduce power user logins from this, or escalate privileges.

That's kind of blasphemy, with the HN crowd, I know, but we aren't interested in selling anything. It's a pure service.

I won't limit retries, because locking users is about as bad as you can get, with userabusability. I just make sure that the fox ain't worth the chase, and make the chase just a bit more difficult, so hackers will waste their time on low-hanging fruit (that tastes pretty bad).

pixl97

3 months ago

Depending on what jurisdiction you're in, it still could be a legal risk due to PII leaking.

ChrisMarshallNY

3 months ago

That’s why we drastically limit collection of PII.

It really seems to be unusual, for folks to limit data collection. I’m always surprised, when folks seem surprised at how little we collect (and we don't actually "collect" the data, as it never leaves the server, and we don't really do anything with the bit we have. It's just enough to give the user a unique ID, and allow other users to anonymously contact them).

It does make administration and forensics a bit more challenging, but that’s our problem; not the user’s.

louthy

3 months ago

[EDIT: I’d only read up to: “So why aren’t these errors better? “Password is incorrect, try again.” or, “No account exists for this email.” Is that so hard?” — then I bailed, my bad, I should have kept reading. So my response below isn’t fair]

This ignores the security risks from being too verbose and/or specific with error messages, especially if they’re coming from a server. You’ll usually fail security/pen-test audit.

I agree that doing a better job of helping the user is laudable, but you need to know which battles to fight.

Giving a unique error number that can be referenced by a support team (who could look up the event, look at stack traces, etc.) is the best way to deal with truly exceptional events. Otherwise, if it comes to authentication or authorisation, you have to extremely careful what information you share.

username923409

3 months ago

You should try reading past the first page of the article before making a judgment like that.

louthy

3 months ago

Who has time? ;)

But fair enough, I had stopped at the point where the advice was bad.

My bad. I’ve clarified in my original comment.

xg15

3 months ago

I think the article conflates different scenarios: I don't think many people expect that a web service gives detailed technical error messages to external users who might be attackers. But the really annoying appearances of the "oopsie woopsie" messages are if they happen on a local system, where there is no other party - neither a potential attacker, nor any "administrator" who should be trusted instead of the user. So if there is no one else except the user, who should fix the error?

upofadown

3 months ago

When things go wrong there are three main entities to consider:

The User

They need a coherent explanation of what happened, based on the concepts they can be expected to understand. Ideally enough information/explanation should be provided to allow them to overcome the condition.

The Maintenance

This should be a super detailed description of the error and include any possible context. The user will not reliably preserve this sort of information.

The Attracker

Could they conceivably get access to the data?

Could they get any use out of the data?

Could they cause the error condition that generated the error.

user

3 months ago

[deleted]