RFC 10008: The new HTTP Query Method

396 pointsposted a day ago
by schappim
(rfc-editor.org)

163 Comments

100ms

a day ago

Including a strong motivating example might have helped sell this, using an example that could trivially be expressed as a GET is extremely distracting.

Even imagining a QUERY with a large JSON filtering structure, or say an image input as request body, it feels extremely odd to include the request body as part of the cache key. It also implies an unbounded and user-controlled cache key, with the only really meaningful general caching strategy being bitwise compare of the request body (or a hash), which in a hostile scenario implies cache busting would be trivial.

This invokes multiple semantic oddities in one go with obvious difficulties for a very niche use case. If I'm writing a service that needs complex filtering or complex input like an image, any form of caching (e.g. individual data columns of a join, or embeddings keyed by perceptual hashes of a decoded image input) is going to be far away from the HTTP layer and certainly unrelated to the exact bit representation of the request on the wire.

Why even bother trying to capture this in a generic way?

I would be far more inclined to try and capture this caching semantic as a new header for POST. Something like "Vary: request-body" or similar. Perfectly backwards compatible and perfectly ignorable for all but the 0.1% of CDN use cases where the behaviour might turn out useful

Joker_vD

a day ago

> It also implies an unbounded and user-controlled cache key,

The query part of GET's URI is also barely bounded in practice and user-controlled, and is indeed used as part of the cache key (because it's a part of URI), so I am not sure why you raise this objection at all.

giancarlostoro

a day ago

> and user-controlled

I've found some sites that tack on a session ID and if you try to tamper with the URL in any way, it sends you back to "Page 1" really annoys me lol at that point let me skip to any page with your web UI.

PunchyHamster

a day ago

Well, because it is more code. Current caching software caches by headers + query string. It now needs to be expaned to cache by body too.

It feels very pointless and there is no drawback of just using POST

OvervCW

a day ago

There is: your browser or other type of client does not know it can repeat a POST request if it fails, whereas a QUERY request can be freely repeated in case of errors.

spockz

20 hours ago

Not freely. It is idempotent, not safe. So it still can have serious load consequences.

Joker_vD

20 hours ago

    Unlike POST, however, the method is explicitly safe and idempotent, allowing
    functions like caching and automatic retries to operate.

account42

7 hours ago

Putting something in a spec does not automatically make it true. In the real world if you repeat expensive queries more than an undefined amount you get blocked or at least bot-checked.

projektfu

5 hours ago

From the browser POV it doesn't need to ask if you want to re-send data.

themgt

3 hours ago

Putting something in a spec does not automatically make it true.

Terrible news for computing.

afavour

a day ago

Is caching not the primary reason to use this over POST? You should never want to cache POST requests.

drdexebtjl

21 hours ago

No. Being idempotent, it also lets the browser/client/reverse proxy retry it if it fails.

nfw2

19 hours ago

Technically a put or a patch is also idempotent. The benefits are idempotent and safe (and semantically appropriate). Post (generally) communicates something is changing whereas a query doesn't

gamache

18 hours ago

PUT is idempotent, PATCH is not always. The semantics of a PATCH payload are up to the server and standards like JSON Patch (RFC 6902, https://datatracker.ietf.org/doc/html/rfc6902) allow non-idempotent operations like adding an item to a list.

nfw2

11 hours ago

I stand corrected although using patch this way seems goofy to me.

speleding

6 hours ago

I like the proposal, but I agree they could have sold it better.

This is basically a GET request that can have a body. I've found myself in need of that more than once when I did not want huge URLs with encoded data showing up in logs. Using POST request there is not appropriate because it signals data could be modified (i.e. cannot be sent to read-only instances). I guess modifying the spec to allow GET to have a body would pose too many problems.

CodesInChaos

a day ago

The browser can simply store a collision resistant hash (e.g. SHA-256) of the body, if it wants a smaller cache key. I can't really think of any caching related attacks that don't equally apply to a query parameter. Generating a unique 30 character query parameter is just as easy as generating a 30 MB request body, if you want to flood the cache.

ralferoo

a day ago

Not necessarily that simple, as you'd have sort all the input parameters to maintain a useable cache key. Not especially difficult, but if the data is large and so re-allocation and sorting is required, then you're starting to open up the attack surface where bugs might have been introduced.

dagss

19 hours ago

Do you have to? Is it common to treat ?a=1&b=2 the same as ?b=2&a=1 in browser/CDNs/etc?

Seems the spec puts this as a MAY. I think I doubt it will be implemented in generic ways, except perhaps for urlencoded payloads. After all you cannot normalize in general without knowing the query language. At the backend it does not matter, may as well cache one level deeper based on the parsed input irrespective of QUERY or not.

ralferoo

18 hours ago

No, that was my point. In a GET request, a caching proxy cannot assume the URL is URL encoded parameters, because the URL can contain data encoded in any form. So, you could only cache a GET on an exact URL.

But for a QUERY that explicitly marked the data as multipart or url-encoded, then semantically the order of parameters no longer matters.

That said, it's hypothetical because the only thing that uses those at the moment is POST and that explicitly should never be cached.

But there's another reply above to my comment that points out that a caching implementation is free to do what it likes, and if it fails to cache when parameters are in a different order, then it would still be correct, which is a fair point. That comment was https://news.ycombinator.com/item?id=48578024

CodesInChaos

8 hours ago

I think the only sane approach for caching comparing if the body is identical and not apply any content type specific transformations. It's the safest choice, and the cache not being effective in some edge cases isn't a big deal. The client can always canonicalize the data itself, if it matters for its use-case.

Even if the spec says that different representations are equivalent, it's quite common for applications to treat them differently. For example field ordering in json objects is supposed to not matter, but some serializers care if a type discriminator is the first field.

ygouzerh

a day ago

Regarding the body used as a key for the caching: in the RFC, from my understanding, it's indicated that we can use Location as well:

Exemple:

``` QUERY /search HTTP/1.1 Content-Type: application/json

{ "filters": { "region": "asia", "status": "active" }, "sort": "created_at", "limit": 500 } ```

can answer

``` HTTP/1.1 303 See Other Location: /queries/results/f3a9c1d7 ```

And then you can access later `/queries/results/f3a9c1d7` using a pure GET call, and cache this instead

inigyou

a day ago

Not all usage scenarios are the public internet, and something doesn't have to be useful on the public internet to be standardized.

Realistically, systems for the public internet will use a secure hash as the cache key so it'll always be the same size. The cache key already includes a URL that can be very long, and an arbitrary set of header values.

ralferoo

a day ago

Except that by definition, in a URL the data has no implicit meaning so for a cache hit you need an exact match, including order and case, but for a list of POST parameters, they could legitimately be in any order and so you can't just hash it all as a blob, you need to sort the keys, possibly copy data around (unless using keys plus hash), probably allocating more memory, etc. I'm pretty certain we'll see at least one CVE out of the first few implementations of this!

inigyou

a day ago

POST/QUERY data can be in any format. Who are you to say order doesn't matter? Are you sure you can even parse it? Mine is in DES-encrypted (with key "password") base85 DER, you really gonna implement that in your proxy?

ralferoo

20 hours ago

Maybe my knowledge is out of date in terms of how people generally use POST nowadays, but AFAIK multipart/form-data is still the most common encoding for data and occasionally application/x-www-form-urlencoded.

Both of these, the key values can be in any order with the same interpretation. That's kind of a moot point for POST method, because they should never be cached anyway, but for the new QUERY method it'd be reasonable to expect a cache hit whenever the parameters are the same regardless of order.

My point is that for a GET, you can't assume that the order isn't important, because the URL is an opaque string by the time it hits the cache. However, POST (and now QUERY) explicitly says what the coding is, so for instance with application/x-www-form-urlencoded we can be sure that the parameters can be in any order without changing the meaning. You cannot infer that from a URL itself.

As to your point, yes you can use any other encoding you like to. But most systems don't do that, they use multipart/form-data.

drdexebtjl

20 hours ago

This RFC does not require caching to be implemented at all, so it wouldn’t be reasonable to expect a cache hit, no. But if your implementation does that, cool :)

ralferoo

19 hours ago

An RFC would never mandate caching. But the table in the article says "cacheable: yes" for GET and QUERY. There is no "my implementation" because this is a proposal that has only just been proposed and there is currently "no implementations". I'm simply saying that QUERY will be harder to get caching correct compared to GET, and I'm almost certain there will be end up being CVEs resulting from its implementation.

drdexebtjl

18 hours ago

It’s been an IETF Internet-Draft for a few years at this point, so there are some implementations already in the wild.

What I mean is that implementations are free to choose do something as complex as what you suggest, but also something as simple as hashing the body as a blob, and they can even bail on caching completely (for example if the payload is too large).

All of those options would be correct behavior per the RFC.

Of course we may still see CVEs from this, but they will be self-inflicted, not caused by a complex standard.

tanepiper

a day ago

One example - I'm building an MCP server at the moment for a database I'm working on. In ChatGPT I want to do dry-run posts first that roll back before committing - both are POST requests with a property - and it loves to trigger the safety layer in the tools (for various reasons, it's hard to debug exact causes)

But I think this would make it better - QUERY before POST means different request types, not just the same with a safety flag.

friendzis

a day ago

> It also implies an unbounded and user-controlled cache key.

While the concern is valid, caching is entirely optional at query level, therefore it is totally valid to cache only certain "filters".

cryptonym

a day ago

Sure you can provide an image as request body, but you could already do it with b64 query parameter. If you try hard enough, you can poorly use any proposed standard. GET with query parameters already is opaque and makes cache busting trivial.

layer8

a day ago

Query parameters are length-limited, because HTTP URIs are: https://www.rfc-editor.org/info/rfc9110/#section-4.1-5. There is no expectation for arbitrarily long HTTP URLs to be functioning.

cryptonym

a day ago

Your link doesn't say URIs are length-limited

Draiken

a day ago

I'm guessing you never hit this issue then, but it's a real issue. Whether or not it's in the RFC as a hard limit it doesn't matter, no HTTP server will allow unlimited sized URIs.

You simply can't base64 large payloads and you're stuck with workarounds.

cryptonym

a day ago

You are guessing wrong. Thanks, I know specific implementation will come with their limits. This will equally apply to QUERY body size and caching strategy.

Are we seriously ok with linking the RFC as source while providing a statement that doesn't match? RFC does matter.

ralferoo

a day ago

The RFC does say "It is RECOMMENDED that all senders and recipients support, at a minimum, URIs with lengths of 8000 octets in protocol elements."

One can infer from the RFC that you can reasonably expect many implementations to fail beyond 8000 characters, and that there are no guarantees up to that either.

True, the RFC doesn't specify a limit, but it does clearly indicate that it's not unbounded, nor should you expect it to be.

drdexebtjl

20 hours ago

This RFC (10008) does not require caching to be implemented at all, so it would make no sense to make a recommendation here for what is a reasonable limit to expect caching to work.

layer8

21 hours ago

They are in the sense that the recommended supported length is only 8000 bytes. There are no such specified length recommendations for HTTP body size.

cryptonym

10 hours ago

Recommended supported length is at least 8k.

Of course I don't advocate oversize URLs. That's a point of RFC10008.

Let's say we build a service for image transformation or image information extraction. Get isn't practical. QUERY with image as body could be a valid usage, regardless of caching. It conveys information that request is idempotent and can be retried with no impact on data, contrary to POST. If your http client is configured to support this, it can potentially improve reliability.

epolanski

a day ago

> Why even bother trying to capture this in a generic way?

I guess it's about resolving the odd semantics of using POST which is not idempotent and thus allowing easier control flow of caches and retrys.

Your perspective is 100% correct if you think at the application-layer, but with a dedicated method, you can have that behaviour out-of-the-box out of your HTTP infrastructure (whether it's at your hyperscaler's router or your apache/nginx/browser whatever) and stop implementing yourself the post-as-a-query edge case.

davidkwast

a day ago

I would use a hash of the body content (the query) as a URL parameter

/?hash=123456789

Joker_vD

a day ago

Why? That's pushing more work to do both on yourself and the cache.

WorldMaker

21 hours ago

Actually this is a use-case supported by this RFC [1]. You accept an arbitrary QUERY /search/ and you cache it on your side (or in a middle box somewhere such as a CDN edge) you can return in your response:

    Location: /search/?queryHash=SOMECDNHASH
The browser can then cache that Location and the next time convert that same QUERY /search/ into GET /search/?queryHash=SOMECDNHASH.

Sure, it is more work for your webserver to compute that and potentially the browser to cache it's knowledge of that QUERY, but it potentially gives you an advantage in keeping things like CDN edge caches generally aware of client/browser caches in a way that can be performance optimized.

[1] https://www.rfc-editor.org/info/rfc10008/#section-2.4

wang_li

a day ago

If you control the full stack then the functionality described here can be implemented with POST. The only way this comes into play is if some second party client of your service is trying to impose rules on how your backend works. My answer to that is no. I will be defining the contract by which my services operate.

CodesInChaos

a day ago

I wonder if HTML forms will add support for QUERY:

    <form action="..." method="query">
This would avoid the annoying re-submission warnings you're getting if you refresh a page that was returned by a POST form submission, since QUERY is required to be idempotent.

acabal

a day ago

Supporting more than GET/POST in HTML forms has been my dream for decades. There's a WHATWG proposal to do just that if you want to add your voice: https://github.com/whatwg/html/pull/11347

CodesInChaos

20 hours ago

I'm not convinced of the benefit of allowing PUT/PATCH/DELETE in forms. But QUERY sounds more useful, and its lack of side-effects (plus the lack of legacy code using it) would avoid the need for CORS preflight requests.

There is a separate issue for QUERY: https://github.com/whatwg/html/issues/12594

MBCook

21 hours ago

What’s the use case for that?

acabal

18 hours ago

Because if HTTP is the language of the web, then HTML forms are how humans speak that language to computers. Right now we humans can only speak GET and POST.

In other words, right now if a human wants to DELETE a widget, the human has click on an HTML form to `POST /widgets/123/delete` - i.e. use an incorrect verb on an incorrect URL/object - or use some other workaround like smuggling a special `_method=DELETE` variable. This is unnatural and semantically incorrect, resulting in ugly hacks that break HTTP-level expectations like idempotency; and it also requires additional app-level logic to process.

Meanwhile a machine is allowed to simply `DELETE /widgets/123` because their interface to HTTP is not clicking on HTML forms.

We humans could converse with websites in semantically correct HTTP, have clean URLs in which both REST APIs and human-facing URLs are identical without hacks, and require no extra app/framework logic, if HTML forms simply allowed all (human-relevant) HTTP verbs.

imtringued

7 hours ago

Things like that are why it's completely unreasonable to expect websites to work without first party Javascript. Utterly basic and mundane functionality is gated behind Javascript.

Building the mundane in the absence of JavaScript isn't some small thing you can build. It's going to require you to rebuild your entire site around the idea. Things like a searchable dropdown will now turn a single page form into a horrifying multi page mess where you get sent to a page for just the dropdown and the search box and selectable options, which then send you back to the original form. That is going to require you to store a lot of the form data on the server to keep track of what you entered across pages rather than let the browser do its thing.

The noscript crowd should have lobbied for a better HTML instead of rallying against JavaScript.

amluto

a day ago

One oddity of forms: the result of a form POST is a page that has a location (the URL) but that cannot loaded via that location. As far as I know, the fact that the page is a POST and not a GET is not stored anywhere visible to the user or to JS. And refresh works oddly.

If method=QUERY were added, there would be a new variety of this weirdness.

sheept

a day ago

At least browsers wouldn't have to warn users that they'd be resubmitting data if they reload the page after submitting a query form, since query requests are intended to be idempotent

amluto

a day ago

You still get the nastiness that the Sec-Fetch-* state gets mostly trashed when you hit refresh. And someone would need to figure out how CORS preflight interacts with refresh, which is not currently an issue with POST. (The current "simple request" behavior or whatever it's called is a real mess and is the cause of a lot of CSRF vulnerabilities.)

bob1029

a day ago

This is better solved with the post redirect get pattern.

CodesInChaos

a day ago

The redirect pattern makes sense for a POST request that creates a resource, where you can then redirect to the newly created resource.

QUERY on the other hand makes sense for cases where the request doesn't cause any state changes on the server, and there is no resource to redirect to.

diroussel

a day ago

That is the good old fashion workaround. But why is it better than a form causing an HTTP QUERY.

If we can do QUERY forms, it would be an ideal time to add JSON encoding for forms.

XYen0n

11 hours ago

If new method support is added, PUT should be used in this scenario.

paradox460

a day ago

They never added support for any other verbs, but it's a brave new world, so who knows

100ms

a day ago

Forms, HTTP implementations, public API surfaces, and all for what exactly. Introducing a new verb for this feels profoundly misplaced

jagged-chisel

a day ago

Idempotency is an important attribute for correctness. Yep, you can document that POSTing to $ENDPOINT is idempotent, but you can't communicate that to caching layers throughout the network. QUERY, by definition, is idempotent and cacheable.

resters

a day ago

Great point. I wish more people realized that intuitively.

alpinisme

a day ago

At least support - or lack thereof - for a new verb is unambiguous (compared to changing the semantics of GET)

tempfile

a day ago

Depends whether your form submission should expect side effects or not. Most forms submissions have side effects. If the effect is truly idempotent, wouldn't PUT be a better verb? That is also supposed to be idempotent.

echoangle

a day ago

You can’t use PUT as a form action though.

tempfile

8 hours ago

I always forget this, because it makes no sense. Fair enough!

diroussel

a day ago

GET and QUERY are both idempotent.

tempfile

8 hours ago

I know, but neither GET nor QUERY may have side effects.

ctdinjeu7

a day ago

Now HN’s UX can finally be decent.

The team will have to wait for a new header and textarea specs to fix the rest of the jank.

This site is so awful lol. Why don’t they update it?

CodesInChaos

a day ago

Where does HN use POST for safe operations? I can't think of any.

Comment submission isn't safe, so QUERY can't be used there. And it doesn't suffer from the problem anyways, since HN returns a 3XX on successful submission, so refreshing doesn't show a warning.

ynac

a day ago

Just in case anyone wants to pretend it's still that other century:

https://www.rfc-editor.org/rfc/rfc10008.txt

jjice

a day ago

I'll forever love a long, totally plain text document like this. So many good times with video game FAQs as a kid. It really is a superior form of information in a lot of ways (not all).

riffic

a day ago

I have a thesis brewing that explores how rich text WYSIWYG editors create a "what you see is all there is" cognitive bias, while plain text overcomes it.

riffic

a day ago

beautiful formatting. I should crib this style template for internal work memos, it's timeless.

piterrro

a day ago

> GET request with a body was heavily considered by the IETF working group, but it was ultimately rejected in favor of creating the new QUERY method. The decision to create a distinct method came down to historical interoperability issues and strict compliance with the core architectural definitions of HTTP.

I've been sending request body along GET method for years now

notatoad

13 hours ago

this was the response the last time this came up here.

you can do all kinds of nonstandard stuff if you control the server, the client, and any steps in between. the point of standards is for when you don't control it all.

put your server behind a managed load balancer or a caching proxy, and your get requests with bodies aren't going to do so well anymore.

huskyr

a day ago

Apparently some load balancers drop the body.

inigyou

a day ago

I expect all sorts of intermediaries may drop the body, since having a body is forbidden by the standard.

When it's your client talking to your server you can obviously do whatever you want - it doesn't cause problems until you want to involve third-party code, such as a reverse proxy (such as nginx) or a CDN. This includes proxies your customers may be using.

jmaw

a day ago

Where is it forbidden by the standard? I don't see anything in the GET definition in RFC 9110 [1] forbidding that. My understanding was that this is just undefined behavior. And not recommended due to your point about some third-party CDNs and RPs handling that UB in different ways.

[1]: https://datatracker.ietf.org/doc/html/rfc9110#name-get

giobox

21 hours ago

It's never been explicitly forbidden, just heavily discouraged in virtually every version of the spec. In the current 9110:

> "A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported. An origin server SHOULD NOT rely on private agreements to receive content, since participants in HTTP communication are often unaware of intermediaries along the request chain."

"Content" is what the RFC calls a request body. The simplest argument against IMO is it makes caching more complex, as now the request body would have to be part of any cache keying etc, you can't just blindly key off the request + URI params for every single GET. There are plenty of other reasons to not do it.

The spec expects responses to GETs generally to be cacheable:

> "The response to a GET request is cacheable; a cache MAY use it to satisfy subsequent GET and HEAD requests unless otherwise indicated by the Cache-Control header field (Section 5.2 of [CACHING])."

Zardoz84

9 hours ago

I work with a custom own maded web framewrok for Java (from before Spring was a thing!). A funny thing it's that treats GET and POST petitons identical when try to recover parameters from the petitions. So GET and POST get parametertes from URL and from BODY. And this made some weird bugs many years ago, when in some case, we got repeated parameters in URL and BODY, trigering the framewrok to think that must put it on a List and pass it to the service method.

pseudohadamard

16 hours ago

It does seem like a boil-the-ocean "fix". The entire planet has been kludging this with GET or POST as appropriate for the entire lifetime of HTTP, and now there's a proposal to do something that looks identical to POST but with slightly different semantics? So in exchange for solving a problem that's been dealt with (if not perfectly) forever you'd need to convince the entire world to adopt a new, incompatible mechanism? I'll give this one as much of a chance as BEEP.

You've never heard of BEEP? Well, that's why.

drob518

a day ago

Aside: Wow, we’ve hit 5-digit RFC numbers now!

robertlagrant

4 hours ago

> https://www.rfc-editor.org/info/rfc10008/#section-2.8

Having a standard way to do pagination would be extremely useful. If we could add an items (or similar) range header type that would be excellent!

For example, in the HEAD response (and the QUERY response):

  accept-ranges: items
  items-length: 342
In the QUERY request:

  Range: items=0-9

smashed

a day ago

Use the QUERY method in your http query to query search results. Do not add query parameters.

I think the name is confusing because the term 'query' is already used to refer to http requests in general.

Just the title of the RFC confused me.

comfydragon

a day ago

> the term 'query' is already used to refer to http requests in general

In what circles is this the case? I sometimes colloquially refer to a GET request as a query, but definitely not so on a POST, PUT or DELETE.

rehevkor5

a day ago

Yeah, and it doesn't even have to be a query, it could be an idempotent effect. I think they'd be better off calling it IPOST (for idempotent post).

Edit: ah, they declare QUERY as "safe" meaning no side effects, for cacheability. My mistake.

andltsemi3

a day ago

If this is actually going to replace GET requests w/ query strings in the wild, Im very much hoping for browser bookmarks to support keeping request parameters.

inigyou

a day ago

Probably won't. Probably will replace whenever POST is currently used for a query.

anitil

15 hours ago

That's a good point, I do like to be able to bookmark specific issues (like in Jira I bookmark 'all issues that are assigned to me and not done' )

toybeaver

a day ago

This makes me happy tbh, I was never a fan of creating `POST /search` endpoints when working with robust APIs

pwdisswordfishq

a day ago

Wait, it's already past 10 thousand?

rhplus

a day ago

Someone has an ambiguous bet predicting when RFC 10000 will be published, but the numbers went straight from 9998 to 10008. No-one wins!

https://manifold.markets/CollectedOverSpread/when-will-rfc-1...

echoangle

a day ago

> This question resolves to the month of publication of the lowest-numbered RFC with a number greater than or equal to 10000.

So of 10008 is the first one after 10000, that date is the one to bet on.

schoen

20 hours ago

Is anyone formulating prediction market questions asking AIs to brainstorm about edge cases in order to leave fewer of them uncovered by the market definition?

We do have humans brainstorming about such things, but this feels like something LLMs might be good at.

Imustaskforhelp

a day ago

Everytime I think that prediction markets bets can't get worse, they do, all in weird ways. I never expected someone betting over when RFC 10,000 will be published but somehow its fits just about right for prediction markets.

just wow, people seem to be having too much money it seems for them to bet over when RFC's are gonna get released.

This isn't even one of the worst offenders on prediction market or even comparable to it but I am just amazed (in a negative manner, surprised? its just strange) by the depth on what people actually bet on these markets.

networked

a day ago

People aren't betting real money on this. Manifold uses "mana" points similar to HN karma, which is why you get more for-fun silly bets. I don't see anything inherently wrong with it. Disclosure: my mana net worth is 75k; I haven't been active on Manifold.

Imustaskforhelp

a day ago

Ah okay. I didn't know that.

Interesting thing actually. Seems similar to the trend in South Korea recently where you can online shop to get the thrill of shopping but you aren't actually paying with money.

But I am unsure of the overlap between manifold and polymarket/kalshi. I imagine that some might win in manifold and try to bet on polymarket to win "real" money which ends up being a bit gambling-esque.

But good for manifold for atleast not playing with real money but rather points like this. I would argue that Manifold might be better than polygon/kalshi in terms of net positive outcome of its existence for the world perhaps.

networked

a day ago

Interesting trend. I have found a short article about it: https://www.odditycentral.com/news/south-koreas-fake-online-.... It seems nothing long-form has been written about it in English yet.

There is an overlap between Manifold and Polymarket/Kalshi. At the very least, Polymarket is more liquid, which creates opportunities for arbitrage and incentives for Manifold users to follow Polymarket. There is something at stake on Manifold itself if you choose to pursue it. There have been ways to convert mana to charitable donations (to your preferred charity), tickets to Manifest (the Manifold conference), and also merch and now prize drawings. Mana is like HN karma in that being at the top gives you status and bragging rights and suggests technical competence.

yoshi389111

4 hours ago

RFC numbers finally reached 10000. It's interesting to think that the RFC series started in 1969 and has continued for more than 50 years.

inlined

a day ago

I know agents are out of scope of this RFC but I love that this could easily be extended to make the JS EventSource to work on streaming AI queries.

Due to the need of bodies in requests, everyone uses POST and streaming results often use the text/event-stream protocol for responses. But this is technically a bad fit because no state is actually changing and because EventSource can only use GET for some obstinate reason. So many APIs reimplement the functionality with their own parser

tonympls

a day ago

As "just a guy that programs" (ok, now guides agents to program) and tries to follow the rules (with a big dose of pragmatism), this totally makes sense to me. This is also the first time I've seen or heard about this coming.

I like that we now have a way to not being forced to define Resources when we want to query. It always felt like I was missing something that there could be an infinite, defined-on-the-fly number of Resources for a "part" of a given Resource. Do I really want to define "all cats that sleep more than 20 hours a day and like sunbeams and want to eat breakfast at 3 am" as a Resource? (ok, we all know that is actually the full set of cats). I'm ok that you want to define that as a Resource but in my system, it makes more sense that Cats is the Resource and I just need some accepted way to query.

I like the implementation (again, as just a guy that programs). I don't see how it could have done it better or simpler which probably hides the complexity of getting there.

I also especially appreciate how the spec is written. Opening a spec, I wonder how far I'll get before I don't know what the heck they're talking about (and, again, as just a guy that programs). I don't think it's easy to write a spec that is complete and approachable like this. Really appreciate that.

inigyou

a day ago

The standards have always been a bit more abstract than you use in practice. Common practice prior to this would be /catsearch?sleep=20&sunbeams=yes&breakfast=0300 where the "resource" is /catsearch and the rest are query parameters, but you could also use /catsearch/sleep/20/sunbeams/yes/breakfast/0300 which looks like a "resource", but nobody is actually enforcing that it is a "resource".

barbazoo

a day ago

> GET: Content (body) "no defined semantics"

I thought it wouldn't be a terrible idea to open up the GET method to contain a body but according to the original spec the GET body is to be ignored completely. There's also caching which would break because the important bit of the request would live in the stripped body.

angrybards

18 hours ago

GET with only a URI has the semantics of retrieving the current representation of the resource. This is the most basic form of hyperlinking and quite important to how the web works. Adding a body parameter to GET would break that constraints of the method as you couldn't treat two requests using the same URI as referencing the same thing.

mlhpdx

a day ago

Wow, it still isn’t a standard? I’ve been building with the QUERY method for years now.

I’ve enjoyed the combination with Range headers for paging, despite this tidbit:

> It is expected that these built-in features will be used instead of HTTP Range Requests

Using the QUERY request as the definition of a set, and Range to retrieve subsets seems very natural.

ygouzerh

a day ago

The exemples section at the end, with csv and sql are quite powerful. It open the door to easy caching of raw data and probably other use cases, quite interesting!

nottorp

a day ago

It's as bookmarkable as a query with its parameters in the POST data...

cosmotic

a day ago

Why not just define the semantics of a GET request body?

advisedwang

a day ago

There's countless proxies in the wild that would not behave correctly with an RFC-defined GET-with-body, and there's no way for a client to know if that's the case.

QUERY has the advantage of getting default behaviour from most proxies (which at least is well behaved even if inefficient). If there are any proxies that just drop QUERY requests, at least they won't silently mangle the request.

This is the same way that instead of improving how HTTP 301 was specified, HTTP 308 was created. It's a pragmatic move.

cosmotic

18 hours ago

Would those same proxies work with the new method?

WorldMaker

21 hours ago

It would be a potentially breaking change so out of an abundance of caution likely would require (or need to wait for) a new version of HTTP like HTTP/4.0. On the other hand, adding an entirely new Method potentially can be applied across HTTP versions. It's possible for an HTTP/1.1 server to accept QUERY today under this RFC, it doesn't need to wait for and upgrade to HTTP/4.0 service, whatever that would entail. (Especially in the Postel's Law world where HTTP/1.1, HTTP/2.0, and HTTP/3.0 are all living side-by-side and are very different protocols under the hood.)

chadgpt3

a day ago

Proxies often delete it

elAhmo

a day ago

They could be updated to not delete it, like they would require for this new method anyway.

doctorpangloss

a day ago

"Checkmate, IETF."

i actually agree with you, i don't comprehend why we're adding anything to HTTP like this, it is basically already deprecated. The modern API is WebTransport, if you want to make caching better for WebTransport, do it there.

moralestapia

a day ago

Agree.

They should not delete the body in the first place.

jerf

a day ago

If we're going to play the "should" game, whatever originated the body "shouldn't" have because the spec says that's illegal.

Although we could also go with, a proxy shouldn't delete the body, it "should" reject the request outright as ill-formed.

The meta-point I'm making here, which I'm sure will be missed if I don't spell it out, is that if we're going to talk about what "should" be done when it is explicily out of the scope of a standard, there's no way around the fact that there are multiple completely sensible ways to extend the standard and there's every reason to expect that in the real world people aren't going to agree. Sometimes they manage to, but even then often quite imperfectly. Our human intuitions that standards are something that are "built" is perhaps not wrong, but you can also productively look at standards as taking the raw material of all possible things two systems could send to each other and removing possibilities. If you reach into a space that has been explicitly removed, you can't expect everyone else to do so in exactly the same way you will.

Joker_vD

a day ago

> the spec says that's illegal.

For a very long time, the spec did not say it's illegal. In fact, RFC 2616 (that has been defining the HTTP/1.1 for 15 years) says that

                                   A message-body MUST NOT be included in
    a request if the specification of the request method (section 5.1.1)
    does not allow sending an entity-body in requests. A server SHOULD
    read and forward a message-body on any request; if the request method
    does not include defined semantics for an entity-body, then the
    message-body SHOULD be ignored when handling the request.
but if you go into the section that describes the semantics of a GET request, well — that section says nothing at all whether a GET request is allowed or not to have a message-body. So it's not prohibited, and the servers should simply ignore it when processing it (and proxies should forward it up).

flufluflufluffy

a day ago

The HTTP standard does not define a body for GET requests. Therefore, proxy implementations will typically only copy the data from the request header when sending it off to its next destination. They don’t technically “delete” anything. This saves compute (and possibly bandwidth, if the request happened to have a body, although the whole point is that one can safely assume a GET request does not have a body per the standard).

brookst

a day ago

Wouldn’t just putting an etag on POST requests accomplish the same thing? If I’m understanding it the server has to maintain state to ensure idempotency.

CodesInChaos

a day ago

QUERY is GET with a request body. So it must be safe, not just idempotent. Where safe means it has no significant side-effects. Typically servers will not keep any state for QUERY requests.

There is one interesting variant though, which uses state: The client sends a QUERY containing the full query, and the server returns a url usable with GET with which this query can be triggered in the future. Similar to prepared statements in SQL databases.

Using QUERY for GraphQL queries (not mutations) would be a good match. These only read data, but are sometimes bigger than the url length limit.

brookst

a day ago

Thanks for the explanation!

I still don’t get how idempotency can typically be ensured without state. It very much depends on data model and application design. Even side effects like using a user’s lookup quota need to be handled at a higher layer than HTTP (I think?).

wongarsu

a day ago

Imagine a forum where comment ids are client-generated UUIDs, and comments are inserted with "ON CONFLICT IGNORE". Submitting the same comment twice would simply be a noop

But what the Query method really targets are things like a graphql query that can be multiple kb for a single query, but only reads data. Sure, it might count against rate limits, trigger logs, etc. But at a conceptual level resubmitting the same query should give the same result (if the data didn't change). And since you are only reading data, resubmitting is safe

CodesInChaos

a day ago

Minor side-effects like quotas or request logging are generally ignored when considering the semantics of http methods. I don't see any complications for QUERY that don't already apply to GET. It just allows you to bypass the url length limit by putting the data in the body instead of the url itself.

uberex

a day ago

Exactly. "GET / is idempotent" ... "But I just launched a DDOS against / and now it returns 502...

Joker_vD

a day ago

> I still don’t get how idempotency can typically be ensured without state.

Well, how is "GET /index.html HTTP/1.1" made idempotent in practice without (additional) state?

brookst

17 hours ago

Ah, QUERY is explicitly scoped to be read-only, unlike PUT, POST, DELETE, etc. I had missed that part.

inigyou

a day ago

Yes it varies. Using the QUERY method doesn't automatically mean your app is idempotent - it means the browser, and any intermediaries, can assume it's idempotent. So when you go forward and then back they're free to reissue the request and you won't get the "this may repeat whatever you just did" popup.

If it's not actually idempotent but you're telling the browser it is, of course you may cause bugs. Same as GET.

CodesInChaos

a day ago

I think PUT, which is idempotent, would still require the "repeat action" warning, since it will overwrite any changes that happened to the same resource in the mean time (unless used with `if-match` or similar). QUERY won't require such warnings because it's safe, not just idempotent.

nfw2

19 hours ago

HTTP can't ensure what happens in the application layer is safe or idempotent. It is up to the developer to ensure that the implications communicated by their api design are in fact true.

trollbridge

a day ago

Ideally, libraries like FastAPI, etc. could be configured to translate QUERYs to GETs, until you can rewrite your code to automatically support both.

n_e

a day ago

Interestingly, despite the QUERY request being safe, the RFC says it's subject to preflight requests:

> A QUERY request from user agents implementing Cross-Origin Resource Sharing (CORS) will require a "preflight" request, as QUERY does not belong to the set of CORS-safelisted methods (see [FETCH]).

CodesInChaos

a day ago

That paragraph merely describes how existing browsers behave, it doesn't specify how future browsers must behave. After all, a HTTP RFC isn't really the right place to specify browser specific behavior like CORS, that belongs in a W3C/WHATWG specification.

Joker_vD

a day ago

    Unlike POST, however, the method is explicitly safe and idempotent, allowing
    functions like caching and automatic retries to operate.
Essentially, it's for things that are inherently safe/idempotent already (e.g. search or indeed, anything that you don't mind being retried) but require a lot of data passed in the request.

exabrial

a day ago

Just curious, was it Google/Facebook that sponsored this?

userbinator

13 hours ago

Two of the authors are from Akamai and Cloudflare.

exabrial

2 hours ago

Interesting. The reason I asked is both companies were on a user hostile (per usual) campaign to hide query parameters from the regular consumer in an attempt to track them better.

stymaar

a day ago

Why not standardize a body in the GET request (which isn't forbidden per spec and works in many places already but isn't supported everywhere because it's not mandated to support it)?

mholt

a day ago

Too many servers ignore/drop/reject body in GET requests. RFC 9110 does allow it, but is only recommended if server documentation states that it is supported.

stymaar

a day ago

These servers will likely also reject the QUERY request though…

nfw2

19 hours ago

Yeah those servers need to be updated anyway to support the new standard

Also, anyone using these servers is not currently putting params in GET body because doing so wouldn't work.

Those that oversee evolving standards seem to take extreme cautious to not rock the boat, and so more and more gotchas for the sake of backwards-compatibility keep getting added to the mountain of random details new devs have to learn. Which will soon include the difference between a GET and a QUERY I guess.

greghines

a day ago

But at least in that case you may get a much more meaningful 405 Method Not Allowed response rather than the server just silently dropping the GET body content.

lanycrost

a day ago

query strings always had size limit, seems this new type will solve it which will be really good.

TheGRS

a day ago

I think I'm seeing the advantages of using this, but I can't help feel like momentum is going to be strongly against adoption. Already feeling like if I suggested this it would meet a ton of eye-rolls, there'd be all this new plumbing needed just to support something we can already do in similar ways.

drzaiusx11

a day ago

I don't hate it. Covers all the bases: 1.1, 2, 3/quic and solves real problems: get query limitations vs body content & post-without-mutation. Yes there are preexisting workarounds, but they're non-obvious.

etchalon

a day ago

Why does this feel like GraphQL demanding everyone else solve their problems?

AtNightWeCode

a day ago

Kinda pointless since traffic is encrypted. If you can terminate TLS you can apply any rule based on the content as well. Like headers which is more reliable.

paulddraper

a day ago

What no one has been able to explain:

Why is that better than a longer URL?

Semantically it identifies the resource. And it must be included in the cache key.

True, not everything supports long URLs. But not everything supports QUERY either, so it’s an absurd argument.

userbinator

13 hours ago

So this is basically just a "big GET"?

Edit: refute or agree, please.

haeseong

a day ago

QUERY has existed in spirit for nearly two decades as WebDAV's SEARCH method https://www.rfc-editor.org/rfc/rfc5323 and the thing that always killed it in practice was intermediaries. Plenty of proxies, WAFs, and load balancers either strip the body from methods they do not recognize or reject the request outright, so the guarantee that sending a body is safe evaporates the moment traffic crosses a middlebox you do not control. Until gateway and CDN support is real rather than just on paper, POST with a header marking the body as part of the cache key stays the pragmatic choice.

geth101

a day ago

Just allow body for get. Problem solved.

CodeWriter23

a day ago

Nah, you need the 405 Method Not Allowed to be returned by legacy systems and proxies along the way to your bleeding edge server rather than silently failing whilst dropping the params in the request body.

simon84

a day ago

This whole thing is non sense. It basically mixes technical constraints (body or not body) with a functional requirement that arises from people that are tied to semantics of the protocol.

HTTP is transfer protocol. It should not ever imply anything at the business level.

Yes REST made it's worst mistake out if it by giving a meaning to the verb.

Yes proxies rule how the body is re-interpreted in spite of the will of the sender (wtf).

But the original RFC states clearly that any verb can be used. This is how WebDav normalised its own.

But playing fancy by introducing a change that all HTTP implementation will have to honor is a very bad and irrational choice.

pie_flavor

a day ago

Yes this, yes that, yes the other, because proxies are in agreement with patterns are in agreement with the HTTP spec that methods exist and have semantic and functional requirements. Your 'should' seems to be discussing a hypothetical technology that is not HTTP, because HTTP has worked this way since 1997.

simon84

11 hours ago

You are correct, better concede than argue.

What I meant was a tangent: the HTTP was designed with a strong assumption that the application implementing it (the web server) would be the one providing the logic. Hence all the RFC terminology "should", "must", targeted at the implementor.

But very quickly, the logic was deferred to a layer on top (PHP,...) which would focus on the business aspect. The wiring was strong but the contract intent is loose (the requirements on the transport do not apply to the function).

Different layers, different people involved. What survived are the more or less conventions about it, which for the ease and limitations imposed by the protocol layer, led to infinite discussions about GET having a body or not.

The whole question arises because there is this clash between transport and logic that is wired on top, not built-in.

So while indeed, introducing QUERY solves a protocol gap, the people designing the business method never cared (or even knew) about that gap in the first place. This was another people's job to try to reconcile the two.

That's why I'm saying that digging into the initial assumption that the implementor of the HTTP is bound to business-level contraints is not reflecting the reality that has been going on since the early days of the dynamic web.