Origin private file system

77 pointsposted 2 days ago
by thunderbong
(developer.mozilla.org)

48 Comments

mg

2 days ago

From my perspective, the biggest flaw in FireFox is that it does not support the File System Access API.

Because with it, we can offer users to hold their data natively on their devices. Instead of storing everything in the cloud.

Chrome on the desktop supports it. Here is a demo:

https://googlechromelabs.github.io/text-editor/

A text editor that works just like a native application.

And mobile support is also in the making:

https://issues.chromium.org/issues/40101963

I can't wait to use this in my web applications. Finally a way to build proper tools in the browser.

The "Origin private file system" that this story links to is just like the infamous IndexDB. Something that the user cannot backup, cannot use in other software and that is only kept around for as long as FireFox feels like.

lioeters

2 days ago

> kept around for as long as FireFox feels like

I've heard Firefox keeps OPFS data in browser as long as there's enough space (which can be checked from JS). It's been reliable from my experience, but backup is still necessary by saving to remote server or export as local file. Safari has a stricter strategy and removes OPFS data and local storage more aggressively, for example, if a site domain hasn't been visited in 7 days.

On the File System Access API, it seems doubtful that Mozilla will ever implement it in a useful way.

> Mozilla's Position

> There's a subset of this API we're quite enthusiastic about (in particular providing a read/write API for files and directories as alternative storage endpoint), but it is wrapped together with aspects for which we do not think meaningful end user consent is possible to obtain (in particular cross-site access to the end user's local file system). Overall we consider this harmful therefore, but Mozilla could be supportive of parts, provided this were segmented better.

https://mozilla.github.io/standards-positions/#native-file-s...

jauntywundrkind

2 days ago

The restrictions as Mozilla has created them value security so highly that they won't allow webapps to access users files (for fear that maybe the user allows multiple sites access to data that sites side-channel through).

Just one idiots opinion here, but that seems like an incredibly bone headed way to take the whole project of the web platform & make sure it remains utterly unusable at replacing far less secure regular apps. And consigning the web to forever be dependant on cloud data systems, since there's no local access.

I really wish there were more than three parties, wish there was more than Google and Google alone that actually wanted the web platform to succeed. I'd kept some reservation that maybe Mozilla was being cautious, but after seizing the opportunity to grand-stand with Apple against a lot of pretty boring nice APIs that are fine behind permissions prompts in 2020, it feels like the have kept such a strong will to limit and restrain the web. https://www.zdnet.com/article/apple-declined-to-implement-16...

lioeters

a day ago

I agree, Mozilla and Apple are holding back the web with their approach to developing new browser features.

> Technologies that Apple declined to include in Safari: Bluetooth, MIDI, USB, serial interfaces like microcontrollers, 3D printers..

All these could be available by opt-in user permissions, but instead it's Apple's decision to not provide them at all.

They claim it's for security but it sure smells like a business decision.

hedora

2 days ago

As I understand it, Safari won’t auto delete data stored by web apps that are launched from the home screen.

But, yeah, handling of this stuff should be better. I’d personally prefer it if the API’s required user opt in, and did not include auto deleting durably stored data.

cxr

2 days ago

Browsers have existing file APIs that programmers could use but don't. To reiterate a comment from the last time this came up:

> You could literally run the Golang compiler in your browser if you wanted to[...] A lot of the stuff that people think they need Chrome's proposed filesystem APIs for are not actually things they need.

<https://news.ycombinator.com/item?id=41176674>

jitl

a day ago

Literally no one wants to work like this. A save dialog every time I press cmd-s? If we shipped local file editing for Notion that worked like this our users would egg our office. For any app outside maybe like an image macro meme generator this sucks. For anything else like for example a notes app, photo library, more serious image editor, PowerPoint, video editor, animation software, anything people spend >1 hour a day in - you want to auto save, and often you want to manage a tree of files in a root folder as a “library” of the users’ data.

Electron only exists because browsers are too limited. That’s all electron is anyways, a browser with much better APIs for writing apps then the ones included in Chrome and especially Firefox.

cxr

a day ago

Geez, you fucking people...

An API is not by definition fine-grained control over UI/UX.

Browser makers can implement the new file system API with better UX than what they have now. They can also do that with the existing file APIs; there's no contract involving the existing APIs that says things have to work the way they do right now. (They could also implement the new APIs in a way that technically works but has just as shitty or worse UX than the existing ones.)

If you want good/better UX for file save operations, then lobby for that. A whole new (sprawling, bad) set of APIs for giving websites access to the user's filesystem is an orthogonal matter. You're conflating two separate things.

jitl

a day ago

I do not want better UX to read and write a single file. None of the example apps I mentioned work well with R/W to a single file except maybe the image editor example.

I want permission to create, read, write, move/rename files and directories within a root directory selected by the user. Anything less is not sufficient for AAA quality local-first apps.

I think window.showDirectoryPicker() is fine. I hope Firefox will implement this or a similar API, but I am also okay with launching major features without Firefox support. I have done so before to no discernable adverse effects.

https://developer.mozilla.org/en-US/docs/Web/API/Window/show...

cxr

a day ago

Nothing I just said is limited to the narrow use case of dealing with single files. If you want improvements for multiple files, then congratulations, you can have that, too!

> I want permission to [...] move/rename files [...] Anything less is not sufficient for AAA quality local-first apps.

And robbers want to be able to slip their fingers into the wallets of whomever they want. So?

Very few apps outside of file managers actually need to be able to move and rename the user's files, and it would be much better if they those that don't would assume they aren't going to be able to. You can call this a prereq for high-quality apps, and I'll maintain my view that apps that refuse to work unless you grant an overbroad set of capabilities that they don't really need are ipso facto low-quality.

Besides, if you actually want all this, then you can build an app for remoteStorage.io right now, without needing to know whether the files live on the user's disk or not—and if the user wants the storage area they connect to your app to to be backed by local files, then that's a choice that they can make. Your app doesn't need to be able to discriminate between whether they are or they aren't.

jitl

a day ago

Why are you comparing renaming a file inside a directory picked or created by the user to robbers picking a pocket? It's quite typical to implement "atomic writes" by writing to a temporary adjacent file, and once the write is complete, move it to the actual destination filename, possibly overwriting the existing file. For example when exporting a video in a video editor, you might write it first to "$APPDIR/Trip to Spain.webm.incomplete" and then move it to "$APPDIR/Trip to Span.webm" once the file is valid.

I am fine if Firefox or any other user-agent mandates that the user can only grant write access to empty directories in order to protect their users from robbers. I'd still be able to build and organize a library of the user's data there, and they'd still be able to edit files with other native apps without needed to upload or download anything.

I think the remotestorage.io suggestion a little odd, since the only way it seems one can expose a local filesystem to remotestorage.io apps would be to ask my users to download, install, run, and secure a native, non-web HTTP server program, or to sign up for a third party hosting provider where the files will be off in a cloud. That does not sound like a "local first" app to me :(

I find it ironic that a protocol called "remotestorage" touts that it is offline-first design because it keeps user data in inaccessible user-agent storage until the user goes back online, at which point it writes that data to a remote storage provider. Only once the file is written to a 3rd-party server can the user can download the file back to their filesystem. Wouldn't writing a file to the user's filesystem be a more convenient and simple system?

porridgeraisin

2 days ago

Can't imagine a worse alternative. The filesystem api is really good. For example, I use the google office suite entirely offline through the official extension that uses the filesystem api. You can edit a local file just like that.

Same goes for photopea.

You can't seriously suggest manual save as an alternative. Ctrl+S, choose file, do you want to overwrite, yes. That's way too much friction, come on.

PittleyDunkin

2 days ago

Ok but that outlines a miserable user experience.

user

a day ago

[deleted]

jeroenhd

2 days ago

Call me old-fashioned, but I don't want browsers to alter the files I've downloaded after I downloaded them. Chrome gives absolutely no indication that it can still access the files I've "saved".

zb3

2 days ago

Seems that Mozilla is detached from reality, not only because of this, but they also refuse to implement other APIs like WebUSB providing bizarre claims about security at the same time neglecting valid arguments about web apis being safer than native apps..

DrillShopper

2 days ago

Why even bother to sandbox the browser if it can scribble all over my USB devices?

exe34

2 days ago

it can't, you have to pick which device it can use.

M95D

2 days ago

I don't remember Windows or Linux providing this kind of security granularity. Or a security prompt such as the one you describe. USB is all-or-nothing.

rcxdude

a day ago

No, it's the browser that mediates the access from the websites.

DrillShopper

a day ago

Okay, and what makes sure the browser isn't stepping outside the granted access?

zb3

a day ago

The bounty you'd get if you found a way to bypass this.

exe34

a day ago

the same thing that makes sure the browser doesn't give them full access to your files. nothing.

hoten

2 days ago

I've had a couple webapps where I needed to use the FS API (which allows users to provide real, accessible folders to a webapp, unlike OPFS).

I can't understand why anyone would want to use OPFS or what problems it solves that a slim interface over IndexDB wouldn't provide. The only use I've had for it has been a almost-ok substitute for the actual FS API (given its the same interface), but since the goal is to allow users to have easy access to your webapp's data, it has to be paired with a cumbersome import/export feature for browsers that only have OPFS.

I guess it's probably more performant for FS-like queries (like listing the contents of a directory...), but that performance benefit (for me at least) seems secondary to giving users control over the data.

jauntywundrkind

2 days ago

It bristles me a lot that browsers are sometimes only offering highspeed FileSystemAccess to OPFS only, not real files. Letting users interact with their files well is a basic decency. Webapps which makes it possible to expose their state & let users see that is a service to users, shows respect. It's good to share your state, bad to hide it in some secret hidden away internal place.

As for why not indexeddb, there were some great great details on Notion & their wasm sqlite decision. They tried LocalStorage, sucked, tried IndexedDB, but they said theres a very significant cost per row written or read & that it just was way too slow for their use case of lots of small rows. Really sad to hear, really sucks that the browser once again built some fancy safe secure developers-wont-be-able-to-do-harm approach that ended up being so slow as to be unusable. https://news.ycombinator.com/item?id=40989990

The too slow to be usable feels like the last File System API, which lets us do files but in a ridiculously slow fully-buffered form. Because again the spec authors prioritized making a safe secure API that was free of any potential pitfalls that no developer could ever mess up. There-by making something essentially unusably slow except for the trivialest of uses.

tommiegannert

2 days ago

> To summarize how the OPFS differs from the user-visible file system:

> * The OPFS is subject to browser storage quota restrictions, just like any other origin-partitioned storage mechanism (for example IndexedDB API). You can access the amount of storage space the OPFS is using via navigator.storage.estimate().

> * Clearing storage data for the site deletes the OPFS.

> * Permission prompts and security checks are not required to access files in the OPFS.

> * Browsers persist the contents of the OPFS to disk somewhere, but you cannot expect to find the created files matched one-to-one. The OPFS is not intended to be visible to the user.

> * It also has a set of synchronous calls available (other File System API calls are asynchronous) that can be run inside web workers only so as not to block the main thread.

TheRealPomax

2 days ago

This feels a lot like giving companies a way to store data on my device without me being in control of that data. If your web app is going to store data, I want the power to audit that data, not have the browser go "oh, sorry, can't let you do that, that would hurt someone else's bottom line".

drdaeman

2 days ago

They already do this with cookies, local storage, IndexDB and WebSQL. OPFS is not changing anything - I'm pretty sure it'll be visible in Developer Tools just like other data storages.

The issue is that browser vendors are making design decisions that do not respect power users, but this issue is way bigger and more complicated (and quite different) than some data storage oddities. There are only a few browsers out there (forks don't matter, they don't change architecture), and no single one of them has modular design with user-replaceable parts talking over well-defined interfaces, classic UNIX-way style.

If anything, inability of a website to put a known-content file in user-accessible file system is a security feature. If a website would be allowed to place a file with arbitrary content under a well-known or mostly-known location somewhere in user's home directory, that could lead to unwanted consequences.

behringer

2 days ago

If it doesn't change anything why is it needed? It clearly changes something and I'm not convinced it's for my, the user's, benefit.

azornathogron

2 days ago

It just provides a different API for storage. You could say the same thing about IndexedDB being added when LocalStorage already existed - it's just a different API with some different performance characteristics.

And no, it's not for the benefit of you the user, but it's not to your detriment either. It's mostly for the benefit of the web developer. (It might be good for users indirectly if web developers make good use of it)

behringer

a day ago

I the user would prefer the developer use regular file storage.

bflesch

2 days ago

You didnt read the website. It says that you wont be able to find the files on your disk, even when searching for the contents. So Firefox will be actively encrypting / obfuscating them.

That's not the case for localStorage or cookies or any other current option, because all of them you can access both on the disk and via devtools.

drdaeman

2 days ago

I'm sorry? I did, and I just skimmed over it again, and I still don't see where it says anything along the lines of "even when searching for the contents".

The only relevant piece I see in the article is this (emphasis mine):

> Browsers persist the contents of the OPFS to disk somewhere, but you cannot expect to find the created files matched one-to-one. The OPFS is not intended to be visible to the user.

And that matches how cookies and localStorage works. They aren't files either, they're in a SQLite database somewhere out there.

Can you please give me a quote where it says so?

Upd: There's Chrome (web.dev) article linked from MDN that explicitly mentions a way for end-user to browse this filesystem: https://web.dev/articles/origin-private-file-system#debug_th... - I'm pretty sure Firefox has (or will have, idk how mature is this standard) something similar.

How those files are stored on disk is an implementation detail. All article says that user should not expect them to match their local filesystem conventions, hinting that it's probably going to live in some kind of a database rather than match host filesystem conventions.

Upd2: I've checked out the demo (https://sqlite-wasm-opfs.glitch.me/) and it looks like, indeed, Firefox is missing an ability to introspect the storage at the moment. Or at least I don't see anything in DevTools on Firefox 131. This is disappointing and concerning, but I strongly suspect it's a bug/missing feature rather than a deliberate design decision.

Upd3: I see that demo's site data near-pristine content-wise under ~/.mozilla/firefox/$profile/storage/default/https+++sqlite-wasm-opfs.glitch.me/fs/... While filenames are nonsensical, the SQLite3 demo database (with that single table "t") that's created in OPFS is accessible in my host machine's filesystem verbatim:

    $ sqlite3 KF3XI6X7GVF3O6IT3QGODMSZ4OMH56DTTUBOK23MNTB6ODM5HWEA
    SQLite version 3.45.3 2024-04-15 13:34:05
    Enter ".help" for usage hints.
    sqlite> .tables
    t
    sqlite> SELECT * FROM t LIMIT 3;
    20|40
    21|42
    22|44

TheRealPomax

2 days ago

Note that if it shows up in the dev tools "Storage" tab, then that's fine, that's how you audit cookies and local/sessionStorage just fine already. But from what I can tell, part of the "no no, we call it private for a reason" is that they're not getting exposed that way. If finding a sqlite3 db and manually working in that is the only way to audit the data, that's a dark pattern right there.

I'd be happy to be wrong about that, though.

drdaeman

2 days ago

For some reason it's not visible in DevTools' "Storage" tab (Firefox 131.0.2). I strongly suspect it's negligence rather than malice, though.

TheRealPomax

a day ago

Probably because this spec hasn't been accepted yet, it's still a draft.

DrillShopper

2 days ago

I'm definitely turning it off if there's a toggle in about:config

user

2 days ago

[deleted]

zb3

2 days ago

Anything that makes the web platform more powerful is a good thing. The web is a secure, sandboxed platform, but it's also an open platform. Screw Apple and their extortions, develop web apps instead.

jitl

a day ago

Apple users pay for good apps, which is nice if you’re a developer wants to make a good app and also make some money.

zb3

a day ago

Then shouldn't free apps also be free for the developers to make? Sure, users should have a choice, but now it's apple who has both choice and the 30% ransom.