I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand.

But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.

You don't want a browser that takes 10GB of extra RAM (on top of the memory hog that is having just 3 or 4 complex tabs open on its own already) and pegs your CPU at 99% usage for minutes at a time. And not in an era when mass market consumer laptops are still commonly 8GB or 16GB of total system RAM. Many of those with integrated-into-CPU onboard graphics (eg: not a gaming laptop with a discrete GPU on the PCI-E bus).

It'll be a catastrophe for laptop battery life, among other resource use problems. And an LLM that fits in under 8 to 10GB of RAM for CPU-only inference is not going to be nearly as capable as an off-device inference system.

I wish they had just done this with a very clear up front opt in (not enabled by default) thing that explains what Mistral is, that it's not some big American cloud company but a relatively small startup in France, and that your prompts/LLM interactions will go to their servers. And some documentation on how it will be handled/stored in a supposedly trustworthy manner.

The privacy policy for Smart Window[1] indicates that prompts are not only sent to an LLM provider (Mistral in this case), but also to Mozilla servers first.

  > The assistant sends the full prompt (including your query, any relevant Memories, and any additional relevant browsing context) to a Mozilla server.
  > 
  > Upon receipt, Mozilla forwards the request to a third-party large language model (LLM) on your behalf. The LLM receives the request from Mozilla, not directly from you, and sees a Mozilla IP address rather than your own. The request includes only the information needed to generate a response or perform the requested action.

  > When you submit a query in the Smart Bar, the assistant uses a local (on-device) intent classification model to determine whether the query is best addressed by a chat or a search.

I can't fathom how they'd consider "Private" something that uploads all the sensitive information in readable form to a third party service.

In fact, If I were asked to explain the opposite of private, I might give this kind of design as an example.

> This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud.

I certainly agree that this option would be nice to have, but running firefox without an ad blocker on the modern web will ALREADY spin up my fans just rendering all the garbage, and I have a decent laptop.

You can't expect the median Firefox-running device to be able to run a capable enough local model effectively.

> This is an excellent use case for completely local, small model inference

Is it really ? LLM take lot of ram and drain battery. People run Firefox on low end computer.

> This is an excellent use case for completely local, small model inference,

How small is this 'small model' inference?

Browsers especially are expected to operate on a very wide number of devices. It makes perfect sense to me to start off with a cloud driven approach and later allow people to configure it to run locally if they have the hardware to do so. Anyone with the hardware to do so is already signaling that they'd have the technical interest to make that change while having the default 'works for everyone' mode be one that doesn't require a powerful local device.

I say this as someone using a 6 year old computer and only upgraded to 8GB of ram when the latest WoW expansion was released.

the average user has nowhere near enough the capability to run a model fast enough to do useful things

Anyone else's extensions not work on support.mozilla.org?

This is ridiculous. Not only does Mozilla support BYOM with Smart Window, but do we seriously expect your average user to be able to locally run it with ease?

Imagine for a moment the complaints that would arise the very next day because their browser is now revving up their laptop fans and hoarding precious RAM.

Local models aren't yet light enough to handle the required processing at the lowest end, so Mozilla chose the most responsible stewards they could find. There is no news here.

Thanks for mentioning the mistake on the link, it's been reported internally.

zdr is good, really confidential inference with attestation would be better. the same as a confidential vm TEE which confirms the integrity and privacy.

there are good providers available, i think it should become the standard.

two high quality companies: near.ai tinfoil.sh

its up to us to make extensions that are so famous that they can be integrated into the browser functionality i guess

Except ... this is not local. It's a remote service. Here is something that is:

    docker run ghcr.io/open-webui/open-webui:v0.11.3
    docker run mcr.microsoft.com/playwright/mcp:latest

I don't think it is comparable. Nowhere in the news they say it is offline but IMO Gemini Nano is offline (?)

I'm not a Google fan but I never understood the outrage as it is a local model ... sure, you have to trust them, but if you use their browser you already do.

That’s a great idea, and would be a much better feature than the unimaginative things Mozilla has been implementing. It’s so disappointing.

I think that's roughly what duck.ai is doing? If I type that in there I see it generating some search queries and then generating a list of links based on the different result pages.

It wouldn't be cool because it would still hand your data to Google.

Both are true, a lot of people are multilingual but a lot are also not.

I know a lot of foreigner for which the feature is really useful (local department store, booking a doctor appointment etc.)

I think that beyond marketing this is also a way to collect "AI browsing" trajectories. Not an expert but I think this is the kind of great quality data that can be useful when training LLMs.

It's pretty nice for tedious tasks. I tried to have it look through job listings based on my criteria. It burned through my 5h Claude quota, but what I got was pretty impressive.

I agree on the privacy concerns but this could be useful with a local model, I think.

Palemoon doesn't embed clankers as far as I know.

SeaMonkey exists. But, I'm sure Mozilla will fuck it up if they ever remember it exists.

Waterfox also works well!

AI are excellent at ingesting tons of data and getting something relevant from what you asked. I ask it about linux behavior and it will dig through hundreds of patches & mails and will return me informations sourced with links to the patches, mails containing it's affirmations.

I understand why they built this. They probably should and this is probably the right way to do it.

And also yes, I’ve tried Dia and OpenAI’s thing and I can’t for the life of me figure out what to do with the feature.

And then it dawns on me what I really want is a modernized bookmarks system…

All previous similar issues got hand waved with "Mozilla has to make money somehow". I am afraid one day remaining users will decide that Chromium based browsers provide a better experience in most use cases, and now have same privacy guarantees as Firefox. Looking at market share maybe that had already happened and pragmatism won over ideology, and we will all suffer the consequences of such monoculture.