Gemini 3.8 Flash and 3.8 Flash Cyber

76 pointsposted 5 hours ago
by simonsarris

20 Comments

kamranjon

4 hours ago

They've interestingly left out any mention of speed.

I have been testing 3.7 flash against 3.5 flash and it seems to lose every time in overall latency. Every benchmark I've seen seems to suggest the opposite[1] - that 3.7 flash is significantly (at times 2x) faster than 3.5 flash - but I have never been able to prove this out in real world use cases.

Has anyone found their latency numbers to actually be accurate? Is this why they've toned it down in this release? For context, I'm testing larger generation payloads that take 8-10 seconds in 3.5 flash and 15-25 seconds in 3.7 flash. Lowest reasoning settings in both cases.

1: https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...

film42

4 hours ago

It depends on how you're querying Gemini models. OpenRouter is the fastest by far. I'm guessing they bought the dedicated pipe from Google. Gemini via VertexAI and consumer API has pretty bad latency.

kamranjon

4 hours ago

Yea I am testing through OpenRouter - have you noticed 3.7 flash being significantly faster?

film42

2 hours ago

I guess it might be relative, but switching from VertexAI endpoint to OpenRouter was like 2-3x faster for us.

atemerev

4 hours ago

Everyone is censoring models now with anything remotely resembling cyber or bio. I already have problems with my research in mathematical epidemiology because of that - both Sol and Fable simply refuse. They keep pushing people towards Chinese models that can be decensored.

vehemenz

4 hours ago

Supposedly Fable 5.1 is better, but I haven't tried it yet. I've run into the same thing with mundane work that is barely bio/cyber adjacent.

Re: Chinese models, even if the model itself isn't censored, some of the big model providers have guardrails now that you can't exceed, which somewhat defeats the purpose.

atemerev

an hour ago

"Uncensored" means "weights modified to remove refusals". Abliterated. Providers do not serve such models, at least not frontier-grade. You have to run the weights yourself. For Kimi K3, this is about $60/hour for hardware rental. But you can have about 100 sessions simultaneously.

And yes, Fable 5.1 has the same refusal rate, and significantly nerfed reasoning.

sergiotapia

4 hours ago

Who coined the phrase "cyber" for security related things lol. It's so 1999.

estearum

4 hours ago

Hasn't the field been called "cybersecurity" since... forever?

user

4 hours ago

[deleted]

sumeno

3 hours ago

I assure you that in 1999 "cyber" meant something very different

josefresco

4 hours ago

Cyber is more of an early 1990's thing, and I have no issue with it unlike most in the tech field. I feel like it dropped off in the late 90's and early 00's but made a comeback as hacking became a mainstream security issue.

cleverpotato479

4 hours ago

Amusingly, "cyber" comes from the word "kubernetes"!