The same stuff that powers this?

https://3d.kalidoface.com/

It's pretty impressive that this runs on-device. It's better than a lot of commercial mocap offerings.

AND this was marked deprecated/unsupported over 3 years ago despite the fact it's a pretty mature solution.

Google has been sleeping on their tech or not evangelizing it enough.

I got it all set up and tested out Gemma3 1B on a Pixel 8a. That only took a few minutes, which was nice.

But it was garbage. It barely parsed the question, didn't even attempt to answer it, and replied in what was barely serviceable English. All I asked was how it was small enough to run locally on my phone. It was bad enough for me to abandon the model entirely, which is saying a lot, because I feel like I have pretty low expectations for AI work in the first place.

Presumably because this an "experimental Alpha release".

Beta releases are possibly on the Play Store, but it sounds like they're not ready yet.

Some of the mediapipe models are nice, but mediapipe has been around forever (or 2019). It has always been about running AI on the edge, back when the exciting frontier of AI were visual tasks.

For stuff like face tracking it's still useful, but for some other tasks like image recognition the world has changed drastically

I would say the target audience is anyone deploying ML models cross-platform, specifically ones that would require supporting code beyond the TFLite runtime to make it work.

LLMs and computer vision tasks are good examples of this.

For example, a hand-gesture recognizer might require: - Pre-processing of input image to certain color space + image size - Copy of image to GPU memory - Run of object detection TFLite model to detect hand - Resize of output image - Run of gesture recognition TFLite model to detect gesture - Post processing of gesture output to something useful

Shipping this to iOS+Android requires a lot of code beyond executing TFLite models.

The Google Mediapipe approach is to package this graph pipeline, and shared processing "nodes" into a single C++ library where you can pick and choose what you need and re-use operations across tasks. The library also compiles cross-platform and the supporting tasks can offer GPU acceleration options.

One internal debate Google likely had was whether it was best to extend TFLite runtime with these features, or to build a separate library (Mediapipe). TFLite already supports custom compile options with additional operations.

My guess is they thought it was best to keep TFLite focused on "tensor based computation" tasks and offload broader operations like LLM and image processing into a separate library.

In the model's community section Goog confirms they're working on a gguf version so you can host it like most other models.

https://huggingface.co/google/gemma-3n-E4B-it-litert-preview...

As far as I know it's based on Gemini nano architecture which exclusively runs on Android and Chrome. So I'm guessing you can't run it on Linux outside Chrome.

This isn't really true. They are different offerings.

CoreML is specific to the Apple ecosystem and lets you convert a PyTorch model to a CoreML .mlmodel that will run with acceleration on iOS/Mac.

Google Mediapipe is a giant C++ library for running ML flows on any device (iOS/Android/Web). It includes Tensorflow Lite (now LiteRT) but is also a graph processor that helps with common ML preprocessing tasks like image resizing, annotating, etc.

Google killing products early is a good meme but Mediapipe is open source so you can at least credit them with that. https://github.com/google-ai-edge/mediapipe

I used a fork of Mediapipe for a contract iOS/Android computer vision product and it was very complex but worked well. A cross-platform solution would not have been possible with CoreML.

Tensorflow light has been battle tested on literal billions of devices over the years and this looks like a rebrand/extension of that plus media pipe, one of the biggest users of it. Google has been serious about on device ML for over 5 years now, I don't think they are going to kill this. Confusingly rebrand it maybe :)

The generative AI piece is not available in Apple ecosystems right? I think that would be huge and I really hope Apple gives us something similar. And I gotta say the chat piece of this seems really useful too.

Also where the f is Swift Assist already

Its just a rebranded tensorflow lite, Ive been using that on edge devices since 2019... CoreML is great too!

My brother in Christ, CoreML only exists because Apple saw Tensorflow and wanted the featureset without cooperating on a common standard. TF was like 2 years old (and fairly successful) by the point CoreML was announced. To this day CoreML is little more than a proprietary BLAS interface, with nearly zero industry buy-in.

Terrifying what being an iOS dev does to a feller.

is it possible to go to the iPhone app store and get an app that is essentially an ollama like model downloader and launcher?

ONNX exists but since they don't change their name very often not a whole lot of people know about it.

For one thing, executorch is currently full of sharp edges. No idea about this, but i had bad experience with ET. If i were starting over, I might start from torch.fx and automate from there. fx is stable and should be around for a while.

[dead]

it does support python and web, and runs on raspberry pi.

Perhaps you missed the associated documentation? This is a classification tool which requires input labels "uses an EfficientNet architecture and was trained using ImageNet to recognize 1,000 classes, such as trees, animals, food, vehicles".

The full list [1] doesn't seem to include a human. You can tweak the score threshold to reduce false positives.

1: https://storage.googleapis.com/mediapipe-tasks/image_classif...