Repository navigation
can we convert existing quantized tflite files to LiteRT AOT ones? #1147
Description
FYI, as May 29th, 2026 Taiwan Time.
| model name | compiled |
|---|---|
| Mobilenet V4 | only works for Q SoCs |
| MobileDET | works for G, Q, and most of M (except MT6893 and MT6895) |
| Mobile Segmenter | works for G, Q, and most of M |
| Mobile BERT | works for G and Q |
| EDSR | G (except G6) and Q |
Nope, I didn't try to run them. Only tried to convert/compile them.
Colab notebook I used, https://colab.research.google.com/drive/1aJCNhaq5QS-UTb5nXaWnkUueXFRNR7Ow?usp=sharing
Activity
[-]can we convert existing tflite files to LiteRT AOT ones?[/-][+]can we convert existing quantized tflite files to LiteRT AOT ones?[/+]freedomtan commentedon Jun 16, 2026
I did try the converted models on Pixel 10 Pro. They work to some extent (I won't say great, because performance numbers are much lower than I expected).
freedomtan commentedon Jun 16, 2026
all ops will be wrapped into a single op called DISPATCH op.
@freedomtan to share numbers on Pixel 10 Pro.
freedomtan commentedon Jun 17, 2026
the 6.0 app on Pixel 9 and 10 Po.
As you can see, Pixel 10 numbers are clear worse than Pixel 10. It's clear that he MobilerBERT on Pixel 10 fell back to CPU.
With benchmark_model for tflite and benchmark_model for litert, we can get
unit: ms
LiteRT numbers on Pixel 10 are not better than TFLite + NNAPI.
Mostelk commentedon Jun 18, 2026
@freedomtan can you point us to the CL apk you are testing with
freedomtan commentedon Jun 18, 2026
I used
benchmark_model.for CompiledModel API (LiteRT new API): https://developers.google.com/edge/litert/next/benchmark
for Interpreter API (the TFLite one): https://developers.google.com/edge/litert/models/measurement
freedomtan commentedon Jun 23, 2026
unit: ms
Google provided Android 17 for Pixel devices, with that, we can have JIT work.
freedomtan commentedon Jun 23, 2026
@freedomtan to try if we can make Samsung's AOT work with samsung pip wheel.
freedomtan commentedon Jun 29, 2026
With this, I could compile/convert the MOSAIC (all others failed).
freedomtan commentedon Aug 18, 2026
benchmark_model for litert: https://developers.google.com/edge/litert/next/benchmark
benchmark_model for tflite: https://developers.google.com/edge/litert/models/measurement
@Mostelk
freedomtan commentedon Oct 22, 2026
I revisited model conversion just now, https://colab.research.google.com/drive/1XV39i6LhaDktMyie8HOQLLu38UgLCpW6?usp=sharing
with ai-edge-litert-nightly and ai-edge-litert-sdk-*-nightly
Both Google's and MediaTek's supports for compiled_model are improved.
@Mostelk for your reference.