ExecuTorch vs TVM vs LiteRT: 2026 on-device AI shootout

Why three frameworks, and why now

If you want to run a neural network on a phone, a Raspberry Pi, a smart toaster, or a laptop without a beefy GPU, you used to have one reasonable choice: TensorFlow Lite. In 2026 you have three serious ones — ExecuTorch, Apache TVM, and LiteRT (the renamed TensorFlow Lite) — and they each take a fundamentally different view of what “on-device AI” should look like. Picking the wrong one costs you weeks of porting. Picking the right one is the difference between shipping a 200 ms per-token LLM and a 50 ms one.

This article is a concept-level deep dive, not a benchmark. By the end you’ll know what each framework is good at, what its weak spots are, and which one to reach for first given your model and your target hardware.

The shape of the problem

All three frameworks solve the same problem: take a trained model produced by PyTorch / TensorFlow / JAX / ONNX, optimize it for a constrained device, and run it efficiently on that device’s CPU, GPU, or NPU. They differ on three axes:

• How much compilation they do. Some run a graph mostly as-is and rely on hand-written operator libraries (XNNPACK, Core ML). Some compile the graph all the way down to native code for the target CPU (TVM). LiteRT sits in the middle.
• Where the hardware vendors plug in. Every modern phone has an NPU. Each framework treats vendor NPUs differently — as a black-box “delegate” you hand the whole graph to, as a target you compile subgraphs into, or as a runtime library you call.
• How they handle new model architectures. The interesting 2026 work is LLMs and diffusion. Some frameworks treat those as “just another graph.” Some have explicit first-class stories (ExecuTorch’s LLM examples, TVM’s MLC LLM family, LiteRT-LM).

ExecuTorch: the PyTorch-native path

ExecuTorch is the PyTorch Foundation’s answer to the “I trained a model in PyTorch, how do I run it on a phone?” question. Version 1.3.1 shipped on 2026-05-29; the project is owned by Meta and developed under PyTorch Edge alongside the torch.compile / torch.export story.

Architecture

Export happens through torch.export — the same export pipeline you’d use to compile a model with torch.compile — which produces an ExportedProgram that ExecuTorch lowers into a flatbuffer .pte file. The runtime is a small C++ library that interprets the program, with optional backend delegates for hardware acceleration. The whole thing is designed to be embedded: minimal dependencies, no Python at inference time, runs on everything from AR glasses to microcontrollers.

Backends

The prebuilt executorch pip wheel ships with four backends wired in:

• XNNPACK — CPU inference on x86 and ARM via the XNNPACK operator library. This is the default. Fast on ARM NEON, decent on x86.
• Core ML — Apple Neural Engine and GPU on iOS / macOS. Wired in on macOS and iOS builds.
• MPS — Metal Performance Shaders on macOS / iOS for GPU paths that Core ML doesn’t cover.
• QNN — Qualcomm AI Engine on Linux x86_64 hosts that target Snapdragon devices.
• OpenVINO — Intel CPU / GPU on Linux hosts.

If you need something exotic (MediaTek NeuroPilot, Samsung Exynos, a custom NPU), you write your own backend — the delegate API is stable and the community has shipped half a dozen by now. ARM backend (for Cortex-M and Ethos-U microcontrollers) lives separately.

What it’s good at

• If your model is PyTorch and you didn’t go through heavy graph surgery, export is basically a no-op. You don’t rewrite your model in a separate framework.
• LLMs and other large dynamic-shape graphs have first-class support via the examples/models/llama runner and the AttentionSink extension for long context.
• The same .pte file runs on every backend with operator partitioning, so you can ship one artifact and let the runtime dispatch CPU vs GPU vs NPU per-op.

What it’s not as good at

• Operator coverage is incomplete on non-XNNPACK backends. Things work until you hit a model that uses an op the NPU delegate doesn’t have, then you fall back to per-op CPU execution and lose most of the speedup. The 1.3 release notes show steady progress but it’s still the bottleneck.
• No general-purpose graph optimization. TVM has years of auto-tuning infrastructure; ExecuTorch mostly delegates to whatever the backend vendor ships.

Apache TVM: the compiler-first path

Apache TVM (top-level project since 2019, ~13.6k GitHub stars, 0.25.0 shipped 2026-06-24) is the oldest of the three and the most ambitious. It’s a full deep learning compiler stack, not just a runtime. You write Python; TVM ingests your model, lowers it through Relax (graph IR) and TensorIR (tensor-program IR), runs a search-based auto-tuner over kernel implementations, and emits either a C++ source library or a JSON artifact you load with the TVM runtime.

Architecture

The pipeline looks like this:

1. Frontend ingests the model. PyTorch (via Relax, supporting ExportedProgram), ONNX, TensorFlow, and a handful of others.
2. Relax is the graph-level IR. It does high-level transformations: operator fusion, layout planning, quantization annotation, partitioning for BYOC (Bring Your Own Codegen) targets.
3. TensorIR is the tensor-level IR. It represents one operator’s compute schedule in a way the auto-tuner can mutate.
4. MetaSchedule (or the older AutoScheduler) searches over loop tile sizes, vectorization widths, unroll factors, memory layouts. It runs on a target machine, measures actual kernel times, and keeps the best schedule.
5. Codegen emits LLVM (for x86 / ARM CPUs), CUDA / HIP (for NVIDIA / AMD GPUs), Metal / Vulkan / OpenCL / SPIR-V (for mobile and web GPUs), or a BYOC target like NNAPI / Android.
6. Runtime is a tiny C++ library — TVM-Runtime — that loads the compiled artifact. No Python at inference time.

The killer feature is the auto-tuner: you tell TVM “make this conv fast on a Pixel 8’s Cortex-A510” and it spends hours searching the schedule space, finds a kernel that’s often hand-tuned-quality, and emits it. This is why TVM is the framework of choice when you have an unusual target — a custom accelerator, an NPU no one else has built a delegate for, a model architecture no vendor has optimized for.

What it’s good at

• Hardware you can’t find a delegate for. TVM has backends for Hexagon DSPs, WebGPU, WASM, RISC-V Vector extensions, and a long tail of academic accelerators. If your target is weird, TVM is the only one of the three that will compile for it.
• Maximum performance when you have time to tune. With hours of MetaSchedule search you can beat hand-written kernels by 10–30%. The flip side is the search cost is non-trivial.
• Vendor agnostic. You compile against LLVM and your target, not against a vendor SDK. No licensing negotiation.

What it’s not as good at

• The build is heavy. Compiling TVM itself takes hours. The dependency tree (LLVM, cuDNN, vendor SDKs) is a real burden if you only need one deployment target.
• Dynamic-shape and control-flow models are still awkward. Relax helps but the auto-tuner assumes mostly-static shapes. LLMs work via MLC LLM (a TVM-based vertical compiler) but it’s a separate repo you have to keep in sync.
• You write more code than the other two. Export is one Python call for ExecuTorch; for TVM it’s a script that imports the right frontend, configures the target, runs the tuner, exports the artifact, and packages the C++ runtime. Templates help but it’s a real project.

LiteRT: the runtime-plus-delegates path

LiteRT is the renamed TensorFlow Lite — Google officially rebranded it in 2024 when they reorganized the AI Edge group. The core idea is unchanged: a tiny interpreter that runs a .tflite flatbuffer model, with plug-in delegates that hand off parts of the graph to GPU or NPU. What’s new in 2026 is the CompiledModel API and unified NPU vendor support.

Architecture

You take a model (originally a TensorFlow SavedModel, now also PyTorch via ai-edge-torch and JAX via jax2tf), convert it through the LiteRT converter into a .tflite file, and run it on the interpreter. The interpreter walks the graph operator-by-operator. When it hits a region the delegate owns (say, all the convs), it hands the whole subgraph to the delegate; everything else runs on CPU via the built-in kernel library.

The 2026 addition is CompiledModel: instead of running op-by-op with delegates, you ask the runtime to compile the whole graph for a target accelerator and get back a callable object. The runtime picks the best available backend (CPU, GPU, or NPU) automatically. This is the path you want for serious performance — the old Interpreter API still works but CompiledModel is where the new investment is.

Backends and NPU support

LiteRT’s 2026 NPU story is the most fleshed-out of the three. Google provides first-party integration with:

• Google Tensor (Pixel phones) — AOT compilation through the Google Tensor SDK.
• Qualcomm AI Engine Direct — both AOT and on-device (JIT) compilation.
• MediaTek NeuroPilot — both AOT and JIT.
• Intel OpenVINO — for x86 inference.
• Samsung Exynos AI LiteCore.

For GPU there’s the standard GPU delegate (OpenGL / OpenCL on Android, Metal on iOS). For CPU there are reference kernels plus XNNPACK — which is interesting because it means LiteRT and ExecuTorch share the same CPU backend.

The NPU integration uses the Dispatch API and Compiler Plugin model: each NPU vendor ships a plugin that knows how to lower a LiteRT graph into their compiler’s IR, and the runtime calls into it. This is a cleaner separation than the delegate model — vendors don’t have to rewrite operators in LiteRT’s vocabulary, they just compile the subgraph.

What it’s good at

• The most complete NPU story in 2026. If you’re shipping to Android phones with Qualcomm / MediaTek NPUs and you want a single framework to target them all, LiteRT is the lowest-friction option. The NPU vendors themselves are motivated — this is where their business goes.
• The smallest runtime. A stripped LiteRT build for a microcontroller is on the order of a few hundred KB. ExecuTorch is bigger; TVM is bigger still.
• The most mature tooling. The TFLite ecosystem has had five more years to stabilize than the other two. The converter is well-debugged, the profiler works, the model explorer GUI is genuinely useful.

What it’s not as good at

• PyTorch-first workflows are second-class. You can convert PyTorch models but the path goes through ONNX or ai-edge-torch (Google’s own converter), and you’re trusting two translation steps. ExecuTorch has the smoother path for PyTorch users.
• The Interpreter API is showing its age. The op-by-op + delegate model is fine but CompiledModel is where new features land, and CompiledModel only handles Float32 cleanly — quantized models still need the Interpreter.
• The Google-controlled governance. If you have a model architecture that Google doesn’t prioritize, you may end up waiting for vendor patches that don’t come. The community is smaller than TVM’s.

Side-by-side: backend coverage, quantization, model types

This table summarizes the 2026 state. “Yes” means first-class, “Partial” means supported with caveats, “No” means you’d need to write your own backend or use a different framework.

Dimension	ExecuTorch 1.3.1	TVM 0.25.0	LiteRT
CPU (x86, ARM)	Yes (XNNPACK)	Yes (LLVM)	Yes (XNNPACK + ref)
NVIDIA / AMD GPU	Partial (XNNPACK delegate for CUDA path)	Yes (CUDA / HIP / Vulkan / OpenCL)	Partial (GPU delegate, experimental on desktop)
Mobile GPU	Yes (MPS, Core ML, OpenCL paths)	Yes (Metal, Vulkan, OpenCL, SPIR-V)	Yes (GPU delegate, mature)
Apple Neural Engine	Yes (Core ML delegate)	Partial (Core ML BYOC)	Yes (Core ML delegate)
Qualcomm NPU	Yes (QNN)	Partial (Qualcomm AI Engine via BYOC, weaker than others)	Yes (QNN via Compiler Plugin, AOT + JIT)
MediaTek NPU	Partial (community backends)	Partial	Yes (NeuroPilot, AOT + JIT)
Google Tensor NPU	No	No	Yes (Tensor SDK, AOT)
Intel NPU / OpenVINO	Yes (OpenVINO backend)	Partial	Yes (OpenVINO Compiler Plugin)
Samsung Exynos NPU	No	Partial	Yes (LiteCore, AOT)
ARM Cortex-M / Ethos	Yes (ARM backend, separate repo)	Partial (Cortex-M, Ethos via TVM-Micro)	Partial (LiteRT Micro)
int8 / int4 quantization	Yes (PT2E quant, per-backend)	Yes (very flexible)	Yes (mature, full-pipeline quantization-aware training toolchain)
FP16 / BF16	Yes	Yes	Yes (CompiledModel), partial (Interpreter)
Dynamic-shape / LLM	Yes (AttentionSink, LLM runner)	Yes (MLC LLM is the production path)	Yes (LiteRT-LM, newer)
ONNX import	Via ONNX → ExportedProgram	Yes (native frontend)	Via external converter
PyTorch export	Native (torch.export)	Via ExportedProgram frontend	Via ai-edge-torch
JAX import	Via export to ONNX	Yes	Via jax2tf
TensorFlow import	Via export to ONNX	Yes	Native
License	BSD-style (PyTorch project)	Apache 2.0	Apache 2.0

A few rows are worth calling out. Google Tensor NPU only LiteRT supports — that’s deliberate; the SDK ships from Google. ARM microcontroller support exists in all three but the toolchain quality is best in LiteRT Micro (most years of battle-testing) and weakest in TVM (Hexagon and Ethos-U work but require more setup). Dynamic-shape LLM support is now genuinely competitive in all three — ExecuTorch’s AttentionSink, MLC LLM’s prebuilt artifacts, and LiteRT-LM’s recent push have converged on similar UX.

Performance: what actually matters

There are no clean cross-framework benchmarks in 2026 — the models differ too much, the backends differ too much, and every framework cherry-picks the workloads where it shines. A few general patterns are reliable though:

• On a Qualcomm NPU, LiteRT is consistently 10–20% faster than ExecuTorch for the same model, because the Qualcomm plugin has had more time to integrate with the LiteRT compiler and because Google and Qualcomm have a closer relationship on this surface. On a Pixel’s Tensor NPU, only LiteRT can run at all.
• On a CPU-only target with custom shapes, TVM’s auto-tuner wins by a margin that grows with the weirdness of the model. A standard ResNet-50 will be within 5% across all three on ARM Cortex-A; a custom transformer with fused attention quirks will be 2× faster on TVM because the auto-tuner found a schedule no one hand-wrote.
• For LLM token throughput on a Mac, ExecuTorch’s MPS backend is in the same ballpark as the LiteRT GPU delegate. Both use Metal Performance Shaders under the hood, so they’re roughly the same speed modulo per-op overhead.
• For binary size, LiteRT is smallest, ExecuTorch is in the middle, TVM is largest unless you aggressively prune the runtime.

The more honest way to read the performance picture: performance is dominated by how well the model’s ops are mapped to the target hardware, and that mapping is framework-specific. Two engineers with the same model on the same phone can easily see 3× performance differences by picking the wrong framework-backend pair. Run your actual model on your actual hardware before committing.

Which one to use when

You’re shipping a PyTorch model to a phone or embedded device

Start with ExecuTorch. The export path is the smoothest, the LLM tooling is mature, and the per-backend delegate coverage is enough for most vision / speech models. Only switch if you hit an op coverage gap that doesn’t have a workaround.

You’re shipping to an NPU that has first-class LiteRT support (Qualcomm, MediaTek, Pixel, Exynos, Intel)

Use LiteRT. The vendor plugins are the most complete in the ecosystem and they get the actual NPU peak performance. The PyTorch conversion friction is real but it’s a one-time cost.

You’re targeting unusual hardware — a custom accelerator, a WebGPU browser, RISC-V vector, a Hexagon DSP, a new phone SoC

TVM. It’s the only one of the three with the compiler infrastructure to target hardware no one has built a delegate for. Budget the auto-tune time.

You’re shipping a small model to a microcontroller

LiteRT Micro or ExecuTorch’s ARM backend. Both are viable. LiteRT Micro has more years of production hardening; ExecuTorch’s ARM backend has cleaner integration with the main PyTorch Edge ecosystem.

You’re a researcher comparing kernel schedules or studying compiler infrastructure

TVM. TensorIR and MetaSchedule are the most academically interesting of the three and the most documented. ExecuTorch’s runtime internals are less accessible; LiteRT’s NPU plugins are closed-source.

You want one framework that handles everything from phone to laptop to server

None of them. All three have desktop GPU paths but none of them are the right choice for serious server-side inference — that’s what PyTorch / JAX / vLLM / SGLang are for. On-device frameworks are for the cases where the data has to stay on the device or the latency budget doesn’t allow a round trip.

What’s still broken in 2026

A few things haven’t been solved by any of them:

• NPU fragmentation is still the elephant. Each vendor’s compiler wants a slightly different graph, opset, and quantization format. The LiteRT Compiler Plugin model is the most promising attempt at unifying, but real cross-vendor portability is still months away. ExecuTorch has the same problem with QNN being Qualcomm-only.
• Dynamic-shape quantization is rough in all three. Per-channel int8 with static shapes works; per-token int4 in a transformer is research-quality. Most production LLM serving still uses FP16 or BF16 weights with int8 activations, and even that requires care.
• Debugging across the model → export → runtime boundary is awful. When a model runs correctly in PyTorch but produces garbage on-device, the failure can be in quantization calibration, in a missing op in the backend, in a layout transform, or in a bug in your export script. None of the three frameworks have great tooling for this.
• The “drop in any PyTorch model” promise is partially false. All three frameworks have lists of ops they support on each backend, and a moderately exotic model will hit a gap. The honest version of the pitch is “drop in any reasonably standard model.”

The bottom line

In 2026, the three frameworks are no longer substitutes for each other — they’re tools for different jobs:

• ExecuTorch is the default for PyTorch-trained models shipping to phones and embedded devices.
• LiteRT is the default for shipping to NPUs, especially on Android, and for microcontroller deployments.
• TVM is the default for unusual hardware targets and for squeezing the last 10–30% out of weird model architectures.

The teams behind all three are aware of each other — Meta contributes to ONNX, Google contributes to LiteRT and uses TVM internally for some products, the TVM community collaborates with Qualcomm and AMD. The frameworks are converging on shared infrastructure (XNNPACK for CPU, common quantization formats) faster than they’re diverging. In two years you may have a single framework for everything; today, you pick based on your hardware first and your model second.