Modern AI Engineering

Lesson 17.6 · 25 min · video lesson

On-Device ML: A TensorFlow Lite Android Walkthrough

How does a model trained in Python end up recognising a hand-drawn digit inside an Android app, with no internet at all?

In short: TensorFlow Lite (now also called LiteRT) is a runtime for running trained models on phones and small devices. We train a model in Python, convert it to a compact .tflite file (usually quantised to 8-bit integers), bundle it in the app's assets, load it with an Interpreter in Kotlin, and call run() on preprocessed input. We also simulate int8 inference in numpy to see why the quantised model is 4× smaller yet gives almost the same answers.

In this lesson

  1. Why TensorFlow Lite exists
  2. The end-to-end workflow
  3. Converting and quantising the model
  4. Setting up the Android project
  5. The Kotlin integration code
  6. Simulating int8 inference in numpy
  7. Hardware delegates and the alternatives
  8. Worked example, step by step
  9. Practice: try it yourself
  10. Common mistakes and when not to use it

Key terms in this lesson

TensorFlow Lite / LiteRT
Google's runtime for running trained models on mobile and embedded devices; renamed LiteRT in 2024.
.tflite file
A FlatBuffer file holding the converted model graph and weights.
Interpreter
The TFLite runtime object that loads a .tflite model and runs inference.
Tensor
A multi-dimensional array of numbers, such as an image of shape [1, 28, 28, 1].
Post-training quantisation
Converting a trained float model to lower-precision integers without retraining.
Representative dataset
A small set of real inputs the converter uses to measure value ranges for quantisation.
Delegate
A plugin that runs part or all of a TFLite model on other hardware such as a GPU or NPU.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 17.5 Cloud vs Edge: Where Should Your Model Run? · 17.7 LLM Routing: Directing Each Query to the Best Model →