Google Opens ML Drift for Phone AI and Desktop Previews

Google has opened the code for ML Drift, software that runs AI calculations on graphics chips inside phones and computers. Announced October 8 under Apache 2.0, it powers GPU acceleration in LiteRT, Google’s runtime for putting trained models into applications. Developers can also use the engine separately. Announcement

That gives application teams two different starting points: a runtime that handles models for them, or lower-level code for building their own execution system. Both target a problem familiar to mobile developers: customers use different graphics chips, operating systems and drivers, so software tuned for one device may need different instructions on another. Project

Google describes Windows and Linux language-model support as desktop previews. Mobile and edge devices remain its main production focus. Google says YouTube Shorts uses it to separate subjects from backgrounds in video effects, with up to 40% less time to process frames across Android and iOS. Announcement

The October LiteRT release connects that existing engine to changes developers must account for now. Version 2.3.0 raises Android’s minimum supported software level to API 24, adds Swift interfaces for Apple devices and distributes GPU components separately. These packaging decisions help determine which users an updated application can serve and which libraries must ship with it. Release notes

Which graphics software runs on each device

ML Drift separates a model’s calculations from the instructions each graphics interface understands. Its repository describes a shared representation, GpuModel, and templates that generate device-specific shader code. A shader is a small program executed on a graphics processor. The engine can adjust how arrays of numbers occupy graphics memory without requiring an entirely separate model for every backend. Project

Google’s LiteRT guide lists OpenCL and OpenGL for Android, Metal for macOS, WebGPU through Vulkan for Linux and WebGPU through Direct3D for Windows. This is a map of software routes to graphics hardware. On Windows, the instructions additionally require two DirectX shader-compiler libraries beside the application executable. Installing the AI runtime alone does not supply every platform dependency. GPU guide

The standalone WebGPU example shows another requirement: developers first build Dawn, the software implementing WebGPU, and configure the project to use it. Its documented build and test steps run on Linux or macOS. Windows appears in Google’s broader platform support, but this particular quickstart supplies neither a Windows build sequence nor a Windows test result. WebGPU example

Browser applications have a separate route. Chrome’s built-in AI interfaces let websites ask browser-managed models to summarize, rewrite, translate or answer prompts. Chrome manages those models rather than requiring each website to distribute its own model files. These higher-level interfaces also have their own release and availability rules, separate from compiling ML Drift’s open-source engine. Chrome APIs

Chrome’s current requirements illustrate how deployment can narrow further. Its foundation-model APIs require supported desktop operating systems, sufficient storage and either suitable GPU memory or a qualifying CPU configuration. Chrome for Android and iOS do not yet support those foundation-model APIs. A phone can therefore support a native application using ML Drift without supporting the same feature through Chrome’s built-in language model. Chrome requirements

Chrome provides an availability check that distinguishes a model ready to use from one still downloading, one that needs a download and an unsupported device. Websites also need user interaction before starting some downloads. These states affect what a customer sees before the first answer, even when local generation works well after setup. Chrome requirements

App integration and engine development need different tests

For an Android application written in Kotlin, Google’s GPU guide adds both the core LiteRT package and its GPU package, then selects GPU acceleration when creating a compiled model. C++ applications need the runtime and accelerator components included in their build. The guide also supports passing graphics buffers directly into inference, reducing copies between processor and graphics memory. GPU guide

Developers working directly with the engine start closer to the hardware. The Android OpenCL quickstart builds an addition test with the Android development toolchain, copies the executable onto a device and runs it there. Only the native development kit is needed to build the core library in that example; the full software development kit can be skipped. Android example

The sample checks addition across several ways of storing arrays on a graphics chip. Passing it would establish that a small set of operations works in that environment. A complete application still needs its own model and inputs exercised. SnowRock reviewed these instructions and source documents; it did not build the engine or run device benchmarks. Android example

ML Drift’s language-model sample takes another step: it extracts downloaded model weights into binary tensors and can compress them into four-bit numbers. Its supported presets include Gemma 3, Gemma 4 and Qwen 3 variants. The accompanying text-generation runner prioritizes readable example code over advanced scheduling, according to the repository. Model samples

The profiler can instead allocate synthetic weights quickly, avoiding a large model download while measuring individual operations. That is useful for studying where GPU time goes. It cannot establish whether a real model answers correctly. Its separate settings for model size, context length and numerical precision also give comparisons a concrete configuration to record. Model samples

Prompt processing and answer generation have different costs

Google’s researchers explained the underlying optimization in a May 2025 paper. A language model first processes the prompt, a stage called prefill, then generates new tokens one at a time, called decoding. Prefill can use large amounts of arithmetic in parallel; decoding often waits on moving data. ML Drift chooses different programs for those stages and reduces intermediate memory transfers. Research

The October announcement’s mobile tests process 512 input tokens and generate 128; its desktop tests use 8,192 inputs and 1,024 outputs. Both specify four-bit weights, a half-precision attention cache and synchronization after every generated token. Those settings describe different workloads, so the charts cannot supply one speed estimate for all phones and computers. Announcement

The earlier paper also describes reusing memory for temporary arrays whose lifetimes do not overlap. One illustrated Stable Diffusion configuration reduced activation storage from 4.31 gigabytes to 387 megabytes. Those figures concern intermediate data in that example, rather than every byte needed by every model. Weight storage, temporary arrays and conversation caches each contribute to what fits on a device. Research

Google says the older TensorFlow Lite GPU delegate will stop receiving new features. Its October 8 notice directed Android developers to standalone LiteRT packages, with Google Play services support to follow. Announcement