Google releases 'LiteRT.js,' which makes AI inference in browsers up to 3 times faster than existing runtimes.
On July 9, 2026, Google announced LiteRT.js, a JavaScript runtime for rapidly executing AI models within web browsers.
LiteRT.js, Google's high performance Web AI Inference - Google Developers Blog
Real-time object detection from webcam footage, text conversion from recorded audio, and image resolution enhancement all require 'inference' using AI models. Performing inference directly within the browser has the advantage of eliminating the need to wait for communication and avoiding the sending of input data outside the device.
Google has long offered a browser-based machine learning library called 'TensorFlow.js,' but the previous version of TensorFlow.js used a JavaScript-based processing mechanism. This JavaScript-based mechanism made it difficult to fully utilize the performance of the CPU and GPU, resulting in TensorFlow.js struggling to achieve the same speed as native applications.
LiteRT.js brings the LiteRT execution environment, which has been optimized for smartphones and desktops, to the browser via WebAssembly, making it easier to utilize the processing power of the device. When inference is performed on the CPU, it uses the neural network computation library 'XNNPACK,' which allows the use of multiple CPU cores and SIMD instructions. When using the GPU, it combines the 'WebGPU' standard for controlling the GPU from the browser with Google's acceleration technology 'ML Drift,' and for NPUs, which are processors dedicated to AI processing, it uses the 'Web Neural Network API (WebNN API).'
The AI models used by developers are in the same '.tflite' format as LiteRT. Existing LiteRT models can be imported into the browser, and models created with PyTorch, JAX, and TensorFlow can also be converted and used. PyTorch models can be converted with 'LiteRT Torch,' and 'quantization,' which reduces model size and speeds up processing by representing numerical values within the model with fewer bits, can be adjusted layer by layer using the AI Edge Quantizer.
Furthermore, it's not necessary to completely rebuild existing apps using TensorFlow.js; pre-inference processing such as image formatting and processing of inference results can remain in TensorFlow.js, and only the part that executes the model can be replaced with LiteRT.js. Google explains that, especially for PyTorch models, the complex process of 'converting from PyTorch to ONNX, TensorFlow, and then to TensorFlow.js' is avoided, and direct conversion from PyTorch to LiteRT format is possible.
According to tests conducted by Google using a 2024 M4-based MacBook Pro, LiteRT.js was up to three times faster than other web runtimes for computer vision and speech processing models.
On the other hand, tests comparing LiteRT.js execution methods showed that inference using WebGPU or WebNN was 5 to 60 times faster than standard CPU execution.
The publicly released demo allows users to perform real-time object detection using YOLO, depth estimation that calculates depth from webcam footage and converts it into a 3D space, and image upscaling to 4x resolution using Real-ESRGAN, all within a browser. LiteRT.js is provided as the npm package '@litertjs/core,' and it is possible to load the model and obtain inference results using JavaScript or TypeScript.
Google plans to further enhance its WebNN support and NPU-based inference, as well as expand its optimizations for generative AI running on devices. They also announced 'LiteRT-LM.js,' which allows large-scale language models to run in a browser, and stated that they will continuously improve LiteRT.js's performance, the range of supported models, and developer tools.
Related Posts:
