Each time a brand new Pixel telephone comes out, you would possibly hear that “on-device processing” makes its cool new options potential. Simply check out the new Pixel 9 telephones — issues like Pixel Studio and Name Notes run “on machine.” And it’s not simply telephones: Nest cameras, Pixel smartwatches and Fitbit gadgets additionally use this entire “on-device processing” factor. Given the gadgets that use it and the options it’s powering, it sounds fairly essential.
It’s protected to imagine that the, er, processing, is occurring on the, uh…effectively, the machine. However to get a greater understanding of what which means, we talked to Trystan Upstill, who has been at Google for almost 20 years engaged on engineering groups throughout Android, Google Information and Search.
You had been on a staff that helped develop a number of the thrilling options that shipped with our new Pixel gadgets — are you able to inform me just a little about what you labored on?
Most just lately, I labored inside Android the place I led a staff that focuses on melding Google’s varied expertise stack into an incredible expertise that’s significant to the consumer. Then determining how one can construct it and ship it.
Since we’re enhancing applied sciences and introducing new ones very often, it looks like that may be a unending job.
Precisely! Inside latest years, there’s been this explosion in generative AI capabilities. At first after we began eager about operating giant language fashions on gadgets, we thought it was sort of a joke — like, “Certain we are able to do this, however perhaps by 2026.” However then we started scoping it out, and the expertise efficiency advanced so rapidly that we had been in a position to launch options utilizing Gemini Nano, our on-device mannequin, on Pixel 8 Professional in December 2023.
That’s what I wish to know extra about: “on-device processing.” Let’s break it down and begin with what precisely “processing” means.
The principle processor, or system-on-a-chip (SoC), in your gadgets, has various what are referred to as Processing Models designed particularly to deal with the duties you wish to do with that machine. That is why you will see the chip (just like the Tensor chip present in Pixels) known as a “system-on-a-chip: There’s not only one processor, however a number of processing models, reminiscence, interfaces and way more, all collectively on one piece of silicon.
Let’s use Pixel smartphones for instance: The processing models embrace a Central Processing Unit, or CPU, as the principle “engine” of types; a Graphics Processing Unit, or GPU, which renders visuals; and now right this moment we’ve got a Tensor Processing Unit, or TPU, specifically designed by Google to run AI/ML workloads on a tool. These all work collectively to assist your telephone get issues carried out — aka, processing.
For instance, while you take images, you’re typically utilizing all components of your telephone’s processing energy to good impact. The CPU can be busy operating core duties that management what the telephone is doing, the GPU can be serving to render what the lens is seeing and, on a premium Android machine like a Pixel, there’s additionally quite a lot of work occurring on the TPU to course of what the optical lens sees to make your images look superior.
Received it. “On-device” processing implies there’s off-device. The place is “off-device processing” occurring, precisely?
Off-device processing occurs within the cloud. Your machine connects to the web and sends your request to servers elsewhere, which carry out the duty, after which ship the output again to your telephone. So if we wished to take that course of and make it occur on machine, we’d take the massive machine studying mannequin that powered that activity within the cloud and make it smaller and extra environment friendly so it could actually run in your machine’s working system and {hardware}.
What {hardware} makes that potential?
New, extra highly effective chipsets. For instance, with the Pixel 9 Professional, that’s occurring because of our SoC referred to as Tensor G4. Tensor G4 allows these telephones to run fashions like Gemini Nano — it’s in a position to deal with these high-performance computations.
So principally, Tensor is designed particularly to run Google AI, which is additionally what powers quite a lot of Pixel’s new gen AI capabilities.
Proper! And the generative AI options are undoubtedly a part of it, however there are many different issues on-device processing makes potential, too. Rendering video, enjoying video games, HDR picture enhancing, language translation — most every part you do along with your telephone. These are all occurring in your telephone, not being despatched as much as a server for processing.
