NPU in mobile SoCs: what it is, what it's for, and how it revolutionizes artificial intelligence in your devices.

  • The NPU is a specialized AI processor found in today's SoCs.
  • Allows you to run artificial intelligence tasks efficiently and locally
  • Its integration improves the experience, autonomy and privacy on mobile phones and laptops.
  • Software development is key to taking full advantage of the NPU
Isaac

12 minutes

NPU in mobile SoCs

In recent years, technology Artificial Intelligence It has crept into our daily lives, completely changing the experience of using our mobile devices. It's no longer just about faster phones or better cameras, but about real machines capable of understanding us, anticipating our needs, and offering us features that, not long ago, seemed like science fiction. Much of this is due to a component that, although not always the star of the specifications, is becoming essential: the NPU.

The acronym NPU may sound like something technical or very specialized to you, but you're probably already taking advantage of its benefits every time you unlock your phone with your face, apply a smart filter to a photo, or use voice assistants and real-time translation. Let's unravel in depth what exactly an NPU is, how it works within the SoC processors current, what it is used for and what its advantages and real applications are in modern mobile devices and other smart equipment.

What is an NPU and how did it get into our devices?

The term NPU corresponds to the acronym Neural Processing Unit, which literally translated means Neural Processing Unit. This component is, in essence, a specialized processor that is integrated into the SoC (System on a Chip) of mobile phones, tablets, laptops, and even IoT devices. It is specifically designed to accelerate and improve the performance of all those tasks that depend on neural networks and algorithms Artificial Intelligence y machine learning algorithm .

In a traditional SoC we found mainly two protagonists: the CPU (for general processing) and the GPU (for graphics and parallel computing). However, as AI increasingly made its way into everyday applications, it became clear that a dedicated chip was needed that could handle those workloads much more efficiently. And that's where the NPU comes in.

This evolution has been relatively recent. The first major leap came in 2017, when Huawei introduced the first NPU integrated into its chip. Kirin 970 and Apple did the same with the Neural Engine of the A11 Bionic. Since then, the proliferation of NPUs in SoCs from brands such as Qualcomm, MediaTek, Samsung, Intel and AMD has been unstoppable. This has allowed artificial intelligence to not depend so much on the cloud and to run locally, fast and efficient, on your phone or laptop.

How does an NPU work with respect to the CPU and GPU?

NPU SoC architecture

To understand the importance of NPUs, it's worth reviewing how the different SoC processors work on AI tasks:

  • CPU: It's the "generalist brain." It executes general-purpose tasks, program instructions, and process coordination, but it's not optimized for parallelizing large volumes of data as required by AI.
  • GPU: Specializes in graphics and parallel computing. Its architecture makes it very good for massive mathematical calculations, but its focus is still broader, and it consumes more power than an NPU for AI tasks.
  • NPU: Specifically designed for neural network calculations. Works with matrices, multiplications and convolutions (core operations in AI and machine learning) with much greater efficiency and speed, while consuming less battery.

In practice, an NPU frees it to the CPU and GPU for intensive AI processes. For example, when you use facial recognition on your phone, it's not the CPU that compares the image with stored data, but the NPU, which can do so almost instantly, saving time and energy.

In addition, modern NPUs are optimized for computing tasks. inferenceThat is, once a neural network has been trained (usually in large data centers or specialized servers), the NPU is responsible for applying and using it to execute intelligent functions on the end device.

Why are NPUs necessary in today's SoCs?

The integration of NPUs addresses the massive processing demands of modern real-time AI applications. Think voice assistants, instant translators, cameras that automatically enhance photos, or voice dictations… All of these functions require analyzing, processing, and returning results in a matter of milliseconds.

The NPU allows these tasks to be solved locally on the device itself, without depending on the cloud, which translates into more privacy, lower latency y lower energy consumption. In addition, the workload offloaded to the CPU and GPU frees up resources for other functions, improving the overall user experience and increasing the battery life.

For example, manufacturers such as Qualcomm, MediaTek, and Samsung have developed SoCs with NPUs so powerful that they are capable of running smart assistants, translating languages ​​on the fly offline, or applying complex AI camera effects in real time, without the user noticing delays or excessive power consumption.

Practical applications of the NPU in mobile phones and other devices

If you're wondering when you might be using an NPU without realizing it, here's a detailed look at its real-life, everyday applications:

  • Facial and biometric recognition: Allows you to unlock your mobile phone by quickly and reliably comparing the user's face.
  • Advanced image and video processing: Improve photo quality with Smart HDR, blur backgrounds during video calls, create artistic effects, or stabilize videos with AI.
  • Instant offline translation: Runs neural translation models locally, enabling real-time translations even without internet access.
  • Voice assistants and natural language recognition: Processes spoken commands and conversations, improving the accuracy and speed of assistants such as Siri, Google Assistant or Copilot.
  • Smart Recommendations: Analyzes your usage habits and preferences to recommend content in apps, music, TV shows, etc.
  • Dictation and correction of texts: Allows you to type by voice, corrects grammatical errors, and even suggests stylistic improvements in real time.
  • AI Image Creation: With generative models, you can create images from textual descriptions, apply artistic styles, or modify the appearance of photographs.
  • Augmented Reality: Improves the experience of games and apps that overlay digital information on the real world, identifying objects and faces with greater precision.
  • Improvements in autonomy and energy efficiency: The NPU's targeted approach reduces battery drain compared to if the same task were performed by the CPU or GPU.

All of this, which we see as commonplace in next-generation mobile phones today, would not be possible without the power and specialization of a dedicated NPU in the SoC.

Key differences between NPU, CPU, GPU and other specialized units

One of the common questions is why the CPU or GPU is not enough to handle artificial intelligence applications. The answer lies in the architecture and optimization that each one offers. Let's put it simply:

  • CPU: Flexibility and overall control. Ideal for multitasking and sequential processes, but inefficient for the millions of simultaneous calculations required by neural networks.
  • GPUParallel computing. Very good at heavy math, especially in graphics and simulations, but not as specific or efficient as an NPU for AI.
  • NPU: Ultra-detailed design for AI and machine learning tasks. It operates on specialized matrices and operations, resulting in lower power consumption and faster inference speeds.
  • TPU (Tensor Processing Unit): A variant designed by Google for its data centers and servers, focused on massive AI loads and model training.

It is worth mentioning that in consumer devices, the NPU is primarily dedicated to inference of neural networks, not training, since the latter requires resources and data volumes that are only available on servers and cloud environments.

Advantages and limitations of NPUs: strengths and weaknesses

The main advantages of including an NPU in the SoC of a mobile phone or laptop are:

  • Accelerated performance on AI tasksProcesses such as image recognition, instant translation, or photo enhancement are executed in milliseconds.
  • Reduction of energy consumption: Being optimized for AI, they consume much less power than the CPU or GPU on the same task, which helps the battery last longer.
  • Low latencyBy working locally, AI applications respond in real time, without relying on external networks or servers.
  • Enhanced Privacy: Data such as faces, voices, or habits can be processed within the device itself, without being sent to the cloud.
  • Enable new smart features: They enable developments that were previously unfeasible on portable devices.

On the other hand, there are also Limitations and challenges associated with the use of NPUs:

  • Higher speed can sometimes be a problemIf the data storage system is not fast enough, the NPU can “choke” the flow of information, creating bottlenecks.
  • They only run already trained models: Training large neural networks still requires even more powerful hardware or the cloud.
  • Dependence on software developmentIt's critical that operating systems and apps truly take advantage of the NPU. Without APIs and custom code, the NPU is wasted, and the workload falls to the GPU or CPU.
  • Not all devices integrate it yetAlthough they are becoming more common, older mobile phones and many basic laptops still lack NPUs or have underpowered versions.

Where can we find an NPU and how is it evolving?

Nowadays, the presence of an NPU is no longer exclusive to the most advanced smartphones. Let's look at some examples:

  • Smartphones and tabletsVirtually all high-end and mid-range models released since 2019 include an NPU. Apple has been integrating an NPU into its iPhones and iPads since the A11 Neural Engine, Huawei and its HiSilicon since the Kirin 970, Qualcomm has been integrating it into the latest Snapdragon devices, and MediaTek has been integrating it into the Helio and Dimensity.
  • Laptops and ultrabooksWith the rise of AI in Windows 11 and features like Copilot, recent models with Intel Core Ultra and AMD Ryzen AI processors now come with dedicated NPUs. Qualcomm and other manufacturers have also done the same in their PC SoCs.
  • IoT devices and home automationMany surveillance cameras, smart assistants, connected speakers, and other gadgets integrate miniature NPUs for tasks such as voice recognition or motion detection.
  • Autonomous vehicles and robotics: NPUs allow you to process data from multiple sensors and cameras in real time, crucial for autonomous navigation and decision-making.

The future points to an even greater presence of NPUs in all types of smart devices, from televisions to watches and household appliances.

Featured Implementations: Apple, Google, Intel, AMD, and More

Each manufacturer has developed its own version of the NPU, or neural engine, in recent years. Some of the most well-known and widely used in the industry are:

  • Apple neural engine: Released in 2017 with the iPhone X (A11 Bionic), it has evolved with each new generation and is also present in Macs with Apple Silicon. It's responsible for Face ID, camera improvements, dictation, and all the AI ​​features of the Apple ecosystem.
  • Qualcomm AI Engine: The latest generations of Snapdragon incorporate specialized NPUs, capable of performing tasks such as offline translation, image recognition, and smart assistants without connectivity.
  • MediaTek APU (AI Processing Unit): Present in its Helio and Dimensity SoCs, it is used in both camera tasks and in voice assistants and intelligent data processing.
  • google tensor: The Pixel 6 mobile line and later incorporates an integrated scale “TPU” (Tensor Processing Unit), specialized in computational photography and AI assistants.
  • Intel AI Boost and AMD Ryzen AI: Recent laptop processors with integrated NPU, focused on optimizing AI effects for video calls, energy savings, camera framing, etc.

This competition has driven a revolution in the smart features we can enjoy on consumer devices.

The role of software and NPU integration in apps and systems

Powerful hardware would be of little use if the software couldn't take advantage of it. That's why both operating system and application vendors are working to offer APIs and development tools that allow AI operations to be transparently delegated to the NPU.

For example, Apple makes available to developers the CoreML so their apps can take advantage of the Neural Engine. Google is doing the same with TensorFlow Lite and its integrated TPU in the Pixels. Microsoft has incorporated APIs to take advantage of the NPU in Windows 11, making it easier to use Copilot and other intelligent features.

This allows both third-party apps and native features (camera, gallery, assistants, keyboard, etc.) to take advantage of the NPU, making the user experience smoother and smarter.

How to choose a device with a good NPU and what to look for?

If you're looking for a mobile phone, tablet, or laptop that truly takes advantage of artificial intelligence, there are several aspects to consider when it comes to the NPU:

  • NPU power: It is generally measured in TOPS (Tera Operations Per Second). The higher the number, the better the performance for AI.
  • Recent SoC versionsChoose models with current chips from Apple, Qualcomm, MediaTek, Samsung, Intel, or AMD, as they include more powerful NPUs.
  • Software ecosystem: Make sure the operating system and apps you use actually take advantage of the NPU.
  • Updates and support: Reference models receive more continuous AI improvements that are compatible with the latest developments.

In today's high-end and mid-range mobile phones, it's rare to see a SoC without an NPU. But in laptops and tablets, it's a good idea to read the specifications carefully to make sure they have dedicated AI.

The impact of NPUs on the future of consumer AI

The presence of NPUs will continue to be key as AI evolves, enabling smarter and more efficient functions across all devices. Their ability to process complex tasks locally will enhance privacy, reduce reliance on the cloud, and boost technological innovation in areas such as facial recognition, advanced photography, real-time translation, and more. The integration of NPUs into SoCs has transformed the concept of smart devices, making them faster, more efficient, and more tailored to user needs. Their advancement will continue to pave the way for an increasingly connected and intelligent future.


Add as preferred source