Edge AI Memory Guide: Optimizing Power, Bandwidth, and Latency

One of the most common questions during the schematic phase is: how much memory does edge AI need? The answer is never a fixed number; it depends on model parameters, bit precision, and active workspace buffers.

• Model Parameter Storage: A model with 10 million parameters using 16-bit (FP16) precision will require 20MB just for the static weights.
• Quantization Impact: By utilizing 8-bit integer (INT8) or even 4-bit quantization, engineers can reduce the memory footprint by 50% to 75%. This is essential for fitting complex models into the internal SRAM of a microcontroller.
• Activation Buffers: During inference, the system must store intermediate results (activations) for each layer. For high-resolution computer vision models, these activations can often exceed the size of the model itself.
• Inference Engine Overhead: Modern runtimes like TensorFlow Lite or ONNX require a workspace memory (heap) to manage tensors during execution.

Requirements span orders of magnitude depending on the workload. A quantized classification or detection model can run in tens of megabytes, often within an MCU’s internal SRAM. Multi-stream vision pipelines typically need hundreds of megabytes of external DRAM. Edge LLM and vision-language workloads on NPU- or SoC-class hardware are where 4GB to 16GB becomes realistic. Sizing against the wrong tier is expensive in both directions: underestimate and you hit out-of-memory errors during inference, overestimate and you carry BOM cost and standby power you never use. Underestimating this need can cause severe out-of-memory (OOM) errors during inference, while overestimating raises both the BOM cost and standby power consumption. For detailed guidance on choosing the right components, see our technical blog: Memory IC essentials: selecting the right components for your project.

Choosing between LPDDR4x and LPDDR5 is a key technical decision for high-performance edge devices. While both provide low-power features, they differ markedly in architecture, affecting efficiency and maximum throughput.

LPDDR4x is the dominant memory solution in medical and industrial sectors, providing proven reliability and lower supply risk. It features a reduced I/O voltage to conserve power during fast data transfers.

LPDDR5 reaches 6400 MT/s, 50% higher than the first version of LPDDR4 [2]. Much of its efficiency gain comes from the split clocking scheme: commands run on a quarter-speed master clock while the high-speed WCK data strobes are gated off when idle — which matters directly for edge devices doing periodic inference between long sleep intervals. Features like Data-Copy and Write-X further reduce internal data movement. Jeju Semiconductor (JSC), an authorized partner of Suntsu, focuses on developing these low-power Integrated Circuits and memory modules. Their significance in the market is further detailed in Tiny Chips, Big Impact: The Rise of JSC in the Memory Semiconductor Sector.

Non-volatile storage permanently holds the AI model weights. The process of retrieving these weights into RAM during a “cold start” or model swap can become a major bottleneck.

• eMMC (embedded Multi-Media Controller): eMMC 5.1 uses a parallel, half-duplex interface, allowing only read or write operations at a time. With a maximum speed of about 400 MB/s, it suits smaller models where fast boot times are not essential. When specifying highly reliable eMMC solutions, Suntsu’s Engineering Services team frequently qualifies high-grade NAND flash from authorized partners like ESMT and Jeju Semiconductor (JSC).
• UFS (Universal Flash Storage): UFS employs a serial, full-duplex interface modeled after SCSI architecture, enabling concurrent read and write operations. UFS adopts the SCSI Architecture Model with command queuing, allowing multiple outstanding commands rather than eMMC’s one-at-a-time processing [3b]. Throughput has scaled accordingly: UFS 4.0 delivers up to 5.8 GB/s across two lanes, UFS 4.1 followed in December 2024, and UFS 5.0 published in February 2026 [3a]. Against eMMC 5.1’s ~400 MB/s ceiling, that’s better than an order of magnitude — which is the difference between a model swap the user notices and one they don’t. To meet these high-speed, demanding UFS requirements, Suntsu partners with Flexxon to deliver storage that guarantees industrial-grade longevity and rapid data transfer.

For applications requiring real-time model switching, like a smart camera switching from object detection to facial recognition based on a trigger, UFS is the best option. It minimizes the “blind time” during transitions.

Although NAND flash (eMMC/UFS) is favored for high-density storage, NOR flash continues to be essential for certain Edge AI applications because of its distinct architectural benefits.

The main benefit of NOR flash is its Execute-in-Place (XiP) feature. Unlike NAND flash, which needs code to be copied to RAM before running, NOR enables the processor to run instructions directly from the flash memory. This is essential for ultra-low latency applications where every millisecond of boot-up or response time matters.

In safety-critical industrial applications, NOR flash offers much greater reliability and quicker random read access. Suntsu collaborates with ESMT to deliver industrial-grade memory solutions built to endure tough environments while ensuring data integrity. For a comprehensive comparison of these technologies, see the NOR Flash Guide 2026: Architecture, Reliability, and NAND vs NOR.