As more AI systems appear with large pools of shared or unified memory, it is easy to compare them directly with discrete GPUs based on memory capacity alone.
For example, compact AI systems and workstations from NVIDIA, AMD and Apple may provide 128 GB of unified or shared memory. At first glance, this can look significantly more capable than a discrete GPU with 48 GB of dedicated VRAM, but the two should not be compared on memory capacity alone.
That comparison is useful, but incomplete.
The main difference is that these systems have very different memory architectures, performance characteristics and infrastructure designs.
Dedicated GPU memory
A conventional discrete GPU has its own high-speed memory, or VRAM, dedicated to the GPU.
The CPU has separate system memory, and data may need to move between CPU RAM and GPU memory across PCIe or another interconnect.
The main advantage is performance. Modern GPU memory such as GDDR and HBM is designed to provide very high bandwidth to the GPU, which is important for AI workloads that continually move model weights and data through the processor.
The limitation is capacity. If a GPU has 48 GB of VRAM, the model and its working data generally need to fit within that memory unless multiple GPUs, model partitioning or offloading are used.
Unified or shared memory
Systems such as Apple Silicon and NVIDIA DGX Spark use a different approach.
Instead of maintaining completely separate CPU and GPU memory pools, the CPU and GPU can access a common pool of memory.
This has a major advantage for AI: it allows relatively compact systems to run models that may be too large to fit into the VRAM of a single conventional GPU.
A system with 128 GB of unified memory may therefore be able to load a model that simply cannot fit onto a 24 GB or 48 GB GPU.
However, 128 GB of unified memory should not be considered the same thing as a 128 GB graphics card.
Capacity is not performance
One of the most important distinctions is between memory capacity and memory bandwidth.
Memory capacity largely determines what can fit.
Memory bandwidth has a major influence on how quickly the GPU can process that data.
Large language model inference is often memory-bandwidth sensitive because the GPU repeatedly reads large amounts of model data from memory.
This means a unified-memory system may be able to run a larger model, while a discrete GPU with less memory may run a smaller model considerably faster.
A useful way to think about it is:
Capacity determines what you can load. Bandwidth and compute determine how quickly you can run it.
Scalability is also different – important for enterprises
Discrete GPU systems have a significant advantage when it comes to expansion.
A workstation or server may support multiple GPUs, allowing additional compute and memory capacity to be added as requirements grow.
For example, a server with four 48 GB GPUs has 192 GB of aggregate VRAM.
This does not mean every application automatically sees one 192 GB memory pool. Models may need to be split across GPUs, and performance depends on the software and interconnect between the GPUs.
However, it provides a clear scale-up path.
Unified-memory systems are usually much more fixed. Memory is generally selected when the system is purchased and cannot be expanded later. If the workload outgrows the system, the normal options are to replace the platform or scale out to additional systems.
Production infrastructure is about more than GPU memory
There is another important distinction when these systems move from development into production.
Many unified-memory systems are delivered as compact workstations or AI appliances. They may provide excellent AI performance, but they do not necessarily provide the same infrastructure redundancy and serviceability as an enterprise GPU server.
A typical enterprise server may support:
- Redundant hot-swappable power supplies
- Redundant cooling fans
- Multiple network interfaces
- Remote management
- Replaceable drives
- Multiple GPUs
- Expandable storage and networking
- Field-replaceable components
- Rack integration and standard data-centre power
This is not an argument that unified-memory systems use less reliable components.
The distinction is redundancy and availability.
If a single power supply fails in a workstation-class system, the system stops. In a server with redundant power supplies, the workload can continue operating while the failed unit is replaced.
That difference becomes increasingly important as an AI workload moves from experimentation into production.
Cost can be a major advantage
One of the strongest arguments for unified-memory systems is cost.
Large amounts of dedicated GPU memory are expensive. Moving from a 24 GB or 32 GB GPU to 48 GB, 96 GB or beyond quickly moves into professional and data-centre GPU territory, and building a server with several GPUs adds further cost for the GPUs themselves, chassis, power, cooling and supporting infrastructure.
Unified-memory platforms can provide a surprisingly large GPU-accessible memory pool at a much lower entry price.
This makes them particularly attractive where the main requirement is being able to fit a large model into memory, rather than delivering the maximum possible throughput.
The important qualification is that the comparison should not be made on memory capacity alone.
A 128 GB unified-memory system may be considerably cheaper than assembling 128 GB or more of discrete GPU VRAM, but the discrete-GPU platform may also provide:
- Higher memory bandwidth
- Greater inference throughput
- Better support for concurrent users
- Additional GPUs as requirements grow
- Enterprise redundancy and serviceability
- Higher-performance networking and storage options
In other words, unified memory can provide a very good cost per GB of GPU-accessible memory, while discrete GPU systems generally provide a stronger performance, scalability and production infrastructure proposition.
For development, experimentation and running large models locally, that cost difference can make unified-memory systems extremely compelling. For heavily used production workloads, the economics may move back towards discrete GPUs because throughput and concurrency become more important than simply fitting the model into memory.
A simple comparison
| Area | Unified / Shared Memory | Discrete GPU |
| Cost per GB of accessible memory | Usually much lower | Usually much higher |
| Initial system cost | Lower | Higher, particularly multi-GPU |
| Large model capacity | Very good | Limited by GPU VRAM unless multi-GPU |
| Memory bandwidth | Good to very good | Usually higher |
| Raw inference performance | Good | Generally stronger |
| Multiple concurrent users | More limited | Stronger |
| Memory expansion | Usually fixed | Add GPUs |
| Compute expansion | Usually fixed | Add GPUs |
| Multi-GPU capability | Limited or platform-specific | Common in servers |
| Hardware redundancy | Often limited | Strong on enterprise servers |
| Serviceability | Usually appliance/workstation style | Enterprise server options |
| Development and testing | Excellent | Excellent |
| Production scaling | Usually scale-out | Scale-up and scale-out |
So which is better?
Neither architecture is automatically better.
Unified memory can be extremely useful where the priority is fitting a large model into a relatively compact and cost-effective system. It is particularly attractive for development, experimentation and some single-user or low-concurrency workloads.
Discrete GPUs are generally stronger where the requirement is higher throughput, more concurrent users, hardware expansion and production infrastructure with redundancy.
The important point is not to compare systems purely on the number of gigabytes advertised.
A 128 GB unified-memory system and a server with discrete GPU VRAM may both be excellent AI platforms, but they solve slightly different problems.
For AI infrastructure sizing, the more useful questions are:
- How large is the model?
- How much memory does it require?
- What memory bandwidth is available?
- How many users or agents will run concurrently?
- Can the platform be expanded?
- What level of redundancy is required?
- Is this a development system or production infrastructure?
Those questions usually tell you far more than the headline memory figure.
