FREE AI INFRASTRUCTURE TOOL
GPU Fit Calculator
Find the right GPU for your AI model
Estimate whether an AI model is likely to fit and run on a selected GPU based on model size, GPU memory, quantisation and workload type.
The Focus Group Technologies GPU Fit Calculator helps IT teams compare AI model requirements against GPU capabilities across Q4, Q8 and FP16 precision. Explore GPU and model compatibility, understand VRAM and unified memory requirements, and identify potential infrastructure options before moving into detailed AI workload testing and benchmarking.
What is the GPU Fit Calculater
The GPU Fit Calculator is a free AI infrastructure sizing tool developed by Focus Group Technologies to help organisations estimate whether a particular AI model is likely to fit and run on a selected GPU or system.
It compares model size and architecture with available GPU memory while accounting for model precision and quantisation, including Q4, Q8 and FP16. You can start with a GPU to explore which AI models may fit, or select an AI model to compare potential GPU options.
The calculator is designed as an early-stage sizing guide rather than a replacement for real-world workload testing and benchmarking. Actual AI performance will also depend on factors including memory bandwidth, compute capability, context length, workload characteristics and the wider infrastructure supporting the deployment.
GPU Sizing TOOL
Compare AI Models and GPU Requirements
Select a GPU to explore which AI models may fit, or choose an AI model to compare potential GPU options. Use Q4, Q8 or FP16 precision to see how model quantisation changes memory requirements.
Q4 is the practical local-inference default – what most people actually run via Ollama/llama.cpp. Q8 and FP16 use more memory but are closer to full precision.
Choose a GPU
See every model it can run, color-coded by fit.
Unified/coherent-unified memory pools give higher capacity than an equivalent dedicated-VRAM card, but not necessarily equivalent speed – capacity and performance are different axes.
Choose a model
See every GPU that can run it, color-coded by fit.
All models
Reference table, not filtered by your selection above.
All GPUs
Reference table, not filtered by your selection above.
GPU & AI MODEL TERMINOLOGY
Understanding GPU and AI Model Requirements
GPU sizing for AI workloads depends on more than GPU model alone. Model architecture, parameter count, memory type and quantisation all affect whether an AI workload can fit and how effectively it can run.
Model Precision and Quantisation:
Q4 / Q8 / FP16: These describe how precisely a model’s weights are stored. Q4 (4-bit) uses the least memory and is fastest to load, Q8 (8-bit) uses more memory but retains more precision, while FP16 (16-bit floating point) generally offers the highest fidelity but requires substantially more VRAM.
Dedicated VRAM vs Unified Memory:
Dedicated VRAM is memory physically attached to the GPU and is typically very fast, but limited to the capacity of the GPU. Unified memory is a shared memory pool available to both CPU and GPU, allowing much larger models to fit, although bandwidth and performance characteristics can differ from dedicated GPU memory.
Dense Model:
A dense model activates essentially all of its model parameters for each inference request. This makes memory requirements relatively predictable: a 20B-parameter model broadly needs enough memory to hold the entire 20B model.
Multimodal Model:
A multimodal model can process more than one type of input, typically text plus images, and sometimes audio or video. These models may require additional memory and compute for components such as vision encoders alongside the main language model.
Mixture of Experts (MoE):
An MoE model contains multiple specialised groups of parameters, or “experts”, but activates only a subset for each token. This can reduce the compute required per token, although the full model may still need substantial memory because all or most experts must remain available.
Encoder–Decoder Model:
These models use one component to encode the input and another to generate the output. They are commonly used for tasks such as translation, summarisation and other sequence-to-sequence workloads.
Diffusion Model:
A diffusion model generates content by starting with noise and progressively refining it over multiple processing steps. It is most commonly associated with image generation, and its GPU requirements are influenced heavily by image resolution, model size and the number of generation steps.
FREQUENTLY ASKED QUESTIONS
GPU Fit Calculator FAQs
Common questions about GPU memory, AI model sizing, quantisation and selecting infrastructure for AI workloads.
How much GPU memory does an AI model need?
The amount of GPU memory an AI model requires depends on its parameter count, model architecture, quantisation level, context length and workload. Lower-precision formats such as Q4 generally require considerably less memory than Q8 or FP16, allowing larger models to run on GPUs with lower memory capacity.
What GPU do I need to run an AI model?
The right GPU depends on the AI model being used and the workload you intend to run. Model size and memory requirements determine whether the model can fit, while GPU architecture, memory bandwidth and compute performance influence how effectively it performs. The GPU Fit Calculator provides an early indication of compatible options before detailed testing.
What is the difference between Q4, Q8 and FP16?
Q4, Q8 and FP16 describe different levels of numerical precision used to store AI model weights. Lower precision reduces memory requirements, while higher precision retains more information but consumes more GPU memory. The appropriate choice depends on the model, workload and required performance or output quality.
Does more VRAM mean a GPU will run an AI model faster?
Not necessarily. VRAM capacity determines whether a model can fit into GPU memory, but performance also depends on factors including memory bandwidth, GPU architecture, compute capability and workload characteristics. Capacity and speed should therefore be assessed separately.
Can AI models run using unified memory instead of dedicated VRAM?
Some systems allow AI workloads to use unified memory shared between the CPU and GPU. This can provide access to larger memory pools and allow larger models to fit, although performance characteristics can differ substantially from systems using dedicated GPU memory.
Can an AI model run across multiple GPUs?
Yes. Some AI workloads can be distributed across multiple GPUs using techniques such as model parallelism. Whether this is practical depends on the model, software framework, GPU interconnect, workload and overall system architecture.
Is the GPU Fit Calculator a performance benchmark?
No. The GPU Fit Calculator is an early-stage sizing guide designed to estimate whether an AI model is likely to fit within available GPU or system memory. It does not replace workload testing, benchmarking or detailed infrastructure design.
Why should AI infrastructure be assessed before selecting GPUs?
GPU selection is only one part of AI infrastructure planning. Storage performance, networking, CPU resources, memory, power, cooling, software architecture and deployment location can all affect the performance, scalability and cost of an AI environment.
AI INFRASTRUCTURE PLANNING
Found a GPU fit? Let’s look at the infrastructure behind it.
GPU compatibility is only one part of an AI deployment. Compute architecture, memory, storage, networking, power and the wider infrastructure environment all influence performance, scalability and cost.
If you are planning an AI workload, Focus Group Technologies can help you evaluate the infrastructure, hardware and deployment options around it.