The Big Idea
A Graphics Processing Unit (GPU) is designed to perform huge numbers of mathematical operations at once. Most of these operations involve vectors — ordered sets of numbers that describe positions, directions, colors, or velocities.
While a CPU usually works on one number at a time (called a scalar), a GPU works on many numbers at once (a vector). This is why GPUs are so fast at tasks like drawing 3D images, running physics simulations, and training machine learning models.
What Is a Vector?
A vector is a quantity that has both a size and a direction. In computing, a vector is simply a group of related values.
Examples:
- A 2D screen position might be written as (200, 150).
- A 3D direction in space might be (0.3, 0.8, -0.5).
- A color might be represented as (0.7, 0.2, 0.1) for red, green, and blue components.
Each component of the vector is stored in the GPU’s memory as a floating-point number.
Why GPUs Are Built for Vector Math
Every image, model, and lighting calculation can be represented as vectors. The GPU’s architecture is made to perform the same operation on thousands or even millions of vectors at once.
Inside each GPU are:
- Thousands of arithmetic logic units (ALUs) that carry out simple operations such as addition and multiplication on vector components.
- Vector registers that hold sets of related values (for example, the x, y, z, and w coordinates of a vertex).
- Vector instruction units that tell the ALUs to apply the same calculation to all components in parallel.
This design is called SIMD — single instruction, multiple data — and it allows the GPU to perform vector operations on vast amounts of data very quickly.
Common Vector Operations on the GPU
| Operation | Example | Purpose in Graphics |
|---|---|---|
| Vector addition | Add one vector to another, such as (2,3,1) + (1,0,-2) | Moves or combines positions and motion vectors |
| Scalar multiplication | Multiply a vector by a single number, such as 2 × (3,4,1) | Scales an object or changes its speed |
| Dot product | Multiply two vectors and add their products | Used to find the angle between two directions or the brightness of a surface |
| Cross product | Combine two 3D vectors to get a perpendicular one | Used to calculate surface normals for lighting |
| Normalization | Adjust a vector so its length equals 1 | Used to represent direction only, without affecting magnitude |
All these are executed by the GPU’s vector processing hardware in floating-point form.
How GPUs Use Vector Math
- Transformations (Geometry Stage)
Each 3D vertex of a model is transformed by multiplying it with a transformation matrix. This step positions, rotates, and scales the object in the 3D world. Because every vertex transformation is independent, GPUs can process millions of these calculations at the same time. - Lighting and Shading (Pixel Stage)
When light hits a surface, the GPU calculates how bright each pixel should be. It compares the direction of the light with the direction of the surface’s normal vector using a dot product. This determines how much light is reflected toward the viewer — making surfaces appear bright or dim depending on the angle. - Motion and Physics
In simulations, velocity and acceleration are represented as vectors. The GPU continuously updates these vectors to simulate movement, gravity, and collisions. - Image Processing
Operations like blur, sharpening, and edge detection treat each pixel as a small vector of color components (red, green, blue). The GPU performs vector arithmetic on these color vectors across the entire image in parallel.
Why Vector Math Fits the GPU
GPUs excel at parallelism — doing many similar calculations at the same time. Vector math fits this model perfectly because each component of a vector (x, y, z, etc.) can be processed independently by a different ALU.
This is why a GPU is sometimes described as a “vector engine.” Whether it is rendering a 3D world, calculating reflections, or processing an image filter, every task boils down to millions of small vector operations happening simultaneously.
In Summary
Vector math is the foundation of all modern graphics processing. Every line drawn, every object rotated, every shadow cast, and every color blended depends on vector arithmetic.
A GPU is specialized hardware that performs these vector operations thousands of times faster than a CPU could — by handling many components at once using floating-point arithmetic in parallel.