Stack, Heap, and the CPU: Where Your Data Actually Lives

You write new Order() in Java and it works. But have you ever wondered where that object actually goes? Or why a local variable inside a method disappears when the method returns, but an object you return from a method sticks around?

The answer comes down to two regions of memory: the stack and the heap. Every Java program uses both. They serve completely different purposes, have completely different behaviors, and map to different parts of the CPU hardware underneath.

This post explains all of that in plain language.

Start with an analogy

Imagine you are working at a desk.

Your desk surface is where you put the things you are working on right now. You open a file, do some work, close it, and put it away. The desk is small, fast to reach, and automatically clears itself when you finish a task.

The filing cabinet in the corner is where things live long-term. It is large, it holds everything, but it takes more effort to find something. When a file is no longer needed, someone eventually has to come around and clean it out.

Inside your head right now is the fastest memory of all. The number you are holding in your mind while doing mental math. It does not even touch the desk.

Keep this analogy in mind. Everything in this post maps back to it.

Part 1: Why Memory Is Divided at All

The problem it solves

A running program needs two fundamentally different kinds of memory:

1. Memory that lives and dies with a function call. When you call a method, you need somewhere to put its local variables, its parameters, and a note about where to return when the method is done. When the method returns, all of that can be thrown away immediately.

2. Memory that survives across function calls. When you create an object and return it from a method, that object must outlive the method. It needs to live somewhere the caller can reach it.

These two needs are so different that treating them the same way would be wasteful and complicated. The stack handles the first. The heap handles the second.

Without it

If there were only one kind of memory, every piece of data would need to be tracked manually. You would have to say “keep this until I say so” for every variable, even the ones you only need for one line. And you would have to say “throw this away now” for everything when you were done. One mistake and you either run out of memory or access data that no longer exists.

The stack and heap division makes most of this automatic. The stack cleans itself up by design. The heap has the garbage collector.

This is not a JVM invention

The stack and heap have been part of computer architecture since the 1950s. The CPU itself has a dedicated register (the stack pointer) that tracks the stack. When you call a function in any language, at any level, the hardware gets involved. The JVM is just one layer on top of that.

Part 2: The Stack

The problem it solves

Every time a method is called, the program needs to remember several things:

• What are the values of the parameters?
• What are the local variables inside this method?
• When this method finishes, where in the code should we go back to?

The stack is where all of that lives. It is designed to be fast, simple, and self-cleaning.

How it works

The stack is a LIFO structure: Last In, First Out. Think of a stack of plates. You can only add to the top and take from the top.

Each method call adds a stack frame to the top. A frame contains:

• The method’s parameters
• The method’s local variables
• The return address: where to jump back to when the method finishes

When the method returns, its frame is popped off the top. The previous frame is now on top again and execution continues from the return address.

Here is a simple example:

public static void main(String[] args) {
    int result = add(3, 4);          // calls add()
    System.out.println(result);
}

static int add(int a, int b) {
    int sum = a + b;                 // sum lives on the stack
    return sum;                      // frame is popped, sum is gone
}

While add() is running, the stack looks like this:

When add() returns, its frame is removed. main() is now on top, and result gets the value 7.

Production example

Every method you call in Java creates a stack frame. A typical Spring Boot request might stack up dozens of frames: your controller, your service, your repository, Spring’s dispatcher, Tomcat’s thread handler, and more.

// Each of these adds a frame to the stack while it runs
public ResponseEntity<Order> getOrder(Long id) {         // frame 1
    Order order = orderService.findById(id);              // frame 2
    OrderDTO dto = mapper.toDTO(order);                   // frame 3
    return ResponseEntity.ok(dto);
}

All three frames exist simultaneously on the stack while toDTO() is running. When toDTO() returns, its frame is gone. When findById() returns, its frame is gone. The stack shrinks as each method finishes.

Without it: StackOverflowError

The stack has a fixed maximum size (usually 512KB to 1MB per thread by default). If method calls keep stacking without ever returning, the stack runs out of space.

// This never stops calling itself
static int countDown(int n) {
    return countDown(n - 1);   // adds a frame every time, never pops
}

The result:

Exception in thread "main" java.lang.StackOverflowError

This is almost always caused by infinite recursion: a method that calls itself (directly or indirectly) without a base case that stops it.

Trade-offs

The stack is the fastest memory a program uses. Allocating space on the stack is just moving the stack pointer register by a few bytes. It requires no coordination, no searching for free space, and no garbage collection. It is automatic and instant.

The cost is its fixed size. You cannot put large data on the stack. You cannot put data there that needs to outlive its method. And each thread gets its own stack, so a large number of threads means a large amount of stack memory even if the stacks are mostly empty.

Part 3: The Heap

The problem it solves

The stack is great for temporary data. But what about objects? An Order object might be created in one method, returned to the caller, stored in a list, passed to another service, and used for the entire duration of a request. It needs to live somewhere that is not tied to any single method call.

That somewhere is the heap.

How it works

The heap is a large pool of memory, shared by all threads in the JVM. When you write new Order(), the JVM finds a free block of memory on the heap, puts the object there, and gives you a reference (an address) pointing to it.

public Order createOrder(String customerId) {
    Order order = new Order(customerId);   // object goes on the heap
    return order;                          // reference is returned, object stays
}

Here, order is a local variable on the stack. But it holds only an address, a pointer to where the actual Order object lives on the heap. When createOrder() returns, the local variable order disappears from the stack. But the object it was pointing to is still on the heap. The caller now holds that address.

Stack                    Heap
─────────────────        ─────────────────────────────
order = 0x7f3a  ───────► Order { customerId="C001",
                                  total=0,
                                  items=[] }

What lives where

This is one of the most important things to know as a Java developer:

Production example

A request comes in and your service creates several objects:

public InvoiceDTO generateInvoice(Long orderId) {
    Order order = orderRepo.findById(orderId).orElseThrow();  // heap
    List<LineItem> items = order.getItems();                   // heap
    BigDecimal total = calculateTotal(items);                  // heap (BigDecimal is an object)
    return new InvoiceDTO(order, items, total);                // heap
}

All of these objects live on the heap. When the request is done and nothing holds a reference to them anymore, they become eligible for garbage collection. The GC will eventually find them and reclaim the memory.

Without it: OutOfMemoryError

The heap has a maximum size set at startup. When the heap is full and the garbage collector cannot free enough space, you get:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

The most common causes:

• A memory leak: objects are kept alive by references that should have been cleared
• A cache that grows without bound
• Processing a file or dataset that is much larger than expected

You set the heap size with JVM flags:

java -Xms512m -Xmx2g MyApp
# -Xms: starting heap size
# -Xmx: maximum heap size

Trade-offs

The heap is flexible: it grows as needed (up to -Xmx), it holds objects of any size, and objects can outlive any method. But heap allocation is slower than stack allocation. The JVM has to find a free block of the right size, update internal bookkeeping, and eventually the garbage collector has to scan and collect dead objects.

The heap is also shared across threads, which means concurrent access to objects needs careful handling (synchronized, volatile, or thread-safe data structures).

Part 4: How They Map to the CPU

The problem it solves

The stack and heap are not just software concepts. They map directly to hardware. Understanding the hardware explains why the stack is fast, why the heap can be slow, and why memory access patterns matter for performance.

The stack is in the CPU spec

Every modern CPU has a dedicated register called the stack pointer (SP on ARM, RSP on x86-64). It always holds the memory address of the top of the current stack.

When you call a method, the CPU executes a CALL instruction. That instruction does two things automatically:

1. Pushes the return address onto the stack (moves RSP down by 8 bytes and writes the address)
2. Jumps to the method’s code

When the method returns, the CPU executes a RET instruction. That instruction:

1. Reads the return address from the top of the stack
2. Moves RSP back up (pops the frame)
3. Jumps to the return address

This is not managed by the JVM or the OS. It is baked into the CPU’s instruction set. The stack exists because the CPU was designed around it.

The heap is just RAM

The heap has no dedicated CPU register. No CALL instruction. No hardware support. It is just a region of RAM that the JVM carves out and manages in software.

When you write new Order(), the JVM calls its internal allocator, which finds a free block in the heap, marks it as used, and returns the address. This is all software. The CPU just sees memory reads and writes.

Registers: the fastest memory

Below the stack is one more level: CPU registers. These are tiny storage locations built directly into the CPU chip. A modern CPU has 16 to 32 general-purpose registers, each holding 8 bytes.

Registers are so fast that they have no latency in the traditional sense: the CPU accesses them in a single clock cycle, under a nanosecond. When the JIT compiler optimizes your code (see the HotSpot post), it tries to keep your most-used local variables in registers rather than stack memory.

static int add(int a, int b) {
    return a + b;
}

After JIT compilation, a and b might never touch the stack at all. They live in registers for the entire duration of the method call.

The memory hierarchy

Here is the full picture of where data can live, from fastest to slowest:

The stack lives mostly in L1 and L2 cache because you access it constantly and sequentially. The heap lives in RAM, and how quickly you access heap objects depends on whether they happen to be in the CPU cache at that moment.

Trade-offs

The deeper in the hierarchy your data is, the slower it is to access. A register access takes 0.3 nanoseconds. A RAM access takes around 100 nanoseconds. That is 300 times slower.

For a tight loop running a million times, the difference between data in L1 cache and data in RAM can be the difference between 10ms and 3000ms.

Part 5: Why Memory Location Affects Speed

The problem it solves

Two pieces of code that do the same logical work can have very different performance depending on how their data is arranged in memory. This is one of the most underappreciated sources of slowness in Java applications.

Cache lines

The CPU does not fetch one byte from RAM at a time. It fetches a cache line: 64 bytes in one go. When you read one value from memory, the 63 bytes next to it come along for free and sit in L1 cache.

This means accessing memory sequentially (one element after the next, in order) is extremely cache-friendly. The CPU prefetches the next cache line before you need it. It feels like register speed.

Accessing memory randomly (jumping around to different addresses) is cache-unfriendly. Every jump is likely to be a cache miss, requiring a trip all the way to RAM.

Without it: pointer chasing

A linked list is a classic example of cache-unfriendly data access:

// Each node is a separate object on the heap, scattered in memory
LinkedList<Integer> linked = new LinkedList<>();
for (int i = 0; i < 1_000_000; i++) linked.add(i);

long sum = 0;
for (int n : linked) sum += n;   // each step follows a pointer to a random location

Each Node object has a next pointer. Following that pointer jumps to a different memory address, likely not in cache. Each iteration of the loop potentially causes a cache miss, forcing a round trip to RAM.

An ArrayList stores its elements in one contiguous array:

// All elements sit next to each other in one block of memory
ArrayList<Integer> array = new ArrayList<>();
for (int i = 0; i < 1_000_000; i++) array.add(i);

long sum = 0;
for (int n : array) sum += n;    // sequential access, stays in cache

The CPU prefetches the next chunk automatically. The whole array fits in a few thousand cache lines.

Production example

A real benchmark of iterating over one million integers:

// Measure both
long start = System.nanoTime();
long linkedSum = 0;
for (int n : linked) linkedSum += n;
long linkedTime = System.nanoTime() - start;

start = System.nanoTime();
long arraySum = 0;
for (int n : array) arraySum += n;
long arrayTime = System.nanoTime() - start;

System.out.printf("LinkedList: %,d ns%n", linkedTime);
System.out.printf("ArrayList:  %,d ns%n", arrayTime);

Typical results on a modern machine:

LinkedList: 28,450,000 ns
ArrayList:   3,120,000 ns

Same number of elements. Same sum operation. Nine times slower, purely because of where the data lives in memory.

Trade-offs

This does not mean you should always use arrays. LinkedList is better when you insert or delete from the middle frequently. ArrayList is better when you mostly read sequentially. The trade-off is between memory layout (cache performance) and structural operations (insert/delete cost). Most of the time, ArrayList wins for iteration-heavy code.

Part 6: Common Failures and What They Mean

StackOverflowError

What happened: A method called itself (or a chain of methods called each other) without ever stopping. The stack kept growing until it hit its size limit.

// Missing base case: this never stops
static long factorial(int n) {
    return n * factorial(n - 1);   // should stop at n == 0
}

How to fix it:

1. Find the infinite recursion and add a base case
2. Convert deep recursion to iteration
3. If the recursion is legitimately deep (parsing a very deep structure), increase the stack size: java -Xss4m MyApp

OutOfMemoryError: Java heap space

What happened: The heap is full. The GC ran but could not free enough memory to fulfill the allocation request.

Common causes:

• Objects being held in a list or cache that never gets cleared
• Processing a large dataset by loading it all into memory at once
• A bug where references are never released

How to investigate:

# Take a heap dump to see what is using memory
java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/dump.hprof MyApp

Open the dump in Eclipse MAT or IntelliJ’s memory profiler. Look for the largest object clusters.

OutOfMemoryError: GC overhead limit exceeded

What happened: The GC is running almost constantly (spending more than 98% of time collecting) but freeing less than 2% of the heap each time. The JVM gives up.

This usually means there is a genuine memory leak: objects are being created faster than they can be collected, or objects are being kept alive longer than they should be.

How to fix it: Find what is holding references. Common culprits: static collections that grow without bound, event listeners that are never unregistered, thread-local variables that are never cleaned up.

Part 7: What to Do With This

Prefer small, short-lived objects in hot paths. The JVM’s escape analysis (covered in the HotSpot post) can stack-allocate objects that never leave their method. A ValidationContext used only inside a loop iteration does not need to go to the heap at all. Keep objects small and local where possible.

Prefer arrays and ArrayList over LinkedList for iteration. Cache locality matters more than most developers expect. Unless you have a specific reason to use LinkedList (frequent mid-list insertion and deletion), ArrayList is almost always faster to iterate because its elements sit next to each other in memory.

Each thread has its own stack. If your application spawns many threads (or uses a thread pool), each thread gets its own stack. The default stack size is typically 512KB to 1MB. 1,000 threads means up to 1GB of stack memory, even if the stacks are mostly empty. Virtual threads (Java 21+) solve this: they use a tiny initial stack that grows only as needed.

Set your heap size explicitly. Never run a production JVM without setting -Xmx. Without it, the JVM picks a default (usually 25% of available RAM) that may be too small or too large for your use case.

java -Xms512m -Xmx2g MyApp
# Start at 512MB, grow up to 2GB

When you get OutOfMemoryError, take a heap dump immediately. The heap dump shows exactly what is in memory at the moment of failure. Without it, you are guessing. Always run with -XX:+HeapDumpOnOutOfMemoryError in production.

Summary

The stack and heap exist because programs need two different kinds of memory. The stack is for things that live and die with a method call: fast, automatic, self-cleaning, but small. The heap is for things that outlive their method: flexible, large, but needing the GC to clean up.

Underneath both is the CPU. The stack has hardware support: the stack pointer register and the CALL/RET instructions. The heap is just RAM managed in software. Between the CPU and RAM sits the cache hierarchy: registers, L1, L2, L3, then RAM. The closer your data is to the CPU, the faster your program runs.

The practical takeaways:

• Local primitives and references live on the stack
• Objects live on the heap
• Access heap memory sequentially when you can
• Each thread has its own stack; the heap is shared
• StackOverflowError means infinite recursion; OutOfMemoryError means the heap is full or leaking