What Is Memory Hierarchy?

October 26, 2023


Computer memory is a vital component that stores and retrieves data and instructions. Therefore, it enables a computer to perform tasks effectively. Establishing a memory hierarchy is essential for the machine to function correctly.

In this text, you will learn what memory hierarchy is.

What Is Memory Hierarchy?

What Is Memory Hierarchy in Computer Systems?

Memory hierarchy organizes memory components based on their access and response times. Moreover, several levels of memory exist, each with different access speeds and performance. This structure ensures that frequently accessed information is obtained faster than less frequently used data.

Why Is Memory Hierarchy Important?

The hierarchy consists of storage devices arranged based on size, cost, access speed, etc. The memory hierarchy simplifies memory management and facilitates data distribution across different types. The purpose is better security, shorter access times, and data availability

Additionally, this hierarchical structure facilitates features like demand paging and pre-paging, all while decreasing the system's per-bit cost.

Memory Hierarchy Types

The memory hierarchy includes different components, generally split into one of the two types:

  • Internal Memory (Primary Memory). Registers, main memory, and cache. Internal memory is directly accessible by the processor.
  • External Memory (Secondary Memory). Secondary storage (HDDsSSDs) and tertiary storage (magnetic disk, magnetic tape, and optical disk. This is peripheral storage accessible by the processor via an I/O Module.

The components' detailed descriptions are in the following headings.


These high-speed memory units are located in the CPU and keep often-used data. Registers have the fastest access time as they are closest to the CPU.

However, registers have the smallest storage capacity and can process limited information at a time.

Cache Memory

The cache is a small, fast memory unit close to the CPU. It allows the processor to quickly access frequently used data by temporarily holding a copy of the information from the main memory.

Additionally, the cache memory improves overall system performance and reduces access times. This is possible via algorithms specifically designed to predict which data is likely to be accessed so it can be preloaded for efficient CPU optimization.

Main Memory

Main memory, or RAM (Random Access Memory), is the primary memory of a computer system. It keeps data and instructions the CPU currently uses. However, while it has a large storage capacity, RAM is slower than registers and cache memory. 

RAM has two types:

  • Static RAM (SRAM). Stores the binary info in flip-flops. As long as there is power, information remains valid. 
  • Dynamic RAM (DRAM). Keeps binary data by utilizing the charge stored on capacitors. DRAM has higher memory cell density per unit of space than SRAM.

Secondary Storage

Secondary storage is a non-volatile memory with greater storage capacity than main memory. It is where the CPU keeps data it doesn't frequently access but still needs to keep for later usage. However, secondary storage has the slowest access time and is generally the most cost-effective option among the memory hierarchy levels.

This storage type is also important for permanent data retention when the computer is powered off. Common examples of secondary storage include hard disk drives (HDDs) and solid-state drives (SSDs).

Tertiary Memory

Tertiary memory refers to storage technologies that offer capacities larger than primary and secondary memory but with significantly slower access times. Tertiary memory examples are:

  • Magnetic disks. Circular plates fabricated with metal, plastic, or magnetized material. These disks offer a balance between access speed and storage capacity.
  • Magnetic tape. Magnetic recording device covered with a plastic film. It is used for data backup due to high storage capacity.
  • Optical disk (CDs, DVDs, Blu-ray Discs). Storage type that uses laser technology to read and write data. The disks offer different capacity options but are becoming obsolete.

Memory Hierarchy Design

Memory hierarchy optimizes data access and storage. Design choices depend on the specific computer architecture, intended use, and the trade-off between speed, capacity, and cost.

Memory hierarchy design creates levels of memory based on different types and their characteristics. The memory hierarchy design looks like this:

  • Level 0: Registers.
  • Level 1: Cache.
  • Level 2: Main memory.
  • Level 3: Secondary memory, magnetic disks, or solid-state memory.
  • Level 4: Tertiary memory.
Memory hierarchy pyramid

Memory Hierarchy Characteristics

The hierarchy is based on characteristics which optimally balance performance, capacity, and cost:

  • Performance. Increases when users need to access lower memory hierarchy levels less frequently. Without the memory hierarchy, a speed gap exists between the main memory and CPU registers. 
  • Capacity. Represents a volume of information the memory is able to store. 
  • Access Time. The interval between the read/write request and the data availability.
  • Cost Per Bit. This metric represents dividing the overall memory cost by the total number of bits accessed. 

Each characteristic either increases or decreases, going from CPU registers (level 0) to the fourth level, as represented in the table below:

CharacteristicsFrom Level 0 to Level 4
Access TimeDecreases
Cost per BitIncreases


After reading this article, you know what memory hierarchy is and the different memory types.

Next, learn about different types of data storage and data storage solutions.

Was this article helpful?
Sara Zivanov
Sara Zivanov is a technical writer at phoenixNAP who is passionate about making high-tech concepts accessible to everyone. Her experience as a content writer and her background in Engineering and Project Management allows her to streamline complex processes and make them user-friendly through her content.
Next you should read
In-Memory Database Explained
November 3, 2022

In-memory databases use RAM and other different volatile memory types to store the database. As a result, they're fast and can be used in real-time data systems. Learn more about in-memory databases in this...
Read more
How to Check Memory Usage in Linux, 5 Simple Commands
March 28, 2024

In this tutorial, learn the five most commonly used commands to check memory usage in Linux. We also provide with...
Read more
Stack vs. Heap: Explaining the Difference
September 21, 2023

This article explains the differences between stack and heap memory and their impact on different memory management techniques and program execution...
Read more
How to Create a Linux Swap File
June 1, 2023

Linux swap files enable managing memory-intensive workloads. They do that by moving inactive files from RAM to a swap file. Learn how swap space works and how to create Linux swap files...
Read more