Ollie
In the previous parts of this tutorial series, we introduced Apache Iceberg at a high level: what it is, what it is for, and where it fits in a modern data system.
This tutorial series is a 101-level introduction, so we are not going to dive deep into Iceberg internals. Instead, we will focus more on how to use Iceberg properly and how to think about it clearly.
Before we go further, it helps to get familiar with a few basic Iceberg concepts and terms. You will see these words often when reading documentation, configuring a catalog, or working with tables from Spark or other engines. Having a clear mental model for them will make the rest of the tutorial much easier to follow.
Naming and Organization
Table
At its core, an Apache Iceberg table is simply a table: a named dataset with rows, columns, and a defined schema.
If you have used tables in systems such as PostgreSQL, BigQuery, or Snowflake, the basic idea is the same. You read from a table, write to it, filter it, join it with other tables, and use it to organize structured data for analytics.
The biggest difference is not in the basic idea of a table itself, but in the kind of environment Iceberg is designed for. Iceberg tables are built for data lake and lakehouse systems, where data may be accessed by multiple engines such as Spark, Trino, or Flink.
For a beginner, the simplest and most useful mental model is this:
An Iceberg table is the table abstraction for the data lake.
That is the level you should start with. Under the hood, there is more going on, but from a user’s point of view, it is still a table.
View
A view can be understood as a named query that behaves like a table.
You can query a view much like a table, but the view itself is defined based on other tables rather than storing its own data. That is a good mental model to start with, and for everyday use it is close enough.
In Iceberg environments, the internal definition of a view can be a little more complicated than in a single-engine system, because the same view may be read by multiple query engines. But you do not need to worry about that yet.
For now, it is enough to think of a view like this:
A view is a reusable named definition of query results that users can work with like a table.
That mental model will serve you well in most discussions.
Namespace
In Apache Iceberg, a namespace is a logical grouping of tables.
If you have used PostgreSQL, it is helpful to think of an Iceberg namespace as being similar to a PostgreSQL schema. Both are used to organize related tables under a common name.
For example, you might have a namespace called sales, with tables such as:
sales.orderssales.customerssales.payments
This makes it easier to keep things organized as the number of tables grows.
The exact features of namespaces can depend on the catalog implementation, but at a basic level, a namespace is simply a way to group and manage tables logically within a larger catalog.
Catalog
An Apache Iceberg catalog is the system that keeps track of tables and lets engines find them by name.
It is the entry point for working with Iceberg tables. Through a catalog, you can do things such as:
- list namespaces
- create tables
- load existing tables
- manage tables as named objects
In many query engines, tables are referred to by fully qualified names such as:
catalog.namespace.table
That means the catalog sits at the top of the naming hierarchy.
This also makes it possible for a single compute engine to work with multiple catalogs at once, as long as those catalogs are configured.
At a basic level, the catalog is the system that lets tables be identified and managed as named objects.
Warehouse
In Apache Iceberg, a warehouse is the top-level storage area where table data and table metadata are kept.
If a namespace is like a logical folder for organizing tables, the warehouse is the broader home that contains all of that data.
In practice, a warehouse usually points to a storage location such as:
- an S3 bucket
- a cloud storage path
- a filesystem directory
As a user, you typically do not think about the warehouse every time you query a table. But it is the storage foundation underneath the catalog, namespaces, and tables.
It defines where the actual contents of the tables live.
A simple way to distinguish the two is this:
The warehouse is about where table data lives. The catalog is about how tables are named, organized, and discovered.
Table Structure and Change Over Time
Table Schema
A table schema defines the structure of a table.
It tells you:
- what columns the table has
- what each column is called
- what type of data each column can hold
For example, a table might have columns such as:
order_idcustomer_idcreated_attotal_amount
and types such as:
- integer
- string
- timestamp
- decimal
If the table is the dataset itself, the schema is the definition of its shape.
This is the same basic idea you may already know from systems such as PostgreSQL, BigQuery, or Snowflake. The schema describes the columns of the table and the kind of values stored in them.
Schema Evolution
Schema evolution means the schema of a table can change over time without recreating the table from scratch or rewriting all of its existing data.
In practice, this can include changes such as:
- adding a new column
- renaming a column
- changing the order of columns
- sometimes removing a column
This matters because real data systems do not stay fixed forever. New requirements appear, old fields become less useful, and tables need to adapt over time.
In Iceberg, schema evolution is built in. That means the table can evolve along with your data model, while still preserving compatibility with the table’s existing data and history.
For a modern data system, that is a very important capability.
Partition
A partition is a way of dividing a table into smaller logical pieces based on the values of one or more columns.
For example, a table might be partitioned by:
- day derived from a timestamp column
countryregion
The purpose of partitioning is to make large tables more efficient to query.
If you only need records from one particular day or one particular country, the engine can avoid reading unrelated parts of the table. That makes queries faster and more efficient.
Just like schema, partitioning does not have to remain fixed forever. As the way a table is used changes over time, its partitioning strategy can also change.
That means an Iceberg table can adapt not only in its columns and types, but also in how its data is organized for efficient access.
Snapshot
A snapshot is a point-in-time version of an Iceberg table.
Whenever data in the table changes, Iceberg creates a new snapshot that represents the table’s state at that moment.
This is a very important concept in Iceberg.
A table is not just one current state. It is a sequence of versions over time. Each write creates a new snapshot, and each snapshot captures what the table looked like at that point.
That idea is what makes features such as time travel possible. Because Iceberg tracks table versions through snapshots, a query engine can read the table as it existed at an earlier point in time.
For now, the important idea is simple:
Each write creates a new snapshot, and each snapshot represents one version of the table.
Summary
In this part, we covered some of the most important concepts and terms you will encounter when working with Apache Iceberg:
- table
- view
- namespace
- catalog
- warehouse
- schema
- schema evolution
- partition
- snapshot
You do not need to memorize every detail right away. What matters most is building a clear mental model.
An Iceberg table is still a table. A namespace groups tables. A catalog keeps track of them. A warehouse is where the table data and metadata live. A schema defines the table’s structure. Partitioning helps large tables perform well. And snapshots let Iceberg track the table as it changes over time.
With those concepts in place, the next parts of the tutorial will be much easier to follow.