Iceberg REST Catalog Overview #11 — Managing Tables
Register for 2025 Apache Iceberg Summit
Free Copy of Apache Iceberg: The Definitive Guide
2025 Apache Iceberg Architecture Guide
Ultimate Iceberg Resource Guide
Apache Iceberg’s REST Catalog API provides a robust way to manage tables, ensuring seamless table creation, updates, and deletion within a distributed data lakehouse environment. The /v1/{prefix}/namespaces/{namespace}/tables/{table} endpoint plays a central role in handling table metadata and modifications.
This blog covers:
Loading a Table’s Metadata
Updating Table Metadata and Committing Changes
Dropping a Table
Checking Table Existence
1. Loading a Table’s Metadata (GET /tables/{table})
When a client requests a table from the catalog, the response includes table metadata and configuration details. The configuration may consist of access tokens for further requests and override catalog-wide settings.
Example Request: Retrieve Table Metadata
GET /v1/warehouse/namespaces/sales/tables/orders HTTP/1.1
Host: iceberg.catalog.com
Authorization: Bearer <your-access-token>Example Response: Table Metadata
{
"table-identifier": "sales.orders",
"metadata-location": "s3://data-lake/sales/orders/metadata/v1.metadata.json",
"schemas": [
{
"schema-id": 1,
"fields": [
{ "id": 1, "name": "order_id", "type": "long" },
{ "id": 2, "name": "customer_id", "type": "long" },
{ "id": 3, "name": "order_date", "type": "date" }
]
}
],
"snapshots": [
{
"snapshot-id": "987654321",
"timestamp-ms": 1700000000000,
"manifest-list": "s3://data-lake/sales/orders/metadata/v1.manifests"
}
]
}📌 Key Features:
✔ Retrieves table metadata, including schemas and snapshots
✔ Returns override configurations if any exist
✔ Supports If-None-Match headers to avoid unnecessary metadata retrieval
2. Updating Table Metadata and Committing Changes (POST /tables/{table})
Updating an Iceberg table requires atomic commit operations to ensure consistency. Each commit includes requirements (conditions to validate before applying changes) and updates (metadata modifications).
Example Request: Commit Table Updates
POST /v1/warehouse/namespaces/sales/tables/orders HTTP/1.1
Host: iceberg.catalog.com
Authorization: Bearer <your-access-token>
Content-Type: application/json
{
"requirements": {
"assert-ref-snapshot-id": 987654321
},
"updates": [
{
"update-type": "AddSchemaUpdate",
"schema": {
"schema-id": 2,
"fields": [
{ "id": 1, "name": "order_id", "type": "long" },
{ "id": 2, "name": "customer_id", "type": "long" },
{ "id": 3, "name": "order_date", "type": "date" },
{ "id": 4, "name": "order_status", "type": "string" }
]
}
}
]
}🔹 assert-ref-snapshot-id ensures no concurrent modifications break consistency.
🔹 updates modify the schema by adding a new field (order_status).
3. Dropping a Table (DELETE /tables/{table})
When removing a table, the purge option determines whether to delete only the catalog entry or metadata and data files.
Example Request: Delete a Table (Without Purging Data)
DELETE /v1/warehouse/namespaces/sales/tables/orders HTTP/1.1
Host: iceberg.catalog.com
Authorization: Bearer <your-access-token>Example Request: Delete a Table and Purge Data
DELETE /v1/warehouse/namespaces/sales/tables/orders?purgeRequested=true HTTP/1.1
Host: iceberg.catalog.com
Authorization: Bearer <your-access-token>📌 Key Considerations:
✔ Purging deletes all metadata and data files.
✔ If purgeRequested=false, only the catalog entry is removed, leaving data untouched.
4. Checking Table Existence (HEAD /tables/{table})
To check if a table exists, a HEAD request is sent. The response contains no body, but a 204 No Content status confirms existence.
Example Request: Check Table Existence
HEAD /v1/warehouse/namespaces/sales/tables/orders HTTP/1.1
Host: iceberg.catalog.com
Authorization: Bearer <your-access-token>Possible Responses
Status Code Meaning
204 No ContentTable exists
404 Not FoundTable does not exist
403 ForbiddenInsufficient permissions
Handling Common Errors
Error Code — Cause — Resolution
400 Bad RequestInvalid request format. Ensure the request body is correctly formatted.
401 UnauthorizedMissing or invalid authentication token. Use the correct authentication headers.
403 ForbiddenInsufficient permissions. Check access control policies.
404 Not FoundTable or namespace does not exist. Verify namespace and table names.
409 ConflictCommit failed due to concurrent modification. Retry with the latest snapshot ID.
503 Service UnavailableCatalog service is temporarily down. Retry with exponential backoff.
Best Practices for Managing Iceberg Tables
✅ Use HEAD Requests → Quickly check if a table exists before performing actions.
✅ Handle Concurrent Commits → Use assertions to ensure data consistency.
✅ Purge Data Carefully → Only purge if you want to remove both metadata and data.
✅ Monitor Table Snapshots → Maintain version control over schema changes.
Conclusion
Managing tables in Apache Iceberg’s REST Catalog API provides flexibility while maintaining strong governance and consistency. These API endpoints streamline Lakehouse Table management, whether you are loading metadata, modifying schemas, or dropping tables.