org.apache.iceberg.catalog

Interface Catalog

All Known Implementing Classes:

BaseMetastoreCatalog, BaseSessionCatalog.AsCatalog, CachingCatalog, DynamoDbCatalog, EcsCatalog, GlueCatalog, HadoopCatalog, HiveCatalog, InMemoryCatalog, JdbcCatalog, NessieCatalog, RESTCatalog, SnowflakeCatalog

public interface Catalog

A Catalog API for table create, drop, and load operations.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Catalog.TableBuilder A builder used to create valid tables or start create/replace transactions.

Method Summary

Modifier and Type	Method and Description
default Catalog.TableBuilder	buildTable(TableIdentifier identifier, Schema schema) /** Instantiate a builder to either create a table or start a create/replace transaction.
default Table	createTable(TableIdentifier identifier, Schema schema) Create an unpartitioned table.
default Table	createTable(TableIdentifier identifier, Schema schema, PartitionSpec spec) Create a table.
default Table	createTable(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> properties) Create a table.
default Table	createTable(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> properties) Create a table.
default boolean	dropTable(TableIdentifier identifier) Drop a table and delete all data and metadata files.
boolean	dropTable(TableIdentifier identifier, boolean purge) Drop a table; optionally delete data and metadata files.
default void	initialize(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> properties) Initialize a catalog given a custom name and a map of catalog properties.
default void	invalidateTable(TableIdentifier identifier) Invalidate cached table metadata from current catalog.
java.util.List<TableIdentifier>	listTables(Namespace namespace) Return all the identifiers under this namespace.
Table	loadTable(TableIdentifier identifier) Load a table.
default java.lang.String	name() Return the name for this catalog.
default Transaction	newCreateTableTransaction(TableIdentifier identifier, Schema schema) Start a transaction to create a table.
default Transaction	newCreateTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec) Start a transaction to create a table.
default Transaction	newCreateTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> properties) Start a transaction to create a table.
default Transaction	newCreateTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> properties) Start a transaction to create a table.
default Transaction	newReplaceTableTransaction(TableIdentifier identifier, Schema schema, boolean orCreate) Start a transaction to replace a table.
default Transaction	newReplaceTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec, boolean orCreate) Start a transaction to replace a table.
default Transaction	newReplaceTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> properties, boolean orCreate) Start a transaction to replace a table.
default Transaction	newReplaceTableTransaction(TableIdentifier identifier, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> properties, boolean orCreate) Start a transaction to replace a table.
default Table	registerTable(TableIdentifier identifier, java.lang.String metadataFileLocation) Register a table with the catalog if it does not exist.
void	renameTable(TableIdentifier from, TableIdentifier to) Rename a table.
default boolean	tableExists(TableIdentifier identifier) Check whether table exists.

Method Detail

name

default java.lang.String name()

Return the name for this catalog.

Returns:

this catalog's name

listTables

java.util.List<TableIdentifier> listTables(Namespace namespace)

Return all the identifiers under this namespace.

Parameters:

namespace - a namespace

Returns:

a list of identifiers for tables

Throws:

NoSuchNamespaceException - if the namespace is not found

createTable

default Table createTable(TableIdentifier identifier,
                          Schema schema,
                          PartitionSpec spec,
                          java.lang.String location,
                          java.util.Map<java.lang.String,java.lang.String> properties)

Create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

properties - a string map of table properties

Returns:

a Table instance

Throws:

AlreadyExistsException - if the table already exists

createTable

default Table createTable(TableIdentifier identifier,
                          Schema schema,
                          PartitionSpec spec,
                          java.util.Map<java.lang.String,java.lang.String> properties)

Create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

properties - a string map of table properties

Returns:

a Table instance

Throws:

AlreadyExistsException - if the table already exists

createTable

default Table createTable(TableIdentifier identifier,
                          Schema schema,
                          PartitionSpec spec)

Create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

Returns:

a Table instance

Throws:

AlreadyExistsException - if the table already exists

createTable

default Table createTable(TableIdentifier identifier,
                          Schema schema)

Create an unpartitioned table.

Parameters:

identifier - a table identifier

schema - a schema

Returns:

a Table instance

Throws:

AlreadyExistsException - if the table already exists

newCreateTableTransaction

default Transaction newCreateTableTransaction(TableIdentifier identifier,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.lang.String location,
                                              java.util.Map<java.lang.String,java.lang.String> properties)

Start a transaction to create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

properties - a string map of table properties

Returns:

a Transaction to create the table

Throws:

AlreadyExistsException - if the table already exists

newCreateTableTransaction

default Transaction newCreateTableTransaction(TableIdentifier identifier,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.util.Map<java.lang.String,java.lang.String> properties)

Start a transaction to create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

properties - a string map of table properties

Returns:

a Transaction to create the table

Throws:

AlreadyExistsException - if the table already exists

newCreateTableTransaction

default Transaction newCreateTableTransaction(TableIdentifier identifier,
                                              Schema schema,
                                              PartitionSpec spec)

Start a transaction to create a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

Returns:

a Transaction to create the table

Throws:

AlreadyExistsException - if the table already exists

newCreateTableTransaction

default Transaction newCreateTableTransaction(TableIdentifier identifier,
                                              Schema schema)

Start a transaction to create a table.

Parameters:

identifier - a table identifier

schema - a schema

Returns:

a Transaction to create the table

Throws:

AlreadyExistsException - if the table already exists

newReplaceTableTransaction

default Transaction newReplaceTableTransaction(TableIdentifier identifier,
                                               Schema schema,
                                               PartitionSpec spec,
                                               java.lang.String location,
                                               java.util.Map<java.lang.String,java.lang.String> properties,
                                               boolean orCreate)

Start a transaction to replace a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

properties - a string map of table properties

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

Throws:

NoSuchTableException - if the table doesn't exist and orCreate is false

newReplaceTableTransaction

default Transaction newReplaceTableTransaction(TableIdentifier identifier,
                                               Schema schema,
                                               PartitionSpec spec,
                                               java.util.Map<java.lang.String,java.lang.String> properties,
                                               boolean orCreate)

Start a transaction to replace a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

properties - a string map of table properties

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

Throws:

NoSuchTableException - if the table doesn't exist and orCreate is false

newReplaceTableTransaction

default Transaction newReplaceTableTransaction(TableIdentifier identifier,
                                               Schema schema,
                                               PartitionSpec spec,
                                               boolean orCreate)

Start a transaction to replace a table.

Parameters:

identifier - a table identifier

schema - a schema

spec - a partition spec

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

Throws:

NoSuchTableException - if the table doesn't exist and orCreate is false

newReplaceTableTransaction

default Transaction newReplaceTableTransaction(TableIdentifier identifier,
                                               Schema schema,
                                               boolean orCreate)

Start a transaction to replace a table.

Parameters:

identifier - a table identifier

schema - a schema

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

Throws:

NoSuchTableException - if the table doesn't exist and orCreate is false

tableExists

default boolean tableExists(TableIdentifier identifier)

Check whether table exists.

Parameters:

identifier - a table identifier

Returns:

true if the table exists, false otherwise

dropTable

default boolean dropTable(TableIdentifier identifier)

Drop a table and delete all data and metadata files.

Parameters:

identifier - a table identifier

Returns:

true if the table was dropped, false if the table did not exist

dropTable

boolean dropTable(TableIdentifier identifier,
                  boolean purge)

Drop a table; optionally delete data and metadata files.

If purge is set to true the implementation should delete all data and metadata files.

Parameters:

identifier - a table identifier

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if the table did not exist

renameTable

void renameTable(TableIdentifier from,
                 TableIdentifier to)

Rename a table.

Parameters:

from - identifier of the table to rename

to - new table name

Throws:

NoSuchTableException - if the from table does not exist

AlreadyExistsException - if the to table already exists

loadTable

Table loadTable(TableIdentifier identifier)

Load a table.

Parameters:

identifier - a table identifier

Returns:

instance of Table implementation referred by tableIdentifier

Throws:

NoSuchTableException - if the table does not exist

invalidateTable

default void invalidateTable(TableIdentifier identifier)

Invalidate cached table metadata from current catalog.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.

Parameters:

identifier - a table identifier

registerTable

default Table registerTable(TableIdentifier identifier,
                            java.lang.String metadataFileLocation)

Register a table with the catalog if it does not exist.

Parameters:

identifier - a table identifier

metadataFileLocation - the location of a metadata file

Returns:

a Table instance

Throws:

AlreadyExistsException - if the table already exists in the catalog.

buildTable

default Catalog.TableBuilder buildTable(TableIdentifier identifier,
                                        Schema schema)

/** Instantiate a builder to either create a table or start a create/replace transaction.

Parameters:

identifier - a table identifier

schema - a schema

Returns:

the builder to create a table or start a create/replace transaction

initialize

default void initialize(java.lang.String name,
                        java.util.Map<java.lang.String,java.lang.String> properties)

Initialize a catalog given a custom name and a map of catalog properties.

A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

Parameters:

name - a custom name for the catalog

properties - catalog properties