Deprecation of Lake Formation’s Ruled Tables Function

After cautious consideration, now we have made the choice to finish assist for Ruled Tables, efficient December 31, 2024, to concentrate on open supply transactional desk codecs akin to Apache Iceberg, Apache Hudi, and Linux Basis Delta Lake. This determination stems from buyer choice for these open supply options, which supply ACID-compliant transactions, compaction, time journey, and different options beforehand offered by Ruled Tables. Amazon Net Providers (AWS) has enhanced our assist for these codecs throughout numerous analytics companies, together with Amazon Athena, Amazon EMR (Elastic MapReduce), AWS Glue, and Amazon Redshift, with options that embody automated compaction assist for Apache Iceberg, retention and snapshot expiration and orphan file deletion for Apache Iceberg, enhancements to AWS Glue Knowledge Catalog CreateTable API to create Apache Iceberg tables, and AWS Glue Crawlers schema detection assist throughout Apache Iceberg, Apache Hudi, and Linux Basis Delta Lake. Clients can now use these open supply codecs to realize ACID-compliant transactions with Amazon Easy Storage Service (Amazon S3) backed information, benefiting from their wealthy options and large compatibility.

After December 31, 2024, clients will not be capable of create Ruled Tables transactions (lakeformation:StartTransaction), write to Ruled Tables (lakeformation:UpdateTableObjects), or question your Ruled Tables utilizing Amazon Athena. Clients will nonetheless be capable of entry their desk state info by calling the lakeformation:GetTableObjects and transaction info by calling lakeformation:ListTransactions till February 17, 2025. After February 17, 2025, all Ruled Desk APIs will begin to fail. Ruled Tables metadata will live on inside the AWS Glue Knowledge Catalog, and the Ruled Tables information will stay in your S3 buckets. No different desk varieties shall be affected by this alteration, together with Hive (Apache Parquet, CSV, ORC, and so forth), Iceberg, Hudi, and Delta Lake tables.

Migrating your Ruled Tables

Clients can migrate their tables from Ruled Tables to one of many open supply codecs by copying their ruled desk information on to Apache Iceberg utilizing Amazon Athena. Emigrate information to Iceberg, you should use the Amazon Athena CREATE TABLE AS (CTAS) assertion, as proven within the following code instance.

CREATE TABLE my_iceberg_table WITH ( 
    table_type="ICEBERG",
    is_external = false,
    location = 's3://mybucket/myicebergdata/'
) AS 
SELECT * FROM my_governed_table

You may specify extra table-level properties, that are listed within the Amazon Athena Person Information. In case you specify partitions or buckets as a part of the Apache Iceberg desk definition, then you might run into the 100 partition per bucket limitation. On this case, seek advice from Use CTAS and INSERT INTO to work across the 100 partition restrict.

In case you require any help migrating your tables, or have any questions, attain out to us at governed-tables-support@amazon.com.

In regards to the writer

Mert Hocanin is a Principal Large Knowledge Architect with AWS Lake Formation.