HIVE-29450: Iceberg: Multi-table, multi-statement txn support

[deniskuzZ]

What changes were proposed in this pull request?

The commit introduces an atomic multi-table commit path for Iceberg tables through HMS. The flow:

1. HiveIcebergOutputCommitter stages Iceberg snapshots via HiveTxnCoordinator instead of committing individually
2. HiveTxnCoordinator.commit() gathers all staged metadata, acquires locks, and performs a single updateTableParams RPC
3. A new thrift endpoint update_table_params atomically updates params for multiple tables with CAS (compare-and-swap) semantics
4. DbTxnManager is extended with a pluggable TxnCoordinator abstraction

Why are the changes needed?

Enables multi-table, multi-statement transactions support

Does this PR introduce any user-facing change?

No

How was this patch tested?

mvn test -Dtest=TestIcebergCliDriver -Dqfile=iceberg_multi_table_txn.q

[Copilot]

Pull request overview

This PR implements atomic multi-table, multi-statement transaction support for Iceberg tables through HMS. The implementation introduces a pluggable transaction coordinator architecture that enables Iceberg to stage metadata changes across multiple tables and commit them atomically via a new HMS batch update endpoint.

Changes:

• Adds update_table_params Thrift RPC for atomic multi-table parameter updates with CAS semantics
• Introduces TxnCoordinator abstraction in DbTxnManager for pluggable transaction coordination
• Implements HiveTxnCoordinator to batch Iceberg snapshot commits across tables in explicit transactions or multi-table inserts
• Refactors SQL direct update logic into reusable base classes (DirectSqlBase, DirectSqlUpdateParams)

Reviewed changes

Copilot reviewed 64 out of 67 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
hive_metastore.thrift	Adds TableParamsUpdate struct and update_table_params RPC
RawStore.java, CachedStore.java, ObjectStore.java	Implements updateTableParams interface
MetaStoreDirectSql.java	Adds updateTableParams with CAS validation and batch SQL updates
DirectSqlBase.java, DirectSqlUpdateParams.java	Refactors parameter update logic into reusable components
DirectSqlUpdatePart.java	Extends DirectSqlBase and delegates to DirectSqlUpdateParams
TxnCoordinator.java	New interface for transaction-level coordination hooks
HiveTxnManager.java, DbTxnManager.java	Adds pluggable TxnCoordinator support
AcidTxnCoordinator.java	Default ACID transaction coordinator implementation
HiveTxnCoordinator.java	Iceberg multi-table atomic commit coordinator
HiveTransaction.java, StagingTableOperations.java	Stages Iceberg metadata for batch HMS updates
HiveIcebergOutputCommitter.java, IcebergAcidUtil.java	Integrates staged transactions
Driver.java, DriverTxnHandler.java, DriverContext.java	Tracks output table count and improves transaction handling
SessionStateUtil.java	Adds output table count tracking
ReExecuteOnWriteConflictPlugin.java	Enhances retry logic for commit failures
Test files	Adds comprehensive concurrent execution tests

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sonarqubecloud]

Quality Gate passed

Issues
46 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.1% Duplication on New Code

See analysis details on SonarQube Cloud