Skip to content

Support update expressions in single request update#6471

Open
anasatirbasa wants to merge 5 commits intoaws:masterfrom
anasatirbasa:feature/support-update-expressions-in-single-request-update
Open

Support update expressions in single request update#6471
anasatirbasa wants to merge 5 commits intoaws:masterfrom
anasatirbasa:feature/support-update-expressions-in-single-request-update

Conversation

@anasatirbasa
Copy link
Contributor

@anasatirbasa anasatirbasa commented Oct 13, 2025

Motivation and Context

#5554

This enhancement adds support for explicit UpdateExpression input in the DynamoDB Enhanced Client while preserving existing behavior by default.

Previously, users could update items using POJO attributes and extension-generated expressions, but could not directly provide request-level update expressions. That made it harder to use advanced DynamoDB update patterns (for example, atomic counters, list operations, and custom expression logic) in the same request flow.

To keep this change safe for existing users, we introduced an opt-in merge strategy flag instead of changing merge behavior globally.

What Changed

1) Enhanced Request Models

  • Added updateExpression() support to:
    • UpdateItemEnhancedRequest
    • TransactUpdateItemEnhancedRequest
  • Added updateExpressionMergeStrategy() to both request builders.

2) Backward Compatibility Preserved by Default

  • Default strategy is UpdateExpressionMergeStrategy.LEGACY.
  • If users do not set the new flag, behavior remains as before:
    • update actions from POJO, extensions, and request are concatenated as-is
    • overlapping paths may still be rejected by DynamoDB (Two document paths overlap)

3) New Opt-In Merge Strategy

  • Added UpdateExpressionMergeStrategy.PRIORITIZE_HIGHER_SOURCE.
  • In this mode, conflicts are resolved per top-level attribute with precedence:
    • request expression > extension expression > POJO actions

Top-level examples:

  • list, list[0], list[1] -> same top-level attribute (list)
  • object.list[0] -> top-level attribute (object)

So list, list[0], and list[1] compete in the same group, while object.list[0] is resolved in a different group.

4) Existing Null-Remove Behavior Kept

  • The pre-existing null-remove suppression behavior is still preserved:
    • if POJO null handling would generate REMOVE attr
    • and the same attribute is explicitly updated by extension/request
    • that POJO REMOVE is suppressed

Testing

Coverage Added/Updated

Tests were updated and expanded to cover:

  1. Legacy default behavior

    • backward-compatible concatenation when strategy is unset
  2. Opt-in strategy behavior

    • PRIORITIZE_HIGHER_SOURCE precedence resolution across sources
  3. Path grouping scenarios

    • overlapping list paths: list, list[0], list[1]
    • nested paths: object.list[0]
  4. Conflict and suppression scenarios

    • request vs extension vs POJO precedence
    • legacy null-remove suppression behavior
  5. Non-overlapping attributes

    • independent top-level attributes continue to merge correctly

Test Results

  • Existing tests continue to pass (backward compatibility preserved)
  • New and updated tests pass for both strategies and all covered scenarios

Test Coverage on modified classes:

image

Test Coverage Checklist

Scenario Done Comments if Not Done
1. Different TableSchema Creation Methods
a. TableSchema.fromBean(Customer.class) [x]
b. TableSchema.fromImmutableClass(Customer.class) for immutable classes [x]
c. TableSchema.documentSchemaBuilder().build() [ ] Not applicable for UpdateExpression feature
d. StaticTableSchema.builder(Customer.class) [x]
2. Nesting of Different TableSchema Types
a. @DynamoDbBean with annotated auto-generated key [x]
b. @DynamoDbImmutable with annotated auto-generated key [x]
c. Auto-generated key combined with partition/sort key [x]
3. CRUD Operations
a. scan() [x]
b. query() [ ] Not directly tested with UpdateExpression
c. updateItem() preserves existing value or generates when absent [x]
d. putItem() with no key set (auto-generation occurs) [x]
e. putItem() with key set manually [x]
f. getItem() retrieves auto-generated key [x]
g. deleteItem() [x]
h. batchGetItem() [x]
i. batchWriteItem() [x]
j. transactGetItems() [x]
k. transactWriteItems() [x]
4. Data Types and Null Handling
a. Annotated attribute is null → key auto-generated [x]
b. Annotated attribute non-null → value preserved [x]
c. Validation rejects non-String annotated attribute [x]
5. AsyncTable and SyncTable
a. DynamoDbAsyncTable Testing [ ] Focus was on sync operations
b. DynamoDbTable Testing [x]
6. New/Modification in Extensions
a. Works with other extensions like VersionedRecordExtension [x]
b. Test with Default Values in Annotations [ ] Not applicable for UpdateExpression
c. Combination of Annotation and Builder passes extension [ ] Not applicable for UpdateExpression
7. New/Modification in Converters
a. Tables with Scenario in Scenario No.1 (All table schemas are Must) [ ] Not applicable for UpdateExpression
b. Test with Default Values in Annotations [ ] Not applicable for UpdateExpression
c. Test All Scenarios from 1 to 5 [ ] Not applicable for UpdateExpression

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@anasatirbasa anasatirbasa changed the title Feature/support update expressions in single request update Support update expressions in single request update Oct 20, 2025
@anasatirbasa anasatirbasa force-pushed the feature/support-update-expressions-in-single-request-update branch 2 times, most recently from bfa9f26 to a7fe576 Compare October 22, 2025 06:42
@anasatirbasa anasatirbasa marked this pull request as ready for review October 27, 2025 15:09
@anasatirbasa anasatirbasa requested a review from a team as a code owner October 27, 2025 15:09
@anasatirbasa anasatirbasa force-pushed the feature/support-update-expressions-in-single-request-update branch 2 times, most recently from f26a4ec to 8791c1f Compare December 15, 2025 10:05
@bhoradc bhoradc mentioned this pull request Jan 6, 2026
12 tasks
@anasatirbasa anasatirbasa force-pushed the feature/support-update-expressions-in-single-request-update branch from ef8f4c0 to a0eeea7 Compare January 11, 2026 14:53
@anasatirbasa anasatirbasa force-pushed the feature/support-update-expressions-in-single-request-update branch from 716b7c8 to dabb912 Compare March 9, 2026 00:39
.ignoreNulls(ignoreNulls)
.ignoreNullsMode(ignoreNullsMode)
.conditionExpression(conditionExpression)
.returnValuesOnConditionCheckFailure(returnValuesOnConditionCheckFailure);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add updateExpression here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should be added to the buillder. I've made the changes.

* request expressions (highest).
*
* <p><b>Steps:</b> Identify attributes used by extensions/requests to prevent REMOVE conflicts →
* create item SET/REMOVE actions → merge extensions (override item) → merge request (override all).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not happening, this concatenates and DynamoDB will throw a conflict error, see updateExpressionInRequest_whenAttributeAlsoInPojo_shouldThrowConflictError test.

.collect(Collectors.toSet());
}

public static UpdateExpression generateItemSetExpression(Map<String, AttributeValue> itemMap,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be package private as it is used only by unit tests?
Same for generateItemRemoveExpression.

Copy link
Contributor Author

@anasatirbasa anasatirbasa Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have converted them into package private methods.

.build();
}

public static UpdateExpression generateItemRemoveExpression(Map<String, AttributeValue> itemMap,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be public ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated it to private.

.collect(Collectors.toSet());
}

public static UpdateExpression generateItemSetExpression(Map<String, AttributeValue> itemMap,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be public ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated it to private.

}

public UpdateExpressionResolver build() {
return new UpdateExpressionResolver(this);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If nonKeyAttributes() is never called on the builder, this.nonKeyAttributes remains null. The resolve() method calls nonKeyAttributes.isEmpty() which will throw NullPointerException.

}

}
} No newline at end of file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have new line at the end

* if there are attributes to be updated (most likely). If both exist, they are merged and the code generates a final
* Expression that represent the result.
* Merges UpdateExpressions from three sources in priority order: POJO attributes (lowest),
* extensions (medium), request (highest). Higher priority sources override conflicting actions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to clarify the javadoc here, If I understand this right,

Given a POJO with: item.setCounter(100L)
And a request UpdateExpression: SET counter = counter + :inc

The resolver produces BOTH actions (concat, not override):
SET #AMZN_MAPPED_counter = :AMZN_MAPPED_counter, counter = counter + :inc

DynamoDB rejects this with: "Two document paths overlap with each other"

The only conflict that IS auto-resolved: if item.setCounter(null), the POJO would generate REMOVE counter, but the resolver suppresses that REMOVE because counter appears in the request expression.

* Tests transactWriteItems() operation Verifies that transactional write operations work correctly.
*/
@Test
public void transactWriteItems_withUpdateExpression() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test does NOT exercise TransactUpdateItemEnhancedRequest.updateExpression(), it only performs transactional delete + put operations. There is no test that verifies the request-level
UpdateExpression flows through the TransactUpdateItemEnhancedRequest → UpdateItemOperation →
UpdateExpressionResolver path (the right-side of the Either).

A test using:

  .addUpdateItem(mappedTable, TransactUpdateItemEnhancedRequest.builder(...)
      .item(keyRecord).updateExpression(expression).build())

is needed to cover this code path.

@anasatirbasa
Copy link
Contributor Author

The available approaches were analyzed and the main goal was to avoid breaking backward compatibility.
So instead of changing behavior globally, an opt-in flag was added: updateExpressionMergeStrategy.

If this is not set, behavior stays LEGACY (same as today).
In LEGACY, pojo, extension, and request update actions are concatenated as-is. There’s no broad conflict resolution for overlapping paths, so combinations like list and list[0] can still be rejected by DynamoDB with "Two document paths overlap".

The one existing exception is null-remove suppression: if a POJO field is null, it would normally produce a REMOVE, but that REMOVE is skipped when the same attribute is explicitly updated by an extension or request expression. This existing protective behavior is unchanged.

If users want conflict handling, they can opt into PRIORITIZE_HIGHER_SOURCE.
When multiple sources touch the same top-level attribute, we choose one winner with this order:
request expression > extension expression > POJO

Examples:

  • list, list[0], and list[1] are all paths under the same top-level attribute: list
  • object.list[0] is under top-level object
    So when conflicts are resolved, the first three compete in the same list bucket, while object.list[0] is handled separately.

Tha Javadoc was also updated to reflect all of this: why the flag exists, default behavior, available values, precedence, and path grouping.

Tests were updated and expanded to cover these scenarios end-to-end, including:

  • legacy default behavior
  • opt-in precedence behavior
  • overlapping paths (list, list[0], list[1])
  • nested paths (object.list[0])
  • null-remove suppression in legacy mode
  • non-overlapping attributes merging as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants