Skip to content

Conversation

@Russole
Copy link
Contributor

@Russole Russole commented Jan 8, 2026

What changes were proposed in this pull request?

  • Fix inconsistent default handling for OZONE_DATANODE_PIPELINE_LIMIT
  • Use conf.getInt(..., DEFAULT) instead of falling back to 0 when the config is unset
  • Ensure the behavior matches OZONE_DATANODE_PIPELINE_LIMIT_DEFAULT (default = 2)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14369

How was this patch tested?

All CI checks passed.
https://github.com/Russole/ozone/actions/runs/20823196572

Copy link
Contributor

@Gargi-jais11 Gargi-jais11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Russole for working on the patch.
There are two more classes where OZONE_DATANODE_PIPELINE_LIMIT_DEFAULT value should be used instead of 0.
SCMNodeManager : https://github.com/Gargi-jais11/ozone/blob/64bb019407bc001fbe2c6339141908c3e7d59b8f/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java#L198-L199
PipelinePlacementPolicy : https://github.com/Gargi-jais11/ozone/blob/64bb019407bc001fbe2c6339141908c3e7d59b8f/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelinePlacementPolicy.java#L79-L80

Each layer needs the correct default value because they all independently read the configuration and use it in different ways. Fixing only one would leave inconsistencies in the others, causing incorrect behavior.

@Gargi-jais11
Copy link
Contributor

It is also good if you add test cases for this part in TestPipelinePlacementPolicy, TestRatisPipelineProvider and TestSCMNodeManager.
1. RatisPipelineProvider

Test flow:
- Creates config without setting pipeline limit
- Creates pipelines up to expected limit (6 pipelines with 10 nodes)
- Verifies 7th pipeline creation throws SCMException
- Validates error message mentions default limit of 2

Coverage:
- Default value (2) used instead of 0 
- Pipeline limit enforcement works 
- Error messages accurate 

2. PipelinePlacementPolicy

Test flow:
- Creates policy with config without limit set
- Adds exactly 2 pipelines to test node (default limit)
- Verifies node is filtered out when choosing nodes for new pipeline

3. SCMNodeManager

Test flow:
- Creates node manager with config without limit set
- Registers datanode with healthy volumes
- Calls pipelineLimit() and verifies returns 2

@Russole
Copy link
Contributor Author

Russole commented Jan 10, 2026

Thanks @Gargi-jais11 for the reviews. I’ve addressed the comments and updated the patch accordingly.

@Russole Russole requested a review from Gargi-jais11 January 10, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants