Skip to content

autotuner: add unit tests for parse_flow_variables and parse_tunable_variables#4148

Open
rohithsiddi wants to merge 1 commit intoThe-OpenROAD-Project:masterfrom
rohithsiddi:autotuner-test-utils
Open

autotuner: add unit tests for parse_flow_variables and parse_tunable_variables#4148
rohithsiddi wants to merge 1 commit intoThe-OpenROAD-Project:masterfrom
rohithsiddi:autotuner-test-utils

Conversation

@rohithsiddi
Copy link
Copy Markdown

@rohithsiddi rohithsiddi commented Apr 15, 2026

Problem

Two helper functions in tools/AutoTuner/src/autotuner/utils.py had explicit TODO: Tests. comments with no accompanying tests:

  • parse_flow_variables (line 186) — parses environment variables from Tcl scripts and a generated vars.tcl file
  • parse_tunable_variables (line 221) — reads variables.yaml and returns the set of variables marked as tunable

Note: parse_flow_variables was the original validation approach in parse_config before being superseded by parse_tunable_variables in 4458cacb. The function remains defined with an explicit TODO and is tested here for completeness.

These functions are on the critical path: parse_tunable_variables is called by parse_config on every tuning run to validate which variables are allowed.

Changes

  • tools/AutoTuner/test/test_utils.py — new test file covering both functions
  • flow/test/test_autotuner.sh — invoke the new tests before the smoke tests

Test coverage

parse_flow_variables

Test What it covers
test_parse_flow_variables_from_scripts_and_vars_tcl Happy path: reads scripts/*.tcl and vars.tcl, extracts env vars via regex, normalises to uppercase, verifies make vars call signature
test_parse_flow_variables_exits_when_make_fails make returns non-zero → prints [ERROR TUN-0018] and exits
test_parse_flow_variables_exits_when_vars_tcl_is_missing make succeeds but vars.tcl is absent → prints [ERROR TUN-0019] and exits

parse_tunable_variables

Test What it covers
test_parse_tunable_variables_returns_only_tunable_keys Returns only keys with tunable=1; excludes tunable=0 and keys with no tunable field

subprocess.run and yaml.safe_load are mocked at the module level so no Makefile or real variables.yaml is required. A real temporary filesystem is used for the Tcl file parsing to exercise the actual glob/regex/file-read path.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces unit tests for the AutoTuner utility and the rule file generation logic. It adds tools/AutoTuner/test/test_utils.py, which is integrated into the test_autotuner.sh script, and a new test suite flow/test/test_genRuleFile.py. Review feedback highlights that test_genRuleFile.py is currently not invoked by any test scripts and should be integrated into the CI pipeline. Additionally, a workaround in the wildcard matching test points to a logic bug in genRuleFile.py that should be addressed directly.

Comment thread flow/test/test_genRuleFile.py Outdated
Comment thread flow/test/test_genRuleFile.py Outdated
@rohithsiddi rohithsiddi marked this pull request as draft April 15, 2026 16:47
…variables

Signed-off-by: rohithsiddi <rohithsiddi7@gmail.com>
@rohithsiddi rohithsiddi force-pushed the autotuner-test-utils branch from 94af63b to 56dff7c Compare April 15, 2026 16:50
@rohithsiddi rohithsiddi marked this pull request as ready for review April 15, 2026 16:54
@maliberty maliberty requested a review from luarss April 15, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant