Skip to content

Determine-reboot-cause: wait for database before running#340

Merged
qiluo-msft merged 1 commit intosonic-net:masterfrom
jianyuewu:reboot_cause_parse_db_cfg
Feb 2, 2026
Merged

Determine-reboot-cause: wait for database before running#340
qiluo-msft merged 1 commit intosonic-net:masterfrom
jianyuewu:reboot_cause_parse_db_cfg

Conversation

@jianyuewu
Copy link
Contributor

@jianyuewu jianyuewu commented Jan 13, 2026

Issue

When try upgrade internal SW and reboot, in some rare case, it will report error:

2025 Dec 20 13:47:54.048784 sonic_testbed ERR determine-reboot-cause: :- parseDatabaseConfig: Sonic database config file doesn't exist at /var/run/redis/sonic-db/database_config.json
2025 Dec 20 13:47:54.048904 sonic_testbed INFO determine-reboot-cause[16038]: Traceback (most recent call last):
2025 Dec 20 13:47:54.048942 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/bin/determine-reboot-cause", line 298, in <module>
2025 Dec 20 13:47:54.048972 sonic_testbed INFO determine-reboot-cause[16038]:     main()
2025 Dec 20 13:47:54.049002 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/bin/determine-reboot-cause", line 257, in main
2025 Dec 20 13:47:54.049039 sonic_testbed INFO determine-reboot-cause[16038]:     previous_reboot_cause, additional_reboot_info = determine_reboot_cause()
2025 Dec 20 13:47:54.049070 sonic_testbed INFO determine-reboot-cause[16038]:                                                     ^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.049099 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/bin/determine-reboot-cause", line 185, in determine_reboot_cause
2025 Dec 20 13:47:54.049142 sonic_testbed INFO determine-reboot-cause[16038]:     hardware_reboot_cause = find_hardware_reboot_cause()
2025 Dec 20 13:47:54.049172 sonic_testbed INFO determine-reboot-cause[16038]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.049204 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/bin/determine-reboot-cause", line 126, in find_hardware_reboot_cause
2025 Dec 20 13:47:54.049232 sonic_testbed INFO determine-reboot-cause[16038]:     hardware_reboot_cause_major, hardware_reboot_cause_minor = get_reboot_cause_from_platform()
2025 Dec 20 13:47:54.049261 sonic_testbed INFO determine-reboot-cause[16038]:                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.049289 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/bin/determine-reboot-cause", line 114, in get_reboot_cause_from_platform
2025 Dec 20 13:47:54.049330 sonic_testbed INFO determine-reboot-cause[16038]:     platform  = sonic_platform.platform.Platform()
2025 Dec 20 13:47:54.049359 sonic_testbed INFO determine-reboot-cause[16038]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.049387 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/lib/python3.11/dist-packages/sonic_platform/platform.py", line 35, in __init__
2025 Dec 20 13:47:54.052263 sonic_testbed INFO determine-reboot-cause[16038]:     self._chassis = SmartSwitchChassis()
2025 Dec 20 13:47:54.052303 sonic_testbed INFO determine-reboot-cause[16038]:                     ^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.052332 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/lib/python3.11/dist-packages/sonic_platform/chassis.py", line 1264, in __init__
2025 Dec 20 13:47:54.053215 sonic_testbed INFO determine-reboot-cause[16038]:     self.initialize_modules()
2025 Dec 20 13:47:54.053256 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/lib/python3.11/dist-packages/sonic_platform/chassis.py", line 1301, in initialize_modules
2025 Dec 20 13:47:54.053479 sonic_testbed INFO determine-reboot-cause[16038]:     self.initialize_single_module(index=index)
2025 Dec 20 13:47:54.053518 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/lib/python3.11/dist-packages/sonic_platform/chassis.py", line 1293, in initialize_single_module
2025 Dec 20 13:47:54.053748 sonic_testbed INFO determine-reboot-cause[16038]:     module = DpuModule(index)
2025 Dec 20 13:47:54.053788 sonic_testbed INFO determine-reboot-cause[16038]:              ^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.053817 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/local/lib/python3.11/dist-packages/sonic_platform/module.py", line 288, in __init__
2025 Dec 20 13:47:54.054556 sonic_testbed INFO determine-reboot-cause[16038]:     self.chassis_state_db = SonicV2Connector(host="127.0.0.1")
2025 Dec 20 13:47:54.054595 sonic_testbed INFO determine-reboot-cause[16038]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.054624 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 2254, in __init__
2025 Dec 20 13:47:54.057262 sonic_testbed INFO determine-reboot-cause[16038]:     for db_name in self.get_db_list():
2025 Dec 20 13:47:54.057304 sonic_testbed INFO determine-reboot-cause[16038]:                    ^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.057333 sonic_testbed INFO determine-reboot-cause[16038]:   File "/usr/lib/python3/dist-packages/swsscommon/swsscommon.py", line 2191, in get_db_list
2025 Dec 20 13:47:54.057642 sonic_testbed INFO determine-reboot-cause[16038]:     return _swsscommon.SonicV2Connector_Native_get_db_list(self)
2025 Dec 20 13:47:54.057687 sonic_testbed INFO determine-reboot-cause[16038]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025 Dec 20 13:47:54.057718 sonic_testbed INFO determine-reboot-cause[16038]: RuntimeError: Sonic database config file doesn't exist at /var/run/redis/sonic-db/database_config.json

Fix

Add database.service to Requires/After, so determine-reboot-cause runs only after Redis/StateDB is up, avoiding failures when database_config.json is not yet present.

@jianyuewu
determine-reboot-cause: wait for database before running
87aac55 
Add database.service to Requires/After, so determine-reboot-cause runs
only after Redis/StateDB is up, avoiding failures when
database_config.json is not yet present.

Signed-off-by: Jianyue Wu <jianyuew@nvidia.com>
@jianyuewu
Copy link
Contributor Author

jianyuewu commented Jan 13, 2026

@r12f @prgeor Could you help review this PR for fixing start up dependency issue? Thank you~

@jianyuewu jianyuewu mentioned this pull request Jan 13, 2026
Open
@qiluo-msft qiluo-msft merged commit 88df6ff into sonic-net:master Feb 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qiluo-msft qiluo-msft qiluo-msft approved these changes

+3 more reviewers

@dgsudharsan dgsudharsan dgsudharsan approved these changes

@Junchao-Mellanox Junchao-Mellanox Junchao-Mellanox approved these changes

@gpunathilell gpunathilell gpunathilell approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Request for 202511 Branch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@jianyuewu @qiluo-msft @dgsudharsan @Junchao-Mellanox @gpunathilell @volodymyrsamotiy