[build] Add configurable timeout for apt-get build hook to prevent long hangs#26865
[build] Add configurable timeout for apt-get build hook to prevent long hangs#26865yijingyan2 wants to merge 2 commits intosonic-net:masterfrom
Conversation
Signed-off-by: Yijing Yan <yijingyan@microsoft.com>
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Adds a build-time configuration knob to apply APT HTTP(S) Acquire timeouts during SONiC image builds, aiming to reduce long hangs in package operations executed via sonic-build-hooks.
Changes:
- Extend the
apt-getbuild hook to optionally passAcquire::http::Timeout/Acquire::https::Timeoutbased onBUILD_APT_GET_HTTP_TIMEOUT. - Persist the new timeout variable into the buildinfo config generated during the build.
- Introduce a default value for
BUILD_APT_GET_HTTP_TIMEOUTand propagate/export it throughMakefile.work.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/sonic-build-hooks/hooks/apt-get | Adds conditional APT Acquire timeout options to hook-invoked apt/apt-get commands. |
| scripts/generate_buildinfo_config.sh | Writes BUILD_APT_GET_HTTP_TIMEOUT into the generated buildinfo config for hook consumption. |
| rules/config | Defines default BUILD_APT_GET_HTTP_TIMEOUT value. |
| Makefile.work | Passes and exports BUILD_APT_GET_HTTP_TIMEOUT into the buildinfo generation and slave build invocation. |
| PIP_HTTP_TIMEOUT=$(PIP_HTTP_TIMEOUT) \ | ||
| BUILD_APT_GET_HTTP_TIMEOUT=$(BUILD_APT_GET_HTTP_TIMEOUT) \ | ||
| BUILD_PROCESS_TIMEOUT=$(BUILD_PROCESS_TIMEOUT) \ |
There was a problem hiding this comment.
This line’s indentation uses tabs and is noticeably inconsistent with surrounding lines in the SONIC_BUILD_INSTRUCTION block. While it won’t change behavior, it makes the Makefile harder to read and can lead to accidental formatting issues in future edits; please align it with the neighboring lines’ indentation.
| APT_OPTIONS= | ||
| if [[ "${BUILD_APT_GET_HTTP_TIMEOUT}" =~ ^[1-9][0-9]*$ ]]; then | ||
| APT_OPTIONS="-o Acquire::http::Timeout=${BUILD_APT_GET_HTTP_TIMEOUT} -o Acquire::https::Timeout=${BUILD_APT_GET_HTTP_TIMEOUT}" | ||
| fi |
There was a problem hiding this comment.
The PR description/title imply a hard timeout for apt-get to prevent long hangs, but this implementation only sets APT Acquire HTTP/HTTPS timeouts. apt-get can still hang for non-network reasons (e.g., waiting on apt/dpkg locks or postinst scripts). If the goal is a wall-clock limit, consider wrapping the apt invocation with a timeout-style guard (or update the PR description/comments to clarify this is only a network Acquire timeout).
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Yijing Yan <yijingyan@microsoft.com>
db068b7 to
1872596
Compare
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Why I did it
Some apt-get operations can hang for hours and result in build failure due to timeout.
This change adds a configurable timeout for apt-get commands executed through build hooks, so builds fail fast and can be retried again instead of hanging for hours.
Work item tracking
How I did it
Apply timeout wrapper in apt-get hook
How to verify it
tested with debug messages
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)