Problem
When launching a cluster with a docker image (image_id: docker:...), every docker setup command is run with stream_logs=False (sky/provision/docker_utils.py, DockerInitializer._run). That is the right default for probe/identity commands (whoami, inspect, daemon-wait retries) whose outputs are parsed, but it also silences docker pull — the one setup step whose duration scales with image size.
For a multi-GB image (e.g. anyscale/ray-llm, vLLM images, CUDA training images), the provision log shows nothing between:
and
for many minutes. Anyone tailing the launch — sky launch clients, the dashboard request log, or an API server's /api/stream — sees a launch that looks hung, with no way to distinguish "pulling 20 GB of layers" from "wedged". The pull's non-TTY output is bounded, line-oriented layer progress, so streaming it is cheap and readable.
Reproduction
resources:
image_id: docker:vllm/vllm-openai:latest
sky launch on any cloud with --cluster + watch the logs: silence from "Instance is up." until the container starts.
Proposal
Add a stream_logs passthrough to DockerInitializer._run (default False, so all existing call sites keep today's behavior) and enable it only for the two docker pull call sites in initialize(). Output is teed, so it still lands in the per-instance provision log and is still returned to callers for parsing.
Happy to contribute the PR (ready to submit).
Problem
When launching a cluster with a docker image (
image_id: docker:...), every docker setup command is run withstream_logs=False(sky/provision/docker_utils.py,DockerInitializer._run). That is the right default for probe/identity commands (whoami,inspect, daemon-wait retries) whose outputs are parsed, but it also silencesdocker pull— the one setup step whose duration scales with image size.For a multi-GB image (e.g.
anyscale/ray-llm, vLLM images, CUDA training images), the provision log shows nothing between:and
for many minutes. Anyone tailing the launch —
sky launchclients, the dashboard request log, or an API server's/api/stream— sees a launch that looks hung, with no way to distinguish "pulling 20 GB of layers" from "wedged". The pull's non-TTY output is bounded, line-oriented layer progress, so streaming it is cheap and readable.Reproduction
sky launchon any cloud with--cluster+ watch the logs: silence from "Instance is up." until the container starts.Proposal
Add a
stream_logspassthrough toDockerInitializer._run(defaultFalse, so all existing call sites keep today's behavior) and enable it only for the twodocker pullcall sites ininitialize(). Output is teed, so it still lands in the per-instance provision log and is still returned to callers for parsing.Happy to contribute the PR (ready to submit).