`--lifetime` would be great to have, to limit execution time

[yegor256]

Currently, I use lychee to validate external links in my blog. I have more than 600 Markdown pages and more than a thousand of links. It takes 15+ minutes to check them all. Sometimes it may take even longer. Since the checking process is rather non-deterministic, it won't hurt if it stops after the first five minutes. Some links won't be checked, but some 404 links may be detected.

I suggest adding --lifetime=42 parameter, which means: "stop running after 42 seconds and report what's found, showing how many links were skipped."

[mre]

Hey @yegor256,

why does lychee take so long? Can you share the repo if possible?

You can tweak the delay between requests in case the server can handle more. It depends a lot on your setup and how many different hosts you connect to. (lychee has per-host rate limiting to avoid overloading servers.)

If you set --host-concurrency 20 or even higher if the server can handle more requests. (The default is 10.)

https://lychee.cli.rs/guides/cli/#--host-request-interval

[yegor256]

@mre sure, this is the latest execution: https://github.com/yegor256/blog/actions/runs/22711543878/job/65850585935

[mre]

Thanks, I found that the majority of links complete relatively quickly. There is a long tail of slow hosts, though. The --host-stats flag helps uncover the bad hosts. I've posted the results of my local run in this gist here.

A couple ideas:

• Enable --host-stats to check yourself
• Reduce the number of retries with --max-retries 0. I noticed this is set to 16 in your config, which means you retry every request up to 16 times. (You might have thought it was the total number across all requests.)
• Use a cache so that only new links get checked, and old ones don't for a configurable amount of time. See this page.

[yegor256]

@mre thanks for the advice, I will definitely try to do what you suggested. On top of that, would be great to have the --lifetime feature which would help me protect my build against resource overuse.

[mre]

The value of lychee is giving you a trustworthy picture of your link health. A run that silently skips an unknown number of links undermines that. You'd never know whether the unchecked tail contained broken links, making the output harder to act on in CI.

The better path here is fixing the root cause: --max-retries 0 alone should dramatically reduce your runtime (16 retries per request adds up fast), and enabling --cache means already-checked links are skipped on subsequent runs entirely.

I can understand your use-case, but in my opinion, a --lifetime flag would paper over the issue rather than solve it. Let's try to solve the problem by tweaking the existing flags first.

[cristiklein]

@yegor256 I wonder why you prefer lychee to limit its execution time, instead of relying on the CI to do that. As far as I can tell, both GitHub Actions and GitLab CI have such a feature. Personally, I'd rather trust limiting execution time at the CI than the tool level.

[yegor256]

@cristiklein if GitHub stops my job on timeout, it's a failure. If lychee stops itself, it's a success (with reduced accuracy, but it's acceptable for me). We already accept individual timeouts (per link), why can't we accept a global timeout?

[cristiklein]

@yegor256 I see, so you want something like:

lychee --accept-timeout --global-timeout 42 ...  # exits 0 after 42 seconds
lychee --global-timeout 42 ...  # exits non-zero after 42 seconds

This will be an interesting feature to implement. I have zero experience with cancelling asynchronous calls in Rust.

[yegor256]

@cristiklein with the --global-timeout 42 it should exit zero after 42 seconds if no errors were found during 42 seconds. It should simply ignore all other links, that were not yet checked. With this feature lychee may be used for "smoke" testing, instead of detailed and thorough testing.

[mre]

As a workaround you could try something like that:

timeout 42 lychee **/*.md
EXIT_CODE=$?

if [ $EXIT_CODE -eq 124 ]; then
  echo "lychee timed out; partial results only"
  exit 0
fi

exit $EXIT_CODE

The 124 exit code is documented in
https://cgit.git.savannah.gnu.org/cgit/coreutils.git/tree/src/timeout.c

[yegor256]

@mre looks good, but I can't use this workaround in the Lychee GitHub Action