Use case causing 503: Service Unavailable.

[yarikoptic]

(dandisets-2) dandi@drogon:/mnt/backup/dandi/tmp$ wget     --recursive     --span-hosts     --domains=webdav.dandiarchive.org,api.dandiarchive.org     --no-parent     --content-disposition     --reject "index.html*"     https://webdav.dandiarchive.org/dandisets/000026/draft/sub-I38/ses-SPIM/micr
--2024-09-20 13:09:31--  https://webdav.dandiarchive.org/dandisets/000026/draft/sub-I38/ses-SPIM/micr
Resolving webdav.dandiarchive.org (webdav.dandiarchive.org)... 18.205.36.100, 54.162.128.250, 52.204.242.176, ...
Connecting to webdav.dandiarchive.org (webdav.dandiarchive.org)|18.205.36.100|:443... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2024-09-20 13:10:01 ERROR 503: Service Unavailable.

000026 is an "exceptional" one as this folder contains thousands of files directly under that folder.

[jwodder]

@yarikoptic Is any action needed for this issue, or is this just informational?

[yarikoptic]

Isn't any 503 a sign of an error in implementation?

FWIW, now with timeouts implemented it is 408 "Request Timeout."

But I do not think that this is desired/correct functioning -- we should aim to provide access to that data, so neither 503 nor 408 is acceptable long term solution. So as to me - it is a real issue.

[jwodder]

Isn't any 503 a sign of an error in implementation?

Wikipedia describes it as:

The server cannot handle the request (because it is overloaded or down for maintenance)

That's more of an external resource issue than an implementation issue. I believe the actual 503 was generated by Heroku due to dandidav exceeding Heroku's timeout.

As for why this is happening, requesting https://webdav.dandiarchive.org/dandisets/000026/draft/sub-I38/ses-SPIM/micr is asking dandidav to display a page with 20136 items, and dandidav currently needs to make a separate request for each of the assets. It all comes back to dandi/dandi-archive#1837.