Handling responses with fetchLater

[DerGernTod]

I'm having a hard time understanding why the current implementation of fetchLater doesn't yield any response. I understand that it doesn't make sense if the context is already gone (similar to sendBeacon only yielding a bool), but what if the context is not gone and the response is actually interesting? Imho this is the most important part of fetch keepAlive: you can use the fetch api the same way you used to and don't have to fall back to sendBeacon if you decide to send during unload, especially if you started sending and during the request, the page unloads. with fetch keepAlive you're safe. Without keepAlive, if you send the request before the unload event, you never have the chance to decide to send with sendBeacon. So for us, keepAlive resolved this issue completely.

However, with fetchLater, the whole issue comes back. Our implementation leverages beacons to retrieve monitoring configuration updates (which of course can be ignored if the page is already gone, but is important if the page is still active, say for SPAs that aren't reloaded for hours or days). Thus we still have to fall back to fetch keepAlive and cannot leverage fetchLater at all for our beacons. We would fire request solely for config updates every now and then, and use fetchLater for beacons, but that would cause additional requests for no gain (in 99% of the cases, the config stays the same).

Can someone explain to me the narrative as to why fetchLater never yields a response?

[fergald]

So for us, keepAlive resolved this issue completely.

we are not proposing to alter fetch's behaviour so if fetch with keepalive solves your problem, you can continue to use it.

I don't understand your use case. Could you give some example code of how you would use fetchLater if it did give a response and where just using fetch would not have worked?

[DerGernTod]

fetch with keepAlive works for us because we can work with its response, but the response is not mandatory for us.

We still have to handle all the page hide/visibility change/unload mechanisms though where we need to decide when to use which mechanism. For firefox for example, which didn't support keepAlive for a long time (and I think it's still experimental), we have to check during page dismissal if we should rather send via sendBeacon or keep sending with fetch because the browser supports keepAlive.

ATM we send beacons right after the load event and 2 seconds after data has been created (to send it in bulk in case there's more). Of course this delay requires us to handle page dismissal.

This proposal would be good for us since it literally tries to solve exactly our use case - send monitoring beacons at arbitrary times, whatever the user agent decides would be best, but definitely before the page is dismissed. If I understood this proposal correctly, this is exactly what we need. We would basically be able to get rid of all of our page dismissal complexity, timeouts and listeners by simply using fetchLater and use its response to update our config whenever necessary.

[Sora2455]

@DerGernTod What about firing a fetch and fetchLater at the same time, but having an AbortSignal passed to the fetchLater that is triggered if the fetch resolves?

[DerGernTod]

That would mean I still need the page dismissal and timeout handling in order to be able to properly abort one or the other, and it would introduce quite some complexity to prevent sending duplicate data. This would essentially negate the use-case for fetchLater entirely. My question was just about why the response is omitted in fetchLater. What's the reasoning, and what's the advantage?

[fergald]

The main reason is simplicity and trying to provide only one way to do something (I realize we are supplying yet another way to do beacons, however all of the older ways are discouraged, there will be only one recommended way). We already have an API for fetching data with a response. Adding another would not be a good idea. We could have added "later" options to fetch but then we would have to make fetch more complex, including the possibility of rejecting the fetch entirely due to quota (which requires immediate feedback, so cannot be part of the Promise that is normally returned).

It seems like you want to do 2 distinct things in a single request

• deliver data
• receive config

and you are taking advantage of the first to get a performance boost on the second. I understand why this is desirable but something seems off sending back config like this. You're often sending config back to clients that are already destroyed and presumably sending back config updates more frequently than desirable.

I wonder if there isn't a better way to do it. E.g. if there was a way to supply a hint on a regular fetch that told the browser that this request is not urgent and should be piggy-backed on another request to the same origin, then you could neatly separate the requests while still getting the performance boost (there would be a little overhead in there being 2 requests but header compression etc should minimize this). You could disentangle the beacon code from the config update code (client and server).

[DerGernTod]

The thing is, in most cases, there won't be ever a configuration change. Usually our users configure their application monitoring once and it runs like this for a long time, but we still want the possibility (especially for simplification on setup processes) to update the configuration in a single page context.

This means that sending a specific config update request is useless in most cases, thus leveraging the response makes most sense because it doesn't require our business code to decide when a config update request has to be sent.

With every monitoring beacon, we send the current config revision (which is part of the injected script) in the payload and only if there was an update, the response contains the updated config.

If the browser would allow some kind of piggy-backing, it would mean we'd have to queue a piggy-backed config request that should be sent with every fetchLater call, and when a fetchLater request is done, the config request has to be enqueued again to be sent with the next fetchLater call (since on an SPA where people spend hours or days, we wouldn't get an updated config otherwise).

I understand that you want to prevent overloading fetch with features, but isn't introducing APIs that do the same but "a little different" the reason why we have sendBeacon, fetch keepAlive and now fetchLater in the first place? Because the previous implementations just don't cut it. I think fetch has a pretty good API that is easily extensible - at least from a user perspective (I never worked on the native sides of that so can't speak for these). Please don't get me wrong, I'm sure a lot of brain power went into this, but I feel like our use case is exactly the one you're trying to solve, but the current implementation of the draft fails to do so, even though it could.

[mingyc]

you can use the fetch api the same way you used to and don't have to fall back to sendBeacon if you decide to send during unload, especially if you started sending and during the request, the page unloads. with fetch keepAlive you're safe. Without keepAlive, if you send the request before the unload event, you never have the chance to decide to send with sendBeacon. So for us, keepAlive resolved this issue completely.

fetchLater() is semantically equivalenet to fetch(..., {keepalive}) in unload and you don't have to "decide" when to use it.

However, with fetchLater, the whole issue comes back.

We still have to handle all the page hide/visibility change/unload mechanisms though where we need to decide when to use which mechanism.

Could you please explain with some concrete example? I'm sorry but I still fail to understand why you can't just fire request like how you are using sendBeacon.

but isn't introducing APIs that do the same but "a little different" the reason why we have sendBeacon, fetch keepAlive and now fetchLater in the first place

sendBeacon is supposed to be replaced by fetch keepalive but the former is still widely used and there is no way for us to deprecate it. However, we are not proposing fetchLater to replace fetch.

We did consider adding Promise initially, but there are some concerns around data race. See #74 (comment) and #70 (comment) for full details.

[DerGernTod]

fetchLater() is semantically equivalenet to fetch(..., {keepalive}) in unload and you don't have to "decide" when to use it.

I have to "decide" when to use it if I actually want to have a response (if available). In my scenario, I would always like to be able to handle a response, but don't want to manage when the request is sent by the browser.

Could you please explain with some concrete example? I'm sorry but I still fail to understand why you can't just fire request like how you are using sendBeacon.

Because I want a response whenever possible, but if I don't get it for every request, or when the page is dismissed, it's not an issue. I can use our current approach, but it's rather complex since it needs to handle page dismissal. Whenever data is generated, we wait for 2 seconds before sending a beacon (to accumulate as much data as possible, in case more data is generated subsequently e.g. via mutation or performance observers). But we need to have page dismissal handling in order to not lose any data that is currently within this 2 second sending delay. IIUC this is exactly why fetchLater exists - so that I don't need to care about handling this stuff. But if fetchLater doesn't yield a response, I can't use it (because of the config update reasons I already explained).

We're using sendBeacon (or fetch keepalive) during page dismissal because it's the most reliable way and we know that we really have to send a beacon at that time, otherwise we'll lose the data. At this point, we don't care about the response anymore since the page is dismissed anyway. But for all other beacons that are sent before dismissal, we do care about the response. Especially for SPAs that are open for a long period of time.

I'll try to showcase a simple scenario:

const observer = new PerformanceObserver(reportData);
observer.observe();
// [...]
let timeoutId = -1;
let curData = [];
let abortController = new AbortController();

function reportData(data) {
  abortController.abort();
  abortController = new AbortController();
  curData = [...curData, ...data];
  clearTimeout(timeoutId);

  timeoutId = setTimeout(() => {
    fetch(reportUrl, {
      body: JSON.stringify(curData),
      keepAlive: true,
      signal: abortController.signal
    })
      .then(response => response.json())
      .then(responseJson => updateConfig(responseJson))
      .then(() => curData = []);
  }, 2000);

  window.addEventListener("pagehide", () => {
    clearTimeout(timeoutId);
    // fire and forget
    fetch(reportUrl, { body: JSON.stringify(curData), keepAlive: true });
    curData = [];
  }, { once: true });

  window.addEventListener("visibilitychange", () => { /* you get the point */ });
}

This is a very simplified scenario but I think it demonstrates why fetchLater would be the perfect use case, but is not feasible because it doesn't yield a response. If fetchLater would yield a response, we could get rid of all the timeout and listener handling and could simply call fetchLater whenever we get data. I hope this makes sense and explains my concerns.

sendBeacon is supposed to be replaced by fetch keepalive but the former is still widely used and there is no way for us to deprecate it. However, we are not proposing fetchLater to replace fetch.

Hmm I think I understand the concerns about the data race and the activated field is at least a resolution for immediate checks. But why not give the user still the possibility to act upon a response? The FetchLaterResult could yield a response Promise a user can await on which would solve both issues you mentioned in the comments and still provide the possibility for those who need it to consume response data and also act upon status codes (e.g. reporting issues in sending the beacons which afaik is currently not possible at all). From a racing perspective, checking the activated field of the FetchLaterResult from within the promise resolution function would always consistently yield true, while consumers can still use the activated field with BFCache restorations.