WIP: Add a mechanism to fetch to track progress

[lukewarlow]

This is not ready for review this is WIP.

• At least two implementers are interested (and none opposed):
  • …
  • …
• Tests are written and can be reviewed and commented upon at:
  • …
• Implementation bugs are filed:
  • Chromium: …
  • Gecko: …
  • WebKit: …
  • Deno (not for CORS changes): …
• MDN issue is filed: …
• The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

---

Preview | Diff

[lukewarlow]

Need to define the event somewhere properly. (potentially taking the event definitions from XHR?)

[lukewarlow]

This is taken from XHR and should be removed from XHR in a corresponding PR.

[lukewarlow]

Should fetch really fire sync events? It would match XHR but I wonder if these should be queued instead?

[lukewarlow]

Most of these events aren't used in Fetch so this table should probably be kept in the XHR spec.

[lukewarlow]

If we want to do progress tracking for response we'll need to do it differently to XHR, XHR consumes the body inside of this algorithm but fetch obviously can't.

One idea is to wrap response's body with a wrapper readable stream that allows use to fire progress events when the body actually gets read. This way we don't consume anything to report progress, (we can make it so we only wrapper if there's a responseObserver with a progress event listeners).

[jakearchibald]

Why couldn't the event be fired here https://whatpr.org/fetch/1843.html#http-network-fetch:~:text=Increase%20response%E2%80%99s%20body%20info%E2%80%99s%20encoded%20size%20by%20bytes%E2%80%99s%20length%2E ?

[lukewarlow]

Essentially because there's no access to it from within the JS fetch() definition.

[lukewarlow]

Firefox has the below IDL. I split it so the observers still fire events named progress but it's maybe fine to keep it as a single eventtarget and just change the event names?

[Exposed=Window]
callback interface ObserverCallback {
  undefined handleEvent(FetchObserver observer);
};

enum FetchState {
  // Pending states
  "requesting", "responding",
  // Final states
  "aborted", "errored", "complete"
};

[Exposed=(Window,Worker),
 Pref="dom.fetchObserver.enabled"]
interface FetchObserver : EventTarget {
  readonly attribute FetchState state;

  // Events
  attribute EventHandler onstatechange;
  attribute EventHandler onrequestprogress;
  attribute EventHandler onresponseprogress;
};

[lukewarlow]

Should probably be hasRequestListeners

[lukewarlow]

An alternative design that could be considered is to just make Request and event target and fire progress events at the request object directly. Would require authors to construct a request attach a listener and then pass to fetch but might be okay?

[jakearchibald]

@lukewarlow I wonder if that could get confusing in cases where request/response isn't used in a fetch. Eg, if we made these things transferrable/clonable, would you expect progress events when getting a Response from indexeddb?

[lukewarlow]

Yeah that's a good point. Perhaps it's just okay that they only fire progress within fetch? Or perhaps it would be doable to fire them elsewhere (if it would even make sense)?

My thinking is if the vast majority of cases support progress events then the nicer API is probably a worthy trade off?

Response is actually where things are less clear in general though.

[annevk]

Request and Response are very much designed to be data classes. Note also how fetch() and fetchLater() essentially create a copy as one of their first steps. I don't think they should become more stateful than what is needed for the Body mixin.

[lukewarlow]

Okay in that case I think the addition should be moved to a fetchInit rather than being part of requestInit, easy enough to do. That way it keeps it separate from request entirely.

[mindplay-dk]

Okay in that case I think the addition should be moved to a fetchInit rather than being part of requestInit, easy enough to do. That way it keeps it separate from request entirely.

what does that look like?

a third argument to fetch?

or like interface FetchInit extends RequestInit with extra options?

[lukewarlow]

Yeah the FetchInit dictionary would extend RequestInit

[jakearchibald]

I wonder if this should be a single observer object, making it easier to pass around. This also makes more sense if we want somewhere to put an abort event.

Also, this would be a good opportunity to move to a more modern pattern of having the details on the target object, rather than on the event object.

interface FetchObserver extends EventTarget {
  requestLengthComputable: boolean;
  responseLengthComputable: boolean;

  requestLoaded: number;
  responseLoaded: number;

  requestTotal: number;
  responseTotal: number;

  onrequestprogress: (event: Event) => void;
  onresponseprogress: (event: Event) => void;
}

We could include requeststart and responsestart events which indicate when the LengthComputable properties have settled.

[Terreii]

Awsome!
But should/could this work be synced with the AI/Translation people? They are working on a progress method, too.
It would be too bad if we had two (three if you count XMLHttpRequest) methods for progress observing on the web.

const translator = await Translator.create({
  sourceLanguage: 'es',
  targetLanguage: 'fr',
  monitor(m) {
    m.addEventListener('downloadprogress', (e) => {
      console.log(`Downloaded ${e.loaded * 100}%`);
    });
  },
});

I think @domenic is responsible for it.

[jakearchibald]

@Terreii thanks for raising this. monitor is a good name actually.

[jakearchibald]

webmachinelearning/translation-api#61

[noamr]

Perhas we can have requestFinished and responseFinished promises as well?

[lukewarlow]

Just coming back to this to say it's still on my radar and will try to make progress on this. I think from discussions this overall direction seems okay though?