Flush backbuffer of lower quality segments for looped video's

[AndreasFurster]

Is your feature request related to a problem? Please describe.

This is a feature request based on issue #4315.

If hls.js switches to higher quality levels, the previous segments are never refetched in better quality, which is crucial if video is on a loop.

For example:

1. Quality level is estimated on 720p
2. Segment 1 is buffered in 720p
3. Quality level is upgraded to 1080p
4. Segment 2 is buffered in 1080p
5. The video loops
6. Segment 1 is played in 720p again while we have the juice to buffer at 1080p level

As mentioned by @cjpillsbury, hls.bufferController.flushBackBuffer() on Hls.Events.LEVEL_SWITCHED could also flush segments of higher quality.

I'm also not sure if buffered segments later on in the video are considered backbuffer. So if we are for example on segment 4. Would it also flush segment 6?

Another thing is that it should wait flushing before the new segment has been buffered. Because when the connection decreases again and the higher quality segment cannot be buffered, we should fall back on the lower quality buffered segment that has already been buffered.

Describe the solution you'd like

An additional Hls option that magickly fixes the mentioned issues 😬

Additional context

No response

[robwalch]

The stream-controller could perform this kind of back-buffer ejection using the fragment tracker to identify buffered content of a lesser quality further back than the currently playing fragment. See nextLevelSwitch and its use of getAppendedFrag(currentTime) in stream-controller; the first half of nextLevelSwitch clears some of the back buffer without removing what is needed to continue smooth playback. Somewhere in checkFragmentChanged might be a good point to detect playback up-switch and an end point for flushing any remaining back buffer of a lower quality.

[iDVB]

Is there a traction on this feature/issue? Any ideas on a rough workaround for the time being?

[robwalch]

Hi @iDVB,

You could use a FRAG_CHANGED event handler to observe changes in playback quality, tracking the time ranges as you go, and some time after a change (preferably not immediately), flush the back-buffer by triggering a BUFFER_FLUSHING event. So something like:

hls.on(Events.FRAG_CHANGED, ({ frag }) => {
   // if the `hls.levels[frag.level]` `bitrate`, `width` or `height` is greater than the last fragment,
   // and `frag.start` is >= the lower quality `previousFrag.end`
   // then wait for currentTime to advance another target or segment duration and flush the video buffer:
      hls.trigger(Events.BUFFER_FLUSHING, {
        startOffset: 0,
        endOffset: previousFragment.end - 0.1,
        'video',
      });
  // keep track of `frag` as `previousFrag` to compare to `hls.levels[frag.level]` on next update
  // or maintain whatever list of time-ranges by quality attribute you like to be handled when it is safe to do so.
});

To avoid interfering with playback wait until currentTime is at least one target duration past the end of the lower quality buffer. If this is specifically for looping, you might wait until playback is close to the end.

Note that this does not solve for looping. There is nothing in HLS.js that will reload the first segment as the playhead approaches video.duration when video.loop is true. Further more there is nothing that optimizes or modifies default buffer length and size settings for looping. These are things that should be considered if HLS.js were to be optimized to support media elements which are set to loop.

[iDVB]

Thanks for those ideas.
Is there anything even simpler we can do that would work for loop?
Like preemptively setting the startLevel or currentLevel for looped videos to the highest level just before it starts playing the first time / everytime ?
This way, even if it loops it will only ever loop back to the best quality.

For us this would be an acceptable trade off. Just not exactly sure what that code looks like or if it would work.

It's odd for us, because the loop video looks GREAT on first load and loop, but then we load a different video and then load back that loop and it's that reloading the loop video that seems to start it at first level.

[bennlich]

It's odd for us, because the loop video looks GREAT on first load and loop, but then we load a different video and then load back that loop and it's that reloading the loop video that seems to start it at first level.

Sounds like this might be a different issue then. My understanding is that this issue is about loading low quality segments when bandwidth is low, and then never replacing them when bandwidth improves.

I think it might be worth opening a new issue where you describe the steps you take when switching videos.

[jasan-s]

This seems like a very sought after feature. My current solution of detecting max level and clearing lower level buffer causes a delay in the loop which is not optimized.