Memory Leak

[fschrempf]

I seem to have problems with memory leaking from the plugin. Here is an example of running the "Main Mix FLAC Interactive" channel for some time and memory usage of the server process continuously increasing with each song. At 21:00 I switched from the RP plugin to a normal internet radio station. As you can see the leak stopped.

Running the RP stream continuously will drain all the memory on my server machine and finally causes an OOM situation where the server process gets killed.

Below is the information on the server. I switched to a unreleased version of LMS (HEAD from the public/9.0 branch from a few days ago) to see if this helps but I can try to retest with an official release or other versions if this helps.

The RP plugin is on v3.6.0.

    Version: 9.0.2 - TRUNK @ UNKNOWN
    Plattformarchitektur: x86_64-linux
    Perl-Version: 5.40.0 - x86_64-linux-thread-multi
    Audio::Scan: 1.10
    IO::Socket::SSL: 2.083
    Datenbankversion: DBD::SQLite 1.74 (sqlite 3.47.2)

Thanks for any help!

[michaelherger]

What's your "streaming method" in Settings/Advanced/Networking?

[fschrempf]

It's "normal"

[michaelherger]

Hmm... left it playing all night. And yes, I do indeed see an increase in memory usage. But by far not what you're seeing. I never noticed, probably because I'm sometimes updating daily. And I've never seen complaints before... are people not using the plugin 😁?

That'll likely be a tough one to fix. Hopefully I can reproduce by skipping tracks or something. You wouldn't have Qobuz or TIDAL by chance? It might be helpful to understand whether it has to do with the fact that we're online streaming large (Flac) files, or whether it was specific to RP.

[fschrempf]

Ok, thanks anyway. I've been running the plugin for years on an older setup (older LMS version). I only noticed the problem now after migrating to new hardware and using the latest versions of LMS and the plugin.

By the way I've tested with the latest public/9.1 branch now and the issue is the same.

Unfortunately I currently don't have any streaming service for comparison. But maybe I can test with one of the trial subscriptions. I will also try to test with other RP options (non-FLAC, non-interactive).

Let me know, if I can help in any other way. I'm happy to test something or provide additional information if possible.

[fschrempf]

Some new information: I observed some strange phenomenon. The system was idle for some time (all players off) and all of a sudden I'm seeing that the slimserver process is hogging all available physical memory and the OOM kicks in.

I have no idea what's going on there. Probably that is something that is unrelated to the leak reported above, but obviously that's even more fatal.

On the other hand I remembered that I also occasionally use the AccuRadio plugin which maybe could serve as comparison to see if the issue is related to the RP plugin or not. Here is how the memory usage looks with this plugin:

This looks pretty similar to the RP plugin, which makes me think that the cause for this is either in both plugins or more likely in the core. I even suspect that this might not be an issue at all because the memory gets finally released when the pressure is high enough (not verified this so far) and the real issue is the dramatic memory hogging seen above.

[michaelherger]

What other 3rd party plugins are you using? I could imagine that some of them (eg. Music and Artist Information) could consume some memory, too.

[fschrempf]

Actually I only have RadioParadise, Accuradio and YouTube as third-party plugins. The others are all community plugins that are enabled by default. I now disabled some of the community plugins that I don't need to see if this makes any difference.

[fschrempf]

I think that I found out how to reproduce the excessive and sudden memory drainage:

1. Play RP on a single player
2. Power-cut the player while playback is running
3. In LMS the stream now still seems to be running even if no players are online
4. After the night, turn on the player in the morning
5. LMS starts constantly draining memory (around 400MB per minute) until the OOM kicks in

Could it be that LMS somehow fails to detect that all players are offline and continues to buffer data somehow and when the player comes back online and reconnects it tries to resync the player by sending it all of the buffered data? Just a wild guess without knowing anything about how this works at all.

@michaelherger Maybe you can make something out of this?

[michaelherger]

Ok, that's interesting. What kind of player are you using?

[fschrempf]

It's a RPi with piCorePlayer. And I just noticed that I probably should run some updates for this ;)

piCorePlayer v3.10 | linux 4.4.39-pcpAudioCore+ | piCore v8.0 | Squeezelite v1.8.7-1052

[fschrempf]

The player is now updated to

piCorePlayer v10.0.0 | www v10.005 | linux 6.6.67-pcpCore (32) | piCore v15.0 | Squeezelite v2.0.0-1518-pCP

The problem remains the same.