Monitored Folders

[jacobdhendrix]

It would be nice to have an option to monitor a folder and automatically update the bucket when prerolls are added or removed.

[chadwpalm]

Monitoring folders/file in Node JS isn't a difficult task. As it currently stands, and I say currently because I'm always open to changes that will make the user experience easier, buckets are not tied to directories, so while renaming or deleting a file could result in updating a bucket to rename or remove a file, where would added files go?

In other words, if you added a file to the file system, which bucket would it go in? This would require buckets to be tied to directories (and/or its subdirectories).

I'm going to mention @imsorrywhat here because they had a similar request to monitor files for when their filenames get changed. I think this can be an interesting discussion.

One solution could be to simply tie a bucket to a directory. Instead of picking files to be in the bucket from the UI you would just pick what directory that bucket is connected to, and then when the backend creates the string for Plex it just looks in that directory and picks the random file from there instead of from the list of files in the config file. Then any change to the directory (adding, removing, renaming) would trigger the string to be regenerated so if something was renamed or removed it wouldn't be in the string anymore.

Pros:

1. You can add, delete, rename files all you want and Preroll Plus will keep up to date with it because it is monitoring directories on the file system, not lists of file in the buckets in the config file.
2. The config file would be smaller because each bucket would just need a directory name, not the list of files in that bucket/directory.

Cons:

1. If you want the same file(s) in more than one bucket you would have to have that file in multiple directories unless you created hard/soft links yourself.
2. It might affect file weights as it currently stands. If buckets are just directories it would be more difficult to add more weight to a particular file (unless that's not a general concern). You could add symbolic links to a file multiple times without copying it, but now you are getting into ideas a general user might not understand.

I'm open to discussion and perhaps alternatives I haven't considered.

[jacobdhendrix]

My thought was the monitoring would be optional, so it would continue to work as-is for the "Files in bucket". The optional monitoring could be "Folders in bucket" that monitors for new files which are then given a weight of 1. There could also be a general file check that removes missing files from buckets.

[imsorrywhat]

Maybe I'm not understanding this correctly so let me know if that's the case. When editing existing buckets, they all show the same file list on the right side. So if a new file is added to the directory, wouldn't it just show up on the right side window? Then one would just edit the desired bucket and add the new file.

I will say that I like not having to manage directories on the host side. I'm deploying with docker so I would have to edit the compose and restart the container anytime I want to add a new directory.

[chadwpalm]

I've made additional updates that I pushed to develop as version 0.1.3.

As of right now, Preroll Plus is monitoring your preroll folders, so if you rename, delete, or move a file it should automatically scan your Buckets and update (or delete) that file in any Bucket it is found in. It will also auto-update the Plex string as well, so that there aren't any prerolls in the string that aren't in the file system. Same if you create/update an existing Bucket in the UI. It will also update the Plex string and if you delete a Bucket it should also delete that Bucket from any Sequences that Bucket was in.

Adding a file currently does not do anything, but I will still be looking into creating an option in the buckets to either add/remove files from Buckets like you can now, or the option to tie a Bucket to a directory as requested above. This should satisfy both worlds. It might take a little time to do that since it will involve updating several pieces of code. Maybe sometime next week.

I'm also looking into having the frontend update in real-time when files are changed in the file system. Right now, if you make any changes to the file system you will need to refresh your browser to pull those changes in.

Chad

[jacobdhendrix]

How often is the scan performed? I just removed a file and refreshed the browser but it's still in the bucket. In case this is a bug, my files are stored in this way: /Prerolls/Holiday/HolidayPreroll.mp4, /Prerolls/Normal/NormalPreroll.mp4, etc. Let me know if you'd like me to open an issue.

[chadwpalm]

I created an issue here with my reply: #4

[chadwpalm]

Adding a file currently does not do anything, but I will still be looking into creating an option in the buckets to either add/remove files from Buckets like you can now, or the option to tie a Bucket to a directory as requested above. This should satisfy both worlds. It might take a little time to do that since it will involve updating several pieces of code. Maybe sometime next week.

I'm also looking into having the frontend update in real-time when files are changed in the file system. Right now, if you make any changes to the file system you will need to refresh your browser to pull those changes in.

I've finally completed both of these features and they are ready to test.

1. I've added the ability to tie a bucket to a directory. That way you can manage your files at the filesystem level. This is optional, so you can still populate the buckets the old fashion way as well. You will see a new option in the bucket creation/editor called "Source of file". Manual is the original way, and Directory is the new way.
2. I've also completed the feature where both the bucket list and file listing update in the GUI in real-time. You shouldn't have to refresh the page when you add/move/rename/delete a file or directory. Also, unlike before where adding files didn't do anything, I was able to fix that and now adding of files is monitored too.

The new tag is chadwpalm/prerollplus:feature-bucket-dir-0.1.5.24

[chadwpalm]

Folder monitoring is mostly working. Sometimes it grabs @eadir instead of an actual file which causes Plex to not play a preroll. I grabbed a screenshot from Plex showing what it has in the preroll field.

I'll need to do what I did for the file listings on the front end where I filter out the Synology folders/files from being selected.

The bucket listing seems to be working on the back end, but I need to refresh the browser to see that a file has been removed from a bucket. Without refreshing it looks like it's still in the bucket.

I believe this is related to the last thing you wrote about the lost connection to the backend. I'll address that below.

Some unexpected behavior is if I move a file to another folder the path is updated instead of removing it from the bucket. This may be intentional but as the end user I would have expected it to be removed (maybe I only want this preroll to show up during a certain time but it's still in the default bucket after moving it, for example). Here's the log if this is a bug:

[2024-10-11 10:49:33] [INFO] File /prerolls/Christmas/Plex Pre-Roll - Christmas Edition.mp4 was moved to /prerolls/Temp/Plex Pre-Roll - Christmas Edition.mp4 [2024-10-11 10:49:33] [INFO] Handling rename from /prerolls/Christmas/Plex Pre-Roll - Christmas Edition.mp4 to /prerolls/Temp/Plex Pre-Roll - Christmas Edition.mp4 in buckets [2024-10-11 10:49:33] [INFO] Updated settings for renamed file: /prerolls/Christmas/Plex Pre-Roll - Christmas Edition.mp4 to /prerolls/Temp/Plex Pre-Roll - Christmas Edition.mp4 in bucket "Christmas" [2024-10-11 10:49:33] [INFO] Settings file saved [2024-10-11 10:49:33] [INFO] Sending update

Please help me if I was not understanding your original request. Your request in the first message of this thread was, "It would be nice to have an option to monitor a folder and automatically update the bucket when prerolls are added or removed."

This is what I added in this version. The ability to select the directory you want the bucket to monitor and then you are free to add, rename, delete files in that folder and it will reflect that in the bucket. This only happens when you select Directory as "Source of files" when creating/editing the bucket.

The original request by another user to monitor the files in the backend was so that if files got renamed, deleted, or moved, it would automatically update the settings file to reflect those changes. This includes the moving of a file. While you are suggesting a scenario where moving a file from a folder should delete the file from the bucket, I can counter that with scenarios where someone might have put the file in the wrong folder or are reorganizing their folders and they don't lose that file's connection with the bucket it is in.

The idea here is that buckets and file folders are not synonymous. If you want buckets and folders to be one and the same, then you can tie the bucket to the folder with the new option I just added.

You can make as many buckets as you want and put whatever files you want into any bucket....even have the same file in two buckets. This should open the door to ANY scenario you can think of. Pair this with the scheduling of the sequences you can still do what you described in your example. You shouldn't need to be creating Temp folders to accomplish your goals. If there are some files in a bucket you don't want during a certain time, then create two buckets, one with those files and one without, then choose which bucket you want to use during the sequence that is schedule for that time, and use the other bucket in a sequence outside of that scheduled time.

If I'm not making any sense, please outline in detail the scenario you are looking for or trying to accomplish, and maybe I can help guide you on how to accomplish that.

And finally, I'm not sure if this is causing any issues, but I've got an error message on the buckets screen: Lost connection to backend, trying to reconnect...

That message happens when the frontend browser can't connect with the backend server for keeping the file list and bucket list up to date in real-time. I believe that is why you are not seeing the changes in real-time when you update the bucket.

This happens when either the browser can't connect to the backend, or if the backend server stopped or got reset.....like if you restart the Docker container for example. In that case you would need to just refresh the browser and it should reconnect. If it doesn't connect and you are seeing this message all the time, could you press F12 when this happens, then click on Console and send me the output of what you see there? Then I can diagnose the error.

[jacobdhendrix]

Please help me if I was not understanding your original request. Your request in the first message of this thread was, "It would be nice to have an option to monitor a folder and automatically update the bucket when prerolls are added or removed."

This is what I added in this version. The ability to select the directory you want the bucket to monitor and then you are free to add, rename, delete files in that folder and it will reflect that in the bucket. This only happens when you select Directory as "Source of files" when creating/editing the bucket.

Other than the hidden Synology files, this is working as expected.

The original request by another user to monitor the files in the backend was so that if files got renamed, deleted, or moved, it would automatically update the settings file to reflect those changes. This includes the moving of a file. While you are suggesting a scenario where moving a file from a folder should delete the file from the bucket, I can counter that with scenarios where someone might have put the file in the wrong folder or are reorganizing their folders and they don't lose that file's connection with the bucket it is in.

Sounds like it's working correctly, then. I was just reporting it in case it was a bug. I like the renaming part of the feature. The temp folder was just to run some tests to make sure the app was working correctly.

That message happens when the frontend browser can't connect with the backend server for keeping the file list and bucket list up to date in real-time. I believe that is why you are not seeing the changes in real-time when you update the bucket.

This happens when either the browser can't connect to the backend, or if the backend server stopped or got reset.....like if you restart the Docker container for example. In that case you would need to just refresh the browser and it should reconnect. If it doesn't connect and you are seeing this message all the time, could you press F12 when this happens, then click on Console and send me the output of what you see there? Then I can diagnose the error.

I think I know what the cause of the issue is. I am running this behind a Cloudflare tunnel and Traefik. In Safari the app loads and works other than the backend error. In the Safari console I get an error indicating the connection to the backend needs to be served over HTTPS, since the app is being served over HTTPS.

[blocked] The page at https://prerollplus.mydomain.com/buckets requested insecure content from ws://prerollplus.mydomain.com/. This content was blocked and must be served over HTTPS.

When I load this in a Chromium based browser (I tried both Brave and Arc) I only get a white screen and this in the console:

Mixed Content: The page at 'https://prerollplus.mydomain.com/' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://prerollplus.mydomain.com/'. This request has been blocked; this endpoint must be available over WSS.
connectWebSocket @ App.jsx:131
react-dom.production.min.js:189 SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
at cs.connectWebSocket (App.jsx:131:15)
at cs.componentDidMount (App.jsx:49:10)
at wl (react-dom.production.min.js:261:217)
at xl (react-dom.production.min.js:260:446)
at bl (react-dom.production.min.js:259:431)
at react-dom.production.min.js:283:96
at Au (react-dom.production.min.js:281:398)
at au (react-dom.production.min.js:270:269)
at A (scheduler.production.min.js:13:203)
at MessagePort.P (scheduler.production.min.js:14:128)
fs @ react-dom.production.min.js:189
Show 1 more frame
Show less
react-dom.production.min.js:127 Uncaught SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
at cs.connectWebSocket (App.jsx:131:15)
at cs.componentDidMount (App.jsx:49:10)
at wl (react-dom.production.min.js:261:217)
at xl (react-dom.production.min.js:260:446)
at bl (react-dom.production.min.js:259:431)
at react-dom.production.min.js:283:96
at Au (react-dom.production.min.js:281:398)
at au (react-dom.production.min.js:270:269)
at A (scheduler.production.min.js:13:203)
at MessagePort.P (scheduler.production.min.js:14:128)

Next I tried HTTP over Tailscale, which is essentially the same as local (my server is at another location) and it works perfectly in both Safari and Chromium. So, if you feel like trying to fix HTTPS issues I'm happy to test, but don't feel like you need to fix it on my behalf since this is probably an edge case.

[chadwpalm]

Next I tried HTTP over Tailscale, which is essentially the same as local (my server is at another location) and it works perfectly in both Safari and Chromium. So, if you feel like trying to fix HTTPS issues I'm happy to test, but don't feel like you need to fix it on my behalf since this is probably an edge case.

Actually, it wasn't that difficult. I just added in for it to see if the http connection is secure or not and then select either the secure or non-secure websocket connection accordingly.

From what I saw online, for Cloudflare you'll need to make sure it is correctly configured to support WebSockets. This is typically supported by default, but check the settings in your Cloudflare dashboard under Network > WebSockets.

I also fixed the directory monitoring. I had forgotten to filter out directories when it grabbed the directory list. Should be okay now.

The update is in chadwpalm/prerollplus:feature-bucket-dir-0.1.5.25

If you want to test the Cloudflare secure connection again you can. I'll be curious if it works.

Thanks for the additional info!

[jacobdhendrix]

It's working perfectly now over HTTPS. Realtime folder updates and Synology filtering are working as well. Great job on this app! I'm not sure I have any more feature ideas but if I come up with anything I'll let you know.

[chadwpalm]

Thanks for the feedback (and kind words). I'll go ahead a roll this into the develop branch.