Larger bookmark files (5.8.3 --> 5.8.6)

[merlinpimpim]

Which version of floccus are you using?

5.8.6

How many bookmarks do you have, roughly?

60k

Are you using other means to sync bookmarks in parallel to floccus?

Yes. Google Sync

Sync method

WebDAV

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

Chrome 143.0.7499.170

Which version of Nextcloud Bookmarks are you using? (if relevant)

No response

Which version of Nextcloud? (if relevant)

No response

What kind of WebDAV server are you using? (if relevant)

No response

Describe the Bug

Hello,

I received version 5.8.6 yesterday and was on version 5.8.3 just before that:

The problem since version 5.8.6 is as follows:
(Obviously, I'm opening this case because I haven't added 14% more links (maybe just 3 or 4 links), including on another Chrome browser synchronized via Google sync).

I had to disable security for the synchronization to start:

After synchronization, the .xbel file was much larger (which probably explains the longer synchronization times):

7 MB more!

A bug ?

It's important to know because I have another PC with Chrome that is still on version 5.8.3, and there is some kind of conflict between the sizes of these two versions when they try to synchronize.
Should I force my second PC to upgrade to 5.8.6, or is there a problem with version 5.8.6 (or an intermediate version)?

Thank you.

Expected Behavior

Same index file size if no links are added when there is a Floccus version upgrade.

To Reproduce

Did not attempt to reproduce

Debug log provided

• I have provided a debug log file

[github-actions]

Hello! 👋

Thank you for taking the time to open this issue with floccus. I know it's frustrating when software causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at and if possible solved. Let me give you a short introduction on what to expect from this issue tracker to avoid misunderstandings. I'm Marcel. I created floccus a few years ago, and have been maintaining it since. I currently work for Nextcloud which leaves me with less time for side projects like this one than I used to have. I still try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it. Until then, please be patient. It helps when you stick around to answer follow up questions I may have, as very few bugs can be fixed directly from the first bug report, without any interaction. If information is missing in your bug report and the issue cannot be solved without it, I will have to close the issue after a while. Note also that GitHub in general is a place where people meet to make software better together. Nobody here is under any obligation to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can collaborate to make this software better. For everyone. Thus, if you can, you could also have a look at other issues to see whether you can help other people with your knowledge and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and try to fix the odd bug yourself. Everyone will be thankful for extra helping hands! If you cannot lend a helping hand, to continue the development and maintenance of this project in a sustainable way, I ask that you donate to the project when opening an issue (or at least once your issue is solved), if you're not a donor already. You can find donation options at https://floccus.org/donate/. Thank you!

One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the Nextcloud forum, to twitter or somewhere else. But this is a technical issue tracker, so please make sure to focus on the tech and keep your opinions to yourself.

Thank you for reading through this primer. I look forward to working with you on this issue! Cheers! 💙

[marcelklehr]

Hi @merlinpimpim
I apologize for the delay.

I'm opening this case because I haven't added 14% more links
I had to disable security for the synchronization to start

These two statements summarize a large part of the issue, I think. The idea of the failsafe is that if the user hasn't caused the inflation of their bookmarks collection and sees this warning, that the user should not disable the failsafe, but check the bookmarks collections on both the server and in the browser and then use the manual override buttons to fix the situation. If you disable the failsafe you literally open the doors for failure. :/ I'm sorry this happened to you and I suppose I need to make this more clear in the UI messaging.

The deeper question that you are probably also wondering about is of course: Why was there an increase in the bookmarks count when you didn't cause it? It could be that you have hit a rare and obscure bug in floccus that I haven't managed to fix yet, or it could be that this was caused by syncing with Google in parallel to floccus, which is known to cause issues like this (the reason for this, if you're curious, is that browser-builtin sync often takes the liberty to change the IDs of the bookmarks in the browser, but floccus relies on these to.. well ... identify the bookmarks, so when their IDs change, it will think they are new.)

Going forward: What do I recommend?

You don't need to update any floccus instance, if it doesn't update on its own. It's great that you still have the old xbel file. The broad strokes of how to fix this would be: Put the content of the old version of the file into the current file synced by floccus. If, like you say above, you had added 3-4 legitimate additions to your collections before things went south, you can rescue them from the new, corrupted file as follows: Since XBEL is just text, you can open it in a text editor and read and write to it and you can use a diff tool to perhaps figure out what those were and insert them into the new file at the right place (make sure to overtake the "highest ID" section at the start as well).

I hope this helps, let me know how it goes.

[merlinpimpim]

First, thank you for your reply.

check the bookmarks collections on both the server and in the browser and then use the manual override buttons to fix the situation. If you disable the failsafe you literally open the doors for failure.

In my case, what should I have done? Are the manual override buttons the arrows (download/upload)?
Because before disabling the failsafe, I checked that the bookmarks seemed OK on the browser side (I say “seemed” because I have over 60k bookmarks in numerous directories).

But after that I also tried deleting the Xbel file and re-uploading my bookmarks from scratch. The result is an Xbel file of the same “new” larger size.
Does this mean that my browser bookmarks are corrupted?

Why was there an increase in the bookmarks count when you didn't cause it

Thanks for the explanations.

Since XBEL is just text, you can open it in a text editor and read and write to it and you can use a diff tool to perhaps figure out what those were and insert them into the new file at the right place (make sure to overtake the "highest ID" section at the start as well).

I didn't specify that bookmarks are password-encrypted using the mechanism implemented in Floccus. So I don't think what you're suggesting is possible?
Just in case: is it possible that an encryption issue caused the file to grow artificially (between versions 5.8.3 and 5.8.6)?

Well, having said all that, I don't see any functional problems whatsoever!
Is the file strangely larger? Yes.
Do I have corrupted bookmarks in the browser? Not found any.
Does it still sync? Yes.

Maybe it's not worth investigating further?

[marcelklehr]

Are the manual override buttons the arrows (download/upload)?

Yes, exactly.

I didn't specify that bookmarks are password-encrypted using the mechanism implemented in Floccus. So I don't think what you're suggesting is possible?

ah, indeed. Yes, the encryption complicates things.

is it possible that an encryption issue caused the file to grow artificially (between versions 5.8.3 and 5.8.6)?

I don't think so, nothing was changed in that regard.

Well, having said all that, I don't see any functional problems whatsoever!
Is the file strangely larger? Yes.
Do I have corrupted bookmarks in the browser? Not found any.
Does it still sync? Yes.
Maybe it's not worth investigating further?

Mmmh, it's quite unlikely that the bookmarks file grew larger without an increase in bookmarks, especially since you received the failsafe error. (As you checked your local bookmarks in that moment, it seems likely that this is either connected with syncing with Google in parallel or due to an actual bug). Could you perhaps use a browser extension to check for duplicates to see if there are any new duplicates that you weren't aware of?

[merlinpimpim]

I used https://chromewebstore.google.com/detail/quick-duplicate-bookmarks/hjiaieibgociindhongdoeokppalmabk to deduplicate my bookmarks.
I think about 1k bookmarks were cleaned up, but the Xbel file only went from 22058kb to 21523kb :(
Therefore, we can assume that the problem lies elsewhere than in the duplication of bookmarks?

[marcelklehr]

Are you syncing the root folder?

[merlinpimpim]

Yes, from the beginning.

[marcelklehr]

When syncing the root folder, I've sometimes seen bookmarks ending up in the root folder and or in the Mobile bookmarks folder where you cannot find them easily in the Browser UI, maybe that happened to you. Could you try exporting your bookmarks from your browser as HTML and checking what this reveals?

[merlinpimpim]

I don't quite understand what I should be looking for in the HTML export, but I can say right away that:

• I only have a dozen favorites in the root folder
• I also have favorites in Mobile Bookmarks (which is normal and was there before: these are the bookmarks from my mobile device that Google Sync synchronizes with my PC).

The deduplication tool mentioned above took these folders into account. And for example, I have no duplicates between my mobile favorites and the rest.
(I have about 500 favorites in the mobile bookmarks folder).

For your information, I use Floccus as follows:
Mobile device <--GSync--> Personal PC <--Floccus and webdav on a self-hosted space--> Work PC (where the admin has blocked Google Sync for everyone).

[merlinpimpim]

When I have some time, I will try the following procedure:
Install a separate “new” version of Chrome (Chrome beta, for example), install Floccus, import my old Xbel (lightweight version), export an HTML file of the favorites imported by Floccus, and compare (full text) this HTML file with the one exported from my usual Chrome.
What do you think?