Skip to content

Zstd support#4

Open
mehmet-karaman wants to merge 14 commits intouvoigt:masterfrom
mehmet-karaman:zstd_support
Open

Zstd support#4
mehmet-karaman wants to merge 14 commits intouvoigt:masterfrom
mehmet-karaman:zstd_support

Conversation

@mehmet-karaman
Copy link
Contributor

Implemented Zstd support upon the Apache commons compress zip archive implementation.

@mehmet-karaman mehmet-karaman force-pushed the zstd_support branch 2 times, most recently from 8d91b8d to 1e141a1 Compare March 28, 2025 08:49
Copy link

@iloveeclipse iloveeclipse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If the zstd support doesn't work or disabled, editor opens and is empty etc.
    So far so good. But editor should not allow to save or change anything, right now it allows to add entries, it will be dirty after that but fails to save via NPE:
java.lang.NullPointerException: Cannot invoke "org.eclipse.core.runtime.content.IContentType.getFileSpecs(int)" because the return value of "org.eclipse.core.runtime.content.IContentTypeManager.getContentType(String)" is null
	at zipeditor.model.ZipContentDescriber.getContentTypeForFileExtension(ZipContentDescriber.java:99)
	at zipeditor.ZipEditor.internalSave(ZipEditor.java:384)
	at zipeditor.ZipEditor.doSave(ZipEditor.java:364)

Ideally editor would disable all actions, or, if it is toomuch work, at least do not fail with NPE but with a message that the file can't be changed.

  1. If the zip can't be opened (zstd is disabled) and also sometimes if it is enabled (so it seem to be existing issue) I get following errors in the log:
java.lang.NullPointerException: Cannot invoke "zipeditor.ZipOutlinePage.setInput(zipeditor.model.Node)" because "this.this$0.fOutlinePage" is null
	at zipeditor.ZipEditor$19.run(ZipEditor.java:1125)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:5047)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4527)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1151)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:339)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1042)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:153)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:668)

Fix:

diff --git a/ZipEditor/src/zipeditor/ZipEditor.java b/ZipEditor/src/zipeditor/ZipEditor.java
index 5561f2c..1869e05 100644
--- a/ZipEditor/src/zipeditor/ZipEditor.java
+++ b/ZipEditor/src/zipeditor/ZipEditor.java
@@ -549,2 +549,3 @@
 		fZipActionGroup.dispose();
+		fModel = null;
 		super.dispose();		
@@ -1124,3 +1125,5 @@
 				public void run() {
-					fOutlinePage.setInput(fModel.getRoot());
+					if(fOutlinePage != null && fModel != null) {
+						fOutlinePage.setInput(fModel.getRoot());
+					}
 				}
  1. Right clicking on a zip file with zstd disabled shows an error popup with the message that zstd support is not there. I believe this error popup is not needed here, logging should be enough.
    Stack:
zipeditor.model.ZipEditorZstdException: Support for zstd is deactivated. Please check the zip editor settings.
   at zipeditor.model.zstd.ZstdUtilities.getInputStream(ZstdUtilities.java:16)
   at zipeditor.model.ZipModel$1.createZstdInputStream(ZipModel.java:115)
   at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.getNextZipEntry(ZipArchiveInputStream.java:791)
   at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.getNextEntry(ZipArchiveInputStream.java:648)
   at zipeditor.model.ZipModel.detectType(ZipModel.java:119)
   at zipeditor.model.ZipContentDescriber.describe(ZipContentDescriber.java:144)
   at org.eclipse.core.internal.content.ContentTypeCatalog.describe(ContentTypeCatalog.java:295)
   at org.eclipse.core.internal.content.ContentTypeCatalog.collectMatchingByContents(ContentTypeCatalog.java:266)
   at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesForSubset(ContentTypeCatalog.java:490)
   at org.eclipse.core.internal.content.ContentTypeCatalog.internalFindContentTypesFor(ContentTypeCatalog.java:549)
   at org.eclipse.core.internal.content.ContentTypeCatalog.findContentTypesFor(ContentTypeCatalog.java:383)
   at org.eclipse.core.internal.content.ContentTypeMatcher.findContentTypesFor(ContentTypeMatcher.java:57)
   at org.eclipse.lsp4e.LSPEclipseUtils.getFileContentTypes(LSPEclipseUtils.java:1327)
   at org.eclipse.lsp4e.LanguageServersRegistry.canUseLanguageServer(LanguageServersRegistry.java:463)
   at org.eclipse.lsp4e.HasLanguageServerPropertyTester.test(HasLanguageServerPropertyTester.java:29)
   at org.eclipse.core.internal.expressions.Property.test(Property.java:65)
   at org.eclipse.core.expressions.TestExpression.evaluate(TestExpression.java:107)
   at org.eclipse.core.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:54)
   at org.eclipse.core.internal.expressions.AdaptExpression.evaluate(AdaptExpression.java:121)
   at org.eclipse.core.expressions.CompositeExpression.evaluateOr(CompositeExpression.java:68)
   at org.eclipse.core.expressions.OrExpression.evaluate(OrExpression.java:26)
   at org.eclipse.core.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:54)
   at org.eclipse.core.internal.expressions.IterateExpression.evaluate(IterateExpression.java:163)
   at org.eclipse.e4.ui.internal.workbench.ContributionsAnalyzer.isCoreExpressionVisible(ContributionsAnalyzer.java:263)
   at org.eclipse.e4.ui.internal.workbench.ContributionsAnalyzer.isVisible(ContributionsAnalyzer.java:237)
   at org.eclipse.e4.ui.workbench.renderers.swt.ContributionRecord.computeVisibility(ContributionRecord.java:167)
   at org.eclipse.e4.ui.workbench.renderers.swt.ContributionRecord.updateVisibility(ContributionRecord.java:98)
   at org.eclipse.e4.ui.workbench.renderers.swt.MenuManagerRendererFilter.updateElementVisibility(MenuManagerRendererFilter.java:173)
   at org.eclipse.e4.ui.workbench.renderers.swt.MenuManagerRendererFilter.updateElementVisibility(MenuManagerRendererFilter.java:184)
   at org.eclipse.e4.ui.workbench.renderers.swt.MenuManagerShowProcessor.showMenu(MenuManagerShowProcessor.java:248)
  1. If zstd support is disabled, I expect "usual" zip files to work. However, creating a new empty zip file, adding few entries and save results in an error. Same happens if I open existing regular zip file, add an entry and try to save:
java.lang.NullPointerException: Cannot invoke "org.eclipse.core.runtime.content.IContentType.getFileSpecs(int)" because the return value of "org.eclipse.core.runtime.content.IContentTypeManager.getContentType(String)" is null
	at zipeditor.model.ZipContentDescriber.getContentTypeForFileExtension(ZipContentDescriber.java:99)
	at zipeditor.ZipEditor.internalSave(ZipEditor.java:384)
	at zipeditor.ZipEditor.doSave(ZipEditor.java:364)

That should not happen.

@iloveeclipse
Copy link

Another observation: starting without jnilib and aircompressor: the preference page doesn't disable the comboand the checkbox for zstd support is enabled and checked. Should be all disabled and some error label shown like "to enable zstd install jnilib or aircompressor libraries"

@mehmet-karaman
Copy link
Contributor Author

Another observation: starting without jnilib and aircompressor: the preference page doesn't disable the comboand the checkbox for zstd support is enabled and checked. Should be all disabled and some error label shown like "to enable zstd install jnilib or aircompressor libraries"

Good observation, as the libraries are optional, we have to set the default "zstd active" flag to false. And this should be cared by the user if he wants to activate this, he has to take care of the installed libraries..

@mehmet-karaman
Copy link
Contributor Author

Could reproduce the bug in the Package Explorer..

Zstd support was active, the expander besides the zip elements tree item was visible.. Disabled the Zstd Support and expandede that node and it has thrown the NPE.. Now this is also fixed..

Copy link

@iloveeclipse iloveeclipse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nnow I see that updating entries in existing not-zstd compressed file create zstd entries:
image

The file below was created with default compression and .classpath inside was updated while the preference "ZSTD Default Compression" was enabled.

However, the preference should affect only new files, not existing.

Additionally to that, if such file is saved, next attempt to open it with zstd disabled results in an error message is shown "An error occurred trying to save":

image

After that the zip file is opened but the entry encoded with zstd compression is not shown (OK).

However, after that one could modify any other entry in the zip & save the zip, loosing the zstd encoded entry (NOK).

@uvoigt
Copy link
Owner

uvoigt commented Jul 5, 2025

Hi,
I did not get notified by email about this activities. That's why I was so silent ;)
At the moment, I don't have the time to check the feature. But I do in a couple of weeks.
@iloveeclipse do you think the feature is complete?

@iloveeclipse
Copy link

@uvoigt : at some point we assumed there is no one interested and most resent patches are not pushed here. We have few commits on top which clean up preferences page and "mavenize" the build to produce a fully self contained p2 repository with all dependencies included.
We wil push on Monday. With that it would be complete.

@uvoigt
Copy link
Owner

uvoigt commented Jul 5, 2025

Thanks @iloveeclipse,
I will have a look as soon as I get the time. I see that I pushed my latest patches up to the current release and I am not aware of other recent patches.
Anyway, thank you for your contribution and interest!
Uwe

@mehmet-karaman
Copy link
Contributor Author

Pushed the latest changes.

  • Mavenified zip editor (uses tycho and eclipse orbit similar building)
  • fixed smaller ui glitches in the preference page

The maven build can be started with "mvn clean install". The built artifacts can be found in the following path "/zipeditor/ZipEditor.UpdateSite/target/repository"

@uvoigt
Copy link
Owner

uvoigt commented Jul 18, 2025

Pushed the latest changes.

  • Mavenified zip editor (uses tycho and eclipse orbit similar building)
  • fixed smaller ui glitches in the preference page

The maven build can be started with "mvn clean install". The built artifacts can be found in the following path "/zipeditor/ZipEditor.UpdateSite/target/repository"
Please give me some time @mehmet-karaman to have a look and test. I will definitely accept your changes at last. The only problem is that my time is somewhat limited these days.
Best regards,
Uwe

mehmet-karaman and others added 4 commits October 15, 2025 22:24
- fixed wrong "wrapped" m2e bundle dependency on airlift in tests
- fixed wrong classpath entry for aircompressor in zipeditor
- fixed project settings to refer to proper Java 21 environment
- bumped zipeditor bundle/feature/update site version to 1.2.0
- moved to a central utility
- added handlers to handle these library usages. This helps to avoid the
ClassNotFoundExceptions
- Added message in Zstd Preferences Page to show if the Library is
available or not.
- active flag is used also with checking if any library is available
- fixed NPEs
- Added hint in zip editor if zstd isn't available.
- fix Project Explorer expanding bug (if afterwards the zstd was
deactivated)
- changed resource close() to try with resources.
- Add new files by Drag And Drop with Zstd compression if the new pref
is true.
- Fixed Zip entry properties bug with zsdt in method combobox.
- Implemented PR comments.
- changed two dim. array of libraries (combobox edit field) to more
readable records and conversion to two dim. array.
- added checkbox selection of zstd default in dialog according the
default zstd compression state.
- added additional checks if zstd is used as default but this method is
not used in this zip file.
Was checking before that JNI lib is selected, now its checking that
aircompressor is not selected.
@mehmet-karaman
Copy link
Contributor Author

mehmet-karaman commented Oct 16, 2025

Hello @uvoigt ,

I've done the rebase, there are no conflicts anymore.

@mehmet-karaman
Copy link
Contributor Author

The maven build can be started with "mvn clean install". The built artifacts can be found in the following path "/zipeditor/ZipEditor.UpdateSite/target/repository"

After that the target platform can be selected as /ZipEditor/latest.target ..

@uvoigt
Copy link
Owner

uvoigt commented Mar 6, 2026

@mehmet-karaman

when building I get

[INFO] �[1m----< �[0;36morg.eclipse.orbit.maven-bnd:org.eclipse.orbit.maven-bnd.site�[0;1m >----�[m
[INFO] �[1mBuilding org.eclipse.orbit.maven-bnd.site 1.0.0-SNAPSHOT           [4/8]�[m
[INFO]   from maven-bnd/site/pom.xml
[INFO] �[1m-------------------------[ eclipse-repository ]-------------------------�[m
[INFO] Resolving dependencies of MavenProject: org.eclipse.orbit.maven-bnd:org.eclipse.orbit.maven-bnd.site:1.0.0-SNAPSHOT @ /Users/voigt/eclipse-workspace-memet/zipeditor/maven-bnd/site/pom.xml
[INFO] Resolving target definition file:/Users/voigt/eclipse-workspace-memet/zipeditor/maven-bnd/tp/MavenBND.target for environments=[win32/win32/x86_64, macosx/cocoa/x86_64, macosx/cocoa/aarch64, linux/gtk/x86_64, linux/gtk/aarch64], include source mode=honor, referenced repository mode =ignore, execution environment=StandardEEResolutionHints [executionEnvironment=OSGi profile 'JavaSE-21' { source level: 21, target level: 21}] with Tycho Provisioning Agent (extension>org.eclipse.tycho:tycho-maven-plugin:4.0.13)
[INFO] Resolving MavenDependencyRoots = [GroupId = io.airlift, ArtifactId = aircompressor, Version = 2.0.2, ArtifactType = jar], IncludeDependencyScope = [compile], MissingManifestStrategy = GENERATE, IncludeSource = true
[INFO] default instructions = {Export-Package=*;version="${version}";-noimport:=true, Bundle-SymbolicName=${mvnGroupId}.${mvnArtifactId}, Bundle-Version=${version}, Bundle-Name=Bundle derived from maven artifact ${mvnGroupId}:${mvnArtifactId}:${mvnVersion}, DynamicImport-Package=*, version=${version_cleanup;${mvnVersion}}, Import-Package=*;resolution:=optional}
[INFO] io.airlift:aircompressor:2.0.2 is wrapped as a bundle with bundle symbolic name io.airlift.aircompressor
[INFO] The artifact can be referenced in feature files with the following data: <plugin id="io.airlift.aircompressor" version="2.0.2"/>
[INFO] Resolving MavenDependencyRoots = [GroupId = org.apache.commons, ArtifactId = commons-compress, Version = 1.28.0-SNAPSHOT, ArtifactType = jar, GroupId = org.apache.commons, ArtifactId = commons-lang3, Version = 3.18.0-SNAPSHOT, ArtifactType = jar], IncludeDependencyScope = [compile], MissingManifestStrategy = GENERATE, IncludeSource = true
[INFO] Downloading from apache snapshot: https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-compress/1.28.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading from apache snapshot: https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-compress/1.28.0-SNAPSHOT/commons-compress-1.28.0-SNAPSHOT.pom
[WARNING] The POM for org.apache.commons:commons-compress:jar:1.28.0-SNAPSHOT is missing, no dependency information available
[INFO] Downloading from apache snapshot: https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-compress/1.28.0-SNAPSHOT/commons-compress-1.28.0-SNAPSHOT.jar
[INFO] �[1m------------------------------------------------------------------------�[m
[INFO] �[1mReactor Summary:�[m
[INFO] 
[INFO] ZipEditor.parent 1.0.0-SNAPSHOT .................... �[1;32mSUCCESS�[m [  0.054 s]
[INFO] org.eclipse.orbit.maven-bnd.parent 1.0.0-SNAPSHOT .. �[1;32mSUCCESS�[m [  0.020 s]
[INFO] org.eclipse.orbit.maven-bnd.tp 1.0.0-SNAPSHOT ...... �[1;32mSUCCESS�[m [  2.064 s]
[INFO] org.eclipse.orbit.maven-bnd.site 1.0.0-SNAPSHOT .... �[1;31mFAILURE�[m [  0.335 s]
[INFO] ZipEditor 1.2.1-SNAPSHOT ........................... �[1;33mSKIPPED�[m
[INFO] Zip Editor Feature 1.2.1-SNAPSHOT .................. �[1;33mSKIPPED�[m
[INFO] ZipEditor.targetPlatform 1.0.0-SNAPSHOT ............ �[1;33mSKIPPED�[m
[INFO] ZipEditor.site 1.0.0-SNAPSHOT ...................... �[1;33mSKIPPED�[m
[INFO] �[1m------------------------------------------------------------------------�[m
[INFO] �[1;31mBUILD FAILURE�[m

Is it correct to depend on snapshots?

@uvoigt
Copy link
Owner

uvoigt commented Mar 11, 2026

I would like to have the maven build for the next release that should include the changes of open-file-performance and master + fallback implementation to ZipArchiveInputStream.
To achieve this, I simply copy the Maven files from here.
Or you prepare another PR with the mavenize commit. That might be easier.

Btw., I asked AI and it generates nearly the same stuff.

@mehmet-karaman
Copy link
Contributor Author

mehmet-karaman commented Mar 11, 2026

I would like to prepare a PR that contains the mavenization + ZipArchiveInputStream fallback..

@uvoigt
Copy link
Owner

uvoigt commented Mar 11, 2026

Today I had so much time. I added the maven things + Github action to the open-file-performance branch
It is already working with a new RC-release in the update manager.
Pardon if the code now differs from your solution. It seems to me, there are some files that are unnecessary.
Actually, the dependencies for commons-compress are not yet added. I will do when merging into master.

@mehmet-karaman
Copy link
Contributor Author

Ok, would you like to continue with the commons-compress integration (as fallback) or shall i continue from here? Do you have time to continue?

@uvoigt
Copy link
Owner

uvoigt commented Mar 11, 2026

Maybe tomorrow I have some time too. So I will try that.

@mehmet-karaman
Copy link
Contributor Author

Let me know if you need some help. Thanks.

@uvoigt
Copy link
Owner

uvoigt commented Mar 15, 2026

@mehmet-karaman I would offer to merge master into your branch because I caused the conflicts.
Not sure which would be the best way, e.g. I create a branch on the base commit of your branch and you retarget the PR. Then this can be fast-forwarded and I merge master into it. On top of that we can discuss.

I created fewer files when adding Maven support. It already runs in the Github CI so I want to stay with it. New release 1.2.0 is already out.
I would wish to integrate the Zstd support in smaller steps because there is potential for problems. I have seen that in the other PR.
Still, I am not familiar with the content of this PR. I gave up when I saw that the build failed.
Within the next days I probably will have time to work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants