Skip to content

RELEASE-NOTES.md

Latest commit

 

History

History
306 lines (237 loc) · 16.9 KB

RELEASE-NOTES.md

File metadata and controls

306 lines (237 loc) · 16.9 KB

dataone-indexer Release Notes

Note

The Helm chart now assumes you have the RabbitMQ Cluster Operator pre-installed on your Kubernetes cluster. Alternatively, it can be configured to use your existing RabbitMQ instance - see the README file for details.

helm chart version 2.0.1

Release date: 2026-02-09

This is a patch release to fix a bug with the new RabbitMQ Cluster Operator-based deployment introduced in version 2.0.0, and to refine some performance settings for RabbitMQ and Solr.

  • Bug fix: Setting rabbitmq.enabled: false in the Helm values.yaml did not disable the installation of the RabbitMQ operator as expected (Issue #308)

  • Indexing Performance: Configuration Changes for Index Worker and Solr (Issue #319):

    • Indexer is now configured to perform automatic Solr commits instead of forcing a hard commit after every Solr update
      • Auto-commit time periods can be overridden using solr.extraEnvVars to set SOLR_OPTS to "-Dsolr.autoCommit.maxTime=<hardPeriod> -Dsolr.autoSoftCommit.maxTime=<softPeriod>". Default values are 1 second for auto soft commits, and 120 seconds for auto hard commits.
      • Forced commits per update can be controlled using commitPerUpdate.hardCommit and commitPerUpdate.softCommit. Both default to false.
      • See https://solr.apache.org/guide/solr/latest/configuration-guide/commits-transaction-logs.html for details
    • solr.javaMem changed from "-Xms512m -Xmx2g" to "-XX:MaxRAMPercentage=50"

  • RabbitMQ Changes (Issue #311)

    • Added rabbitmq.autoPodAntiAffinity to spread RabbitMQ pods across nodes - recommended for production installations with multiple replicas, to increase availability.
      • Anti-affinity can be "required" or "preferred", and weight is configurable
      • Can be overridden disabled, or overridden using new rabbitmq.affinity section
    • rabbitmq.additionalConfig: removed additionalConfig: disk_free_limit.relative: "1.0" and replaced with disk_free_limit.absolute: 1500MB

dataone-indexer version 3.2.0 & helm chart version 2.0.0

Release date: 2025-12-02

dataone-indexer version 3.2.0

  • Performance Improvement: New Approach to index resource map objects (see Issue #101)
  • Handle pids/sids in isPartOf/hasPart, within resource maps (see Issue #291)
  • Remove hasPart from the json-ld subprocessor (see Issue #297)
  • Dependency updates:
    • bump commons-beanutils to 1.11.0
    • bump Apache commons-lang3 to 3.18.0
    • Update Docker base image from eclipse-temurin:17.0.15_6-jre-noble to eclipse-temurin:17.0.17_10-jre-noble
  • Testing bugs resolved:
    • LegacyStoreObjManagerTest no longer requires the existence of /var/metacat/data & /var/metacat/documents
    • Spring no longer tries and fails to create httpService bean when running IndexWorkerTest

helm chart version 2.0.0

Note

This chart now assumes you have the RabbitMQ Cluster Operator pre-installed on your Kubernetes cluster. Alternatively, it can be configured to use your existing RabbitMQ instance - see the README file for details.

Caution

ENSURE THAT THE RABBITMQ QUEUE IS EMPTY, before upgrading, because the new chart version will store the queue on a newly-created PV/PVC, and any jobs already in the old queue will be lost!

  • RabbitMQ has been upgraded from version 3.1.7 to version 4.1.3, and is deployed using the RabbitMQ Cluster Operator (instead of the Bitnami RabbitMQ Helm subchart).
  • livenessProbe and readinessProbe parameters are now fully configurable in values.yaml

helm chart version 1.3.3

Release date: 2025-07-29

Caution

We strongly recommend that you upgrade to this version of the helm chart before August 28th, 2025, since previous versions will stop working after Bitnami introduces restrictions to container image availability!

This is a patch release to account for upcoming changes to Bitnami container image availability; see Bitnami's announcements:

This latest chart pulls the Bitnami RabbitMQ, Solr, and Zookeeper image versions used by dataone-indexer from the bitnamilegacy repository, which will remain functional after the August cutoff

helm chart version 1.3.3

Release date: 2025-07-29

Caution

We strongly recommend that you upgrade to this version of the helm chart before August 28th, 2025, since previous versions will stop working after Bitnami introduces restrictions to container image availability!

This is a patch release to account for upcoming changes to Bitnami container image availability; see Bitnami's announcements:

This latest chart pulls the Bitnami RabbitMQ, Solr, and Zookeeper image versions used by dataone-indexer from the bitnamilegacy repository, which will remain functional after the August cutoff

dataone-indexer version 3.1.5 & helm chart version 1.3.2

Release date: 2025-06-26

dataone-indexer version 3.1.5

This is a patch release with the following minor fixes and upgrades

  • Dataone-indexer can handle legacy Metacat object repository (Issue #222)
  • Remove some extra log statements (for version conflict retries) that are confusing to users (Issue #243)
  • Indexer performance improvement: Decrease the re-try waiting time for a version conflict error (Issue #245)
  • Remove unnecessary dependency on PostrgeSQL jar (Issue #247)

helm chart version 1.3.2

  • Bump indexer App version to 3.1.5

dataone-indexer version 3.1.4 & helm chart version 1.3.1

Release date: 2025-05-20

dataone-indexer version 3.1.4

This is a patch release with the following minor fixes and upgrades:

  • Use org.apache.commons.logging throughout; removed direct references to org.apache.log4j.Logger (Issue #178)
  • Remove the spring-web component from the dependency library. (Issue #216)
  • Disable livenessProbe check for non-k8s deployments. (Issue #225)
  • Bump log4j-layout-template-json version to 2.24.3

helm chart version 1.3.1

  • Add a "warm up" postStart hook for Lustre/HPC Filesystems (disabled by default - see idxworker.enableMountWarmupHook in values.yaml. Addresses visibility issues with Lustre and similar HPC filesystems.) (Issue #227)
  • Remove subPath for log4j properties configmap to allow dynamic reloading (Issue #210)
  • Allow configuration of Java Heap Memory for index workers, via values.yaml (idxworker.javaMem) (Issue #231)
  • Added extraVolumes and extraVolumeMounts to values.yaml, allowing users to specify additional volumes and mounts in their deployments (Issue #223)
  • Full Support for granular configuration of security context in Containers and InitContainers (Issue #224)
    • Add podSecurityContext.runAsUser: 59997 (applies to dependencies initContainer and indexer container)
    • Add runAsNonRoot: true to dependencies initContainer (was previously applied only to indexer container)
  • Introduced idxworker.tripleDbStorageDefinition in values.yaml to allow custom storage configurations for the tripleDB volume. Supports alternative storage types such as hostPath and emptyDir (Issue #228).
  • Bump indexer App version to 3.1.4

Caution

If upgrading from 1.2.0, note the section entitled: Caution - ENSURE THAT THE RABBITMQ QUEUE IS EMPTY, in the release notes for helm chart v1.3.0 below!,

dataone-indexer version 3.1.3 & helm chart version 1.3.0

Release date: 2025-05-01

dataone-indexer version 3.1.3

  • This is a patch release to update logging dependencies and fix a connectivity issue:
  • Restore disconnected RabbitMQ connections - The fix in version 3.1.2 failed to fully resolve RabbitMQ disconnects under some circumstances, leading to the indexer failing to dequeue new jobs. We believe version 3.1.3 corrects the problem.
  • Bump the log4j version from 2.17.1 to 2.24.3
  • Bump slf4j-api from 1.6.1 to 2.0.17
  • Bump slf4j-reload4j from 1.7.36 to 2.0.17

helm chart version 1.3.0

Caution

ENSURE THAT THE RABBITMQ QUEUE IS EMPTY, before upgrading or installing a new chart version for the first time, because each new chart version will store the queue on a newly-created PV/PVC!

This applies only the initial installation or upgrade. After this, RabbitMQ will continue to use the same newly-created PV/PVC, and the queue will not be lost. The new PVC will be named: data-[release-name]-rabbitmq-[rmq-version]-[idx], where [rmq-version] is the rabbitmq app version (not the chart version), with periods replaced by dashes, and [idx] is the statefulset ordinal index; e.g.: data-metacatarctic-rabbitmq-3-13-7-0

When upgrade or installation is complete, you can then safely kubectl delete both the old PVC and the old PV (provided you're certain the queue was empty).

Note

This behavior can be overridden by setting .Values.rabbitmq.nameOverride to the same name as the previous version, but this is NOT recommended, since the RabbitMQ installation then becomes an upgrade instead of a fresh install, and may require significant manual intervention; see: https://www.rabbitmq.com/docs/feature-flags#version-compatibility

dataone-indexer version 3.1.2 & helm chart version 1.2.0

dataone-indexer helm chart version 1.1.2

  • Release date: 2025-02-24
    • Add support for hostPath volume
    • Increase .Values.idxworker.solrVerConflictMaxTries from 50 to 25000 to resolve solr version-conflict errors
    • (Application software version remains unchanged at 3.1.1)

dataone-indexer version 3.1.1 & helm chart version 1.1.1

  • Release date: 2024-12-17
  • dataone-indexer version 3.1.1
    • This is a minor release to fix logging issues. Changes were made to the slf4j and log4j dependencies to ensure that log messages are written correctly (see PR #155).
  • helm chart version 1.1.1
    • Bump Application version to 3.1.1
    • Change log4j properties configMap mount-point from log4j.properties to log4j2.properties

dataone-indexer version 3.1.0 & helm chart version 1.1.0

  • Release date: 2024-11-21
  • dataone-indexer version 3.1.0
    • Integrate with the new Dataone hash-based storage library hashstore-java.
      • Indexer no longer needs access to an aut token in order to index private datasets.
    • Update Docker base image to eclipse-temurin:17.0.12_7-jre-jammy
    • Upgrade log4j-core to 2.24.0 to fix "method can't be found" issue
    • Bump dependencies:
      • org.apache.commons:commons-lang3 from 3.4 to 3.17.0
      • org.slf4j:slf4j-api from 1.7.36 to 2.0.16
      • org.springframework.data:spring-data-commons from 1.6.5.RELEASE to 3.3.4
      • org.apache.maven.plugins:maven-compiler-plugin from 2.0.1 to 3.13.0
      • com.coderplus.maven.plugins:copy-rename-maven-plugin from 1.0 to 1.0.1
      • org.apache.logging.log4j:log4j-jcl from 2.17.1 to 2.24.0
      • org.apache.maven.plugins:maven-clean-plugin from 3.2.0 to 3.4.0
      • com.fasterxml.jackson.core:jackson-annotations from 2.13.3 to 2.18.0
  • helm chart version 1.1.0
    • Bump Application version to 3.1.0
    • Add storage to values.yaml for new hashstore integration

dataone-indexer version 3.0.2 & helm chart version 1.0.2

  • Release date: 2024-07-29
  • dataone-indexer version 3.0.2
    • Bug fix - RabbitMQ Channel timeouts (PR #119)
  • helm chart version 1.0.2
    • Bump Application version to 3.0.2
    • Make .Values.rabbitmq.auth.existingPasswordSecret a required value

dataone-indexer version 3.0.1 & helm chart version 1.0.1

  • Release date: 2024-07-08
  • dataone-indexer version 3.0.1
    • Bump rmq amqp client to 5.21.0
    • Add healthcheck code
    • Exit app if unrecoverable exception occurs when started from main() method
  • helm chart version 1.0.1
    • Change .Values.idxworker.cn_url to .Values.global.d1ClientCnUrl
    • Get fullname from metacat chart or provide in values.yaml
    • Add simple 'exec' liveness probe. Remove readiness probe

dataone-indexer version 3.0.0 & helm chart version 1.0.0

  • Release date: 2024-04-25
  • dataone-indexer version 3.0.0 -- first release of dataone-indexer
  • helm chart version 1.0.0 -- first release of helm chart