add property wildcard test with eq operator

Author: thomas loubrieu

.github/workflows/codeql-analysis.yml

@@ -77,30 +77,3 @@ jobs:
       with:
         name: codeql-artifacts
         path: ${{ env.RESULTS_DIR }}
-
-  sloc-count:
-      name: SLOC Count
-      runs-on: ubuntu-latest
-      if: github.actor != 'pdsen-ci'
-      steps:
-          -
-              name: 💳 Checkout
-              uses: actions/checkout@v5
-              with:
-                  lfs: true
-                  fetch-depth: 0
-                  token: ${{secrets.ADMIN_GITHUB_TOKEN}}
-
-          -
-            name: Count Lines of Code (cloc)
-            uses: djdefi/cloc-action@6
-            with:
-                options: --report-file=cloc.md
-
-
-          -
-            name: Upload SLOC
-            uses: actions/upload-artifact@v5
-            with:
-              name: sloc-count
-              path: ${{ github.workspace }}/cloc.md

.gitignore

@@ -72,6 +72,7 @@ var/
 *.swp
 bin/
 typescript
+.env
 
 # Java stuff
 *.javac

CHANGELOG.md

@@ -1,16 +1,25 @@
 # Changelog
 
-## [«unknown»](https://github.com/NASA-PDS/registry/tree/«unknown») (2025-11-19)
+## [release/1.5.0](https://github.com/NASA-PDS/registry/tree/release/1.5.0) (2026-01-08)
 
-[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.4.0...«unknown»)
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/release/1.6.0...release/1.5.0)
+
+## [release/1.6.0](https://github.com/NASA-PDS/registry/tree/release/1.6.0) (2026-01-07)
+
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.4.0...release/1.6.0)
 
 **Requirements:**
 
 - As a user, I want to receive an alert when my registry credentials are readable by someone else than myself [\#424](https://github.com/NASA-PDS/registry/issues/424)
 - As a node operator, I want the the registry schema to update autonomously / dynamically when new data is ingested. [\#113](https://github.com/NASA-PDS/registry/issues/113)
 
+**Improvements:**
+
+- Document and script the monthly legacy registry statusing [\#460](https://github.com/NASA-PDS/registry/issues/460)
+
 **Defects:**
 
+- Some properties return "null" values [\#456](https://github.com/NASA-PDS/registry/issues/456) [[s.medium](https://github.com/NASA-PDS/registry/labels/s.medium)]
 - lid and pds:Identification\_Area.pds:logical\_identifier do not behave the same in the API [\#423](https://github.com/NASA-PDS/registry/issues/423) [[s.medium](https://github.com/NASA-PDS/registry/labels/s.medium)]
 - Some of product urn:nasa:pds:lro\_diviner\_derived2::1.0's fields are not indexed [\#422](https://github.com/NASA-PDS/registry/issues/422) [[s.high](https://github.com/NASA-PDS/registry/labels/s.high)]
 - Error `FIPS mode: only SunJSSE TrustManagers may be used` [\#440](https://github.com/NASA-PDS/registry/issues/440) [[s.high](https://github.com/NASA-PDS/registry/labels/s.high)]
@@ -23,12 +32,17 @@
 
 - Troubleshoot SBN 403 Error [\#414](https://github.com/NASA-PDS/registry/issues/414)
 - Add the --legacy-sync command option to the EN sweeper in production [\#390](https://github.com/NASA-PDS/registry/issues/390)
+- Document password rotation requirements [\#320](https://github.com/NASA-PDS/registry/issues/320)
 - Update registry documentation [\#288](https://github.com/NASA-PDS/registry/issues/288)
 - Respond to User Questions Regarding Supporting Migration to OpenSearch Serverless [\#275](https://github.com/NASA-PDS/registry/issues/275)
 
 ## [v1.4.0](https://github.com/NASA-PDS/registry/tree/v1.4.0) (2025-07-16)
 
-[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.3.1...v1.4.0)
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/release/1.4.0...v1.4.0)
+
+## [release/1.4.0](https://github.com/NASA-PDS/registry/tree/release/1.4.0) (2025-07-16)
+
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.3.1...release/1.4.0)
 
 **Requirements:**
 
@@ -108,7 +122,11 @@
 
 ## [v1.3.1](https://github.com/NASA-PDS/registry/tree/v1.3.1) (2023-11-16)
 
-[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.3.0...v1.3.1)
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/release/1.3.1...v1.3.1)
+
+## [release/1.3.1](https://github.com/NASA-PDS/registry/tree/release/1.3.1) (2023-11-16)
+
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.3.0...release/1.3.1)
 
 **Requirements:**
 
@@ -237,18 +255,11 @@
 
 ## [v1.0.0](https://github.com/NASA-PDS/registry/tree/v1.0.0) (2022-05-03)
 
-[Full Changelog](https://github.com/NASA-PDS/registry/compare/v1.0.0-SNAPSHOT...v1.0.0)
+[Full Changelog](https://github.com/NASA-PDS/registry/compare/1f4d45e5a395b8d05b58a0924066dcd3cd0b3565...v1.0.0)
 
 **Requirements:**
 
 - Upgrade the initialization of the ES/OS database following latest upgrade of components [\#31](https://github.com/NASA-PDS/registry/issues/31)
-
-## [v1.0.0-SNAPSHOT](https://github.com/NASA-PDS/registry/tree/v1.0.0-SNAPSHOT) (2022-01-20)
-
-[Full Changelog](https://github.com/NASA-PDS/registry/compare/1f4d45e5a395b8d05b58a0924066dcd3cd0b3565...v1.0.0-SNAPSHOT)
-
-**Requirements:**
-
 - As a developer, I want to include supplemental file data sizes in the registry [\#112](https://github.com/NASA-PDS/registry/issues/112)
 - As a node operator, I want to ingest metadata regarding secondary collections that belong to a bundle. [\#109](https://github.com/NASA-PDS/registry/issues/109)
 - As a node operator, I want to ingest metadata regarding secondary products that belong to a collection. [\#108](https://github.com/NASA-PDS/registry/issues/108)

README.md

@@ -19,13 +19,43 @@ This repository specifically contains these PDS registry application resources:
 - the source for the user/administrator documentation, see `docs` folder
 - docker compose script to start/test the full application with its required components, see https://github.com/NASA-PDS/registry/blob/main/docker/README.md for details. This also includes integration tests (in `docker/postman`). The integration test datasets are maintained in NASA-PDS/registry-ref-data repository.
 - **For developers:** See [Integration Testing Guide](https://nasa-pds.github.io/registry/developer/integration-testing.html) for instructions on adding and running integration tests.
+- utility scripts for registry management and reporting, see `scripts` folder
 
 
 ## Code of Conduct
 
 All users and developers of the NASA-PDS software are expected to abide by our [Code of Conduct](https://github.com/NASA-PDS/.github/blob/main/CODE_OF_CONDUCT.md). Please read this to ensure you understand the expectations of our community.
 
 
+## Scripts
+
+### Registry Status Report Generator
+
+Generate CSV reports for missing and staged products in the PDS Registry:
+
+```bash
+# Generate reports and commit/push to GitHub (default)
+./scripts/generate_registry_status_reports.py
+
+# Generate reports only, without committing
+./scripts/generate_registry_status_reports.py --no-commit
+```
+
+**Purpose:** Generates reports for missing and staged products in the registry.
+
+**Output:** Creates four CSV files in `docs/status/`:
+- `missing_bundles_in_registry.csv` - Missing Product_Bundle records
+- `missing_collections_in_registry.csv` - Missing Product_Collection records
+- `staged_bundles_in_registry.csv` - Staged Product_Bundle records
+- `staged_collections_in_registry.csv` - Staged Product_Collection records
+
+By default, the script commits and pushes these files to GitHub. Use `--no-commit` to disable this.
+
+**Requirements:**
+- Python 3.12 or higher
+- `pds-registry-client` and PDS Registry credentials
+- See `scripts/README.md` for full configuration details
+
 ## Utilities
 * Treks
     * To deploy the package run one of these commands from the root directory:

conf/status/missing_bundles_per_node.json

@@ -0,0 +1,21 @@
+
+{
+  "_source": ["node", "lidvid", "product_class"],
+  "size": 2000,
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Bundle"
+          }
+        },
+        {
+          "match_phrase": {
+            "found_in_registry": "false"
+          }
+        }
+      ]
+    }
+  }
+}

conf/status/missing_collections_per_node.json

@@ -0,0 +1,21 @@
+
+{
+  "_source": ["node", "lidvid", "product_class"],
+  "size": 2000,
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Collection"
+          }
+        },
+        {
+          "match_phrase": {
+            "found_in_registry": "false"
+          }
+        }
+      ]
+    }
+  }
+}

conf/status/staged_bundles_per_node.json

@@ -0,0 +1,21 @@
+
+{
+  "_source": ["ops:Harvest_Info/ops:node_name", "lidvid", "product_class", "ops:Harvest_Info/ops:harvest_date_time"],
+  "size": 2000,
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Bundle"
+          }
+        },
+        {
+          "match_phrase": {
+            "ops:Tracking_Meta/ops:archive_status": "staged"
+          }
+        }
+      ]
+    }
+  }
+}

conf/status/staged_collections_per_node.json

@@ -0,0 +1,21 @@
+
+{
+  "_source": ["ops:Harvest_Info/ops:node_name", "lidvid", "product_class", "ops:Harvest_Info/ops:harvest_date_time"],
+  "size": 2000,
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Collection"
+          }
+        },
+        {
+          "match_phrase": {
+            "ops:Tracking_Meta/ops:archive_status": "staged"
+          }
+        }
+      ]
+    }
+  }
+}

docker/.env

@@ -22,7 +22,7 @@ ES_DISCOVERY_TYPE=single-node
 # --------------------------------------------------------------------
 
 # Docker image of the Registry API
-REG_API_IMAGE=nasapds/registry-api-service:1.6.2
+REG_API_IMAGE=nasapds/registry-api-service:develop
 
 # Absolute path of the application.properties file to be used for the Registry API
 REG_API_APP_PROPERTIES_FILE=./default-config/application.properties
@@ -32,7 +32,7 @@ REG_API_APP_PROPERTIES_FILE=./default-config/application.properties
 # --------------------------------------------------------------------
 
 # Docker image of the Registry Loader
-REG_LOADER_IMAGE=nasapds/registry-loader:1.1.1
+REG_LOADER_IMAGE=nasapds/registry-loader:latest
 
 # --------------------------------------------------------------------
 # Registry Sweepers
@@ -73,35 +73,6 @@ TEST_DATA_URL=https://github.com/NASA-PDS/registry-ref-data/releases/download/La
 # The lidvid of the test data, which is used to set the archive status (only required, if executing with test data)
 TEST_DATA_LIDVID=urn:nasa:pds:mars2020.spice::1.0 urn:nasa:pds:mars2020.spice::2.0 urn:nasa:pds:mars2020.spice::3.0
 
-# --------------------------------------------------------------------
-# Registry Harvest Service
-# --------------------------------------------------------------------
-
-# Docker image of the Registry Harvest Service
-REGISTRY_HARVEST_SERVICE_IMAGE=nasapds/registry-harvest-service:1.2.1
-
-# Absolute path of the Registry Harvest Service configuration file in the host machine (E.g.: /tmp/cfg/harvest-server.cfg)
-HARVEST_SERVER_CONFIG_FILE=./default-config/harvest-server.cfg
-
-# --------------------------------------------------------------------
-# Registry Crawler Service
-# --------------------------------------------------------------------
-
-# Docker image of the Registry Crawler Service
-REGISTRY_CRAWLER_SERVICE_IMAGE=nasapds/registry-crawler-service:1.1.0
-
-# Absolute path of the Registry Crawler Service configuration file in the host machine (E.g.: /tmp/cfg/crawler-server.cfg)
-CRAWLER_SERVER_CONFIG_FILE=./default-config/crawler-server.cfg
-
-# --------------------------------------------------------------------
-# Registry Harvest CLI
-# --------------------------------------------------------------------
-
-# Docker image of the Registry Harvest CLI
-REGISTRY_HARVEST_CLI_IMAGE=nasapds/registry-harvest-cli:1.1.0
-
-# Absolute path of the Registry Harvest CLI configuration file in the host machine (E.g.: /tmp/conf/harvest-client.cfg)
-HARVEST_CLIENT_CONFIG_FILE=./default-config/harvest-client.cfg
 
 # --------------------------------------------------------------------
 # Postman Collection Test

docker/docker-compose.yml

@@ -85,6 +85,10 @@ services:
   elasticsearch:
     profiles: ["dev-api", "dev-loader", "pds-core-registry", "int-registry-batch-loader", "os"]
     image: ${ES_IMAGE}
+    ulimits:
+      nofile:
+        soft: 65536
+        hard: 65536
     environment:
       - discovery.type=${ES_DISCOVERY_TYPE}
     # Uncomment the following environment variable to use OpenSearch without security