Lutaml integration

.gitignore

@@ -13,3 +13,4 @@
 # rspec failure tracking
 .rspec_status
 Gemfile.lock
+.claude/

.rubocop.yml

@@ -7,6 +7,6 @@ require: rubocop-rails
 inherit_from:
   - https://raw.githubusercontent.com/riboseinc/oss-guides/master/ci/rubocop.yml
 AllCops:
-  TargetRubyVersion: 2.7
+  TargetRubyVersion: 3.2
 Rails:
   Enabled: false

CLAUDE.md

@@ -0,0 +1,72 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+relaton-itu is a Ruby gem for retrieving ITU (International Telecommunication Union) standards metadata. Part of the Relaton family of gems maintained by Ribose Inc.
+
+## Commands
+
+```bash
+bundle exec rake spec          # Run full test suite
+bundle exec rspec spec/relaton/itu/              # Run new-namespace tests only
+bundle exec rspec spec/relaton/itu/item_spec.rb  # Run a single spec file
+bundle exec rspec spec/relaton/itu/item_spec.rb:15  # Run a specific test by line
+bin/console                    # Interactive Ruby console with gem loaded
+```
+
+No separate lint command is configured; RuboCop can be run via `bundle exec rubocop`.
+
+## Architecture
+
+### Namespace Migration (In Progress)
+
+The codebase is migrating from flat `RelatonItu` namespace (`lib/relaton_itu/`) to nested `Relaton::Itu` (`lib/relaton/itu/`). Both namespaces coexist:
+
+- **`lib/relaton/itu/`** — New namespace. Model classes, DataFetcher, DataParserR, Processor, Util, Version are here.
+- **`lib/relaton_itu/`** — Old namespace. ItuBibliography, XMLParser, ItuBibliographicItem, HitCollection, and others still live here.
+
+The `Processor` class (`Relaton::Itu::Processor`) bridges both: it lives in the new namespace but calls old-namespace classes (`::RelatonItu::ItuBibliography`, `::RelatonItu::XMLParser`, etc.) for functionality not yet migrated.
+
+### Model Layer (Lutaml::Model)
+
+All model classes use `Lutaml::Model::Serializable` for XML/YAML serialization:
+
+- **`Item`** → extends `Bib::Item` (main bibliographic item)
+- **`ItemData`** → extends `Bib::ItemData` (used by DataParserR for parsed documents)
+- **`Bibitem`** / **`Bibdata`** → extend `Item`, mix in shared behavior from `Bib`
+- **`Ext`** → extends `Bib::Ext` with ITU-specific fields (doctype, structuredidentifier, question, recommendationstatus, ip_notice_received, meeting, meeting_place, meeting_date, intended_type, source)
+- **`Doctype`**, **`StructuredIdentifier`**, **`EditorialGroup`**, **`Bureau`**, **`Group`**, **`ApprovalStage`**, **`RecommendationStatus`**, **`Question`**, **`Meeting`**, **`MeetingDate`** — ITU-specific metadata types
+
+### Data Fetching
+
+- **`DataFetcher`** extends `Core::DataFetcher` — orchestrates fetching ITU-R documents from `extranet.itu.int`
+- **`DataParserR`** — module that parses ITU-R JSON search API results into `ItemData` instances (sets `flavor: "itu"` on all parsed documents)
+- Sources: recommendations (JSON index), questions, reports, handbooks, resolutions (HTML indices)
+
+### Processor
+
+`Relaton::Itu::Processor` extends `Relaton::Core::Processor` and is the entry point for the Relaton plugin system. Provides `get`, `fetch_data`, `from_xml`, `hash_to_bib`, `grammar_hash`, and `remove_index_file`.
+
+## Testing
+
+- **Index fixture:** `spec/fixtures/index-v1.zip` is pre-loaded into `Relaton::Index` pool in `before(:suite)` (configured in `spec/support/webmock.rb`). Run `rake spec:update_index` to refresh from relaton-data-itu-r.
+- **Framework:** RSpec with VCR cassettes for HTTP mocking and WebMock
+- **Fixtures:** `spec/fixtures/` contains sample YAML/XML documents for round-trip tests
+- **VCR cassettes:** `spec/vcr_cassettes/` — 22 cassettes recording real HTTP responses
+- **Coverage:** SimpleCov, target near 100%
+
+Round-trip tests (serialize → deserialize → compare) are the primary pattern for model classes.
+
+## Key Dependencies
+
+- `relaton-bib` — base bibliographic model classes (`Bib::Item`, `Bib::Ext`, etc.)
+- `relaton-core` — `Core::Processor`, `Core::DataFetcher` base classes
+- `lutaml-model` — serialization framework (XML/YAML mapping via `Lutaml::Model::Serializable`)
+- `mechanize` — web scraping for data fetching
+- `relaton-index` — document indexing
+
+## Ruby Version
+
+Requires Ruby >= 3.1.0.

README.adoc

@@ -1,4 +1,4 @@
-= RelatonItu: retrieve ITU Standards for bibliographic use using the BibliographicItem model
+= Relaton::Itu: retrieve ITU Standards for bibliographic use using the BibliographicItem model
 
 image:https://img.shields.io/gem/v/relaton-itu.svg["Gem Version", link="https://rubygems.org/gems/relaton-itu"]
 image:https://github.com/relaton/relaton-itu/workflows/macos/badge.svg["Build Status (macOS)", link="https://github.com/relaton/relaton-itu/actions?workflow=macos"]
@@ -16,7 +16,7 @@ Add this line to your application's Gemfile:
 
 [source,ruby]
 ----
-gem 'relaton-bib'
+gem 'relaton-itu'
 ----
 
 And then execute:
@@ -25,129 +25,150 @@ And then execute:
 
 Or install it yourself as:
 
-    $ gem install relaton-bib
+    $ gem install relaton-itu
 
 == Usage
 
 === Search for a standard using keywords
 
 [source,ruby]
 ----
-require 'relaton_itu'
+require 'relaton/itu'
 => true
 
-hit_collection = RelatonItu::ItuBibliography.search("ITU-T L.163")
-=> <RelatonItu::HitCollection:0x007fbf5fb4ab60 @ref=ITU-T L.163 @fetched=false>
+hit_collection = Relaton::Itu::Bibliography.search("ITU-T L.163")
+=> <Relaton::Itu::HitCollection:0x00000000001700 @ref=ITU-T L.163 @fetched=false>
 
-item = hit_collection[1].fetch
-=> #<RelatonItu::ItuBibliographicItem:0x007fbf1fc89548
+item = hit_collection[1].item
+=> #<Relaton::Itu::ItemData:0x000000012541b8f8
  ...
 ----
 
 === XML serialization
+
 [source,ruby]
 ----
 item.to_xml
-=> "<bibitem id="ITU-TLSuppl.39" type="standard" schema-version="v1.2.1">
-      <fetched>2022-12-05</fetched>
-      <title type="title-intro" format="text/plain" language="en" script="Latn">ITU-T L.1470</title>
+=> "<bibitem id="ITUTLSuppl39092020" type="standard" schema-version="v1.4.1">
+      <fetched>2026-02-24</fetched>
+      <title language="en" script="Latn" type="title-intro">ITU-T L.1470</title>
+      <title language="en" script="Latn" type="title-main">Optical fibre cable Recommendations and standardization guideline</title>
+      <title language="en" script="Latn" type="main">ITU-T L.1470 - Optical fibre cable Recommendations and standardization guideline</title>
+      <uri type="src">https://handle.itu.int/11.1002/1000/14547</uri>
+      <uri type="pdf">https://www.itu.int/rec/dologin_pub.asp?lang=e&amp;id=T-REC-L.Sup39-202009-I!!PDF-E&amp;type=items</uri>
+      <docidentifier type="ITU" primary="true">ITU-T L Suppl. 39 (09/2020)</docidentifier>
       ...
     </bibitem>"
 ----
+
 With `bibdata: true` option XML output is wrapped with `bibdata` element and `ext` element added.
+
 [source,ruby]
 ----
 item.to_xml bibdata: true
-=> "<bibdata type="standard" schema-version="v1.2.1">
-      <fetched>2022-12-05</fetched>
-      <title type="title-intro" format="text/plain" language="en" script="Latn">ITU-T L.1470</title>
+=> "<bibdata type="standard" schema-version="v1.4.1">
+      <fetched>2026-02-24</fetched>
+      <title language="en" script="Latn" type="title-intro">ITU-T L.1470</title>
+      <title language="en" script="Latn" type="title-main">Optical fibre cable Recommendations and standardization guideline</title>
+      <title language="en" script="Latn" type="main">ITU-T L.1470 - Optical fibre cable Recommendations and standardization guideline</title>
+      <uri type="src">https://handle.itu.int/11.1002/1000/14547</uri>
+      <uri type="pdf">https://www.itu.int/rec/dologin_pub.asp?lang=e&amp;id=T-REC-L.Sup39-202009-I!!PDF-E&amp;type=items</uri>
+      <docidentifier type="ITU" primary="true">ITU-T L Suppl. 39 (09/2020)</docidentifier>
       ...
-      <ext schema-version="v1.0.0">
+      <ext schema-version="v1.0.2">
         <doctype>recommendation</doctype>
-        ...
+        <flavor>itu</flavor>
       </ext>
     </bibdata>"
 ----
 
 === Get document by code and year
 [source,ruby]
 ----
-RelatonItu::ItuBibliography.get("ITU-T L.163", "2018", {})
-[relaton-itu] (ITU-T L.163) Fetching from www.itu.int ...
-[relaton-itu] (ITU-T L.163) Found: `ITU-T L.163`
-=> #<RelatonItu::ItuBibliographicItem:0x007fbf5f94b9b8
+Relaton::Itu::Bibliography.get("ITU-T L.163", "2018", {})
+[relaton-itu] INFO: (ITU-T L.163 (2018)) Fetching from www.itu.int ...
+[relaton-itu] INFO: (ITU-T L.163 (2018)) Found: `ITU-T L.163 (11/2018)`
+=> #<Relaton::Itu::ItemData:0x000000010f92a288
 ...
 ----
 
 === Get amendment
 [source,ruby]
 ----
-RelatonItu::ItuBibliography.get "ITU-T G.989.2/Amd 1"
-[relaton-itu] (ITU-T G.989.2/Amd 1) Fetching from www.itu.int ...
-[relaton-itu] (ITU-T G.989.2/Amd 1) Found: `ITU-T G.989.2/Amd 1`
-=> #<RelatonItu::ItuBibliographicItem:0x007fe5409e5840
+Relaton::Itu::Bibliography.get "ITU-T G.989.2 (2019)/Amd 1"
+[relaton-itu] INFO: (ITU-T G.989.2 (2019)) Fetching from www.itu.int ...
+[relaton-itu] INFO: (ITU-T G.989.2 (2019)) Found: `ITU-T G.989.2 (02/2019)`
+=> #<Relaton::Itu::ItemData:0x000000010fa4f3c0
 ...
 ----
 
 === Get ITU-R documents
 
 [source,ruby]
 ----
-RelatonItu::ItuBibliography.get "ITU-R BO.600-1"
-[relaton-itu] (ITU-R BO.600-1) Fetching from www.itu.int ...
-[relaton-itu] (ITU-R BO.600-1) Found: `ITU-R BO.600-1`
-=> #<RelatonItu::ItuBibliographicItem:0x00007fb0cf8a1308
+Relaton::Itu::Bibliography.get "ITU-R BO.600-1"
+[relaton-itu] INFO: (ITU-R BO.600-1) Fetching from Relaton repository ...
+[relaton-itu] INFO: (ITU-R BO.600-1) Found: `ITU-R BO.600-1`
+=> #<Relaton::Itu::ItemData:0x000000010fa4a3c0
 ...
 ----
 
 === Get ITU-R Radio Regulations (RR)
 
 [source,ruby]
 ----
-RelatonItu::ItuBibliography.get 'ITU-R RR (2020)'
-[relaton-itu] (ITU-R RR) Fetching from www.itu.int ...
-[relaton-itu] WARNING: Invalid doctype: `publication`
-[relaton-itu] (ITU-R RR) Found: ITU-R RR
-=> #<RelatonItu::ItuBibliographicItem:0x007fb4b55eb660
+Relaton::Itu::Bibliography.get 'ITU-R RR (2020)'
+[relaton-itu] INFO: (ITU-R RR (2020)) Fetching from www.itu.int ...
+[relaton-itu] INFO: (ITU-R RR (2020)) Found: `ITU-R RR (2020)`
+=> #<Relaton::Itu::ItemData:0x000000010fa41e00
 ...
 ----
 
-=== Typed links
+=== Typed source links
 
-Each ITU document has `src` type link and optional `obp` and `word`  link types.
+Each ITU document has `src` type link and optional `obp` and `word` link types.
 
 [source,ruby]
 ----
-item.link
-=> [#<RelatonBib::TypedUri:0x00007f82d665f530 @content=#<Addressable::URI:0xc170 URI:https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=13786&lang=en>, @type="src">,
- #<RelatonBib::TypedUri:0x00007f82d665f0f8 @content=#<Addressable::URI:0xc184 URI:https://www.itu.inthttp//handle.itu.int/11.1002/1000/13786-en?locatt=format:pdf&auth>, @type="obp">]
+item.source[0].type
+=> "src"
+
+item.source[0].content
+=> "https://handle.itu.int/11.1002/1000/14547"
 ----
 
-=== Create a bibliographic item from YAML
+=== Create a bibliographic item from XML
+
 [source,ruby]
 ----
-hash = YAML.load_file 'spec/examples/itu_bib_item.yml'
-=> {"id"=>"ITU-T L.163 (11/2018)",
+item = Relaton::Itu::Item.from_xml File.read("spec/fixtures/bibdata.xml")
+=> #<Relaton::Itu::ItemData:0x000000010fa28ea0
 ...
+----
+
+=== Create a bibliographic item from YAML
 
-RelatonItu::ItuBibliographicItem.from_hash hash
-=> #<RelatonItu::ItuBibliographicItem:0x007fd88ac02aa0
+[source,ruby]
+----
+item = Relaton::Itu::Item.from_yaml File.read("spec/fixtures/item.yaml")
+=> #<Relaton::Itu::ItemData:0x000000010f9848a0
 ...
 ----
 
 === Fetch data
 
-This gem uses the (https://extranet.itu.int/brdocsearch) dataset as a data source. +
+This gem uses the https://extranet.itu.int/brdocsearch dataset as a data source.
 
-The method `RelatonItu::DataFetcher.fetch(output: "data", format: "yaml")` fetches all the documents from the dataset and saves them to the `./data` folder in YAML format.
+The method `Relaton::Itu::DataFetcher.fetch("itu-r", output: "data", format: "yaml")` fetches all the documents from the dataset and saves them to the `./data` folder in YAML format.
 Arguments:
 
-- `output` - folder to save documents (default 'data').
+- `source` - source name (`itu-r`).
+- `output` - folder to save documents (default `data`).
 - `format` - the format in which the documents are saved. Possible formats are: `yaml`, `xml`, `bibxxml` (default `yaml`).
 
 [source,ruby]
 ----
-RelatonItu::DataFetcher.fetch output: "dir", format: "xml"
+Relaton::Itu::DataFetcher.fetch "itu-r", output: "dir", format: "xml"
 Started at: 2023-05-27 09:21:16 -0400
 Stopped at: 2023-05-27 09:27:45 -0400
 Done in: 390 sec.
@@ -162,11 +183,13 @@ RelatonItu uses the relaton-logger gem for logging. By default, it logs to STDOU
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
+To update the index test fixture (used by tests), run `rake spec:update_index`. This downloads the latest `index-v1.zip` from the https://github.com/relaton/relaton-data-itu-r[relaton-data-itu-r] repository.
+
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 == Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/metanorma/relaton-bib.
+Bug reports and pull requests are welcome on GitHub at https://github.com/relaton/relaton-itu.
 
 == License
 

Rakefile

@@ -4,3 +4,25 @@ require "rspec/core/rake_task"
 RSpec::Core::RakeTask.new(:spec)
 
 task :default => :spec
+
+namespace :spec do
+  desc "Download latest ITU index fixture from relaton-data-itu-r"
+  task :update_index do
+    require "net/http"
+    require "uri"
+
+    url = "https://raw.githubusercontent.com/relaton/relaton-data-itu-r/data-v2/index-v1.zip"
+    dest = File.join(__dir__, "spec", "fixtures", "index-v1.zip")
+
+    puts "Downloading \#{url} ..."
+    uri = URI.parse(url)
+    response = Net::HTTP.get_response(uri)
+
+    if response.is_a?(Net::HTTPSuccess)
+      File.binwrite(dest, response.body)
+      puts "Updated \#{dest} (\#{response.body.bytesize} bytes)"
+    else
+      abort "Failed to download: HTTP \#{response.code}"
+    end
+  end
+end