Conversation
- Configuration holds index_prefix, batch_size, auto_indexing, async_indexing, and index_job_class settings with sensible defaults - Registry is a thread-safe Singleton for tracking model-to-index mappings with register, find_by_class, find_by_table, all, and reset! - Add rails-specific spec_helper that loads without ActiveRecord - Full spec coverage for both classes (20 examples, 0 failures) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement the core index definition classes for Manticore::Rails: - Index: holds fields, attributes, properties, and associations for a model - Field: represents a full-text search field with SQL, association, and method detection - Attribute: extends Field with type mapping (integer->bigint, datetime->timestamp, etc.) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Create lib/manticore/rails.rb with configuration, registry, and auto_indexing helpers plus dedicated Zeitwerk loader - Create lib/manticore/rails/railtie.rb to load rake tasks in Rails - Update lib/manticore/client.rb to ignore rails files from main loader Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- AssociationProxy captures method_missing chains (tags.name → "tags.name") and method calls with args (transcript(:approved_text) → "transcript(approved_text)") - IndexBuilder provides indexes, has, reindex_on_change, set_property DSL methods evaluated via instance_eval, building Index objects with Fields and Attributes - Handles all legacy patterns: bare symbols, association proxies, SQL strings, and method calls with arguments - 15 specs covering all DSL patterns including full Episode-style block Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Schema: generates CREATE/DROP TABLE DDL for ManticoreSearch, executes via UtilsApi
- Indexer: serializes AR records to documents, bulk upsert via IndexApi.bulk NDJSON
- Searcher/Result: lazy-loading search with pagination, filter translation, ordering
- Searchable: ActiveRecord concern with define_manticore_index DSL and callbacks
- Rake tasks: manticore:schema:{create,drop,rebuild} and manticore:index:rebuild
- Fix Field#sql? to detect CAST, IF, CONVERT and other SQL function patterns
- All 141 specs passing
- Define Manticore module before Zeitwerk loader references it - Parse version from file in gemspec instead of requiring version.rb, avoiding Manticore module definition at gem specification time - WebMock detects `defined?(Manticore)` and tries to load its JRuby Manticore HTTP client adapter, which fails with our gem
…core-rails/ - Rename ManticoreSearch -> ManticoreClient across all files to avoid namespace collision with JRuby's Manticore HTTP client gem - Move Rails DSL layer from lib/manticore/rails/ to lib/manticore-rails/ for cleaner separation (require "manticore-rails") - Fix Zeitwerk loader push_dir namespace references
- Add manticore:setup task (drop + create + populate all indexes) - Indexer#reindex_all now includes associations to avoid N+1 - Indexer#reindex_all yields progress count for reporting - Rake tasks eager-load models so registry is populated - Fix railtie rake task path after lib/manticore-rails/ move - Auto-require railtie when Rails is present
Rename ManticoreClient::Rails namespace to top-level ManticoreRails and add manticore-rails.gemspec alongside manticore-client.gemspec. Two gems, one repo — the Rails layer depends on manticore-client ~> 1.0. Adds version.rb, standalone README-rails.md, and fixes indentation issues introduced during the namespace rename.
Replace N individual API calls with one bulk request, matching the pattern already used by bulk_replace.
The manticore:schema:create and manticore:schema:drop tasks called each_index which was never defined, causing NoMethodError at runtime.
Accepts both string and class for index_job_class config, using ActiveSupport's constantize instead of the unrestricted Object.const_get.
Previously mutated shared Configuration state which affected all threads in multi-threaded servers like Puma. Now uses thread-local flag so only the current thread is affected.
Move ManticoreRails::Result to ManticoreRails::Searcher::Result for clearer ownership. Result is the search subsystem's concern and should not be a top-level class in the ManticoreRails namespace.
association_name, column_name, and association_path all called
column.to_s.split('.') independently. Extract to a single memoized
private method.
Property values containing single quotes would produce broken SQL. Use SQL-standard escaping (double single quotes) for safety.
Avoid creating a new IndexApi object on every bulk_replace and delete_records call. The client is stateless and safe to reuse.
Results from ManticoreSearch are ranked by relevance, but WHERE IN (...) does not preserve that order. Use index_by and filter_map to reorder loaded records to match the original search ranking. Missing records are silently skipped.
Prevent infinite recursion when circular associations are encountered during field value extraction. Raises an error if association depth exceeds 10 levels.
Property keys are now validated against /\A[a-z_][a-z0-9_]*\z/i to ensure only safe identifiers are used in CREATE TABLE DDL. Values were already escaped; this closes the key injection vector.
Use Mutex to synchronize lazy initialization of @@default class variables in both ApiClient and Configuration, preventing race conditions when multiple threads access them concurrently.
…piError Only set known attributes (code, response_headers, response_body) instead of blindly setting any key from the options hash as an instance variable.
batch_size must be positive. index_prefix only accepts alphanumeric characters and underscores to prevent injection in table names.
Maintain a secondary @by_table hash index alongside @indexes to avoid linear scan when looking up indexes by table name.
Replace case/when with a ternary since all non-Symbol branches produce the same result.
Add on_error callback to Configuration that defaults to warn. Users can now customize error handling for indexing failures (e.g. report to error tracking services, raise in test).
Passing nil filters through as-is instead of silently converting to 0, which would incorrectly match records with value 0.
When wrapping the base query with filters in a bool.must clause, only pass the relevant query field to QueryFilter to avoid ambiguous query semantics.
Use MANTICORESEARCH_URI consistently across all workflows. The release workflow was using MANTICORE_URL which the test suite does not recognize.
Report failed batches via on_error callback and continue reindexing remaining batches instead of aborting the entire operation.
Emit search.manticore_rails, bulk.manticore_rails, and delete.manticore_rails events for monitoring search latency and indexing throughput in production.
Creates a config/initializers/manticore.rb template with all configuration options and sensible defaults.
When a subclass doesn't have its own index, find_by_class now checks ancestor classes, enabling STI models to inherit the parent's search index.
…tation Cache the constantized job class in Configuration to avoid repeated string-to-class resolution on every after_commit callback. Add a note about zero-downtime reindex limitations in the README.
Check circuit_open? in manticore_should_index? to skip indexing when ManticoreSearch is unresponsive. Call record_success!/record_failure! after each index/remove operation to track consecutive failures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Report via on_error when some requested IDs are not found in the database, helping diagnose stale callbacks or race conditions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevent accidental mutation of Index objects once they are registered in the Registry. Adding fields, attributes, associations, or properties after registration raises FrozenError. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Support must_not filters via without: option, mirroring the existing with: inclusion filters. Can be combined for complex queries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Setting on_error to :raise converts it to a lambda that re-raises the original error, making indexing failures visible in development and test environments instead of silently logging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update README with instrumentation events table, circuit breaker section, without: exclusion filter example, and :raise mode for on_error. Update CHANGELOG with all new features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Protect @failure_count with @circuit_mutex to prevent races in multi-threaded servers like Puma. Initialize both at module level. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Allows tests to restore all configuration defaults with a single call instead of manually resetting individual attributes in ensure blocks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Convert doubles for BulkResponse, SearchResponse, SearchResponseHits, and HitsHits to instance_double to catch API contract drift. Use real Index objects in registry specs. Re-enable RSpec/VerifiedDoubles for specs that don't mock ActiveRecord internals. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Memoize referenced_associations on Index since the object is frozen after registration. Remove Attribute#sortable? which was defined but never called anywhere in the codebase. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cover the SQL error raise path in Schema.execute and add a test for single-quote escaping in property values. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add commented circuit_breaker_threshold option and enable on_error :raise mode for local development environments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provides a clear intent method name for manually resetting the circuit breaker, e.g. after ManticoreSearch recovers from an outage. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix generator template: use Rails.env.production? instead of Rails.env.local? which requires Rails 7.1+ - Remove unused Field#method_call? method - Remove exposed attr_readers from IndexBuilder (internal only) - Complete CHANGELOG with all features added in 0.1.0 - Document rails generator in Installation section - Document reset_circuit! in Circuit breaker section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The mva/multi type was defined in MANTICORE_TYPE_MAP but no user-facing type in Attribute::TYPE_MAP maps to :mva, making it unreachable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…SQL responses - Move record_success! inside sync block so async enqueue does not falsely reset the circuit breaker - Handle beginless/endless Ruby ranges in search filters instead of sending nil bounds to ManticoreSearch - Guard Schema.execute against empty/nil API responses Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ering Reset circuit breaker in before/after hooks to prevent failure count pollution across test groups. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When using manticore_serialize, associations accessed in custom
serialization aren't auto-detected by referenced_associations.
The new includes directive lets models declare extra associations
to eager-load during batch reindex, eliminating N+1 queries.
define_manticore_index do
indexes :name
includes :transcripts, anchors: :dvags
end
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.