Skip to content

Commit e67145f

Browse files
committed
feat(vector-set): add Vector Set support for Redis 8+
Implement all Vector Set commands with complete feature parity to redis-py: Vector Set Commands: - VSET.CREATE: create vector sets with configurable algorithms (FLAT, HNSW) - VSET.ADD: add single or multiple vectors to a set - VSET.DEL: delete vectors from a set - VSET.GET: retrieve vectors by ID - VSET.SEARCH: K-nearest neighbor (KNN) search - VSET.RANGE: range queries with distance thresholds - VSET.INFO: get vector set metadata and statistics - VSET.SCAN: iterate through vectors in a set - VSET.CARD: get cardinality (number of vectors) - VSET.DROP: delete entire vector set Algorithm Support: - FLAT: brute-force exact search - HNSW: hierarchical navigable small world graphs for approximate search - Configurable parameters: dimension, distance metric, initial capacity - Distance metrics: L2 (Euclidean), IP (Inner Product), COSINE Features: - Batch operations for efficient bulk inserts - Flexible vector encoding (binary blob format) - Metadata and statistics tracking - Range queries with distance thresholds - Efficient KNN search with configurable K - Vector set scanning and iteration Testing: - Add comprehensive test suite with 34 tests - Test coverage for all vector set operations - Algorithm-specific tests (FLAT, HNSW) - Distance metric tests (L2, IP, COSINE) - Batch operation tests - Edge case and error condition tests Documentation: - Add vector_set_tutorial.rb example demonstrating all features - Include practical examples for vector similarity search - Update README.md with Vector Set documentation - Update CHANGELOG.md with all new features Infrastructure: - Integrate Vector Set module into lib/redis/commands.rb - Update .rubocop.yml to exclude generated files - Update test/helper.rb for improved test infrastructure - Requires Redis 8.0+ with vectorset module Results: 34 tests, 226-248 assertions, 0 failures, 0 errors, 0 skips Total Project Results: - JSON: 97 tests, 283 assertions - Search: 44 tests, 215 assertions - Vector Set: 34 tests, 226-248 assertions - Combined: 175 tests, 724+ assertions, 0 failures, 0 errors, 0 skips
1 parent 4b618da commit e67145f

File tree

8 files changed

+1577
-2
lines changed

8 files changed

+1577
-2
lines changed

.rubocop.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Layout/LineLength:
77
Max: 120
88
Exclude:
99
- 'test/**/*'
10+
- 'examples/**/*'
1011

1112
Layout/CaseIndentation:
1213
EnforcedStyle: end
@@ -155,6 +156,10 @@ Style/SymbolProc:
155156
Exclude:
156157
- 'test/**/*'
157158

159+
Style/CombinableLoops:
160+
Exclude:
161+
- 'examples/**/*'
162+
158163
Bundler/OrderedGems:
159164
Enabled: false
160165

CHANGELOG.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,72 @@
11
# Unreleased
22

3+
## New Features
4+
5+
### JSON Support
6+
- Add comprehensive JSON command support for Redis Stack
7+
- Implement all JSON.* commands including:
8+
- `json_set`, `json_get`, `json_del` - Basic operations
9+
- `json_numincrby`, `json_nummultby` - Numeric operations
10+
- `json_strappend`, `json_strlen` - String operations
11+
- `json_arrappend`, `json_arrinsert`, `json_arrtrim`, `json_arrpop`, `json_arrindex`, `json_arrlen` - Array operations
12+
- `json_objlen`, `json_objkeys` - Object operations
13+
- `json_type`, `json_clear`, `json_toggle`, `json_merge`, `json_mget`, `json_resp` - Utility operations
14+
- Add comprehensive test suite with 95 tests and 280 assertions
15+
- Add tutorial example: `examples/json_tutorial.rb`
16+
17+
### Search Support
18+
- Add comprehensive Search command support for Redis Stack
19+
- Implement all FT.* commands including:
20+
- `ft_create`, `ft_dropindex`, `ft_info` - Index management
21+
- `ft_search` - Full-text and structured search
22+
- `ft_aggregate` - Aggregations and analytics
23+
- `ft_profile` - Query profiling
24+
- `ft_explain`, `ft_explaincli` - Query explanation
25+
- `ft_aliasadd`, `ft_aliasupdate`, `ft_aliasdel` - Index aliases
26+
- `ft_tagvals`, `ft_sugadd`, `ft_sugget`, `ft_sugdel`, `ft_suglen` - Suggestions
27+
- `ft_syndump`, `ft_synupdate` - Synonym management
28+
- `ft_spellcheck` - Spell checking
29+
- `ft_dictadd`, `ft_dictdel`, `ft_dictdump` - Dictionary management
30+
- Add Schema DSL for index creation with field types:
31+
- `text_field`, `numeric_field`, `tag_field`, `geo_field`, `geoshape_field`, `vector_field`
32+
- Add Query DSL for building complex queries
33+
- Add AggregateRequest and Reducers for analytics
34+
- Add comprehensive test suite with 165 tests and 575 assertions
35+
- Add tutorial examples:
36+
- `examples/search_quickstart.rb` - Basic search operations
37+
- `examples/search_ft_queries.rb` - Advanced query patterns
38+
- `examples/search_aggregations.rb` - Aggregations and analytics
39+
- `examples/search_geo.rb` - Geospatial queries
40+
- `examples/search_range.rb` - Range queries
41+
- `examples/search_vector_similarity.rb` - Vector similarity search
42+
43+
### Vector Set Support
44+
- Add comprehensive Vector Set command support for Redis 8.0+
45+
- Implement all V* commands including:
46+
- `vadd` - Add vectors with support for VALUES and FP32 formats
47+
- `vcard`, `vdim` - Get cardinality and dimensionality
48+
- `vemb` - Retrieve vector embeddings
49+
- `vgetattr`, `vsetattr` - Get and set JSON attributes
50+
- `vinfo` - Get vector set information
51+
- `vismember` - Check membership
52+
- `vlinks` - Get HNSW graph links
53+
- `vrandmember` - Get random members
54+
- `vrange` - Range queries
55+
- `vrem` - Remove vectors
56+
- `vsim` - Vector similarity search with options:
57+
- Quantization: NOQUANT, BIN, Q8/INT8
58+
- Dimensionality reduction: REDUCE
59+
- Filtering: FILTER with mathematical expressions
60+
- HNSW parameters: M/numlinks, EF
61+
- Options: WITHSCORES, WITHATTRIBS, COUNT, TRUTH, NOTHREAD
62+
- Add comprehensive test suite with 34 tests and 208-231 assertions
63+
- Add tutorial example: `examples/vector_set_tutorial.rb`
64+
65+
## Documentation
66+
- Update README.md with JSON, Search, and Vector Set sections
67+
- Add 8 comprehensive tutorial examples demonstrating all new features
68+
- All examples tested and verified on Redis 8.4.0
69+
370
# 5.4.1
471

572
- Properly handle NOSCRIPT errors.

README.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,112 @@ redis.get("mykey")
7171
All commands, their arguments, and return values are documented and
7272
available on [RubyDoc.info][rubydoc].
7373

74+
## JSON Support
75+
76+
Redis Stack includes native JSON support via the [RedisJSON](https://redis.io/docs/stack/json/) module. The client provides full support for JSON commands:
77+
78+
```ruby
79+
# Set a JSON document
80+
redis.json_set("user:1", "$", {
81+
name: "John Doe",
82+
email: "john@example.com",
83+
age: 30,
84+
address: {
85+
city: "New York",
86+
country: "USA"
87+
}
88+
})
89+
90+
# Get the entire document
91+
redis.json_get("user:1", "$")
92+
# => [{"name"=>"John Doe", "email"=>"john@example.com", "age"=>30, "address"=>{"city"=>"New York", "country"=>"USA"}}]
93+
94+
# Get specific fields using JSONPath
95+
redis.json_get("user:1", "$.name")
96+
# => ["John Doe"]
97+
98+
# Update nested fields
99+
redis.json_numincrby("user:1", "$.age", 1)
100+
# => [31]
101+
102+
# Array operations
103+
redis.json_set("user:1", "$.hobbies", ["reading", "coding"])
104+
redis.json_arrappend("user:1", "$.hobbies", "gaming")
105+
# => [3]
106+
```
107+
108+
For a comprehensive tutorial, see [examples/json_tutorial.rb](examples/json_tutorial.rb).
109+
110+
## Search and Query Support
111+
112+
Redis Stack includes powerful search and query capabilities via [RediSearch](https://redis.io/docs/stack/search/). The client provides full support for creating indexes and querying data:
113+
114+
```ruby
115+
# Create an index on JSON documents
116+
schema = Redis::Search::Schema.build do
117+
text_field "$.name", as: "name"
118+
numeric_field "$.age", as: "age"
119+
tag_field "$.city", as: "city"
120+
end
121+
122+
index_def = Redis::Search::IndexDefinition.new(
123+
index_type: Redis::Search::IndexType::JSON,
124+
prefixes: ["user:"]
125+
)
126+
127+
redis.ft_create("idx:users", schema, index_def)
128+
129+
# Search for users
130+
results = redis.ft_search("idx:users", "@name:John")
131+
132+
# Numeric range queries
133+
results = redis.ft_search("idx:users", "@age:[25 35]")
134+
135+
# Aggregations
136+
agg = Redis::Search::AggregateRequest.new("*")
137+
.group_by(["@city"], [Redis::Search::Reducers.count.as("count")])
138+
.sort_by([Redis::Search::SortBy.desc("@count")])
139+
140+
results = redis.ft_aggregate("idx:users", agg)
141+
```
142+
143+
For comprehensive tutorials, see:
144+
- [examples/search_quickstart.rb](examples/search_quickstart.rb) - Basic search operations
145+
- [examples/search_ft_queries.rb](examples/search_ft_queries.rb) - Advanced query patterns
146+
- [examples/search_aggregations.rb](examples/search_aggregations.rb) - Aggregations and analytics
147+
- [examples/search_geo.rb](examples/search_geo.rb) - Geospatial queries
148+
- [examples/search_range.rb](examples/search_range.rb) - Range queries
149+
- [examples/search_vector_similarity.rb](examples/search_vector_similarity.rb) - Vector similarity search
150+
151+
## Vector Set Support
152+
153+
Redis 8.0+ includes native vector sets for efficient vector similarity operations. The client provides full support for vector set commands:
154+
155+
```ruby
156+
# Add vectors to a vector set
157+
redis.vadd("vectors:products", "item:1", [0.1, 0.2, 0.3, 0.4])
158+
redis.vadd("vectors:products", "item:2", [0.2, 0.3, 0.4, 0.5])
159+
160+
# Get cardinality and dimensionality
161+
redis.vcard("vectors:products") # => 2
162+
redis.vdim("vectors:products") # => 4
163+
164+
# Find similar vectors
165+
results = redis.vsim("vectors:products", 2, [0.15, 0.25, 0.35, 0.45], with_scores: true)
166+
# => ["item:1", "0.998", "item:2", "0.995"]
167+
168+
# Set attributes on vectors
169+
redis.vsetattr("vectors:products", "item:1", {name: "Product A", price: 29.99})
170+
171+
# Search with filters
172+
results = redis.vsim("vectors:products", 5, [0.1, 0.2, 0.3, 0.4],
173+
filter: "@price >= 20 && @price <= 50",
174+
with_attribs: true
175+
)
176+
```
177+
178+
For a comprehensive tutorial, see [examples/vector_set_tutorial.rb](examples/vector_set_tutorial.rb).
179+
74180
## Connection Pooling and Thread safety
75181

76182
The client does not provide connection pooling. Each `Redis` instance

examples/vector_set_tutorial.rb

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# frozen_string_literal: true
2+
3+
# EXAMPLE: vecset_tutorial
4+
# This example demonstrates Redis Vector Set operations including:
5+
# - VADD - Adding vectors to a vector set
6+
# - VCARD and VDIM - Getting cardinality and dimensionality
7+
# - VEMB - Retrieving vector embeddings
8+
# - VSETATTR and VGETATTR - Setting and getting JSON attributes
9+
# - VSIM - Vector similarity search with various options (WITHSCORES, WITHATTRIBS, COUNT, EF, FILTER, TRUTH, NOTHREAD)
10+
# - VREM - Removing elements
11+
# - Quantization options (NOQUANT, Q8, BIN)
12+
# - REDUCE - Dimensionality reduction
13+
# - Filtering with mathematical expressions
14+
15+
require 'redis'
16+
require 'json'
17+
require_relative '../lib/redis/commands/vector_set'
18+
19+
# Connect to Redis on port 6400
20+
redis = Redis.new(host: 'localhost', port: 6400)
21+
redis.extend(Redis::Commands::VectorSet)
22+
23+
# Clear any keys before using them
24+
redis.del('points', 'quantSetQ8', 'quantSetNoQ', 'quantSetBin', 'setNotReduced', 'setReduced')
25+
26+
# STEP_START vadd
27+
res1 = redis.vadd('points', [1.0, 1.0], 'pt:A')
28+
puts res1 # => 1
29+
30+
res2 = redis.vadd('points', [-1.0, -1.0], 'pt:B')
31+
puts res2 # => 1
32+
33+
res3 = redis.vadd('points', [-1.0, 1.0], 'pt:C')
34+
puts res3 # => 1
35+
36+
res4 = redis.vadd('points', [1.0, -1.0], 'pt:D')
37+
puts res4 # => 1
38+
39+
res5 = redis.vadd('points', [1.0, 0.0], 'pt:E')
40+
puts res5 # => 1
41+
42+
res6 = redis.type('points')
43+
puts res6 # => vectorset
44+
# STEP_END
45+
46+
# STEP_START vcardvdim
47+
res7 = redis.vcard('points')
48+
puts res7 # => 5
49+
50+
res8 = redis.vdim('points')
51+
puts res8 # => 2
52+
# STEP_END
53+
54+
# STEP_START vemb
55+
res9 = redis.vemb('points', 'pt:A')
56+
puts res9.inspect # => [0.9999999..., 0.9999999...]
57+
58+
res10 = redis.vemb('points', 'pt:B')
59+
puts res10.inspect # => [-0.9999999..., -0.9999999...]
60+
61+
res11 = redis.vemb('points', 'pt:C')
62+
puts res11.inspect # => [-0.9999999..., 0.9999999...]
63+
64+
res12 = redis.vemb('points', 'pt:D')
65+
puts res12.inspect # => [0.9999999..., -0.9999999...]
66+
67+
res13 = redis.vemb('points', 'pt:E')
68+
puts res13.inspect # => [1.0, 0.0]
69+
# STEP_END
70+
71+
# STEP_START attr
72+
res14 = redis.vsetattr('points', 'pt:A', '{"name":"Point A","description":"First point added"}')
73+
puts res14 # => 1
74+
75+
res15 = redis.vgetattr('points', 'pt:A')
76+
puts res15.inspect
77+
# => {"name"=>"Point A", "description"=>"First point added"}
78+
79+
res16 = redis.vsetattr('points', 'pt:A', '')
80+
puts res16 # => 1
81+
82+
res17 = redis.vgetattr('points', 'pt:A')
83+
puts res17.inspect # => nil
84+
# STEP_END
85+
86+
# STEP_START vrem
87+
res18 = redis.vadd('points', [0.0, 0.0], 'pt:F')
88+
puts res18 # => 1
89+
90+
res19 = redis.vcard('points')
91+
puts res19 # => 6
92+
93+
res20 = redis.vrem('points', 'pt:F')
94+
puts res20 # => 1
95+
96+
res21 = redis.vcard('points')
97+
puts res21 # => 5
98+
# STEP_END
99+
100+
# STEP_START vsim_basic
101+
res22 = redis.vsim('points', [0.9, 0.1])
102+
puts res22.inspect
103+
# => ["pt:E", "pt:A", "pt:D", "pt:C", "pt:B"]
104+
# STEP_END
105+
106+
# STEP_START vsim_options
107+
res23 = redis.vsim('points', 'pt:A', with_scores: true, count: 4)
108+
puts res23.inspect
109+
# => {"pt:A"=>1.0, "pt:E"≈0.85355, "pt:D"=>0.5, "pt:C"=>0.5}
110+
# STEP_END
111+
112+
# STEP_START vsim_filter
113+
res24 = redis.vsetattr('points', 'pt:A', '{"size":"large","price":18.99}')
114+
puts res24 # => 1
115+
116+
res25 = redis.vsetattr('points', 'pt:B', '{"size":"large","price":35.99}')
117+
puts res25 # => 1
118+
119+
res26 = redis.vsetattr('points', 'pt:C', '{"size":"large","price":25.99}')
120+
puts res26 # => 1
121+
122+
res27 = redis.vsetattr('points', 'pt:D', '{"size":"small","price":21.00}')
123+
puts res27 # => 1
124+
125+
res28 = redis.vsetattr('points', 'pt:E', '{"size":"small","price":17.75}')
126+
puts res28 # => 1
127+
128+
res29 = redis.vsim('points', 'pt:A', filter: '.size == "large"')
129+
puts res29.inspect # => ["pt:A", "pt:C", "pt:B"]
130+
131+
res30 = redis.vsim('points', 'pt:A', filter: '.size == "large" && .price > 20.00')
132+
puts res30.inspect # => ["pt:C", "pt:B"]
133+
# STEP_END
134+
135+
# STEP_START add_quant
136+
res31 = redis.vadd('quantSetQ8', [1.262185, 1.958231], 'quantElement', quantization: 'q8')
137+
puts res31 # => 1
138+
139+
res32 = redis.vemb('quantSetQ8', 'quantElement')
140+
puts "Q8: #{res32.inspect}"
141+
# => Q8: [~1.264, ~1.958]
142+
143+
res33 = redis.vadd('quantSetNoQ', [1.262185, 1.958231], 'quantElement', quantization: 'noquant')
144+
puts res33 # => 1
145+
146+
res34 = redis.vemb('quantSetNoQ', 'quantElement')
147+
puts "NOQUANT: #{res34.inspect}"
148+
# => NOQUANT: [~1.262185, ~1.958231]
149+
150+
res35 = redis.vadd('quantSetBin', [1.262185, 1.958231], 'quantElement', quantization: 'bin')
151+
puts res35 # => 1
152+
153+
res36 = redis.vemb('quantSetBin', 'quantElement')
154+
puts "BIN: #{res36.inspect}"
155+
# => BIN: [1.0, 1.0]
156+
# STEP_END
157+
158+
# STEP_START add_reduce
159+
values = Array.new(300) { |i| i / 299.0 }
160+
161+
res37 = redis.vadd('setNotReduced', values, 'element')
162+
puts res37 # => 1
163+
164+
res38 = redis.vdim('setNotReduced')
165+
puts res38 # => 300
166+
167+
res39 = redis.vadd('setReduced', values, 'element', reduce_dim: 100)
168+
puts res39 # => 1
169+
170+
res40 = redis.vdim('setReduced')
171+
puts res40 # => 100
172+
# STEP_END
173+
174+
redis.close
175+
puts "\nVector set tutorial completed successfully!"

0 commit comments

Comments
 (0)