Skip to content

Commit 4b618da

Browse files
author
Brian Sam-Bodden
committed
feat(search): add comprehensive Redis Query Engine support
Implement modular Search architecture with complete feature parity to redis-py: Core Components: - Schema and field definitions (TextField, NumericField, TagField, GeoField, VectorField, GeoShapeField) - Query builder with fluent API and advanced query syntax - Index management and operations (create, drop, alter, info) - Aggregation framework with reducers and grouping - Hybrid search combining text and vector queries - Result parsing and formatting Search Features: - Full-text search with stemming, phonetic matching, and stop words - Vector similarity search supporting FLAT, HNSW, and SVS-VAMANA algorithms - Geospatial search with radius and polygon queries - Numeric and tag filtering - Aggregations with grouping, sorting, applying, and reducing - Hybrid search with score combination (RRF, linear) - Auto-complete/suggestions with fuzzy matching - Spell checking and synonym management - Query profiling and explain Field Types: - TextField: full-text search with weights, phonetic matching, withsuffixtrie - NumericField: range queries with sortable option - TagField: exact-match filtering with case sensitivity and separators - GeoField: geospatial queries with radius search - VectorField: vector similarity with multiple algorithms and distance metrics - GeoShapeField: polygon-based geospatial queries Advanced Features: - IndexDefinition for fine-grained index control - Query parameters for dynamic queries - Multiple dialect support (1, 2, 3, 4) - Cursor-based pagination for large result sets - Score explanations and custom scorers - Highlighting and summarization Bug Fixes: - Fix PARAMS handling to preserve binary vector data (don't call .to_s on values) - Add DIALECT 2 requirement for KNN queries in Redis 8 - Add convenience API for SORTBY (:sort_by + :asc parameters) - Fix field option ordering for proper Redis command syntax Testing: - Add comprehensive test suite with 44 tests across 8 test files - Test coverage for all search features and edge cases - Hybrid search tests with vector + text queries - Aggregation tests with complex pipelines - Vector similarity tests with multiple algorithms Documentation: - Add 6 example files demonstrating all features: - search_quickstart.rb: basic search operations - search_ft_queries.rb: advanced query syntax - search_aggregations.rb: aggregation pipelines - search_geo.rb: geospatial queries - search_range.rb: numeric range queries - search_vector_similarity.rb: vector search - search_with_hashes.rb: search with Redis hashes Results: 44 tests, 215 assertions, 0 failures, 0 errors, 0 skips
1 parent 2edd8b1 commit 4b618da

25 files changed

Lines changed: 4846 additions & 0 deletions

examples/search_aggregations.rb

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
# Redis Search Aggregations Example
5+
#
6+
# This example demonstrates Redis Search aggregation and analytics capabilities including:
7+
# - FT.AGGREGATE with various reducers (COUNT, SUM, AVG, MIN, MAX)
8+
# - GROUPBY operations with multiple reducers
9+
# - APPLY transformations and expressions
10+
# - SORTBY for ordering results
11+
# - LIMIT for pagination
12+
# - FILTER for conditional filtering
13+
# - Complex aggregation pipelines
14+
#
15+
# Run this example with: ruby examples/search_aggregations.rb
16+
17+
require 'redis'
18+
require 'json'
19+
require_relative '../lib/redis/commands/json'
20+
require_relative '../lib/redis/commands/search'
21+
22+
# Connect to Redis
23+
redis = Redis.new(host: 'localhost', port: 6400)
24+
redis.extend(Redis::Commands::JSON)
25+
redis.extend(Redis::Commands::Search)
26+
27+
# Clean up any existing index
28+
begin
29+
redis.ft_dropindex('idx:bicycle', delete_documents: true)
30+
rescue Redis::CommandError
31+
# Index doesn't exist, continue
32+
end
33+
34+
# Create index
35+
schema = Redis::Commands::Search::Schema.build do
36+
text_field '$.brand', as: 'brand'
37+
text_field '$.model', as: 'model'
38+
text_field '$.description', as: 'description'
39+
numeric_field '$.price', as: 'price'
40+
tag_field '$.condition', as: 'condition'
41+
end
42+
43+
definition = Redis::Commands::Search::IndexDefinition.new(
44+
prefix: ['bicycle:'],
45+
index_type: Redis::Commands::Search::IndexType::JSON
46+
)
47+
48+
redis.create_index('idx:bicycle', schema, definition: definition)
49+
50+
# Bicycle data
51+
bicycle_data = [
52+
{
53+
brand: 'Velorim',
54+
model: 'Jigger',
55+
price: 270,
56+
description: 'Small and powerful, the Jigger is the best ride for the smallest of tikes!',
57+
condition: 'new'
58+
},
59+
{
60+
brand: 'Bicyk',
61+
model: 'Hillcraft',
62+
price: 1200,
63+
description: 'Kids want to ride with as little weight as possible.',
64+
condition: 'used'
65+
},
66+
{
67+
brand: 'Nord',
68+
model: 'Chook air 5',
69+
price: 815,
70+
description: 'The Chook Air 5 gives kids aged six years and older a durable bike.',
71+
condition: 'used'
72+
},
73+
{
74+
brand: 'Eva',
75+
model: 'Eva 291',
76+
price: 3400,
77+
description: 'The sister company to Nord, Eva launched in 2005.',
78+
condition: 'used'
79+
},
80+
{
81+
brand: 'Noka Bikes',
82+
model: 'Kahuna',
83+
price: 3200,
84+
description: 'Whether you want to try your hand at XC racing.',
85+
condition: 'used'
86+
},
87+
{
88+
brand: 'Breakout',
89+
model: 'XBN 2.1 Alloy',
90+
price: 810,
91+
description: 'The XBN 2.1 Alloy is our entry-level road bike.',
92+
condition: 'new'
93+
},
94+
{
95+
brand: 'ScramBikes',
96+
model: 'WattBike',
97+
price: 2300,
98+
description: 'The WattBike is the best e-bike for people who still feel young at heart.',
99+
condition: 'new'
100+
},
101+
{
102+
brand: 'Peaknetic',
103+
model: 'Secto',
104+
price: 430,
105+
description: 'If you struggle with stiff fingers or a kinked neck.',
106+
condition: 'new'
107+
},
108+
{
109+
brand: 'nHill',
110+
model: 'Summit',
111+
price: 1200,
112+
description: 'This budget mountain bike from nHill performs well.',
113+
condition: 'new'
114+
},
115+
{
116+
brand: 'BikeShind',
117+
model: 'ThrillCycle',
118+
price: 815,
119+
description: 'An artsy, retro-inspired bicycle.',
120+
condition: 'refurbished'
121+
}
122+
]
123+
124+
# Add bicycle documents
125+
bicycle_data.each_with_index do |bike, i|
126+
redis.json_set("bicycle:#{i}", '$', bike)
127+
end
128+
129+
puts "Added #{bicycle_data.length} bicycle documents\n\n"
130+
131+
# STEP_START agg1
132+
# Example 1: APPLY transformation - Calculate discounted price for new bicycles
133+
puts "Example 1: APPLY transformation - Calculate discounted price for new bicycles"
134+
req1 = Redis::Commands::Search::AggregateRequest.new('@condition:{new}')
135+
.load('__key', 'price')
136+
.apply(discounted: '@price - (@price * 0.1)')
137+
138+
res1 = redis.ft_aggregate('idx:bicycle', req1)
139+
puts "Total results: #{res1[0]}"
140+
puts "Results:"
141+
(1...res1.length).each do |i|
142+
row = res1[i]
143+
puts " Key: #{row[row.index('__key') + 1]}, " \
144+
"Price: #{row[row.index('price') + 1]}, " \
145+
"Discounted: #{row[row.index('discounted') + 1]}"
146+
end
147+
puts
148+
# STEP_END
149+
150+
# STEP_START agg2
151+
# Example 2: GROUPBY with SUM reducer - Count affordable bikes by condition
152+
puts "Example 2: GROUPBY with SUM reducer - Count affordable bikes by condition"
153+
req2 = Redis::Commands::Search::AggregateRequest.new('*')
154+
.load('price')
155+
.apply(price_category: '@price<1000')
156+
.group_by('@condition', Redis::Commands::Search::Reducers.sum('@price_category').as('num_affordable'))
157+
158+
res2 = redis.ft_aggregate('idx:bicycle', req2)
159+
puts "Total results: #{res2[0]}"
160+
puts "Results:"
161+
(1...res2.length).each do |i|
162+
row = res2[i]
163+
puts " Condition: #{row[row.index('condition') + 1]}, " \
164+
"Num Affordable: #{row[row.index('num_affordable') + 1]}"
165+
end
166+
puts
167+
# STEP_END
168+
169+
# STEP_START agg3
170+
# Example 3: GROUPBY with COUNT reducer - Count total bicycles
171+
puts "Example 3: GROUPBY with COUNT reducer - Count total bicycles"
172+
req3 = Redis::Commands::Search::AggregateRequest.new('*')
173+
.apply(type: "'bicycle'")
174+
.group_by('@type', Redis::Commands::Search::Reducers.count.as('num_total'))
175+
176+
res3 = redis.ft_aggregate('idx:bicycle', req3)
177+
puts "Total results: #{res3[0]}"
178+
puts "Results:"
179+
(1...res3.length).each do |i|
180+
row = res3[i]
181+
puts " Type: #{row[row.index('type') + 1]}, " \
182+
"Total: #{row[row.index('num_total') + 1]}"
183+
end
184+
puts
185+
# STEP_END
186+
187+
# STEP_START agg4
188+
# Example 4: GROUPBY with TOLIST reducer - List bicycles by condition
189+
puts "Example 4: GROUPBY with TOLIST reducer - List bicycles by condition"
190+
req4 = Redis::Commands::Search::AggregateRequest.new('*')
191+
.load('__key')
192+
.group_by('@condition', Redis::Commands::Search::Reducers.tolist('__key').as('bicycles'))
193+
194+
res4 = redis.ft_aggregate('idx:bicycle', req4)
195+
puts "Total results: #{res4[0]}"
196+
puts "Results:"
197+
(1...res4.length).each do |i|
198+
row = res4[i]
199+
condition_idx = row.index('condition')
200+
bicycles_idx = row.index('bicycles')
201+
puts " Condition: #{row[condition_idx + 1]}"
202+
puts " Bicycles: #{row[bicycles_idx + 1]}"
203+
end
204+
puts
205+
# STEP_END
206+
207+
# STEP_START agg5
208+
# Example 5: GROUPBY with multiple reducers - Statistics by condition
209+
puts "Example 5: GROUPBY with multiple reducers - Statistics by condition"
210+
req5 = Redis::Commands::Search::AggregateRequest.new('*')
211+
.load('price')
212+
.group_by('@condition',
213+
Redis::Commands::Search::Reducers.count.as('count'),
214+
Redis::Commands::Search::Reducers.sum('@price').as('total_price'),
215+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'),
216+
Redis::Commands::Search::Reducers.min('@price').as('min_price'),
217+
Redis::Commands::Search::Reducers.max('@price').as('max_price'))
218+
219+
res5 = redis.ft_aggregate('idx:bicycle', req5)
220+
puts "Total results: #{res5[0]}"
221+
puts "Results:"
222+
(1...res5.length).each do |i|
223+
row = res5[i]
224+
puts " Condition: #{row[row.index('condition') + 1]}"
225+
puts " Count: #{row[row.index('count') + 1]}"
226+
puts " Total Price: #{row[row.index('total_price') + 1]}"
227+
puts " Avg Price: #{row[row.index('avg_price') + 1]}"
228+
puts " Min Price: #{row[row.index('min_price') + 1]}"
229+
puts " Max Price: #{row[row.index('max_price') + 1]}"
230+
end
231+
puts
232+
# STEP_END
233+
234+
# STEP_START agg6
235+
# Example 6: SORTBY - Sort results by price descending
236+
puts "Example 6: SORTBY - Sort results by price descending"
237+
req6 = Redis::Commands::Search::AggregateRequest.new('*')
238+
.load('__key', 'price', 'brand')
239+
.sort_by(Redis::Commands::Search::Desc.new('@price'))
240+
241+
res6 = redis.ft_aggregate('idx:bicycle', req6)
242+
puts "Total results: #{res6[0]}"
243+
puts "Results (top 5):"
244+
(1...[res6.length, 6].min).each do |i|
245+
row = res6[i]
246+
puts " Brand: #{row[row.index('brand') + 1]}, " \
247+
"Price: #{row[row.index('price') + 1]}"
248+
end
249+
puts
250+
# STEP_END
251+
252+
# STEP_START agg7
253+
# Example 7: LIMIT - Paginate results
254+
puts "Example 7: LIMIT - Paginate results"
255+
req7 = Redis::Commands::Search::AggregateRequest.new('*')
256+
.load('__key', 'price', 'brand')
257+
.sort_by(Redis::Commands::Search::Asc.new('@price'))
258+
.limit(2, 3) # Skip 2, return 3
259+
260+
res7 = redis.ft_aggregate('idx:bicycle', req7)
261+
puts "Total results: #{res7[0]}"
262+
puts "Results (offset 2, limit 3):"
263+
(1...res7.length).each do |i|
264+
row = res7[i]
265+
puts " Brand: #{row[row.index('brand') + 1]}, " \
266+
"Price: #{row[row.index('price') + 1]}"
267+
end
268+
puts
269+
# STEP_END
270+
271+
# STEP_START agg8
272+
# Example 8: FILTER - Filter aggregated results
273+
puts "Example 8: FILTER - Filter aggregated results"
274+
req8 = Redis::Commands::Search::AggregateRequest.new('*')
275+
.load('price')
276+
.group_by('@condition',
277+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'))
278+
.filter('@avg_price > 1000')
279+
280+
res8 = redis.ft_aggregate('idx:bicycle', req8)
281+
puts "Total results: #{res8[0]}"
282+
puts "Results (conditions with avg price > 1000):"
283+
(1...res8.length).each do |i|
284+
row = res8[i]
285+
puts " Condition: #{row[row.index('condition') + 1]}, " \
286+
"Avg Price: #{row[row.index('avg_price') + 1]}"
287+
end
288+
puts
289+
# STEP_END
290+
291+
# STEP_START agg9
292+
# Example 9: Complex aggregation pipeline
293+
puts "Example 9: Complex aggregation pipeline - Price analysis by condition"
294+
req9 = Redis::Commands::Search::AggregateRequest.new('*')
295+
.load('price', 'brand')
296+
.apply(price_range: '@price >= 1000 ? "high" : "low"')
297+
.group_by(['@condition', '@price_range'],
298+
Redis::Commands::Search::Reducers.count.as('count'),
299+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'))
300+
.sort_by(Redis::Commands::Search::Desc.new('@count'))
301+
302+
res9 = redis.ft_aggregate('idx:bicycle', req9)
303+
puts "Total results: #{res9[0]}"
304+
puts "Results:"
305+
(1...res9.length).each do |i|
306+
row = res9[i]
307+
puts " Condition: #{row[row.index('condition') + 1]}, " \
308+
"Price Range: #{row[row.index('price_range') + 1]}, " \
309+
"Count: #{row[row.index('count') + 1]}, " \
310+
"Avg Price: #{row[row.index('avg_price') + 1]}"
311+
end
312+
puts
313+
# STEP_END
314+
315+
puts "All aggregation examples completed successfully!"

0 commit comments

Comments
 (0)