Skip to content

Commit 820929f

Browse files
author
bosd
committed
[ADD] base_sparse_field_jsonb
1 parent 1718574 commit 820929f

File tree

16 files changed

+1100
-0
lines changed

16 files changed

+1100
-0
lines changed

base_sparse_field_jsonb/README.md

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
## Introduction
2+
3+
This module upgrades the `base_sparse_field` module to use PostgreSQL's native JSONB
4+
column type instead of TEXT for serialized fields.
5+
6+
**Why JSONB?**
7+
8+
The standard `base_sparse_field` stores serialized attributes as JSON text in a TEXT
9+
column. While functional, this has limitations:
10+
11+
- No database-level indexing on JSON content
12+
- Filtering requires fetching all records and processing in Python
13+
- Less efficient storage (text vs binary)
14+
15+
**What this module provides:**
16+
17+
- **JSONB Storage**: Serialized fields use PostgreSQL JSONB column type
18+
- **GIN Indexes**: Automatic creation of GIN indexes for fast key/value lookups
19+
- **Transparent Upgrade**: Drop-in replacement, no code changes needed
20+
- **Migration Support**: Automatically converts existing TEXT columns to JSONB
21+
22+
**Performance Benefits:**
23+
24+
| Operation | TEXT (before) | JSONB (after) |
25+
| ------------------- | ------------------------ | ------------------ |
26+
| Key existence check | Python loop | GIN index O(log n) |
27+
| Value filtering | Full table scan + Python | Index-assisted |
28+
| Storage size | ~30% larger | Binary compressed |
29+
30+
This module is particularly beneficial when used with `attribute_set` and
31+
`product_attribute_set` for managing dynamic product attributes on e-commerce websites
32+
where filtering performance is critical.
33+
34+
## Configuration
35+
36+
No configuration is required. The module works automatically upon installation.
37+
38+
## Optional: Verify Installation
39+
40+
After installation, you can verify JSONB columns exist:
41+
42+
```sql
43+
SELECT column_name, data_type
44+
FROM information_schema.columns
45+
WHERE table_name = 'product_template'
46+
AND column_name LIKE 'x_custom%';
47+
```
48+
49+
Expected output:
50+
51+
```
52+
column_name | data_type
53+
------------------------+-----------
54+
x_custom_json_attrs | jsonb
55+
```
56+
57+
## Optional: Verify GIN Index
58+
59+
```sql
60+
SELECT indexname, indexdef
61+
FROM pg_indexes
62+
WHERE tablename = 'product_template'
63+
AND indexname LIKE '%gin%';
64+
```
65+
66+
## Usage
67+
68+
## Installation
69+
70+
Simply install this module. It will:
71+
72+
1. Override the `Serialized` field class to use JSONB
73+
2. Migrate any existing TEXT columns to JSONB
74+
3. Create GIN indexes on all serialized field columns
75+
76+
No configuration is required.
77+
78+
## Compatibility
79+
80+
This module is compatible with:
81+
82+
- `attribute_set` - Dynamic attributes for any model
83+
- `product_attribute_set` - Product-specific attributes
84+
- `website_attribute_set` - E-commerce attribute display and filtering
85+
86+
All modules using `base_sparse_field` automatically benefit from JSONB storage.
87+
88+
## Technical Details
89+
90+
### Column Type Change
91+
92+
Before:
93+
94+
```sql
95+
x_custom_json_attrs TEXT
96+
```
97+
98+
After:
99+
100+
```sql
101+
x_custom_json_attrs JSONB
102+
```
103+
104+
### GIN Index
105+
106+
The module creates GIN indexes for fast lookups:
107+
108+
```sql
109+
CREATE INDEX idx_product_template_x_custom_json_attrs_gin
110+
ON product_template USING GIN (x_custom_json_attrs);
111+
```
112+
113+
### Querying JSONB (Advanced)
114+
115+
With JSONB, you can use PostgreSQL's native JSON operators in raw SQL:
116+
117+
```sql
118+
-- Find products where x_capacity > 5000
119+
SELECT * FROM product_template
120+
WHERE x_custom_json_attrs->>'x_capacity' > '5000';
121+
122+
-- Find products with a specific attribute
123+
SELECT * FROM product_template
124+
WHERE x_custom_json_attrs ? 'x_fire_suppression_system';
125+
126+
-- Find products matching multiple criteria
127+
SELECT * FROM product_template
128+
WHERE x_custom_json_attrs @> '{"x_power_type": "electric"}';
129+
```
130+
131+
## Migration from TEXT
132+
133+
If you have existing data in TEXT format, the post-install hook automatically handles
134+
the migration:
135+
136+
```sql
137+
ALTER TABLE product_template
138+
ALTER COLUMN x_custom_json_attrs TYPE jsonb
139+
USING x_custom_json_attrs::jsonb;
140+
```
141+
142+
Empty strings and NULL values are handled gracefully.
143+
144+
## Roadmap
145+
146+
## Planned Enhancements
147+
148+
### ORM Search Integration
149+
150+
Currently, sparse field filtering is done in Python after fetching records. Future
151+
versions may include ORM-level search operators that translate to PostgreSQL JSON
152+
queries:
153+
154+
```python
155+
# Future: Direct ORM filtering on sparse fields
156+
products = env['product.template'].search([
157+
('x_capacity', '>', 5000), # Translated to JSONB query
158+
])
159+
```
160+
161+
### Dynamic Index Creation
162+
163+
Add support for creating targeted indexes on frequently filtered attributes:
164+
165+
```python
166+
# Future: Per-attribute index
167+
attribute.create_search_index()
168+
```
169+
170+
### Search Panel Integration
171+
172+
Native integration with Odoo's search panel widget for attribute filtering on website
173+
product listings.
174+
175+
## Contributors
176+
177+
- OBS Solutions B.V. <https://www.obs-solutions.com>
178+
179+
## Credits
180+
181+
## Development
182+
183+
This module was developed based on discussions and research from:
184+
185+
- [OCA/odoo-pim Issue #153](https://github.com/OCA/odoo-pim/issues/153) - Storage and
186+
search optimization
187+
- [Akretion Issue #62](https://github.com/akretion/ak-odoo-incubator/issues/62) -
188+
base_sparse_field evolution

0 commit comments

Comments
 (0)