Describe the bug
AzureAISearchDocumentStore's in filter operator builds a search.in(field, valueList, delimiter) OData expression by joining values with a comma (_in() in filters.py), the same character Azure uses as the delimiter argument. Any string metadata value that itself contains a literal comma gets split apart by Azure's own server side parsing of that value list, so the filter never matches the value you passed in. A value like "New York, NY" becomes three fragments (New York, NY, Austin in a two-value list), none of which equal the original string.
_escape_odata_literal only escapes single quotes for the eq/ne operators; _in calls it too, but escaping a quote does nothing for a comma, since the comma is consumed by search.in's own delimiter parsing, not by OData string-literal parsing.
To Reproduce
from haystack_integrations.document_stores.azure_ai_search.filters import _normalize_filters
_normalize_filters({"operator": "in", "field": "meta.city", "value": ["New York, NY", "Austin"]})
# "search.in(city,'New York, NY,Austin',',')"
Per Azure's documented search.in(variable, valueList, delimiter) semantics, the service splits the value-list string on the delimiter before comparing, so the query above resolves to matching against ["New York", " NY", "Austin"], never "New York, NY". I reproduced the query-construction output above in a clean container against current main (39ba76eb3); I do not have a live Azure Search index to confirm the server-side split independently, but it follows Azure's own documented behavior for search.in.
A fix would need a delimiter that cannot appear in the values (or per-value escaping of the chosen delimiter before joining), plus a regression test alongside the existing _in/_escape_odata_literal tests.
Describe your environment
- Integration:
azure-ai-search-haystack
- File:
integrations/azure_ai_search/src/haystack_integrations/document_stores/azure_ai_search/filters.py, _in() (around line 91)
- Reproduced against
main at 39ba76eb3
Describe the bug
AzureAISearchDocumentStore'sinfilter operator builds asearch.in(field, valueList, delimiter)OData expression by joining values with a comma (_in()infilters.py), the same character Azure uses as the delimiter argument. Any string metadata value that itself contains a literal comma gets split apart by Azure's own server side parsing of that value list, so the filter never matches the value you passed in. A value like"New York, NY"becomes three fragments (New York,NY,Austinin a two-value list), none of which equal the original string._escape_odata_literalonly escapes single quotes for theeq/neoperators;_incalls it too, but escaping a quote does nothing for a comma, since the comma is consumed bysearch.in's own delimiter parsing, not by OData string-literal parsing.To Reproduce
Per Azure's documented
search.in(variable, valueList, delimiter)semantics, the service splits the value-list string on the delimiter before comparing, so the query above resolves to matching against["New York", " NY", "Austin"], never"New York, NY". I reproduced the query-construction output above in a clean container against currentmain(39ba76eb3); I do not have a live Azure Search index to confirm the server-side split independently, but it follows Azure's own documented behavior forsearch.in.A fix would need a delimiter that cannot appear in the values (or per-value escaping of the chosen delimiter before joining), plus a regression test alongside the existing
_in/_escape_odata_literaltests.Describe your environment
azure-ai-search-haystackintegrations/azure_ai_search/src/haystack_integrations/document_stores/azure_ai_search/filters.py,_in()(around line 91)mainat39ba76eb3