[Bug]: ES/OS backend ignore scope and link attributes

[Manik2708]

What happened?

ES dbmodel has no place to store scope and link attributes and therefore FromDBModel and ToDBModel ignored them. This is leading to information loss which needs to be fixed. v1 Adapter has also missed these attributes because v1 model has no place to store them, following this Badger and Cassandra too have missed these attributes and this should be taken care during ugrading them to v2

Expected behavior

ES/OS backend should properly store and return traces with link and scope attributes.

[yurishkuro]

do we have integration tests for link attributes? It should be part of the storage certification harness.

[Manik2708]

do we have integration tests for link attributes? It should be part of the storage certification harness.

yes, working on them

[yurishkuro]

The issue description is misleading - the reason v1 adapter ignores those is because there's no place in the v1 model to store them. I wouldn't want to spend cycles improving that, instead once a storage backend is fully migrated to v2 API it would be expected to satisfy e2e tests that would include these extra attributes, without any v1 conversion.

[Manik2708]

The issue description is misleading - the reason v1 adapter ignores those is because there's no place in the v1 model to store them.

agree, will update the description

I wouldn't want to spend cycles improving that, instead once a storage backend is fully migrated to v2 API it would be expected to satisfy e2e tests that would include these extra attributes, without any v1 conversion.

this issue requires changing dbmodels of ES and Cassandra and because ES upgrade took place long time back so I thought it to cover in a separate issue. So changing the scope of this issue to ES only and leave rest to storage backend upgrades will be right? @yurishkuro

[Manik2708]

@yurishkuro I have trimmed the scope to only ES/OS, if that's correct please remove the badger and cassandra labels and added this point to #8080

[SoumyaRaikwar]

The issue description is misleading - the reason v1 adapter ignores those is because there's no place in the v1 model to store them. I wouldn't want to spend cycles improving that, instead once a storage backend is fully migrated to v2 API it would be expected to satisfy e2e tests that would include these extra attributes, without any v1 conversion.

@yurishkuro
To support this "v2-first" strategy for ES/OS, if we set the minimum supported versions for v2 (Elasticsearch 7.9+ and OpenSearch 2.0+). Establishing this would: Let us use data streams by default in v2 without legacy fallback logic (as discussed in #7768) and Allow us to design the v2-native database model for links and scopes without being constrained by the needs of older ES 6.x/7.x versions that don't run in the v2 CI anyway.

Does it make sense to formalize these minimums as part of the full v2 migration?