Get a list of unique messages based on certain header

[MitchelNijdam-Rockstars]

In order to analyze a higher number of triaged records (let say in the order of 50+), it would be useful to group the records based on a header value, for example exception message. This would make it easier to get a list of the unique exceptions that occurred.

Since exception messages could contain variables (such as offset/faulty value), a 'normal' GROUP BY might not be enough.

Postgres provides pg_trgm with similarity(text, text) method, which might be useful.

Possible nice to have is to dynamically change the similarity percentage.

[alexanderabramov]

One step before showing a list of records with a certain exception message would be to show a list of all distinct exception messages with counters, sorted by how many there are.

Something like this:

select count(r.key), h.value 
from record r 
inner join header h on r.id = h.record_id 
where r.topic = '%topic' 
and r.triaged = false 
and h.key = 'kafka_dlt-exception-message' 
group by h.value;

Ideally taking into account variable parameters.

Ideally also flexible to handle both spring-kafka and spring-cloud-streams header names.