alevin-fry quant slightly inconsistent results

[nschcolnicov]

This isn't really an issue, but I want to understand why is it that alevin-fry quant always produces different results even when using the same input.
I see that for these files:

• featureDump.txt
• gene_eqclass.txt.gz
• geqc_counts.mtx
• quants_mat.mtx
• quants_mat_rows.txt
  The rows are always in different order, and the values in each row have slightly different values.

For example, I compared the featureDump.txt files across two executions that used the same inputs. I sorted both files and then ran the diff command and I got this output:

As you can see, after sorting, the values are almost the same, but not exactly the same.

The reason why I'm asking about this is because I need to create some unit tests around this tool and need to find a way to make the results fully reproducible. Is there a way to achieve this?

This is the command that I'm using to run the tool:

alevin-fry quant \
-i ./af_permit_list_P10_T_collate -m gencode.v42.transcripts_tg2.txt \
--resolution cr-like-em -o ./alevin_fry_quant_P10_T \
--dump-eqclasses \
--use-mtx -t 16

[rob-p]

Hi @nschcolnicov,

Apologies for missing this issue. I was not notified of it for some reason. The differences appear to be only in floating point values. I believe this is due to order dependence of the sum in floating point addition of sequences of numbers. That is, in different runs, individual barcodes may be ordered differently in the resulting matrix, and thus summer in a different order, resulting in small differences of aggregate values. My recommendation would be to allow small epsilons for comparison in unit tests. If precise, identical values are required, I recommend taking a look at the code that produces the feature dump file. Perhaps we could impose some order there. However, in general, there are aspects that are expected to vary between runs (e.g. barcode order) based on the concurrency of the implementation.