diaPASEF first tests

[KlemensFroehlich]

hi Michael

Please forgive me for bringing this up again... In my defense: You said nothing is preventing users from converting .d folders to mzml and analyze diaPASEF data :)

I have tried a few things to see how different softwares currently handly diaPASEF. For this I generated a ~10ng, 5min human HEK active gradient diaPASEF run on a timsTOF Ultra.
file size as .d folder:
1.4GB

I have now tried 3 different ways to generate an mzml file:

1. MSConvert, standard settings, so I think it came down to 4.5 million scan events which led to a final size of 25GB.
2. MSConvert, Combine ion mobility scans 6.3GB
3. timsconvert, standard settings 5.7GB

I searched the data with sage and for comparison I also included a 30min ddaPASEF run, which was directly analyzed from .d

the ddaPASEF seems to work nicely looking at the q value distribution of spectrum, peptide or protein level, but for all other searches the q value distribution looks odd to me.

I would be happy to share the data with you, or do more testing benchmarking. As I already mentioned there is currently no non-proprietary option to analyze diaPASEF data with large search spaces, so I am highly motivated to get this off the ground.

You also said earlier that it is not trivial to collapse / handle the ion mobility in tims data.
Would you be open to have a look at the msconvert ion mobility combine option and at the resulting data structure?
Maybe this can already solve the problem and sage "only" needs to be adapted to be able to handle this specific mzml ?

Oh and btw working with sage made me smile more than once! It is a lot of fun to work with a tool so fast! I still remember the days I had to run semitryptic searches in MaxQuant for a week xD

Best, Klemens

[lazear]

I did say theoretically possible 😉 - there are clearly some practical issues.

I'm definitely interested in supporting diaPASEF. To be honest though, it will probably be ~2-3 months or so before I have time to really dig in and experiment (I have a lot going on until late spring!). It seems like there are a couple other people interested in perhaps trying it out too, and I am always happy to provide guidance on Sage internals - if there's a way to collapse into a single spectrum, plugging it into Sage should be straightforward.

If you're willing to share the files, I can download them and hack on them when I get a chance!

[KlemensFroehlich]

Thank you so much for even considering this!

please find all data (also a DIA-NN analysis directly from .d) here:
SwitchDrive

If supporting diaPASEF is a longterm possibility, may I ask what you are thinking about very roughly (I will not hold you to it I promise expect if you say that I can do something theoretically :P)?

More specifically, I was thinking about whether you would consider reaching out to Vadim Demichev to see whether it is possible to create the library with SAGE and then do the main DIA analysis with DIA-NN. It is not open-source though, so it might not be ideal. But looking at FragPipe, this could be a powerful combination of tools.

Again, if I can help in any way, please let me know! Especially, the files that I provided: If there is any other metric I should check or summarize that might be helpful, I would be happy to do so!

Best, Klemens

[prvst]

Thank you so much for even considering this!

please find all data (also a DIA-NN analysis directly from .d) here: SwitchDrive

If supporting diaPASEF is a longterm possibility, may I ask what you are thinking about very roughly (I will not hold you to it I promise expect if you say that I can do something theoretically :P)?

More specifically, I was thinking about whether you would consider reaching out to Vadim Demichev to see whether it is possible to create the library with SAGE and then do the main DIA analysis with DIA-NN. It is not open-source though, so it might not be ideal. But looking at FragPipe, this could be a powerful combination of tools.

Again, if I can help in any way, please let me know! Especially, the files that I provided: If there is any other metric I should check or summarize that might be helpful, I would be happy to do so!

Best, Klemens

Hello @KlemensFroehlich and @lazear. I would like to point out that the FragPipe DIA workflow uses a program called easypqp (OpenMS) to build the library. The library is then passed down to DIANN to perform the quantification.

If you go to the easypqp GitHub page, you'll see that someone already made this request to them.

Glad to see that people share the same interests and strategies.

Best

[lazear]

Yes, sage will write all matched fragments when using the --annotate-matches CLI flag, so it should be possible to plug into easypqp at some point, or other library building tools!

[RobbinBouwmeester]

@KlemensFroehlich @lazear , could this repo be useful? https://github.com/mafreitas/tdf2mzml, in the description they claim to have support for DIA bruker data (and based on a closed discussion/issue on that repo this concerns diaPASEF).

[jsnedeco]

To add to this: I tried out diaPASEF file to just see if I could use something besides DIA-NN for quantification (both for closed source reasons and also because it seems to take a very long time to generate a spectral library from a diaPASEF run).

I used tdf2mzml to generate an mzML file from a test bruker TimsTOF HT file and tested out the released version of Sage (0.14.7) with chimera, wide_window and report_psms to 5 (I can send the full config file if it helps). The top level identification stats look like this

[2024-07-18T19:37:40Z INFO sage] discovered 109719 target peptide-spectrum matches at 1% FDR
[2024-07-18T19:37:40Z INFO sage] discovered 21658 target peptides at 1% FDR
[2024-07-18T19:37:40Z INFO sage] discovered 4533 target proteins at 1% FDR

Not bad! Running on DIA-NN gives the following top line stats:

[34:35] Number of IDs at 0.01 FDR: 117535
[34:38] Calculating protein q-values
[34:38] Number of genes identified at 1% FDR: 8749 (precursor-level), 7925 (protein-level) (inference performed using proteotypic peptides only)

So it would be nice to boost the protein level identifications a touch, but it doesn't look bad as a first pass. It's also possible that DIA-NN is using different metrics for protein counts or something, I haven't investigated that yet.

Then I saw this PR (#140) and got extremely excited to try out the raw .d inputs to see if I could skip the conversion step altogether since it's pretty slow. I built a new version off of the master branch used the same parameters and...

[2024-07-18T20:34:23Z INFO sage] discovered 2776 target peptide-spectrum matches at 1% FDR
[2024-07-18T20:34:23Z INFO sage] discovered 593 target peptides at 1% FDR
[2024-07-18T20:34:23Z INFO sage] discovered 263 target proteins at 1% FDR

That seems quite a bit worse. I know that this is still very much an experimental use case, but I'm wondering if I'm doing something wrong?

[sander-willems-bruker]

Hi @jsnedeco . This indeed is very experimental, @jspaezp is working on a PR that should boost these numbers quite a bit (MannLabs/timsrust#21). Without looking at the dia.d folder and your config, I can't tell if you are doing anything wrong of course, but I am happy to share my findings with the implementaiton you also have available (MannLabs/timsrust#15 (comment)). I used this config: MannLabs/timsrust#15 (comment).

[jsnedeco]

Hi @sander-willems-bruker! I used these parameters:
results_test.json

It looks pretty similar, except your report_psms is a lot higher. I probably can't share the specific .d file I was testing, but I'll see if I can get a representative sample that I can share.

[jspaezp]

As a reference ... #140 this PR added support for dia-pasef direct reading.

@jsnedeco (wont discuss diann, since it does a completely different thing) but from what I see in message there is something wrong ...
PROVIDED THE SAME PARAMETERS FOR SAGE ARE USED, tdf2mzml->sage and timsrust->sage should give the same result or at least VERY similar (which has been the case for my data) ... since both pieces of software IN ESSENCE do the same thing.

I have seen on occasion that mass calibration looks different between them, so try running it with +- 50 ppm on ms2 (its overkill but... who knows :P and if that works let us know, since it would point to an error in how the mz is calculated in timsrust).

LMK if you can send over a file with representative data that suffers from the same issue.

[jsnedeco]

DIANN was mostly intended as a rough benchmark, not as something to achieve complete parity with :)

PROVIDED THE SAME PARAMETERS FOR SAGE ARE USED, tdf2mzml->sage and timsrust->sage should give the same result or at least VERY similar (which has been the case for my data) ... since both pieces of software IN ESSENCE do the same thing.

I used the same parameters for both, but I did use the released version of sage for the mzml file and the unreleased version of sage for the .d analysis (because it needed to be.) I just tested again using the unreleased version for both with the same parameters I used originally and the results look the same:

tdf2mzml->sage
[2024-07-19T19:24:55Z INFO sage] discovered 109719 target peptide-spectrum matches at 1% FDR
[2024-07-19T19:24:55Z INFO sage] discovered 21658 target peptides at 1% FDR
[2024-07-19T19:24:55Z INFO sage] discovered 4533 target proteins at 1% FDR

timsrust->sage
[2024-07-18T20:34:23Z INFO sage] discovered 2776 target peptide-spectrum matches at 1% FDR
[2024-07-18T20:34:23Z INFO sage] discovered 593 target peptides at 1% FDR
[2024-07-18T20:34:23Z INFO sage] discovered 263 target proteins at 1% FDR

timsrust->sage +/- 50 ppm ms2
[2024-07-19T19:35:27Z INFO sage] discovered 1764 target peptide-spectrum matches at 1% FDR
[2024-07-19T19:35:27Z INFO sage] discovered 506 target peptides at 1% FDR
[2024-07-19T19:35:27Z INFO sage] discovered 247 target proteins at 1% FDR

In both these cases I'm using report_psms: 5, just so I'm not tweaking too many parameters.
I'll see if I can get an example that I can actually share.

[jspaezp]

All of my samples are 22 min either ddaPASEF/diaPASEF

timsrust dda -> discovered 10519 target peptides at 1% FDR
timsrust dia -> discovered 3951 target peptides at 1% FDR
tdf2mzml dda -> discovered 10498 target peptides at 1% FDR
tdf2mzml dia -> discovered 4530 target peptides at 1% FDR

    "./231121_RH30_NMIAA_E1_DDA.d"
[2024-07-20T16:09:54Z INFO  sage] - file IO:     1210 ms
[2024-07-20T16:10:09Z INFO  sage] - search:     14355 ms (3409 spectra/s)
[2024-07-20T16:10:09Z INFO  sage_core::ml::retention_alignment] aligning file #0: y = 1.0000x + 0.0000
[2024-07-20T16:10:09Z INFO  sage_core::ml::retention_alignment] aligned retention times across 1 files
[2024-07-20T16:10:09Z INFO  sage_core::ml::retention_model] - fit retention time model, rsq = 0.8335711996562211
[2024-07-20T16:10:09Z INFO  sage_core::ml::mobility_model] - fit mobility model, rsq = 0.9504371376944695, mse = 0.00047716065311882407
[2024-07-20T16:10:09Z INFO  sage] discovered 18122 target peptide-spectrum matches at 1% FDR
[2024-07-20T16:10:09Z INFO  sage] discovered 10519 target peptides at 1% FDR
[2024-07-20T16:10:09Z INFO  sage] discovered 2225 target proteins at 1% FDR
[2024-07-20T16:10:09Z INFO  sage] finished in 27s
[2024-07-20T16:10:09Z INFO  sage] cite: "Sage: An Open-Source Tool for Fast Proteomics Searching and Quantification at Scale" https://doi.org/10.1021/acs.jproteome.3c00486

    "./231121_RH30_NMIAA_E1_DIA.d"
[2024-07-20T16:11:06Z INFO  sage] - file IO:    38320 ms
[2024-07-20T16:11:54Z INFO  sage] - search:     47667 ms (863 spectra/s)
[2024-07-20T16:11:54Z INFO  sage_core::ml::retention_alignment] aligning file #0: y = 1.0000x + 0.0000
[2024-07-20T16:11:54Z INFO  sage_core::ml::retention_alignment] aligned retention times across 1 files
[2024-07-20T16:11:54Z INFO  sage_core::ml::retention_model] - fit retention time model, rsq = 0.8302852190047076
[2024-07-20T16:11:54Z INFO  sage_core::ml::mobility_model] - fit mobility model, rsq = 0.9679125124575643, mse = 0.0007812790787113548
[2024-07-20T16:11:54Z INFO  sage] discovered 16037 target peptide-spectrum matches at 1% FDR
[2024-07-20T16:11:54Z INFO  sage] discovered 3951 target peptides at 1% FDR
[2024-07-20T16:11:54Z INFO  sage] discovered 1101 target proteins at 1% FDR
[2024-07-20T16:11:55Z INFO  sage] finished in 96s
[2024-07-20T16:11:55Z INFO  sage] cite: "Sage: An Open-Source Tool for Fast Proteomics Searching and Quantification at Scale" https://doi.org/10.1021/acs.jproteome.3c00486

    "./231121_RH30_NMIAA_E1_DDA.mzml"
[2024-07-20T16:12:08Z INFO  sage] generated 319009266 fragments, 12240058 peptides in 7125ms
[2024-07-20T16:12:08Z INFO  sage] processing files 0 .. 1 
[2024-07-20T16:12:09Z INFO  sage] - file IO:     1570 ms
[2024-07-20T16:12:18Z INFO  sage] - search:      8046 ms (6081 spectra/s)
[2024-07-20T16:12:18Z INFO  sage_core::ml::retention_alignment] aligning file #0: y = 1.0000x + 0.0000
[2024-07-20T16:12:18Z INFO  sage_core::ml::retention_alignment] aligned retention times across 1 files
[2024-07-20T16:12:18Z INFO  sage_core::ml::retention_model] - fit retention time model, rsq = 0.8349065613194171
[2024-07-20T16:12:18Z INFO  sage_core::ml::mobility_model] - fit mobility model, rsq = 0.950576663864528, mse = 0.0005678266815400485
[2024-07-20T16:12:18Z INFO  sage] discovered 18242 target peptide-spectrum matches at 1% FDR
[2024-07-20T16:12:18Z INFO  sage] discovered 10498 target peptides at 1% FDR
[2024-07-20T16:12:18Z INFO  sage] discovered 2247 target proteins at 1% FDR
[2024-07-20T16:12:18Z WARN  sage] parquet output format is currently unstable! There may be failures or schema changes!
[2024-07-20T16:12:18Z INFO  sage] finished in 17s
[2024-07-20T16:12:18Z INFO  sage] cite: "Sage: An Open-Source Tool for Fast Proteomics Searching and Quantification at Scale" https://doi.org/10.1021/acs.jproteome.3c00486

    "./231121_RH30_NMIAA_E1_DIA.mzml"
[2024-07-20T16:12:31Z INFO  sage] generated 319009266 fragments, 12240058 peptides in 7424ms
[2024-07-20T16:12:31Z INFO  sage] processing files 0 .. 1 
[2024-07-20T16:12:50Z INFO  sage] - file IO:    18955 ms
[2024-07-20T16:13:38Z INFO  sage] - search:     48018 ms (857 spectra/s)
[2024-07-20T16:13:39Z INFO  sage_core::ml::retention_alignment] aligning file #0: y = 1.0000x + 0.0000
[2024-07-20T16:13:39Z INFO  sage_core::ml::retention_alignment] aligned retention times across 1 files
[2024-07-20T16:13:39Z INFO  sage_core::ml::retention_model] - fit retention time model, rsq = 0.835649318485739
[2024-07-20T16:13:39Z INFO  sage_core::ml::mobility_model] - fit mobility model, rsq = NaN, mse = 0
[2024-07-20T16:13:39Z INFO  sage] discovered 18558 target peptide-spectrum matches at 1% FDR
[2024-07-20T16:13:39Z INFO  sage] discovered 4530 target peptides at 1% FDR
[2024-07-20T16:13:39Z INFO  sage] discovered 1236 target proteins at 1% FDR
[2024-07-20T16:13:39Z WARN  sage] parquet output format is currently unstable! There may be failures or schema changes!
[2024-07-20T16:13:39Z INFO  sage] finished in 75s
[2024-07-20T16:13:39Z INFO  sage] cite: "Sage: An Open-Source Tool for Fast Proteomics Searching and Quantification at Scale" https://doi.org/10.1021/acs.jproteome.3c00486

my config:

{
    "database": {
        "bucket_size": 8192,
        "enzyme": {
            "missed_cleavages": 2,
            "min_len": 7,
            "max_len": 50,
            "cleave_at": "KR",
            "restrict": "P"
        },
        "fragment_min_mz": 150.0,
        "fragment_max_mz": 2000.0,
        "peptide_min_mass": 500.0,
        "peptide_max_mass": 5000.0,
        "ion_kinds": [
            "b",
            "y"
        ],
        "min_ion_index": 2,
        "max_variable_mods": 3,
        "static_mods": {
            "C": 57.0215
        },
        "variable_mods": {
            "M": [15.994],
            "C": [71.03711]
        },
        "decoy_tag": "rev_",
        "generate_decoys": true
    },
    "precursor_tol": {
        "ppm": [
            -20.0,
            20.0
        ]
    },
    "fragment_tol": {
        "ppm": [
            -20.0,
            20.0
        ]
    },
    "deisotope": true,
    "min_peaks": 15,
    "max_peaks": 250,
    "max_fragment_charge": 1,
    "min_matched_peaks": 4,
    "predict_rt": true,
    "wide_window": true,
    "chimera": false,
    "report_psms": 1
}

I wonder what other things are different from the bruker sdk and timsrust ... LMK if you can upload data anywhere that we could work on together. RN I am also failing to find a file in PRIDE that is not zipped in a stupid way (zipping all the .d of the repo in a single file ...)

Edit: Downloading these two guys for testing ...

wget https://ftp.pride.ebi.ac.uk/pride/data/archive/2022/02/PXD028735/LFQ_timsTOFPro_PASEF_Ecoli_03.d.zip
wget https://ftp.pride.ebi.ac.uk/pride/data/archive/2022/02/PXD028735/LFQ_timsTOFPro_diaPASEF_Ecoli_01.d.zip

[sander-willems-bruker]

Hi @sander-willems-bruker! I used these parameters: results_test.json

It looks pretty similar, except your report_psms is a lot higher. I probably can't share the specific .d file I was testing, but I'll see if I can get a representative sample that I can share.

I'd definately increase frag_ppm (try 15ppm first, as @jspaezp mentions there are samples where you need to go to 50 due to calibration issues with timsrust), max_peaks (200+ for sure) and report_psms (20+ wont hurt in this case).

What might be going wrong on your particular search is that your mzml might be generated per scan, making them not too complex (meaning the max_peaks and report_psms indeed are not too influential for mzml), whereas timsrust projects the whole window (potentially 500+ scans aggregated, thuis far more peaks and far more complex).

[sander-willems-bruker]

I wonder what other things are different from the bruker sdk and timsrust ... LMK if you can upload data anywhere that we could work on together. RN I am also failing to find a file in PRIDE that is not zipped in a stupid way (zipping all the .d of the repo in a single file ...)

The raw data is identical, as well as all info in the tdf. The primary things that are different are mass and im calibration and centroiding. Depending on sample, there might also be a difference in raw intensities, as timsrust does not yet take ICC into account by default (with the api you can correct for this though, this is just a simple correction factor applied per frame based on TIMS ramp times)

[jsnedeco]

Hi @sander-willems-bruker! I used these parameters: results_test.json
It looks pretty similar, except your report_psms is a lot higher. I probably can't share the specific .d file I was testing, but I'll see if I can get a representative sample that I can share.

I'd definately increase frag_ppm (try 15ppm first, as @jspaezp mentions there are samples where you need to go to 50 due to calibration issues with timsrust), config_test.json(200+ for sure) and report_psms (20+ wont hurt in this case).

What might be going wrong on your particular search is that your mzml might be generated per scan, making them not too complex (meaning the max_peaks and report_psms indeed are not too influential for mzml), whereas timsrust projects the whole window (potentially 500+ scans aggregated, thuis far more peaks and far more complex).

I tried again with the attached config file and the results looked like this

[2024-07-22T22:05:09Z INFO sage] discovered 1980 target peptide-spectrum matches at 1% FDR
[2024-07-22T22:05:09Z INFO sage] discovered 340 target peptides at 1% FDR
[2024-07-22T22:05:09Z INFO sage] discovered 167 target proteins at 1% FDR

So worse than all the previous tests.

Annoyingly, I tried it on another file that I can share and I'm not seeing the same results!
Test file 2

timsrust->sage
[2024-07-22T20:55:37Z INFO sage] discovered 101933 target peptide-spectrum matches at 1% FDR
[2024-07-22T20:55:37Z INFO sage] discovered 18568 target peptides at 1% FDR
[2024-07-22T20:55:37Z INFO sage] discovered 4091 target proteins at 1% FDR

tdf2mzml->sage
[2024-07-22T22:12:25Z INFO sage] discovered 107831 target peptide-spectrum matches at 1% FDR
[2024-07-22T22:12:25Z INFO sage] discovered 19217 target peptides at 1% FDR
[2024-07-22T22:12:25Z INFO sage] discovered 4223 target proteins at 1% FDR

I am going to dig into the fragment annotations on the first unshareable file and see if there are any patterns. If there's any interest, I can share the second test file but it's not showing the behavior so it probably won't be that helpful.

[edit]

For what it's worth, here are the .d and mzml files for Test file 2.

https://filebin.net/frqq1bks9pbtzm8s

And just for fun, I took a look at the fragment_ppm for the psm ids that were called both in the mzml file and the .d file:
Test File 1 (not shared)

Test File 2 (shared in filebin)

[jspaezp]

It's also worth mentioning ... Can you share the .tdf without the tdf_bin? It would give information on the instrument, methods and calibration but not the actual scans (technically it also includes the TIC ). (Also I'm assuming it is done on the same instrument .. I've only tested it on SCP/ultra data ... The one publicly available is HT I think ... Which I haven't run just yet..)