Week 13: The Final Week

In the final week of GSOC I implemented and tested the correction function for the database suitability.

Implementing

Implementing didn't take that long. This was mostly because the main functionalities were already implemented by me for the other try of correcting suitability. I just had to change some minor stuff for them to work.

The only thing I did need to add was the functionality to calculate some decoy entries. Since I want to sample the given database, it's not really possible to take a file already containing decoys. Unfortunately the decoy calculations are not, as for now, exported into a library class. So, I just copied some code from the DecoyDatabase UTIL. That's not good style and might be changed later on, but for now it was the fastest solution. And fast solutions are needed considering it's the last week.

Benchmarking

I than tested the corrected suitability on the human data from before and also on some data from a roundworm (C. Elegans). The files for the latter I also got from the PRIDE folder of the original paper.

Human:


C. Elegans:

As you can see the correction does work pretty well!

Let's first ignore the outliers around ratio of 0.5. When we do that, the correction does a pretty good job of linearization of the suitability. The red lines depicts the suitability/deNovo hits before correction and the blue lines after correction.

For the human data the corrected suitability graph almost perfectly represents the diagonal. This is exactly what we wanted. Because now the suitability corresponds to the ratio of the used database. This is something that's far easier to explain to users. Before that suitability could only be used as a comparing tool and now the score actual means something.

Looking at the C. Elegans plot the graph is a little bit more off. That's because the "right" database scores way lower than in the human example.

Now to the outliers: I am not quite sure why this is happening. The internal extrapolation should execute suitability calculations for ratios of 0.2 to 0.3 here and looking at the non-corrected data the values at those ratios are very stable. It could of cause very well be a bug in my code. Since those outliers appear in both cases and also around a ratio of 0.5, it might also be a general problem with the approach. To find out what is happening here I'm going to give some debug outputs in the suitability class. A fix could be to do multiple subsampling runs, but it's hard to say as long as I'm not sure what exactly is causing the behaviour.

 

Unfortunately the week is over now. For now this will be the final point of this project. I will write a summary of all the things that I did and post it on this blog.

Post-GSOC I'm probably finishing the correction of the suitability to get this functionality to a working state.

Comments

Post a Comment

Popular posts from this blog

GSOC Final Report

Image
What was the goal? A recent publication (found here ) describes a metric to evaluate the suitability of a database used for an identification search of LC-MS/MS data. This is useful because sometimes you don't now from which organism the data is coming from. In that case it is also very challenging to find the best database to search with. OpenMS (codebase located on github ) is a huge open source library with a lot of tools to analyze  mass spectrometry  data. The goal of this project was to implement the freshly proposed metric as a tool in OpenMS. The paper also describes another metric so score the quality of the recorded spectra. This metric was also to be implemented, but it is a lot more trivial and not doesn't need that much attention. Was the goal achieved? Yes, it was. One week ago the final version of the database suitability tool was merged. This tool calculates database suitability and spectral quality according to the algorithms presented in the paper. All algorit
Read more

Week 12: Suitability Correction (& some PR Work)

Image
At the start of this week only the PR containing the changes to the adapters passed. I still had to work on some stuff for the suitability functions. But since these changes were not that complex and reviewing of my changes also took some time, I merged my suitability class on a local branch to start working on the corrected suitability. This is would I did while waiting for reviews or tests to finish: Changes for Suitability Correction For this correction to work a lot of additional functions needed to be coded. Also the input of the compute function had to change drastically because for the two internal identification searches a lot of additonal information is needed. First of all a mzML input to have something to search at all. Second of all the two databases, the original database and the deNovo "database" (with one entry which is a concatination of all deNovo sequences), to have something to search with. And third of all the parameters to know which search adapter to use
Read more

Week 11: Preparing the New Suitability Calculation

The plan on how to correct the suitability to behave more linear is to correct the number of top deNovo hits found. This is done with a factor which is calculated like this: #identifications with only deNovo / #identifications with only the database To calculate this factor two individual identification searches need to be performed, one with only the database without the deNovo peptide and one the other way around. It is very crucial that those searches are done with the same settings as the original database+deNovo one. Currently those infomations are not completely exported by the OpenMS-Adapters that bind the search engines into the library. Only a few informations are written. So that's what I did first this week. Export Adapter Parameters I added a static function to DefaultParamHandler (This is the OpenMS class that is build for parameter stuff.) which takes a Param object (storage for parameters), a MetaInfoInterface object (here meta values can be written) and a string whi
Read more