Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch/sq mass loader low mem #154

Merged
merged 8 commits into from
Nov 6, 2024
Merged

Patch/sq mass loader low mem #154

merged 8 commits into from
Nov 6, 2024

Conversation

jcharkow
Copy link
Collaborator

@jcharkow jcharkow commented Oct 30, 2024

Description

  • Make libraryFiles specific to only GenericSpectrumLoaders (e.g. MzMLLoader) as these are the only files that require a library
  • Remove OSW "HashTables" so that initiation of SqMassFiles does not take a long time and is more memory efficient.
  • Added a "cache" function to loaders so that peptides already loaded do not have to be reloaded from disk

Contents (#154)

Other

  • move library access to MzMLLoader
  • fix tests
  • cached loaded peptides
  • move library checks to GenericSpectrumLoader
  • remove print statements
  • import statement accidentally deleted
  • fix bug that would not allow libraryFile to be None

Uncategorised!

  • Speedup OSWDataAccess initialization

library access is not needed for SqMassLoader, so move to MzMLLoader.
This saves both loading time and memory
remove hashtables or only initiate hashtables if needed
fix tests now that result file does not require a library
this is quite convienient in a jupyter notebook setting when using the
plotChromatograms() function. In this case, the chromatograms do not
have to be loaded from disk because they are already present.
theoretically a library would be needed for extraction in any
SpectrumLoader, (not just .mzml) so move the library checks to the
abstract class
@jcharkow jcharkow requested a review from singjc October 30, 2024 16:37
Copy link
Collaborator

@singjc singjc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The docs CI seems to be failing though?

@jcharkow
Copy link
Collaborator Author

jcharkow commented Nov 1, 2024

I fixed the docs failing so should be good to merge now?

@singjc singjc merged commit 126f995 into dev Nov 6, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants