Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gzip compression for files #204

Open
3 tasks
rheimus opened this issue Oct 10, 2020 · 3 comments
Open
3 tasks

Add gzip compression for files #204

rheimus opened this issue Oct 10, 2020 · 3 comments

Comments

@rheimus
Copy link
Collaborator

rheimus commented Oct 10, 2020

Easy enough to zip up files before transfer.

Thoughts:

  • Do we pre zip files during server startup or zip files on the fly when requested by clients 🤔
  • If we pre zip files we probably need to set up cache / cache busting etc
  • Probably make it configurable for what file types get compressed, or if compression is on at all.

Compression should be

  • User configurable (types of files to compress if any)
  • Cacheable, we don't want to have to recompress every transfer
  • Automatic, neither server nor client should need to manually work with compressed files
@Hadyark
Copy link
Contributor

Hadyark commented Oct 11, 2020

I don't think this feature is interesting.
I have compressed my packmods (RLCraft) with WinRar and I have a compression rate of 92% (160mo -> 150mo). If I compress only .jar files, for example the 2 biggest mods in the pack I go from 91mb to 90mb.
This gain in size is to the detriment of the client's compression / decompression time.

@rheimus
Copy link
Collaborator Author

rheimus commented Oct 12, 2020

Compressing and Decompressing gzip is pretty fast, the server-side would most likely only compress once at startup anyway and only recompress files if they change.

ServerSync is not restricted to jar files, many users sync text-based files that benefit greatly from compression.

Some very dodgy numbers here 😊: https://docs.google.com/spreadsheets/d/1ni19pnle73FCxE-GJsEOVRBcczRO3yoJNCDWLhwpauc/edit?usp=sharing

The basic test above already shows a 10% decrease in traffic over the socket, which is a win in my books, the slow part of server sync is really the transfer speed.

@rheimus
Copy link
Collaborator Author

rheimus commented Oct 12, 2020

We could probably combine this with saving the generated file manifest, could do something basic like a last modified check to cache bust files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants