Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for the jemalloc allocator, dusted up dependencies. Somewhat improved benchmarks #83

Merged
merged 9 commits into from
Jan 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
578 changes: 308 additions & 270 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[workspace]
members = ["vach", "vach-cli", "vach-benchmarks"]
resolver = "2"

[profile.bench]
debug = true
4 changes: 2 additions & 2 deletions vach-benchmarks/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ license = "MIT"
publish = false

[dependencies]
criterion = "0.3.5"
vach = { path = "../vach", features = ["compression", "crypto"] }
criterion = "0.5.1"
vach = { path = "../vach", features = ["all"] }

[[bench]]
name = "benchmark"
Expand Down
30 changes: 18 additions & 12 deletions vach-benchmarks/benches/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,23 @@ use vach::prelude::*;
use vach::crypto_utils::gen_keypair;

// Remove io overhead by Sinking data into the void
struct Sink;
struct Sink(u64);

impl Sink {
fn new() -> Sink {
black_box(Sink)
black_box(Sink(0))
}
}

impl io::Seek for Sink {
fn seek(&mut self, _: io::SeekFrom) -> io::Result<u64> {
Ok(0)
fn seek(&mut self, seek: io::SeekFrom) -> io::Result<u64> {
match seek {
io::SeekFrom::Start(s) => self.0 = s,
io::SeekFrom::Current(s) => self.0 = (self.0 as i64 + s) as u64,
io::SeekFrom::End(s) => self.0 = (self.0 as i64 + s) as u64,
}

Ok(self.0)
}
}

Expand Down Expand Up @@ -86,7 +92,7 @@ pub fn criterion_benchmark(c: &mut Criterion) {
builder.add(data_3, "d3").unwrap();

// Dump data
builder.dump(&mut target, &b_config).unwrap();
black_box(builder.dump(&mut target, &b_config).unwrap());
}

// Load data
Expand All @@ -97,18 +103,18 @@ pub fn criterion_benchmark(c: &mut Criterion) {
throughput_group.bench_function("Archive::fetch(---)", |b| {
// Load data
b.iter(|| {
archive.fetch("d1").unwrap();
archive.fetch("d2").unwrap();
archive.fetch("d3").unwrap();
black_box(archive.fetch("d1").unwrap());
black_box(archive.fetch("d2").unwrap());
black_box(archive.fetch("d3").unwrap());
});
});

throughput_group.bench_function("Archive::fetch_mut(---)", |b| {
// Load data
b.iter(|| {
archive.fetch_mut("d1").unwrap();
archive.fetch_mut("d2").unwrap();
archive.fetch_mut("d3").unwrap();
black_box(archive.fetch_mut("d1").unwrap());
black_box(archive.fetch_mut("d2").unwrap());
black_box(archive.fetch_mut("d3").unwrap());
});
});

Expand All @@ -117,7 +123,7 @@ pub fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("Archive::LOAD_NEW", |b| {
// How fast it takes to load a new archive
b.iter(|| {
Archive::with_config(&mut target, &h_config).unwrap();
black_box(Archive::with_config(&mut target, &h_config).unwrap());
})
});
}
Expand Down
11 changes: 4 additions & 7 deletions vach-cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "vach-cli"
version = "0.5.0"
version = "0.5.5"
edition = "2021"
authors = [
"Jasper Fortuin <zeskeertwee@gmail.com>",
Expand All @@ -19,15 +19,12 @@ name = "vach"
path = "src/main.rs"

[dependencies]
indicatif = "0.17.6"
vach = { version = "0.5.5", features = ["all"] }
clap = "3.1.15"
indicatif = "0.17.6"
anyhow = "1.0.57"
tabled = "0.14.0"
tabled = "0.15.0"
log = "0.4.17"
walkdir = "2.3.2"
pretty_env_logger = "0.5.0"
term_size = "0.3.2"

[dependencies.vach]
path = "../vach"
features = ["compression", "multithreaded", "crypto"]
15 changes: 8 additions & 7 deletions vach/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "vach"

# NOTE: Make sure spec.txt and vach::VERSION constants are all synced up
version = "0.5.0"
version = "0.5.5"

edition = "2021"
authors = [
Expand All @@ -19,28 +19,29 @@ readme = "README.md"
include = ["Cargo.toml", "Cargo.lock", "src/*"]

[dependencies]
thiserror = "1.0.31"
thiserror = "1.0.56"

# Authentication dependencies
ed25519-dalek = { version = "2.0.0", optional = true }
ed25519-dalek = { version = "2.1.0", optional = true }
rand = { version = "0.8.5", optional = true }

# Encryption dependencies
aes-gcm = { version = "0.10.2", features = ["aes"], optional = true }
aes-gcm = { version = "0.10.3", features = ["aes"], optional = true }

# Compression dependencies
lz4_flex = { version = "0.11.1", optional = true, default-features = false, features = [
"frame",
] }
snap = { version = "1.0.5", optional = true }
brotli = { version = "3.3.4", optional = true }
snap = { version = "1.1.1", optional = true }
brotli = { version = "3.4.0", optional = true }

# Multithreaded features
parking_lot = { version = "0.12.1" }
rayon = { version = "1.5.2", optional = true }
rayon = { version = "1.8.0", optional = true }

[features]
default = ["builder", "archive"]
all = ["default", "compression", "crypto", "multithreaded"]

archive = []
builder = []
Expand Down
2 changes: 0 additions & 2 deletions vach/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@
<a href="https://docs.rs/vach">Docs</a> | <a href="https://github.com/zeskeertwee/vach">Repo</a>
</p>

---

`vach`, pronounced like "puck" but with a "v", is an archiving and resource transmission format. It was built to be secure, contained and protected. It was, in fact, designed by the [SCP](https://en.wikipedia.org/wiki/SCP_Foundation) to keep your anomalous assets compact and secure during transmission. A big benefit of `vach` is the fine grained control it grants it's users, as it allows for per-entry independent configuration. `vach` also has in-built support for multiple compression schemes (LZ4, Snappy and Brolti), [data signing](https://github.com/dalek-cryptography/ed25519-dalek), leaf [bitflags](https://docs.rs/vach/latest/vach/archive/struct.Flags.html), [encryption](https://docs.rs/aes-gcm/latest/aes_gcm/) and some degree of archive customization. Check out the `vach` spec at **[spec.txt](https://github.com/zeskeertwee/vach/blob/main/spec/main.txt)**. Any and *all* help will be much appreciated, especially proof reading the docs and code review.

## ⛏ Who is this for?
Expand Down
2 changes: 2 additions & 0 deletions vach/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ It was built to be secure, contained and protected. A big benefit of `vach` is t
- `multithreaded`: Pulls [rayon](https://crates.io/crates/rayon) as a dependency and adds [`Send`] as a trait bound to many generic types. This allows for the auto-parallelization of the `Builder::dump(---)` function.
- `compression`: Pulls `snap`, `lz4_flex` and `brotli` as dependencies and allows for compression in `vach` archives.
- `crypto`: Enables encryption and authentication functionality by pulling the `ed25519_dalek` and `aes_gcm` crates
- `default`: Enables the `archive` and `builder` features.
- `all`: Enables all the features listed above

### 🀄 Show me some code _dang it!_

Expand Down
Loading