Avril
b47a27c60f
(partial-merge from (probably) now-defunct branch `hugetlb`.)
...
Fortune for collect's current commit: Half blessing − 半吉
2 years ago
Avril
35c6cabce3
memfile::hp: Changed `get_masks()` to return an iterator of `SizedMask` instead of just `Mask` to retain the size of the huge-page itself in bytes (for `ftruncate()` calls, and `mmap()` calls.)
...
memfile::hp: Switched from `fallocate()` to `ftruncate()`: This system-call actually does allow the size to be set, as long as it is a multiple of the huge-page size used to create it.
memfile:: Added `RawFile::truncate_size()`: Set size using `ftruncate()` instead of `fallocate()`: This difference matters a lot with hugetlbfs-backed FDs.
Fortune for collect's current commit: Blessing − 吉
2 years ago
Avril
dbf2dbffde
Removed `hugetlb` feature flag: It cannot be used for our use-case.
...
The branch is merged anyway, just in case any of the extra code used in it can be extracted later.
Fortune for collect's current commit: Future small blessing − 末小吉
2 years ago
Avril
8ea3a23e27
Using huge-pages for this is folly from the start: For one, we never actually map the fallocate()"d memory. And two... hugetlbfs does not support write()s in any way, splice()s, send_file()s... It only supports read(), and mmap() (its primary use-case...); which isn"t relevant for us.
...
This branch will be merged but the `hugetlb` feature flag will be removed.
This matter is now closed.
Fortune for collect's current commit: Future blessing − 末吉
2 years ago
Avril
872ea74421
It seems creating HUGETLB memory files either just doesn"t work or changes their behaviour so that any (or at least, small arbitrary) writing or fallocate()ing to them fails... Read up on MFD_HUGETLB more then re-do a test like `memfd_create_wrapper()` to find out why... and if it depends on the MAP_HUGE_ mask, and if so, find one that works... (We know when masks are invalid, since the error message is different. The masks collected via `get_masks()` *are* valid for this system, they just prevent the fd from being any way useful.)
...
Fortune for collect's current commit: Blessing − 吉
2 years ago
Avril
bdfd0a6268
memfile::hp: Added `PartialEq` impl for `c_int` (checks `MAP_HUGE` from `.raw()`), and `c_uint` (checks `memfd_create()` useable constant from `.mask()`.)
...
memfile::hp: Added tests for `get_masks()`
Fortune for collect's current commit: Great blessing − 大吉
2 years ago
Avril
573845a667
memfile::hp: Added `get_masks()`: Returns an iterator over all `MAP_HUGE` masks found on system.
...
Added (default) feature-flag for `hugetlb`: Will use `memfile::hp` for `memfile` enabled feature.
Added tests for both checked and unchecked `Mask` creation.
Added feature `hugepage-checked-masks`: Use checked `Mask` creation in `get_masks{,_in}()`. Only useful for debugging or for ensuring no invalid values slip past mask creation. Disabled by default and offers not much of an improvement other than a bit more safety (integer arithmatic checking).
Fortune for collect's current commit: Half curse − 半凶
2 years ago
Avril
9b4bb475c0
Added extension methods for flattening `eyre::Result<eyre:Result<T>>`s and related constructs.
...
Fortune for collect's current commit: Curse − 凶
2 years ago
Avril
e7b96af012
memfile::hp: Added test for `Mask`"s `.raw()` (`MAP_HHUGE_` flag generation.)
...
Fortune for collect's current commit: Future small blessing − 末小吉
2 years ago
Avril
c4f73ccfa0
memfile::hp: `find_size_bytes()` fixed and tested; should change function to return `eyre::Result<usize>` instead of `Option<usize>` considering how many different failure-paths exist.
...
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
9c18a5b940
memfile::hp: Added `Mask`: Converts bytes into a suitable `MAP_HUGE_` constant via its `.raw()` method, and a suitable flag for `memfd_create()` via its `.mask()` method.
...
The above is all either `const fn` or checked, and can be bitwise OR"d together, producing a higher `Mask`, or bitwise OR"d with an arbitrary `c_uint` to produce a valid mask for `memfd_create()` (in the latter case, the OR with MFD_HUGETLB is unneeded, as `Mask` applies this itself.)
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
b882f0ae97
Completed `hp::find_size_bytes()`, and added const-generated lookup table for non "k" seperators.
...
TODO: Add a test for this function.
Fortune for collect's current commit: Middle blessing − 中吉
3 years ago
Avril
d46efebbf7
Added huge-page mask calculation algorithm to comment of module file `memfile::hp`.
...
Fortune for collect's current commit: Middle blessing − 中吉
3 years ago
Avril
b1a64475f6
Merge branch 'master' into safe-memfd: memfile: Merged `RawFile::try_link_from()`: Analogue to `try_link_to()`.
3 years ago
Avril
c6032467fa
memfile: Added `RawFile::try_link_from()`: Analogue to `try_link_to()`. Instrumented and added panicking versions of both functions. Added `clone_from()` impl that uses `try_link_from()` (i.e. `dup2()`)
...
Fortune for collect's current commit: Future small blessing − 末小吉
3 years ago
Avril
59363848a2
Started: `memfile::hp`: `MFD_HUGETLB` can be used when a size is known to exceed into the boundary of a hugepage. Currently in the process of writing function to collect all possible hugepage sizes and generate the correct flag masks for `memfd_create(, MFD_HUGETLB | ...)` for each one; and the smallest one that contains the known size can be used for the `memfd_create()` call (if the size fits any at all.)
...
TODO: This will be a seperate comptime feature: `hugetlb`: Extend `memfile` with the ability to allocate apporpirately sized huge-pages when input size is known to extend into them.
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
bbed62e738
Version bump (revision): v1.0.2
...
Fortune for collect's current commit: Middle blessing − 中吉
3 years ago
Avril
2c839da13b
Bumped stackalloc version: Use PLT for trampoline: Causing random linker errors on some machines.
...
Fortune for collect's current commit: Small curse − 小凶
3 years ago
Avril
0262ca88d9
Added TODOs for specific bugs (and impl related ideas for how to go about preventing/warning in the event of `mode-memfd` system OOM) ; moved some RawFd-related functions into a submodule
...
Fortune for collect's current commit: Small curse − 小凶
3 years ago
Avril
8dd7ddaed8
Version bump v1.0.1
...
Fortune for collect's current commit: Curse − 凶
3 years ago
Avril
9ecd0ea3d8
Fixed silly bug preventing building on rust-stable.
...
Fortune for collect's current commit: Small blessing − 小吉
3 years ago
Avril
1e207ca453
Create LICENSE
3 years ago
Avril
9125199d80
Added Cargo keywords (tags)
...
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
225a0a08f0
Fix README
...
Fortune for collect's current commit: Half blessing − 半吉
3 years ago
Avril
8fe92b0951
Version 1.0.0!
...
Switched to Rust implementation: It"s faster, more configurable, and better maintainable.
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
f05c1466c7
Added README.md
...
Fortune for collect's current commit: Future small blessing − 末小吉
3 years ago
Avril
8390072309
Reached version 1.0.0!
...
Added `mode-*` feature flags for specific working methods (default: `mode-memfile` [+logging])
Fortune for collect's current commit: Half curse − 半凶
3 years ago
Avril
515a63c3a2
Fixed bug with `memfile` feature not setting stream length properly without `memfile-preallocate`
...
Fortune for collect's current commit: Future small blessing − 末小吉
3 years ago
Avril
23d022b5fe
Added working memfile implementation
...
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
0628853b43
Working memfile implementation
...
Fortune for collect's current commit: Future blessing − 末吉
3 years ago
Avril
edb57c7c95
memfile: Disambiguated memfile errors resulting in `fallocate()` sizes being too large or the call itself failing.
...
Fortune for collect's current commit: Half blessing − 半吉
3 years ago
Avril
b3dbb30deb
Fix imports for when feature `memfile` is disabled.
...
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
ed957bcec8
feature memfile: added `RawFile::open_mem()`
...
Needs testing, and the XXX comment about `memfd_create()` name lifetimes also needs testing. (If they do need to be static; `stackalloc` as a dependancy can be removed entirely, then: `DEFAULT_NAME` will be changed to a CString, function parameter `name` and `error::MemfileCreationStep::Create(name)` will be changed to `Option<&"static CStr>`.
Fortune for collect's current commit: Future blessing − 末吉
3 years ago
Avril
65c297b228
Started adding `memfile` feature: Use a `memfd_create()`d in-memory temporary file.
...
Fortune for collect's current commit: Curse − 凶
3 years ago
Avril
80a2d65262
Made spantrace capture optional at the compilation level.
...
Added `logging` (default) feature for enabling/disabling spantrace captures entirely on build.
Fortune for collect's current commit: Half curse − 半凶
3 years ago
Avril
3af34dc78f
Better documented `jemalloc` feature.
...
Unsure if the massive page mapping is an issue or not...
Fortune for collect's current commit: Blessing − 吉
3 years ago
Avril
cc37f604f1
Fixed tracing causing huge unneeded bottleneck in non `bytes` builds.
...
Fortune for collect's current commit: Future small blessing − 末小吉
3 years ago
Avril
bea5cda4a1
Fixed not copying to buffer when not using feature `bytes`.
...
Fortune for collect's current commit: Curse − 凶
3 years ago
Avril
1cdd5b2187
Made `bytes` an optional feature.
...
Fortune for collect's current commit: Future small blessing − 末小吉
3 years ago
Avril
4cf1c88a3f
Update gitignore
...
Fortune for collect's current commit: Middle blessing − 中吉
3 years ago
Avril
a5ad02a6de
Working primitive, slow, prototype.
...
Fortune for collect's current commit: Small blessing − 小吉
3 years ago
Avril
2aa8c832de
initial commit: must figure out how splice() works before we continue here.
...
Fortune for collect's current commit: Curse − 凶
3 years ago