Added feature `mapped-file`: Attempt to `mmap()` input where possible to avoid reads. (NOTE: This should be replaced with a cfg-target-unix check, for now it is a default feature that can be disabled.)

Started module `part`: Partitioning of memory areas (halve-then-`{r,m}emchr("\n")` pivot calculation) (parallelised via rayon + crossbeam-queue (+maybe tokio_uring?) if `threads` feature enabled.

Fortune for reverse's current commit: Middle blessing − 中吉
refactor-search-capext
Avril 9 months ago
parent 5144539191
commit ff898fc9b3
Signed by: flanchan
GPG Key ID: 284488987C31F630

@ -7,7 +7,7 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[features]
default = []
default = ["mapped-file"]
unstable = ["parking_lot?/nightly", "memchr/compiler_builtins", "crossbeam-queue?/nightly", "futures?/unstable", "color-eyre/track-caller"]
@ -32,9 +32,7 @@ threads-async = ["threads", "dep:futures", "dep:tokio-uring", "dep:tokio-stream"
#XXX ["output-lines", "buffer-output", "ignore-output-errors", "ignore-invalid-args"]
#TODO: Branch repo, and begin refactor:
#
#TODO: Move these features to Clap runtime arg parse options; Major refactor, bump to 0.5.0 or to 1.0.0[r0]
#TODO: Move these features to Clap runtime arg parse options.
#TODO: Add feature 'mapped-file': Attempt to map input file and process data backwards from end if possible.
#TODO: Add feature 'threads': Use rayon to parallelise line-searching in buffer (or mapped file ^) (partition in half, move partition line to closest {r,}memchr('\n'); repeat N times recursively on each side (parallel) where N is number of logical CPU cores and/or to get the chunks below/in a certain size range.

@ -1,4 +1,6 @@
mod part;
//#[inline]
fn reverse<T>(slice: &mut [T])
{

@ -0,0 +1,3 @@
//! Partitioning even areas by delimitor byte.
use super::*;
Loading…
Cancel
Save