You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
119 lines
5.8 KiB
119 lines
5.8 KiB
# leanify-many
|
|
|
|
Spawn [leanify] as subprocesses to enable parallel & concurrent compression.
|
|
|
|
[leanify]: https://github.com/JayXon/Leanify
|
|
|
|
## About
|
|
[leanify] is a nice tool that losslessly compresses files for you, it works on a large number of file types.
|
|
However, it has a couple drawbacks; the main one is it is entirely single threaded, and each file blocks without letting the others also be operated on.
|
|
|
|
I'm too stupid to try to fork it and add concurrent processing myself, so I made this hack that just spawns child processes of `leanify` on the file-list instead. This means we can run `leanify` in parallel for a bunch of files easily, with options to set the max (or have no max) number of `leanify` operations allowed to happen at once.
|
|
|
|
This can greatly improve speed when using on multiple files.
|
|
|
|
## How to use
|
|
Usually `leanify-many` can determine where `leanify` is installed by checking your `PATH`.
|
|
|
|
``` shell
|
|
$ leanify-many *.jpg *.png *.gif
|
|
```
|
|
|
|
If not, you can tell it by setting the `LEANIFY` environment variable to the path of the binary
|
|
|
|
``` shell
|
|
$ LEANIFY=~/bin/leanify leanify-many .
|
|
```
|
|
|
|
### Changing number of children
|
|
|
|
By default, there is no limit to the number of children spawned. This can cause "too many open files" errors if used with a lot of files, and can also cause slowdowns when trying to spawn many more processes than the CPU has processors.
|
|
|
|
You can set the number of children with `--max-children <number>`, and/or you can pass `-m` to limit the max number of children to the number of processors the system currently has.
|
|
|
|
### Recursion
|
|
|
|
`leanify-many` handles resolving pathnames before sending them to `leanify`, by default there is a max recursion depth of 1 (so, no recursion). Passing a directory instead of the list of files to `leanify-many` does not count towards this limit, so `leanify-many dir/` and `leanify-many dir/*` are the same (except the files she shell excludes from `*`).
|
|
|
|
You can specify the max recursion depth with `--recursive <number>`, or set it to unlimited with `-r`.
|
|
|
|
### Passing arguments to the children
|
|
|
|
You can also pass arguments through to the children with `--passthrough` or `-p`, and also with the ones aliased here.
|
|
|
|
The full list of arguments can be found [here](https://github.com/JayXon/Leanify#usage).
|
|
|
|
| Option | Description |
|
|
|------------------------------|----------------------------------------------|
|
|
| `-i`, `--iteration <number>` | Number of iterations |
|
|
| `-d`, `--max_depth <number>` | Max depth of leanify's archive compressesion |
|
|
| `-f`, `--fastmode` | Fast mode, no recompression |
|
|
| `-q`, `--quiet` | No output to stdout |
|
|
| `-v`, `--verbose` | Verbose output |
|
|
| `--keep-exif` | Do not remove exif |
|
|
|
|
The following are the same:
|
|
|
|
``` shell
|
|
$ leanify-many --passthrough '--max_depth 10 -f -v' -m files/
|
|
$ leanify-many --max_depth 10 -f -v -m files/
|
|
```
|
|
|
|
But, the following is not:
|
|
|
|
``` shell
|
|
$ leanify-many --passthrough '--max_depth 0 -f -v' -m files/
|
|
$ leanify-many --max_depth 0 -f -v -m files/
|
|
```
|
|
|
|
Without the `checked_pass` feature enabled, the top command will cause all subprocesses to silently error, since `0` is not a valid argument for `--max_depth`. The same is true of any invalid arguments passed to `--passthrough`.
|
|
The other aliases, however, make these checks for validity before starting the subprocesses.
|
|
|
|
If you want to pass something 'as is' to the subprocesses, do not enable the `checked_pass` feature flag when building, and pass the args with `--passthrough` instead of using the aliases. Alternatively, if the `checked_pass` feature was enabled when the binary was compiled (check the output of `--help`, and see below for features legend), pass `-P` instead of `-p` or `--passthough`.
|
|
|
|
### Misc.
|
|
|
|
| Option | Description | Notes |
|
|
|-----------------|-------------------------------------------------------------|-----------------------------|
|
|
| `--no-progress` | Do not display progress bar | Requires `progress` feature |
|
|
| `--colour` | Always display colour | Requires `colour` feature |
|
|
| `--no-colour` | Never display colour | Requires `colour` feature |
|
|
| `-` | Treat all remaining arguments as inputs, stop parsing flags | |
|
|
|
|
## Optional features
|
|
There are a few compile-time features that can be enabled/disabled for additional functionality.
|
|
|
|
| Name | Description | Default |
|
|
|----------------|--------------------------------------------------------|---------|
|
|
| `splash` | Show program information when printing help | On |
|
|
| `colour` | Enable colouring of certain outputs, like warnings | On |
|
|
| `progress` | Enable progress bar | On |
|
|
| `threads` | Enable threaded scheduler | Off |
|
|
| `checked_pass` | Check the arguments sent to leanify with `passthrough` | Off |
|
|
|
|
When building with Rust nightly, some other optimisations and features will be present.
|
|
|
|
### Legend
|
|
|
|
In `--help` the compiled-with features are listed as so:
|
|
* `+feature` means feature enabled
|
|
* `-feature` means feature disabled
|
|
|
|
When printing with colour:
|
|
* Red means enabled by default
|
|
* Bright red means enabled specifically
|
|
* Blue means disabled by default
|
|
* Bright blue means disabled specifically
|
|
|
|
# Additional
|
|
|
|
I included a Gentoo ebuild for [leanify] in [other]
|
|
|
|
[other]: ./other/Leanify
|
|
|
|
And also an ebuild for `leanify-many` in [other](./ebuild/leanify-many)
|
|
|
|
# License
|
|
|
|
GPL'd with <3
|