reasoning: we're going to be doing a lot of heap allocations
TODO: async and sync compression of this data