Expose prelude helpers and bfuse struct fields
#73
Conversation
Making prelude public allows other developers to gain access to methods such as `bfuse::hash_of_hash` and `HashSet::xor_from` in order to manually calculate the indices in a filter.
There was a problem hiding this comment.
I think this is reasonable, but let's make a few changes:
-
Let's not expose
preludepublicly asprelude, but instead expose a moduleinternalthat exposes only certain functions that the prelude makes public (like,fingerprint!) -
I would somewhat prefer that
BinaryFuse8/16/32expose a functionattributesthat returns the data on seed/segments rather than exposing it directly on the struct. -
Let's do this for all filters, not just Fuse variants.
-
Would you be willing to add documentation on the exposed fields, and maybe a section in the library docs explaining the strategy you've used? It sounds like it might be valuable in general.
Sounds like a good idea.
So long as the function would return the exact same data, this would work!
I would if I knew better! Truthfully, I grasp enough conceptually about these types of filters to take advantage of them, but not enough to be able to explain what each field is. |
What?
In order to allow developers to be more fine-grained with their usage of this crate, I am proposing simply
pubplacements that I am currently using to expose certain methods and fields necessary to compute the hash indices manually.Why?
I found in benchmarking that I could significantly improve the lookup function in my code by only reading the initial metadata about the filter (seed and in the context of bfuse, segment information), computing the hash myself, and
seek_reading the individual bytes from the file in which I am storing the built filters to determine if a key is contained within the filter. I am not knowledgeable enough with the filters themselves to be able to properly document the segment information or the prelude functions, so that may still be needed.