Show HN: Typed-arrow – compile‑time Arrow schemas for Rust
github.comHi community, we just released https://github.com/tonbo-io/typed-arrow.
When working with arrow-rs, we noticed that schemas are declared at runtime. This often leads to runtime errors and makes development less safe.
typed-arrow takes a different approach:
- Schemas are declared at compile time with Rust’s type system.
- This eliminates runtime schema errors.
- And introduces no runtime overhead — everything is checked and generated by the compiler.
If you’ve run into Arrow runtime schema issues, and your schema is stable (not defined or switched at runtime), this project might be useful.
I'm pretty deep into the arrow ecosystem and have been dreaming about something like this for a while now. Nice to see!
I'm also imagining what this would look like in other languages (Java, Go, Python) - you can't do as much at compile time of course, but there are still marginal improvements to be made over the dynamism of the standard arrow libraries, when the use case benefits from it.
Cool, curious if there is any willingness or interest in adding a geoarrow feature, or generally some form of integration with that crate? (Though to be honest I'm not currently clear up to date on the stability/usability of geoarrow at this current point in time)
Sure I'd love to, could you share your practice or what you are doing on top of Parquet?
I had no idea what Arrow is: https://arrow.apache.org or arrow-rs: https://github.com/apache/arrow-rs
How is this different from https://github.com/Swoorup/arrow-convert?
typed-arrow avoids any kinds of runtime side effects: cost, errors, etc.
I congratulate you. This is a clever extension of a great framework.