Originally IO used a version of SQLite that was translated into Go and then compiled as Go code. This allowed SQLite to be used without CGO, which allowed IO to be run in lightweight Alpine containers which include musl instead of libc.
But since IO is used alongside Envoy, this benefit is moot. Envoy depends on libc and the project has no interest in supporting musl. Since Envoy will always depend on libc, it seems reasonable for IO to also.
Surveying possible sources for a Go CGO-based wrapper, the landscape is complex:
- The “OG” implementation, crawshaw/sqlite, is unmaintained.
- The original author and several collaborators are currently maintaining tailscale/sqlite, which is essentially a publicly-visible internal library used by Tailscale.
- The original implementation was forked into a “community-managed” one at go-llsqlite/crawshaw.
This third implementation was the easiest to move to from zombiezen/go-sqlite, the library that IO originally used. However, I’m concerned about the long term stability of this repo, so it seems prudent to maintain a fork, which is now at agentio/sqlite. If you use this, fork your own version! At some point I may replace it with the Tailscale-maintained library, which had enough API-level differences that I didn’t just immediately switch to it.
Pros#
- This allows IO to directly use the C SQLite and reduces compatibility risks and performance overhead of the Go reimplementation.
- It’s fast! After some tuning, IO became performance-limited by the SQLite write-ahead logging of traffic events. For one simple test, throughput went from under 4000 requests per second using the Go-based library to over 25000 using agentio/sqlite and SQLite 3.51.0.
Cons#
- With this, IO won’t be usable in Alpine containers.