Migrate to modernc.org/sqlite for CGO-free builds #727
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Migrates the main litestream binary from github.com/mattn/go-sqlite3 to modernc.org/sqlite to enable CGO-free builds and automatic cross-platform compilation. This enables signed macOS releases without cross-compilation toolchains. Key changes: - Replace mattn/go-sqlite3 with modernc.org/sqlite for main binary - Implement PERSIST_WAL using FileControl API (full feature parity) - Add build tags to conditionally compile VFS support (requires CGO) - Keep VFS functionality with mattn/go-sqlite3 for compatibility - Update all SQLite driver references from "sqlite3" to "sqlite" The VFS feature (cmd/litestream-vfs) intentionally remains CGO-dependent as it's experimental and may require write support in the future. Fixes #723 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
The modernc.org/sqlite package auto-registers as 'sqlite' driver. Removed custom driver registration and switched all connections to use the standard 'sqlite' driver name to fix CI build failures.
The cmd/litestream-vfs/main.go file needs a main function for non-CGO builds. Added a stub that panics with an informative message when the VFS command is built without the required build tags.
The VFS functionality depends on sqlite3vfs which requires the C SQLite library. Since we've migrated to modernc.org/sqlite (pure Go), VFS must be explicitly enabled with the 'vfs' build tag to avoid linking errors. This change: - Requires -tags vfs to enable VFS support - Prevents sqlite3vfs from being linked in default builds - Fixes CI build failures from missing SQLite C library - Maintains VFS compatibility when explicitly requested
- Added instructions for testing VFS builds with CGO and build tags - Added warnings about not committing binary files - Updated .gitignore to exclude common binary names - Documented different build configurations to test before commits
MIGRATION_TO_MODERNC_SQLITE.md was for internal planning only
The 'cgo' build tag was redundant since: - VFS requires CGO due to sqlite3vfs dependency - The main binary is now pure Go with modernc.org/sqlite - The 'vfs' tag expresses the actual intent This simplifies the build tags to focus on what we're building (VFS) rather than implementation details (CGO requirement).
- Removed redundant binary exclusions from .gitignore - Updated CLAUDE.md to always build binaries into bin/ directory - The bin/ directory is already gitignored, preventing accidental commits
The main() function already exists in litestream-vfs.go with the SQLITE3VFS_LOADABLE_EXT build tag. The stub was causing 'unreachable code' and wasn't needed - the package is invisible to builds without the appropriate build tags.
The VFS stub was not needed - VFS types are only referenced in files that have the appropriate build tags. Without the 'vfs' tag, the VFS code doesn't exist and there are no references to it.
|
✅ Manual integration tests have been run by @corylanou |
- Removed empty init() function that only contained a comment - Removed redundant connection comment (PERSIST_WAL handling is documented in setPersistWAL) - Kept original comments unmodified for clarity
- Revert WAL removal comment to original simple form - Focus on functional descriptions rather than implementation details
benbjohnson
approved these changes
Sep 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR migrates the main litestream binary from
github.com/mattn/go-sqlite3tomodernc.org/sqliteto enable CGO-free builds and automatic cross-platform compilation, particularly for signed Silicon Mac releases.Changes
Key Benefits
Testing
Notes
cmd/litestream-vfs) intentionally remains CGO-dependent as it's experimental and primarily for Fly.io projectsMIGRATION_TO_MODERNC_SQLITE.mdfor detailed migration notesFixes #723
🤖 Generated with Claude Code