@@ -109,6 +109,77 @@ pre-commit install
109
109
- Ensure existing tests pass before submitting PRs
110
110
- Integration tests require specific environment setup (see test files for details)
111
111
112
+ #### Testing the 1GB Lock Page Edge Case
113
+
114
+ SQLite reserves a special "lock page" at exactly 1GB (1,073,741,824 bytes)
115
+ that cannot be written to due to file locking mechanisms, particularly on
116
+ Windows. Litestream must skip this page during replication.
117
+
118
+ ** Why Test Databases >1GB?**
119
+
120
+ - Ensures lock page skipping logic works correctly
121
+ - Verifies replication doesn't break when crossing the 1GB boundary
122
+ - Tests that restoration properly handles databases with lock pages
123
+ - Validates LTX format handling of non-data pages
124
+
125
+ ##### Lock Page Calculation
126
+
127
+ The lock page number varies by database page size:
128
+
129
+ - 4KB pages: page 262145 (most common, ~ 99% of databases)
130
+ - 8KB pages: page 131073
131
+ - 16KB pages: page 65537
132
+ - 32KB pages: page 32769
133
+
134
+ Formula: ` LockPgno = (0x40000000 / pageSize) + 1 `
135
+
136
+ ##### Testing Approach
137
+
138
+ 1 . Create test databases larger than 1GB:
139
+
140
+ ``` bash
141
+ # Create a test database with specific page size
142
+ sqlite3 test.db " PRAGMA page_size=4096; CREATE TABLE t(data BLOB);"
143
+
144
+ # Fill database to exceed 1GB (adjust iterations as needed)
145
+ for i in {1..300000}; do
146
+ sqlite3 test.db " INSERT INTO t VALUES(randomblob(4000));"
147
+ done
148
+
149
+ # Verify size and page count
150
+ sqlite3 test.db " PRAGMA page_count; PRAGMA page_size;"
151
+ ```
152
+
153
+ 1 . Test with different page sizes:
154
+
155
+ ``` bash
156
+ # Test with 8KB pages
157
+ sqlite3 test_8k.db " PRAGMA page_size=8192; CREATE TABLE t(data BLOB);"
158
+
159
+ # Test with 16KB pages
160
+ sqlite3 test_16k.db " PRAGMA page_size=16384; CREATE TABLE t(data BLOB);"
161
+ ```
162
+
163
+ 1 . Verify lock page handling in replication:
164
+
165
+ ``` bash
166
+ # Configure litestream for the test database
167
+ litestream replicate test.db s3://mybucket/test.db
168
+
169
+ # Restore and verify
170
+ litestream restore -o restored.db s3://mybucket/test.db
171
+
172
+ # Check that lock page region is handled correctly
173
+ sqlite3 restored.db " PRAGMA integrity_check;"
174
+ ```
175
+
176
+ ##### Automated Testing Considerations
177
+
178
+ - Use sparse files where filesystem supports it to avoid disk space issues
179
+ - Consider using in-memory databases with appropriate size limits for CI
180
+ - Test both snapshot and incremental WAL replication across 1GB boundary
181
+ - Verify page maps correctly skip the lock page
182
+
112
183
## Security
113
184
114
185
If you discover a security vulnerability, please:
0 commit comments