gh-135386: Fix "unable to open database file" errors on readonly DB #135566
Conversation
…M errors on readonly DB
|
All commit authors signed the Contributor License Agreement. |
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
There was a problem hiding this comment.
Please add a test case. You can refer to the devguide for how to do that.
Misc/NEWS.d/next/Library/2025-06-16-15-00-13.gh-issue-135386.lNrxLc.rst
Outdated
Show resolved
Hide resolved
707f4a3 to
779faf2
Compare
779faf2 to
3e3a3ba
Compare
|
No need to force-push, we squash at the end. |
Got it, thanks for the reminder! |
|
FYI, tests are failing on Windows. |
Thanks for the suggestion. I've skipped the test on Windows using |
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Sure, that makes sense — I'd be happy to look into making the test work on Windows as well. Also, thanks for the suggestions — I've adopted the URI change and simplified the flag check as suggested. : ) |
I've updated the test so it no longer skips on Windows — instead, it now explicitly closes the database before setting read-only permissions, and restores the original permissions after the test. |
Lib/test/test_dbm_sqlite3.py
Outdated
| def test_readonly_open_without_wal_shm(self): | ||
| wal_path = self.filename + "-wal" | ||
| shm_path = self.filename + "-shm" | ||
|
|
||
| for suffix in wal_path, shm_path: | ||
| os_helper.unlink(suffix) | ||
|
|
||
| try: | ||
| self.db.close() | ||
| except Exception: | ||
| pass | ||
|
|
||
| os.chmod(self.filename, stat.S_IREAD) | ||
|
|
||
| db = dbm_sqlite3.open(self.filename, "r") | ||
| try: | ||
| self.assertEqual(db[b"key1"], b"value1") | ||
| self.assertEqual(db[b"key2"], b"value2") | ||
| finally: | ||
| db.close() | ||
|
|
||
| os.chmod(self.filename, stat.S_IWRITE) |
There was a problem hiding this comment.
IIUC correctly, this change makes it so that the WAL/SHM files are no longer created in read-only mode? In that case, we could simplify the test to simply open the database in read-only mode and check that the auxiliary files are not written. I'm thinking something like this:
class Immutable(unittest.TestCase):
def setUp(self):
self.filename = os_helper.TESTFN
self.db = dbm_sqlite3.open(self.filename, "r")
def tearDown(self):
self.db.close()
def test_readonly_open_without_wal_shm(self):
wal_path = self.filename + "-wal"
shm_path = self.filename + "-shm"
self.assertFalse(os.path.exists(wal_path))
self.assertFalse(os.path.exists(shm_path))Or does this also affect the database file itself? i.e. does passing immutable=1 allow opening read-only files whereas otherwise it would be impossible?
There was a problem hiding this comment.
Thanks, I’ve replaced the original test with your suggested version in a new Immutable test class. It’s much cleaner and focused.
There was a problem hiding this comment.
Or does this also affect the database file itself? i.e. does passing
immutable=1allow opening read-only files whereas otherwise it would be impossible?
Yes, exactly — without immutable=1, even in mode=ro, SQLite may still attempt operations that require write access (like writing lock files or checking for journal/WAL state), and will fail if the database file is read-only or in a read-only directory.
Passing immutable=1 tells SQLite to treat the database as entirely static and read-only, which avoids those operations and allows the file to be opened successfully even with 0444 permissions or from a read-only mount.
|
In order to use https://www.sqlite.org/uri.html#recognized_query_parameters says
|
That's a good point — I agree That said, in the context of The use of If needed, we could consider adding a separate flag like 'ri' in the future for stricter use cases, but I think using |
What
This PR adds the SQLite URI parameter
immutable=1when opening a database in read-only mode inLib/dbm/sqlite3.py. This explicitly informs SQLite that the database is read-only and avoids attempts to create or write to auxiliary WAL/SHM files.Why
Without this flag, opening a read-only SQLite database may fail with errors such as "unable to open database file" when the WAL or SHM files cannot be created due to filesystem permissions or other restrictions. This is especially common when the database file is read-only and the environment prevents creation of additional files.
Adding
immutable=1allows SQLite to operate correctly without needing to create WAL/SHM files, thus preventing these errors.Where
The change is made in the
_Database.__init__constructor inLib/dbm/sqlite3.py. The URI used to open the database connection is appended with&immutable=1only if theflagis"ro"(read-only).Related issue
#135386
This is my first contribution to CPython. I appreciate the opportunity and look forward to feedback from the community. Thank you!