8000 gh-135386: Fix "unable to open database file" errors on readonly DB by GeneralK1ng · Pull Request #135566 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

gh-135386: Fix "unable to open database file" errors on readonly DB #135566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
+93 −4
Open

gh-135386: Fix "unable to open database file" errors on readonly DB #135566

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
761f178
Fix: add immutable=1 flag for read-only SQLite access to avoid WAL/SH…
GeneralK1ng Jun 16, 2025
3e3a3ba
gh-135386: Fix dbm.sqlite3 readonly open error by using immutable=1
blurb-it[bot] Jun 16, 2025
7f988dd
Add test: readonly sqlite db without wal/shm (skip on Windows)
GeneralK1ng Jun 17, 2025
775683e
optimize(code): Update Lib/dbm/sqlite3.py
GeneralK1ng Jun 23, 2025
66899ff
optimize(code): update Lib/dbm/sqlite3.py
GeneralK1ng Jun 23, 2025
4561075
Test: make read-only dbm.sqlite3 test compatible with Windows
GeneralK1ng Jun 23, 2025
7c40be3
Refactor: remove unused _readonly flag, simplify readonly WAL/SHM test
GeneralK1ng Jun 24, 2025
759f218
Remove unused import
GeneralK1ng Jun 24, 2025
73d9e24
Test: add read-only directory and file tests for dbm.sqlite3
GeneralK1ng Jul 9, 2025
c4c8065
Test: improve read-only filesystem tests for dbm.sqlite3 per review f…
GeneralK1ng Jul 9, 2025
b586fc3
Fix test and avoid unclosed DB connection.
GeneralK1ng Jul 9, 2025
bd405f2
Skip platform-dependent write failure tests in readonly directory
GeneralK1ng Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions Lib/dbm/sqlite3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,18 +60,23 @@ def __init__(self, path, /, *, flag, mode):
# We use the URI format when opening the database.
uri = _normalize_uri(path)
uri = f"{uri}?mode={flag}"
if flag == "ro":
# Add immutable=1 to allow read-only SQLite access even if wal/shm missing
uri += "&immutable=1"

try:
self._cx = sqlite3.connect(uri, autocommit=True, uri=True)
except sqlite3.Error as exc:
raise error(str(exc))

self._readonly = (flag == "ro")
# This is an optimization only; it's ok if it fails.
with suppress(sqlite3.OperationalError):
self._cx.execute("PRAGMA journal_mode = wal")
if flag != "ro":
with suppress(sqlite3.OperationalError):
self._cx.execute("PRAGMA journal_mode = OFF")

if flag == "rwc":
self._execute(BUILD_TABLE)
if flag == "rwc":
self._execute(BUILD_TABLE)

def _execute(self, *args, **kwargs):
if not self._cx:
Expand Down
25 changes: 25 additions & 0 deletions Lib/test/test_dbm_sqlite3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import stat
import os
import sys
import unittest
from contextlib import closing
Expand Down Expand Up @@ -89,6 +91,29 @@ def test_readonly_keys(self):
def test_readonly_iter(self):
self.assertEqual([k for k in self.db], [b"key1", b"key2"])

def test_readonly_open_without_wal_shm(self):
wal_path = self.filename + "-wal"
shm_path = self.filename + "-shm"

for suffix in wal_path, shm_path:
os_helper.unlink(suffix)

try:
self.db.close()
except Exception:
pass

os.chmod(self.filename, stat.S_IREAD)

db = dbm_sqlite3.open(self.filename, "r")
try:
self.assertEqual(db[b"key1"], b"value1")
self.assertEqual(db[b"key2"], b"value2")
finally:
db.close()

os.chmod(self.filename, stat.S_IWRITE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC correctly, this change makes it so that the WAL/SHM files are no longer created in read-only mode? In that case, we could simplify the test to simply open the database in read-only mode and check that the auxiliary files are not written. I'm thinking something like this:

class Immutable(unittest.TestCase):
    def setUp(self):
        self.filename = os_helper.TESTFN
        self.db = dbm_sqlite3.open(self.filename, "r")

    def tearDown(self):
        self.db.close()

    def test_readonly_open_without_wal_shm(self):
        wal_path = self.filename + "-wal"
        shm_path = self.filename + "-shm"

        self.assertFalse(os.path.exists(wal_path))
        self.assertFalse(os.path.exists(shm_path))

Or does this also affect the database file itself? i.e. does passing immutable=1 allow opening read-only files whereas otherwise it would be impossible?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I’ve replaced the original test with your suggested version in a new Immutable test class. It’s much cleaner and focused.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or does this also affect the database file itself? i.e. does passing immutable=1 allow opening read-only files whereas otherwise it would be impossible?

Yes, exactly — without immutable=1, even in mode=ro, SQLite may still attempt operations that require write access (like writing lock files or checking for journal/WAL state), and will fail if the database file is read-only or in a read-only directory.

Passing immutable=1 tells SQLite to treat the database as entirely static and read-only, which avoids those operations and allows the file to be opened successfully even with 0444 permissions or from a read-only mount.



class ReadWrite(_SQLiteDbmTests):

Expand Down
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Library/2025-06-16-15-00-13.gh-issue-135386.lNrxLc.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix :exc:`sqlite3.OperationalError` error when using :func:`dbm.open` with a read-only file object.
Loading
0