8000 bpo-42213: Remove redundant cyclic GC hack in sqlite3 by erlend-aasland · Pull Request #26462 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

bpo-42213: Remove redundant cyclic GC hack in sqlite3 #26462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

bpo-42213: Remove redundant cyclic GC hack in sqlite3 #26462

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
70511f3
bpo-42213: Remove redundant cyclic GC hack in sqlite3
May 29, 2021
6cb435b
Split open tests into their own test case, and cleanup more explicitly
Jun 2, 2021
1e3f589
clean up test_trace_callback_content explicitly
Jun 2, 2021
5ff1a80
Document workarounds
Jun 2, 2021
866e226
Use test.support.gc_collect iso. gc.collect
Jun 2, 2021
063878c
Revert gc workarounds
Jun 2, 2021
b53aacb
Address review: Explicitly close opened database files
Jun 2, 2021
e0ffe76
Try to harden connection close
Jun 2, 2021
4ebb593
sqlite3_close_v2() always returns SQLITE_OK
Jun 2, 2021
7d8014a
Check connection on __enter__
Jun 2, 2021
1b23df1
Add tests that exercise stuff against a closed database
Jun 2, 2021
d31403e
Harden test clean up
Jun 2, 2021
aba8dd7
Add managed resource helper
Jun 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
sqlite3_close_v2() always returns SQLITE_OK
  • Loading branch information
Erlend E. Aasland committed Jun 2, 2021
commit 4ebb593d5f95549223faa8804b64b617e0217af9
17 changes: 5 additions & 12 deletions
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,17 +250,14 @@ connection_clear(pysqlite_Connection *self)
return 0;
}

static int
static void
connection_close(pysqlite_Connection *self)
{
int rc = SQLITE_OK;

if (self->db) {
rc = sqlite3_close_v2(self->db);
int rc = sqlite3_close_v2(self->db);
assert(rc == SQLITE_OK);
self->db = NULL;
}

return rc;
}

static void
Expand All @@ -271,7 +268,7 @@ connection_dealloc(pysqlite_Connection *self)
tp->tp_clear((PyObject *)self);

/* Clean up if user has not called .close() explicitly. */
(void)connection_close(self);
connection_close(self);

tp->tp_free(self);
Py_DECREF(tp);
Expand Down Expand Up @@ -367,11 +364,7 @@ pysqlite_connection_close_impl(pysqlite_Connection *self)
Py_CLEAR(self->statements);
Py_CLEAR(self->cursors);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you use the connection after you call close()? Are we checking for NULL for these veriables everywhere else? (Cannot check myself as I'm on my phone)

Copy link
Contributor Author
@erlend-aasland erlend-aasland Jun 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll get a sqlite3.ProgrammingError. There's a lot of sanity checks in the code for this. The sqlite3 test suite runs fine, but of course, we do not have 100% code coverage yet. (Digression: see issue 43553 for improving sqlite3 code coverage.)

After close, if you try to use a cursor, fetch from a pending statement, or manipulate the connection, you'll get a sqlite3.ProgrammingError:

>>> cx = sqlite3.connect(":memory:")
>>> cu = cx.cursor()
>>> res = cu.execute("select 1")
>>> cx.close()
>>> res.fetchall()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: Cannot operate on a closed database.
>>> cu.execute("select 1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: Cannot operate on a closed database.
>>> cx.create_function("test", 1, lambda x: x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: Cannot operate on a closed database.
>>>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some more tests wrt. this: 1b23df1
Also, there's some regression tests that exercise operations on closed connections (and closed cursors).

Let me know if you want me to add more tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a strange observation:
I did a ref leak test on Windows yesterday, and it segfaulted on test_bpo31770. After some testing, I reverted the change highlighted in this conversation; I removed the three Py_CLEAR, added pysqlite_do_all_statements(self, ACTION_FINALIZE, 1) back, but kept connection_close(). That fixed the ref leak test on Windows, and the test suite still worked; it did not "leak" open test database files. So, for the new PR, I'll keep this modification. We'll see how the CI fares :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc. @vstinner

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI seems to be fine with this: erlend-aasland#9


int rc = connection_close(self);
if (rc != SQLITE_OK) {
_pysqlite_seterror(self->db);
return NULL;
}
connection_close(self);
}

Py_RETURN_NONE;
Expand Down
0