The fix involves using pysqlite_check_remaining_sql(), not only to check
for multiple statements, but now also to strip leading comments and
whitespace from SQL statements, so we can improve DML query detection.
pysqlite_check_remaining_sql() is renamed lstrip_sql(), to more
accurately reflect its function, and hardened to handle more SQL comment
corner cases.
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.
Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
The following sqlite3 features were deprecated in 3.10, scheduled for
removal in 3.12:
- sqlite3.OptimizedUnicode (gh-23163)
- sqlite3.enable_shared_cache (gh-24008)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Signed-off-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
* Map SQLITE_MISUSE to sqlite3.InterfaceError
SQLITE_MISUSE implies misuse of the SQLite C API, which, if it happens,
is _not_ a user error; it is an sqlite3 extension module error.
* Raise better errors when binding parameters fail.
Instead of always raising InterfaceError, guessing what went wrong,
raise accurate exceptions with more accurate error messages.
Unless sqlite3_blob_open() returns SQLITE_MISUSE, the error code and
message are available on the connection object. This means we have to
handle SQLITE_MISUSE error messages explicitly.
* Improve exception compliance with PEP 249
* Raise InterfaceError instead of ProgrammingError for SQLITE_MISUSE.
If SQLITE_MISUSE is raised, it is a sqlite3 module bug. Users of the
sqlite3 module are not responsible for using the SQLite C API correctly.
* Don't overwrite BufferError with ValueError when conversion to BLOB fails.
* Raise ProgrammingError instead of Warning if user tries to execute() more
than one SQL statement.
* Raise ProgrammingError instead of ValueError if an SQL query contains null characters.
* Make sure `_pysqlite_set_result` raises an exception if it returns -1.
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules.
The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).
https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.
The core of the change is in:
* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers
I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config.
The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.
The following are not changed (yet):
* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init
https://bugs.python.org/issue46541
* bpo-45512: Raise sqlite3.Connection.__init__ is called with bad isolation level
* Also explicitly test allowed isolation levels
* Use subTest for better error messages if something goes wrong
* Update Lib/test/test_sqlite3/test_dbapi.py
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
defined to get access to internal _PyObject_CallNoArgs().
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
* Constructors of subclasses of some buitin classes (e.g. tuple, list,
frozenset) no longer accept arbitrary keyword arguments.
* Subclass of set can now define a __new__() method with additional
keyword parameters without overriding also __init__().
- all callbacks are now named xxx_callback
- normalise callable naming in set_*() functions
- normalise context argument naming in callbacks
The sqlite code is being "touched" in bpo-42064 (and related issues);
this style change makes it easier to work with and review.
- acquire the GIL at the very start[1]
- release the GIL at the very end
[1] The trace callback performs a sanity check before acquiring the GIL
Automerge-Triggered-By: GH:encukou
* MemoryError is now raised instead of sqlite3.Warning when
memory is not enough for encoding a statement to UTF-8
in Connection.__call__() and Cursor.execute().
* UnicodEncodeError is now raised instead of sqlite3.Warning when
the statement contains surrogate characters
in Connection.__call__() and Cursor.execute().
* TypeError is now raised instead of ValueError for non-string
script argument in Cursor.executescript().
* ValueError is now raised for script containing the null
character instead of truncating it in Cursor.executescript().
* Correctly handle exceptions raised when getting boolean value
of the result of the progress handler.
* Add many tests covering different corner cases.
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
MemoryError raised in user-defined functions will now preserve
its type. OverflowError will now be converted to DataError.
Previously both were converted to OperationalError.
Prepare for module state:
- Add "get state by defining class" and "get state by module def" stubs
- Add AC defining class when needed
- Add state pointer to connection context
- Pass state as argument to utility functions
Automerge-Triggered-By: GH:encukou
This implies that SQLite now takes care of destroying the callback
context (the PyObject callable it has been passed), so we can strip the
collation dict from the connection object.
With this, all sqlite3 static globals have been moved to the global state.
There are a couple of global static strings left, but there should be no need for adding them to the state.
https://bugs.python.org/issue42064
Both `executescript` methods contain the same docstring typo:
_"Executes a multiple SQL statements at once."_ => _"Executes multiple SQL statements at once."_
Automerge-Triggered-By: GH:pablogsal
Erlend Egeberg Aasland
Erlend Egeberg Aasland
Serhiy Storchaka
neonene
Kalyan
Victor Stinner
Jelle Zijlstra
Petr Viktorin
Eric Snow
Dong-hee Na
Christian Heimes