Commit Graph

252 Commits

Author SHA1 Message Date
mpage df73179048
gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)
Most mutable data is protected by a striped lock that is keyed on the
referenced object's address. The weakref's hash is protected using the
weakref's per-object lock.
 
Note that this only affects free-threaded builds. Apart from some minor
refactoring, the added code is all either gated by `ifdef`s or is a no-op
(e.g. `Py_BEGIN_CRITICAL_SECTION`).
2024-04-08 10:58:38 -04:00
Mariusz Felisiak 1a10437a14
gh-91602: Add iterdump() support for filtering database objects (#114501)
Add optional 'filter' parameter to iterdump() that allows a "LIKE"
pattern for filtering database objects to dump.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-02-06 12:34:56 +01:00
Serhiy Storchaka 771bd3c94a
Add private _PyUnicode_AsUTF8NoNUL() function (GH-111957)
Like PyUnicode_AsUTF8(), but check for embedded null characters.
2023-11-10 21:31:36 +02:00
Victor Stinner 11e83488c5
gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"

This reverts commit d9b606b3d0.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"

This reverts commit cde1071b2a.

* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"

This reverts commit d731579bfb.

* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"

This reverts commit d8f32be5b6.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"

This reverts commit 37e4e20eaa.
2023-11-07 22:36:13 +00:00
Victor Stinner 37e4e20eaa
gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)
PyUnicode_AsUTF8() now raises an exception if the string contains
embedded null characters.
2023-10-20 20:04:27 +02:00
Victor Stinner be5e8a0103
gh-110964: Remove private _PyArg functions (#110966)
Move the following private functions and structures to
pycore_modsupport.h internal C API:

* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()

Changes:

* Python/getargs.h now includes pycore_modsupport.h to export
  functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
  is used.
* Add pycore_modsupport.h includes when a C extension uses one of
  these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
  directly or indirectly (via code generated by Argument Clinic)
  pycore_modsupport.h:

  * _csv
  * _curses_panel
  * _dbm
  * _gdbm
  * _multiprocessing.posixshmem
  * _sqlite.row
  * _statistics
  * grp
  * resource
  * syslog

* _testcapi: bad_get() no longer uses METH_FASTCALL calling
  convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
  PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
  by _testcapi sub-modules which need the internal C API
  (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
  watchers.c.
* Remove Include/cpython/modsupport.h header file.
  Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
2023-10-17 14:30:31 +02:00
Victor Stinner 79823c103b
gh-106320: Remove private _PyErr_ChainExceptions() (#108713)
Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and
_PyErr_SetFromPyStatus() functions from the public C API.

* Move the private _PyErr_ChainExceptions() and
  _PyErr_ChainExceptions1() function to the internal C API
  (pycore_pyerrors.h).
* Move the private _PyErr_SetFromPyStatus() to the internal C API
  (pycore_initconfig.h).
* No longer export the _PyErr_ChainExceptions() function.
* Move run_in_subinterp_with_config() from _testcapi to
  _testinternalcapi.
2023-08-31 13:53:19 +02:00
Erlend E. Aasland 0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Erlend E. Aasland 4116592b6f
gh-108278: Deprecate passing the three first params as keyword args for sqlite3 UDF creation APIs (#108281)
Deprecate passing name, number of arguments, and the callable as keyword
arguments, for the following sqlite3.Connection APIs:

- create_function(name, nargs, callable, ...)
- create_aggregate(name, nargs, callable)

The affected parameters will become positional-only in Python 3.15.
2023-08-28 13:32:07 +00:00
Victor Stinner b32d4cad15
gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
Change generated by the command:

sed -i -e 's!_PyLong_AsInt!PyLong_AsInt!g' \
    $(find -name "*.c" -o -name "*.h")
2023-08-25 01:01:30 +02:00
Erlend E. Aasland 1a1bfc2891
gh-105539: Emit ResourceWarning if sqlite3 database is not closed explicitly (#108015) 2023-08-22 13:10:29 +02:00
Erlend E. Aasland fd19509220
gh-108083: Don't ignore exceptions in sqlite3.Connection.__init__() and .close() (#108084)
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 11:39:12 +00:00
Erlend E. Aasland 13c36dc9ae
gh-93057: Deprecate positional use of optional sqlite3.connect() params (#107948) 2023-08-15 08:09:56 +00:00
Erlend E. Aasland 6fbaba552a
gh-107938: Synchonise the signature of of sqlite3.connect and sqlite3.Connection.__init__ (#107939) 2023-08-14 16:37:44 +02:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner c9ce983ae1
gh-106320: Remove private pylifecycle.h functions (#106400)
Remove private pylifecycle.h functions: move them to the internal C
API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No
longer export most of these functions.

Move _testcapi.test_atexit() to _testinternalcapi.
2023-07-04 09:41:43 +00:00
Victor Stinner 2e92edbf6d
gh-106320: Remove private _PyImport C API functions (#106383)
* Remove private _PyImport C API functions: move them to the internal
  C API (pycore_import.h).
* No longer export most of these private functions.
* _testcapi avoids private _PyImport_GetModuleAttrString().
2023-07-03 23:02:07 +00:00
Victor Stinner c38da1e3e1
gh-105927: Add _PyWeakref_IS_DEAD() function (#105992)
* Add _PyWeakref_IS_DEAD() internal function.
* Modify is_dead_weakref() of Modules/_weakref.c and
  _pysqlite_drop_unused_cursor_references() to replace
  PyWeakref_GET_OBJECT() with _PyWeakref_IS_DEAD().
* Replace "int i" with "Py_ssize_t i" to iterate on cursors
  in _pysqlite_drop_unused_cursor_references().
2023-06-22 21:56:44 +02:00
Erlend E. Aasland 6849acb3fe
gh-105875: Require SQLite 3.15.2 or newer (#105876)
SQLite 3.15.2 was released 2016-11-28.
2023-06-19 00:29:08 +02:00
Erlend E. Aasland a24a780d93
gh-105375: Improve error handling in sqlite3 collation callback (#105412)
Check for error after each call to PyUnicode_FromStringAndSize().
2023-06-07 13:10:28 +02:00
Eric Snow 26baa747c2
gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization (gh-104437)
With the move to a per-interpreter GIL, this check slipped through the cracks.
2023-05-15 13:59:26 -06:00
Erlend E. Aasland a05bad3254
gh-100370: fix OverflowError in sqlite3.Connection.blobopen for 32-bit builds (#103902) 2023-05-07 12:55:31 +02:00
Erlend E. Aasland bb8aa7a2b4
gh-103489: Add get/set config methods to sqlite3.Connection (#103506) 2023-04-26 19:57:48 +00:00
Erlend E. Aasland 222c63fc6b
gh-103015: Add entrypoint keyword param to sqlite3.Connection.load_extension (#103073) 2023-04-26 21:22:03 +02:00
Irit Katriel 2db23d10bf
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Modules/) (#102196) 2023-02-24 21:43:03 +00:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Erlend E. Aasland c95f554a40
gh-83638: Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour (#93823)
Introduce the autocommit attribute to Connection and the autocommit
parameter to connect() for PEP 249-compliant transaction handling.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Géry Ogam <gery.ogam@gmail.com>
2022-11-12 23:44:41 +01:00
Erlend Egeberg Aasland a3d4d15f53
gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory (#95146)
This PR partially reverts gh-24421 (PR) and fixes the remaining concerns
given in gh-93044 (issue):

- keyword arguments are passed as positional arguments to factory()
- if an argument is not passed to sqlite3.connect(), its default value
  is passed to factory()

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-07-23 09:51:28 +02:00
Erlend Egeberg Aasland 6446592c89
gh-93925: Improve clarity of sqlite3 commit/rollback, and close docs (#93926)
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
2022-06-19 21:17:28 +02:00
Erlend Egeberg Aasland 4e9fa71d7e
gh-93829: In sqlite3, replace Py_BuildValue with faster APIs (#93830)
- In Modules/_sqlite/connection.c, use PyLong_FromLong
- In Modules/_sqlite/microprotocols.c, use PyTuple_Pack
2022-06-15 10:42:49 +02:00
Serhiy Storchaka 6fd4c8ec77
gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
2022-06-14 07:15:26 +03:00
neonene d8395eb38d
gh-92434: Silence compiler warning in Modules/_sqlite/connection.c on 32-bit systems (#93090) 2022-06-10 10:10:04 +02:00
Serhiy Storchaka d853758092
gh-91922: Fix sqlite connection on nonstardard locales and paths (GH-92926) 2022-05-20 11:53:05 +03:00
Erlend Egeberg Aasland d9ec553194
gh-89289: Fix compiler warning in _sqlite/connection.c (#92258) 2022-05-03 22:21:56 +02:00
Erlend Egeberg Aasland c278474df9
gh-89289: Harden sqlite3.Connection init (#92214)
- Make sure SQLite resources are freed if database open fails
- Remove unneeded branches if init is aborted
2022-05-03 12:18:11 -06:00
Erlend Egeberg Aasland 1d4a9a45b7
gh-92206: Improve scoping of sqlite3 register cursor helper (#92212) 2022-05-03 06:33:24 -06:00
Erlend Egeberg Aasland 721aa96540
gh-89301: Fix regression with bound values in traced SQLite statements (#92053) 2022-05-02 08:14:35 -06:00
Erlend Egeberg Aasland c06a4ffe81
gh-69093: improve sqlite3.Connection.blobopen() error handling (GH-91571)
Unless sqlite3_blob_open() returns SQLITE_MISUSE, the error code and
message are available on the connection object. This means we have to
handle SQLITE_MISUSE error messages explicitly.
2022-04-15 09:27:39 -07:00
Erlend Egeberg Aasland ee475430d4
gh-69093: Support basic incremental I/O to blobs in `sqlite3` (GH-30680)
Authored-by: Aviv Palivoda <palaviv@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@innova.no>
Co-authored-by: palaviv <palaviv@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-14 17:02:56 -07:00
Erlend Egeberg Aasland 9ebcece82f
gh-79097: Add support for aggregate window functions in sqlite3 (GH-20903) 2022-04-11 17:55:59 -07:00
Erlend Egeberg Aasland a7551247e7
bpo-41930: Add support for SQLite serialise/deserialise API (GH-26728)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-04-05 07:15:25 -07:00
Erlend Egeberg Aasland 4674fd4e93
bpo-44859: Raise more accurate exceptions in `sqlite3` (GH-27695)
* Improve exception compliance with PEP 249
* Raise InterfaceError instead of ProgrammingError for SQLITE_MISUSE.
  If SQLITE_MISUSE is raised, it is a sqlite3 module bug. Users of the
  sqlite3 module are not responsible for using the SQLite C API correctly.
* Don't overwrite BufferError with ValueError when conversion to BLOB fails.
* Raise ProgrammingError instead of Warning if user tries to execute() more
  than one SQL statement.
* Raise ProgrammingError instead of ValueError if an SQL query contains null characters.
* Make sure `_pysqlite_set_result` raises an exception if it returns -1.
2022-03-16 22:58:25 -07:00
Erlend Egeberg Aasland e801e88744
bpo-45138: Revert GH-28240: Expand traced SQL statements (GH-31788)
This reverts commit d1777515f9.

Automerge-Triggered-By: GH:JelleZijlstra
2022-03-09 09:39:49 -08:00
Erlend Egeberg Aasland d1777515f9
bpo-45138: Expand traced SQL statements in `sqlite3` trace callback (GH-28240) 2022-03-08 18:46:40 -08:00
Erlend Egeberg Aasland 4d95fa1ac5
bpo-46878: Purge 'non-standard' from sqlite3 docstrings (GH-31612) 2022-03-07 20:18:41 -08:00
Erlend Egeberg Aasland 88567a9970
bpo-46874: Speed up sqlite3 user-defined aggregate 'step' method (GH-31604) 2022-03-03 22:54:36 +09:00
Erlend Egeberg Aasland b2077117d1
bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351) 2022-02-17 00:24:44 +09:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Erlend Egeberg Aasland 3eb3b4f270
bpo-43853: Expand test suite for SQLite UDF's (GH-27642) 2022-01-26 08:26:16 -08:00
Erlend Egeberg Aasland 9d6a239a34
bpo-44092: Don't reset statements/cursors before rollback (GH-26026)
In SQLite versions pre 3.7.11, pending statements would block a rollback.  This is no longer the case, so remove the workaround.
2022-01-03 19:02:39 +00:00