Commit Graph

60 Commits

Author SHA1 Message Date
Dino Viehland 05f2f0ac92
gh-90815: Add mimalloc memory allocator (#109914)
* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  https://github.com/python/cpython/pull/31164/

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-30 15:43:11 +00:00
Mienxiu f07ca27709
C-API docs: Clarify the size of arenas (#110895)
Clarify the size of arenas

From 3.10.0 alpha 7, the pymalloc allocator uses arenas with a fixed size of 1
MiB on 64-bit platforms instead of 256 KiB on 32-bit platforms.
2023-10-16 19:52:13 +01:00
Ezio Melotti 718391f475
gh-110631: Fix reST indentation (#110724)
* Fix wrong indentation in the other dirs.

* Fix more wrong indentation.
2023-10-11 22:43:03 +02:00
Eric Snow db361a340a
gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) 2023-07-31 23:11:15 +00:00
Serhiy Storchaka 983305268e
gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) 2023-07-27 18:44:32 +03:00
Victor Stinner 8d61a71f9c
gh-107298: Fix more Sphinx warnings in the C API doc (#107329)
Declare the following functions as macros, since they are actually
macros. It avoids a warning on "TYPE" or "macro" argument.

* PyMem_New()
* PyMem_Resize()
* PyModule_AddIntMacro()
* PyModule_AddStringMacro()
* PyObject_GC_New()
* PyObject_GC_NewVar()
* PyObject_New()
* PyObject_NewVar()

Add C standard C types to nitpick_ignore in Doc/conf.py:

* int64_t
* uint64_t
* uintptr_t

No longer ignore non existing "__int" type in nitpick_ignore.

Update Doc/tools/.nitignore
2023-07-27 00:52:40 +00:00
Serhiy Storchaka f8b7fe2f26
gh-106948: Add standard external names to nitpick_ignore (GH-106949)
It includes standard C types, macros and variables like "size_t",
"LONG_MAX" and "errno", and standard environment variables like "PATH".
2023-07-22 21:35:22 +03:00
Serhiy Storchaka fcc816dbff
gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) 2023-07-21 10:52:07 +03:00
Pablo Galindo Salgado c053284e39
gh-96997: Clarify the contract of PyMem_SetAllocator() (#98977) 2022-11-02 21:49:12 +00:00
Adam Turner 0031e62973
gh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (#97768)
:c:type:`<C type>` -> :c:expr:`<C type>`

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-10-05 11:01:14 -07:00
Adam Turner 8b211b4cdb
gh-93738: Documentation C syntax (:c:type:`TYPE` -> :c:expr:`TYPE`) (#97770)
:c:type:`TYPE` -> :c:expr:`TYPE`
2022-10-04 16:26:36 -07:00
Ned Batchelder 3440d197a5
Docs: remove redundant "adverb-adjective" hyphens from compound modifiers (GH-94551)
Discussion: https://discuss.python.org/t/slight-grammar-fix-throughout-adverbs-dont-need-hyphen/17021
2022-07-05 11:16:10 +02:00
Jelle Zijlstra 4c92427fb8
c-api docs: There are five fields, not four (GH-32379) 2022-04-07 07:02:54 -07:00
Jelle Zijlstra 897bc6f928
More minor fixes to C API docs (GH-31525)
* wording fixes in type.rst

* grammar and punctuation in sys.rst

* set: grammar fixes

* structures: capitalization fix

* grammar fixes for sequence

* objects: point to Py_TYPE instead of direct object access

* numbers: add more explicit Python equivalences

* method: add missing period

* memory: grammar fix

* mapping: grammar fixes

* long: grammar fix

* iter: fix grammar for PyAIter_Check

* init: grammar fix
2022-04-02 12:31:05 -07:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) a79e2b6497
Fix a small typo in the docs (GH-26991) 2021-07-03 11:04:57 -03:00
Victor Stinner 6cd0446ef7
bpo-44113: Deprecate old functions to config Python init (GH-26060)
Deprecate the following functions to configure the Python
initialization:

* PySys_AddWarnOption()
* PySys_AddWarnOptionUnicode()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Use the new PyConfig API of the Python Initialization Configuration
instead (PEP 587).
2021-05-12 23:59:25 +02:00
JT e08c67372d
bpo-43928: Fix 'succesfully' typo in document (GH-25569) 2021-04-30 18:35:07 -04:00
Victor Stinner 645ed62fb4
bpo-43774: Remove unused PYMALLOC_DEBUG macro (GH-25711)
Enhance also the documentation of debug hooks on memory allocators.
2021-04-29 10:47:47 +02:00
Victor Stinner a41782cc84
bpo-43774: Document configure options (GH-25283)
Add Doc/using/configure.rst documentation to document configure,
preprocessor, compiler and linker options.

Add a new section about the "Python debug build".
2021-04-08 22:32:21 +02:00
Victor Stinner 0d6bd1ca7c
bpo-3329: Fix typo in PyObjectArenaAllocator doc (GH-24795) 2021-03-09 12:16:42 +01:00
Pablo Galindo b0478d738a
Fix Sphynx syntax in the memory.rst document 2021-02-02 20:45:36 +00:00
Pablo Galindo e485be5b6b
Add a clarification for the object-domain allocators regarding pointer validity (GH-24253) 2021-01-19 13:09:06 +00:00
Pablo Galindo bc450f9694
Add a paragraph about allocation domains to the C-API docs (GH-24252) 2021-01-18 22:20:57 +00:00
Victor Stinner 474652fe93
bpo-40204, doc: Fix syntax of C variables (GH-21846)
For example, fix the following Sphinx 3 errors:

Doc/c-api/buffer.rst:102: WARNING: Error in declarator or parameters
Invalid C declaration: Expected identifier in nested name. [error at 5]
  void \*obj
  -----^

Doc/c-api/arg.rst:130: WARNING: Unparseable C cross-reference: 'PyObject*'
Invalid C declaration: Expected end of definition. [error at 8]
  PyObject*
  --------^

The modified documentation is compatible with Sphinx 2 and Sphinx 3.
2020-08-13 22:11:50 +02:00
Julien Danjou d4d17fd2cf
fix(doc): set correct RST syntax for c:function (GH-18589)
The current content is not rendered since the syntax is not correct.
2020-02-21 11:47:41 +01:00
Serhiy Storchaka e835b31d2b
bpo-38600: NULL -> ``NULL``. (GH-17001)
Also fix some other formatting.
2019-10-30 21:37:16 +02:00
Serhiy Storchaka 25fc088607
bpo-38600: Change the mark up of NULL in the C API documentation. (GH-16950)
Replace all *NULL* with ``NULL``.
2019-10-30 12:03:20 +02:00
Nick Coghlan 17a058ed6f
Remove trailing .0 from version changed note (GH-14987) 2019-07-28 21:40:47 +10:00
Hai Shi 39a5d17a7f closes bpo-37508: Fix name of type in memory.rst. (GH-14604) 2019-07-05 21:03:13 -07:00
Stéphane Wirtel cbb6484573 Doc: Replace the deprecated highlightlang directive by highlight. (#13377)
highlightlang is deprecated since April 2018 in Sphinx.
See https://github.com/sphinx-doc/sphinx/pull/4845
2019-05-17 15:25:34 +05:30
Victor Stinner 4c409beb4c
bpo-36389: Change PyMem_SetupDebugHooks() constants (GH-12782)
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
2019-04-11 13:01:15 +02:00
Kevin Adler a407004391 Fix typo in Memory Management doc. (GH-10798) 2018-11-30 09:42:47 +02:00
Andrés Delfino 5092439c2c bpo-33892: Doc: Use gender neutral words (GH-7770) 2018-06-18 13:34:30 +09:00
Serhiy Storchaka 5bb0005f9f
Make formatting of some return codes conforming to the general style. (#5587) 2018-02-09 13:31:19 +02:00
Victor Stinner 5d39e04290
bpo-32030: Rework memory allocators (#4625)
* Fix _PyMem_SetupAllocators("debug"): always restore allocators to
  the defaults, rather than only caling _PyMem_SetupDebugHooks().
* Add _PyMem_SetDefaultAllocator() helper to set the "default"
  allocator.
* Add _PyMem_GetAllocatorsName(): get the name of the allocators
* main() now uses debug hooks on memory allocators if Py_DEBUG is
  defined, rather than calling directly malloc()
* Document default memory allocators in C API documentation
* _Py_InitializeCore() now fails with a fatal user error if
  PYTHONMALLOC value is an unknown memory allocator, instead of
  failing with a fatal internal error.
* Add new tests on the PYTHONMALLOC environment variable
* Add support.with_pymalloc()
* Add the _testcapi.WITH_PYMALLOC constant and expose it as
   support.with_pymalloc().
* sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so
   replace it with support.with_pymalloc().
* pythoninfo: add _testcapi collector for pymem
2017-11-29 17:20:38 +01:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Victor Stinner ec2cbdd1df
bpo-20064: Document PyObject_Malloc() (#4199)
Document the following functions:

* PyObject_Malloc()
* PyObject_Calloc()
* PyObject_Realloc()
* PyObject_Free()

Fix also PyMem_RawFree() documentation.
2017-10-31 09:37:25 -07:00
Victor Stinner 5ea4c06773 bpo-30054: Expose tracemalloc C API (#1236)
* Make PyTraceMalloc_Track() and PyTraceMalloc_Untrack() functions
  public (remove the "_" prefix)
* Remove the _PyTraceMalloc_domain_t type: use directly unsigned
  int.
* Document methods

Note: methods are already tested in test_tracemalloc.
2017-06-20 17:46:36 +02:00
INADA Naoki f669ffff60 fix minor bug in pymalloc. (#335)
reported by Alexis Lopez-Garcia.
2017-02-27 14:42:37 +01:00
Victor Stinner f5c4b99034 PyMem_Malloc() now uses the fast pymalloc allocator
Issue #26249: PyMem_Malloc() allocator family now uses the pymalloc allocator
rather than system malloc(). Applications calling PyMem_Malloc() without
holding the GIL can now crash: use PYTHONMALLOC=debug environment variable to
validate the usage of memory allocators in your application.
2016-04-22 16:26:23 +02:00
Victor Stinner d1700a9360 Fix typo in doc: avoid the french "& cie" :-) 2016-03-23 16:57:51 +01:00
Victor Stinner 9b46a57302 Doc: fix typos, patch written by Stefan Behnel 2016-03-18 15:10:43 +01:00
Victor Stinner c2fc56836f Enhance documentation on malloc debug hooks
Issue #26564, #26516, #26563.
2016-03-18 11:04:31 +01:00
Victor Stinner 0611c26a58 On memory error, dump the memory block traceback
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
2016-03-15 22:22:13 +01:00
Victor Stinner c4aec3628b Check the GIL in PyObject_Malloc()
Issue #26558: The debug hook of PyObject_Malloc() now checks that the GIL is
held when the function is called.
2016-03-14 22:26:53 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Victor Stinner 29bf27fe7f Issue #26516: Enhance Python mem allocators doc
* add link to PYTHONMALLOCSTATS env var
* add parameters to PyMem macros like PyMem_MALLOC()
* fix PyMem_SetupDebugHooks(): add Calloc functions
* add some newlines for readability
2016-03-09 14:49:52 +01:00
R David Murray f3cae79883 Merge: #23957: fix typo. 2015-04-14 16:42:49 -04:00
R David Murray 5be7f1513d #23957: fix typo. 2015-04-14 16:42:08 -04:00
Victor Stinner d8f0d922d5 Issue #21233: Rename the C structure "PyMemAllocator" to "PyMemAllocatorEx" to
make sure that the code using it will be adapted for the new "calloc" field
(instead of crashing).
2014-06-02 21:57:10 +02:00