Commit Graph

39 Commits

Author SHA1 Message Date
Mark Shannon b11a951f16
bpo-44032: Move data stack to thread from FrameObject. (GH-26076)
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.

* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.

* Move locals, cells and freevars out of frame object into separate memory buffer.

* Use per-threadstate allocated memory chunks for local variables.

* Move globals and builtins from frame object to per-thread stack.

* Move (slow) locals frame object to per-thread stack.

* Move internal frame functions to internal header.
2021-05-21 10:57:35 +01:00
Vladimir Matveev 24a54c0bd4
Delete PyGen_Send (#22663) 2020-10-12 12:10:42 -07:00
Vladimir Matveev 037245c5ac
bpo-41756: Add PyIter_Send function (#22443) 2020-10-09 17:15:15 -07:00
Vladimir Matveev 2b05361bf7
bpo-41756: Introduce PyGen_Send C API (GH-22196)
The new API allows to efficiently send values into native generators
and coroutines avoiding use of StopIteration exceptions to signal 
returns.

ceval loop now uses this method instead of the old "private"
_PyGen_Send C API. This translates to 1.6x increased performance
of 'await' calls in micro-benchmarks.

Aside from CPython core improvements, this new API will also allow 
Cython to generate more efficient code, benefiting high-performance
IO libraries like uvloop.
2020-09-18 18:38:38 -07:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
Victor Stinner ae00a5a885
bpo-40428: Remove PyTuple_ClearFreeList() function (GH-19769)
Remove the following function from the C API:

* PyAsyncGen_ClearFreeLists()
* PyContext_ClearFreeList()
* PyDict_ClearFreeList()
* PyFloat_ClearFreeList()
* PyFrame_ClearFreeList()
* PyList_ClearFreeList()
* PySet_ClearFreeList()
* PyTuple_ClearFreeList()

Make these functions private, move them to the internal C API and
change their return type to void.

Call explicitly PyGC_Collect() to free all free lists.

Note: PySet_ClearFreeList() did nothing.
2020-04-29 02:29:20 +02:00
Victor Stinner 7c59d7c986
bpo-40421: Add pyframe.h header file (GH-19755)
Add a new separated pyframe.h header file of the PyFrame public C
API: it is included by Python.h.

Add PyFrame_GetLineNumber() to the limited C API.

Replace "struct _frame" with "PyFrameObject" in header files.
PyFrameObject is now defined as struct _frame by pyframe.h which is
included early enough in Python.h.
2020-04-28 16:32:48 +02:00
Dong-hee Na d905df766c
bpo-39573: Add Py_IS_TYPE() function (GH-18488)
Co-Author: Neil Schemenauer <nas-github@arctrix.com>
2020-02-13 18:37:17 +01:00
Pablo Galindo cd7db76a63
bpo-39372: Clean header files of declared interfaces with no implementations (GH-18037)
The public API symbols being removed are:

_PyBytes_InsertThousandsGroupingLocale, _PyBytes_InsertThousandsGrouping, _Py_InitializeFromArgs, _Py_InitializeFromWideArgs, _PyFloat_Repr, _PyFloat_Digits,
_PyFloat_DigitsInit, PyFrame_ExtendStack, _PyAIterWrapper_Type, PyNullImporter_Type, PyCmpWrapper_Type, PySortWrapper_Type, PyNoArgsFunction.
2020-01-18 03:14:59 +00:00
Yury Selivanov fc4a044a3c
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468) 2019-09-29 22:59:11 -07:00
Joannah Nanjekye 74b662cf20 bpo-15088 : Remove PyGen_NeedsFinalizing() (GH-15702)
Remove PyGen_NeedsFinalizing(): it was not
documented, tested or used anywhere within CPython after
the implementation of PEP 442.
2019-09-06 17:41:38 +02:00
Victor Stinner ed48866c55
bpo-35134: Split traceback.h header (GH-13430)
Add new Include/cpython/traceback.h and Include/internal/traceback.h
header files.
2019-05-20 00:14:57 +02:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
Yury Selivanov faa135acbf bpo-31709: Drop support for asynchronous __aiter__. (#3903) 2017-10-06 02:08:57 -04:00
Serhiy Storchaka 60e49aa756 Issue #23996: Added _PyGen_SetStopIterationValue for safe raising
StopIteration with value. More safely handle non-normalized exceptions
in -_PyGen_FetchStopIterationValue.
2016-11-06 18:47:03 +02:00
Serhiy Storchaka 24411f8a8d Issue #23996: Added _PyGen_SetStopIterationValue for safe raising
StopIteration with value. More safely handle non-normalized exceptions
in -_PyGen_FetchStopIterationValue.
2016-11-06 18:44:42 +02:00
Yury Selivanov 833c626e67 Issue #28544: Fix compilation of _asynciomodule.c on Windows 2016-10-28 18:48:50 -04:00
Yury Selivanov eb6364557f Issue #28003: Implement PEP 525 -- Asynchronous Generators. 2016-09-08 22:01:51 -07:00
Yury Selivanov a6f6edbda8 Issue #27243: Fix __aiter__ protocol 2016-06-09 15:08:31 -04:00
Yury Selivanov c724bae51c coroutines: Error when awaiting on coroutine that's being awaited
Issue #25888
2016-03-02 11:30:46 -05:00
Yury Selivanov 5376ba9630 Issue #24400: Introduce a distinct type for 'async def' coroutines.
Summary of changes:

1. Coroutines now have a distinct, separate from generators
   type at the C level: PyGen_Type, and a new typedef PyCoroObject.
   PyCoroObject shares the initial segment of struct layout with
   PyGenObject, making it possible to reuse existing generators
   machinery.  The new type is exposed as 'types.CoroutineType'.

   As a consequence of having a new type, CO_GENERATOR flag is
   no longer applied to coroutines.

2. Having a separate type for coroutines made it possible to add
   an __await__ method to the type.  Although it is not used by the
   interpreter (see details on that below), it makes coroutines
   naturally (without using __instancecheck__) conform to
   collections.abc.Coroutine and collections.abc.Awaitable ABCs.

   [The __instancecheck__ is still used for generator-based
   coroutines, as we don't want to add __await__ for generators.]

3. Add new opcode: GET_YIELD_FROM_ITER.  The opcode is needed to
   allow passing native coroutines to the YIELD_FROM opcode.

   Before this change, 'yield from o' expression was compiled to:

      (o)
      GET_ITER
      LOAD_CONST
      YIELD_FROM

   Now, we use GET_YIELD_FROM_ITER instead of GET_ITER.

   The reason for adding a new opcode is that GET_ITER is used
   in some contexts (such as 'for .. in' loops) where passing
   a coroutine object is invalid.

4. Add two new introspection functions to the inspec module:
   getcoroutinestate(c) and getcoroutinelocals(c).

5. inspect.iscoroutine(o) is updated to test if 'o' is a native
   coroutine object.  Before this commit it used abc.Coroutine,
   and it was requested to update inspect.isgenerator(o) to use
   abc.Generator; it was decided, however, that inspect functions
   should really be tailored for checking for native types.

6. sys.set_coroutine_wrapper(w) API is updated to work with only
   native coroutines.  Since types.coroutine decorator supports
   any type of callables now, it would be confusing that it does
   not work for all types of coroutines.

7. Exceptions logic in generators C implementation was updated
   to raise clearer messages for coroutines:

   Before: TypeError("generator raised StopIteration")
   After: TypeError("coroutine raised StopIteration")
2015-06-22 12:19:30 -04:00
Yury Selivanov 7544508f02 PEP 0492 -- Coroutines with async and await syntax. Issue #24017. 2015-05-11 22:57:16 -04:00
Victor Stinner 40ee30181f Issue #21205: Add a new ``__qualname__`` attribute to generator, the qualified
name, and use it in the representation of a generator (``repr(gen)``). The
default name of the generator (``__name__`` attribute) is now get from the
function instead of the code. Use ``gen.gi_code.co_name`` to get the name of
the code.
2014-06-16 15:59:28 +02:00
Antoine Pitrou 58720d6145 Issue #17934: Add a clear() method to frame objects, to help clean up expensive details (local variables) and break reference cycles. 2013-08-05 23:26:40 +02:00
Antoine Pitrou 9396356948 Backout c89febab4648 following private feedback by Guido.
(Issue #17807: Generators can now be finalized even when they are part of a reference cycle)
2013-05-14 20:37:52 +02:00
Antoine Pitrou 04e70d19e7 Issue #17807: Generators can now be finalized even when they are part of a reference cycle. 2013-05-08 18:12:35 +02:00
Nick Coghlan c40bc09942 Issue #13783: the PEP 380 implementation no longer expands the public C API 2012-06-17 15:15:49 +10:00
Benjamin Peterson 2afe6aeae8 perform yield from delegation by repeating YIELD_FROM opcode (closes #14230)
This allows generators that are using yield from to be seen by debuggers. It
also kills the f_yieldfrom field on frame objects.

Patch mostly from Mark Shannon with a few tweaks by me.
2012-03-15 15:37:39 -05:00
Antoine Pitrou b4a92376f8 Fix regression after c8d1df9ac987
(PPC buildbot)
2012-03-10 23:43:12 +01:00
Nick Coghlan 76e1bb0ef9 Fix indenting 2012-01-14 16:08:08 +10:00
Nick Coghlan 1f7ce62bd6 Implement PEP 380 - 'yield from' (closes #11682) 2012-01-13 21:43:40 +10:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Christian Heimes af98da18c0 Merged revisions 60284-60349 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r60286 | christian.heimes | 2008-01-25 15:54:23 +0100 (Fri, 25 Jan 2008) | 1 line

  setup.py doesn't pick up changes to a header file
........
  r60287 | christian.heimes | 2008-01-25 16:52:11 +0100 (Fri, 25 Jan 2008) | 2 lines

  Added the Python core headers Include/*.h and pyconfig.h as dependencies for the extensions in Modules/
  It forces a rebuild of all extensions when a header files has been modified
........
  r60291 | raymond.hettinger | 2008-01-25 20:24:46 +0100 (Fri, 25 Jan 2008) | 4 lines

  Changes 54857 and 54840 broke code and were reverted in Py2.5 just before
  it was released, but that reversion never made it to the Py2.6 head.
........
  r60296 | guido.van.rossum | 2008-01-25 20:50:26 +0100 (Fri, 25 Jan 2008) | 2 lines

  Rewrite the list_inline_repeat overflow check slightly differently.
........
  r60301 | thomas.wouters | 2008-01-25 22:09:34 +0100 (Fri, 25 Jan 2008) | 4 lines


  Use the right (portable) definition of the max of a Py_ssize_t.
........
  r60303 | thomas.wouters | 2008-01-26 02:47:05 +0100 (Sat, 26 Jan 2008) | 5 lines


  Make 'testall' work again when building in a separate directory.
  test_distutils still fails when doing that.
........
  r60305 | neal.norwitz | 2008-01-26 06:54:48 +0100 (Sat, 26 Jan 2008) | 3 lines

  Prevent this test from failing if there are transient network problems
  by retrying the host for up to 3 times.
........
  r60306 | neal.norwitz | 2008-01-26 08:26:12 +0100 (Sat, 26 Jan 2008) | 12 lines

  Use a condition variable (threading.Event) rather than sleeps and checking a
  global to determine when the server is ready to be used.  This slows the test
  down, but should make it correct.  There was a race condition before where the
  server could have assigned a port, yet it wasn't ready to serve requests.  If
  the client sent a request before the server was completely ready, it would get
  an exception.  There was machinery to try to handle this condition.  All of
  that should be unnecessary and removed if this change works.  A NOTE was
  added as a comment about what needs to be fixed.

  The buildbots will tell us if there are more errors or
  if this test is now stable.
........
  r60307 | neal.norwitz | 2008-01-26 08:38:03 +0100 (Sat, 26 Jan 2008) | 3 lines

  Fix exception in tearDown on ppc buildbot.  If there's no directory,
  that shouldn't cause the test to fail.  Just like it setUp.
........
  r60308 | raymond.hettinger | 2008-01-26 09:19:06 +0100 (Sat, 26 Jan 2008) | 3 lines

  Make PySet_Add() work with frozensets.  Works like PyTuple_SetItem() to build-up values in a brand new frozenset.
........
  r60309 | neal.norwitz | 2008-01-26 09:26:00 +0100 (Sat, 26 Jan 2008) | 1 line

  The OS X buildbot had errors with the unavailable exceptions disabled.  Restore it.
........
  r60310 | raymond.hettinger | 2008-01-26 09:37:28 +0100 (Sat, 26 Jan 2008) | 4 lines

  Let marshal build-up sets and frozensets one element at a time.
  Saves the unnecessary creation of a tuple as intermediate container.
........
  r60311 | raymond.hettinger | 2008-01-26 09:41:13 +0100 (Sat, 26 Jan 2008) | 1 line

  Update test code for change to PySet_Add().
........
  r60312 | raymond.hettinger | 2008-01-26 10:31:11 +0100 (Sat, 26 Jan 2008) | 1 line

  Revert PySet_Add() changes.
........
  r60314 | georg.brandl | 2008-01-26 10:43:35 +0100 (Sat, 26 Jan 2008) | 2 lines

  #1934: fix os.path.isabs docs.
........
  r60316 | georg.brandl | 2008-01-26 12:00:18 +0100 (Sat, 26 Jan 2008) | 2 lines

  Add missing things in re docstring.
........
  r60317 | georg.brandl | 2008-01-26 12:02:22 +0100 (Sat, 26 Jan 2008) | 2 lines

  Slashes allowed on Windows.
........
  r60319 | georg.brandl | 2008-01-26 14:41:21 +0100 (Sat, 26 Jan 2008) | 2 lines

  Fix markup again.
........
  r60320 | andrew.kuchling | 2008-01-26 14:50:51 +0100 (Sat, 26 Jan 2008) | 1 line

  Add some items
........
  r60321 | georg.brandl | 2008-01-26 15:02:38 +0100 (Sat, 26 Jan 2008) | 2 lines

  Clarify "b" mode under Unix.
........
  r60322 | georg.brandl | 2008-01-26 15:03:47 +0100 (Sat, 26 Jan 2008) | 3 lines

  #1940: make it possible to use curses.filter() before curses.initscr()
  as the documentation says.
........
  r60324 | georg.brandl | 2008-01-26 15:14:20 +0100 (Sat, 26 Jan 2008) | 3 lines

  #1473257: add generator.gi_code attribute that refers to
  the original code object backing the generator. Patch by Collin Winter.
........
  r60325 | georg.brandl | 2008-01-26 15:19:22 +0100 (Sat, 26 Jan 2008) | 2 lines

  Move C API entries to the corresponding section.
........
  r60326 | christian.heimes | 2008-01-26 17:43:35 +0100 (Sat, 26 Jan 2008) | 1 line

  Unit test fix from Giampaolo Rodola, #1938
........
  r60327 | gregory.p.smith | 2008-01-26 19:51:05 +0100 (Sat, 26 Jan 2008) | 2 lines

  Update docs for new callpack params added in r60188
........
  r60329 | neal.norwitz | 2008-01-26 21:24:36 +0100 (Sat, 26 Jan 2008) | 3 lines

  Cleanup the code a bit.  test_rfind is failing on PPC and PPC64 buildbots,
  this might fix the problem.
........
  r60330 | neal.norwitz | 2008-01-26 22:02:45 +0100 (Sat, 26 Jan 2008) | 1 line

  Always try to remove the test file even if close raises an exception
........
  r60331 | neal.norwitz | 2008-01-26 22:21:59 +0100 (Sat, 26 Jan 2008) | 3 lines

  Reduce the race condition by signalling when the server is ready
  and not trying to connect before.
........
  r60334 | neal.norwitz | 2008-01-27 00:13:46 +0100 (Sun, 27 Jan 2008) | 5 lines

  On some systems (e.g., Ubuntu on hppa) the flush()
  doesn't cause the exception, but the close() does.

  Will backport.
........
  r60335 | neal.norwitz | 2008-01-27 00:14:17 +0100 (Sun, 27 Jan 2008) | 2 lines

  Consistently use tempfile.tempdir for the db_home directory.
........
  r60338 | neal.norwitz | 2008-01-27 02:44:05 +0100 (Sun, 27 Jan 2008) | 4 lines

  Eliminate the sleeps that assume the server will start in .5 seconds.
  This should make the test less flaky.  It also speeds up the test
  by about 75% on my box (20+ seconds -> ~4 seconds).
........
  r60342 | neal.norwitz | 2008-01-27 06:02:34 +0100 (Sun, 27 Jan 2008) | 6 lines

  Try to prevent this test from being flaky.  We might need a sleep in here
  which isn't as bad as it sounds.  The close() *should* raise an exception,
  so if it didn't we should give more time to sync and really raise it.

  Will backport.
........
  r60344 | jeffrey.yasskin | 2008-01-27 06:40:35 +0100 (Sun, 27 Jan 2008) | 3 lines

  Make rational.gcd() public and allow Rational to take decimal strings, per
  Raymond's advice.
........
  r60345 | neal.norwitz | 2008-01-27 08:36:03 +0100 (Sun, 27 Jan 2008) | 3 lines

  Mostly reformat.  Also set an error and return NULL if neither MS_WINDOWS
  nor UNIX is defined.  This may have caused problems on cygwin.
........
  r60346 | neal.norwitz | 2008-01-27 08:37:38 +0100 (Sun, 27 Jan 2008) | 3 lines

  Use int for the sign rather than a char.  char can be signed or unsigned.
  It's system dependent.  This might fix the problem with test_rfind failing.
........
  r60347 | neal.norwitz | 2008-01-27 08:41:33 +0100 (Sun, 27 Jan 2008) | 1 line

  Add stdarg include for va_list to get this to compile on cygwin
........
  r60348 | raymond.hettinger | 2008-01-27 11:13:57 +0100 (Sun, 27 Jan 2008) | 1 line

  Docstring nit
........
  r60349 | raymond.hettinger | 2008-01-27 11:47:55 +0100 (Sun, 27 Jan 2008) | 1 line

  Removed an unnecessary and confusing paragraph from the namedtuple docs.
........
2008-01-27 15:18:18 +00:00
Christian Heimes 90aa7646af #1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT. 2007-12-19 02:45:37 +00:00
Martin v. Löwis 9f2e346911 Merged revisions 56467-56482 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r56477 | martin.v.loewis | 2007-07-21 09:04:38 +0200 (Sa, 21 Jul 2007) | 11 lines

  Merged revisions 56466-56476 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r56476 | martin.v.loewis | 2007-07-21 08:55:02 +0200 (Sa, 21 Jul 2007) | 4 lines

    PEP 3123: Provide forward compatibility with Python 3.0, while keeping
    backwards compatibility. Add Py_Refcnt, Py_Type, Py_Size, and
    PyVarObject_HEAD_INIT.
  ........
................
  r56478 | martin.v.loewis | 2007-07-21 09:47:23 +0200 (Sa, 21 Jul 2007) | 2 lines

  PEP 3123: Use proper C inheritance for PyObject.
................
  r56479 | martin.v.loewis | 2007-07-21 10:06:55 +0200 (Sa, 21 Jul 2007) | 3 lines

  Add longintrepr.h to Python.h, so that the compiler can
  see that PyFalse is really some kind of PyObject*.
................
  r56480 | martin.v.loewis | 2007-07-21 10:47:18 +0200 (Sa, 21 Jul 2007) | 2 lines

  Qualify SHIFT, MASK, BASE.
................
  r56482 | martin.v.loewis | 2007-07-21 19:10:57 +0200 (Sa, 21 Jul 2007) | 2 lines

  Correctly refer to _ob_next.
................
2007-07-21 17:22:18 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Martin v. Löwis 8d97e33bb7 Patch #966493: Cleanup generator/eval_frame exposure. 2004-06-27 15:43:12 +00:00
Martin v. Löwis e440e47e91 Patch #957398: Add public API for Generator Object/Type. 2004-06-01 15:22:42 +00:00