Commit Graph

10927 Commits

Author SHA1 Message Date
Quan Tian 3bd0d620d6 Fix typos in comments (GH-9905) 2018-10-19 23:30:03 +02:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి) f192aeb95a bpo-20216: Correct docstrings of digest() methods in hashlib. (GH-9873) 2018-10-19 20:42:53 +03:00
Serhiy Storchaka 55f8249d65
bpo-34741: Get rid of tp_getattro and tp_setattro in pyexpat.xmlparser. (GH-9422)
Use tp_members and tp_getset instead.
2018-10-19 18:00:51 +03:00
Serhiy Storchaka f081fd8303
bpo-35013: Add more type checks for children of Element. (GH-9944)
It is now guarantied that children of xml.etree.ElementTree.Element
are Elements (at least in C implementation). Previously methods
__setitem__(), __setstate__() and __deepcopy__() could be used for
adding non-Element children.
2018-10-19 12:12:57 +03:00
Zackery Spytz 68def052dc Fix several reference counting bugs in pyexpat.c. (GH-9955) 2018-10-19 09:57:38 +03:00
Serhiy Storchaka 6f906b3d72
bpo-35008: Fix possible leaks in Element.__setstate__(). (GH-9924)
C implementation of xml.etree.ElementTree.Element.__setstate__()
leaked references to children when called for already initialized
element.
2018-10-18 09:49:54 +03:00
Gregory P. Smith 9d4712bc8f
bpo-35011: Restore use of pyexpatns.h in libexpat (GH-9939)
Restores the use of pyexpatns.h to isolate our embedded copy of the expat C
library so that its symbols do not conflict at link or dynamic loading time
with an embedding application or other extension modules with their own
version of libexpat.

5dc3f23b5f (diff-3afaf7274c90ce1b7405f75ad825f545) inadvertently removed it when upgrading expat.
2018-10-17 18:10:46 -07:00
Stéphane Wirtel 74a8b6ea7e bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
2018-10-18 01:05:04 +02:00
Serhiy Storchaka b11c5667f9
bpo-34941: Fix searching Element subclasses. (GH-9766)
Methods find(), findtext() and findall() of xml.etree.ElementTree.Element
were not able to find chldren which are instances of Element subclasses.
2018-10-14 10:32:19 +03:00
Serhiy Storchaka 9b8c2e7676
bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751)
for the SHAKE algorithm in the hashlib module.
2018-10-11 07:41:00 +03:00
Raymond Hettinger de2e448414
bpo-34925: Optimize common case for bisect() argument parsing (#9753) 2018-10-08 08:02:41 -07:00
Zackery Spytz 365ad2ead5 bpo-34824: Fix a possible NULL pointer dereference in _ssl.c (GH-9606)
On failure, _PyBytes_Resize() will deallocate the bytes object and set
"result" to NULL.


https://bugs.python.org/issue34824
2018-10-06 10:41:45 -07:00
Stéphane Wirtel 683281f536 bpo-34906: Doc: Fix typos (2) (GH-9735)
Fix typos
2018-10-06 16:35:53 +02:00
Serhiy Storchaka addf8afb43
Fix a compiler warning added in bpo-34872. (GH-9722) 2018-10-05 21:20:02 +03:00
Stéphane Wirtel 07fbbfde1b bpo-34906: Doc: Fix typos (GH-9712) 2018-10-05 16:17:18 +02:00
E. M. Bray 7a7693e9cb bpo-28441: Ensure `.exe` suffix in `sys.executable` on MinGW and Cygwin (GH-4348)
This is needed to even the run the test suite on buildbots for affected platforms; e.g.:

```
./python.exe  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
/home/embray/src/python/test-worker/3.x.test-worker/build/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
Traceback (most recent call last):
  File "./Tools/scripts/run_tests.py", line 56, in <module>
    main(sys.argv[1:])
  File "./Tools/scripts/run_tests.py", line 52, in main
    os.execv(sys.executable, args)
PermissionError: [Errno 13] Permission denied
make: *** [Makefile:1073: buildbottest] Error 1
```
2018-10-05 20:38:50 +09:00
Serhiy Storchaka 5f5a7781c8
bpo-34740: Get rid of tp_getattro in ossaudiodev.oss_audio_device. (GH-9421)
Use tp_members and tp_getset instead.
2018-10-04 10:42:06 +03:00
Serhiy Storchaka b2953fa3dd
bpo-34739: Get rid of tp_getattro in xml.etree.ElementTree.XMLParser. (GH-9420)
Use tp_members and tp_getset instead.
2018-10-04 10:41:27 +03:00
Elvis Pranskevichus 0c797a6aca bpo-34872: Fix self-cancellation in C implementation of asyncio.Task (GH-9679)
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.

    async def task1():
        async def task2():
            await task3     # task3 is never cancelled

        asyncio.current_task().cancel()
        await asyncio.create_task(task2())

The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.

Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario.
2018-10-03 10:30:31 -04:00
William Orr 81574b80e9 closes bpo-34862: Guard definition of convert_sched_param with POSIX_SPAWN_SETSCHEDULER. (GH-9658)
Fixes broken build on OpenBSD-current.
2018-10-01 22:19:56 -07:00
Raymond Hettinger df8101517a
Speed-up math.dist() by 30% (GH-9628) 2018-09-29 14:30:38 -07:00
Zackery Spytz 4b430e5f69 Fix incorrect error handling in _pickle.Unpickler.__init__() (GH-9630)
_pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1.
2018-09-29 08:48:46 +03:00
Pablo Galindo e2f48bf0e8 bpo-34797: Convert heapq to the argument clinic (GH-9560) 2018-09-28 12:39:43 -07:00
Tal Einat 1fba2ffc37 bpo-34736: improve error message for invalid length b64decode inputs (GH-9563)
Improvements:
1. Include the number of valid data characters in the error message.
2. Mention "number of data characters" rather than "length".


https://bugs.python.org/issue34736
2018-09-27 22:57:22 -07:00
Zsolt Cserna 9df346bf98 bpo-34248: Add filename to error raised in {gnu,ndbm}.open() (GH-8590)
Report the filename to the exception when raising {gdbm,dbm.ndbm}.error in
dbm.gnu.open() and dbm.ndbm.open() functions, so it gets printed when the
exception is raised, and can also be obtained by the filename attribute of the
exception object.
2018-09-27 22:54:34 +03:00
Yury Selivanov 994269ccee
bpo-34762: Update PyContext* to PyObject* in asyncio and decimal (GH-9609)
This fixes various compiler warnings.
2018-09-27 14:55:55 -04:00
Bjorn Andersson bb8165172a bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706)
The AF_QIPCRTR address family was introduced in Linux v4.7.

Co-authored-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2018-09-26 16:47:52 +03:00
Joe Pamer c8c0249c9e bpo-32557: allow shutil.disk_usage to take a file path on Windows also (GH-9372)
https://bugs.python.org/issue32557
2018-09-25 07:57:36 -07:00
Tim Hoffmann a0fd7f1b55 Migrate datetime.date.fromtimestamp to Argument Clinic (GH-8535) 2018-09-24 10:39:02 +02:00
Lisa Roach 9718b59ee5
bpo-34659: Adds initial kwarg to itertools.accumulate() (GH-9345) 2018-09-23 17:34:59 -07:00
Christian Heimes 9fb051f032 bpo-34670: Add TLS 1.3 post handshake auth (GH-9460)
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.

Signed-off-by: Christian Heimes <christian@python.org>q


https://bugs.python.org/issue34670
2018-09-22 23:32:31 -07:00
Nathaniel J. Smith c0da582b22 bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468)
OpenSSL follows the convention that whenever you call a function, it
returns an error indicator value; and if this value is negative, then
you need to go look at the actual error code to see what happened.

Commit c6fd1c1c3a introduced a small mistake in
_ssl__SSLSocket_shutdown_impl: instead of checking whether the error
indicator was negative, it started checking whether the actual error
code was negative, and it turns out that the error codes are never
negative. So the effect was that 'unwrap()' lost the ability to raise
SSL errors.


https://bugs.python.org/issue34759
2018-09-21 21:44:12 -07:00
Yury Selivanov 2ec872b31e
bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) 2018-09-21 15:33:56 -04:00
Raymond Hettinger b46ad5431d
Minor performance tweak for deque.index() with a start argument (GH-9440) 2018-09-21 01:46:41 -07:00
Serhiy Storchaka fb3e9c00ed
bpo-34755: Add few minor optimizations in _asynciomodule.c. (GH-9455) 2018-09-21 09:11:32 +03:00
Zackery Spytz 91e6c8717b bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418)
There was a missing PyMem_Free(format) in time_strftime().
2018-09-21 09:09:48 +03:00
Benjamin Peterson c510c6b8b6
Simplify PyInit_timezone. (GH-9467)
Reduce the knotty preprocessor conditional logic, dedent unnecessarily nested
code, and handle errors properly.

The first edition of this change (afde1c1a05)
failed (bpo-34715) because FreeBSD doesn't define the timezone globals. That's
why we're now checking for HAVE_DECL_TZNAME.
2018-09-20 19:52:18 -07:00
Benjamin Peterson a4ae828ee4
closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261) 2018-09-20 18:36:40 -07:00
Berker Peksag b10a64d117
bpo-34743: Fix test_database_source_name under SQLite 3.7.9 (GH-9426) 2018-09-20 14:14:33 +03:00
Berker Peksag 8d1e190fc5
bpo-32215: Fix performance regression in sqlite3 (GH-8511) 2018-09-20 14:10:49 +03:00
Victor Stinner 06e7608207
Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" (GH-9430)
* Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)"

This reverts commit dbdee0073c.

* Revert "bpo-34589: C locale coercion off by default (GH-9073)"

This reverts commit 7a0791b699.

* Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)"

This reverts commit 188ebfa475.
2018-09-19 14:56:36 -07:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Christian Heimes cb5778f00c bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146)
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue34623
2018-09-18 05:38:58 -07:00
Serhiy Storchaka 0185f34ddc
bpo-33721: Make some os.path functions and pathlib.Path methods be tolerant to invalid paths. (#7695)
Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(),
os.path.isfile(), os.path.islink(), and os.path.ismount() now return False
instead of raising ValueError or its subclasses UnicodeEncodeError
and UnicodeDecodeError for paths that contain characters or bytes
unrepresentative at the OS level.
2018-09-18 11:28:51 +03:00
Serhiy Storchaka 7bdf28265a
bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610)
Add C API function PyCompile_OpcodeStackEffectWithJump().
2018-09-18 09:54:26 +03:00
Victor Stinner 7a0791b699
bpo-34589: C locale coercion off by default (GH-9073)
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).

test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
2018-09-17 16:22:29 -07:00
Victor Stinner 188ebfa475
bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)
_PyCoreConfig:

* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn

These fields are now private (name prefixed by "_").
2018-09-17 15:13:17 -07:00
Victor Stinner 1fb399ba4e
bpo-34715: Revert "Simplify PyInit_timezone. (GH-9323)" (GH-9366)
This reverts commit afde1c1a05.
2018-09-17 13:56:17 -07:00
Steve Dower c6fd1c1c3a
bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) 2018-09-17 11:34:47 -07:00
Serhiy Storchaka 12a69db908
Convert os.readlink() to Argument Clinic. (GH-8778)
Also convert os.get_blocking() and os.set_blocking().
2018-09-17 15:38:27 +03:00
Alexandru Ardelean b3a271fc0c bpo-34710: fix SSL module build (GH-9347)
Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
2018-09-17 04:53:31 -07:00
Vladimir Matveev 7843caeb90 bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) 2018-09-15 22:36:29 -07:00
Benjamin Peterson afde1c1a05
Simplify PyInit_timezone. (GH-9323)
Assume tzname exists. Only use a hack to compute altzone if it's not defined.
2018-09-14 16:21:32 -07:00
Eric Snow 5903296045
bpo-34651: Only allow the main interpreter to fork. (gh-9279)
When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter.

https://bugs.python.org/issue34651
2018-09-14 14:17:20 -07:00
Benjamin Peterson b93062b7fb
bpo-34672: Don't pass NULL to gmtime_r. (GH-9312) 2018-09-14 10:39:13 -07:00
Benjamin Peterson 5633c4f342
bpo-34672: Try to pass the C library's own timezone strings back to it. (GH-9288) 2018-09-14 09:09:04 -07:00
Benjamin Peterson ea13740a37
bpo-34674: Assume unistd.h exists on Unix. (GH-9290) 2018-09-13 21:57:31 -07:00
Raymond Hettinger 1401018da1
Remove wording that could be deemed to be perjorative (GH-9287) 2018-09-13 21:17:40 -07:00
Gregory P. Smith a20b6adb5a bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255)
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.



https://bugs.python.org/issue34658
2018-09-13 04:30:10 -07:00
Alexey Izbyshev 6f82bffd2d bpo-34649: Add missing NULL checks to _encoded_const() (GH-9225)
Reported by Svace static analyzer.
2018-09-13 00:05:20 +03:00
Oren Milman 0bd1a2dcfd bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) 2018-09-12 22:14:35 +03:00
Benjamin Peterson e502451781
closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) 2018-09-12 12:06:42 -07:00
Sergey Fedoseev 7f0d59f3a8 Simplified implementation of _sre.ascii_iscased(). (GH-9097) 2018-09-12 15:49:09 +03:00
Cheryl Sabella 731ff68eee closes bpo-25041: Document AF_PACKET socket address format. (GH-4092) 2018-09-11 17:32:15 -07:00
Benjamin Peterson b9bf9d025e
Delete old expat comment. (GH-9197) 2018-09-11 17:17:39 -07:00
Max Bélanger 4859ba0d2c closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) 2018-09-11 16:14:00 -07:00
Sergey Fedoseev ec014a101a bpo-34636: Use fast path for more chars in SRE category macros. (GH-9170)
When handling \s, \d, or \w (and their inverse) escapes in bytes regexes this a small but measurable performance improvement.

<!-- issue-number: [bpo-34636](https://www.bugs.python.org/issue34636) -->
https://bugs.python.org/issue34636
<!-- /issue-number -->
2018-09-11 15:47:59 -07:00
Tal Einat c4bccd3c76 bpo-20180: convert most of itertoolsmodule.c to use Argument Clinic (GH-9164) 2018-09-11 14:49:13 -07:00
Oren Milman 24bd50bdcc closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) 2018-09-11 11:46:55 -07:00
Berker Peksag b690b9b047
bpo-29386: Pass -1 to epoll_wait() when timeout is < -1 (GH-9040)
Although the kernel accepts any negative value for timeout, the
documented value to block indefinitely is -1.

This commit also makes the code similar to select.poll.poll().
2018-09-11 20:29:48 +03:00
Andrew Svetlov 0baa72f4b2
bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) 2018-09-11 10:13:04 -07:00
Oren Milman 735171e334 closes bpo-29832: Remove "getsockaddrarg" from error messages. (GH-3163) 2018-09-11 09:51:29 -07:00
Benjamin Peterson 5033aa77aa
bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) 2018-09-10 21:04:00 -07:00
Gregory P. Smith ce34410b8b
bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242)
When subprocess.Popen() stdin= stdout= or stderr= handles are specified
and appear in pass_fds=, don't close the original fds after dup'ing them.

This implementation and unittest primarily came from @izbyshev (see the PR)

See also b89b52f284

This also removes the old manual p2cread, c2pwrite, and errwrite closing logic
as inheritable flags and _close_open_fds takes care of that properly today without special treatment.

This code is within child_exec() where it is the only thread so there is no
race condition between the dup and _Py_set_inheritable_async_safe call.
2018-09-10 17:46:22 -07:00
Tal Einat 3286ce4ade bpo-20180: itertools.groupby Argument Clinic conversion (GH-4170) 2018-09-10 11:33:08 -07:00
Peter Eisentraut 0e0bc4e221 Fix misleading mentions of tp_size in comments (GH-9093)
Many type object initializations labeled a field "tp_size" in the
comment, but the name of that field is tp_basicsize.
2018-09-10 09:46:08 -07:00
Serhiy Storchaka d700f97b62
bpo-20104: Change the file_actions parameter of os.posix_spawn(). (GH-6725)
* Make its default value an empty tuple instead of None.
* Make it a keyword-only parameter.
2018-09-08 14:48:18 +03:00
William Grzybowski 28658485a5 bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098)
Pass the user/group name as Unicode to the formatting function,
instead of always decoding a bytes string from UTF-8.
2018-09-07 19:10:39 +02:00
Pablo Galindo 254a4663d8
bpo-20104: Add flag capabilities to posix_spawn (GH-6693)
Implement the "attributes objects" parameter of `os.posix_spawn` to complete the implementation and fully cover the underlying API.
2018-09-07 16:44:24 +01:00
William Grzybowski 23e65b2555 bpo-33625: Release GIL for grp.getgr{nam,gid} and pwd.getpw{nam,uid} (GH-7081)
Release GIL on grp.getgrnam(), grp.getgrgid(), pwd.getpwnam() and
pwd.getpwuid() if reentrant variants of these functions are available.

Patch by William Grzybowski.
2018-09-07 14:06:15 +02:00
Sergey Fedoseev f9925d86c9 _sre.c: Removed unused SRE_IS_ALNUM macro (GH-9090) 2018-09-07 10:56:09 +02:00
Erik Janssens 874809ea38 closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) 2018-09-04 23:29:42 -07:00
Alexander Buchkovsky 266f4904a2 bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027)
Fix for invalid assert on big output of multiprocessing.Process.
2018-09-04 18:10:28 +02:00
Pablo Galindo e9ba3705de
bpo-33083 - Make math.factorial reject arguments that are not int-like (GH-6149)
math.factorial() was accepting non-integral Decimal instances. This is inconsistent with the actual behaviour for floats, which are not accepted.
2018-09-03 22:20:06 +01:00
Victor Stinner 73b00becbd
bpo-34544: pymain_read_conf() don't change LC_ALL (GH-9045)
bpo-34485, bpo-34544: Again, pymain_read_conf() leaves LC_ALL locale
unchanged: only modify LC_CTYPE.
2018-09-03 17:32:31 +02:00
Victor Stinner 2094c2bea4
bpo-34567: pythoninfo gets coreconfig (GH-9043)
* Add _testcapi.get_coreconfig() to get the _PyCoreConfig of the
  interpreter
* test.pythoninfo now gets the core configuration using
  _testcapi.get_coreconfig()
2018-09-03 17:06:39 +02:00
Victor Stinner f01b2a1b84
bpo-34544: Fix setlocale() in pymain_read_conf() (GH-9041)
bpo-34485, bpo-34544: On some FreeBSD, nl_langinfo(CODESET) fails if
LC_ALL or LC_CTYPE is set to an invalid locale name. Replace
_Py_SetLocaleFromEnv(LC_CTYPE) with _Py_SetLocaleFromEnv(LC_ALL) to
initialize properly locales.

Partially revert commit 177d921c8c.
2018-09-03 14:38:21 +02:00
Raymond Hettinger f326714426
Minor improvement to code clarity (GH-9036)
Make it clear that the n==0 case is included. Otherwise, you have to know that max==0.0 whenever n==0.
2018-09-02 13:34:21 -07:00
Thomas Herzog 2d7102e726 closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) 2018-09-01 15:13:57 -07:00
Raymond Hettinger 745c0f3980
Simplify vector_norm() by eliminating special cases in the main loop (GH-9006)
The *max* value is no longer treated as a special case in the main loop.  Besides making the main loop simpler and branchless, this also lets us relax the input restriction of *vec* to contain only non-negative values.
2018-08-31 11:22:13 -07:00
Victor Stinner fbca90856d
bpo-34523: Use _PyCoreConfig instead of globals (GH-9005)
Use the core configuration of the interpreter, rather
than using global configuration variables. For example, replace
Py_QuietFlag with core_config->quiet.
2018-08-30 00:50:45 +02:00
Victor Stinner 3d4226a832
bpo-34523: Support surrogatepass in locale codecs (GH-8995)
Add support for the "surrogatepass" error handler in
PyUnicode_DecodeFSDefault() and PyUnicode_EncodeFSDefault()
for the UTF-8 encoding.

Changes:

* _Py_DecodeUTF8Ex() and _Py_EncodeUTF8Ex() now support the
  surrogatepass error handler (_Py_ERROR_SURROGATEPASS).
* _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx() now use
  the _Py_error_handler enum instead of "int surrogateescape" to pass
  the error handler. These functions now return -3 if the error
  handler is unknown.
* Add unit tests on _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx()
  in test_codecs.
* Rename get_error_handler() to _Py_GetErrorHandler() and expose it
  as a private function.
* _freeze_importlib doesn't need config.filesystem_errors="strict"
  workaround anymore.
2018-08-29 22:21:32 +02:00
Victor Stinner c5989cd876
bpo-34523: Py_DecodeLocale() use UTF-8 on Windows (GH-8998)
Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding on
Windows if Py_LegacyWindowsFSEncodingFlag is zero.

pymain_read_conf() now sets Py_LegacyWindowsFSEncodingFlag in its
loop, but restore its value at exit.
2018-08-29 19:32:47 +02:00
Victor Stinner b2457efc78
bpo-34523: Add _PyCoreConfig.filesystem_encoding (GH-8963)
_PyCoreConfig_Read() is now responsible to choose the filesystem
encoding and error handler. Using Py_Main(), the encoding is now
chosen even before calling Py_Initialize().

_PyCoreConfig.filesystem_encoding is now the reference, instead of
Py_FileSystemDefaultEncoding, for the Python filesystem encoding.

Changes:

* Add filesystem_encoding and filesystem_errors to _PyCoreConfig
* _PyCoreConfig_Read() now reads the locale encoding for the file
  system encoding.
* PyUnicode_EncodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize()
  now use the interpreter configuration rather than
  Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
  global configuration variables.
* Add _Py_SetFileSystemEncoding() and _Py_ClearFileSystemEncoding()
  private functions to only modify Py_FileSystemDefaultEncoding and
  Py_FileSystemDefaultEncodeErrors in coreconfig.c.
* _Py_CoerceLegacyLocale() now takes an int rather than
  _PyCoreConfig for the warning.
2018-08-29 13:25:36 +02:00
Victor Stinner 177d921c8c
bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988)
On Windows, the LC_CTYPE is now set to the user preferred locale at
startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the
Python initialization. Previously, the LC_CTYPE locale was "C" at
startup, but changed when calling setlocale(LC_CTYPE, "") or
setlocale(LC_ALL, "").

pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to
behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the
LC_ALL anymore.

On Windows, standard streams like sys.stdout now always use
surrogateescape error handler by default (ignore the locale).
2018-08-29 11:25:15 +02:00
Raymond Hettinger 21786f5186
Improve commutativity of math.hypot() and math.dist() (GH-8984) 2018-08-28 22:47:24 -07:00
Victor Stinner 124b9eb4e4
bpo-34485: Add _Py_ClearStandardStreamEncoding() (GH-8982)
* Move Py_SetStandardStreamEncoding() from pylifecycle.c
  to coreconfig.c
* Add _Py_ClearStandardStreamEncoding() private function.
* pymain_free() now calls _Py_ClearStandardStreamEncoding().
* Add assertions add the end of _PyCoreConfig_Read()
* _PyCoreConfig_Copy(): rename COPY_STR_ATTR() macro
  to COPY_WSTR_ATTR().
* Fix get_stdio_errors() indentation.
2018-08-29 01:29:06 +02:00
Victor Stinner 2c8ddcf4f1
bpo-34485: Fix _Py_InitializeCore() for C locale coercion (GH-8979)
* _Py_InitializeCore() now sets the LC_CTYPE locale to the user
  preferred locale before checking if the C locale should be coerced
  or not in _PyCoreConfig_Read().
* Fix pymain_read_conf(): remember if the C locale has been coerced
  when the configuration should be read again if the encoding has
  changed.
2018-08-29 00:16:53 +02:00
Victor Stinner 9e4994d410
bpo-34485: Enhance init_sys_streams() (GH-8978)
Python now gets the locale encoding with C code to initialize the encoding
of standard streams like sys.stdout. Moreover, the encoding is now
initialized to the Python codec name to get a normalized encoding name and
to ensure that the codec is loaded. The change avoids importing
_bootlocale and _locale modules at startup by default.

When the PYTHONIOENCODING environment variable only contains an encoding,
the error handler is now is now set explicitly to "strict".

Rename also get_default_standard_stream_error_handler() to
get_stdio_errors().

Reduce the buffer to format the "cpXXX" string (Windows locale encoding).
2018-08-28 23:26:33 +02:00
Raymond Hettinger 74734f73ca
Fast path for exact floats in math.hypot() and math.dist() (GH-8949) 2018-08-26 19:38:31 -05:00
Zackery Spytz 1871a4a351 Remove unneeded PyErr_Clear() in _winapi_SetNamedPipeHandleState_impl() (GH-8281) 2018-08-26 01:10:14 +08:00
Sergey Fedoseev 90555eca44 bpo-34395: Don't free allocated memory on realloc fail in load_mark() in _pickle.c. (GH-8788) 2018-08-25 13:41:58 +03:00
Sergey Fedoseev 86b89916d1 Fix upsizing of marks stack in pickle module. (GH-8860)
Previously marks stack was upsized even there was space for additional item.
2018-08-25 10:54:40 +03:00
Gregory P. Smith 76be0fffff
bpo-13312: Avoid int underflow in time year. (GH-8912)
Avoids an integer underflow in the time module's year handling code.
2018-08-24 18:08:50 -07:00
Alexey Izbyshev 498845368f closes bpo-34471: _datetime: Add missing NULL check to tzinfo_from_isoformat_results. (GH-8869)
Reported by Svace static analyzer.
2018-08-24 08:53:16 -07:00
Paul Ganssle 096329f0b2 bpo-34454: fix .fromisoformat() methods crashing on inputs with surrogate code points (GH-8862)
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.

Important notes:

1.  It is possible to pass a non-UTF-8 string as a separator to the
    `.isoformat()` methods.
2.  The pure-Python `datetime.fromisoformat()` implementation accepts
    strings with a surrogate as the separator.

In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
2018-08-23 18:06:20 +03:00
Michael Osipov 48ce4897f8 bpo-34412: Make signal.strsignal() work on HP-UX (GH-8786)
Introduce a configure check for strsignal(3) which defines HAVE_STRSIGNAL for
signalmodule.c. Add some common signals on HP-UX. This change applies for
Windows and HP-UX.
2018-08-23 16:27:19 +03:00
Victor Stinner 89487f51b8
bpo-34207: Fix pymain_read_conf() for UTF-8 Mode (GH-8868)
bpo-34170, bpo-34207: pymain_read_conf() now sets Py_UTF8Mode to
config->utf8_mode. pymain_read_conf() calls indirectly
Py_DecodeLocale() and Py_EncodeLocale() which depend on Py_UTF8Mode.
2018-08-23 12:23:46 +02:00
Alexey Izbyshev c583919ffc bpo-34462: Add missing NULL check to _copy_raw_string() (GH-8863)
Reported by Svace static analyzer.
2018-08-22 21:27:32 +03:00
Berker Peksag e7d4b2f205
bpo-2122: Make mmap.flush() behave same on all platforms (GH-8692)
Previously, its behavior was platform-dependent and there was no error checking
under Windows.
2018-08-22 21:21:05 +03:00
Alexey Izbyshev f8c06b0280 bpo-34456: pickle: Add missing NULL check to save_global(). (GH-8851)
Reported by Svace static analyzer.
2018-08-22 07:51:25 +03:00
Alexey Izbyshev cdbf50cba1 bpo-34441: Fix ABC.__subclasscheck__ crash on classes with invalid __subclasses__ (GH-8835)
The missing NULL check was reported by Svace static analyzer.
2018-08-20 23:04:19 +03:00
Michael Osipov 0e6e7a1e52 bpo-34419: selectmodule.c does not compile on HP-UX due to bpo-31938 (GH-8796)
Fix compile errors reported by HP aCC by fixing bugs introduced in 6dc57e2a20
which do not cause trouble on clang or GCC.

Patch by Michael Osipov.
2018-08-17 14:43:02 +03:00
Erik Janssens e6a4755e67 bpo-34217: Use lowercase for windows headers (GH-8472) 2018-08-16 15:40:50 +09:00
Sergey Fedoseev 67b9cc8e60 bpo-34395: Fix memory leaks caused by incautious usage of PyMem_Resize(). (GH-8756) 2018-08-16 07:27:50 +03:00
Berker Peksag e0b5b2096e
bpo-34384: Fix os.readlink() on Windows (GH-8740)
os.readlink() now accepts path-like and bytes objects on Windows.
Previously, support for path-like and bytes objects was only
implemented on Unix.

This commit also merges Unix and Windows implementations of
os.readlink() in one function and adds basic unit tests to increase
test coverage of the function.
2018-08-15 13:03:41 +03:00
Raymond Hettinger a47f394af8
Minor code clean-up. Don't alter the input vector. Use variables instead. GH-8748 2018-08-12 14:18:56 -07:00
Raymond Hettinger 00414597b4
Add more tests and assertions for math.hypot() and math.dist() (GH-8747) 2018-08-12 12:15:23 -07:00
Raymond Hettinger c630e10440
Factor-out common code. Also, optimize common cases by preallocating space on the stack. GH-8738
Improves speed by 9 to 10ns per call.
2018-08-11 18:39:05 -07:00
Raymond Hettinger 1399074535
Replace straight addition with Kahan summation and move max to the end (GH-8727) 2018-08-11 11:26:36 -07:00
Alex Grönholm a7548230ff Fixed inconsistency in string handling in the Task C implementation (GH-8717) 2018-08-09 16:49:49 -04:00
Alex Grönholm cca4eec3c0 bpo-34270: Make it possible to name asyncio tasks (GH-8547)
Co-authored-by: Antti Haapala <antti.haapala@anttipatterns.com>
2018-08-08 17:06:47 -04:00
Zackery Spytz 8b58468032 Remove unneeded PyErr_Clear() calls after PyErr_Print(). (GH-8699)
PyErr_Print() always clears the error indicator, so there is no need
to call PyErr_Clear() immediately afterwards.
2018-08-07 17:26:58 +03:00
Victor Stinner d807862677
bpo-34170: Fix pymain_run_file() (GH-8660)
bpo-34170, bpo-34326: Fix pymain_run_file(): use
PyRun_AnyFileExFlags(closeit=1) instead of calling fclose(fp)
explicitly to close the input file before running the code.
2018-08-03 23:54:06 +02:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
INADA Naoki 2ebd3813af
Fix docstring of Profiler class (GH-8651) 2018-08-03 18:09:57 +09:00
Victor Stinner 72ec3193b5
bpo-34170: Cleanup pymain_run_filename() (GH-8631)
* Inline pymain_run_file() and pymain_open_filename() into
  pymain_run_filename()
* Created pymain_run_stdin() which is pymain_run_filename() with
  filename=NULL
* Rename pymain_run_filename() to pymain_run_file()
2018-08-02 19:34:20 +02:00
jdemeyer fc512e3e06 bpo-34287: Do not use second argument of METH_NOARGS functions (GH-8582) 2018-08-02 20:14:54 +09:00
Victor Stinner 80b762f010
bpo-31650: Remove _Py_CheckHashBasedPycsMode global config var (GH-8608)
bpo-31650, bpo-34170: Replace _Py_CheckHashBasedPycsMode with
_PyCoreConfig._check_hash_pycs_mode. Modify PyInit__imp() and
zipimport to get the parameter from the current interpreter core
configuration.

Remove Include/internal/import.h file.
2018-08-01 18:18:07 +02:00
Victor Stinner 6c785c0ebd
bpo-34170: Add Python/coreconfig.c for _PyCoreConfig (GH-8607)
* Add Include/coreconfig.h
* Move config_*() and _PyCoreConfig_*() functions from Modules/main.c
  to a new Python/coreconfig.c file.
* Inline _Py_ReadHashSeed() into config_init_hash_seed()
* Move global configuration variables to coreconfig.c
2018-08-01 17:56:14 +02:00
Victor Stinner fc96437db4
bpo-33499: Fix pymain_init_pycache_prefix() (GH-8596)
Fix a memory leak in pymain_init_pycache_prefix()
when PYTHONPYCACHEPREFIX and -X pycache_prefix are used.
2018-08-01 16:16:46 +02:00
Victor Stinner ea68d83933
bpo-34170: _PyCoreConfig_Read() defaults to argc=0 (GH-8595)
Add unit tests for argc and argv of _PyCoreConfig.
2018-08-01 03:07:18 +02:00
Victor Stinner 9851227382
bpo-34170: Rename _PyCoreConfig.unbuffered_stdip (GH-8594)
* Rename _PyCoreConfig.unbuffered_stdio to buffered_stdio
* Rename _PyCoreConfig.debug to parser_debug
2018-08-01 03:07:00 +02:00
Victor Stinner a4d20b2e5e
bpo-34170: Py_Main() updates config when setting Py_InspectFlag (GH-8593) 2018-08-01 02:57:45 +02:00
Victor Stinner b75d7e2435
bpo-34170: Add _PyCoreConfig._frozen parameter (GH-8591)
Modify frozenmain.c to use _Py_InitializeFromConfig().
2018-08-01 02:13:04 +02:00
Raymond Hettinger 9c18b1ae52
bpo-33089: Add math.dist() for computing the Euclidean distance between two points (GH-8561) 2018-07-31 00:45:49 -07:00
Serhiy Storchaka 9d5727326a
bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)
* Fix integer overflow in os.readv(), os.writev(), os.preadv()
  and os.pwritev() and in os.sendfile() with headers or trailers
  arguments (on BSD-based OSes and MacOS).

* Fix sending the part of the file in os.sendfile() on MacOS.
  Using the trailers argument could cause sending more bytes from
  the input file than was specified.

Thanks Ned Deily for testing on 32-bit MacOS.
2018-07-31 10:24:54 +03:00
Serhiy Storchaka f1d36d8efa
bpo-33729: Fix issues with arguments parsing in hashlib. (GH-8346)
* help(hashlib) didn't work because of incorrect module name in blake2b and
  blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
  accepted keyword argument "string" for binary data, but documented as
  accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
  keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
  which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
  positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
  shake_*() and keccak_*() incorrectly referred to sha3_224.

Also made the following enhancements:

* More accurately specified input and result types for strings, bytes and
  bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.
2018-07-31 09:50:16 +03:00
Sergey Fedoseev b229b072a9 Remove creation of a list for row_cast_map in pysqlite_cursor_init() (GH-8494)
This list is never used: if detect_types is on, this list will be
replaced with another one before row_cast_map is used, if
detect_types is off, row_cast_map is not used at all.
2018-07-31 00:11:50 +03:00
Stéphane Wirtel b7fd73896d bpo-34231: PYTHONBREAKPOINT is not documented on python --help (GH-8475) 2018-07-29 11:27:16 +01:00
Raymond Hettinger c6dabe37e3
bpo-33089: Multidimensional math.hypot() (GH-8474) 2018-07-28 07:48:04 -07:00
Sergey Fedoseev aee632dfbb Remove some unused code in _pysqlite_query_execute() (GH-8495)
Unused since commit ab994ed8b9.
2018-07-27 13:06:03 +03:00
Victor Stinner d145775b45
bpo-34170: Cleanup pymain_read_conf() (GH-8476)
* Config: Rename ignore_environment field to use_environment.
* _PyCoreConfig_Read(): if isolated is set, use_environment and
  site_import are now always set to 0.
* Inline pymain_free_raw() into pymain_free()
* Move config_init_warnoptions() call into pymain_read_conf_impl()
* _PyCoreConfig_Read(): don't replace values if they are already set:
  faulthandler, pycache_prefix, home.
2018-07-26 16:04:56 +02:00
Serhiy Storchaka e0d67f17cc
bpo-34201: Make ndarray.readonly a bool and use stricter tests in test_buffer. (GH-8414) 2018-07-26 13:23:03 +03:00
Serhiy Storchaka 7cb7bcff20
bpo-20260: Implement non-bitwise unsigned int converters for Argument Clinic. (GH-8434) 2018-07-26 13:22:16 +03:00
Serhiy Storchaka 323748ad74
bpo-34197: Make _csv.Dialect attributes booleans. (GH-8440)
Attributes skipinitialspace, doublequote and strict are now
booleans instead of integers 0 or 1.
2018-07-26 13:21:09 +03:00
Victor Stinner ecf411c59e
bpo-34170: Enhance _PyCoreConfig_Read() (GH-8468)
* Inline cmdline_get_env_flags() into config_read_env_vars():
  _PyCoreConfig_Read() now reads much more environment variables like
  PYTHONVERBOSE.
* Allow to override faulthandler and allocator even if dev_mode=1.
  PYTHONMALLOC is now the priority over PYTHONDEVMODE.
* Fix _PyCoreConfig_Copy(): copy also install_signal_handlers,
  coerce_c_locale and coerce_c_locale_warn
* _PyCoreConfig.install_signal_handlers default is now 1: install
  signals by default
* Fix also a compiler warning: don't define _PyPathConfig type twice.
2018-07-26 02:37:22 +02:00
Victor Stinner 60b04c9f6f
bpo-34228: Allow PYTHONTRACEMALLOC=0 (GH-8467)
PYTHONTRACEMALLOC=0 environment variable and -X tracemalloc=0 command
line option are now allowed to disable explicitly tracemalloc at
startup.
2018-07-25 19:23:53 +02:00
Ammar Askar 96d1e69a12 bpo-29097: Forego fold detection on windows for low timestamp values (GH-2385)
On Windows, passing a negative value to local results in an OSError because localtime_s on Windows does not support negative timestamps. Unfortunately this means that fold detection for timestamps between 0 and max_fold_seconds will result in this OSError since we subtract max_fold_seconds from the timestamp to detect a fold. However, since we know there haven't been any folds in the interval [0, max_fold_seconds) in any timezone, we can hackily just forego fold detection for this time range on Windows.
2018-07-25 12:54:58 -04:00
Serhiy Storchaka ee98e7bbde
bpo-34218: Fix a leak in _elementtree.c introduced in GH-6769. (GH-8460) 2018-07-25 14:52:45 +03:00
Victor Stinner d3b191992e
bpo-34170: Cleanup pymain_run_python() (GH-8455)
* Move _PyMain.main_importer_path inside pymain_run_python
* If main_importer_path is non-NULL, update sys.path[0] earlier
2018-07-25 10:21:03 +02:00
Victor Stinner 1dc6e3906a
bpo-34170: Add _Py_InitializeFromConfig() (GH-8454)
* If _Py_InitializeCore() is called twice, the second call now copies
  and apply (partially) the new configuration.
* Rename _Py_CommandLineDetails to _PyCmdline
* Move more code into pymain_init(). The core configuration created
  by Py_Main() is new destroyed before running Python to reduce the
  memory footprint.
* _Py_InitializeCore() now returns the created interpreter.
  _Py_InitializeMainInterpreter() now expects an interpreter.
* Remove _Py_InitializeEx_Private(): _freeze_importlib now uses
  _Py_InitializeFromConfig()
* _PyCoreConfig_InitPathConfig() now only computes the path
  configuration if needed.
2018-07-25 02:49:17 +02:00
erikjanss 6cf8255912 bpo-34217: Use lowercase header for Windows (GH-8453) 2018-07-25 09:41:46 +09:00
Victor Stinner 53b7d4e402
bpo-34170: Add _PyCoreConfig.bytes_warning (GH-8447)
Add more fields to _PyCoreConfig:

* _check_hash_pycs_mode
* bytes_warning
* debug
* inspect
* interactive
* legacy_windows_fs_encoding
* legacy_windows_stdio
* optimization_level
* quiet
* unbuffered_stdio
* user_site_directory
* verbose
* write_bytecode

Changes:

* Remove pymain_get_global_config() and pymain_set_global_config()
  which became useless. These functions have been replaced by
  _PyCoreConfig_GetGlobalConfig() and
  _PyCoreConfig_SetGlobalConfig().
* sys.flags.dont_write_bytecode value is now restricted to 1 even if
  -B option is specified multiple times on the command line.
* PyThreadState_Clear() now uses the config from the current
  interpreter rather than using global Py_VerboseFlag
2018-07-25 01:37:05 +02:00
Victor Stinner d19d8d5279
bpo-34170: Add _PyCoreConfig.isolated (GH-8417)
* _PyCoreConfig: add isolated and site_import attributes
* Replace Py_IgnoreEnvironment with config->ignore_environment when
  reading the current configuration
* _PyCoreConfig_Read() now sets ignore_environment, utf8_mode,
  isolated and site_import from Py_IgnoreEnvironment, Py_UTF8Mode,
  Py_IsolatedFlag and Py_NoSiteFlag
* _Py_InitializeCore() now sets Py_xxx flags from the configuration
* pymain_read_conf() now uses _PyCoreConfig_Copy() to save/restore
  the configuration.
* Rename _disable_importlib of _PyCoreConfig to _install_importlib
* _PyCoreConfig_SetGlobalConfig() now also set
  Py_HashRandomizationFlag
* Replace !Py_NoSiteFlag with core_config->site_import
2018-07-24 13:55:48 +02:00
Serhiy Storchaka 02ec92fa7b
bpo-29209: Remove old-deprecated features in ElementTree. (GH-6769)
Also make getchildren() and getiterator() emitting
a DeprecationWarning instead of PendingDeprecationWarning.
2018-07-24 12:03:34 +03:00
Victor Stinner f2626ce6d4
bpo-34170: _PyCoreConfig_Read() leaves Py_IsolatedFlag unchanged (GH-8361)
* _PyCoreConfig_Read() no longer directly modifies Py_IsolatedFlag
  and Py_NoSiteFlag global configuration flags. The function now
  requires two pointers to integer, so these flags can be set later,
  to avoid side effets in _PyCoreConfig_Read().
* pathconfig_global_init() now leaves Py_IsolatedFlag and
  Py_NoSiteFlag unchanged.
* Fix pathconfig_global_init(): avoid computing the path
  configuration twice, use _PyCoreConfig_SetPathConfig().
2018-07-21 03:54:20 +02:00
Victor Stinner b1147e43da
bpo-34170: Rework _PyCoreConfig_Read() to avoid side effect (GH-8353)
Rework _PyCoreConfig_Read() function which *reads* core configuration
to not *modify* the path configuration.

A new _PyCoreConfig_SetPathConfig() function now recreates the path
configuration from the core configuration. This function is now
called very late in _Py_InitializeCore(), just before calling
initimport().

Changes:

* Add _PyCoreConfig.dll_path
* Py_SetPath() now fails with a fatal python error on memory
   allocation failure.
* Rename _PyPathConfig_Calculate() to _PyPathConfig_Calculate_impl()
* Replace _PyPathConfig_Init() with _PyPathConfig_Calculate(): the
  function now requires a _PyPathConfig
* Add _PyPathConfig_SetGlobal() to set the _Py_path_config global
  variable.
* Add _PyCoreConfig_InitPathConfig(): compute the path configuration
* Add _PyCoreConfig_SetPathConfig(): set path configuration from core
  configuration
* Rename wstrlist_append() to _Py_wstrlist_append()
* _Py_wstrlist_append() now handles integer overflow.
2018-07-21 02:06:16 +02:00
Victor Stinner fb47bca9ee
bpo-34008: Allow to call Py_Main() after Py_Initialize() (GH-8043)
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
2018-07-20 17:34:23 +02:00
ValeriyaSinevich ce75df3031 bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo. (GH-7911) 2018-07-19 15:34:03 -07:00
Serhiy Storchaka 5d4cb54800
bpo-34141: Optimized pickling simple non-recursive values. (GH-8318) 2018-07-18 10:10:49 +03:00
Zackery Spytz 28f07364f0 bpo-34068: _io__IOBase_close_impl could call _PyObject_SetAttrId with an exception set (GH-8282) 2018-07-17 09:31:44 +03:00
Antoine Pitrou 961d54c5c1
bpo-32430: Rename Modules/Setup.dist to Modules/Setup (GH-8229)
bpo-32430: Rename Modules/Setup.dist to Modules/Setup

Remove the necessity to copy the former manually to the latter when updating the local source tree.
2018-07-16 19:03:03 +02:00
Raymond Hettinger 66953f0ec6
Factor-out two substantially identical code blocks. (GH-8219) 2018-07-10 04:17:40 -07:00
INADA Naoki 5ac9e6eee5
bpo-33597: Reduce PyGC_Head size (GH-7043) 2018-07-10 17:19:53 +09:00
Benjamin Peterson 7762e4d387
prefix internal sqlite symbols with _pysqlite_ (GH-8215) 2018-07-09 21:20:23 -07:00
Benjamin Peterson d6d4432724
delete some unused pysqlite forward declarations (GH-8211) 2018-07-09 20:41:26 -07:00
Sergey Fedoseev 0830858aee bpo-34041: Allow creating deterministic functions in Connection.create_function() (GH-8086) 2018-07-08 10:09:20 +03:00
Benjamin Peterson 9b50a7f29b
Make PySimpleQueueType static. (GH-8175) 2018-07-07 15:21:15 -07:00
Benjamin Peterson 6cfe45a5c3
Make various internal _testbuffer symbols static. (GH-8160) 2018-07-07 11:18:38 -07:00
Benjamin Peterson 4629c0d531
Hide some symbols from _xxsubinterpreters. (GH-8151) 2018-07-06 23:28:35 -07:00
Benjamin Peterson cb4bae72c9
Make two PyModuleDef_Slot symbols static in _testmultiphase. (GH-8147) 2018-07-06 21:05:51 -07:00
Benjamin Peterson 3c8aae9ffe
Make TaskStepMethWrapper_Type and TaskWakeupMethWrapper_Type static. (GH-8127) 2018-07-05 22:39:34 -07:00
Benjamin Peterson 97ae32c92e
Make GenericAlias_Type and Generic_Type static. (GH-8076) 2018-07-03 22:39:09 -07:00
Benjamin Peterson b4588c2fff
Don't export pending_threadfunc from _testcapi. (GH-8075) 2018-07-03 22:30:56 -07:00
Tal Einat 6dc57e2a20
bpo-31938: Convert selectmodule.c to Argument Clinic (GH-4265) 2018-06-30 23:02:48 +03:00
Serhiy Storchaka fdb5a50ef3
bpo-25862: Fix several bugs in the _io module. (GH-8026)
They can be exposed when some C API calls fail due to lack of
memory.

* Failed Py_BuildValue() could cause an assertion error in the
  following TextIOWrapper.tell().
* input_chunk could be decrefed twice in TextIOWrapper.seek()
  after failed Py_BuildValue().
* initvalue could leak in StringIO.__getstate__() after failed
  PyDict_Copy().
2018-06-30 20:57:50 +03:00
Tal Einat 0cdf5f4289
bpo-32568: make select.epoll() and its docs consistent (#7840)
* `flags` is indeed deprecated, but there is a validation on its value for
  backwards compatibility reasons.  This adds mention of this in the docs.
* The docs say that `sizehint` is deprecated and ignored, but it is still
  used when `epoll_create1()` is unavailable. This adds mention of this in
  the docs.
* `sizehint=-1` is acceptable again, and is replaced with `FD_SETSIZE-1`.
  This is needed to have a default value available at the Python level,
  since `FD_SETSIZE` is not exposed to Python. (see: bpo-31938)
* Reject `sizehint=0` since it is invalid to pass on to `epoll_create()`.

The relevant tests have also been updated.
2018-06-30 15:43:23 +03:00
Zackery Spytz 23db935bcf bpo-25862: Fix assertion failures in io.TextIOWrapper.tell(). (GH-3918) 2018-06-29 13:14:58 +03:00
Zackery Spytz d2cbfffc84 bpo-25007: Add copy protocol support to zlib compressors and decompressors (GH-7940) 2018-06-27 21:04:51 +03:00
Benjamin Peterson 4e21100fa7
bpo-33956: update vendored expat to 2.2.5 (GH-7925) 2018-06-26 19:25:45 -07:00
jdemeyer 66ecefcfe7 Remove tp_print implementation (GH-7857) 2018-06-23 21:08:43 +09:00
Victor Stinner 9b7cf75721
bpo-33916: Fix bz2 and lzma init when called twice (GH-7843)
bz2, lzma: When Decompressor.__init__() is called twice, free the old
lock to not leak memory.
2018-06-23 10:35:23 +02:00
Xiang Zhang 44742e94c8
suppress compiler warnings in _cursesmodule.c (#7860) 2018-06-23 12:29:30 +08:00
Xiang Zhang b248e957a8
Fix compiling error when missing gdbm version macros (GH-7823) 2018-06-20 21:23:30 +08:00
Victor Stinner 00f9edb98d
bpo-33901: Add _gdbm._GDBM_VERSION (GH-7794)
* Fix also PyInit__gdbm() to catch errors.
* test.pythoninfo: add gdbm.version
* test_dbm_gnu now logs GDBM_VERSION when run in verbose mode.

* pythoninfo: rename function to collect_gdbm()
2018-06-19 23:29:22 +02:00
Giampaolo Rodola c7f02a9659
bpo-33671 / shutil.copyfile: use memoryview() with dynamic size on Windows (#7681)
bpo-33671
* use memoryview() with size == file size on Windows, see https://github.com/python/cpython/pull/7160#discussion_r195405230
* release intermediate (sliced) memoryview immediately
* replace "OSX" occurrences with "macOS"
* add some unittests for copyfileobj()
2018-06-19 08:27:29 -07:00
Pablo Galindo cb970730e3 bpo-33630: Fix using of freed memory in old versions of glicb for posix_spawn(). (GH-7685) 2018-06-19 11:19:50 +03:00
Carl Meyer b193fa996a bpo-33499: Add PYTHONPYCACHEPREFIX env var for alt bytecode cache location. (GH-6834)
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.

Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".

Patch by Carl Meyer.
2018-06-16 14:40:56 +10:00
Victor Stinner 6c5a4b3156
bpo-33824, bpo-32030: Fix pymain_read_conf() (GH-7712)
Fix "LC_ALL=C python3.7 -V": reset properly the command line parser
when the encoding changes after reading the Python configuration.

Fix pymain_read_conf(): use memset(0) to reset properly cmdline.
2018-06-16 00:06:28 +02:00
Wonsup Yoon d134809cd3 bpo-29456: Fix bugs in unicodedata.normalize: u1176, u11a7 and u11c3 (GH-1958)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
2018-06-15 20:03:14 +08:00
Eric Snow ab4a1988fd
bpo-33615: Re-enable subinterpreter tests. (#7552)
All the subinterpreter tests were disabled in gh-7513. This commit re-enables them, but leaves one bad test disabled. The test is partly causing problems because it makes assumptions about the availability of a high-level interpreters module (see PEP 554). So I'm disabling the test until such a high-level module is available.
2018-06-13 08:02:39 -06:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Ned Deily 4531ec74c4
bpo-31432: Revert unrelated code changes to _ssl.c and test_ssl (GH-7650) 2018-06-11 20:26:28 -04:00
Christian Heimes ef24b6c54d bpo-31432: Clarify ssl CERT_NONE/OPTIONAL/REQUIRED docs. (GH-3530)
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
2018-06-11 18:59:45 -04:00
Alexander Belopolsky 877b23202b
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does  not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
 d is naive.

This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.

In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.

* Updated the documentation.

Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification.  Thanks, Tim.
2018-06-10 17:02:58 -04:00
Zackery Spytz 69dccc397a Remove unneeded PyErr_Occurred() check in os_lseek_impl() (GH-7557)
This call became unneeded after the posix module was converted to the
Argument Clinic in 2f93635d34 and should
have been removed as part of that change.
2018-06-10 10:28:01 +03:00
Tal Einat 1b85c71a21
bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
Benjamin Peterson 7c69c1c0fb
update to Unicode 11.0.0 (closes bpo-33778) (GH-7439)
Also, standardize indentation of generated tables.
2018-06-06 20:14:28 -07:00
Victor Stinner 45e4efba7f
bpo-33781: audioop: enhance rounding double as int (GH-7447)
Move the floor() call into fbound() to call floor() on a double
rather than an int. The change should enhance the rounding.

Document also (int)double rounding mode.
2018-06-06 15:50:50 +02:00