Commit Graph

10588 Commits

Author SHA1 Message Date
Victor Stinner 6c5a4b3156
bpo-33824, bpo-32030: Fix pymain_read_conf() (GH-7712)
Fix "LC_ALL=C python3.7 -V": reset properly the command line parser
when the encoding changes after reading the Python configuration.

Fix pymain_read_conf(): use memset(0) to reset properly cmdline.
2018-06-16 00:06:28 +02:00
Wonsup Yoon d134809cd3 bpo-29456: Fix bugs in unicodedata.normalize: u1176, u11a7 and u11c3 (GH-1958)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
2018-06-15 20:03:14 +08:00
Eric Snow ab4a1988fd
bpo-33615: Re-enable subinterpreter tests. (#7552)
All the subinterpreter tests were disabled in gh-7513. This commit re-enables them, but leaves one bad test disabled. The test is partly causing problems because it makes assumptions about the availability of a high-level interpreters module (see PEP 554). So I'm disabling the test until such a high-level module is available.
2018-06-13 08:02:39 -06:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Ned Deily 4531ec74c4
bpo-31432: Revert unrelated code changes to _ssl.c and test_ssl (GH-7650) 2018-06-11 20:26:28 -04:00
Christian Heimes ef24b6c54d bpo-31432: Clarify ssl CERT_NONE/OPTIONAL/REQUIRED docs. (GH-3530)
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
2018-06-11 18:59:45 -04:00
Alexander Belopolsky 877b23202b
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does  not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
 d is naive.

This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.

In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.

* Updated the documentation.

Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification.  Thanks, Tim.
2018-06-10 17:02:58 -04:00
Zackery Spytz 69dccc397a Remove unneeded PyErr_Occurred() check in os_lseek_impl() (GH-7557)
This call became unneeded after the posix module was converted to the
Argument Clinic in 2f93635d34 and should
have been removed as part of that change.
2018-06-10 10:28:01 +03:00
Tal Einat 1b85c71a21
bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
Benjamin Peterson 7c69c1c0fb
update to Unicode 11.0.0 (closes bpo-33778) (GH-7439)
Also, standardize indentation of generated tables.
2018-06-06 20:14:28 -07:00
Victor Stinner 45e4efba7f
bpo-33781: audioop: enhance rounding double as int (GH-7447)
Move the floor() call into fbound() to call floor() on a double
rather than an int. The change should enhance the rounding.

Document also (int)double rounding mode.
2018-06-06 15:50:50 +02:00
Zackery Spytz e9e3976057 bpo-33767: Fix improper use of SystemError by mmap.mmap objects (GH-7381)
Raise TypeError instead of SystemError for unsupported operations.
2018-06-05 15:59:41 +03:00
Eric Snow 63799136e6
bpo-33615: Re-enable a subinterpreter test. (gh-7251)
For bpo-32604 I added extra subinterpreter-related tests (see #6914), which caused a few buildbots to crash. This patch fixes the crash by ensuring that refcounts in channels are handled properly.
2018-06-01 18:45:20 -06:00
Eric Snow 6854e803b7
bpo-33724: Use the right format code for int64_t in subinterpreters code. (gh-7330) 2018-06-01 16:26:01 -06:00
pkerling e905c84494 bpo-30654: Do not reset SIGINT handler to SIG_DFL in finisignal (GH-7146) 2018-06-01 11:47:18 +02:00
Victor Stinner 58d1683255
bpo-33706: Fix pymain_parse_cmdline_impl() (GH-7283)
Fix a crash in Python initialization when parsing the command line
options.

Fix memcpy() size parameter: previously, we read one wchar_t after
the end of _PyOS_optarg. Moreover, don't copy the trailingg NUL
character: we write it manually anyway.

Thanks Christoph Gohlke for the bug report and the fix!
2018-05-31 15:09:28 +02:00
Serhiy Storchaka a5c42284e6
bpo-33677: Fix signatures of tp_clear handlers for AST and deque. (GH-7196) 2018-05-31 07:34:34 +03:00
Serhiy Storchaka c4653c9bf1
bpo-33622: Add checks for exceptions leaks in the garbage collector. (GH-7126)
* Failure in adding to gc.garbage is no longer fatal.
* An exception in tp_clear() no longer lead to crash (though tp_clear() should not leave exceptions).
2018-05-29 18:50:10 +03:00
Yury Selivanov 416c1ebd98
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174) 2018-05-28 17:54:02 -04:00
Yury Selivanov 35230d08e0
bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (#7080) 2018-05-28 11:11:31 -04:00
Serhiy Storchaka 19de8b3dd7
bpo-33644: Fix signatures of tp_finalize handlers in testing code. (GH-7111) 2018-05-26 10:51:58 +03:00
Serhiy Storchaka 17d8830312 bpo-32493: Fix uuid.uuid1() on FreeBSD. (GH-7099)
Use uuid_enc_be() if available to encode UUID to bytes as big endian.
2018-05-25 00:45:09 +02:00
Serhiy Storchaka 301e3cc8a5
bpo-33622: Fix issues with handling errors in the GC. (GH-7078)
* Fixed a leak when the GC fails to add an object with __del__ into
  the gc.garbage list.
* PyGC_Collect() can now be called when an exception is set and
  preserves it.
* Fixed an undefined behavior with comparing a dead pointer with NULL.
2018-05-24 15:19:29 +03:00
INADA Naoki 1179f4b40f
bpo-33583: Add note in PyObject_GC_Resize() doc (GH-7021) 2018-05-21 18:35:41 +09:00
Serhiy Storchaka 6655354afc
bpo-33584: Fix several minor bugs in asyncio. (GH-7003)
Fix the following bugs in the C implementation:

* get_future_loop() silenced all exceptions raised when look up the get_loop
  attribute, not just an AttributeError.
* enter_task() silenced all exceptions raised when look up the current task,
  not just a KeyError.
* repr() was called for a borrowed link in enter_task() and task_step_impl().
* str() was used instead of repr() in formatting one error message (in
  Python implementation too).
* There where few reference leaks in error cases.
2018-05-20 16:30:31 +03:00
Carl Meyer 4857543a09 Fix typo in error message when decoding PYTHONPATH. (GH-6981) 2018-05-19 18:48:22 -04:00
Carl Meyer c0ee341b29 Fix Windows build of Python for latest WinSDK. (GH-6874) 2018-05-17 14:03:59 -04:00
Eric Snow 3ab0136ac5
bpo-32604: Implement force-closing channels. (gh-6937)
This will make it easier to clean up channels (e.g. when used in tests).
2018-05-17 10:27:09 -04:00
Zvezdan Petkovic c2f082e9d1 bpo-13631: Fix the order of initialization for readline libedit on macOS. (GH-6915)
The editline emulation needs to be initialized *after* the name is
defined. This fixes the long open issue.
2018-05-17 02:45:10 -04:00
INADA Naoki d852142cd7
Replace _PyGC_REFS macros with higher level macros (GH-6852)
Only gcmodule.c uses _PyGC_REFS* macros now.
This makes easy to read GC code.
2018-05-17 11:07:21 +09:00
Eric Snow 6d2cd9036c
bpo-32604: Improve subinterpreter tests. (#6914)
Add more tests for subinterpreters. This patch also fixes a few small defects in the channel implementation.
2018-05-16 15:04:57 -04:00
Alex Gaynor 55e53c3093
Remove an unneeded call into OpenSSL (GH-6887) 2018-05-16 13:02:06 -04:00
Serhiy Storchaka d8dcd57edb
bpo-33454: Fix arguments parsing in _xxsubinterpreters.channel_close(). (GH-6747) 2018-05-16 01:08:09 +03:00
Christian Heimes 67c4801663 bpo-32257: Add ssl.OP_NO_RENEGOTIATION (GH-5904)
The ssl module now contains OP_NO_RENEGOTIATION constant, available with
OpenSSL 1.1.0h or 1.1.1.

Note, OpenSSL 1.1.0h hasn't been released yet.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-15 16:25:40 -04:00
Segev Finer 735abadd5b bpo-16865: Support arrays >=2GB in ctypes. (GH-3006) 2018-05-15 02:54:29 +03:00
Alex Gaynor 00717a46a1
Fixed an unused variable warning introduced in GH-6800 (GH-6816) 2018-05-14 16:48:14 -04:00
Alex Gaynor f04224210d
Remove `ifdef` check for an OpenSSL version (0.9.6) we don't support (GH-6800) 2018-05-14 11:51:45 -04:00
Serhiy Storchaka b00854caa0
bpo-20171: Convert the _curses and _curses_panel modules to Argument Clinic. (GH-4251) 2018-05-10 11:27:23 +03:00
Oren Milman d518d8bc8d bpo-21983: Fix a crash in ctypes.cast() when passed a ctypes structured data type (GH-3859) 2018-05-09 14:38:56 -07:00
Serhiy Storchaka d54cfb160c
bpo-33441: Make the sigset_t converter available in other modules. (GH-6720)
* Expose the sigset_t converter via private API _Py_Sigset_Converter().
* Use Argument Clinic for parsing sigset_t in signalmodule.c.
* Raise ValueError instead OverflowError for integers out of
  the C long range.

Based on patch by Pablo Galindo Salgado.
2018-05-08 07:48:50 +03:00
Antoine Pitrou 9d3627e311
bpo-33332: Add signal.valid_signals() (GH-6581) 2018-05-04 13:00:50 +02:00
Serhiy Storchaka ef347535f2
bpo-20104: Improve error handling and fix a reference leak in os.posix_spawn(). (#6332) 2018-05-01 16:45:04 +03:00
Victor Stinner ca405017d5
bpo-27645, sqlite: Fix integer overflow on sleep (#6594)
Use the _PyTime_t type and round away from zero (ROUND_UP,
_PyTime_ROUND_TIMEOUT) the sleep duration, when converting a Python
object to seconds and then to milliseconds. Raise an OverflowError in
case of overflow.

Previously the (int)double conversion rounded towards zero
(ROUND_DOWN).
2018-04-30 12:22:17 +02:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Serhiy Storchaka 2e38cc3933
bpo-33383: Fix crash in get() of the dbm.ndbm database object. (#6630) 2018-04-29 12:38:06 +03:00
Antoine Pitrou 25038ecfb6
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575)
Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
reserved signal numbers between 1 and NSIG.  The `range(1, NSIG)` idiom
is commonly used to select all signals for blocking with `pthread_sigmask`.
So we ignore the sigaddset() return value until we expose sigfillset()
to provide a better idiom.
2018-04-23 20:53:33 +02:00
xdegaye 063db62aab bpo-32232: by default, Setup modules are no longer built with -DPy_BUILD_CORE (GH-6489)
Setup modules are no longer built with -DPy_BUILD_CORE by default,
as using that flag may now require including additional internal-only header files.

Instead, only the modules that specifically need it use that setting.
2018-04-21 01:03:49 +10:00
Serhiy Storchaka e5362eaa75
bpo-33308: Fix a crash in the parser module when convert an ST object. (#6519)
Converting with line_info=False and col_info=True crashed before.
2018-04-19 01:55:37 +03:00
Devon R f178028f37 bpo-33253: Fix xxsubtype.bench() to accept correct str signature. (GH-6439) 2018-04-10 08:21:18 +03:00
Łukasz Langa c51d8c9ba6
bpo-23403: Bump pickle.DEFAULT_PROTOCOL to 4 (#6355)
This makes performance better and produces shorter pickles. This change is backwards compatible up to the oldest currently supported version of Python (3.4).
2018-04-03 23:06:53 -07:00