Commit Graph

24234 Commits

Author SHA1 Message Date
Antoine Pitrou 4bd41c9b52
bpo-32025: Add time.thread_time() (#4410)
* bpo-32025: Add time.thread_time()

* Add missing #endif

* Add NEWS blurb

* Add docs and whatsnew

* Address review comments

* Review comments
2017-11-15 22:52:21 +01:00
Dong-hee Na 762b9571c9 bpo-32018: Fix inspect.signature repr to follow PEP 8 (#4408) 2017-11-15 13:30:59 -05:00
Serhiy Storchaka f8a4c03ede
bpo-30399: Get rid of trailing comma in the repr of BaseException. (#1650) 2017-11-15 17:53:28 +02:00
Serhiy Storchaka 00987f6230
bpo-32011: Revert "Issue #15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (#4381)
Simplify the reverted code.

This reverts commit e9bbe8b87b.
2017-11-15 17:41:05 +02:00
Serhiy Storchaka ddbce13786
bpo-32023: Disallow genexprs without parenthesis in class definitions. (#4400) 2017-11-15 17:39:37 +02:00
Serhiy Storchaka edad8eebee
bpo-31949: Fixed several issues in printing tracebacks (PyTraceBack_Print()). (#4289)
* Setting sys.tracebacklimit to 0 or less now suppresses printing tracebacks.
* Setting sys.tracebacklimit to None now causes using the default limit.
* Setting sys.tracebacklimit to an integer larger than LONG_MAX now means using
  the limit LONG_MAX rather than the default limit.
* Fixed integer overflows in the case of more than 2**31 traceback items on
  Windows.
* Fixed output errors handling.
2017-11-15 17:38:52 +02:00
Serhiy Storchaka 6545256df9
bpo-32032: Test both implementations of module-level pickle API. (#4401) 2017-11-15 14:01:08 +02:00
Serhiy Storchaka 9165f77d5f
bpo-32012: Disallow trailing comma after genexpr without parenthesis. (#4382) 2017-11-15 08:49:40 +02:00
Andrey Egorov e1d62e0b7c bpo-32015: Asyncio looping during simultaneously socket read/write an… (#4386)
* bpo-32015: Asyncio cycling during simultaneously socket read/write and reconnection

* Tests fix

* Tests fix

* News add

* Add new unit tests
2017-11-14 11:18:59 +02:00
Serhiy Storchaka 9b6c60cbce
bpo-31979: Simplify transforming decimals to ASCII (#4336)
in int(), float() and complex() parsers.

This also speeds up parsing non-ASCII numbers by around 20%.
2017-11-13 21:23:48 +02:00
Yury Selivanov ce12629c84
bpo-28369: Enhance transport socket check in add_reader/writer (#4365) 2017-11-13 13:38:22 -05:00
xdegaye d7d4fea4a3
bpo-29181: Skip test_tarfile tests on PermissionError raised by Android (GH-4375) 2017-11-12 18:02:06 +01:00
xdegaye 6a55d09573
bpo-29180: Skip test_os tests on PermissionError raised by Android (GH-4374) 2017-11-12 17:57:04 +01:00
xdegaye 92c2ca7633
bpo-28759: Skip some tests on PermissionError raised by Android (GH-4350)
Access to mkfifo(), mknod() and hard link creation is controled
by SELinux on Android.
Also remove test.support.android_not_root.
2017-11-12 17:31:07 +01:00
xdegaye e0582a37c8
bpo-30696: Fix the REPL looping endlessly when no memory (GH-4160) 2017-11-12 16:50:48 +01:00
xdegaye 1588be66d7
bpo-28180: Fix the implementation of PEP 538 on Android (GH-4334) 2017-11-12 12:45:59 +01:00
benfogle 9703f092ab bpo-31976: Fix race condition when flushing a file is slow. (#4331) 2017-11-10 22:03:40 +01:00
Yury Selivanov 4652bf2acc
Rewrite asyncio test to be more meaningful (#4363) 2017-11-10 15:34:17 -05:00
Brian Curtin 9f914a01af
bpo-31985: Deprecate openfp in aifc, sunau, and wave (#4344)
The openfp functions of aifp, sunau, and wave had pointed to the open
function of each module since 1993 as a matter of backwards
compatibility. In the case of aifc.openfp, it was both undocumented
and untested. This change begins the formal deprecation of those
openfp functions, with their removal coming in 3.9.

This additionally adds a TODO in test_pyclbr around using aifc.openfp,
though it shouldn't be changed until removal in 3.9.
2017-11-10 11:38:25 -05:00
Serhiy Storchaka 5e0df74b3b
bpo-31999: Fix test_venv in case the zlib module is not available. (#4359) 2017-11-10 12:09:39 +02:00
Serhiy Storchaka a1718bc7e0
bpo-31998: Fix test_zipapp in case the zlib module is not available. (#4358) 2017-11-10 12:09:24 +02:00
Paul Ganssle 191e993365 bpo-31222: Make (datetime|date|time).replace return subclass type in Pure Python (#4176) 2017-11-09 13:34:29 -08:00
Antoine Pitrou 0a2ff23fe6
Silence error output in test_concurrent_futures (bpo-21423) (#4347)
* Silence error output in test_concurrent_futures (bpo-21423)
2017-11-09 15:33:43 +01:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Sanyam Khurana 7973e279a2 bpo-21862: Add -m option to cProfile for profiling modules (#4297)
* bpo-21862: Add -m option to cProfile for profiling modules
2017-11-08 11:50:56 +01:00
Suren Nihalani c62f0cb3b1 bpo-31620: have asyncio/queues not leak memory when you've exceptions during waiting (#3813) 2017-11-07 20:35:23 +03:00
Antoine Pitrou 921e9432a1
bpo-31970: Reduce performance overhead of asyncio debug mode. (#4314)
* bpo-31970: Reduce performance overhead of asyncio debug mode.
2017-11-07 17:23:29 +01:00
Serhiy Storchaka 39a156c505
Fix the sizeof test for dicts with shared keys. (#4311)
By accident the size of the empty dict keys object matched the
size of values array.
2017-11-07 15:08:09 +02:00
Marat Sharafutdinov 69cfed1cd7 Add asyncio.Handle.cancelled() method (#2388) 2017-11-07 12:06:05 +03:00
Nir Soffer ad455cd924 bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)
blocksize was hardcoded to 8192, preventing efficient upload when using
file-like body. Add blocksize argument to __init__, so users can
configure the blocksize to fit their needs.

I tested this uploading data from /dev/zero to a web server dropping the
received data, to test the overhead of the HTTPConnection.send() with a
file-like object.

Here is an example 10g upload with the default buffer size (8192):

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 17.53 seconds (584.00m/s)

real	0m17.574s
user	0m8.887s
sys	0m5.971s

Same with 512k blocksize:

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 6.60 seconds (1551.15m/s)

real	0m6.641s
user	0m3.426s
sys	0m2.162s

In real world usage the difference will be smaller, depending on the
local and remote storage and the network.

See https://github.com/nirs/http-bench for more info.
2017-11-06 13:16:37 -08:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
Serhiy Storchaka d4d79bc1ff
bpo-28564: Use os.scandir() in shutil.rmtree(). (#4085)
This speeds up it to 20-40%.
2017-11-04 14:16:35 +02:00
Antoine Pitrou 63ff4131af
bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor (#4241)
* bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor

* Fix docstring
2017-11-04 11:05:49 +01:00
Jack O'Connor dcfb0e3c04 bpo-31933: fix blake2 multi-byte params on big endian platforms (#4250)
All Blake2 params have to be encoded in little-endian byte order. For
the two multi-byte integer params, leaf_length and node_offset, that
means that assigning a native-endian integer to them appears to work on
little-endian platforms, but gives the wrong result on big-endian. The
current libb2 API doesn't make that very clear, and @sneves is working
on new API functions in the GH issue above. In the meantime, we can work
around the problem by explicitly assigning little-endian values to the
parameter block.

See https://github.com/BLAKE2/libb2/issues/12.
2017-11-03 20:02:41 +01:00
Serhiy Storchaka a7723d8b09
bpo-31924: Fix test_curses on NetBSD 8. (#4228) 2017-11-03 20:29:33 +02:00
Antoine Pitrou cbe1756e3e
bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed (#3247)
* bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed

* Avoid mucking with process state in test.
Add a warning if the semaphore process died, as semaphores may then be leaked.

* Add NEWS entry
2017-11-03 14:31:38 +01:00
Antoine Pitrou fc6b348b12
bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary (#3246)
* bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary.

* Fix test on Windows

* Add NEWS entry

* Adopt a different approach: ignore SIGINT and SIGTERM, as in semaphore tracker.

* Fix comment

* Make sure the test doesn't muck with process state

* Also test previously-started processes

* Update 2017-08-30-17-59-36.bpo-31308.KbexyC.rst

* Avoid masking SIGTERM in forkserver.  It's not necessary and causes a race condition in test_many_processes.
2017-11-03 13:34:22 +01:00
Victor Stinner c29b585fd4
bpo-31784: Implement PEP 564: add time.time_ns() (#3989)
Add new time functions:

* time.clock_gettime_ns()
* time.clock_settime_ns()
* time.monotonic_ns()
* time.perf_counter_ns()
* time.process_time_ns()
* time.time_ns()

Add new _PyTime functions:

* _PyTime_FromTimespec()
* _PyTime_FromNanosecondsObject()
* _PyTime_FromTimeval()

Other changes:

* Add also os.times() tests to test_os.
* pytime_fromtimeval() and pytime_fromtimeval() now return
  _PyTime_MAX or _PyTime_MIN on overflow, rather than undefined
  behaviour
* _PyTime_FromNanoseconds() parameter type changes from long long to
  _PyTime_t
2017-11-02 07:28:27 -07:00
Vincent Michel e314853d57 bpo-31307: Make ConfigParser.read() accept bytes objects (GH-3420) 2017-11-02 15:47:04 +03:00
Serhiy Storchaka 4f469c0966
bpo-27666: Fixed stack corruption in curses.box() and curses.ungetmouse(). (#4220) 2017-11-01 20:48:49 +02:00
Serhiy Storchaka 7e68790f3d
bpo-15037: Add a workaround for getkey() in curses for ncurses 5.7 and earlier. (#3826)
Skip a test for unget_wch()/get_wch() on OpenBSD since they are broken
in ncurses 5.7.
2017-11-01 16:03:40 +02:00
jimmylai d1e34031f6 [asyncio] bpo-30423: add regression test for orphan future causes "RuntimeError: Event loop stopped before Future completed." (#3295)
* call remove_done_callback in finally section

* [asyncio] bpo-30423 bug: add regression test for orphan future causes "RuntimeError: Event loop stopped before Future completed."
2017-11-01 15:54:45 +02:00
Serhiy Storchaka 894ebd065e
bpo-31919: Fix building the curses module on OpenIndiana. (#4211) 2017-11-01 14:34:20 +02:00
Victor Stinner 280c22a82a
Fix test_socket.test_create_connection() (#4206)
bpo-31910: test_create_connection() now catchs also EADDRNOTAVAIL to
fix the test on Travis CI.
2017-10-31 19:48:14 -07:00
Victor Stinner afd055a59f
pythoninfo: add Py_DEBUG (#4198) 2017-10-31 08:41:10 -07:00
Victor Stinner e1a470bd6a
test_capi.test__testcapi() becomes more verbose (#4197)
Write the name of each subtest on a new line to help debugging when
a test does crash Python.
2017-10-31 08:40:59 -07:00
Serhiy Storchaka db91e0fe24
bpo-31897: Convert unexpected errors when read bogus binary plists into InvalidFileException. (#4171) 2017-10-31 14:05:53 +02:00
Serhiy Storchaka baac01e629
bpo-31891: Fix building the curses module on NetBSD. (#4165) 2017-10-31 13:56:44 +02:00
Victor Stinner 19f68301a1
bpo-31629: Add support.SaveSignals (#4183)
test_curses now saves/restores signals. On FreeBSD, the curses module
sets handlers of some signals, but don't restore old handlers when
the module is deinitialized.
2017-10-31 03:14:01 -07:00
Quentin Dawans fe4ea9cf1e bpo-31245: Asyncio unix socket datagram (#3164) 2017-10-30 09:43:02 -04:00
Serhiy Storchaka a2314283ff
bpo-20047: Make bytearray methods partition() and rpartition() rejecting (#4158)
separators that are not bytes-like objects.
2017-10-29 02:11:54 +03:00
Terry Jan Reedy 5a4bbcd479
bpo-31836: Test_code_module now passes with sys.ps1, ps2 set (#4070) 2017-10-27 21:45:19 -04:00
Serhiy Storchaka 7351f9e5a9 bpo-31174: Improve the code of test_tools.test_unparse. (#4146) 2017-10-27 15:35:11 +03:00
Ivan Levkivskyi 8c83c23fa3 bpo-28936: Detect lexically first syntax error first (#4097)
Lexically first global and nonlocal syntax errors at given scope should be detected first.
2017-10-27 00:28:35 +03:00
Alexander Belopolsky 66c88ce30c Closes bpo-28281: Remove year (1-9999) limits on the weekday() function. (#4109)
Patch by Mark Gollahon.
2017-10-26 15:34:11 -04:00
Vitor Pereira 52ad72dd0a bpo-30553: Add status code 421 to http.HTTPStatus (GH-2589) 2017-10-26 21:49:19 +03:00
xdegaye 56d1f5ca32 bpo-30697: Fix PyErr_NormalizeException() when no memory (GH-2327) 2017-10-26 15:09:06 +02:00
Serhiy Storchaka 41c56940c6 bpo-21720: Restore the Python 2.7 logic in handling a fromlist. (#4118)
BytesWarning no longer emitted when the fromlist argument of
__import__() or the __all__ attribute of the module contain bytes
instances.
2017-10-26 10:41:59 +03:00
Benjamin Peterson 4eaf7f9490 fixes bpo-31866: remove code pertaining to AtheOS support (#4115)
We stop support this OS in 2007 with commit
19fab761b7. Let's finish.
2017-10-25 23:55:14 -07:00
Mario Corchero 32318930da Closes bpo-31800: Support for colon when parsing time offsets (#4015)
Add support to strptime to parse time offsets with a colon between the hour and the minutes.
2017-10-25 20:35:41 -04:00
Serhiy Storchaka 0f261583ba bpo-31664: Fix test_crypt for the openwall implementation of crypt. (#4116) 2017-10-25 16:30:13 +03:00
Nick Coghlan d7ac06126d bpo-31845: Fix reading flags from environment (GH-4105)
The startup refactoring means command line settings
are now applied after settings are read from the
environment.

This updates the way command line settings are applied
to account for that, ensures more settings are first read
from the environment in _PyInitializeCore, and adds a
simple test case covering the flags that are easy to check.
2017-10-25 12:11:26 +10:00
Serhiy Storchaka 3557b05c5a bpo-31690: Allow the inline flags "a", "L", and "u" to be used as group flags for RE. (#3885) 2017-10-24 23:31:42 +03:00
Alexander Belopolsky fdd9b217c6 Closes bpo-28292: Implemented Calendar.itermonthdays3() and itermonthdays4(). (#4079)
Calendar.itermonthdates() will now consistently raise an exception when a date falls outside of the 0001-01-01 through 9999-12-31 range. To support applications that cannot tolerate such exceptions, the new methods itermonthdays3() and itermonthdays4() are added. The new methods return tuples and are not restricted by the range supported by datetime.date.

Thanks @serhiy-storchaka for suggesting the itermonthdays4() method and for the review.
2017-10-24 13:17:10 -04:00
Serhiy Storchaka eab3ff72eb bpo-31664: Add support for the Blowfish method in crypt. (#3854) 2017-10-24 19:36:17 +03:00
Thomas Kluyver e968bc7357 bpo-30639: Lazily compute repr for error (#2132) 2017-10-24 08:42:36 -04:00
Victor Stinner 8e482bea21 bpo-31174: Fix test_tools.test_unparse (#4102)
test_unparse.DirectoryTestCase now stores the names sample to always
test the same files. It prevents false alarms when hunting reference
leaks.
2017-10-24 03:33:36 -07:00
Victor Stinner 01b5aab7bf bpo-31827: Remove os.stat_float_times() (GH-4061) 2017-10-24 02:02:00 -07:00
Serhiy Storchaka 3b66ebe772 bpo-31847: Fix commented out tests in test_syntax. (#4084)
SyntaxError now is raised instead of SyntaxWarning.
2017-10-24 00:27:14 +03:00
xdegaye 66caacf2f0 bpo-30817: Fix PyErr_PrintEx() when no memory (#2526) 2017-10-23 18:08:41 +02:00
Serhiy Storchaka 4ffd4653a7 bpo-31752: Fix possible crash in timedelta constructor called with custom integers. (#3947)
Bad remainder in divmod() in intermediate calculations caused an assertion failure.
2017-10-23 17:12:28 +03:00
andyclegg 7fed7bd8bb bpo-31756: subprocess.run should alias universal_newlines to text (#4049)
Improve human friendliness of the Popen API: Add text=False as a
keyword-only argument to subprocess.Popen along with a Popen
attribute .text_mode and set this based on the
encoding/errors/universal_newlines/text arguments.

The universal_newlines parameter and attribute are maintained for
backwards compatibility.
2017-10-22 19:01:19 -07:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
Cheryl Sabella 91dc64ba3f bpo-20825: Containment test for ip_network in ip_network. 2017-10-22 23:39:49 +02:00
Serhiy Storchaka bcbdd2f8db bpo-28286: Add tests for the mode argument of GzipFile. (#4074) 2017-10-22 13:18:21 +03:00
Antoine Pitrou daeefd2e04 bpo-28326: Fix multiprocessing.Process when stdout and/or stderr is closed or None. (#4073) 2017-10-22 11:40:31 +02:00
Serhiy Storchaka 73c4708630 Fix bytes warnings in test_struct (added in bpo-29802). (#4068) 2017-10-21 21:59:23 +03:00
Oren Milman db60a5bfa5 bpo-31781: Prevent crashes when calling methods of an uninitialized zipimport.zipimporter object (GH-3986) 2017-10-20 13:42:35 -07:00
Serhiy Storchaka 56cb465cc9 bpo-31825: Fixed OverflowError in the 'unicode-escape' codec (#4058)
and in codecs.escape_decode() when decode an escaped non-ascii byte.
2017-10-20 17:08:15 +03:00
Antoine Pitrou 525f40d231 bpo-31819: Add AbstractEventLoop.sock_recv_into() (#4051)
* bpo-31819: Add AbstractEventLoop.sock_recv_into()

* Add NEWS

* Add doc
2017-10-19 15:46:40 -04:00
jlacoline ea2ef5d0ca bpo-31632: fix set_protocol() in _SSLProtocolTransport (#3817) (#3817) 2017-10-19 13:49:57 -04:00
Łukasz Langa ce9e625445 bpo-31457: Don't omit inner ``process()`` calls with nested LogAdapters (#4044)
This used to be the case on Python 2.  Commit
212b590e11 changed the implementation for Python
3, making the `log()` method of LogAdapter call `logger._log()` directly.  This
makes nested log adapters not execute their ``process()`` method.  This patch
fixes the issue.

Also, now proxying `name`, too, to make `repr()` work with nested log adapters.

New tests added.
2017-10-19 10:24:55 -07:00
Łukasz Langa 0b6a118a45 bpo-31457: Make the `LoggerAdapter.manager` property settable (#4042)
Due to a bug in the initial fix, the setter was in fact creating a different
property.  This is now fixed.
2017-10-18 17:28:51 -07:00
Victor Stinner 884d13a55f time.clock() now emits a DeprecationWarning (GH-4020)
bpo-31803: time.clock() and time.get_clock_info('clock') now emit a
DeprecationWarning warning.

Replace time.clock() with time.perf_counter() in tests and demos.

Remove also hasattr(time, 'monotonic') in test_time since time.monotonic()
is now always available since Python 3.5.
2017-10-17 14:46:45 -07:00
Zane Bitter de86073a76 bpo-28603: Fix formatting tracebacks for unhashable exceptions (#4014) 2017-10-18 00:29:39 +03:00
Riccardo Coccioli 6cfa927ceb bpo-31334: Fix timeout in select.poll.poll() (GH-3277)
Always pass -1, or INFTIM where defined, to the poll() system call when
a negative timeout is passed to the poll.poll([timeout]) method in the
select module. Various OSes throw an error with arbitrary negative
values.
2017-10-17 12:45:07 -07:00
Pablo Galindo 2c15b29aea bpo-31786: Make functions in the select module blocking when timeout is a small negative value. (#4003) 2017-10-17 17:14:41 +03:00
brainfvck c75edabbb6 bpo-31558: Add gc.freeze() (#3705)
Freeze all the objects tracked by gc - move them to a permanent generation
and ignore all the future collections. This can be used before a POSIX
fork() call to make the gc copy-on-write friendly or to speed up collection.
2017-10-16 12:49:41 -07:00
Serhiy Storchaka 676db4bbf2 bpo-31792: Restore os.environ in test_buffer when import numpy. (#4007) 2017-10-16 10:38:14 +03:00
Victor Stinner a505ecdc50 bpo-31676: Fix test_imp.test_load_source() side effect (#3871)
test_load_source() now replaces the current __name__ module with a
temporary module to prevent side effects.
2017-10-13 13:47:49 -07:00
Victor Stinner ccef823939 bpo-25588: Fix regrtest when run inside IDLE (#3962)
When regrtest in run inside IDLE, sys.stdout and sys.stderr are not
TextIOWrapper objects and have no file descriptor associated:
sys.stderr.fileno() raises io.UnsupportedOperation.

Disable faulthandler and don't replace sys.stdout in that case.
2017-10-13 12:59:12 -07:00
INADA Naoki b22273ec5d bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872)
Pattern `[a-z]` with `IGNORECASE` flag can match to some non-ASCII characters.

Straightforward solution for this is using `IGNORECASE | ASCII` flag.
But users may subclass `Template` and override only `idpattern`. So we want to
avoid changing `Template.flags`.

So this commit uses local flag `-i` for `idpattern` and change `[a-z]` to `[a-zA-Z]`.
2017-10-13 16:02:23 +09:00
Serhiy Storchaka de07210077 bpo-30058: Fixed buffer overflow in select.kqueue.control(). (#1095) 2017-10-12 22:17:46 +03:00
Éric Araujo 03b9537dc5 bpo-31567: more decorator markup fixes in docs (GH-3959) (#3966) 2017-10-12 12:28:55 -04:00
Oren Milman 39ecb9c71b bpo-31728: Prevent crashes in _elementtree due to unsafe cleanup of Element.text and Element.tail (#3924) 2017-10-10 23:26:24 +03:00
Victor Stinner 6e3d6b5dc2 bpo-31701: faulthandler: ignore MSC and COM Windows exception (#3929)
bpo-31701: On Windows, faulthandler.enable() now ignores MSC and COM
exceptions.
2017-10-09 09:52:32 -07:00
Oren Milman 0ccc0f6c74 bpo-28280: Make PyMapping_Keys(), PyMapping_Values() and PyMapping_Items() always return a list (#3840) 2017-10-08 11:17:46 +03:00
Serhiy Storchaka f07e2b64df bpo-31642: Restore blocking "from" import by setting None in sys.modules. (#3834) 2017-10-08 10:44:10 +03:00
Zachary Ware 73ffd3f203 bpo-31724: Skip test_xmlrpc_net (GH-3921)
With the upgrade of buildbot.python.org from Buildbot 0.8.x to 0.9.x,
the xmlrpc interface has been removed.  This test is now skipped until
it can be rewritten to query a suitable substitute.
2017-10-08 01:31:23 -05:00
Serhiy Storchaka 79ba471488 bpo-31655: Validate keyword names in SimpleNamespace constructor. (#3909) 2017-10-07 22:59:35 +03:00
Yury Selivanov b8ab9d3fc8 bpo-31708: Allow async generator expressions in synchronous functions (#3905) 2017-10-06 02:58:28 -04:00
Yury Selivanov faa135acbf bpo-31709: Drop support for asynchronous __aiter__. (#3903) 2017-10-06 02:08:57 -04:00
Benjamin Peterson 86566702f3 weaken pthread_getcpuclockid test (more bpo-31596) (#3904)
There just isn't much we can assert in a portable way.
2017-10-05 22:50:42 -07:00
Jelle Zijlstra ac317700ce bpo-30406: Make async and await proper keywords (#1669)
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00
Barry Warsaw 36c1d1f1e5 PEP 553 built-in breakpoint() function (bpo-31353) (#3355)
Implement PEP 553, built-in breakpoint() with support from sys.breakpointhook(), along with documentation and tests.  Closes bpo-31353
2017-10-05 12:11:18 -04:00
Victor K 4d07189788 bpo-31556: asyncio.wait_for can cancel futures faster with timeout <= 0 (#3703) 2017-10-05 12:04:39 -04:00
Victor Stinner 11045c9d8a bpo-31178: Mock os.waitpid() in test_subprocess (#3896)
Fix test_exception_errpipe_bad_data() and
test_exception_errpipe_normal() of test_subprocess: mock os.waitpid()
to avoid calling the real os.waitpid(0, 0) which is an unexpected
side effect of the test.
2017-10-05 06:32:53 -07:00
pdox e14679c784 closes bpo-31596: Add an interface for pthread_getcpuclockid(3) (#3756) 2017-10-05 00:01:56 -07:00
Serhiy Storchaka 77732be801 bpo-30404: The -u option now makes the stdout and stderr streams totally unbuffered. (#1667) 2017-10-04 20:25:40 +03:00
Serhiy Storchaka 0b5e61ddca bpo-30397: Add re.Pattern and re.Match. (#1646) 2017-10-04 20:09:49 +03:00
Serhiy Storchaka 27c623c845 bpo-31675: Fix memory leaks in Tkinter's methods splitlist() and split() (#3866)
when pass a string larger than 2 GiB.

Decrease memory requirements for Tcl's bigmem tests.
2017-10-03 22:39:55 +03:00
diana 543386b7f0 bpo-31657: Add test coverage for the __debug__ case (GH-3450)
Update the compile tests for optimization levels to also check that
__debug__ blocks are included or excluded based on the optimization
level.
Patch by Diana Clarke.
2017-10-03 09:46:56 -07:00
Serhiy Storchaka 85c0b8941f bpo-31619: Fixed a ValueError when convert a string with large number of underscores (#3827)
to integer with binary base.
2017-10-03 14:13:44 +03:00
Thomas Moreau e8c368df22 bpo-31540: Allow passing multiprocessing context to ProcessPoolExecutor (#3682) 2017-10-03 11:53:17 +02:00
Victor Stinner e6cfdefa0c bpo-31510: Fix multiprocessing test_many_processes() on macOS (#3857)
On macOS, a process can exit with -SIGKILL if it is killed "early"
with SIGTERM.
2017-10-02 08:27:34 -07:00
Antoine Pitrou 1023dbbcb7 bpo-31516: current_thread() should not return a dummy thread at shutdown (#3673)
bpo-31516: current_thread() should not return a dummy thread at shutdown
2017-10-02 16:42:15 +02:00
Cornelius Diekmann e6f62f69f0 bpo-31158: Fix nondeterministic read in test_pty (#3808)
* bpo-31158: Fix nondeterministic read in test_pty

* Reuse existing readline implementation from io.

Thx to @pitrou

* Updated comment

Ideally, this commit is fixuped into the previous commit. Since there is
already a comment on github, I won't rebase.
2017-10-02 02:39:55 -07:00
Oren Milman 7dc46d8cf5 bpo-31592: Fix an assertion failure in Python parser in case of a bad unicodedata.normalize(). (#3767) 2017-09-30 20:16:24 +03:00
Serhiy Storchaka f4ea642cb6 bpo-31627: Make test_mailbox be lenient to empty hostname. (#3821) 2017-09-30 17:10:57 +03:00
scoder 101a5e84ac bpo-31648: Improve ElementPath (#3835)
* Allow whitespace inside of ElementPath predicates.
* Add ElementPath predicate support for text comparison of the current node, like "[.='text']".
2017-09-30 16:35:21 +03:00
INADA Naoki 9811e80fd0 bpo-31581: Reduce the number of imports for functools (GH-3757) 2017-09-30 16:13:02 +09:00
James Sexton b24cd055ec bpo-30806 netrc.__repr__() is broken for writing to file (GH-2491)
netrc file format doesn't support quotes and escapes.

See https://linux.die.net/man/5/netrc
2017-09-30 16:10:31 +09:00
Łukasz Langa 574562c5dd bpo-31641: Allow arbitrary iterables in `concurrent.futures.as_completed()` (#3830)
This was possible before.  GH-1560 introduced a regression after 3.6.2 got
released where only sequences were accepted now.  This commit addresses this
problem.
2017-09-29 14:33:34 -07:00
Oren Milman 01c6a8859e bpo-31602: Fix an assertion failure in zipimporter.get_source() in case of a bad zlib.decompress() (GH-3784)
While a rare potential failure (it requires swapping out zlib.decompress() itself and forcing it to return a non-bytes object), this change prevents a potential C-level assertion failure and instead substitutes it with an exception.

Thanks to Oren Milman for the patch.
2017-09-29 11:34:31 -07:00
Zhiming Wang d87b105ca7 bpo-31638: Add compression support to zipapp (GH-3819)
Add optional argument `compressed` to `zipapp.create_archive`, and add
option `--compress` to the command line interface of `zipapp`.
2017-09-29 18:31:52 +01:00
Benjamin Peterson 288d1daada remove support for BSD/OS (closes bpo-31624) (#3812) 2017-09-28 22:44:27 -07:00
Antoine Pitrou a106aec2ed bpo-11063, bpo-20519: avoid ctypes and improve import time for uuid (#3796)
bpo-11063, bpo-20519: avoid ctypes and improve import time for uuid.
2017-09-28 23:03:06 +02:00
Oren Milman d780b2d588 bpo-31478: Fix an assertion failure in random.seed() in case a seed has a bad __abs__() method. (#3596) 2017-09-28 10:50:01 +03:00
Oren Milman 5837d0418f bpo-31588: Validate return value of __prepare__() methods (GH-3764)
Class execution requires that __prepare__() methods return
a proper execution namespace. Check for that immediately
after calling __prepare__(), rather than passing it through
to the code execution machinery and potentially triggering
SystemError (in debug builds) or a cryptic TypeError
(in release builds).

Patch by Oren Milman.
2017-09-28 00:04:37 +10:00
Serhiy Storchaka c740e4fe8a bpo-30347: Stop crashes when concurrently iterate over itertools.groupby() iterators. (#1557) 2017-09-26 21:47:56 +03:00
Victor Stinner e6d9fcbb8d bpo-31170: Write unit test for Expat 2.2.4 UTF-8 bug (#3570)
Non-regression tests for the Expat 2.2.3 UTF-8 decoder bug.
2017-09-25 01:27:34 -07:00
Oren Milman 5d3e80021a bpo-31566: Fix an assertion failure in _warnings.warn() in case of a bad __name__ global. (#3717) 2017-09-24 21:28:42 +03:00
Oren Milman 91fb0afe18 bpo-31285: Fix an assertion failure and a SystemError in warnings.warn_explicit. (#3219) 2017-09-24 21:27:12 +03:00
Serhiy Storchaka c247caf33f bpo-30346: An iterator produced by the itertools.groupby() iterator (#1569)
now becames exhausted after advancing the groupby iterator.
2017-09-24 13:36:11 +03:00
Oren Milman 2b382dd612 bpo-31505: Fix an assertion failure in json, in case _json.make_encoder() received a bad encoder() argument. (#3643) 2017-09-24 12:07:12 +03:00
Barry Warsaw 35425d638c bpo-31389 Add an optional `header` argument to pdb.set_trace() (#3438)
* Give pdb.set_trace() an optional `header` argument

* What's new.

* Give pdb.set_trace() an optional `header` argument

* What's new.
2017-09-22 12:29:42 -04:00
Serhiy Storchaka b3a77964ea bpo-27541: Reprs of subclasses of some classes now contain actual type name. (#3631)
Affected classes are bytearray, array, deque, defaultdict, count and repeat.
2017-09-21 14:24:13 +03:00
Igor Filatov 9adda0cdf8 bpo-31351: Set return code in ensurepip when pip fails (GH-3626)
Previously ensurepip would always report success, even if the
pip installation failed.
2017-09-21 20:07:45 +10:00
Anthony Sottile aaf6fc0982 bpo-26510: make argparse subparsers required by default (#3027)
This fixes a regression from Python 2.  To get optional subparsers,
use the new parameter ``add_subparsers(required=False)``.

Patch by Anthony Sottile.
2017-09-20 17:35:27 -04:00
Victor Stinner 9abee722d4 bpo-31479: Always reset the signal alarm in tests (#3588)
* bpo-31479: Always reset the signal alarm in tests

Use "try: ... finally: signal.signal(0)" pattern to make sure that
tests don't "leak" a pending fatal signal alarm.

* Move two more alarm() calls into the try block

Fix also typo: replace signal.signal(0) with signal.alarm(0)

* Move another signal.alarm() into the try block
2017-09-19 09:36:54 -07:00
Victor Stinner a92941ff12 pythoninfo: ignore OSError(ENOSYS) on getrandom() (#3655) 2017-09-19 07:37:24 -07:00
Oren Milman 865e4b4f63 bpo-31293: Fix crashes in truediv and mul of a timedelta by a float with a bad as_integer_ratio() method. (#3227) 2017-09-19 15:58:11 +03:00
Oren Milman 9974e1bcf3 bpo-31315: Fix an assertion failure in imp.create_dynamic(), when spec.name is not a string. (#3257) 2017-09-19 14:39:47 +03:00
Oren Milman 6db7033192 bpo-31492: Fix assertion failures in case of a module with a bad __name__ attribute. (#3620) 2017-09-19 14:23:01 +03:00
Antoine Pitrou 88c60c9668 Trivial cleanups following bpo-31370 (#3649)
* Trivial cleanups following bpo-31370

* Also cleanup the "importlib._bootstrap_external" module
2017-09-18 23:50:44 +02:00
Antoine Pitrou b43c4caf81 Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) (#3648) 2017-09-18 22:04:20 +02:00
Victor Stinner a8e7d903d7 os.test_utime_current(): tolerate 50 ms delta (#3646) 2017-09-18 08:49:45 -07:00
Victor Stinner e727d41ffc bpo-31499, xml.etree: Fix xmlparser_gc_clear() crash (#3641)
* bpo-31499, xml.etree: Fix xmlparser_gc_clear() crash

xml.etree: xmlparser_gc_clear() now sets self.parser to NULL to prevent a
crash in xmlparser_dealloc() if xmlparser_gc_clear() was called previously
by the garbage collector, because the parser was part of a reference cycle.

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
2017-09-18 05:29:37 -07:00
Raymond Hettinger 132a7d7cdb bpo-31482: Missing bytes support for random.seed() version 1 (#3614)
bpo-31482: Missing bytes support for random.seed() version 1 #3614
2017-09-17 09:04:30 -07:00
Christian Heimes e82c034496 bpo-31431: SSLContext.check_hostname auto-sets CERT_REQUIRED (#3531)
Signed-off-by: Christian Heimes <christian@python.org>
2017-09-15 20:29:57 +02:00
Christian Heimes a170fa162d bpo-31346: Use PROTOCOL_TLS_CLIENT/SERVER (#3058)
Replaces PROTOCOL_TLSv* and PROTOCOL_SSLv23 with PROTOCOL_TLS_CLIENT and
PROTOCOL_TLS_SERVER.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-15 20:27:30 +02:00
Christian Heimes 4df60f18c6 bpo-31386: Custom wrap_bio and wrap_socket type (#3426)
SSLSocket.wrap_bio() and SSLSocket.wrap_socket() hard-code SSLObject and
SSLSocket as return types. In the light of future deprecation of
ssl.wrap_socket() module function and direct instantiation of SSLSocket,
it is desirable to make the return type of SSLSocket.wrap_bio() and
SSLSocket.wrap_socket() customizable.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-15 20:26:05 +02:00
Victor Stinner 11f0807a40 bpo-31234: test_multiprocessing: wait 30 seconds (#3599)
Give 30 seconds to join_process(), instead of 5 or 10 seconds, to
wait until the process completes.
2017-09-15 06:55:31 -07:00
Victor Stinner da3e5cf961 bpo-31234: Join timers in test_threading (#3598)
Call the .join() method of threading.Timer timers to prevent the
"threading_cleanup() failed to cleanup 1 threads" warning.
2017-09-15 05:37:42 -07:00
Feanil Patel 6a396c9807 bpo-31128: Allow pydoc to bind to arbitrary hostnames (#3011)
New -n flag allow overriding localhost with custom value,
for example to run from containers.
2017-09-14 17:54:09 -04:00
Victor Stinner b9b69003d9 bpo-31234: Add support.join_thread() helper (#3587)
join_thread() joins a thread but raises an AssertionError if the
thread is still alive after timeout seconds.
2017-09-14 14:40:56 -07:00
Victor Stinner 167cbde50a bpo-31234: Join threads in test_queue (#3586)
Call thread.join() to prevent the "dangling thread" warning.
2017-09-14 14:04:56 -07:00
Victor Stinner ff40ecda73 bpo-31234: Add test.support.wait_threads_exit() (#3578)
Use _thread.count() to wait until threads exit. The new context
manager prevents the "dangling thread" warning.
2017-09-14 13:07:24 -07:00
Victor Stinner b8c7be2c52 bpo-31234: Join threads in test_threading (#3579)
Call thread.join() to prevent the "dangling thread" warning.
2017-09-14 13:05:21 -07:00
scoder c8d8e15bfc bpo-31455: Fix an assertion failure in ElementTree.XMLParser(). (#3545)
* Avoid calling "PyObject_GetAttrString()" (and potentially executing user code) with a live exception set.

* Ignore only AttributeError on attribute lookups in ElementTree.XMLParser() and propagate all other exceptions.
2017-09-14 23:00:03 +03:00
Oren Milman 0b3a87ef54 bpo-31471: Fix assertion failure in subprocess.Popen() on Windows, in case env has a bad keys() method. (#3580) 2017-09-14 22:30:27 +03:00
Łukasz Langa f350a268a7 bpo-28556: typing.get_type_hints: better globalns for classes and modules (#3582)
This makes the default behavior (without specifying `globalns` manually) more
predictable for users, finds the right globalns automatically.

Implementation for classes assumes has a `__module__` attribute and that module
is present in `sys.modules`.  It does this recursively for all bases in the
MRO.  For modules, the implementation just uses their `__dict__` directly.

This is backwards compatible, will just raise fewer exceptions in naive user
code.

Originally implemented and reviewed at https://github.com/python/typing/pull/470.
2017-09-14 14:33:00 -04:00
Victor Stinner 8dcf22f442 bpo-31234: Join threads in test_hashlib (#3573)
* bpo-31234: Join threads in test_hashlib

Use thread.join() to wait until the parallel hash tasks complete
rather than using events. Calling thread.join() prevent "dangling
thread" warnings.

* test_hashlib: minor PEP 8 coding style fixes
2017-09-14 08:43:22 -07:00
Victor Stinner 18e95b4176 bpo-31234: Join threads in tests (#3572)
Call thread.join() on threads to prevent the "dangling threads"
warning.
2017-09-14 08:43:04 -07:00
Łukasz Langa 1bbd482bcf bpo-31457: Allow for nested LoggerAdapter objects (#3551)
Some of the proxied methods use internal Logger state which isn't proxied,
causing failures if an adapter is applied to another adapter.

This commit fixes the issue, adds a new test for the use case.
2017-09-14 11:34:47 -04:00
Serhiy Storchaka 13ad3b7a82 bpo-31462: Remove trailing whitespaces. (#3564) 2017-09-14 09:38:36 +03:00
Victor Stinner 096ae3373a bpo-31234: Try to fix lock_tests warning (#3557)
Try to fix the "Warning -- threading_cleanup() failed to cleanup 1
threads" warning in test.lock_tests: wait a little bit longer to give
time to the threads to complete.

Warning seen on test_thread and test_importlib.
2017-09-13 16:41:08 -07:00
Ivan Levkivskyi 65bc62052f bpo-28556: Minor updates to typing module (#3550)
* Copy changes to typing from upstream repo
* Add NEWS entry
2017-09-13 19:25:15 -04:00
Victor Stinner ace1ecc00b bpo-31234: threading_cleanup() now warns immediately (#3138)
support.threading_cleanup() waits for 1 second before emitting a
warning if there are threads running in the background. With this
change, it now emits the warning immediately, to be able to catch
bugs more easily.
2017-09-13 13:42:00 -07:00
Victor Stinner b157ce1e58 bpo-31234: Fix dangling thread in test_ftplib (#3544)
Clear also self.server_thread attribute in TestTimeouts.tearDown().
2017-09-13 06:43:58 -07:00
Victor Stinner d165e14e29 bpo-31448, test_poplib: Fix ResourceWarning (#3542)
Call POP3.close(), don't close close directly the sock attribute.
2017-09-13 05:53:10 -07:00
Victor Stinner d403a29c00 bpo-31234: Fix dangling thread in test_ftp/poplib (#3540)
Explicitly clear the server attribute in test_ftplib and test_poplib
to prevent dangling thread.
2017-09-13 03:58:25 -07:00
Victor Stinner b8f4163da3 bpo-31233: socketserver.ThreadingMixIn.server_close() (#3523)
socketserver.ThreadingMixIn now keeps a list of non-daemonic threads
to wait until all these threads complete in server_close().

Reenable test_logging skipped tests.

Fix SocketHandlerTest.tearDown(): close the socket handler before
stopping the server, so the server can join threads.
2017-09-13 01:47:22 -07:00
Victor Stinner 3bcf157c11 bpo-31249: Fix test_concurrent_futures dangling thread (#3521)
ProcessPoolShutdownTest.test_del_shutdown() now closes the call queue
and joins its thread, to prevent leaking a dangling thread.
2017-09-12 17:05:53 -07:00
Victor Stinner a15d155aad bpo-31234: Enhance test_thread.test_forkinthread() (#3516)
* test_thread.test_forkinthread() now waits until the thread completes.
* Check the status in the test method, not in the thread function
* Don't ignore RuntimeError anymore: since the commit
  346cbd351e (bpo-16500,
  os.register_at_fork(), os.fork() cannot fail anymore with
  RuntimeError.
* Replace 0.01 literal with a new POLL_SLEEP constant
* test_forkinthread(): test if os.fork() exists rather than testing
  the platform.
2017-09-12 10:49:22 -07:00
Oren Milman 9d984fd2b0 bpo-31416: Fix assertion failures in case of a bad warnings.filters or warnings.defaultaction. (#3496)
Patch by Oren Milman.
2017-09-12 00:18:09 +03:00
Victor Stinner 50a72af657 test_ssl: Implement timeout in ssl_io_loop() (#3500)
The timeout parameter was not used.
2017-09-11 09:34:24 -07:00
Benjamin Peterson 2bb69a5b4e bpo-31373: remove overly strict float range checks (#3486)
This undoes a853a8ba78 except for the pytime.c
parts. We want to continue to allow IEEE 754 doubles larger than FLT_MAX to be
rounded into finite floats. Tests were added to very this behavior.
2017-09-10 23:50:46 -07:00
Oren Milman 252033d50e bpo-31411: Prevent raising a SystemError in case warnings.onceregistry is not a dictionary. (#3485) 2017-09-11 09:28:39 +03:00
Raymond Hettinger 8b57d73639 bpo-28638: Optimize namedtuple() creation time by minimizing use of exec() (#3454)
* Working draft without _source

* Re-use itemgetter() instances

* Speed-up calls to __new__() with a pre-bound tuple.__new__()

* Add note regarding string interning

* Remove unnecessary create function wrappers

* Minor sync-ups with PR-2736.  Mostly formatting and f-strings

* Bring-in qualname/__module fix-ups from PR-2736

* Formally remove the verbose flag and _source attribute

* Restore a test of potentially problematic field names

* Restore kwonly_args test but without the verbose option

* Adopt Inada's idea to reuse the docstrings for the itemgetters

* Neaten-up a bit

* Add news blurb

* Serhiy pointed-out the need for interning

* Jelle noticed as missing f on an f-string

* Add whatsnew entry for feature removal

* Accede to request for dict literals instead keyword arguments

* Leave the method.__module__ attribute pointing the actual location of the code

* Improve variable names and add a micro-optimization for an non-public helper function

* Simplify by in-lining reuse_itemgetter()

* Arrange steps in more logical order

* Save docstring in local cache instead of interning
2017-09-10 10:23:36 -07:00
Gregory P. Smith efb1d0a3c0 bpo-29639: change test.support.HOST to "localhost"
test.support.HOST should be "localhost" as it was in the past. See the bpo-29639.

Tests that need the IP address should use HOSTv4 (added) or the existing HOSTv6 constant.

This changes the definition and fixes tests that needed updating to deal with HOST being
the hostname rather than the hardcoded IP address.

This is only the first step in addressing https://bugs.python.org/issue29639.
2017-09-09 00:30:15 -07:00
Han Lee 829dacce4f bpo-26669: Fix nan arg value error in pytime.c (#3085)
* Fix #26669

* Modify NaN check function and error message

* Fix pytime.c when arg is nan

* fix whitespace
2017-09-08 16:05:05 -07:00
Christian Heimes b3ad0e5127 bpo-28182: Expose OpenSSL verification results (#3412)
The SSL module now raises SSLCertVerificationError when OpenSSL fails to
verify the peer's certificate. The exception contains more information about
the error.

Original patch by Chi Hsuan Yen

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-08 12:00:19 -07:00
Christian Heimes cb5b68abde bpo-29136: Add TLS 1.3 cipher suites and OP_NO_TLSv1_3 (#1363)
* bpo-29136: Add TLS 1.3 support

TLS 1.3 introduces a new, distinct set of cipher suites. The TLS 1.3
cipher suites don't overlap with cipher suites from TLS 1.2 and earlier.
Since Python sets its own set of permitted ciphers, TLS 1.3 handshake
will fail as soon as OpenSSL 1.1.1 is released. Let's enable the common
AES-GCM and ChaCha20 suites.

Additionally the flag OP_NO_TLSv1_3 is added. It defaults to 0 (no op) with
OpenSSL prior to 1.1.1. This allows applications to opt-out from TLS 1.3
now.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-07 18:07:00 -07:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Christian Heimes 888bbdc192 bpo-27340: Use memoryview in SSLSocket.sendall() (#3384)
* bpo-27340: Use memoryview in SSLSocket.sendall()

SSLSocket.sendall() now uses memoryview to create slices of data. This fix
support for all bytes-like object. It is also more efficient and avoids
costly copies.

Signed-off-by: Christian Heimes <christian@python.org>

* Cast view to bytes, fix typo

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-07 14:18:21 -07:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Łukasz Langa e7c566caf1 bpo-30465: Fix lineno and col_offset in fstring AST nodes (#1800)
For f-string ast nodes, fix the line and columns so that tools such as flake8 can identify them correctly.
2017-09-06 17:27:58 -07:00
R. David Murray 0f6b9d2306 bpo-14191 Add parse_intermixed_args. (#3319)
This adds support for parsing a command line where options and positionals are intermixed as is common in many unix commands. This is paul.j3's patch with a few tweaks.
2017-09-06 20:25:40 -04:00
Zachary Ware 49ce74efe8 Remove all mention of Windows IA-64 support (GH-3389)
It was mostly removed long ago.
2017-09-06 15:45:25 -07:00
caavery effc12f8e9 bpo-27584: New addition of vSockets to the python socket module (#2489)
* bpo-27584: New addition of vSockets to the python socket module

Support for AF_VSOCK on Linux only

* bpo-27584: Fixes for V2

Fixed syntax and naming problems.
Fixed #ifdef AF_VSOCK checking
Restored original aclocal.m4

* bpo-27584: Fixes for V3

Added checking for fcntl and thread modules.

* bpo-27584: Fixes for V4

Fixed white space error

* bpo-27584: Fixes for V5

Added back comma in (CID, port).

* bpo-27584: Fixes for V6

Added news file.
socket.rst now reflects first Linux introduction of AF_VSOCK.
Fixed get_cid in test_socket.py.
Replaced PyLong_FromLong with PyLong_FromUnsignedLong in socketmodule.c
Got rid of extra AF_VSOCK #define.
Added sockaddr_vm to sock_addr.

* bpo-27584: Fixes for V7

Minor cleanup.

* bpo-27584: Fixes for V8

Put back #undef AF_VSOCK as it is  necessary when vm_sockets.h is not installed.
2017-09-06 15:18:10 -07:00
Devin Jeanpierre c5bace2bf7 bpo-29505: Add fuzz tests for float(str), int(str), unicode(str) (#2878)
Add basic fuzz tests for a few common builtin functions.

This is an easy place to start, and these functions are probably safe.
We'll want to add more fuzz tests later.  Lets bootstrap using these.

While the fuzz tests are included in CPython and compiled / tested on a
very basic level inside CPython itself, the actual fuzzing happens as
part of oss-fuzz (https://github.com/google/oss-fuzz). The reason to
include the tests in CPython is to make sure that they're maintained
as part of the CPython project, especially when (as some eventually
will) they use internal implementation details in the test.

(This will be necessary sometimes because e.g. the fuzz test should
never enter Python's interpreter loop, whereas some APIs only expose
themselves publicly as Python functions.)

This particular set of changes is part of testing Python's builtins,
tracked internally at Google by b/37562550.

The _xxtestfuzz module that this change adds need not be shipped with binary distributions of Python.
2017-09-06 11:15:35 -07:00
Ammar Askar 3fc499bca1 bpo-31178: Avoid concatenating bytes with str in subprocess error (#3066)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
2017-09-05 23:41:30 -07:00
Christian Heimes 6877111648 bpo-29781: Fix SSLObject.version before handshake (#3364)
SSLObject.version() now correctly returns None when handshake over BIO has
not been performed yet.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 21:55:40 -07:00
Christian Heimes 3463ee3972 Stop test_xmlrpc from writing to sys.stderr (#3359)
One test case of test_xmlrpc uses HTTPServer with a subclass of
BaseHTTPRequestHandler. The BaseRequestHandler class logs to
sys.stderr by default. Override log_message() to not clobber
test output.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 21:43:46 -07:00
Christian Heimes 305e56c27a bpo-31320: No traceback to sys.stderr in test_ssl (#3360)
In case PROTOCOL_TLS_SERVER is used for both client context and server
context, the test thread dies with OSError. Catch OSError to avoid
traceback on sys.stderr

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 16:37:09 -07:00
Benjamin Peterson 069306312a remove IRIX support (closes bpo-31341) (#3310)
See PEP 11.
2017-09-04 16:36:05 -07:00
Barry Warsaw ba4279683f bpo-1198569: Allow string.Template braced pattern to be different (#3288)
* bpo-1198569: Allow the braced pattern to be different

``string.Template`` subclasses can optionally define ``braceidpattern`` if
they want to specify different placeholder patterns inside and outside the
braces.  If None (the default) it falls back to ``idpattern``.
2017-09-04 16:32:10 -04:00
Christian Heimes 002d64039b bpo-25674: remove sha256.tbs-internet.com ssl test (#3297)
Signed-off-by: Christian Heimes <christian@python.org>
2017-09-04 22:26:01 +02:00
Zhiming Wang 06de1aeff9 bpo-31281: Fix pathlib.Path incompatibility in fileinput (gh-3208)
Fix fileinput with inplace=True to accept pathlib.Path objects.
2017-09-04 13:37:24 -04:00
Ned Deily 918edc0edb bpo-12383: Also ignore __PYVENV_LAUNCHER__ (#3278)
Used in macOS framework builds.
2017-09-04 00:00:21 -04:00
Antoine Pitrou 2ef37607b7 Fix a c.f.as_completed() refleak previously introduced in bpo-27144 (#3270) 2017-09-03 15:09:23 +02:00
Serhiy Storchaka bca4939d80 bpo-31185: Fixed miscellaneous errors in asyncio speedup module. (#3076) 2017-09-03 08:10:14 +03:00
Victor Stinner 8687101589 bpo-31323: Fix reference leak in test_ssl (#3263)
Store exceptions as string rather than object to prevent reference
cycles which cause leaking dangling threads.
2017-09-02 00:26:17 +02:00
Victor Stinner e8a533fbc7 bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() (#3264)
Call doCleanups() to close the loop after calling
executor.shutdown(wait=True): see TestCase.set_event_loop() of
asyncio.test_utils.

Replace also gc.collect() with support.gc_collect().
2017-09-02 00:25:39 +02:00
Grzegorz Grzywacz 97e1b1c814 bpo-27144: concurrent.futures as_complete and map iterators do not keep reference to returned object (#1560)
* bpo-27144: concurrent.futures as_complie and map iterators do not keep
reference to returned object

* Some nits.  Improve wordings in docstrings and comments, and avoid relying on
sys.getrefcount() in tests.
2017-09-01 18:54:00 +02:00
Victor Stinner 16432beadb bpo-31250, test_asyncio: fix dangling threads (#3252)
* Explicitly call shutdown(wait=True) on executors to wait until all
  threads complete to prevent side effects between tests.
* Fix test_loop_self_reading_exception(): don't mock loop.close().
  Previously, the original close() method was called rather than the
  mock, because how set_event_loop() registered loop.close().
2017-09-01 14:46:06 +02:00
Victor Stinner 6c2feabc5d bpo-31217: Fix regrtest -R for small integer (#3260)
Use a pool of integer objects toprevent false alarm when checking for
memory block leaks. Fill the pool with values in -1000..1000 which
are the most common (reference, memory block, file descriptor)
differences.

Co-Authored-By: Antoine Pitrou <pitrou@free.fr>
2017-09-01 13:05:27 +02:00
Oren Milman 631fdee6e6 bpo-31291: Fixed an assertion failure in zipimport.zipimporter.get_data() (#3226)
if pathname.replace('/', '\\') returns non-string.
2017-08-29 20:40:15 +03:00
Serhiy Storchaka 265fcc5fc2 bpo-31286, bpo-30024: Fixed stack usage in absolute imports with (#3217)
binding a submodule to a name.
2017-08-29 15:47:44 +03:00
Oren Milman ba7d736521 bpo-31243: Fixed PyArg_ParseTuple failure checks. (#3171) 2017-08-29 11:58:27 +03:00
Pier-Yves Lessard a30f6d45ac bpo-30987 - Support for ISO-TP protocol in SocketCAN (#2956)
* Added support for CAN_ISOTP protocol

* Added unit tests for CAN ISOTP

* Updated documentation for ISO-TP protocol

* Removed trailing whitespace in documentation

* Added blurb NEWS.d file

* updated Misc/ACKS

* Fixed broken unit test that was using isotp const outside of skippable section

* Removed dependecy over third party project

* Added implementation for getsockname + unit tests

* Missing newline at end of ACKS file

* Accidentally inserted a type in ACKS file

* Followed tiran changes review #1 recommendations

* Added spaces after comma
2017-08-28 10:32:44 +02:00
Paul Moore 0780bf7578 bpo-31072: Rename the new filter argument for zipapp.create_archive. (#3049)
bpo-31072: Rename the new filter argument for zipapp.create_archive (GH-3049)

* Rename the new argument to "filter"
* Improve tests for the new functionality
* Add a "What's New" entry.
2017-08-26 18:04:12 +01:00
Oren Milman a5b4ea15b6 bpo-31271: Fix an assertion failure in io.TextIOWrapper.write. (#3201) 2017-08-25 21:14:54 +03:00
Gregory P. Smith a3a6df36b6 Skip two tests not intended to pass on Windows. (#3202) 2017-08-24 18:15:02 -07:00
Gregory P. Smith 8621bb5d93 bpo-22536: Set the filename in FileNotFoundError. (#3194)
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
2017-08-24 14:58:25 -07:00
Oren Milman de50360ac2 bpo-29741: Update some methods in the _pyio module to also accept integer types. Patch by Oren Milman. (#560) 2017-08-24 11:33:42 -07:00
Łukasz Langa a5fab17fc1 bpo-23835: Restore legacy defaults= behavior for RawConfigParser (#3191)
The fix for bpo-23835 fixed ConfigParser behavior in defaults= handling.
Unfortunately, it caused a backwards compatibility regression with
RawConfigParser objects which allow for non-string values.

This commit restores the legacy behavior for RawConfigParser only.
2017-08-24 09:43:53 -07:00
Oren Milman bf9075a0c5 bpo-31229: Fixed wrong error messages when too many keyword arguments are received. (#3180) 2017-08-23 21:16:48 +03:00
Martijn Pieters 772d809a63 bpo-31161: only check for parens error for SyntaxError (#3082)
Subclasses such as IndentError and TabError should not have this message
applied.
2017-08-22 13:16:23 -07:00
Victor Stinner 41bbd82b6b bpo-31234: test_threaded_import: fix test_side_effect_import() (#3189)
* Don't leak the module into sys.modules
* Avoid dangling thread
2017-08-22 18:05:32 +02:00
Victor Stinner 830d7d2936 bpo-31234: test_httpservers joins the server thread (#3188) 2017-08-22 18:05:07 +02:00
James Tocknell 44e6ad8734 bpo-23835: Enforce that configparser defaults are strings (#2558)
* Enforce that configparser defaults are strings
* Update test_configparser.py
2017-08-21 15:46:30 -07:00
Victor Stinner 9a83f651f3 Add test_subprocess.test_nonexisting_with_pipes() (#3133)
bpo-30121: Test the Popen failure when Popen was created with pipes.
Create also NONEXISTING_CMD variable in test_subprocess.py.
2017-08-21 23:51:31 +02:00
Victor Stinner 489d91c61d bpo-31249: test_concurrent_futures checks dangling threads (#3167)
Add a BaseTestCase class to test_concurrent_futures to check for
dangling threads and processes on all tests, not only tests using
ExecutorMixin.
2017-08-21 23:24:24 +02:00
Oren Milman 1d1d3e9db8 bpo-28261: Fixed err msgs where PyArg_ParseTuple is used to parse normal tuples. (#3119) 2017-08-20 18:35:36 +03:00
Victor Stinner 3d284c081f bpo-31234: test_threading: fix ref cycle (#3150)
test_bare_raise_in_brand_new_thread() now explicitly breaks a
reference cycle to not leak a dangling thread.
2017-08-19 01:54:42 +02:00
Victor Stinner a7719e27b3 bpo-31235: Fix ResourceWarning in test_logging (#3147) 2017-08-19 00:34:00 +02:00
Victor Stinner 6966960468 bpo-30830: test_logging uses threading_setup/cleanup (#3137)
* bpo-30830: test_logging uses threading_setup/cleanup

Replace @support.reap_threads on some methods with
support.threading_setup() in setUp() and support.threading_cleanup()
in tearDown() in BaseTest.

* bpo-30830: test_logging disables threaded socketserver tests

Disable tests because of socketserver.ThreadingMixIn leaks threads,
whereas leaking threads now makes a test to fail on buildbots.

Disable tests until socketserver is fixed: bpo-31233.

* Skip also setup_via_listener()
2017-08-18 23:47:54 +02:00
Victor Stinner c99d41f9c0 bpo-31234: fork_wait tests now join threads (#3139)
fork_wait.py tests now joins threads, to not leak running threads in
the background.
2017-08-18 23:12:26 +02:00
Segev Finer 4d3851727f bpo-30121: Fix debug assert in subprocess on Windows (#1224)
* bpo-30121: Fix debug assert in subprocess on Windows

This is caused by closing HANDLEs using os.close which is for CRT file
descriptors and not for HANDLEs.

* bpo-30121: Suppress debug assertion in test_subprocess when ran directly
2017-08-18 15:18:13 +02:00
Sanyam Khurana a7c449b8c0 bpo-30721: Add missing '?' to new error message (GH-3131) 2017-08-18 22:18:14 +10:00
Sanyam Khurana 5e2eb35bbe bpo-30721: Show correct syntax hint in Py3 when using Py2 redirection syntax (#2345) 2017-08-18 13:37:36 +03:00
Victor Stinner ad7eaed543 bpo-30871: pythoninfo: more sys, os, time data (#3130)
* bpo-30871: pythoninfo: more sys, os, time data

PythonInfo now converts types other than intger to string by default.

* fix typo
2017-08-18 12:08:47 +02:00
syncosmic fe2b56ab92 bpo-31183: `dis` now handles coroutines & async generators (GH-3077)
Coroutines and async generators use a distinct attribute name for their
code objects, so this updates the `dis` module to correctly disassemble
objects with those attributes.

Due to the increase in the test module length, it also fixes some latent
defects in the tests related to how the displayed source line numbers
are extracted.

https://bugs.python.org/issue31230 is a follow-up issue suggesting we
may want to solve this a different way, by instead giving all these object
types a common `__code__` attribute, avoiding the need for special
casing in the `dis` module.
2017-08-18 12:29:21 +10:00
Victor Stinner f6ebd838f0 bpo-30871: pythoninfo: add expat and _decimal (#3121)
* bpo-30871: pythoninfo: add expat and _decimal

* Remove _decimal.__version__

The string is hardcoded, not really interesting.
2017-08-17 22:13:11 +02:00
Victor Stinner b907abc885 bpo-30871: Add test.pythoninfo (#3075)
* Add Lib/test/pythoninfo.py: script collecting various informations
  about Python to help debugging test failures.
* regrtest: remove sys.hash_info and sys.flags from header.
* Travis CI, Appveyor: run pythoninfo before tests
2017-08-17 16:40:51 +02:00
Adrian Wielgosik 7c17e2304b bpo-24700: Add a fast path for comparing array.array of equal type (#3009) 2017-08-17 14:46:06 +02:00
Antoine Pitrou ee84a60858 bpo-18966: non-daemonic threads created by a multiprocessing.Process should be joined on exit (#3111)
* bpo-18966: non-daemonic threads created by a multiprocessing.Process should be joined on exit

* Add NEWS blurb
2017-08-16 20:53:28 +02:00
Victor Stinner 17657bb945 bpo-31069, test_multiprocessing: Fix dangling process (#3103)
Fix a warning about dangling processes in test_rapid_restart() of
_test_multiprocessing: join the process.
2017-08-16 12:46:04 +02:00
Christian Heimes 7b40cb7293 bpo-30714: ALPN changes for OpenSSL 1.1.0f (#2305)
OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client
could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that.
The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN
callback can pretend to not been set.

See https://github.com/openssl/openssl/pull/3158 for more details

Signed-off-by: Christian Heimes <christian@python.org>
2017-08-15 10:33:43 +02:00
Коренберг Марк d4b93e21c2 bpo-31106: Fix handling of erros in posix_fallocate() and posix_fadvise() (#3000) (#3000) 2017-08-14 06:55:16 -07:00
Victor Stinner 5659a72f48 bpo-31008: Fix asyncio test_wait_for_handle on Windows (#3065) 2017-08-11 01:23:22 +02:00
Victor Stinner 957d0e9b59 bpo-26762: _test_multiprocessing reports dangling (#3064)
_test_multiprocessing now marks the test as ENV_CHANGED on dangling
process or thread.
2017-08-10 17:36:50 +02:00
Victor Stinner b97d99cd7f regrtest: count also env changed as failures in progress (#3061) 2017-08-10 16:19:03 +02:00
Victor Stinner b501147980 Enhance support.reap_children() (#3036)
* reap_children() now sets environment_altered to True to detect bugs
  using python3 -m test --fail-env-changed
* Replace bare "except:" with "except OSError:" in reap_children()
* Write an unit test for reap_children() using a timeout of 60
  seconds
2017-08-10 16:01:47 +02:00
Victor Stinner aa8ec34ad5 bpo-31151: Add socketserver.ForkingMixIn.server_close() (#3057)
* Add socketserver.ForkingMixIn.server_close()

bpo-31151: socketserver.ForkingMixIn.server_close() now waits until
all child processes completed to prevent leaking zombie processes.

* Fix test on Windows which doesn't have ForkingMixIn
2017-08-10 15:28:16 +02:00
Victor Stinner 6c8c2943d9 bpo-31160: test_tempfile: Fix reap_children() warning (#3056)
TestRandomNameSequence.test_process_awareness() now calls
os.waitpid() to avoid leaking a zombie process.
2017-08-10 13:05:06 +02:00
Victor Stinner 7b7c6dcfff bpo-31173: Rewrite WSTOPSIG test of test_subprocess (#3055)
The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.

Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
2017-08-10 12:37:39 +02:00
Victor Stinner 3ca9f50f96 bpo-31160: test_builtin: don't check waitpid() status (#3050) 2017-08-09 23:08:22 +02:00
Victor Stinner da5e930187 bpo-31160: Fix test_random for zombie process (#3045)
TestModule.test_after_fork() now calls os.waitpid() to read the exit
status of the child process to avoid creating a zombie process.
2017-08-09 17:59:05 +02:00
Victor Stinner e3510d74aa bpo-31160: regrtest now reaps child processes (#3044)
Add a post_test_cleanup() function which currently only calls
support.reap_children().
2017-08-09 17:44:33 +02:00
Victor Stinner 4baca1b0f7 bpo-31160: Fix test_builtin for zombie process (#3043)
PtyTests.run_child() now calls os.waitpid() to read the exit status
of the child process to avoid creating zombie process and leaking
processes in the background.
2017-08-09 17:44:19 +02:00
Jeffrey Rackauckas b811d664de bpo-31072: Add filter to zipapp (#3021)
bpo-31072: Add a filter argument to zipapp.create_archive (GH-3021)

* Add an include_file argument to allow callers to decide which files to include
* Document the new argument
2017-08-09 14:37:17 +01:00
Ammar Askar 88eee44a91 bpo-31150: Wait for child process in test_forkinthread to avoid thread reaped warnings (#3032) 2017-08-09 10:51:43 +02:00
Shane Harvey c4c9866064 bpo-31107: Fix copyreg mangled slot names calculation. (#2989) 2017-08-04 11:45:00 +03:00
Serhiy Storchaka 5075416b8f bpo-30978: str.format_map() now passes key lookup exceptions through. (#2790)
Previously any exception was replaced with a KeyError exception.
2017-08-03 11:45:23 +03:00
Serhiy Storchaka 25e4f779d7 bpo-31071: Avoid masking original TypeError in call with * unpacking (#2957)
when other arguments are passed.
2017-08-03 11:37:15 +03:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
Preston Landers 6ea56d2ebc bpo-31080: Allowed logging.config.fileConfig() to accept both args and kwargs. (GH-2979) 2017-08-02 21:44:28 +01:00
Alexander Mohr de34cbe9cd bpo-31061: fix crash in asyncio speedup module (GH-2966) 2017-08-02 15:31:07 +09:00
Cooper Lees 173ff4a58a bpo-30897: Add is_mount() to pathlib.Path (#2669)
* Add in is_mount() call to pathlib.Path similiar to os.path.ismount(path)
* Add tests for is_mount()
2017-08-01 15:35:45 -07:00
favll adfe3440f6 bpo-31084: QueueHandler now formats messages correctly. (GH-2954) 2017-08-01 19:12:26 +01:00
Alexander Belopolsky 018d353c1c Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets. (#2896)
* Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets.

* bpo-5288: Implemented %z formatting of sub-minute offsets.

* bpo-5288: Removed mentions of the whole minute limitation on TZ offsets.

* bpo-5288: Removed one more mention of the whole minute limitation.

Thanks @csabella!

* Fix a formatting error in the docs

* Addressed review comments.

Thanks, @haypo.
2017-07-31 10:26:50 -04:00
Vinay Sajip 2543f50033 bpo-30522: Implemented a method to allow setting a logging.StreamHander's stream. (GH-2921) 2017-07-30 10:41:45 +01:00
Avram Lubkin 78c18a9b9a bpo-30962: Added caching to Logger.isEnabledFor() (GH-2752) 2017-07-30 10:36:33 +01:00
Zackery Spytz ceb93f4540 bpo-31027: Fix test_listcomps failure when run directly (#2939)
Bug appears to be incomplete copy-paste-edit.
2017-07-29 12:05:55 -04:00
Victor Stinner 28ce07ae9e bpo-31066: Fix test_httpservers.test_last_modified() (#2933)
Write the temporary file on disk and then get its modification time.
2017-07-28 18:15:02 +02:00
Victor Stinner cc42c121eb bpo-31067: test_subprocess calls reap_children() (#2931)
test_subprocess now also calls reap_children() in tearDown(), not
only on setUp().
2017-07-28 18:00:22 +02:00
INADA Naoki ba9ddb7eea bpo-29585: fix test fail on macOS Framework build (GH-2928) 2017-07-28 21:28:19 +09:00
Ned Deily c22bd58d93 bpo-28095: Re-enable temporarily disabled part of test_startup_imports on macOS (#2927)
The changes for bpo-29585 eliminate the extra imports on macOS that caused
the original test failure.

This reverts commit 8a2150aae6.
2017-07-28 03:02:10 -04:00
Victor Stinner 12953ffe12 bpo-31044: Skip test_posix.test_makedev() on FreeBSD (#2915)
There is a bug in FreeBSD CURRENT with 64-bit dev_t. Skip the test if
dev_t is larger than 32-bit, until the bug is fixed in FreeBSD
CURRENT.
2017-07-27 16:55:54 +02:00
Victor Stinner fd46561167 bpo-31028: Fix test_pydoc when run directly (#2864)
* bpo-31028: Fix test_pydoc when run directly

Fix get_pydoc_link() of test_pydoc to fix "./python
Lib/test/test_pydoc.py": get the absolute path to __file__ to prevent
relative directories.

* Use realpath() instead of abspath()
2017-07-27 11:21:26 +02:00
INADA Naoki 067931dd95 bpo-30188: fix TypeError in test_nntplib (GH-2892)
fixes regression of 5b4feb7
2017-07-26 23:43:22 +09:00
Utkarsh Upadhyay ede9084476 bpo-31043: fixed test_datetime run twice. (GH-2891) 2017-07-26 19:46:17 +09:00
Utkarsh Upadhyay cc5a65cd90 bpo-30302 Make timedelta.__repr__ more informative. (#1493) 2017-07-25 23:51:33 +02:00
Victor Stinner 830080913c bpo-31034: Reliable signal handler for test_asyncio (#2867)
* bpo-31034: Reliable signal handler for test_asyncio

Don't rely on the current SIGHUP signal handler, make sure that it's
set to the "default" signal handler: SIG_DFL.

* Add comments
2017-07-25 19:19:09 +02:00
Victor Stinner 302bbbe9ba bpo-31009: Fix support.fd_count() on Windows (#2862)
* bpo-31009: Fix support.fd_count() on Windows

On Windows, test.support.fd_count() now calls
msvcrt.CrtSetReportMode() to not kill the process nor log any error
on stderr on os.dup(fd) if the file descriptor is invalid.

* Fix for release mode
2017-07-25 17:45:53 +02:00
Stéphane Wirtel 90addd6d1c bpo-31029: test_tokenize Add missing import unittest (#2865) 2017-07-25 16:33:53 +03:00
Victor Stinner b4c52966c8 bpo-26762: test_multiprocessing close more queues (#2855)
* Close explicitly queues to make sure that we don't leave dangling
  threads
* test_queue_in_process(): remove unused queue
* test_access() joins also the process to fix a random warning
2017-07-25 02:40:55 +02:00
Victor Stinner ffb49408f0 test_multiprocessing detects dangling per test case (#2841)
bpo-26762: test_multiprocessing now detects dangling processes and
threads per test case classes:

* setUpClass()/tearDownClass() of mixin classes now check if
  multiprocessing.process._dangling or threading._dangling was
  modified to detect "dangling" processses and threads.
* ManagerMixin.tearDownClass() now also emits a warning if it still
  has more than one active child process after 5 seconds.
* tearDownModule() now checks for dangling processes and threads
  before sleep 500 ms. And it now only sleeps if there is a least one
  dangling process or thread.
2017-07-25 01:55:54 +02:00
Victor Stinner d7e64d9934 test_multiprocessing: Fix dangling process/thread (#2850)
bpo-26762: Fix more dangling processes and threads in
test_multiprocessing:

* Queue: call close() followed by join_thread()
* Process: call join() or self.addCleanup(p.join)
2017-07-25 00:33:56 +02:00
Nir Soffer c648a93ae3 bpo-30980: Fix double close in asyncore.file_wrapper (#2789)
* bpo-30980: Fix close test to fail

test_close_twice was not considering the fact that file_wrapper is
duping the file descriptor. Closing the original descriptor left the
duped one open, hiding the fact that close protection is not effective.

* bpo-30980: Fix double close protection

Invalidated self.fd before closing, handling correctly the case when
os.close raises.

* bpo-30980: Fix fd leak introduced in the fixed test
2017-07-24 23:18:06 +02:00
Victor Stinner 5b4feb7e86 bpo-30188: test_nntplib catch also ssl.SSLEOFError (#2843)
Catch also ssl.SSLEOFError in NetworkedNNTPTests setUpClass().
EOFError was already catched.
2017-07-24 17:41:02 +02:00
Victor Stinner 06634950c5 bpo-26762: Avoid daemon process in _test_multiprocessing (#2842)
test_level() of _test_multiprocessing._TestLogging now uses regular
processes rather than daemon processes to prevent zombi processes
(to not "leak" processes).
2017-07-24 13:02:20 +02:00
Dong-hee Na 2b1e6e9696 bpo-30119: fix ftplib.FTP.putline() to throw an error for a illegal command (#1214) 2017-07-22 19:20:22 +02:00
Antoine Pitrou 896145d9d2 bpo-26732: fix too many fds in processes started with the "forkserver" method (#2813)
* bpo-26732: fix too many fds in processes started with the "forkserver" method

A child process would inherit as many fds as the number of still-running children.

* Add blurb and test comment
2017-07-22 13:22:54 +02:00
Antoine Pitrou ff92ff5366 Fix test failure without ctypes (#2802) 2017-07-21 13:24:05 +02:00
Gareth Rees 3913bad495 bpo-19896: Add typecodes 'q' and 'Q' to multiprocessing.sharedctypes (#2741)
* bpo-19896: Add typcodes 'q' and 'Q' to multiprocessing.sharedctypes. Patch by Antony Lee.

* Add NEWS entry.

* Slightly tweak NEWS entry

Make it clear this is more of a fix rather than a new feature.
2017-07-21 12:35:33 +02:00
Utkarsh Upadhyay 287c5594ed bpo-30822: Fix testing of datetime module. (#2530) (#2783)
Only C implementation was tested.
2017-07-21 02:14:54 +02:00
Victor Stinner 5b392bbaeb bpo-30822: Exclude tzdata from regrtest --all (#2775)
When running the test suite using --use=all / -u all, exclude tzdata
since it makes test_datetime too slow (15-20 min on some buildbots)
which then times out on some buildbots.

-u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata,
to run all test_datetime tests.

Fix also regrtest command line parser to allow passing -u
extralargefile to run test_zipfile64.

Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
2017-07-20 15:46:32 +02:00
Ammar Askar d81bea6520 Use pythontest.net instead of debian.org in test_urllib2net (GH-2755) 2017-07-19 05:27:24 +03:00
Vitor Pereira ba75af7130 bpo-30794: added kill() method to multiprocessing.Process (#2528)
* bpo-30794: added kill() method to multiprocessing.Process

* Added entries to documentation and NEWS

* Refactored test_terminate and test_kill

* Fix SIGTERM and SIGKILL being used on Windows for the tests

* Added "versionadded" marker to the documentation

* Fix trailing whitespace in doc
2017-07-18 17:34:23 +02:00
Steve Dower 68d663cf85 [bpo-30916] Pre-build OpenSSL and Tcl/Tk for Windows (#2688)
Updates ssl and tkinter projects to use pre-built externals
2017-07-17 11:15:48 +02:00
Serhiy Storchaka 49f6449ef4 bpo-30936: Fix a reference leak in json when fail to sort keys. (#2712) 2017-07-16 07:29:16 +03:00
Nick Coghlan f0b6a261bb bpo-30836: fix test_c_locale_coercion on AIX (GH-2713)
AIX uses iso8859-1 in the C locale, not ASCII

AIX doesn't currently provide any of the locale
coercion locales, but we leave locale coercion
enabled in case one gets added in the future.
2017-07-15 22:51:05 +10:00
Serhiy Storchaka d3aaa2f149 bpo-30911: Add tests for bad boolean arguments for accelerated json (#2690)
encoder and decoder.
2017-07-13 10:14:11 +03:00
Brett Cannon ab025e31ab Fix a spelling mistake and tweak tense of output for test.bisect (GH-2683)
Changed "subbset" to "subset". Also made the sentences read like things were happening instead of stating what the code should do (in other words more descriptive than prescriptive).
2017-07-12 12:04:25 -07:00
Louie Lu 50c9435c9b bpo-30899: Add unittests, 100% coverage, for IDLE's two ConfigParser subclasses. (#2662)
Patch by Louie Lu.
2017-07-12 14:05:32 -04:00
Victor Stinner d1cc037d14 bpo-30908: Fix dangling thread in test_os.TestSendfile (#2680)
tearDown() now clears explicitly the self.server variable to make
sure that the thread is completely cleared when tearDownClass()
checks if all threads have been cleaned up.

Fix the following warning:

$ ./python -m test --fail-env-changed -m test.test_os.TestSendfile.test_keywords -R 3:1 test_os
(...)
Warning -- threading_cleanup() failed to cleanup 0 threads after 3 sec (count: 0, dangling: 2)
(...)
Tests result: ENV CHANGED
2017-07-12 16:05:43 +02:00
Serhiy Storchaka 8a9cd20edc bpo-30876: Relative import from unloaded package now reimports the package (#2639)
instead of failing with SystemError.

Relative import from non-package now fails with ImportError rather than
SystemError.
2017-07-12 06:50:03 +03:00
Serhiy Storchaka 1180e5a518 bpo-30879: os.listdir() and os.scandir() now emit bytes names when (#2634)
called with bytes-like argument.
2017-07-11 06:36:46 +03:00
Sylvain 9648088e6c bpo-30878: Fix error message when keyword arguments are passed (#2635)
to staticmethod() and classmethod().
2017-07-09 06:45:06 +03:00
Nir Soffer aa6a4d6ed8 bpo-29854: Skip history-size test on older readline (GH-2621)
Turns out that history-size was added in readline 6.0. This explain why
this tests fail on FreeBSD when using readline 5.2. We skip now the
history size if readline does not support it.

See https://cnswww.cns.cwru.edu/php/chet/readline/CHANGES for
details.
2017-07-08 17:34:27 +03:00
Victor Stinner 1881befb90 bpo-29854: test_readline logs versions (#2619)
* test_readline logs the versions of libreadline when run in verbose
  mode
* Add also readline._READLINE_LIBRARY_VERSION
2017-07-07 16:06:58 +02:00
Nir Soffer fae8f4a9cb bpo-29854: Fix segfault in call_readline() (GH-728)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.

It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.

This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
2017-07-07 09:10:46 +03:00
Gregory P. Smith 529746c905 Make test_shutil test_disk_usage not depend on the cwd fs (#2597)
Make test_shutil test_disk_usage not depend on the current working directory's filesystem.
2017-07-06 17:11:27 -07:00
Serhiy Storchaka b4baacee1a bpo-30814: Fixed a race condition when import a submodule from a package. (#2580) 2017-07-06 08:09:03 +03:00
Yury Selivanov 833a3b0d37 bpo-30828: Fix out of bounds write in `asyncio.CFuture.remove_done_callback() (#2569) 2017-07-05 13:32:03 -04:00
Victor Stinner 8207c17486 Revert "bpo-30822: Fix testing of datetime module." (#2588)
* Revert "bpo-30854: Fix compile error when --without-threads (#2581)"

This reverts commit 0c31163093.

* Revert "NEWS for 30777 (#2576)"

This reverts commit aaa917ff38.

* Revert "bpo-21624: IDLE -- minor htest fixes (#2575)"

This reverts commit 2000150c56.

* Revert "bpo-30777: IDLE: configdialog - add docstrings and improve comments (#2440)"

This reverts commit 7eb5883ac5.

* Revert "bpo-30319: socket.close() now ignores ECONNRESET (#2565)"

This reverts commit 67e1478dba.

* Revert "bpo-30789: Use a single memory block for co_extra. (#2555)"

This reverts commit 378ebb6578.

* Revert "bpo-30845: Enhance test_concurrent_futures cleanup (#2564)"

This reverts commit 3df9dec425.

* Revert "bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)"

This reverts commit 48350412b7.

* Revert "Remove outdated FOX from GUI FAQ (GH-2538)"

This reverts commit d3ed2877a7.

* Revert "bpo-6691: Pyclbr now reports nested classes and functions. (#2503)"

This reverts commit 246ff3bd00.

* Revert "bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)"

This reverts commit 6969eaf468.

* Revert "bpo-30832: Remove own implementation for thread-local storage (#2537)"

This reverts commit aa0aa0492c.

* Revert "bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)"

This reverts commit 5e87592fd1.

* Revert "bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. (#2534)"

This reverts commit 34b54873b5.

* Revert "bpo-30822: Fix testing of datetime module. (#2530)"

This reverts commit 98b6bc3bf7.
2017-07-05 15:44:52 +02:00
Victor Stinner 3df9dec425 bpo-30845: Enhance test_concurrent_futures cleanup (#2564)
* bpo-30845: reap_children() now logs warnings

* bpo-30845: Enhance test_concurrent_futures cleanup

In setUp() and tearDown() methods of test_concurrent_futures tests,
make sure that tests don't leak threads nor processes. Clear
explicitly the reference to the executor to make it that it's
destroyed (to prevent "dangling threads" warning).
2017-07-04 13:14:04 +02:00
Antoine Pitrou 48350412b7 bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)
* bpo-29293: multiprocessing.Condition.notify() lacks parameter `n`

* Add NEWS blurb
2017-07-04 08:59:22 +02:00
csabella 246ff3bd00 bpo-6691: Pyclbr now reports nested classes and functions. (#2503)
Original patch by Guilherme Polo.  Revisions by Cheryl Sabella.
2017-07-03 21:31:25 -04:00
Victor Stinner 5e87592fd1 bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)
--forever now stops if a fail changes the environment.
2017-07-03 11:15:58 +02:00
Serhiy Storchaka 34b54873b5 bpo-30822: Deduplicate ZoneInfoTest classes in test_datetime. (#2534) 2017-07-02 22:36:12 +03:00
Utkarsh Upadhyay 98b6bc3bf7 bpo-30822: Fix testing of datetime module. (#2530)
Only C implementation was tested.
2017-07-02 15:46:04 +03:00
Osvaldo Santana Neto 8a8d28501f bpo-30441: Fix bug when modifying os.environ while iterating over it (#2409) 2017-07-01 20:34:45 +03:00
xdegaye 85f643023f bpo-30695: Add set_nomemory(start, stop) to _testcapi (GH-2406) 2017-07-01 14:14:45 +02:00
Victor Stinner 73528640ff bpo-30818: test_ftplib calls asyncore.close_all() (#2514)
Always clear asyncore socket map using
asyncore.close_all(ignore_all=True) in tearDown() method.

This change should fix this warning:

Warning -- asyncore.socket_map was modified by test_ftplib
  Before: {}
  After:  {4: <test.test_ftplib.DummyTLS_FTPServer 127.0.0.1:0 at 0x805feccf0>}
2017-06-30 17:36:57 +02:00
Victor Stinner 45dba3af50 test_urllib2_localnet: clear server variable (#2510)
Set the server attribute to None in cleanup to avoid dangling
threads.
2017-06-30 17:04:48 +02:00
Victor Stinner 7eebeb8fb8 bpo-30812: Fix test_warnings, restore _showwarnmsg (#2504)
bpo-26568, bpo-30812: Fix test_showwarnmsg_missing(): restore the
attribute after removing it.
2017-06-30 12:58:46 +02:00
Victor Stinner b903067462 bpo-30280: Cleanup threads in ayncio tests (#2501)
* bpo-30280: asyncio now cleans up threads

asyncio base TestCase now uses threading_setup() and
threading_cleanup() of test.support to cleanup threads.

* asyncio: Fix TestBaseSelectorEventLoop cleanup

bpo-30280: TestBaseSelectorEventLoop of
test.test_asyncio.test_selector_events now correctly closes the event
loop: cleanup its executor to not leak threads.

Don't override the close() method of the event loop, only override
the_close_self_pipe() method.
2017-06-30 11:12:33 +02:00
Victor Stinner 21a0a6c9f8 threading_cleanup() failure marks test as ENV_CHANGED (#2500)
If threading_cleanup() fails to cleanup threads, set a a new
support.environment_altered flag to true, flag uses by save_env which
is used by regrtest to check if a test altered the environment. At
the end, the test file fails with ENV_CHANGED instead of SUCCESS, to
report that it altered the environment.
2017-06-30 10:59:52 +02:00
Antoine Pitrou 729780a810 bpo-30807: signal.setitimer() may disable the timer by mistake (#2493)
* bpo-30807: signal.setitimer() may disable the timer by mistake

* Add NEWS blurb
2017-06-30 10:01:05 +02:00
Nick Coghlan 18974c35ad bpo-30647: Check nl_langinfo(CODESET) in locale coercion (GH-2374)
- On some versions of FreeBSD, setting the "UTF-8" locale
  succeeds, but a subsequent "nl_langinfo(CODESET)" fails
- adding a check for this in the coercion logic means that
  coercion will happen on systems where this check succeeds,
  and will be skipped otherwise
- that way CPython should automatically adapt to changes in
  platform behaviour, rather than needing a new release to
  enable coercion at build time
- this also allows UTF-8 to be re-enabled as a coercion
  target, restoring the locale coercion behaviour on Mac OS X
2017-06-30 00:48:14 +10:00
Antoine Pitrou f7d090c165 bpo-30796: Fix failures in signal delivery stress test (#2488)
* bpo-30796: Fix failures in signal delivery stress test

setitimer() can have a poor minimum resolution on some machines,
this would make the test reach its deadline (and a stray signal
could then kill a subsequent test).

* Make sure to clear the itimer after the test
2017-06-29 16:40:14 +02:00
Victor Stinner beeca6e1e5 bpo-30776: regrtest: reduce memleak false positive (#2484)
Only report a leak if each run leaks at least one memory block.
2017-06-29 10:32:49 +02:00
Antoine Pitrou c08177a1cc bpo-30703: Improve signal delivery (#2415)
* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo
2017-06-28 23:29:29 +02:00
INADA Naoki a8f8d5b4bd bpo-29585: optimize site.py startup time (GH-136)
Avoid importing `sysconfig` from `site` by copying minimum code.
Python startup is 5% faster on Linux and 30% faster on macOS
2017-06-29 00:31:53 +09:00
Antoine Pitrou 79d37ae979 Clear potential ref cycle between Process and Process target (#2470)
* Clear potential ref cycle between Process and Process target

Besides Process.join() not being called, this was an indirect cause of bpo-30775.
The threading module already does this.

* Add issue reference
2017-06-28 12:29:08 +02:00
Antoine Pitrou a79f8faccf bpo-30775: Fix refleaks in test_multiprocessing (#2467)
Forgetting to call Process.join() can keep some resources alive.
2017-06-28 11:21:52 +02:00
Serhiy Storchaka f7eae0adfc [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)
Based on patch by Victor Stinner.

Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters.
2017-06-28 08:30:06 +03:00
Victor Stinner 84d9d14a1f bpo-29512: Add test.bisect, bisect failing tests (#2452)
Add a new "python3 -m test.bisect" tool to bisect failing tests.

It can be used to find which test method(s) leak references, leak
files, etc.
2017-06-28 02:24:41 +02:00
Victor Stinner 48b5c422ff bpo-30776: reduce regrtest -R false positives (#2422)
* Change the regrtest --huntrleaks checker to decide if a test file
  leaks or not. Require that each run leaks at least 1 reference.
* Warmup runs are now completely ignored: ignored in the checker test
  and not used anymore to compute the sum.
* Add an unit test for a reference leak.

Example of reference differences previously considered a failure
(leak) and now considered as success (success, no leak):

    [3, 0, 0]
    [0, 1, 0]
    [8, -8, 1]
2017-06-27 02:02:04 +02:00
Joel Hillacre b350c22ebc bpo-30532: Fix whitespace folding in certain cases
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm.  This makes the whitespace handling code consistent.
2017-06-26 17:41:35 -04:00
Victor Stinner 63f54c6893 bpo-30764: regrtest: add --fail-env-changed option (#2402)
* bpo-30764: regrtest: change exit code on failure

* Exit code 2 if failed tests ("bad")
* Exit code 3 if interrupted

* bpo-30764: regrtest: add --fail-env-changed option

If the option is set, mark a test as failed if it alters the
environment, for example if it creates a file without removing it.
2017-06-26 18:33:19 +02:00
Victor Stinner cdee3f14f7 bpo-30764: test_subprocess uses SuppressCrashReport (#2405)
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
2017-06-26 17:23:03 +02:00
Victor Stinner ace56d5836 bpo-30523: regrtest --list-cases --match (#2401)
* regrtest --list-cases now supports --match and --match-file options.
  Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
  messages to stdout when loading test modules.
* Add support._match_test() private function.
2017-06-26 14:18:51 +02:00
Serhiy Storchaka af5392f5c6 bpo-30746: Port more tests for os.spawnvpe() and os.execve() from 2.7. (#2394) 2017-06-25 09:48:54 +03:00
Serhiy Storchaka 77703942c5 bpo-30746: Prohibited the '=' character in environment variable names (#2382)
in `os.putenv()` and `os.spawn*()`.
2017-06-25 07:33:01 +03:00
Antoine Pitrou 13e96cc596 Fix bpo-30596: Add close() method to multiprocessing.Process (#2010)
* Fix bpo-30596: Add close() method to multiprocessing.Process

* Raise ValueError if close() is called before the Process is finished running

* Add docs

* Add NEWS blurb
2017-06-24 19:22:23 +02:00
Serhiy Storchaka d174d24a5d bpo-30730: Prevent environment variables injection in subprocess on Windows. (#2325)
Prevent passing other invalid environment variables and command arguments.
2017-06-23 19:39:27 +03:00
Victor Stinner f87b85f808 bpo-21071: struct.Struct.format type is now str (#845) 2017-06-23 15:11:12 +02:00
Victor Stinner a4b091e135 bpo-30604: Skip CoExtra tests if ctypes is missing (#2356) 2017-06-23 15:08:55 +02:00
Serhiy Storchaka 32cb968a2e bpo-30727: Fix a race condition in test_threading. (#2334) 2017-06-23 13:36:36 +03:00
Benjamin Peterson 279a96206f bpo-30736: upgrade to Unicode 10.0 (#2344)
Straightforward. While we're at it, though, strip trailing whitespace from generated tables.
2017-06-22 22:31:08 -07:00
Gregory P. Smith a3d91b43c2 bpo-29212: Fix the ugly repr() ThreadPoolExecutor thread name. (#2315)
bpo-29212: Fix the ugly ThreadPoolExecutor thread name.

Fixes the newly introduced ugly default thread name for concurrent.futures
thread.ThreadPoolExecutor threads.  They'll now resemble the old <=3.5
threading default Thread-x names by being named ThreadPoolExecutor-y_n.
2017-06-21 23:41:13 -07:00
Dino Viehland f3cffd2b78 bpo-30604: clean up co_extra support (#2144)
bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
2017-06-21 17:44:36 -04:00
Dong-hee Na dcc8ce44c7 bpo-30616: Functional API of enum allows to create empty enums. (#2304)
* bpo-30616: Functional API of enum allows to create empty enums.

* Update NEWS

move addition to avoid conflict
2017-06-21 09:52:32 -07:00
Serhiy Storchaka 26cb4657bc bpo-29755: Fixed the lgettext() family of functions in the gettext module. (#2266)
They now always return bytes.

Updated the gettext documentation.
2017-06-20 17:13:29 +03:00