Commit Graph

45528 Commits

Author SHA1 Message Date
Victor Stinner 56cd3710a1
bpo-39413: Implement os.unsetenv() on Windows (GH-18104)
The os.unsetenv() function is now also available on Windows.

It is implemented with SetEnvironmentVariableW(name, NULL).
2020-01-21 16:13:09 +01:00
Victor Stinner 59e2d26b25
Move test_math tests (GH-18098)
testPerm() and testComb() belong to MathTests, not to IsCloseTests().

test_nextafter() and test_ulp() now use assertIsNaN().
2020-01-21 12:48:16 +01:00
William Chargin eab3b3f1c6 bpo-39389: gzip: fix compression level metadata (GH-18077)
As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.
2020-01-21 13:25:24 +02:00
Victor Stinner 85ead4fc62
bpo-39396: Fix math.nextafter(-0.0, +0.0) on AIX 7.1 (GH-18094)
Move also math.nextafter() on math.ulp() tests from IsCloseTests to
MathTests.
2020-01-21 11:14:10 +01:00
Cheryl Sabella ec64640a2c bpo-32989: IDLE - fix bad editor call of pyparse method (GH-5968)
Fix comments and add tests for editor newline_and_indent_event method.
Remove unused None default for function parameter of pyparse find_good_parse_start method
and code triggered by that default.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-21 05:11:26 -05:00
Andrew Svetlov a96e06db77
bpo-39386: Prevent double awaiting of async iterator (GH-18081) 2020-01-21 00:49:30 +02:00
Inada Naoki 5492bfcefe
bpo-39377: json: Remove the encoding option. (GH-18075) 2020-01-20 13:54:00 +09:00
Inada Naoki e96d954527
bpo-38536: locale: Remove trailing space in formatted currency (GH-16864) 2020-01-20 12:45:50 +09:00
Michael Haas 558f078911 Fix typo from base to based (GH-18055) 2020-01-19 05:29:42 -05:00
Victor Stinner 1d3b0aaa54
bpo-39356, zipfile: Remove code handling DeprecationWarning (GH-18027)
Remove old "except DeprecationWarning:" code path added by
commit bf02e3bb21. It's no longer
needed.

struct.pack() no longer emit DeprecationWarning if getting a float
whereas an integer is expected. It now raises an hard error instead.
2020-01-17 15:17:48 +01:00
Victor Stinner 10fd6b2b9f
bpo-39357: Update bz2 docstring: remove buffering (GH-18036)
Thanks Karthikeyan Singaravelan for the report ;-)
2020-01-17 13:50:39 +01:00
Victor Stinner 9baf242fc7
bpo-39357: Remove buffering parameter of bz2.BZ2File (GH-18028)
Remove the buffering parameter of bz2.BZ2File. Since Python 3.0, it
was ignored and using it was emitting a DeprecationWarning. Pass an
open file object to control how the file is opened.

The compresslevel parameter becomes keyword-only.
2020-01-16 15:33:30 +01:00
Victor Stinner 4691a2f2a2
bpo-39350: Remove deprecated fractions.gcd() (GH-18021)
Remove fractions.gcd() function, deprecated since Python 3.5
(bpo-22486): use math.gcd() instead.
2020-01-16 11:02:51 +01:00
Victor Stinner 210c19e3c5
bpo-39351: Remove base64.encodestring() (GH-18022)
Remove base64.encodestring() and base64.decodestring(), aliases
deprecated since Python 3.1: use base64.encodebytes() and
base64.decodebytes() instead.
2020-01-16 10:24:16 +01:00
Daniel Olshansky 01602ae403 bpo-37958: Adding get_profile_dict to pstats (GH-15495)
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.

The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.

The output of the following script:

```
import cProfile, pstats
import pprint
from pstats import func_std_string, f8

def fib(n):
    if n == 0:
        return 0
    if n == 1:
        return 1
    return fib(n-1) + fib(n-2)

pr = cProfile.Profile()
pr.enable()
fib(5)
pr.create_stats()

ps = pstats.Stats(pr).sort_stats('tottime', 'cumtime')

def get_profile_dict(self, keys_filter=None):
    """
        Returns a dict where the key is a function name and the value is a dict
        with the following keys:
            - ncalls
            - tottime
            - percall_tottime
            - cumtime
            - percall_cumtime
            - file_name
            - line_number

        keys_filter can be optionally set to limit the key-value pairs in the
        retrieved dict.
    """
    pstats_dict = {}
    func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())

    if not func_list:
        return pstats_dict

    pstats_dict["total_tt"] = float(f8(self.total_tt))
    for func in func_list:
        cc, nc, tt, ct, callers = self.stats[func]
        file, line, func_name = func
        ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
        tottime = float(f8(tt))
        percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
        cumtime = float(f8(ct))
        percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
        func_dict = {
            "ncalls": ncalls,
            "tottime": tottime, # time spent in this function alone
            "percall_tottime": percall_tottime,
            "cumtime": cumtime, # time spent in the function plus all functions that this function called,
            "percall_cumtime": percall_cumtime,
            "file_name": file,
            "line_number": line
        }
        func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
        pstats_dict[func_name] = func_dict_filtered

    return pstats_dict

pp = pprint.PrettyPrinter(depth=6)
pp.pprint(get_profile_dict(ps))
```

will produce:

```
{"<method 'disable' of '_lsprof.Profiler' objects>": {'cumtime': 0.0,
                                                      'file_name': '~',
                                                      'line_number': 0,
                                                      'ncalls': '1',
                                                      'percall_cumtime': 0.0,
                                                      'percall_tottime': 0.0,
                                                      'tottime': 0.0},
 'create_stats': {'cumtime': 0.0,
                  'file_name': '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/cProfile.py',
                  'line_number': 50,
                  'ncalls': '1',
                  'percall_cumtime': 0.0,
                  'percall_tottime': 0.0,
                  'tottime': 0.0},
 'fib': {'cumtime': 0.0,
         'file_name': 'get_profile_dict.py',
         'line_number': 5,
         'ncalls': '15/1',
         'percall_cumtime': 0.0,
         'percall_tottime': 0.0,
         'tottime': 0.0},
 'total_tt': 0.0}
 ```

 As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.



https://bugs.python.org/issue37958



Automerge-Triggered-By: @gpshead
2020-01-15 14:51:54 -08:00
Victor Stinner e85a305503
bpo-38630: Fix subprocess.Popen.send_signal() race condition (GH-16984)
On Unix, subprocess.Popen.send_signal() now polls the process status.
Polling reduces the risk of sending a signal to the wrong process if
the process completed, the Popen.returncode attribute is still None,
and the pid has been reassigned (recycled) to a new different
process.
2020-01-15 17:38:55 +01:00
Karthikeyan Singaravelan 54f743eb31 Improve test coverage for AsyncMock. (GH-17906)
* Add test for nested async decorator patch.
* Add test for side_effect and wraps with a function.
* Add test for side_effect with an exception in the iterable.
2020-01-15 09:49:49 +00:00
Dong-hee Na 65a5ce247f bpo-39329: Add timeout parameter for smtplib.LMTP constructor (GH-17998) 2020-01-14 22:42:09 +01:00
Vinay Sajip 7d6378051f
bpo-38901: Allow setting a venv's prompt to the basename of the current directory. (GH-17946)
When a prompt value of '.' is specified, os.path.basename(os.getcwd()) is used to
configure the prompt for the created venv.
2020-01-14 20:49:30 +00:00
Dima 4b0d91aab4 venv: Suppress warning message when bash hashing is disabled. (GH-17966)
When using python's built-in venv activaton script
warnings are printed when hashing is disabled in
bash or zsh, like;

`bash: hash: hashing disabled`

This output is not really useful to the end-user and has
been disabled in `virtualenv` for long.

This commit is based on:
28e85bcd80
2020-01-14 20:47:59 +00:00
Kyle Pollina b4cdb3f60e Fix documentation in code.py (GH-17988) 2020-01-15 01:17:25 +05:30
Pablo Galindo a2ec3f07f7
bpo-39322: Add gc.is_finalized to check if an object has been finalised by the gc (GH-17989) 2020-01-14 12:06:45 +00:00
Géry Ogam 1d1b97ae64 bpo-39048: Look up __aenter__ before __aexit__ in async with (GH-17609)
* Reorder the __aenter__ and __aexit__ checks for async with
* Add assertions for async with body being skipped
* Swap __aexit__ and __aenter__ loading in the documentation
2020-01-14 21:58:29 +10:00
Mark Shannon 9af0e47b17
bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
2020-01-14 10:12:45 +00:00
Dong-hee Na 62e3973395 bpo-39259: smtp.SMTP/SMTP_SSL now reject timeout = 0 (GH-17958) 2020-01-14 08:49:59 +01:00
Dong-hee Na a190e2ade1 bpo-39259: ftplib.FTP/FTP_TLS now reject timeout = 0 (GH-17959) 2020-01-13 20:34:34 +01:00
Chris Withers 31d6de5aba
remove unused __version__ from mock.py (#17977)
This isn't included in `__all__` and could be a source of confusion.
2020-01-13 19:11:34 +00:00
Karthikeyan Singaravelan d8efc14951
bpo-39299: Add more tests for mimetypes and its cli. (GH-17949)
* Add tests for case insensitive check of types and extensions as fallback.
* Add tests for data url with no comma.
* Add tests for read_mime_types.
* Add tests for the mimetypes cli and refactor __main__ code to private function.
* Restore mimetypes.knownfiles value at the end of the test.
2020-01-13 20:09:36 +05:30
Mark Shannon e7c9f4aae1
Cleanup exit code for interpreter. (GH-17756) 2020-01-13 12:51:26 +00:00
Victor Stinner 0b2ab21956
bpo-39310: Add math.ulp(x) (GH-17965)
Add math.ulp(): return the value of the least significant bit
of a float.
2020-01-13 12:44:35 +01:00
Philip McMahon b2b4a51f74 bpo-32021: Support brotli .br encoding in mimetypes (#12200)
Add support for brotli encoding in the encoding_map.
2020-01-12 14:31:49 -08:00
Batuhan Taşkaya 61b14151cc bpo-39313: Add an option to RefactoringTool for using exec as a function (GH-17967)
https://bugs.python.org/issue39313


Automerge-Triggered-By: @pablogsal
2020-01-12 14:13:31 -08:00
Ram Rachum 14dbe4b3f0 Fix outdated comment in _strptime.py (GH-17929)
Can I please get the tags for skipping bpo and skipping a news item?
2020-01-12 12:53:00 -08:00
Guðni Natan Gunnarsson 9f3fc6c5b4 bpo-38293: Allow shallow and deep copying of property objects (GH-16438)
Copying property objects results in a TypeError. Steps to reproduce:

```
>>> import copy
>>> obj = property()
>>> copy.copy(obj)
````

This affects both shallow and deep copying.  
My idea for a fix is to add property objects to the list of "atomic" objects in the copy module.
These already include types like functions and type objects.

I also added property objects to the unit tests test_copy_atomic and test_deepcopy_atomic. This is my first PR, and it's highly likely I've made some mistake, so please be kind :)


https://bugs.python.org/issue38293
2020-01-12 09:41:49 -08:00
Kyle Stanley 0ca7cc7fc0 bpo-38356: Fix ThreadedChildWatcher thread leak in test_asyncio (GH-16552)
Motivation for this PR (comment from @vstinner in bpo issue):
```
Warning seen o AMD64 Ubuntu Shared 3.x buildbot:
https://buildbot.python.org/all/#/builders/141/builds/2593

test_devnull_output (test.test_a=syncio.test_subprocess.SubprocessThreadedWatcherTests) ...
Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
```
The following implementation details for the new method are TBD:

1) Public vs private

2) Inclusion in `close()`

3) Name

4) Coroutine vs subroutine method

5) *timeout* parameter

If it's a private method, 3, 4, and 5 are significantly less important.

I started with the most minimal implementation that fixes the dangling threads without modifying the regression tests, which I think is particularly important. I typically try to avoid directly modifying existing tests as much as possible unless it's necessary to do so. However, I am open to changing any part of this.


https://bugs.python.org/issue38356
2020-01-12 03:02:50 -08:00
Vinay Sajip c12440c371
bpo-16575: Disabled checks for union types being passed by value. (GH-17960)
Although the underlying libffi issue remains open, adding these
checks have caused problems in third-party projects which are in
widespread use. See the issue for examples.

The corresponding tests have also been skipped.
2020-01-12 08:54:00 +00:00
Victor Stinner 100fafcf20
bpo-39288: Add math.nextafter(x, y) (GH-17937)
Return the next floating-point value after x towards y.
2020-01-12 02:15:42 +01:00
Dong-hee Na 1b335ae281 bpo-39259: nntplib.NNTP/NNTP_SSL now reject timeout = 0 (GH-17936)
nntplib.NNTP and nntplib.NNTP_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
2020-01-11 18:39:15 +01:00
Jason R. Coombs 136735c1a2
bpo-39297: Update for importlib_metadata 1.4. (GH-17947)
* bpo-39297: Update for importlib_metadata 1.4. Includes performance updates.

* 📜🤖 Added by blurb_it.

* Update blurb

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-01-11 10:37:28 -05:00
Dong-hee Na 5d978a2e73 bpo-39259: nntplib.NNTP/NNTP_SSL refactoring (GH-17939) 2020-01-11 16:07:36 +01:00
Karthikeyan Singaravelan 43682f1e39
Fix host in address of socket.create_server example. (GH-17706)
Host as None in address raises TypeError since it should be string, bytes or bytearray.
2020-01-11 10:46:30 +05:30
Vinay Sajip ce54519aa0
bpo-39292: Add missing syslog facility codes. (GH-17945) 2020-01-10 19:37:48 +00:00
Dong-hee Na abdc634f33 bpo-39200: Correct the error message for min/max builtin function (GH-17814)
Correct the error message when calling the min() or max() with
no arguments.
2020-01-10 17:31:43 +01:00
Dong-hee Na c39b52f152 bpo-39259: poplib now rejects timeout = 0 (GH-17912)
poplib.POP3 and poplib.POP3_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
2020-01-10 15:34:05 +01:00
Pablo Galindo 4c53e63cc9 bpo-39166: Fix trace of last iteration of async for loops (#17800) 2020-01-10 09:24:22 +00:00
Serhiy Storchaka 850a8856e1
bpo-39235: Check end_lineno and end_col_offset of AST nodes. (GH-17926) 2020-01-10 10:12:55 +02:00
Daniel Hahler 2f65aa4658 Fix typo in test's docstring (GH-17856)
* Fix typo in test's docstring. contination -> continuation.
2020-01-09 22:37:32 +05:30
Steve Dower ed367815ee
bpo-25172: Reduce scope of crypt import tests (GH-17881) 2020-01-09 09:00:29 -08:00
Karthikeyan Singaravelan eef1b027ab Add test cases for dataclasses. (#17909)
* Add test cases for dataclasses.

* Add test for repr output of field.
* Add test for ValueError to be raised when both default and default_factory are passed.
2020-01-09 08:41:46 -05:00
An Long 5907e61a8d bpo-35292: Avoid calling mimetypes.init when http.server is imported (GH-17822) 2020-01-08 10:28:14 -08:00
Dong-hee Na 2e6a8efa83 bpo-39242: Updated the Gmane domain into news.gmane.io (GH-17903) 2020-01-08 16:29:34 +01:00
Dong-hee Na b821173b54 bpo-38871: Fix lib2to3 for filter-based statements that contain lambda (GH-17780)
Correctly parenthesize filter-based statements that contain lambda
expressions in lib2to3.
2020-01-07 18:30:54 +01:00
Dong-hee Na 13a7ee8d62 bpo-38615: Add timeout parameter for IMAP4 and IMAP4_SSL constructor (GH-17203)
imaplib.IMAP4 and imaplib.IMAP4_SSL now have an 
optional *timeout* parameter for their constructors.
Also, the imaplib.IMAP4.open() method now has an optional *timeout* parameter
with this change. The overridden methods of imaplib.IMAP4_SSL and
imaplib.IMAP4_stream were applied to this change.
2020-01-07 18:28:10 +01:00
Derek Brown 950c6795aa bpo-39198: Ensure logging global lock is released on exception in isEnabledFor (GH-17689) 2020-01-07 16:40:23 +00:00
Victor Stinner 5b23f7618d
bpo-39239: epoll.unregister() no longer ignores EBADF (GH-17882)
The select.epoll.unregister() method no longer ignores the EBADF
error.
2020-01-07 15:00:02 +01:00
Andrew Svetlov 10ac0cded2 bpo-39191: Fix RuntimeWarning in asyncio test (GH-17863)
https://bugs.python.org/issue39191
2020-01-07 05:23:01 -08:00
Pablo Galindo 5ec91f78d5
bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860) 2020-01-06 15:59:09 +00:00
Jason R. Coombs 7cdc31a14c
bpo-38907: Suppress any exception when attempting to set V6ONLY. (GH-17864)
Fixes error attempting to bind to IPv4 address.
2020-01-06 07:59:36 -05:00
Jason R. Coombs ee94bdb059
bpo-38907: In http.server script, restore binding to IPv4 on Windows. (GH-17851) 2020-01-05 22:32:19 -05:00
Tal Einat d6c08db853 Minor formatting improvements and fixes to idle.rst (GH-17165) 2020-01-05 18:51:48 -05:00
Pablo Galindo 422ed16fb8
Organise and clean test_positional_only_arg and add more tests (GH-17842) 2020-01-05 18:52:39 +00:00
Pablo Galindo 4b66fa6ce9
bpo-39200: Correct the error message for range() empty constructor (GH-17813)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2020-01-05 17:30:53 +00:00
Anthony Sottile b121a4a45f Fix constant folding optimization for positional only arguments (GH-17837) 2020-01-05 17:03:56 +00:00
Terry Jan Reedy 5ea7bb25e3
bpo-39152: add missing ttk.Scale.configure return value (GH-17815)
tkinter.ttk.Scale().configure([name]) now returns a configuration tuple for name
or a list thereof for all options. Based on patch Giovanni Lombardo.
2020-01-05 11:23:58 -05:00
Serhiy Storchaka b19c0d77e6
bpo-39055: Reject a trailing \n in base64.b64decode() with validate=True. (GH-17616) 2020-01-05 14:15:50 +02:00
Serhiy Storchaka 41ec17e45d
bpo-39056: Fix handling invalid warning category in the -W option. (GH-17618)
No longer import the re module if it is not needed.
2020-01-05 14:15:27 +02:00
Serhiy Storchaka 6a265f0d0c
bpo-39057: Fix urllib.request.proxy_bypass_environment(). (GH-17619)
Ignore leading dots and no longer ignore a trailing newline.
2020-01-05 14:14:31 +02:00
Anthony Sottile ec007cb43f Fix SystemError when nested function has annotation on positional-only argument (GH-17826) 2020-01-05 01:57:21 +00:00
Andrew Svetlov 3a5de51159
Fix #39191: Don't spawn a task before failing (#17796) 2020-01-04 11:10:14 +02:00
Raymond Hettinger 4fcf5c12a3
bpo-39158: ast.literal_eval() doesn't support empty sets (GH-17742) 2020-01-02 22:21:18 -07:00
Batuhan Taşkaya 7b35bef978 bpo-38870: Throw ValueError on invalid yield from usage (GH-17798) 2020-01-02 18:20:04 +00:00
Pablo Galindo 04ec7a1f7a
bpo-39114: Fix tracing of except handlers with name binding (GH-17769)
When producing the bytecode of exception handlers with name binding (like `except Exception as e`) we need to produce a try-finally block to make sure that the name is deleted after the handler is executed to prevent cycles in the stack frame objects. The bytecode associated with this try-finally block does not have source lines associated and it was causing problems when the tracing functionality was running over it.
2020-01-02 11:38:44 +00:00
Jendrik Seipp 5b9077134c bpo-13601: always use line-buffering for sys.stderr (GH-17646) 2020-01-01 23:21:43 +01:00
Vinay Sajip 46abfc1416
bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773)
This uses the heuristic of assuming a named tuple is a subclass of
tuple with a _fields attribute. This change means that contents of
a named tuple wouldn't be converted - if a user wants to have
ConvertingTuple functionality from a namedtuple, they will have to
implement it themselves.
2020-01-01 19:32:11 +00:00
Ned Batchelder 37143a8e3b bpo-39176: Improve error message for 'named assignment' (GH-17777) 2019-12-31 20:40:58 -06:00
Terry Jan Reedy ba82ee894c
Fix idlelib README typo. (GH-17770) 2019-12-31 13:34:22 -05:00
Dong-hee Na 2d5bf568ea bpo-38588: Fix possible crashes in dict and list when calling PyObject_RichCompareBool (GH-17734)
Take strong references before calling PyObject_RichCompareBool to protect against the case
where the object dies during the call.
2019-12-31 01:04:22 +00:00
Zackery Spytz d9e561d23d bpo-38610: Fix possible crashes in several list methods (GH-17022)
Hold strong references to list elements while calling PyObject_RichCompareBool().
2019-12-30 19:32:58 +00:00
Batuhan Taşkaya 09c482fad1 bpo-39019: Implement missing __class_getitem__ for SpooledTemporaryFile (GH-17560) 2019-12-30 16:08:08 +00:00
Batuhan Taşkaya 4dc5a9df59 bpo-39019: Implement missing __class_getitem__ for subprocess classes (GH-17558) 2019-12-30 16:02:04 +00:00
Kyle Stanley 89aa7f0ede bpo-34790: Implement deprecation of passing coroutines to asyncio.wait() (GH-16977) 2019-12-30 13:50:19 +02:00
Mark Shannon 88dce26da6
Fix handling of line numbers around finally-blocks. (#17737) 2019-12-30 09:53:36 +00:00
Pablo Galindo 8f0703ff92
bpo-39157: Skip test_pidfd_send_signal if the system does not have enough privileges to use pidfd (GH-17740) 2019-12-29 21:35:54 +00:00
Pablo Galindo be287c3191
Fix error when running with -uall in test_unparse (GH-17739) 2019-12-29 20:18:36 +00:00
Pablo Galindo 23a226bf3a
bpo-38870: Run always tests that heavily use grammar features in test_unparse (GH-17738) 2019-12-29 19:20:55 +00:00
Gurupad Hegde 6c7bb38ff2 bpo-39136: Fixed typos (GH-17720)
funtion -> function; configuraton -> configuration; defintitions -> definitions;
focusses -> focuses; necesarily -> necessarily; follwing -> following;
Excape -> Escape,
2019-12-28 17:16:02 -05:00
Andrew Svetlov 025eeaa196
Fix import path for asyncio.TimeoutError (#17691) 2019-12-24 12:46:42 +02:00
Pablo Galindo d69cbeb99d
Revert "bpo-38870: Remove dependency on contextlib to avoid performance regression on import (GH-17376)" (GH-17687)
This reverts commit ded8888fbc.
2019-12-23 16:42:48 +00:00
Batuhan Taşkaya 4b3b1226e8 bpo-38870: Refactor delimiting with context managers in ast.unparse (GH-17612)
Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2019-12-23 16:11:00 +00:00
Jürgen Gmach 9f9dac0a4e bpo-38914 Do not require email field in setup.py. (GH-17388)
When checking `setup.py` and when the `author` field was provided, but
the `author_email` field was missing, erroneously a warning message was
displayed that the `author_email` field is required.

The specs do not require the `author_email`field:
https://packaging.python.org/specifications/core-metadata/#author

The same is valid for `maintainer` and `maintainer_email`.

The warning message has been adjusted.

modified:   Doc/distutils/examples.rst
modified:   Lib/distutils/command/check.py


https://bugs.python.org/issue38914
2019-12-23 06:53:18 -08:00
Bar Harel eae87e3e4e bpo-38878: Fix os.PathLike __subclasshook__ (GH-17336)
Quick subclasshook fix using the same method is being used in collections.abc (up to a certain degree).
2019-12-22 09:57:27 +00:00
Łukasz Langa 6202d856d6
Python 3.9.0a2 2019-12-18 22:09:19 +01:00
Victor Stinner 673c39331f
bpo-38546: Fix concurrent.futures test_ressources_gced_in_workers() (GH-17652)
Fix test_ressources_gced_in_workers() of test_concurrent_futures:
explicitly stop the manager to prevent leaking a child process
running in the background after the test completes.
2019-12-18 15:50:04 +01:00
Lysandros Nikolaou 50d4f12958 bpo-39080: Starred Expression's column offset fix when inside a CALL (GH-17645)
Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>
2019-12-18 00:20:55 +00:00
Victor Stinner 9707e8e22d
bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)
Multiprocessing and concurrent.futures tests now stop the resource
tracker process when tests complete.

Add ResourceTracker._stop() method to
multiprocessing.resource_tracker.

Add _cleanup_tests() helper function to multiprocessing.util: share
code between multiprocessing and concurrent.futures tests.
2019-12-17 18:37:26 +01:00
Steve Dower a76ba362c4
bpo-39041: Add GitHub Actions support (GH-17594) 2019-12-16 10:35:22 -08:00
Batuhan Taşkaya 814d687c7d bpo-38348: Extend command line options of ast parsing tool (GH-16540)
Add -i and --indent (indentation level), and --no-type-comments
(type comments) command line options to ast parsing tool.
2019-12-16 19:23:27 +01:00
Batuhan Taşkaya a322f50c36 bpo-38870: Remove dead code related with argument unparsing (GH-17613) 2019-12-16 12:26:58 +00:00
Toke Høiland-Jørgensen 092435e932 bpo-38811: Check for presence of os.link method in pathlib (GH-17225)
Commit 6b5b013bcc ("bpo-26978: Implement pathlib.Path.link_to (Using
os.link) (GH-12990)") introduced a new link_to method in pathlib. However,
this makes pathlib crash when the 'os' module is missing a 'link' method.

Fix this by checking for the presence of the 'link' method on pathlib
module import, and if it's not present, turn it into a runtime error like
those emitted when there is no lchmod() or symlink().

Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
2019-12-16 13:23:55 +01:00
Raymond Hettinger 1ca8fb187e
Add tests and design notes for Counter subset/superset operations. (GH-17625) 2019-12-16 01:54:14 -08:00
Xtreak 79f02fee1a bpo-39033: Fix NameError in zipimport during hash validation (GH-17588)
Patch by Karthikeyan Singaravelan.
2019-12-16 09:34:12 +10:00
Batuhan Taşkaya cb8b946ac1 bpo-38629: implement __floor__ and __ceil__ for float type (GH-16985) 2019-12-15 23:00:28 +01:00
Michael Felt 39afa2d314 bpo-38021: Modify AIX platform_tag so it covers PEP 425 needs (GH-17303)
Provides a richer platform tag for AIX that we expect to be sufficient for PEP 425
binary distribution identification. Any backports to earlier Python versions will be
handled via setuptools.

Patch by Michael Felt.
2019-12-16 00:17:53 +10:00
Daniel Andersson 40c01c3346 Fix typo in site module (GH-17597) 2019-12-14 10:37:58 +00:00
Lysandros Nikolaou 5936a4ce91 Fix elif start column offset when there is an else following (GH-17596) 2019-12-14 10:24:57 +00:00
Xtreak 8289e27393 bpo-36406: Handle namespace packages in doctest (GH-12520) 2019-12-13 10:06:53 -08:00
Lysandros Nikolaou 025a602af7 bpo-39031: Include elif keyword when producing lineno/col-offset info for if_stmt (GH-17582)
When parsing an "elif" node, lineno and col_offset of the node now point to the "elif" keyword and not to its condition, making it consistent with the "if" node.


https://bugs.python.org/issue39031



Automerge-Triggered-By: @pablogsal
2019-12-12 13:40:21 -08:00
Kyle Stanley 1988344a6b Fix warnings in test_asyncio.test_base_events (#17577)
Co-authored-by: tirkarthi
2019-12-12 14:48:20 +01:00
Victor Stinner 7772b1af5e
bpo-38614: Use support timeout constants (GH-17572) 2019-12-11 22:17:04 +01:00
Victor Stinner 0d63bacefd
bpo-38614: Use test.support.SHORT_TIMEOUT constant (GH-17566)
Replace hardcoded timeout constants in tests with SHORT_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.

SHORT_TIMEOUT is 30 seconds by default, but it can be longer
depending on --timeout command line option.

The change makes almost all timeouts longer, except
test_reap_children() of test_support which is made 2x shorter:
SHORT_TIMEOUT should be enough. If this test starts to fail,
LONG_TIMEOUT should be used instead.

Uniformize also "from test import support" import in some test files.
2019-12-11 11:30:03 +01:00
Jason R. Coombs b7a0109cd2
bpo-39022, bpo-38594: Sync with importlib_metadata 1.3 (GH-17568)
* bpo-39022, bpo-38594: Sync with importlib_metadata 1.3 including improved docs for custom finders and better serialization support in EntryPoints.

* 📜🤖 Added by blurb_it.

* Correct module reference
2019-12-10 20:05:10 -05:00
Victor Stinner 1d0f9b316a
bpo-38614: Use test.support.INTERNET_TIMEOUT constant (GH-17565)
Replace hardcoded timeout constants in tests with INTERNET_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.
2019-12-10 22:09:23 +01:00
Victor Stinner c98b0199a9
bpo-38614: Use test.support.LONG_TIMEOUT constant (GH-17562)
Replace hardcoded timeout constants in tests with LONG_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.

LONG_TIMEOUT is 5 minutes by default, but it can be longer depending
on --timeout command line option.
2019-12-10 21:12:26 +01:00
Victor Stinner bbc8b7965b
bpo-38614: Use default join_thread() timeout in tests (GH-17559)
Tests no longer pass a timeout value to join_thread() of
test.support: use the default join_thread() timeout instead
(SHORT_TIMEOUT constant of test.support).
2019-12-10 20:41:23 +01:00
Victor Stinner 07871b256c
bpo-38614: Use test.support.LOOPBACK_TIMEOUT constant (GH-17554)
Replace hardcoded timeout constants in tests with LOOPBACK_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.
2019-12-10 20:32:59 +01:00
Giampaolo Rodola 82374979ec
bpo-39004: increment large sendfile() test timeout (GH-17552) 2019-12-10 17:31:06 +08:00
Pablo Galindo e9df88e8e9
Clean imports in test_unparse (GH-17545) 2019-12-10 00:37:47 +00:00
JohnnyNajera bbc4162baf bpo-38943: Fix IDLE autocomplete window not always appearing (GH-17416)
This has happened on some versions of Ubuntu.
2019-12-09 19:30:01 -05:00
JohnnyNajera 232689b40d bpo-38944: Escape key now closes IDLE completion windows. (GH-17419) 2019-12-09 18:22:16 -05:00
Tim Gates 2ad7651c00 bpo-39009: Fix typo in test__locale (GH-17544) 2019-12-09 22:16:00 +00:00
Steve Dower ee17e37356
bpo-39007: Add auditing events to functions in winreg (GH-17541)
Also allows winreg.CloseKey() to accept same types as other functions.
2019-12-09 11:18:12 -08:00
Pablo Galindo ac229116a3
bpo-39003: Make sure all test are the same when using -R in test_unparse (GH-17537) 2019-12-09 17:57:50 +00:00
Tim Gates c18b805ac6 bpo-39002: Fix simple typo: tranlation -> translation (GH-17517) 2019-12-09 09:42:17 -08:00
Victor Stinner a1a99b4bb7
bpo-20443: No longer make sys.argv[0] absolute for script (GH-17534)
In Python 3.9.0a1, sys.argv[0] was made an asolute path if a filename
was specified on the command line. Revert this change, since most
users expect sys.argv to be unmodified.
2019-12-09 17:34:02 +01:00
Yury Selivanov d219cc4180 bpo-34776: Fix dataclasses to support __future__ "annotations" mode (#9518) 2019-12-09 15:54:20 +01:00
Mark Dickinson bba873e633
bpo-38992: avoid fsum test failure from constant-folding (GH-17513)
* Issue 38992: avoid fsum test failure

* Add NEWS entry
2019-12-09 08:36:34 -06:00
Kyle Stanley ab513a38c9 bpo-37228: Fix loop.create_datagram_endpoint()'s usage of SO_REUSEADDR (#17311) 2019-12-09 15:21:10 +01:00
Victor Stinner 82b4950b5e
bpo-39006: Fix asyncio when the ssl module is missing (GH-17524)
Fix asyncio when the ssl module is missing: only check for
ssl.SSLSocket instance if the ssl module is available.
2019-12-09 15:02:03 +01:00
Victor Stinner 0131aba5ae
bpo-38916: array.array: remove fromstring() and tostring() (GH-17487)
array.array: Remove tostring() and fromstring() methods.  They were
aliases to tobytes() and frombytes(), deprecated since Python 3.2.
2019-12-09 14:09:14 +01:00
Victor Stinner a1838ec259
bpo-38547: Fix test_pty if the process is the session leader (GH-17519)
Fix test_pty: if the process is the session leader, closing the
master file descriptor raises a SIGHUP signal: simply ignore SIGHUP
when running the tests.
2019-12-09 11:57:05 +01:00
Abhilash Raj 3ae4ea1931
bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)
Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
2019-12-08 17:37:34 -08:00
Abhilash Raj 68157da8b4
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
2019-12-08 17:35:38 -08:00
Batuhan Taşkaya 526606baf7 bpo-38994: Implement __class_getitem__ for PathLike (GH-17498)
https://bugs.python.org/issue38994
2019-12-08 12:31:15 -08:00
Elena Oat cd90a52983 bpo-38669: patch.object now raises a helpful error (GH17034)
This means a clearer message is now shown when patch.object is called with two string arguments, rather than a class and a string argument.
2019-12-08 20:14:38 +00:00
AMIR 28c91631c2 bpo-38979: fix ContextVar "__class_getitem__" method (GH-17497)
now contextvars.ContextVar "__class_getitem__" method returns ContextVar class, not None. 


https://bugs.python.org/issue38979



Automerge-Triggered-By: @asvetlov
2019-12-08 03:35:59 -08:00
Victor Stinner 6cac113666
bpo-38991: Remove test.support.strip_python_stderr() (GH-17490)
test.support: run_python_until_end(), assert_python_ok() and
assert_python_failure() functions no longer strip whitespaces from
stderr.
2019-12-08 08:38:16 +01:00
Christian Heimes 2b7de6696b bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190)
test_openssl_version now accepts version 3.0.0.

getpeercert() no longer returns IPv6 addresses with a trailing new line.

Signed-off-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue38820
2019-12-07 08:59:36 -08:00
Daniel Himmelstein 15fb7fa881 bpo-29636: json.tool: Add document for indentation options. (GH-17482)
And updated test to use subprocess.run
2019-12-07 23:14:40 +09:00
idomic 892f9e0777 bpo-37404: Raising value error if an SSLSocket is passed to asyncio functions (GH-16457)
https://bugs.python.org/issue37404
2019-12-07 03:52:35 -08:00
Andrew Svetlov 7ddcd0caa4
bpo-38529: Fix asyncio stream warning (GH-17474) 2019-12-07 13:22:00 +02:00
Batuhan Taşkaya dec367261e bpo-38978: Implement __class_getitem__ for asyncio objects (GH-17491)
https://bugs.python.org/issue38978
2019-12-07 03:05:07 -08:00
Victor Stinner e76ee1a72b
bpo-38982: Fix asyncio PidfdChildWatcher on waitpid() error (GH-17477)
If waitpid() is called elsewhere, waitpid() call fails with
ChildProcessError: use return code 255 in this case, and log a
warning. It ensure that the pidfd file descriptor is closed if this
error occurs.
2019-12-06 16:32:41 +01:00
Mario Corchero b64334cb93 bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
Break cycle generated when saving an exception in socket.py, codeop.py and dyld.py as they keep alive not only the exception but user objects through the ``__traceback__`` attribute.


https://bugs.python.org/issue36820



Automerge-Triggered-By: @pablogsal
2019-12-06 06:27:38 -08:00
wim glenn efefe25443 bpo-27413: json.tool: Add --no-ensure-ascii option. (GH-17472) 2019-12-06 15:44:01 +09:00
Hill Ma 99eb70a9eb bpo-38951: Use threading.main_thread() check in asyncio (GH-17433)
https://bugs.python.org/issue38951
2019-12-05 04:40:12 -08:00
Claudiu Popa bb815499af bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.




https://bugs.python.org/issue38698
2019-12-04 19:14:26 -08:00
Inada Naoki 808769f3a4
bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460) 2019-12-04 18:39:31 +09:00
Pablo Galindo 24f5cac725 bpo-38962: Fix reference leak in test_httpservers (GH-17454) 2019-12-04 10:29:10 +01:00
Daniel Himmelstein 03257949bc bpo-29636: Add --(no-)indent arguments to json.tool (GH-345) 2019-12-04 15:15:19 +09:00
stratakis 894331838b bpo-38270: Fix indentation of test_hmac assertions (GH-17446)
Since c64a1a61e6 two assertions were indented and thus ignored when running test_hmac.

This PR fixes it. As the change is quite trivial I didn't add a NEWS entry.


https://bugs.python.org/issue38270
2019-12-03 07:35:54 -08:00
Matthew Rollings a62ad4730c bpo-38945: UU Encoding: Don't let newline in filename corrupt the output format (#17418) 2019-12-02 14:25:21 -08:00
torsava 34864d1cff bpo-38815: Accept TLSv3 default in min max test (GH-NNNN) (GH-17437)
Make ssl tests less strict and also accept TLSv3 as the default maximum
version. This change unbreaks test_min_max_version on Fedora 32.


https://bugs.python.org/issue38815
2019-12-02 08:15:42 -08:00
Dong-hee Na 2fe4c48917 bpo-38449: Add URL delimiters test cases (#16729)
* bpo-38449: Add tricky test cases

* bpo-38449: Reflect codereview
2019-12-01 15:06:28 -08:00
Daniel Hillier 8d62df60d8 bpo-37523: Raise ValueError for I/O operations on a closed zipfile.ZipExtFile. (GH-14658)
Raises ValueError when calling the following on a closed zipfile.ZipExtFile: read, readable, seek, seekable, tell.
2019-11-30 10:30:47 +02:00
Brett Cannon 1df65f7c6c Fix old mention of virtualenv (GH-17417)
Automerge-Triggered-By: @brettcannon
2019-11-29 15:37:08 -08:00
Steve Dower bea33f5e1d
bpo-38920: Add audit hooks for when sys.excepthook and sys.unraisable hooks are invoked (GH-17392)
Also fixes some potential segfaults in unraisable hook handling.
2019-11-28 08:46:11 -08:00
Tzu-ping Chung d9aa216d49 bpo-38927: Use python -m pip to upgrade venv deps (GH-17403)
I suggest you add `bpo-NNNNN: ` as a prefix for the first commit for future PRs. Thanks!
2019-11-27 20:25:23 +00:00
Inada Naoki ea9835c5d1
bpo-26730: Fix SpooledTemporaryFile data corruption (GH-17400)
SpooledTemporaryFile.rollback() might cause data corruption
when it is in text mode.

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
2019-11-27 22:22:06 +09:00
Bruno P. Kinoshita 9bbcbc9f6d bpo-38688, shutil.copytree: consume iterator and create list of entries to prevent infinite recursion (GH-17098) 2019-11-27 09:10:37 +08:00
HongWeipeng 0b41a922f9 bpo-38045: Improve the performance of _decompose() in enum.py (GH-16483)
* Improve the performance of _decompose() in enum.py

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-11-26 14:36:02 -08:00
HongWeipeng 036fe85bd3 bpo-27145: small_ints[x] could be returned in long_add and long_sub (GH-15716) 2019-11-26 16:54:49 +09:00
Brandt Bucher 6dd9b64770 bpo-38328: Speed up the creation time of constant list and set display. (GH-17114) 2019-11-26 15:16:53 +09:00
Stefan Behnel c6a7bdb356
bpo-20928: support base-URL and recursive includes in etree.ElementInclude (#5723)
* bpo-20928: bring elementtree's XInclude support en-par with the implementation in lxml by adding support for recursive includes and a base-URL.

* bpo-20928: Support xincluding the same file multiple times, just not recursively.

* bpo-20928: Add 'max_depth' parameter to xinclude that limits the maximum recursion depth to 6 by default.

* Add news entry for updated ElementInclude support
2019-11-25 16:36:25 +01:00
Pablo Galindo ded8888fbc bpo-38870: Remove dependency on contextlib to avoid performance regression on import (GH-17376)
https://bugs.python.org/issue38870



Automerge-Triggered-By: @pablogsal
2019-11-25 03:49:17 -08:00
Pablo Galindo 27fc3b6f3f
bpo-38870: Expose a function to unparse an ast object in the ast module (GH-17302)
Add ast.unparse() as a function in the ast module that can be used to unparse an
ast.AST object and produce a string with code that would produce an equivalent ast.AST
object when parsed.
2019-11-24 23:02:40 +00:00
Terry Jan Reedy 6bf644ec82
bpo-38862: IDLE Strip Trailing Whitespace fixes end newlines (GH-17366)
Extra newlines are removed at the end of non-shell files. If the file only has newlines after stripping other trailing whitespace, all are removed, as is done by patchcheck.py.
2019-11-24 16:29:29 -05:00
Claudiu Popa 6f03b236c1 bpo-38876: Raise pickle.UnpicklingError when loading an item from memo for invalid input (GH-17335)
The previous code was raising a `KeyError` for both the Python and C implementation.
This was caused by the specified index of an invalid input which did not exist
in the memo structure, where the pickle stores what objects it has seen.
The malformed input would have caused either a `BINGET` or `LONG_BINGET` load
from the memo, leading to a `KeyError` as the determined index was bogus.

https://bugs.python.org/issue38876



https://bugs.python.org/issue38876
2019-11-24 11:15:08 -08:00
Batuhan Taşkaya e407646b74 Remove unnecessary variable definition (GH-17368) 2019-11-24 16:46:18 +00:00
Zac Hatfield-Dodds 665ad3dfa9 Better runtime TypedDict (GH-17214)
This patch enables downstream projects inspecting a TypedDict subclass at runtime to tell which keys are optional.

This is essential for generating test data with Hypothesis or validating inputs with typeguard or pydantic.
2019-11-24 10:48:48 +00:00
Raymond Hettinger 041d8b48a2
bpo-38881: choices() raises ValueError when all weights are zero (GH-17362) 2019-11-23 02:22:13 -08:00
Brett Cannon 84b1ff6560 bpo-38899: virtual environment activation for fish should use `source` (GH-17359)
The previously documented use of `.` is considered deprecated (https://fishshell.com/docs/current/commands.html#source).


https://bugs.python.org/issue38899



Automerge-Triggered-By: @brettcannon
2019-11-22 23:32:27 -08:00
PypeBros 14a89c4798 bpo-38686: fix HTTP Digest handling in request.py (#17045)
* fix HTTP Digest handling in request.py

There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
>      qop-options       = "qop" "=" <"> 1#qop-value <">
>      qop-value         = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server.  The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection

This is description confirmed by the definition of the [//n//]`#`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.

When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.

* 📜🤖 Added by blurb_it.

* bpo-38686 review fix: remember why.

* fix trailing space in Lib/urllib/request.py

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-11-22 15:19:08 -08:00
bcaller 1b779bfb85 bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).

LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.

The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$

Therefore, a long sequence of spaces can trigger bad performance.

Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")

caused catastrophic backtracking.

The fix removes ambiguity about which \s* should match a particular
space.

You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.

    from http.server import BaseHTTPRequestHandler, HTTPServer

    def make_set_cookie_value(n_spaces):
        spaces = " " * n_spaces
        expiry = f"1-c-1{spaces}!"
        return f"b;Expires={expiry}"

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            self.log_request(204)
            self.send_response_only(204)  # Don't bother sending Server and Date
            n_spaces = (
                int(self.path[1:])  # Can GET e.g. /100 to test shorter sequences
                if len(self.path) > 1 else
                65506  # Max header line length 65536
            )
            value = make_set_cookie_value(n_spaces)
            for i in range(99):  # Not necessary, but we can have up to 100 header lines
                self.send_header("Set-Cookie", value)
            self.end_headers()

    if __name__ == "__main__":
        HTTPServer(("", 44020), Handler).serve_forever()

This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.

Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://localhost:44020/")

The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):

    import requests
    requests.get("http://localhost:44020/")

* Regression test for http.cookiejar REDoS

If we regress, this test will take a very long time.

* Improve performance of http.cookiejar.ISO_DATE_RE

A string like

"444444" + (" " * 2000) + "A"

could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
2019-11-22 15:22:11 +01:00
jacksonriley 138e7bbb0a bpo-38866: Remove asyncore from test_pyclbr.py (GH-17316)
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
2019-11-22 21:51:58 +09:00
Siwon Kang 91daa9d722 bpo-38863: Improve is_cgi() in http.server (GH-17312)
is_cgi() function of http.server library does not currently handle a
cgi script if one of the cgi_directories is located at the
sub-directory of given path. Since is_cgi() in CGIHTTPRequestHandler
class separates given path into (dir, rest) based on the first seen
'/', multi-level directories like /sub/dir/cgi-bin/hello.py is divided
into head=/sub, rest=dir/cgi-bin/hello.py then check whether '/sub'
exists in cgi_directories = [..., '/sub/dir/cgi-bin'].
This patch makes the is_cgi() keep expanding dir part to the next '/'
then checking if that expanded path exists in the cgi_directories.

Signed-off-by: Siwon Kang <kkangshawn@gmail.com>





https://bugs.python.org/issue38863
2019-11-22 01:13:05 -08:00
Raymond Hettinger b4e5eeac26
Defer import of shutil which only needed for help and usage (GH-17334) 2019-11-21 22:51:45 -08:00
benedwards14 0aca3a3a1e bpo-37838: get_type_hints for wrapped functions with forward reference (GH-17126)
https://bugs.python.org/issue37838
2019-11-21 17:24:58 +00:00
Victor Stinner 3ab479a2d1
bpo-38692: Skip test_posix.test_pidfd_open() on EPERM (GH-17290)
Skip the test_posix.test_pidfd_open() test if os.pidfd_open() fails
with a PermissionError. This situation can happen in a Linux sandbox
using a syscall whitelist which doesn't allow the pidfd_open()
syscall yet (like systemd-nspawn).
2019-11-21 12:54:54 +01:00
Victor Stinner 0127bb1c5c
bpo-38875: test_capi: trashcan tests require cpu resource (GH-17314)
test_capi: trashcan tests now require the test "cpu" resource.
2019-11-21 12:54:02 +01:00
Mark Shannon fee552669f
Produce cleaner bytecode for 'with' and 'async with' by generating separate code for normal and exceptional paths. (#6641)
Remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes. Implement finally blocks by code duplication.
Reimplement frame.lineno setter using line numbers rather than bytecode offsets.
2019-11-21 09:11:43 +00:00
Jason Fried 046442d02b bpo-38857: AsyncMock fix for awaitable values and StopIteration fix [3.8] (GH-17269) 2019-11-20 16:27:51 -08:00
xdegaye 559bad1a70 bpo-38841: Skip asyncio test_create_datagram_endpoint_existing_sock_unix (GH-17294)
on platforms lacking a functional bind() for named unix domain sockets



https://bugs.python.org/issue38841



Automerge-Triggered-By: @asvetlov
2019-11-20 12:02:07 -08:00
Federico Bond be5c79e033 bpo-38821: Fix crash in argparse when using gettext (GH-17192) 2019-11-20 15:29:29 +02:00
Terry Jan Reedy b8462477bf
bpo-38636: Fix IDLE tab toggle and file indent width (GH-17008)
These Format menu functions (default shortcuts Alt-T and Alt-U)
were mistakenly disabled in 3.7.5 and 3.8.0.
2019-11-20 01:18:39 -05:00
Benjamin Peterson 7483451577
closes bpo-38712: Add signal.pidfd_send_signal. (GH-17070)
This exposes a Linux-specific syscall for sending a signal to a process
identified by a file descriptor rather than a pid.

For simplicity, we don't support the siginfo_t parameter to the syscall. This
parameter allows implementing a pidfd version of rt_sigqueueinfo(2), which
Python also doesn't support.
2019-11-19 20:39:14 -08:00
Łukasz Langa 1c5a71a7dd
Merge tag 'v3.9.0a1' 2019-11-20 02:05:28 +01:00
Pablo Galindo e0cd8aa70a
bpo-37957: Allow regrtest to receive a file with test (and subtests) to ignore (GH-16989)
When building Python in some uncommon platforms there are some known tests that will fail. Right now, the test suite has the ability to ignore entire tests using the -x option and to receive a filter file using the --matchfile filter. The problem with the --matchfile option is that it receives a file with patterns to accept and when you want to ignore a couple of tests and subtests, is too cumbersome to lists ALL tests that are not the ones that you want to accept and he problem with -x is that is not easy to ignore just a subtests that fail and the whole test needs to be ignored.

For these reasons, add a new option to allow to ignore a list of test and subtests for these situations.
2019-11-19 23:46:49 +00:00
Pablo Galindo 293dd23477
Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246)
Capturing exceptions into names can lead to reference cycles though the __traceback__ attribute of the exceptions in some obscure cases that have been reported previously and fixed individually. As these variables are not used anyway, we can remove the binding to reduce the chances of creating reference cycles.

See for example GH-13135
2019-11-19 21:34:03 +00:00
Jake Tesler c6b20be85c bpo-38707: Fix for multiprocessing.Process MainThread.native_id (GH-17088)
This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.

In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own). 

This change forces the Process object to update its `native_id` attribute during the bootstrap process.

cc @vstinner





https://bugs.python.org/issue38707



Automerge-Triggered-By: @pitrou
2019-11-19 11:50:12 -08:00
Adam Johnson 892221bfa0 bpo-38839: Fix some unused functions in tests (GH-17189) 2019-11-19 11:45:20 -08:00
Vincent Michel 8e0de2a480 bpo-35409: Ignore GeneratorExit in async_gen_athrow_throw (GH-14755)
Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.





https://bugs.python.org/issue35409
2019-11-19 05:53:52 -08:00
Łukasz Langa fd757083df
Python 3.9.0a1 2019-11-19 12:17:21 +01:00
Dong-hee Na 9960230f76 bpo-22367: Update test_fcntl.py for spawn process mode (#17154) 2019-11-19 09:12:42 +01:00
Tomás Farías fe75b62575 bpo-38807: Add os.PathLike to exception message raised by _check_arg_types (#17160) 2019-11-18 21:54:00 -08:00
Steve Dower 00923c6399
bpo-38622: Add missing audit events for ctypes module (GH-17158) 2019-11-18 11:32:46 -08:00
jsnklln e243bae999 bpo-38722: Runpy use io.open_code() (GH-17234)
https://bugs.python.org/issue38722



Automerge-Triggered-By: @taleinat
2019-11-18 11:11:13 -08:00
Victor Stinner 59c80889ff
Revert "bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)" (#17219)
This reverts commit 111772fc27.
2019-11-18 12:26:37 +01:00
Toke Høiland-Jørgensen 111772fc27 bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)
Fix also the Path.symplink() method implementation for the case when
symlinks are not supported.
2019-11-17 19:06:38 +02:00
Andrey Doroschenko 645005e947 bpo-38724: Implement subprocess.Popen.__repr__ (GH-17151) 2019-11-17 16:08:31 +02:00
Jason (Perry) Taylor d0acdfcf34 Fix typo in Lib/socketserver.py (GH-17024)
changed 'This is bad class design, but save some typing'
into 'This is bad class design, but saves some typing'.
2019-11-16 19:14:45 +01:00
Serhiy Storchaka a0652328a2
bpo-28286: Deprecate opening GzipFile for writing implicitly. (GH-16417)
Always specify the mode argument for writing.
2019-11-16 18:56:57 +02:00
Serhiy Storchaka 5fd5cb8d85
bpo-38639: Optimize floor(), ceil() and trunc() for floats. (GH-16991) 2019-11-16 18:00:57 +02:00
Steve Dower 7c6130c8c3
bpo-38453: Ensure correct short path is obtained for test (GH-17184) 2019-11-15 16:04:00 -08:00
Steve Dower abde52cd8e
bpo-38453: Ensure ntpath.realpath correctly resolves relative paths (GH-16967)
Ensure isabs() is always True for \\?\ prefixed paths
Avoid unnecessary usage of readlink() to avoid resolving broken links incorrectly
Ensure shutil tests run in test directory
2019-11-15 09:49:21 -08:00
Kyle Stanley 3f8cebd32c bpo-38692: Add asyncio.PidfdChildWatcher to __all__ (GH-17161)
/cc @asvetlov @1st1 


https://bugs.python.org/issue38692



Automerge-Triggered-By: @benjaminp
2019-11-14 18:47:56 -08:00
Benjamin Peterson 3ccdd9b180
closes bpo-38692: Add a pidfd child process watcher to asyncio. (GH-17069) 2019-11-13 19:08:50 -08:00
Andrew Svetlov dad6be5ffe bpo-38785: Prevent asyncio from crashing (GH-17144)
if parent `__init__` is not called from a constructor of object derived from `asyncio.Future`



https://bugs.python.org/issue38785
2019-11-13 13:36:46 -08:00
Kirill 61289d4366 bpo-38786: Add parsing of https links to pydoc (GH-17143) 2019-11-13 18:13:52 +02:00
Daniel Andersson d89cea15ad bpo-38781: Clear buffer in MemoryHandler flush (GH-17132)
This makes it easier to use a custom buffer when subclassing
MemoryHandler (by avoiding the explicity empty list literal
assignment in the flush method). For example, collection.deque
can now be used without any modifications to MemoryHandler.flush.

The same applies to BufferingHandler.
2019-11-13 09:03:45 +00:00
Zackery Spytz 9c2844927d bpo-4630: Add cursor no-blink option for IDLE (GH-16960)
This immediately toggles shell, editor, and output windows, but does not affect other input widgets.
2019-11-13 02:13:33 -05:00
Benjamin Peterson 74fa9f723f
closes bpo-27805: Ignore ESPIPE in initializing seek of append-mode files. (GH-17112)
This change, which follows the behavior of C stdio's fdopen and Python 2's file object, allows pipes to be opened in append mode.
2019-11-12 14:51:34 -08:00
jsnklln d593881505 bpo-38723: Pdb._runscript should use io.open_code() instead of open() (GH-17127)
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-11-12 14:42:47 -08:00
Serhiy Storchaka 138ccbb022
bpo-38738: Fix formatting of True and False. (GH-17083)
* "Return true/false" is replaced with "Return ``True``/``False``"
  if the function actually returns a bool.
* Fixed formatting of some True and False literals (now in monospace).
* Replaced "True/False" with "true/false" if it can be not only bool.
* Replaced some 1/0 with True/False if it corresponds the code.
* "Returns <bool>" is replaced with "Return <bool>".
2019-11-12 16:57:03 +02:00
Vinay Sajip 106271568c
bpo-16576: Add checks for bitfields passed by value to functions. (GH-17097) 2019-11-12 12:29:34 +00:00
Zackery Spytz c8b53dc3d8 bpo-26353: IDLE adds an unneeded newline when saving a shell window (GH-17103) 2019-11-12 05:54:10 -05:00
Raymond Hettinger 733b9a308e
bpo-38385: Fix iterator/iterable terminology in statistics docs (GH-17111) 2019-11-11 23:35:06 -08:00
Manjusaka 051ff526b5 bpo-38565: add new cache_parameters method for lru_cache (GH-16916) 2019-11-11 23:30:18 -08:00
Brandt Bucher a0ed99bca8 bpo-38438: Simplify argparse "star nargs" usage. (GH-17106) 2019-11-11 12:47:48 -08:00
Raymond Hettinger 84ac437658
bpo-38761: Register WeakSet as a MutableSet (GH-17104) 2019-11-10 20:12:04 -08:00
Serhiy Storchaka e27449da92
bpo-38635: Simplify decoding the ZIP64 extra field and make it tolerant to extra data. (GH-16988) 2019-11-09 13:13:36 +02:00
Dong-hee Na befa032d88 bpo-22367: Add tests for fcntl.lockf(). (GH-17010) 2019-11-07 22:31:41 +02:00
l0rb 991b02dc87 update a deprecated assert in logging tests (GH-17079) 2019-11-07 10:13:36 +00:00
l0rb 519cb8772a bpo-38716: stop rotating handlers from setting inherited namer and rotator to None (GH-17072) 2019-11-06 21:21:40 +00:00
Benjamin Peterson 6c4c45efae
bpo-38692: Add os.pidfd_open. (GH-17063) 2019-11-05 19:21:29 -08:00
Jeroen Demeyer bf17d41826 bpo-37645: add new function _PyObject_FunctionStr() (GH-14890)
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.

CC @vstinner @encukou 


https://bugs.python.org/issue37645



Automerge-Triggered-By: @encukou
2019-11-05 07:48:04 -08:00
Eddie Elizondo b3966639d2 bpo-35381 Remove all static state from posixmodule (GH-15892)
After #9665, this moves the remaining types in posixmodule to be heap-allocated to make it compatible with PEP384 as well as modifying all the type accessors to fully make the type opaque.

The original PR that got messed up a rebase: https://github.com/python/cpython/pull/10854. All the issues in that commit have now been addressed since https://github.com/python/cpython/pull/11661 got committed.

This change also removes any state from the data segment and onto the module state itself.


https://bugs.python.org/issue35381



Automerge-Triggered-By: @encukou
2019-11-05 07:16:14 -08:00
Michael Haas 25fa3ecb98 Fix a typo in wave module docstring (GH-17009)
s/pathing/patching/
2019-11-04 22:32:10 -06:00
Ram Rachum 8d4fef4ee2 bpo-38422: Clarify docstrings of pathlib suffix(es) (GH-16679)
Whenever I use `path.suffix` I have to check again whether it includes the dot or not. I decided to add it to the docstring so I won't have to keep checking. 


https://bugs.python.org/issue38422



Automerge-Triggered-By: @pitrou
2019-11-02 09:46:24 -07:00
Jon Janzen d0d9f7cfa3 Slightly improve plistlib test coverage. (GH-17025)
* Add missing test class (mistake in GH-4455)

* Increase coverage with 4 more test cases

* Rename neg_uid to huge_uid in test_modified_uid_huge

* Replace test_main() with unittest.main()

* Update plistlib docs
2019-11-01 18:45:01 +02:00
MaT1g3R 65c7382c47 Add docstring for shlex.split (GH-16740) 2019-10-31 10:23:20 +00:00
Anthony Sottile b32cb97bce bpo-38312: Add curses.{get,set}_escdelay and curses.{get,set}_tabsize. (GH-16938) 2019-10-31 11:13:48 +02:00
Lucas Cimon b15100fe7d bpo-38586: setting logging.Handler .name property in fileConfig (GH-16918) 2019-10-31 08:06:25 +00:00
Vinay Sajip 79d4ed102a
bpo-16575: Add checks for unions passed by value to functions. (GH-16799) 2019-10-31 08:03:54 +00:00
Victor Stinner a4ed6ed9f3
bpo-38614: Increase asyncio test_communicate() timeout (GH-16995)
Fix test_communicate() of test_asyncio.test_subprocess: use
support.LONG_TIMEOUT (5 minutes), instead of 1 minute.
2019-10-30 16:00:44 +01:00
Pablo Galindo 6c3e66a34b
bpo-38640: Allow break and continue in always false while loops (GH-16992) 2019-10-30 11:53:26 +00:00
Victor Stinner 24c6258269
bpo-38614: Add timeout constants to test.support (GH-16964)
Add timeout constants to test.support:

* LOOPBACK_TIMEOUT
* INTERNET_TIMEOUT
* SHORT_TIMEOUT
* LONG_TIMEOUT
2019-10-30 12:41:43 +01:00
Serhiy Storchaka 865c3b257f
bpo-28029: Make "".replace("", s, n) returning s for any n != 0. (GH-16981) 2019-10-30 12:03:53 +02:00
Daniel Hillier da6ce58dd5 bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656) 2019-10-29 09:24:18 +02:00
Raymond Hettinger 3c88199e0b bpo-38626: Add comment explaining why __lt__ is used. (GH-16978)
https://bugs.python.org/issue38626
2019-10-28 21:38:50 -07:00
Victor Stinner ae7aa42774
Remove code commented for more than 10 years (GH-16965)
test_urllib commented since 2007:

commit d9880d07fc
Author: Facundo Batista <facundobatista@gmail.com>
Date:   Fri May 25 04:20:22 2007 +0000

    Commenting out the tests until find out who can test them in
    one of the problematic enviroments.

pynche code commented since 1998 and 2001:

commit ef30092207
Author: Barry Warsaw <barry@python.org>
Date:   Tue Dec 15 01:04:38 1998 +0000

    Added most of the mechanism to change the strips from color variations
    to color constants (i.e. red constant, green constant, blue
    constant).  But I haven't hooked this up yet because the UI gets more
    crowded and the arrows don't reflect the correct values.

    Added "Go to Black" and "Go to White" buttons.

commit 741eae0b31
Author: Barry Warsaw <barry@python.org>
Date:   Wed Apr 18 03:51:55 2001 +0000

    StripWidget.__init__(), update_yourself(): Removed some unused local
    variables reported by PyChecker.

    __togglegentype(): PyChecker accurately reported that the variable
    __gentypevar was unused -- actually this whole method is currently
    unused so comment it out.
2019-10-28 22:35:31 +01:00
Victor Stinner e471e72977
bpo-37330: open() no longer accept 'U' in file mode (GH-16959)
open(), io.open(), codecs.open() and fileinput.FileInput no longer
accept "U" ("universal newline") in the file mode. This flag was
deprecated since Python 3.3.
2019-10-28 15:40:08 +01:00
Serhiy Storchaka 5c32af7522
bpo-38334: Fix seeking backward on an encrypted zipfile.ZipExtFile. (GH-16937)
Test by Daniel Hillier.
2019-10-27 10:22:14 +02:00
Terry Jan Reedy a8fb9327fb
bpo-37309: First idlelib/NEWS.txt for 3.9.0 (GH-16947) 2019-10-27 01:23:30 -04:00
Terry Jan Reedy e31a79a5b4
bpo-34162: Last idlelib/NEWS.txt items for 3.8.0. (GH-16943) 2019-10-26 22:19:57 -04:00
Terry Jan Reedy e3f90b217a
bpo-38598: Do not try to compile IDLE shell or output windows (GH-16939) 2019-10-26 21:15:10 -04:00
Serhiy Storchaka 26ae9f6d3d
bpo-38535: Fix positions for AST nodes for calls without arguments in decorators. (GH-16861) 2019-10-26 16:46:05 +03:00
Zsolt Dollenstein 96b06aefe2 bpo-33348: parse expressions after * and ** in lib2to3 (GH-6586)
These are valid even in python 2.7


https://bugs.python.org/issue33348



Automerge-Triggered-By: @gpshead
2019-10-23 23:19:07 -07:00
Girts a01ba333af bpo-30618: add readlink to pathlib.Path (GH-8285)
This adds a "readlink" method to pathlib.Path objects that calls through
to os.readlink.


https://bugs.python.org/issue30618



Automerge-Triggered-By: @gpshead
2019-10-23 14:18:40 -07:00
Victor Stinner 1b53a24fb4
bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901)
bpo-34679, bpo-38563: asyncio.ProactorEventLoop.close() now only calls
signal.set_wakeup_fd() in the main thread.
2019-10-23 17:25:29 +02:00
Serhiy Storchaka 10ecbadb79
bpo-31202: Preserve case of literal parts in Path.glob() on Windows. (GH-16860) 2019-10-21 20:37:15 +03:00