[3.13] GH-119496: accept UTF-8 BOM in .pth files (GH-119508)

`Out-File -Encoding utf8` and similar commands in Windows Powershell 5.1 emit
UTF-8 with a BOM marker, which the regular `utf-8` codec decodes incorrectly.

`utf-8-sig` accepts a BOM, but also works correctly without one.

This change also makes .pth files match the way Python source files are handled.

(cherry picked from commit bf5b6467f8)

Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
This commit is contained in:
Miss Islington (bot) 2024-05-24 16:52:20 +02:00 committed by GitHub
parent 0bab0b3a53
commit 217d57fc3c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions

View File

@ -185,7 +185,9 @@ def addpackage(sitedir, name, known_paths):
return
try:
pth_content = pth_content.decode()
# Accept BOM markers in .pth files as we do in source files
# (Windows PowerShell 5.1 makes it hard to emit UTF-8 files without a BOM)
pth_content = pth_content.decode("utf-8-sig")
except UnicodeDecodeError:
# Fallback to locale encoding for backward compatibility.
# We will deprecate this fallback in the future.