gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (#120984)

* Add --app-store-compliance configuration option.

* Added blurb.

* Correct tab-vs-spaces formatting issue.

* Correct source file name in docs.

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

* Correct source code reference in Mac docs

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>

* Only apply the patch forward, and ensure the working directory is correct.

* Make patching reslient to multiple builds.

* Documentation fixes found during review

Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>

* Documentation and configure.ac syntax improvements

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Regenerate configure script.

* Silence the patch echo output.

---------

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Doc/library/urllib.parse.rst

@@ -22,11 +22,19 @@ to an absolute URL given a "base URL."
 
 The module has been designed to match the internet RFC on Relative Uniform
 Resource Locators. It supports the following URL schemes: ``file``, ``ftp``,
-``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``,
+``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``itms-services``, ``mailto``, ``mms``,
 ``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtsps``, ``rtspu``,
 ``sftp``, ``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``,
 ``telnet``, ``wais``, ``ws``, ``wss``.
 
+.. impl-detail::
+
+   The inclusion of the ``itms-services`` URL scheme can prevent an app from
+   passing Apple's App Store review process for the macOS and iOS App Stores.
+   Handling for the ``itms-services`` scheme is always removed on iOS; on
+   macOS, it *may* be removed if CPython has been built with the
+   :option:`--with-app-store-compliance` option.
+
 The :mod:`urllib.parse` module defines functions that fall into two broad
 categories: URL parsing and URL quoting. These are covered in detail in
 the following sections.

Doc/using/configure.rst

@@ -945,6 +945,17 @@ See :source:`Mac/README.rst`.
    Specify the name for the python framework on macOS only valid when
    :option:`--enable-framework` is set (default: ``Python``).
 
+.. option:: --with-app-store-compliance
+.. option:: --with-app-store-compliance=PATCH-FILE
+
+   The Python standard library contains strings that are known to trigger
+   automated inspection tool errors when submitted for distribution by
+   the macOS and iOS App Stores. If enabled, this option will apply the list of
+   patches that are known to correct app store compliance. A custom patch
+   file can also be specified. This option is disabled by default.
+
+   .. versionadded:: 3.13
+
 iOS Options
 -----------
 

Doc/using/ios.rst

@@ -312,3 +312,21 @@ modules in your app, some additional steps will be required:
 
 * If you're using a separate folder for third-party packages, ensure that folder
   is included as part of the ``PYTHONPATH`` configuration in step 10.
+
+App Store Compliance
+====================
+
+The only mechanism for distributing apps to third-party iOS devices is to
+submit the app to the iOS App Store; apps submitted for distribution must pass
+Apple's app review process. This process includes a set of automated validation
+rules that inspect the submitted application bundle for problematic code.
+
+The Python standard library contains some code that is known to violate these
+automated rules. While these violations appear to be false positives, Apple's
+review rules cannot be challenged; so, it is necessary to modify the Python
+standard library for an app to pass App Store review.
+
+The Python source tree contains
+:source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove
+all code that is known to cause issues with the App Store review process. This
+patch is applied automatically when building for iOS.

Doc/using/mac.rst

@@ -188,6 +188,28 @@ distributable application:
 * `PyInstaller <https://pyinstaller.org/>`__: A cross-platform packaging tool that creates
   a single file or folder as a distributable artifact.
 
+App Store Compliance
+--------------------
+
+Apps submitted for distribution through the macOS App Store must pass Apple's
+app review process. This process includes a set of automated validation rules
+that inspect the submitted application bundle for problematic code.
+
+The Python standard library contains some code that is known to violate these
+automated rules. While these violations appear to be false positives, Apple's
+review rules cannot be challenged. Therefore, it is necessary to modify the
+Python standard library for an app to pass App Store review.
+
+The Python source tree contains
+:source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove
+all code that is known to cause issues with the App Store review process. This
+patch is applied automatically when CPython is configured with the
+:option:`--with-app-store-compliance` option.
+
+This patch is not normally required to use CPython on a Mac; nor is it required
+if you are distributing an app *outside* the macOS App Store. It is *only*
+required if you are using the macOS App Store as a distribution channel.
+
 Other Resources
 ===============
 

Mac/Resources/app-store-compliance.patch

@@ -0,0 +1,29 @@
+diff --git a/Lib/test/test_urlparse.py b/Lib/test/test_urlparse.py
+index d6c83a75c1c..19ed4e01091 100644
+--- a/Lib/test/test_urlparse.py
++++ b/Lib/test/test_urlparse.py
+@@ -237,11 +237,6 @@ def test_roundtrips(self):
+               '','',''),
+              ('git+ssh', 'git@github.com','/user/project.git',
+               '', '')),
+-            ('itms-services://?action=download-manifest&url=https://example.com/app',
+-             ('itms-services', '', '', '',
+-              'action=download-manifest&url=https://example.com/app', ''),
+-             ('itms-services', '', '',
+-              'action=download-manifest&url=https://example.com/app', '')),
+             ('+scheme:path/to/file',
+              ('', '', '+scheme:path/to/file', '', '', ''),
+              ('', '', '+scheme:path/to/file', '', '')),
+diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py
+index 8f724f907d4..148caf742c9 100644
+--- a/Lib/urllib/parse.py
++++ b/Lib/urllib/parse.py
+@@ -59,7 +59,7 @@
+                'imap', 'wais', 'file', 'mms', 'https', 'shttp',
+                'snews', 'prospero', 'rtsp', 'rtsps', 'rtspu', 'rsync',
+                'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh',
+-               'ws', 'wss', 'itms-services']
++               'ws', 'wss']
+
+ uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap',
+                'https', 'shttp', 'rtsp', 'rtsps', 'rtspu', 'sip',

Makefile.pre.in

@@ -178,6 +178,9 @@ EXPORTSFROM=	@EXPORTSFROM@
 EXE=		@EXEEXT@
 BUILDEXE=	@BUILDEXEEXT@
 
+# Name of the patch file to apply for app store compliance
+APP_STORE_COMPLIANCE_PATCH=@APP_STORE_COMPLIANCE_PATCH@
+
 # Short name and location for Mac OS X Python framework
 UNIVERSALSDK=@UNIVERSALSDK@
 PYTHONFRAMEWORK=	@PYTHONFRAMEWORK@
@@ -691,7 +694,7 @@ list-targets:
 	@grep -E '^[A-Za-z][-A-Za-z0-9]+:' Makefile | awk -F : '{print $$1}'
 
 .PHONY: build_all
-build_all:	check-clean-src $(BUILDPYTHON) platform sharedmods \
+build_all:	check-clean-src @APP_STORE_COMPLIANCE_PATCH_TARGET@ $(BUILDPYTHON) platform sharedmods \
 		gdbhooks Programs/_testembed scripts checksharedmods rundsymutil
 
 .PHONY: build_wasm
@@ -927,6 +930,18 @@ SRC_GDB_HOOKS=$(srcdir)/Tools/gdb/libpython.py
 $(BUILDPYTHON)-gdb.py: $(SRC_GDB_HOOKS)
 	$(INSTALL_DATA) $(SRC_GDB_HOOKS) $(BUILDPYTHON)-gdb.py
 
+# Compliance with app stores (such as iOS and macOS) sometimes requires making
+# modifications to the Python standard library. If enabled, apply the patch of
+# known modifications to the source tree before building. The patch will be
+# applied in a dry-run mode (validating, but not applying the patch) on builds
+# that *have* a compliance patch, but where compliance has not been enabled.
+build/app-store-compliant:
+	patch @APP_STORE_COMPLIANCE_PATCH_FLAGS@ --forward --strip=1 --directory="$(srcdir)" --input "$(APP_STORE_COMPLIANCE_PATCH)"
+	@if test "@APP_STORE_COMPLIANCE_PATCH_FLAGS@" == ""; then \
+		mkdir -p build ; \
+		echo "$(APP_STORE_COMPLIANCE_PATCH)" > build/app-store-compliant ; \
+	fi
+
 # This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary
 # minimal framework (not including the Lib directory and such) in the current
 # directory.

Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst

@@ -0,0 +1,2 @@
+Added a :option:`--with-app-store-compliance` option to patch out known issues
+with macOS/iOS App Store review processes.

configure

@@ -981,6 +981,9 @@ IPHONEOS_DEPLOYMENT_TARGET
 EXPORT_MACOSX_DEPLOYMENT_TARGET
 CONFIGURE_MACOSX_DEPLOYMENT_TARGET
 _PYTHON_HOST_PLATFORM
+APP_STORE_COMPLIANCE_PATCH_FLAGS
+APP_STORE_COMPLIANCE_PATCH_TARGET
+APP_STORE_COMPLIANCE_PATCH
 INSTALLTARGETS
 FRAMEWORKINSTALLAPPSPREFIX
 FRAMEWORKUNIXTOOLSPREFIX
@@ -1076,6 +1079,7 @@ enable_universalsdk
 with_universal_archs
 with_framework_name
 enable_framework
+with_app_store_compliance
 with_emscripten_target
 enable_wasm_dynamic_linking
 enable_wasm_pthreads
@@ -1855,6 +1859,10 @@ Optional Packages:
                           specify the name for the python framework on macOS
                           only valid when --enable-framework is set. see
                           Mac/README.rst (default is 'Python')
+  --with-app-store-compliance=[PATCH-FILE]
+                          Enable any patches required for compiliance with app
+                          stores. Optional PATCH-FILE specifies the custom
+                          patch to apply.
   --with-emscripten-target=[browser|node]
                           Emscripten platform
   --with-suffix=SUFFIX    set executable suffix to SUFFIX (default is empty,
@@ -4430,6 +4438,71 @@ fi
 printf "%s\n" "#define _PYTHONFRAMEWORK \"${PYTHONFRAMEWORK}\"" >>confdefs.h
 
 
+{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --with-app-store-compliance" >&5
+printf %s "checking for --with-app-store-compliance... " >&6; }
+
+# Check whether --with-app_store_compliance was given.
+if test ${with_app_store_compliance+y}
+then :
+  withval=$with_app_store_compliance;
+    case "$withval" in
+    yes)
+      case $ac_sys_system in
+        Darwin|iOS)
+          # iOS is able to share the macOS patch
+          APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+          APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+          APP_STORE_COMPLIANCE_PATCH_FLAGS=
+          ;;
+        *) as_fn_error $? "no default app store compliance patch available for $ac_sys_system" "$LINENO" 5 ;;
+      esac
+      { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying default app store compliance patch" >&5
+printf "%s\n" "applying default app store compliance patch" >&6; }
+      ;;
+    *)
+      APP_STORE_COMPLIANCE_PATCH="${withval}"
+      APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+      APP_STORE_COMPLIANCE_PATCH_FLAGS=
+      { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying custom app store compliance patch" >&5
+printf "%s\n" "applying custom app store compliance patch" >&6; }
+      ;;
+    esac
+
+else $as_nop
+
+    case $ac_sys_system in
+      iOS)
+        # Always apply the compliance patch on iOS; we can use the macOS patch
+        APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+        APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+        APP_STORE_COMPLIANCE_PATCH_FLAGS=
+        { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying default app store compliance patch" >&5
+printf "%s\n" "applying default app store compliance patch" >&6; }
+        ;;
+      Darwin)
+        # Always *check* the compliance patch on macOS
+        APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+        APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+        APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run"
+        { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: checking (not applying) default app store compliance patch" >&5
+printf "%s\n" "checking (not applying) default app store compliance patch" >&6; }
+        ;;
+      *)
+        # No app compliance patching on any other platform
+        APP_STORE_COMPLIANCE_PATCH=
+        APP_STORE_COMPLIANCE_PATCH_TARGET=
+        APP_STORE_COMPLIANCE_PATCH_FLAGS=
+        { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: not patching for app store compliance" >&5
+printf "%s\n" "not patching for app store compliance" >&6; }
+        ;;
+    esac
+
+fi
+
+
+
+
+
 
 if test "$cross_compiling" = yes; then
 	case "$host" in

configure.ac

@@ -695,6 +695,64 @@ AC_SUBST([INSTALLTARGETS])
 AC_DEFINE_UNQUOTED([_PYTHONFRAMEWORK], ["${PYTHONFRAMEWORK}"],
                    [framework name])
 
+dnl quadrigraphs "@<:@" and "@:>@" produce "[" and "]" in the output
+AC_MSG_CHECKING([for --with-app-store-compliance])
+AC_ARG_WITH(
+  [app_store_compliance],
+  [AS_HELP_STRING(
+    [--with-app-store-compliance=@<:@PATCH-FILE@:>@],
+    [Enable any patches required for compiliance with app stores.
+     Optional PATCH-FILE specifies the custom patch to apply.]
+  )],[
+    case "$withval" in
+    yes)
+      case $ac_sys_system in
+        Darwin|iOS)
+          # iOS is able to share the macOS patch
+          APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+          APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+          APP_STORE_COMPLIANCE_PATCH_FLAGS=
+          ;;
+        *) AC_MSG_ERROR([no default app store compliance patch available for $ac_sys_system]) ;;
+      esac
+      AC_MSG_RESULT([applying default app store compliance patch])
+      ;;
+    *)
+      APP_STORE_COMPLIANCE_PATCH="${withval}"
+      APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+      APP_STORE_COMPLIANCE_PATCH_FLAGS=
+      AC_MSG_RESULT([applying custom app store compliance patch])
+      ;;
+    esac
+  ],[
+    case $ac_sys_system in
+      iOS)
+        # Always apply the compliance patch on iOS; we can use the macOS patch
+        APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+        APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+        APP_STORE_COMPLIANCE_PATCH_FLAGS=
+        AC_MSG_RESULT([applying default app store compliance patch])
+        ;;
+      Darwin)
+        # Always *check* the compliance patch on macOS
+        APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch"
+        APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant"
+        APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run"
+        AC_MSG_RESULT([checking (not applying) default app store compliance patch])
+        ;;
+      *)
+        # No app compliance patching on any other platform
+        APP_STORE_COMPLIANCE_PATCH=
+        APP_STORE_COMPLIANCE_PATCH_TARGET=
+        APP_STORE_COMPLIANCE_PATCH_FLAGS=
+        AC_MSG_RESULT([not patching for app store compliance])
+        ;;
+    esac
+])
+AC_SUBST([APP_STORE_COMPLIANCE_PATCH])
+AC_SUBST([APP_STORE_COMPLIANCE_PATCH_TARGET])
+AC_SUBST([APP_STORE_COMPLIANCE_PATCH_FLAGS])
+
 AC_SUBST([_PYTHON_HOST_PLATFORM])
 if test "$cross_compiling" = yes; then
 	case "$host" in