Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
### Notes
- Please treat as a security fix related to CVE-2023-52425.
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Prior to gh-114269, the iterator returned by ElementTree.iterparse was
initialized with the root attribute as None. This restores the previous
behavior.
The iterator returned by ElementTree.iterparse() may hold on to a file
descriptor. The reference cycle prevented prompt clean-up of the file
descriptor if the returned iterator was not exhausted.
When testing element truth values, emit a DeprecationWarning in all implementations.
This had emitted a FutureWarning in the rarely used python-only implementation since ~2.7 and has always been documented as a behavior not to rely on.
Matching an element in a tree search but having it test False can be unexpected. Raising the warning enables making the choice to finally raise an exception for this ambiguous behavior in the future.
The API documentation for [findtext](https://docs.python.org/3/library/xml.etree.elementtree.html#xml.etree.ElementTree.Element.findtext) states that this function gives back an empty string on "no text content." With the previous implementation, this would give back a empty string even on text content values such as 0 or False. This patch attempts to resolve that by only giving back an empty string if the text attribute is set to `None`. Resolves#91447.
Automerge-Triggered-By: GH:gvanrossum
xml.etree: Remove the ElementTree.Element.copy() method of the pure
Python implementation, deprecated in Python 3.10, use the copy.copy()
function instead. The C implementation of xml.etree has no copy()
method, only a __copy__() method.
Suppress writing an XML declaration in open files in ElementTree.write()
with encoding='unicode' and xml_declaration=None.
If file patch is passed to ElementTree.write() with encoding='unicode',
always open a new file in UTF-8.
ElementTree method write() and function tostring() now use the text file's
encoding ("UTF-8" if not available) instead of locale encoding in XML
declaration when encoding="unicode" is specified.
Remove deprecated __getitem__ methods of xml.dom.pulldom.DOMEventStream,
wsgiref.util.FileWrapper and fileinput.FileInput, deprecated since Python 3.9.
* bpo-39011: Preserve line endings within attributes
Line endings within attributes were previously normalized to "\n" in Py3.7/3.8.
This patch removes that normalization, as line endings which were
replaced by entity numbers should be preserved in original form.
* bpo-20928: bring elementtree's XInclude support en-par with the implementation in lxml by adding support for recursive includes and a base-URL.
* bpo-20928: Support xincluding the same file multiple times, just not recursively.
* bpo-20928: Add 'max_depth' parameter to xinclude that limits the maximum recursion depth to 6 by default.
* Add news entry for updated ElementInclude support
Capturing exceptions into names can lead to reference cycles though the __traceback__ attribute of the exceptions in some obscure cases that have been reported previously and fixed individually. As these variables are not used anyway, we can remove the binding to reduce the chances of creating reference cycles.
See for example GH-13135
* Implement C14N 2.0 as a new canonicalize() function in ElementTree.
Missing features:
- prefix renaming in XPath expressions (tag and attribute text is supported)
- preservation of original prefixes given redundant namespace declarations
* bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree.
* bpo-36673: Rewrite the comment/PI factory handling for the TreeBuilder in "_elementtree" to make it use the same factories as the ElementTree module, and to make it explicit when the comments/PIs are inserted into the tree and when they are not (which is the default).
Add new keyword arguments "default_namespace" and "xml_declaration" to functions ET.tostring() and ET.tostringlist(), as known from ElementTree.write().
Fix xml.dom.minidom cloneNode() on a document with an entity: pass
the correct arguments to the user data handler of an entity (fix an
old copy/paste mistake).
Bug spotted and fix proposed by Charalampos Stratakis, initial
reproducer written by Petr Viktorin.
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
Mikhail B
Sebastian Pipping
Serhiy Storchaka
Sam Gross
Jacob Walls
Nikita Sobolev
Nick Drozd
Kevin Kirsche
Eugene Triguba
Victor Stinner
Jannis Vajen
Hugo van Kemenade
Noah Kantrowitz
E-Paine
Alex Prengère
Felix C. Stegerman
Ammar Askar
scoder
Zackery Spytz
Alex Itkes
mefistotelis
Henry Harutyunyan
Pablo Galindo
Gordon P. Hemsley
Mickaël Schoentgen
Bernt Røskar Brenna
Vladimir Surjaninov
Victor Stinner
Diego Rojas