Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module 

to the urlparse one.  Added a PendingDeprecationWarning in the old
module, it will be deprecated in the future.  Docs and tests updated.
This commit is contained in:
Facundo Batista 2008-09-03 22:35:50 +00:00
parent 69acb43327
commit c585df9476
8 changed files with 177 additions and 128 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

43
Doc/library/cgi.rst
View File

@ -282,49 +282,18 @@ algorithms implemented in this module in other circumstances.
Parse a query in the environment or from a file (the file defaults to
``sys.stdin``). The *keep_blank_values* and *strict_parsing* parameters are
passed to :func:`parse_qs` unchanged.
passed to :func:`urlparse.parse_qs` unchanged.
.. function:: parse_qs(qs[, keep_blank_values[, strict_parsing]])
Parse a query string given as a string argument (data of type
:mimetype:`application/x-www-form-urlencoded`). Data are returned as a
dictionary. The dictionary keys are the unique query variable names and the
values are lists of values for each name.
The optional argument *keep_blank_values* is a flag indicating whether blank
values in URL encoded queries should be treated as blank strings. A true value
indicates that blanks should be retained as blank strings. The default false
value indicates that blank values are to be ignored and treated as if they were
not included.
The optional argument *strict_parsing* is a flag indicating what to do with
parsing errors. If false (the default), errors are silently ignored. If true,
errors raise a :exc:`ValueError` exception.
Use the :func:`urllib.urlencode` function to convert such dictionaries into
query strings.
This function is deprecated in this module. Use :func:`urlparse.parse_qs`
instead. It is maintained here only for backward compatiblity.
.. function:: parse_qsl(qs[, keep_blank_values[, strict_parsing]])
Parse a query string given as a string argument (data of type
:mimetype:`application/x-www-form-urlencoded`). Data are returned as a list of
name, value pairs.
The optional argument *keep_blank_values* is a flag indicating whether blank
values in URL encoded queries should be treated as blank strings. A true value
indicates that blanks should be retained as blank strings. The default false
value indicates that blank values are to be ignored and treated as if they were
not included.
The optional argument *strict_parsing* is a flag indicating what to do with
parsing errors. If false (the default), errors are silently ignored. If true,
errors raise a :exc:`ValueError` exception.
Use the :func:`urllib.urlencode` function to convert such lists of pairs into
query strings.
This function is deprecated in this module. Use :func:`urlparse.parse_qsl`
instead. It is maintained here only for backward compatiblity.
.. function:: parse_multipart(fp, pdict)
@ -332,7 +301,7 @@ algorithms implemented in this module in other circumstances.
Arguments are *fp* for the input file and *pdict* for a dictionary containing
other parameters in the :mailheader:`Content-Type` header.
Returns a dictionary just like :func:`parse_qs` keys are the field names, each
Returns a dictionary just like :func:`urlparse.parse_qs` keys are the field names, each
value is a list of values for that field. This is easy to use but not much good
if you are expecting megabytes to be uploaded --- in that case, use the
:class:`FieldStorage` class instead which is much more flexible.

2
Doc/library/urllib.rst
View File

@ -242,7 +242,7 @@ Utility functions
of the sequence. When a sequence of two-element tuples is used as the *query*
argument, the first element of each tuple is a key and the second is a value.
The order of parameters in the encoded string will match the order of parameter
tuples in the sequence. The :mod:`cgi` module provides the functions
tuples in the sequence. The :mod:`urlparse` module provides the functions
:func:`parse_qs` and :func:`parse_qsl` which are used to parse query strings
into Python data structures.

39
Doc/library/urlparse.rst
View File

@ -101,6 +101,45 @@ The :mod:`urlparse` module defines the following functions:
.. versionchanged:: 2.5
Added attributes to return value.
.. function:: parse_qs(qs[, keep_blank_values[, strict_parsing]])
Parse a query string given as a string argument (data of type
:mimetype:`application/x-www-form-urlencoded`). Data are returned as a
dictionary. The dictionary keys are the unique query variable names and the
values are lists of values for each name.
The optional argument *keep_blank_values* is a flag indicating whether blank
values in URL encoded queries should be treated as blank strings. A true value
indicates that blanks should be retained as blank strings. The default false
value indicates that blank values are to be ignored and treated as if they were
not included.
The optional argument *strict_parsing* is a flag indicating what to do with
parsing errors. If false (the default), errors are silently ignored. If true,
errors raise a :exc:`ValueError` exception.
Use the :func:`urllib.urlencode` function to convert such dictionaries into
query strings.
.. function:: parse_qsl(qs[, keep_blank_values[, strict_parsing]])
Parse a query string given as a string argument (data of type
:mimetype:`application/x-www-form-urlencoded`). Data are returned as a list of
name, value pairs.
The optional argument *keep_blank_values* is a flag indicating whether blank
values in URL encoded queries should be treated as blank strings. A true value
indicates that blanks should be retained as blank strings. The default false
value indicates that blank values are to be ignored and treated as if they were
not included.
The optional argument *strict_parsing* is a flag indicating what to do with
parsing errors. If false (the default), errors are silently ignored. If true,
errors raise a :exc:`ValueError` exception.
Use the :func:`urllib.urlencode` function to convert such lists of pairs into
query strings.
.. function:: urlunparse(parts)

85
Lib/cgi.py
View File

@ -39,7 +39,9 @@ import sys
import os
import urllib
import UserDict
from warnings import filterwarnings, catch_warnings
import urlparse
from warnings import filterwarnings, catch_warnings, warn
with catch_warnings():
if sys.py3kwarning:
filterwarnings("ignore", ".*mimetools has been removed",
@ -173,72 +175,21 @@ def parse(fp=None, environ=os.environ, keep_blank_values=0, strict_parsing=0):
return parse_qs(qs, keep_blank_values, strict_parsing)
# parse query string function called from urlparse,
# this is done in order to maintain backward compatiblity.
def parse_qs(qs, keep_blank_values=0, strict_parsing=0):
"""Parse a query given as a string argument.
"""Parse a query given as a string argument."""
warn("cgi.parse_qs is deprecated, use urlparse.parse_qs \
instead",PendingDeprecationWarning)
return urlparse.parse_qs(qs, keep_blank_values, strict_parsing)
Arguments:
qs: URL-encoded query string to be parsed
keep_blank_values: flag indicating whether blank values in
URL encoded queries should be treated as blank strings.
A true value indicates that blanks should be retained as
blank strings. The default false value indicates that
blank values are to be ignored and treated as if they were
not included.
strict_parsing: flag indicating what to do with parsing errors.
If false (the default), errors are silently ignored.
If true, errors raise a ValueError exception.
"""
dict = {}
for name, value in parse_qsl(qs, keep_blank_values, strict_parsing):
if name in dict:
dict[name].append(value)
else:
dict[name] = [value]
return dict
def parse_qsl(qs, keep_blank_values=0, strict_parsing=0):
"""Parse a query given as a string argument.
Arguments:
qs: URL-encoded query string to be parsed
keep_blank_values: flag indicating whether blank values in
URL encoded queries should be treated as blank strings. A
true value indicates that blanks should be retained as blank
strings. The default false value indicates that blank values
are to be ignored and treated as if they were not included.
strict_parsing: flag indicating what to do with parsing errors. If
false (the default), errors are silently ignored. If true,
errors raise a ValueError exception.
Returns a list, as G-d intended.
"""
pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')]
r = []
for name_value in pairs:
if not name_value and not strict_parsing:
continue
nv = name_value.split('=', 1)
if len(nv) != 2:
if strict_parsing:
raise ValueError, "bad query field: %r" % (name_value,)
# Handle case of a control-name with no equal sign
if keep_blank_values:
nv.append('')
else:
continue
if len(nv[1]) or keep_blank_values:
name = urllib.unquote(nv[0].replace('+', ' '))
value = urllib.unquote(nv[1].replace('+', ' '))
r.append((name, value))
return r
"""Parse a query given as a string argument."""
warn("cgi.parse_qsl is deprecated, use urlparse.parse_qsl instead",
PendingDeprecationWarning)
return urlparse.parse_qs(qs, keep_blank_values, strict_parsing)
def parse_multipart(fp, pdict):
"""Parse multipart input.
@ -645,8 +596,8 @@ class FieldStorage:
if self.qs_on_post:
qs += '&' + self.qs_on_post
self.list = list = []
for key, value in parse_qsl(qs, self.keep_blank_values,
self.strict_parsing):
for key, value in urlparse.parse_qsl(qs, self.keep_blank_values,
self.strict_parsing):
list.append(MiniFieldStorage(key, value))
self.skip_lines()
@ -659,8 +610,8 @@ class FieldStorage:
raise ValueError, 'Invalid boundary in multipart form: %r' % (ib,)
self.list = []
if self.qs_on_post:
for key, value in parse_qsl(self.qs_on_post, self.keep_blank_values,
self.strict_parsing):
for key, value in urlparse.parse_qsl(self.qs_on_post,
self.keep_blank_values, self.strict_parsing):
self.list.append(MiniFieldStorage(key, value))
FieldStorageClass = None

22
Lib/test/test_cgi.py
View File

@ -55,23 +55,6 @@ def do_test(buf, method):
except StandardError, err:
return ComparableException(err)
# A list of test cases. Each test case is a a two-tuple that contains
# a string with the query and a dictionary with the expected result.
parse_qsl_test_cases = [
("", []),
("&", []),
("&&", []),
("=", [('', '')]),
("=a", [('', 'a')]),
("a", [('a', '')]),
("a=", [('a', '')]),
("a=", [('a', '')]),
("&a=b", [('a', 'b')]),
("a=a+b&b=b+c", [('a', 'a b'), ('b', 'b c')]),
("a=1&a=2", [('a', '1'), ('a', '2')]),
]
parse_strict_test_cases = [
("", ValueError("bad query field: ''")),
("&", ValueError("bad query field: ''")),
@ -143,11 +126,6 @@ def gen_result(data, environ):
class CgiTests(unittest.TestCase):
def test_qsl(self):
for orig, expect in parse_qsl_test_cases:
result = cgi.parse_qsl(orig, keep_blank_values=True)
self.assertEqual(result, expect, "Error parsing %s" % repr(orig))
def test_strict(self):
for orig, expect in parse_strict_test_cases:
# Test basic parsing

22
Lib/test/test_urlparse.py
View File

@ -8,6 +8,23 @@ RFC1808_BASE = "http://a/b/c/d;p?q#f"
RFC2396_BASE = "http://a/b/c/d;p?q"
RFC3986_BASE = "http://a/b/c/d;p?q"
# A list of test cases. Each test case is a a two-tuple that contains
# a string with the query and a dictionary with the expected result.
parse_qsl_test_cases = [
("", []),
("&", []),
("&&", []),
("=", [('', '')]),
("=a", [('', 'a')]),
("a", [('a', '')]),
("a=", [('a', '')]),
("a=", [('a', '')]),
("&a=b", [('a', 'b')]),
("a=a+b&b=b+c", [('a', 'a b'), ('b', 'b c')]),
("a=1&a=2", [('a', '1'), ('a', '2')]),
]
class UrlParseTestCase(unittest.TestCase):
def checkRoundtrips(self, url, parsed, split):
@ -61,6 +78,11 @@ class UrlParseTestCase(unittest.TestCase):
self.assertEqual(result3.hostname, result.hostname)
self.assertEqual(result3.port, result.port)
def test_qsl(self):
for orig, expect in parse_qsl_test_cases:
result = urlparse.parse_qsl(orig, keep_blank_values=True)
self.assertEqual(result, expect, "Error parsing %s" % repr(orig))
def test_roundtrips(self):
testcases = [
('file:///tmp/junk.txt',

88
Lib/urlparse.py
View File

@ -5,7 +5,7 @@ UC Irvine, June 1995.
"""
__all__ = ["urlparse", "urlunparse", "urljoin", "urldefrag",
"urlsplit", "urlunsplit"]
"urlsplit", "urlunsplit", "parse_qs", "parse_qsl"]
# A classification of schemes ('' means apply by default)
uses_relative = ['ftp', 'http', 'gopher', 'nntp', 'imap',
@ -267,6 +267,92 @@ def urldefrag(url):
else:
return url, ''
# unquote method for parse_qs and parse_qsl
# Cannot use directly from urllib as it would create circular reference.
# urllib uses urlparse methods ( urljoin)
_hextochr = dict(('%02x' % i, chr(i)) for i in range(256))
_hextochr.update(('%02X' % i, chr(i)) for i in range(256))
def unquote(s):
"""unquote('abc%20def') -> 'abc def'."""
res = s.split('%')
for i in xrange(1, len(res)):
item = res[i]
try:
res[i] = _hextochr[item[:2]] + item[2:]
except KeyError:
res[i] = '%' + item
except UnicodeDecodeError:
res[i] = unichr(int(item[:2], 16)) + item[2:]
return "".join(res)
def parse_qs(qs, keep_blank_values=0, strict_parsing=0):
"""Parse a query given as a string argument.
Arguments:
qs: URL-encoded query string to be parsed
keep_blank_values: flag indicating whether blank values in
URL encoded queries should be treated as blank strings.
A true value indicates that blanks should be retained as
blank strings. The default false value indicates that
blank values are to be ignored and treated as if they were
not included.
strict_parsing: flag indicating what to do with parsing errors.
If false (the default), errors are silently ignored.
If true, errors raise a ValueError exception.
"""
dict = {}
for name, value in parse_qsl(qs, keep_blank_values, strict_parsing):
if name in dict:
dict[name].append(value)
else:
dict[name] = [value]
return dict
def parse_qsl(qs, keep_blank_values=0, strict_parsing=0):
"""Parse a query given as a string argument.
Arguments:
qs: URL-encoded query string to be parsed
keep_blank_values: flag indicating whether blank values in
URL encoded queries should be treated as blank strings. A
true value indicates that blanks should be retained as blank
strings. The default false value indicates that blank values
are to be ignored and treated as if they were not included.
strict_parsing: flag indicating what to do with parsing errors. If
false (the default), errors are silently ignored. If true,
errors raise a ValueError exception.
Returns a list, as G-d intended.
"""
pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')]
r = []
for name_value in pairs:
if not name_value and not strict_parsing:
continue
nv = name_value.split('=', 1)
if len(nv) != 2:
if strict_parsing:
raise ValueError, "bad query field: %r" % (name_value,)
# Handle case of a control-name with no equal sign
if keep_blank_values:
nv.append('')
else:
continue
if len(nv[1]) or keep_blank_values:
name = unquote(nv[0].replace('+', ' '))
value = unquote(nv[1].replace('+', ' '))
r.append((name, value))
return r
test_input = """
http://a/b/c/d

4
Misc/NEWS
View File

@ -56,6 +56,10 @@ C-API
Library
-------
- Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
to the urlparse one. Added a PendingDeprecationWarning in the old
module, it will be deprecated in the future.
- Issue #2562: Fix distutils PKG-INFO writing logic to allow having
non-ascii characters and Unicode in setup.py meta-data.