cpython/Lib
Nir Soffer ad455cd924 bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)
blocksize was hardcoded to 8192, preventing efficient upload when using
file-like body. Add blocksize argument to __init__, so users can
configure the blocksize to fit their needs.

I tested this uploading data from /dev/zero to a web server dropping the
received data, to test the overhead of the HTTPConnection.send() with a
file-like object.

Here is an example 10g upload with the default buffer size (8192):

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 17.53 seconds (584.00m/s)

real	0m17.574s
user	0m8.887s
sys	0m5.971s

Same with 512k blocksize:

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 6.60 seconds (1551.15m/s)

real	0m6.641s
user	0m3.426s
sys	0m2.162s

In real world usage the difference will be smaller, depending on the
local and remote storage and the network.

See https://github.com/nirs/http-bench for more info.
2017-11-06 13:16:37 -08:00
..
asyncio bpo-31245: Asyncio unix socket datagram (#3164) 2017-10-30 09:43:02 -04:00
collections bpo-28638: Optimize namedtuple() creation time by minimizing use of exec() (#3454) 2017-09-10 10:23:36 -07:00
concurrent bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor (#4241) 2017-11-04 11:05:49 +01:00
ctypes time.clock() now emits a DeprecationWarning (GH-4020) 2017-10-17 14:46:45 -07:00
curses
dbm
distutils fixes bpo-31866: remove code pertaining to AtheOS support (#4115) 2017-10-25 23:55:14 -07:00
email bpo-31677: email: Remove re.IGNORECASE flag (GH-3868) 2017-10-04 12:47:38 +09:00
encodings
ensurepip bpo-31351: Set return code in ensurepip when pip fails (GH-3626) 2017-09-21 20:07:45 +10:00
html
http bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279) 2017-11-06 13:16:37 -08:00
idlelib Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
importlib bpo-21720: Restore the Python 2.7 logic in handling a fromlist. (#4118) 2017-10-26 10:41:59 +03:00
json Fix trivial typo in json module docstring (GH-2274) 2017-06-20 13:07:20 -07:00
lib2to3 bpo-30406: Make async and await proper keywords (#1669) 2017-10-05 23:24:46 -04:00
logging bpo-30989: Sort in TimedRotatingFileHandler only when needed. (GH-2812) 2017-10-27 08:04:33 +01:00
msilib Remove all mention of Windows IA-64 support (GH-3389) 2017-09-06 15:45:25 -07:00
multiprocessing bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed (#3247) 2017-11-03 14:31:38 +01:00
pydoc_data Update NEWS and topics for 3.7.0a2 2017-10-16 23:41:55 -04:00
site-packages
sqlite3 bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
test bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279) 2017-11-06 13:16:37 -08:00
tkinter bpo-27319, bpo-31508: Document deprecation in Treeview.selection(). (#3667) 2017-09-24 14:34:09 +03:00
turtledemo time.clock() now emits a DeprecationWarning (GH-4020) 2017-10-17 14:46:45 -07:00
unittest Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
urllib remove a redundant lower in urllib.parse.urlsplit (#3008) 2017-09-02 21:51:39 -07:00
venv bpo-25351: avoid activate failure on strict shells (GH-3804) 2017-09-29 12:48:11 +01:00
wsgiref Remove tiny bit of deadcode from wsgiref.validate (#3996) 2017-10-31 00:07:36 -04:00
xml bpo-31776: Missing "raise from None" in Lib/xml/etree/ElementPath.py (#3978) 2017-10-16 11:24:22 +03:00
xmlrpc bpo-31247: xmlrpc.server: break reference cycle (#3166) 2017-08-21 18:12:58 +02:00
__future__.py
__phello__.foo.py
_bootlocale.py
_collections_abc.py
_compat_pickle.py
_compression.py
_dummy_thread.py Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) (#3648) 2017-09-18 22:04:20 +02:00
_markupbase.py
_osx_support.py
_pydecimal.py Trivial cleanups following bpo-31370 (#3649) 2017-09-18 23:50:44 +02:00
_pyio.py bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#3372) 2017-09-22 10:17:30 -07:00
_sitebuiltins.py
_strptime.py Closes bpo-31800: Support for colon when parsing time offsets (#4015) 2017-10-25 20:35:41 -04:00
_threading_local.py
_weakrefset.py
abc.py
aifc.py
antigravity.py
argparse.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
ast.py
asynchat.py
asyncore.py bpo-29639: change test.support.HOST to "localhost" 2017-09-09 00:30:15 -07:00
base64.py
bdb.py
binhex.py
bisect.py
bz2.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
cProfile.py
calendar.py Closes bpo-28281: Remove year (1-9999) limits on the weekday() function. (#4109) 2017-10-26 15:34:11 -04:00
cgi.py
cgitb.py
chunk.py
cmd.py
code.py
codecs.py
codeop.py
colorsys.py
compileall.py
configparser.py bpo-31307: Make ConfigParser.read() accept bytes objects (GH-3420) 2017-11-02 15:47:04 +03:00
contextlib.py
copy.py
copyreg.py bpo-31107: Fix copyreg mangled slot names calculation. (#2989) 2017-08-04 11:45:00 +03:00
crypt.py bpo-31664: Add support for the Blowfish method in crypt. (#3854) 2017-10-24 19:36:17 +03:00
csv.py
datetime.py Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets. (#2896) 2017-07-31 10:26:50 -04:00
decimal.py
difflib.py
dis.py bpo-31183: `dis` now handles coroutines & async generators (GH-3077) 2017-08-18 12:29:21 +10:00
doctest.py
dummy_threading.py Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) (#3648) 2017-09-18 22:04:20 +02:00
enum.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
filecmp.py
fileinput.py bpo-31281: Fix pathlib.Path incompatibility in fileinput (gh-3208) 2017-09-04 13:37:24 -04:00
fnmatch.py
formatter.py
fractions.py
ftplib.py bpo-31346: Use PROTOCOL_TLS_CLIENT/SERVER (#3058) 2017-09-15 20:27:30 +02:00
functools.py bpo-31581: Reduce the number of imports for functools (GH-3757) 2017-09-30 16:13:02 +09:00
genericpath.py
getopt.py
getpass.py
gettext.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
glob.py
gzip.py
hashlib.py Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
heapq.py Improve clarity (and small speed-up) by using tuple unpacking (#3289) 2017-09-04 11:47:58 -07:00
hmac.py
imaplib.py
imghdr.py
imp.py bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) 2017-06-23 10:35:03 -07:00
inspect.py bpo-30639: Lazily compute repr for error (#2132) 2017-10-24 08:42:36 -04:00
io.py
ipaddress.py bpo-20825: Containment test for ip_network in ip_network. 2017-10-22 23:39:49 +02:00
keyword.py bpo-30406: Make async and await proper keywords (#1669) 2017-10-05 23:24:46 -04:00
linecache.py
locale.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
lzma.py
macpath.py
mailbox.py
mailcap.py
mimetypes.py bpo-30824: Add mimetype for .json (#3048) 2017-08-14 14:39:45 -04:00
modulefinder.py
netrc.py bpo-30806 netrc.__repr__() is broken for writing to file (GH-2491) 2017-09-30 16:10:31 +09:00
nntplib.py
ntpath.py
nturl2path.py
numbers.py Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
opcode.py
operator.py
optparse.py
os.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
pathlib.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
pdb.py bpo-31389 Add an optional `header` argument to pdb.set_trace() (#3438) 2017-09-22 12:29:42 -04:00
pickle.py
pickletools.py
pipes.py
pkgutil.py
platform.py
plistlib.py bpo-31897: Convert unexpected errors when read bogus binary plists into InvalidFileException. (#4171) 2017-10-31 14:05:53 +02:00
poplib.py
posixpath.py
pprint.py
profile.py time.clock() now emits a DeprecationWarning (GH-4020) 2017-10-17 14:46:45 -07:00
pstats.py
pty.py
py_compile.py
pyclbr.py bpo-6691: Pyclbr now reports nested classes and functions. (#2503) 2017-07-03 21:31:25 -04:00
pydoc.py bpo-30406: Make async and await proper keywords (#1669) 2017-10-05 23:24:46 -04:00
queue.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
quopri.py
random.py bpo-31482: Missing bytes support for random.seed() version 1 (#3614) 2017-09-17 09:04:30 -07:00
re.py bpo-31671: re: Convert RegexFlag to int before compile (GH-3862) 2017-10-05 17:19:26 +09:00
reprlib.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
rlcompleter.py
runpy.py
sched.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
secrets.py
selectors.py
shelve.py
shlex.py
shutil.py bpo-28564: Use os.scandir() in shutil.rmtree(). (#4085) 2017-11-04 14:16:35 +02:00
signal.py
site.py bpo-30804: fix macOS build with framework enabled. (#2516) 2017-06-30 19:38:02 +02:00
smtpd.py
smtplib.py
sndhdr.py
socket.py bpo-31234, socket.create_connection(): Fix ref cycle (#3546) 2017-09-13 10:10:10 -07:00
socketserver.py bpo-31233: socketserver.ThreadingMixIn.server_close() (#3523) 2017-09-13 01:47:22 -07:00
sre_compile.py bpo-31690: Allow the inline flags "a", "L", and "u" to be used as group flags for RE. (#3885) 2017-10-24 23:31:42 +03:00
sre_constants.py bpo-31690: Allow the inline flags "a", "L", and "u" to be used as group flags for RE. (#3885) 2017-10-24 23:31:42 +03:00
sre_parse.py bpo-31690: Allow the inline flags "a", "L", and "u" to be used as group flags for RE. (#3885) 2017-10-24 23:31:42 +03:00
ssl.py bpo-31659: Use simple slicing to format PEM cert (GH-3849) 2017-10-02 16:33:42 +09:00
stat.py
statistics.py
string.py bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872) 2017-10-13 16:02:23 +09:00
stringprep.py
struct.py
subprocess.py Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
sunau.py
symbol.py bpo-30406: Make async and await proper keywords (#1669) 2017-10-05 23:24:46 -04:00
symtable.py
sysconfig.py remove support for BSD/OS (closes bpo-31624) (#3812) 2017-09-28 22:44:27 -07:00
tabnanny.py
tarfile.py
telnetlib.py bpo-30397: Add re.Pattern and re.Match. (#1646) 2017-10-04 20:09:49 +03:00
tempfile.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
textwrap.py
this.py
threading.py bpo-31516: current_thread() should not return a dummy thread at shutdown (#3673) 2017-10-02 16:42:15 +02:00
timeit.py
token.py
tokenize.py bpo-30406: Make async and await proper keywords (#1669) 2017-10-05 23:24:46 -04:00
trace.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00
traceback.py bpo-28603: Fix formatting tracebacks for unhashable exceptions (#4014) 2017-10-18 00:29:39 +03:00
tracemalloc.py
tty.py
turtle.py Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
types.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
typing.py bpo-28556: typing.get_type_hints: better globalns for classes and modules (#3582) 2017-09-14 14:33:00 -04:00
uu.py
uuid.py bpo-9678: Fix determining the MAC address in the uuid module. (#4264) 2017-11-04 09:37:32 +02:00
warnings.py
wave.py
weakref.py bpo-30152: Reduce the number of imports for argparse. (#1269) 2017-09-26 00:55:55 +03:00
webbrowser.py
xdrlib.py
zipapp.py bpo-31638: Add compression support to zipapp (GH-3819) 2017-09-29 18:31:52 +01:00
zipfile.py bpo-31370: Remove support for threads-less builds (#3385) 2017-09-07 18:56:24 +02:00