gh-61460: Add a comment describing the multiprocessing.connection protocol (gh-99623)

Describe the multiprocessing connection protocol.

It isn't a good protocol, but it is what it is.  This way we can more
easily reason about making changes to it in a backwards compatible way.
This commit is contained in:
Gregory P. Smith 2022-11-20 10:20:04 -08:00 committed by GitHub
parent 9c4232ae89
commit abf5b6ff43
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 68 additions and 0 deletions

View File

@ -728,6 +728,74 @@ CHALLENGE = b'#CHALLENGE#'
WELCOME = b'#WELCOME#'
FAILURE = b'#FAILURE#'
# multiprocessing.connection Authentication Handshake Protocol Description
# (as documented for reference after reading the existing code)
# =============================================================================
#
# On Windows: native pipes with "overlapped IO" are used to send the bytes,
# instead of the length prefix SIZE scheme described below. (ie: the OS deals
# with message sizes for us)
#
# Protocol error behaviors:
#
# On POSIX, any failure to receive the length prefix into SIZE, for SIZE greater
# than the requested maxsize to receive, or receiving fewer than SIZE bytes
# results in the connection being closed and auth to fail.
#
# On Windows, receiving too few bytes is never a low level _recv_bytes read
# error, receiving too many will trigger an error only if receive maxsize
# value was larger than 128 OR the if the data arrived in smaller pieces.
#
# Serving side Client side
# ------------------------------ ---------------------------------------
# 0. Open a connection on the pipe.
# 1. Accept connection.
# 2. New random 20 bytes -> MESSAGE
# 3. send 4 byte length (net order)
# prefix followed by:
# b'#CHALLENGE#' + MESSAGE
# 4. Receive 4 bytes, parse as network byte
# order integer. If it is -1, receive an
# additional 8 bytes, parse that as network
# byte order. The result is the length of
# the data that follows -> SIZE.
# 5. Receive min(SIZE, 256) bytes -> M1
# 6. Assert that M1 starts with:
# b'#CHALLENGE#'
# 7. Strip that prefix from M1 into -> M2
# 8. Compute HMAC-MD5 of AUTHKEY, M2 -> C_DIGEST
# 9. Send 4 byte length prefix (net order)
# followed by C_DIGEST bytes.
# 10. Compute HMAC-MD5 of AUTHKEY,
# MESSAGE into -> M_DIGEST.
# 11. Receive 4 or 4+8 byte length
# prefix (#4 dance) -> SIZE.
# 12. Receive min(SIZE, 256) -> C_D.
# 13. Compare M_DIGEST == C_D:
# 14a: Match? Send length prefix &
# b'#WELCOME#'
# <- RETURN
# 14b: Mismatch? Send len prefix &
# b'#FAILURE#'
# <- CLOSE & AuthenticationError
# 15. Receive 4 or 4+8 byte length prefix (net
# order) again as in #4 into -> SIZE.
# 16. Receive min(SIZE, 256) bytes -> M3.
# 17. Compare M3 == b'#WELCOME#':
# 17a. Match? <- RETURN
# 17b. Mismatch? <- CLOSE & AuthenticationError
#
# If this RETURNed, the connection remains open: it has been authenticated.
#
# Length prefixes are used consistently even though every step so far has
# always been a singular specific fixed length. This may help us evolve
# the protocol in the future without breaking backwards compatibility.
#
# Similarly the initial challenge message from the serving side has always
# been 20 bytes, but clients can accept a 100+ so using the length of the
# opening challenge message as an indicator of protocol version may work.
def deliver_challenge(connection, authkey):
import hmac
if not isinstance(authkey, bytes):