Issue #20913: improve the SSL security considerations to first advocate using create_default_context().

Doc/library/asyncio-eventloop.rst

@@ -241,6 +241,8 @@ Creating connections
      the transport; if *ssl* is :const:`True`, a context with some
      unspecified default settings is used.
 
+     .. seealso:: :ref:`SSL/TLS security considerations <ssl-security>`
+
    * *server_hostname*, is only for use together with *ssl*,
      and sets or overrides the hostname that the target server's certificate
      will be matched against.  By default the value of the *host* argument

Doc/library/ftplib.rst

@@ -80,14 +80,14 @@ The module defines the following items:
    :rfc:`4217`.
    Connect as usual to port 21 implicitly securing the FTP control connection
    before authenticating. Securing the data connection requires the user to
-   explicitly ask for it by calling the :meth:`prot_p` method.
+   explicitly ask for it by calling the :meth:`prot_p` method.  *context*
-   *keyfile* and *certfile* are optional -- they can contain a PEM formatted
+   is a :class:`ssl.SSLContext` object which allows bundling SSL configuration
-   private key and certificate chain file name for the SSL connection.
+   options, certificates and private keys into a single (potentially
-   *context* parameter is a :class:`ssl.SSLContext` object which allows
+   long-lived) structure.  Please read :ref:`ssl-security` for best practices.
-   bundling SSL configuration options, certificates and private keys into a
+
-   single (potentially long-lived) structure. *source_address* is a 2-tuple
+   *keyfile* and *certfile* are a legacy alternative to *context* -- they
-   ``(host, port)`` for the socket to bind to as its source address before
+   can point to PEM-formatted private key and certificate chain files
-   connecting.
+   (respectively) for the SSL connection.
 
    .. versionadded:: 3.2
 
@@ -96,29 +96,18 @@ The module defines the following items:
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
-   Here's a sample session using the :class:`FTP_TLS` class:
+   Here's a sample session using the :class:`FTP_TLS` class::
 
-   >>> from ftplib import FTP_TLS
+      >>> ftps = FTP_TLS('ftp.pureftpd.org')
-   >>> ftps = FTP_TLS('ftp.python.org')
+      >>> ftps.login()
-   >>> ftps.login()           # login anonymously before securing control channel
+      '230 Anonymous user logged in'
-   >>> ftps.prot_p()          # switch to secure data connection
+      >>> ftps.prot_p()
-   >>> ftps.retrlines('LIST') # list directory content securely
+      '200 Data protection level set to "private"'
-   total 9
+      >>> ftps.nlst()
-   drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 .
+      ['6jack', 'OpenBSD', 'antilink', 'blogbench', 'bsdcam', 'clockspeed', 'djbdns-jedi', 'docs', 'eaccelerator-jedi', 'favicon.ico', 'francotone', 'fugu', 'ignore', 'libpuzzle', 'metalog', 'minidentd', 'misc', 'mysql-udf-global-user-variables', 'php-jenkins-hash', 'php-skein-hash', 'php-webdav', 'phpaudit', 'phpbench', 'pincaster', 'ping', 'posto', 'pub', 'public', 'public_keys', 'pure-ftpd', 'qscan', 'qtc', 'sharedance', 'skycache', 'sound', 'tmp', 'ucarp']
-   drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 ..
-   drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 bin
-   drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 etc
-   d-wxrwxr-x   2 ftp      wheel        1024 Sep  5 13:43 incoming
-   drwxr-xr-x   2 root     wheel        1024 Nov 17  1993 lib
-   drwxr-xr-x   6 1094     wheel        1024 Sep 13 19:07 pub
-   drwxr-xr-x   3 root     wheel        1024 Jan  3  1994 usr
-   -rw-r--r--   1 root     root          312 Aug  1  1994 welcome.msg
-   '226 Transfer complete.'
-   >>> ftps.quit()
-   >>>
 
 
 .. exception:: error_reply
@@ -434,8 +423,8 @@ FTP_TLS Objects
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 .. method:: FTP_TLS.ccc()
 

Doc/library/http.client.rst

@@ -43,10 +43,10 @@ The module provides the following classes:
    For example, the following calls all create instances that connect to the server
    at the same host and port::
 
-      >>> h1 = http.client.HTTPConnection('www.cwi.nl')
+      >>> h1 = http.client.HTTPConnection('www.python.org')
-      >>> h2 = http.client.HTTPConnection('www.cwi.nl:80')
+      >>> h2 = http.client.HTTPConnection('www.python.org:80')
-      >>> h3 = http.client.HTTPConnection('www.cwi.nl', 80)
+      >>> h3 = http.client.HTTPConnection('www.python.org', 80)
-      >>> h3 = http.client.HTTPConnection('www.cwi.nl', 80, timeout=10)
+      >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)
 
    .. versionchanged:: 3.2
       *source_address* was added.
@@ -64,23 +64,27 @@ The module provides the following classes:
    A subclass of :class:`HTTPConnection` that uses SSL for communication with
    secure servers.  Default port is ``443``.  If *context* is specified, it
    must be a :class:`ssl.SSLContext` instance describing the various SSL
-   options.  If *context* is specified and has a :attr:`~ssl.SSLContext.verify_mode`
+   options.
-   of either :data:`~ssl.CERT_OPTIONAL` or :data:`~ssl.CERT_REQUIRED`, then
-   by default *host* is matched against the host name(s) allowed by the
-   server's certificate.  If you want to change that behaviour, you can
-   explicitly set *check_hostname* to False.
 
    *key_file* and *cert_file* are deprecated, please use
-   :meth:`ssl.SSLContext.load_cert_chain` instead.
+   :meth:`ssl.SSLContext.load_cert_chain` instead, or let
+   :func:`ssl.create_default_context` select the system's trusted CA
+   certificates for you.
 
-   If you access arbitrary hosts on the Internet, it is recommended to
+   The recommended way to connect to HTTPS hosts on the Internet is as
-   require certificate checking and feed the *context* with a set of
+   follows::
-   trusted CA certificates::
 
-      context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+      context = ssl.create_default_context()
-      context.verify_mode = ssl.CERT_REQUIRED
+      h = client.HTTPSConnection('www.python.org', 443, context=context)
-      context.load_verify_locations('/etc/pki/tls/certs/ca-bundle.crt')
+
-      h = client.HTTPSConnection('svn.python.org', 443, context=context)
+   Please read :ref:`ssl-security` for more information on best practices.
+
+   .. note::
+      If *context* is specified and has a :attr:`~ssl.SSLContext.verify_mode`
+      of either :data:`~ssl.CERT_OPTIONAL` or :data:`~ssl.CERT_REQUIRED`, then
+      by default *host* is matched against the host name(s) allowed by the
+      server's certificate.  If you want to change that behaviour, you can
+      explicitly set *check_hostname* to False.
 
    .. versionchanged:: 3.2
       *source_address*, *context* and *check_hostname* were added.

Doc/library/imaplib.rst

@@ -69,21 +69,25 @@ There's also a subclass for secure connections:
    This is a subclass derived from :class:`IMAP4` that connects over an SSL
    encrypted socket (to use this class you need a socket module that was compiled
    with SSL support).  If *host* is not specified, ``''`` (the local host) is used.
-   If *port* is omitted, the standard IMAP4-over-SSL port (993) is used.  *keyfile*
+   If *port* is omitted, the standard IMAP4-over-SSL port (993) is used.
-   and *certfile* are also optional - they can contain a PEM formatted private key
+   *ssl_context* is a :class:`ssl.SSLContext` object which allows bundling
-   and certificate chain file for the SSL connection. *ssl_context* parameter is a
+   SSL configuration options, certificates and private keys into a single
-   :class:`ssl.SSLContext` object which allows bundling SSL configuration
+   (potentially long-lived) structure.  Please read :ref:`ssl-security` for
-   options, certificates and private keys into a single (potentially long-lived)
+   best practices.
-   structure. Note that the *keyfile*/*certfile* parameters are mutually exclusive with *ssl_context*,
+
-   a :class:`ValueError` is raised if *keyfile*/*certfile* is provided along with *ssl_context*.
+   *keyfile* and *certfile* are a legacy alternative to *ssl_context* - they
+   can point to PEM-formatted private key and certificate chain files for
+   the SSL connection.  Note that the *keyfile*/*certfile* parameters are
+   mutually exclusive with *ssl_context*, a :class:`ValueError` is raised
+   if *keyfile*/*certfile* is provided along with *ssl_context*.
 
    .. versionchanged:: 3.3
       *ssl_context* parameter added.
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 The second subclass allows for connections created by a child process:
 
@@ -437,14 +441,15 @@ An :class:`IMAP4` instance has the following methods:
 
    Send a ``STARTTLS`` command.  The *ssl_context* argument is optional
    and should be a :class:`ssl.SSLContext` object.  This will enable
-   encryption on the IMAP connection.
+   encryption on the IMAP connection.  Please read :ref:`ssl-security` for
+   best practices.
 
    .. versionadded:: 3.2
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 .. method:: IMAP4.status(mailbox, names)
 

Doc/library/nntplib.rst

@@ -94,6 +94,7 @@ The module itself defines the following classes:
    port *port*.  :class:`NNTP_SSL` objects have the same methods as
    :class:`NNTP` objects.  If *port* is omitted, port 563 (NNTPS) is used.
    *ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object.
+   Please read :ref:`ssl-security` for best practices.
    All other parameters behave the same as for :class:`NNTP`.
 
    Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of
@@ -104,8 +105,8 @@ The module itself defines the following classes:
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 .. exception:: NNTPError
 
@@ -234,9 +235,10 @@ tuples or objects that the method normally returns will be empty.
 
 .. method:: NNTP.starttls(ssl_context=None)
 
-   Send a ``STARTTLS`` command.  The *ssl_context* argument is optional
+   Send a ``STARTTLS`` command.  This will enable encryption on the NNTP
-   and should be a :class:`ssl.SSLContext` object.  This will enable
+   connection.  The *ssl_context* argument is optional and should be a
-   encryption on the NNTP connection.
+   :class:`ssl.SSLContext` object.  Please read :ref:`ssl-security` for best
+   practices.
 
    Note that this may not be done after authentication information has
    been transmitted, and authentication occurs by default if possible during a
@@ -247,8 +249,8 @@ tuples or objects that the method normally returns will be empty.
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 .. method:: NNTP.newgroups(date, *, file=None)
 

Doc/library/poplib.rst

@@ -43,20 +43,23 @@ The :mod:`poplib` module provides two classes:
 
    This is a subclass of :class:`POP3` that connects to the server over an SSL
    encrypted socket.  If *port* is not specified, 995, the standard POP3-over-SSL
-   port is used.  *keyfile* and *certfile* are also optional - they can contain a
+   port is used.  *timeout* works as in the :class:`POP3` constructor.
-   PEM formatted private key and certificate chain file for the SSL connection.
+   *context* is an optional :class:`ssl.SSLContext` object which allows
-   *timeout* works as in the :class:`POP3` constructor. *context* parameter is a
+   bundling SSL configuration options, certificates and private keys into a
-   :class:`ssl.SSLContext` object which allows bundling SSL configuration
+   single (potentially long-lived) structure.  Please read :ref:`ssl-security`
-   options, certificates and private keys into a single (potentially long-lived)
+   for best practices.
-   structure.
+
+   *keyfile* and *certfile* are a legacy alternative to *context* - they can
+   point to PEM-formatted private key and certificate chain files,
+   respectively, for the SSL connection.
 
    .. versionchanged:: 3.2
       *context* parameter added.
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 One exception is defined as an attribute of the :mod:`poplib` module:
 
@@ -198,10 +201,12 @@ An :class:`POP3` instance has the following methods:
 
    *context* parameter is a :class:`ssl.SSLContext` object which allows
    bundling SSL configuration options, certificates and private keys into
-   a single (potentially long-lived) structure.  This method supports
+   a single (potentially long-lived) structure.  Please read :ref:`ssl-security`
-   hostname checking via :attr:`SSLContext.check_hostname`
+   for best practices.
-   :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+
-   :data:`~ssl.HAS_SNI`).
+   This method supports hostname checking via
+   :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+   :data:`ssl.HAS_SNI`).
 
    .. versionadded:: 3.4
 

Doc/library/smtplib.rst

@@ -69,20 +69,15 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions).
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
    *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
-   arguments *local_hostname* and *source_address* have the same meaning as
+   arguments *local_hostname*, *timeout* and *source_address* have the same
-   they do in the :class:`SMTP` class.  *keyfile* and *certfile* are also
+   meaning as they do in the :class:`SMTP` class.  *context*, also optional,
-   optional, and can contain a PEM formatted private key and certificate chain
+   can contain a :class:`~ssl.SSLContext` and allows to configure various
-   file for the SSL connection. *context* also optional, can contain a
+   aspects of the secure connection.  Please read :ref:`ssl-security` for
-   SSLContext, and is an alternative to keyfile and certfile; If it is
+   best practices.
-   specified both keyfile and certfile must be None.  The optional *timeout*
+
-   parameter specifies a timeout in seconds for blocking operations like the
+   *keyfile* and *certfile* are a legacy alternative to *context*, and can
-   connection attempt (if not specified, the global default timeout setting
+   point to a PEM formatted private key and certificate chain file for the
-   will be used). The optional source_address parameter allows to bind to some
+   SSL connection.
-   specific source address in a machine with multiple network interfaces,
-   and/or to some specific source tcp port. It takes a 2-tuple (host, port),
-   for the socket to bind to as its source address before connecting. If
-   omitted (or if host or port are ``''`` and/or 0 respectively) the OS default
-   behavior will be used.
 
    .. versionchanged:: 3.3
       *context* was added.
@@ -92,8 +87,8 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions).
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
-      :data:`~ssl.HAS_SNI`).
+      :data:`ssl.HAS_SNI`).
 
 .. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None, source_address=None)
 

Doc/library/ssl.rst

@@ -1542,7 +1542,7 @@ waiting for clients to connect::
 
    import socket, ssl
 
-   context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+   context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    context.load_cert_chain(certfile="mycertfile", keyfile="mykeyfile")
 
    bindsocket = socket.socket()
@@ -1619,9 +1619,39 @@ to be aware of:
 Security considerations
 -----------------------
 
-Verifying certificates
+Best defaults
-^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^
 
+For **client use**, if you don't have any special requirements for your
+security policy, it is highly recommended that you use the
+:func:`create_default_context` function to create your SSL context.
+It will load the system's trusted CA certificates, enable certificate
+validation, and try to choose reasonably secure protocol and cipher settings.
+
+For example, here is how you would use the :class:`smtplib.SMTP` class to
+create a trusted, secure connection to a SMTP server::
+
+   >>> import ssl, smtplib
+   >>> smtp = smtplib.SMTP("mail.python.org", port=587)
+   >>> context = ssl.create_default_context()
+   >>> smtp.starttls(context=context)
+   (220, b'2.0.0 Ready to start TLS')
+
+If a client certificate is needed for the connection, it can be added with
+:meth:`SSLContext.load_cert_chain`.
+
+By contrast, if you create the SSL context by calling the :class:`SSLContext`
+constructor yourself, it will not have certificate validation enabled by
+default.  If you do so, please read the paragraphs below to achieve a good
+security level.
+
+Manual settings
+^^^^^^^^^^^^^^^
+
+Verifying certificates
+''''''''''''''''''''''
+
+When calling the the :class:`SSLContext` constructor directly,
 :const:`CERT_NONE` is the default.  Since it does not authenticate the other
 peer, it can be insecure, especially in client mode where most of time you
 would like to ensure the authenticity of the server you're talking to.
@@ -1645,7 +1675,7 @@ to specify :const:`CERT_REQUIRED` and similarly check the client certificate.
       by default).
 
 Protocol versions
-^^^^^^^^^^^^^^^^^
+'''''''''''''''''
 
 SSL version 2 is considered insecure and is therefore dangerous to use.  If
 you want maximum compatibility between clients and servers, it is recommended
@@ -1655,11 +1685,11 @@ SSLv2 explicitly using the :data:`SSLContext.options` attribute::
    context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    context.options |= ssl.OP_NO_SSLv2
 
-The SSL context created above will allow SSLv3 and TLSv1 connections, but
+The SSL context created above will allow SSLv3 and TLSv1 (and later, if
-not SSLv2.
+supported by your system) connections, but not SSLv2.
 
 Cipher selection
-^^^^^^^^^^^^^^^^
+''''''''''''''''
 
 If you have advanced security requirements, fine-tuning of the ciphers
 enabled when negotiating a SSL session is possible through the