mirror of https://github.com/python/cpython
559 lines
21 KiB
ReStructuredText
559 lines
21 KiB
ReStructuredText
:mod:`nntplib` --- NNTP protocol client
|
|
=======================================
|
|
|
|
.. module:: nntplib
|
|
:synopsis: NNTP protocol client (requires sockets).
|
|
|
|
|
|
.. index::
|
|
pair: NNTP; protocol
|
|
single: Network News Transfer Protocol
|
|
|
|
**Source code:** :source:`Lib/nntplib.py`
|
|
|
|
--------------
|
|
|
|
This module defines the class :class:`NNTP` which implements the client side of
|
|
the Network News Transfer Protocol. It can be used to implement a news reader
|
|
or poster, or automated news processors. It is compatible with :rfc:`3977`
|
|
as well as the older :rfc:`977` and :rfc:`2980`.
|
|
|
|
Here are two small examples of how it can be used. To list some statistics
|
|
about a newsgroup and print the subjects of the last 10 articles::
|
|
|
|
>>> s = nntplib.NNTP('news.gmane.org')
|
|
>>> resp, count, first, last, name = s.group('gmane.comp.python.committers')
|
|
>>> print('Group', name, 'has', count, 'articles, range', first, 'to', last)
|
|
Group gmane.comp.python.committers has 1096 articles, range 1 to 1096
|
|
>>> resp, overviews = s.over((last - 9, last))
|
|
>>> for id, over in overviews:
|
|
... print(id, nntplib.decode_header(over['subject']))
|
|
...
|
|
1087 Re: Commit privileges for Łukasz Langa
|
|
1088 Re: 3.2 alpha 2 freeze
|
|
1089 Re: 3.2 alpha 2 freeze
|
|
1090 Re: Commit privileges for Łukasz Langa
|
|
1091 Re: Commit privileges for Łukasz Langa
|
|
1092 Updated ssh key
|
|
1093 Re: Updated ssh key
|
|
1094 Re: Updated ssh key
|
|
1095 Hello fellow committers!
|
|
1096 Re: Hello fellow committers!
|
|
>>> s.quit()
|
|
'205 Bye!'
|
|
|
|
To post an article from a binary file (this assumes that the article has valid
|
|
headers, and that you have right to post on the particular newsgroup)::
|
|
|
|
>>> s = nntplib.NNTP('news.gmane.org')
|
|
>>> f = open('/tmp/article.txt', 'rb')
|
|
>>> s.post(f)
|
|
'240 Article posted successfully.'
|
|
>>> s.quit()
|
|
'205 Bye!'
|
|
|
|
The module itself defines the following classes:
|
|
|
|
|
|
.. class:: NNTP(host, port=119, user=None, password=None, readermode=None, usenetrc=False, [timeout])
|
|
|
|
Return a new :class:`NNTP` object, representing a connection
|
|
to the NNTP server running on host *host*, listening at port *port*.
|
|
An optional *timeout* can be specified for the socket connection.
|
|
If the optional *user* and *password* are provided, or if suitable
|
|
credentials are present in :file:`/.netrc` and the optional flag *usenetrc*
|
|
is true, the ``AUTHINFO USER`` and ``AUTHINFO PASS`` commands are used
|
|
to identify and authenticate the user to the server. If the optional
|
|
flag *readermode* is true, then a ``mode reader`` command is sent before
|
|
authentication is performed. Reader mode is sometimes necessary if you are
|
|
connecting to an NNTP server on the local machine and intend to call
|
|
reader-specific commands, such as ``group``. If you get unexpected
|
|
:exc:`NNTPPermanentError`\ s, you might need to set *readermode*.
|
|
:class:`NNTP` class supports the :keyword:`with` statement to
|
|
unconditionally consume :exc:`OSError` exceptions and to close the NNTP
|
|
connection when done. Here is a sample on how using it:
|
|
|
|
>>> from nntplib import NNTP
|
|
>>> with NNTP('news.gmane.org') as n:
|
|
... n.group('gmane.comp.python.committers')
|
|
...
|
|
('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers')
|
|
>>>
|
|
|
|
|
|
.. versionchanged:: 3.2
|
|
*usenetrc* is now False by default.
|
|
|
|
.. versionchanged:: 3.3
|
|
Support for the :keyword:`with` statement was added.
|
|
|
|
.. class:: NNTP_SSL(host, port=563, user=None, password=None, ssl_context=None, readermode=None, usenetrc=False, [timeout])
|
|
|
|
Return a new :class:`NNTP_SSL` object, representing an encrypted
|
|
connection to the NNTP server running on host *host*, listening at
|
|
port *port*. :class:`NNTP_SSL` objects have the same methods as
|
|
:class:`NNTP` objects. If *port* is omitted, port 563 (NNTPS) is used.
|
|
*ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object.
|
|
All other parameters behave the same as for :class:`NNTP`.
|
|
|
|
Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of
|
|
STARTTLS as described below. However, some servers only support the
|
|
former.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
.. exception:: NNTPError
|
|
|
|
Derived from the standard exception :exc:`Exception`, this is the base
|
|
class for all exceptions raised by the :mod:`nntplib` module. Instances
|
|
of this class have the following attribute:
|
|
|
|
.. attribute:: response
|
|
|
|
The response of the server if available, as a :class:`str` object.
|
|
|
|
|
|
.. exception:: NNTPReplyError
|
|
|
|
Exception raised when an unexpected reply is received from the server.
|
|
|
|
|
|
.. exception:: NNTPTemporaryError
|
|
|
|
Exception raised when a response code in the range 400--499 is received.
|
|
|
|
|
|
.. exception:: NNTPPermanentError
|
|
|
|
Exception raised when a response code in the range 500--599 is received.
|
|
|
|
|
|
.. exception:: NNTPProtocolError
|
|
|
|
Exception raised when a reply is received from the server that does not begin
|
|
with a digit in the range 1--5.
|
|
|
|
|
|
.. exception:: NNTPDataError
|
|
|
|
Exception raised when there is some error in the response data.
|
|
|
|
|
|
.. _nntp-objects:
|
|
|
|
NNTP Objects
|
|
------------
|
|
|
|
When connected, :class:`NNTP` and :class:`NNTP_SSL` objects support the
|
|
following methods and attributes.
|
|
|
|
Attributes
|
|
^^^^^^^^^^
|
|
|
|
.. attribute:: NNTP.nntp_version
|
|
|
|
An integer representing the version of the NNTP protocol supported by the
|
|
server. In practice, this should be ``2`` for servers advertising
|
|
:rfc:`3977` compliance and ``1`` for others.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
.. attribute:: NNTP.nntp_implementation
|
|
|
|
A string describing the software name and version of the NNTP server,
|
|
or :const:`None` if not advertised by the server.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
Methods
|
|
^^^^^^^
|
|
|
|
The *response* that is returned as the first item in the return tuple of almost
|
|
all methods is the server's response: a string beginning with a three-digit
|
|
code. If the server's response indicates an error, the method raises one of
|
|
the above exceptions.
|
|
|
|
Many of the following methods take an optional keyword-only argument *file*.
|
|
When the *file* argument is supplied, it must be either a :term:`file object`
|
|
opened for binary writing, or the name of an on-disk file to be written to.
|
|
The method will then write any data returned by the server (except for the
|
|
response line and the terminating dot) to the file; any list of lines,
|
|
tuples or objects that the method normally returns will be empty.
|
|
|
|
.. versionchanged:: 3.2
|
|
Many of the following methods have been reworked and fixed, which makes
|
|
them incompatible with their 3.1 counterparts.
|
|
|
|
|
|
.. method:: NNTP.quit()
|
|
|
|
Send a ``QUIT`` command and close the connection. Once this method has been
|
|
called, no other methods of the NNTP object should be called.
|
|
|
|
|
|
.. method:: NNTP.getwelcome()
|
|
|
|
Return the welcome message sent by the server in reply to the initial
|
|
connection. (This message sometimes contains disclaimers or help information
|
|
that may be relevant to the user.)
|
|
|
|
|
|
.. method:: NNTP.getcapabilities()
|
|
|
|
Return the :rfc:`3977` capabilities advertised by the server, as a
|
|
:class:`dict` instance mapping capability names to (possibly empty) lists
|
|
of values. On legacy servers which don't understand the ``CAPABILITIES``
|
|
command, an empty dictionary is returned instead.
|
|
|
|
>>> s = NNTP('news.gmane.org')
|
|
>>> 'POST' in s.getcapabilities()
|
|
True
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
.. method:: NNTP.login(user=None, password=None, usenetrc=True)
|
|
|
|
Send ``AUTHINFO`` commands with the user name and password. If *user*
|
|
and *password* are None and *usenetrc* is True, credentials from
|
|
``~/.netrc`` will be used if possible.
|
|
|
|
Unless intentionally delayed, login is normally performed during the
|
|
:class:`NNTP` object initialization and separately calling this function
|
|
is unnecessary. To force authentication to be delayed, you must not set
|
|
*user* or *password* when creating the object, and must set *usenetrc* to
|
|
False.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
.. method:: NNTP.starttls(ssl_context=None)
|
|
|
|
Send a ``STARTTLS`` command. The *ssl_context* argument is optional
|
|
and should be a :class:`ssl.SSLContext` object. This will enable
|
|
encryption on the NNTP connection.
|
|
|
|
Note that this may not be done after authentication information has
|
|
been transmitted, and authentication occurs by default if possible during a
|
|
:class:`NNTP` object initialization. See :meth:`NNTP.login` for information
|
|
on suppressing this behavior.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
.. method:: NNTP.newgroups(date, *, file=None)
|
|
|
|
Send a ``NEWGROUPS`` command. The *date* argument should be a
|
|
:class:`datetime.date` or :class:`datetime.datetime` object.
|
|
Return a pair ``(response, groups)`` where *groups* is a list representing
|
|
the groups that are new since the given *date*. If *file* is supplied,
|
|
though, then *groups* will be empty.
|
|
|
|
>>> from datetime import date, timedelta
|
|
>>> resp, groups = s.newgroups(date.today() - timedelta(days=3))
|
|
>>> len(groups)
|
|
85
|
|
>>> groups[0]
|
|
GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m')
|
|
|
|
|
|
.. method:: NNTP.newnews(group, date, *, file=None)
|
|
|
|
Send a ``NEWNEWS`` command. Here, *group* is a group name or ``'*'``, and
|
|
*date* has the same meaning as for :meth:`newgroups`. Return a pair
|
|
``(response, articles)`` where *articles* is a list of message ids.
|
|
|
|
This command is frequently disabled by NNTP server administrators.
|
|
|
|
|
|
.. method:: NNTP.list(group_pattern=None, *, file=None)
|
|
|
|
Send a ``LIST`` or ``LIST ACTIVE`` command. Return a pair
|
|
``(response, list)`` where *list* is a list of tuples representing all
|
|
the groups available from this NNTP server, optionally matching the
|
|
pattern string *group_pattern*. Each tuple has the form
|
|
``(group, last, first, flag)``, where *group* is a group name, *last*
|
|
and *first* are the last and first article numbers, and *flag* usually
|
|
takes one of these values:
|
|
|
|
* ``y``: Local postings and articles from peers are allowed.
|
|
* ``m``: The group is moderated and all postings must be approved.
|
|
* ``n``: No local postings are allowed, only articles from peers.
|
|
* ``j``: Articles from peers are filed in the junk group instead.
|
|
* ``x``: No local postings, and articles from peers are ignored.
|
|
* ``=foo.bar``: Articles are filed in the ``foo.bar`` group instead.
|
|
|
|
If *flag* has another value, then the status of the newsgroup should be
|
|
considered unknown.
|
|
|
|
This command can return very large results, especially if *group_pattern*
|
|
is not specified. It is best to cache the results offline unless you
|
|
really need to refresh them.
|
|
|
|
.. versionchanged:: 3.2
|
|
*group_pattern* was added.
|
|
|
|
|
|
.. method:: NNTP.descriptions(grouppattern)
|
|
|
|
Send a ``LIST NEWSGROUPS`` command, where *grouppattern* is a wildmat string as
|
|
specified in :rfc:`3977` (it's essentially the same as DOS or UNIX shell wildcard
|
|
strings). Return a pair ``(response, descriptions)``, where *descriptions*
|
|
is a dictionary mapping group names to textual descriptions.
|
|
|
|
>>> resp, descs = s.descriptions('gmane.comp.python.*')
|
|
>>> len(descs)
|
|
295
|
|
>>> descs.popitem()
|
|
('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)')
|
|
|
|
|
|
.. method:: NNTP.description(group)
|
|
|
|
Get a description for a single group *group*. If more than one group matches
|
|
(if 'group' is a real wildmat string), return the first match. If no group
|
|
matches, return an empty string.
|
|
|
|
This elides the response code from the server. If the response code is needed,
|
|
use :meth:`descriptions`.
|
|
|
|
|
|
.. method:: NNTP.group(name)
|
|
|
|
Send a ``GROUP`` command, where *name* is the group name. The group is
|
|
selected as the current group, if it exists. Return a tuple
|
|
``(response, count, first, last, name)`` where *count* is the (estimated)
|
|
number of articles in the group, *first* is the first article number in
|
|
the group, *last* is the last article number in the group, and *name*
|
|
is the group name.
|
|
|
|
|
|
.. method:: NNTP.over(message_spec, *, file=None)
|
|
|
|
Send a ``OVER`` command, or a ``XOVER`` command on legacy servers.
|
|
*message_spec* can be either a string representing a message id, or
|
|
a ``(first, last)`` tuple of numbers indicating a range of articles in
|
|
the current group, or a ``(first, None)`` tuple indicating a range of
|
|
articles starting from *first* to the last article in the current group,
|
|
or :const:`None` to select the current article in the current group.
|
|
|
|
Return a pair ``(response, overviews)``. *overviews* is a list of
|
|
``(article_number, overview)`` tuples, one for each article selected
|
|
by *message_spec*. Each *overview* is a dictionary with the same number
|
|
of items, but this number depends on the server. These items are either
|
|
message headers (the key is then the lower-cased header name) or metadata
|
|
items (the key is then the metadata name prepended with ``":"``). The
|
|
following items are guaranteed to be present by the NNTP specification:
|
|
|
|
* the ``subject``, ``from``, ``date``, ``message-id`` and ``references``
|
|
headers
|
|
* the ``:bytes`` metadata: the number of bytes in the entire raw article
|
|
(including headers and body)
|
|
* the ``:lines`` metadata: the number of lines in the article body
|
|
|
|
The value of each item is either a string, or :const:`None` if not present.
|
|
|
|
It is advisable to use the :func:`decode_header` function on header
|
|
values when they may contain non-ASCII characters::
|
|
|
|
>>> _, _, first, last, _ = s.group('gmane.comp.python.devel')
|
|
>>> resp, overviews = s.over((last, last))
|
|
>>> art_num, over = overviews[0]
|
|
>>> art_num
|
|
117216
|
|
>>> list(over.keys())
|
|
['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject']
|
|
>>> over['from']
|
|
'=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= <martin@v.loewis.de>'
|
|
>>> nntplib.decode_header(over['from'])
|
|
'"Martin v. Löwis" <martin@v.loewis.de>'
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
.. method:: NNTP.help(*, file=None)
|
|
|
|
Send a ``HELP`` command. Return a pair ``(response, list)`` where *list* is a
|
|
list of help strings.
|
|
|
|
|
|
.. method:: NNTP.stat(message_spec=None)
|
|
|
|
Send a ``STAT`` command, where *message_spec* is either a message id
|
|
(enclosed in ``'<'`` and ``'>'``) or an article number in the current group.
|
|
If *message_spec* is omitted or :const:`None`, the current article in the
|
|
current group is considered. Return a triple ``(response, number, id)``
|
|
where *number* is the article number and *id* is the message id.
|
|
|
|
>>> _, _, first, last, _ = s.group('gmane.comp.python.devel')
|
|
>>> resp, number, message_id = s.stat(first)
|
|
>>> number, message_id
|
|
(9099, '<20030112190404.GE29873@epoch.metaslash.com>')
|
|
|
|
|
|
.. method:: NNTP.next()
|
|
|
|
Send a ``NEXT`` command. Return as for :meth:`stat`.
|
|
|
|
|
|
.. method:: NNTP.last()
|
|
|
|
Send a ``LAST`` command. Return as for :meth:`stat`.
|
|
|
|
|
|
.. method:: NNTP.article(message_spec=None, *, file=None)
|
|
|
|
Send an ``ARTICLE`` command, where *message_spec* has the same meaning as
|
|
for :meth:`stat`. Return a tuple ``(response, info)`` where *info*
|
|
is a :class:`~collections.namedtuple` with three attributes *number*,
|
|
*message_id* and *lines* (in that order). *number* is the article number
|
|
in the group (or 0 if the information is not available), *message_id* the
|
|
message id as a string, and *lines* a list of lines (without terminating
|
|
newlines) comprising the raw message including headers and body.
|
|
|
|
>>> resp, info = s.article('<20030112190404.GE29873@epoch.metaslash.com>')
|
|
>>> info.number
|
|
0
|
|
>>> info.message_id
|
|
'<20030112190404.GE29873@epoch.metaslash.com>'
|
|
>>> len(info.lines)
|
|
65
|
|
>>> info.lines[0]
|
|
b'Path: main.gmane.org!not-for-mail'
|
|
>>> info.lines[1]
|
|
b'From: Neal Norwitz <neal@metaslash.com>'
|
|
>>> info.lines[-3:]
|
|
[b'There is a patch for 2.3 as well as 2.2.', b'', b'Neal']
|
|
|
|
|
|
.. method:: NNTP.head(message_spec=None, *, file=None)
|
|
|
|
Same as :meth:`article()`, but sends a ``HEAD`` command. The *lines*
|
|
returned (or written to *file*) will only contain the message headers, not
|
|
the body.
|
|
|
|
|
|
.. method:: NNTP.body(message_spec=None, *, file=None)
|
|
|
|
Same as :meth:`article()`, but sends a ``BODY`` command. The *lines*
|
|
returned (or written to *file*) will only contain the message body, not the
|
|
headers.
|
|
|
|
|
|
.. method:: NNTP.post(data)
|
|
|
|
Post an article using the ``POST`` command. The *data* argument is either
|
|
a :term:`file object` opened for binary reading, or any iterable of bytes
|
|
objects (representing raw lines of the article to be posted). It should
|
|
represent a well-formed news article, including the required headers. The
|
|
:meth:`post` method automatically escapes lines beginning with ``.`` and
|
|
appends the termination line.
|
|
|
|
If the method succeeds, the server's response is returned. If the server
|
|
refuses posting, a :class:`NNTPReplyError` is raised.
|
|
|
|
|
|
.. method:: NNTP.ihave(message_id, data)
|
|
|
|
Send an ``IHAVE`` command. *message_id* is the id of the message to send
|
|
to the server (enclosed in ``'<'`` and ``'>'``). The *data* parameter
|
|
and the return value are the same as for :meth:`post()`.
|
|
|
|
|
|
.. method:: NNTP.date()
|
|
|
|
Return a pair ``(response, date)``. *date* is a :class:`~datetime.datetime`
|
|
object containing the current date and time of the server.
|
|
|
|
|
|
.. method:: NNTP.slave()
|
|
|
|
Send a ``SLAVE`` command. Return the server's *response*.
|
|
|
|
|
|
.. method:: NNTP.set_debuglevel(level)
|
|
|
|
Set the instance's debugging level. This controls the amount of debugging
|
|
output printed. The default, ``0``, produces no debugging output. A value of
|
|
``1`` produces a moderate amount of debugging output, generally a single line
|
|
per request or response. A value of ``2`` or higher produces the maximum amount
|
|
of debugging output, logging each line sent and received on the connection
|
|
(including message text).
|
|
|
|
|
|
The following are optional NNTP extensions defined in :rfc:`2980`. Some of
|
|
them have been superseded by newer commands in :rfc:`3977`.
|
|
|
|
|
|
.. method:: NNTP.xhdr(hdr, str, *, file=None)
|
|
|
|
Send an ``XHDR`` command. The *hdr* argument is a header keyword, e.g.
|
|
``'subject'``. The *str* argument should have the form ``'first-last'``
|
|
where *first* and *last* are the first and last article numbers to search.
|
|
Return a pair ``(response, list)``, where *list* is a list of pairs ``(id,
|
|
text)``, where *id* is an article number (as a string) and *text* is the text of
|
|
the requested header for that article. If the *file* parameter is supplied, then
|
|
the output of the ``XHDR`` command is stored in a file. If *file* is a string,
|
|
then the method will open a file with that name, write to it then close it.
|
|
If *file* is a :term:`file object`, then it will start calling :meth:`write` on
|
|
it to store the lines of the command output. If *file* is supplied, then the
|
|
returned *list* is an empty list.
|
|
|
|
|
|
.. method:: NNTP.xover(start, end, *, file=None)
|
|
|
|
Send an ``XOVER`` command. *start* and *end* are article numbers
|
|
delimiting the range of articles to select. The return value is the
|
|
same of for :meth:`over()`. It is recommended to use :meth:`over()`
|
|
instead, since it will automatically use the newer ``OVER`` command
|
|
if available.
|
|
|
|
|
|
.. method:: NNTP.xpath(id)
|
|
|
|
Return a pair ``(resp, path)``, where *path* is the directory path to the
|
|
article with message ID *id*. Most of the time, this extension is not
|
|
enabled by NNTP server administrators.
|
|
|
|
.. deprecated:: 3.3
|
|
The XPATH extension is not actively used.
|
|
|
|
|
|
.. XXX deprecated:
|
|
|
|
.. method:: NNTP.xgtitle(name, *, file=None)
|
|
|
|
Process an ``XGTITLE`` command, returning a pair ``(response, list)``, where
|
|
*list* is a list of tuples containing ``(name, title)``. If the *file* parameter
|
|
is supplied, then the output of the ``XGTITLE`` command is stored in a file.
|
|
If *file* is a string, then the method will open a file with that name, write
|
|
to it then close it. If *file* is a :term:`file object`, then it will start
|
|
calling :meth:`write` on it to store the lines of the command output. If *file*
|
|
is supplied, then the returned *list* is an empty list. This is an optional NNTP
|
|
extension, and may not be supported by all servers.
|
|
|
|
RFC2980 says "It is suggested that this extension be deprecated". Use
|
|
:meth:`descriptions` or :meth:`description` instead.
|
|
|
|
|
|
Utility functions
|
|
-----------------
|
|
|
|
The module also defines the following utility function:
|
|
|
|
|
|
.. function:: decode_header(header_str)
|
|
|
|
Decode a header value, un-escaping any escaped non-ASCII characters.
|
|
*header_str* must be a :class:`str` object. The unescaped value is
|
|
returned. Using this function is recommended to display some headers
|
|
in a human readable form::
|
|
|
|
>>> decode_header("Some subject")
|
|
'Some subject'
|
|
>>> decode_header("=?ISO-8859-15?Q?D=E9buter_en_Python?=")
|
|
'Débuter en Python'
|
|
>>> decode_header("Re: =?UTF-8?B?cHJvYmzDqG1lIGRlIG1hdHJpY2U=?=")
|
|
'Re: problème de matrice'
|