Commit Graph

194 Commits

Author SHA1 Message Date
Serhiy Storchaka 5940b92909 Do not reset the line number because we already set file position to correct
value.

(fixes error in patch for issue #18960)
2014-01-09 20:13:52 +02:00
Serhiy Storchaka 1064a13bb0 Do not reset the line number because we already set file position to correct
value.

(fixes error in patch for issue #18960)
2014-01-09 20:12:49 +02:00
Serhiy Storchaka 7282ff6d5b Issue #18960: Fix bugs with Python source code encoding in the second line.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.

* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.

* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.

* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.

* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.

* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
2014-01-09 18:41:59 +02:00
Serhiy Storchaka 768c16ce02 Issue #18960: Fix bugs with Python source code encoding in the second line.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.

* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.

* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.

* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.

* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.

* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
2014-01-09 18:36:09 +02:00
Serhiy Storchaka c679227e31 Issue #1772673: The type of `char*` arguments now changed to `const char*`. 2013-10-19 21:03:34 +03:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Antoine Pitrou 9ed5f27266 Issue #18722: Remove uses of the "register" keyword in C code. 2013-08-13 20:18:52 +02:00
Benjamin Peterson cb2226cb69 merge 3.3 2013-07-15 20:50:25 -07:00
Benjamin Peterson 265fba40c8 move declaration to top of block 2013-07-15 20:50:22 -07:00
Benjamin Peterson fd9c0203de merge 3.3 (closes #18470) 2013-07-15 20:47:47 -07:00
Benjamin Peterson 2dbfd88245 check the return value of new_string() (closes #18470) 2013-07-15 19:15:34 -07:00
Serhiy Storchaka 9670543a00 Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.
2013-06-09 16:53:55 +03:00
Serhiy Storchaka 3af14aaba5 Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.
2013-06-09 16:51:52 +03:00
Victor Stinner 796977360f Issue #9566: Fix compiler warning on Windows 64-bit 2013-06-05 00:44:00 +02:00
Benjamin Peterson d0845588b8 make _PyParser_TokenNames const 2012-10-24 08:21:52 -07:00
Christian Heimes 0b3847de6d Issue #15096: Drop support for the ur string prefix 2012-06-20 11:17:58 +02:00
Armin Ronacher 6ecf77b3f8 Basic support for PEP 414 without docs or tests. 2012-03-04 12:04:06 +00:00
Antoine Pitrou 3a5d4cb940 Issue #13748: Raw bytes literals can now be written with the `rb` prefix as well as `br`. 2012-01-12 22:46:19 +01:00
Martin v. Löwis bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Martin v. Löwis 1ee1b6fe0d Use identifier API for PyObject_GetAttrString. 2011-10-10 18:11:30 +02:00
Martin v. Löwis afe55bba33 Add API for static strings, primarily good for identifiers.
Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
2011-10-09 10:38:36 +02:00
Martin v. Löwis d63a3b8beb Implement PEP 393. 2011-09-28 07:41:54 +02:00
Jesus Cea c1935d2abf Revert bb62908896fe, but keep the test 2011-04-25 04:03:58 +02:00
Jesus Cea 88f7841be7 Correctly merging #9319 into 3.3? 2011-04-25 03:46:43 +02:00
Victor Stinner c68b6aaec8 Issue #9319: Fix a crash on parsing a Python source code without encoding
cookie and not valid in UTF-8: use "<file>" as the filename instead of
reading from NULL.
2011-04-23 00:41:19 +02:00
Victor Stinner fe7c5b5bdf Issue #9319: Include the filename in "Non-UTF8 code ..." syntax error. 2011-04-05 01:48:03 +02:00
Victor Stinner 7f2fee3640 Issue #10785: Store the filename as Unicode in the Python parser. 2011-04-05 00:39:01 +02:00
Victor Stinner 034c7537d8 Issue #10841: don't translate newlines for pgen 2011-01-07 18:56:19 +00:00
Victor Stinner 89e3436606 Issue #10841: set binary mode on files; the parser translates newlines
On Windows, set the binary mode on stdin, stdout, stderr and all
io.FileIO objects (to not translate newlines, \r\n <=> \n). The Python parser
translates newlines (\r\n => \n).
2011-01-07 18:47:22 +00:00
Victor Stinner aaa4e9a438 Remove arbitrary string length limits
PyUnicode_FromFormat() and PyErr_Format() allocates a buffer of the needed
size, it is no more a fixed-buffer of 500 bytes.
2011-01-05 03:33:26 +00:00
Victor Stinner 83098a4095 Issue #10778: decoding_fgets() decodes the filename from the filesystem
encoding instead of UTF-8.
2010-12-27 20:12:13 +00:00
Victor Stinner 22a351aabf Issue #10095: fp_setreadl() doesn't reopen the file, reuse instead the file
descriptor.
2010-10-14 12:04:34 +00:00
Antoine Pitrou f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Victor Stinner 003a5e702b Use PyErr_Format() in decoding_fgets()
Avoid a buffer of 500 bytes allocated on the stack.
2010-04-28 17:06:46 +00:00
Benjamin Peterson 4893abc77a Merged revisions 79725 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79725 | benjamin.peterson | 2010-04-03 18:03:35 -0500 (Sat, 03 Apr 2010) | 4 lines

  use our own locale independent ctype macros

  requires building pyctype.o into pgen
........
2010-04-03 23:10:01 +00:00
Benjamin Peterson 8daa49ee34 Merged revisions 79723 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79723 | benjamin.peterson | 2010-04-03 17:48:51 -0500 (Sat, 03 Apr 2010) | 1 line

  ensure that the locale does not affect the tokenization of identifiers
........
2010-04-03 22:55:48 +00:00
Victor Stinner 52f6dd7a3d Merged revisions 78826 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78826 | victor.stinner | 2010-03-10 23:30:19 +0100 (mer., 10 mars 2010) | 5 lines

  Issue #3137: Don't ignore errors at startup, especially a keyboard interrupt
  (SIGINT). If an error occurs while importing the site module, the error is
  printed and Python exits. Initialize the GIL before importing the site
  module.
........
2010-03-12 14:45:56 +00:00
Victor Stinner 6aa278ee53 Merged revisions 78603 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78603 | victor.stinner | 2010-03-03 00:20:02 +0100 (mer., 03 mars 2010) | 5 lines

  Issue #7820: The parser tokenizer restores all bytes in the right if the BOM
  check fails.

  Fix an assertion in pydebug mode.
........
2010-03-03 00:18:49 +00:00
Benjamin Peterson 8f326b2369 Merged revisions 76052,76522,76591,76689,76697,76733 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76052 | gregory.p.smith | 2009-11-01 20:02:38 -0600 (Sun, 01 Nov 2009) | 5 lines

  see issue1006238, this merges in the following patch to ease cross
  compiling the printf %zd check.

   http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-lang/python/files/python-2.5-cross-printf.patch?rev=1.1&view=markup
........
  r76522 | barry.warsaw | 2009-11-25 12:38:32 -0600 (Wed, 25 Nov 2009) | 2 lines

  Add mktime_tz to __all__.  It's documented as being available in email.utils.
........
  r76591 | benjamin.peterson | 2009-11-29 16:26:26 -0600 (Sun, 29 Nov 2009) | 4 lines

  now that deepcopy can handle instance methods, this hack can be removed #7409

  Thanks Robert Collins
........
  r76689 | benjamin.peterson | 2009-12-06 11:37:48 -0600 (Sun, 06 Dec 2009) | 1 line

  rewrite translate_newlines for clarity
........
  r76697 | benjamin.peterson | 2009-12-06 15:24:30 -0600 (Sun, 06 Dec 2009) | 2 lines

  fix test_parser from tokenizer tweak
........
  r76733 | benjamin.peterson | 2009-12-09 21:37:59 -0600 (Wed, 09 Dec 2009) | 1 line

  substitute PyDict_Check() for PyObject_IsInstance
........
2009-12-13 02:10:36 +00:00
Benjamin Peterson a0dfa82eca Merged revisions 75149,75260-75263,75265-75267,75292,75300,75376,75405,75429-75433,75437,75445,75501,75551,75572,75589-75591,75657,75742,75868,75952-75957,76057,76105,76139,76143,76162,76223 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r75149 | gregory.p.smith | 2009-09-29 16:56:31 -0500 (Tue, 29 Sep 2009) | 3 lines

  Mention issue6972 in extractall docs about overwriting things outside of
  the supplied path.
........
  r75260 | andrew.kuchling | 2009-10-05 16:24:20 -0500 (Mon, 05 Oct 2009) | 1 line

  Wording fix
........
  r75261 | andrew.kuchling | 2009-10-05 16:24:35 -0500 (Mon, 05 Oct 2009) | 1 line

  Fix narkup
........
  r75262 | andrew.kuchling | 2009-10-05 16:25:03 -0500 (Mon, 05 Oct 2009) | 1 line

  Document 'skip' parameter to constructor
........
  r75263 | andrew.kuchling | 2009-10-05 16:25:35 -0500 (Mon, 05 Oct 2009) | 1 line

  Note side benefit of socket.create_connection()
........
  r75265 | andrew.kuchling | 2009-10-05 17:31:11 -0500 (Mon, 05 Oct 2009) | 1 line

  Reword sentence
........
  r75266 | andrew.kuchling | 2009-10-05 17:32:48 -0500 (Mon, 05 Oct 2009) | 1 line

  Use standard comma punctuation; reword some sentences in the docs
........
  r75267 | andrew.kuchling | 2009-10-05 17:42:56 -0500 (Mon, 05 Oct 2009) | 1 line

  Backport r73983: Document the thousands separator.
........
  r75292 | benjamin.peterson | 2009-10-08 22:11:36 -0500 (Thu, 08 Oct 2009) | 1 line

  death to old CVS keyword
........
  r75300 | benjamin.peterson | 2009-10-09 16:48:14 -0500 (Fri, 09 Oct 2009) | 1 line

  fix some coding style
........
  r75376 | benjamin.peterson | 2009-10-11 20:26:07 -0500 (Sun, 11 Oct 2009) | 1 line

  platform we don't care about
........
  r75405 | neil.schemenauer | 2009-10-14 12:17:14 -0500 (Wed, 14 Oct 2009) | 4 lines

  Issue #1754094: Improve the stack depth calculation in the compiler.
  There should be no other effect than a small decrease in memory use.
  Patch by Christopher Tur Lesniewski-Laas.
........
  r75429 | benjamin.peterson | 2009-10-14 20:47:28 -0500 (Wed, 14 Oct 2009) | 1 line

  pep8ify if blocks
........
  r75430 | benjamin.peterson | 2009-10-14 20:49:37 -0500 (Wed, 14 Oct 2009) | 1 line

  use floor division and add a test that exercises the tabsize codepath
........
  r75431 | benjamin.peterson | 2009-10-14 20:56:25 -0500 (Wed, 14 Oct 2009) | 1 line

  change test to what I intended
........
  r75432 | benjamin.peterson | 2009-10-14 22:05:39 -0500 (Wed, 14 Oct 2009) | 1 line

  some cleanups
........
  r75433 | benjamin.peterson | 2009-10-14 22:06:55 -0500 (Wed, 14 Oct 2009) | 1 line

  make inspect.isabstract() always return a boolean; add a test for it, too #7069
........
  r75437 | benjamin.peterson | 2009-10-15 10:44:46 -0500 (Thu, 15 Oct 2009) | 1 line

  only clear a module's __dict__ if the module is the only one with a reference to it #7140
........
  r75445 | vinay.sajip | 2009-10-16 09:06:44 -0500 (Fri, 16 Oct 2009) | 1 line

  Issue #7120: logging: Removed import of multiprocessing which is causing crash in GAE.
........
  r75501 | antoine.pitrou | 2009-10-18 13:37:11 -0500 (Sun, 18 Oct 2009) | 3 lines

  Add a comment about unreachable code, and fix a typo
........
  r75551 | benjamin.peterson | 2009-10-19 22:14:10 -0500 (Mon, 19 Oct 2009) | 1 line

  use property api
........
  r75572 | benjamin.peterson | 2009-10-20 16:55:17 -0500 (Tue, 20 Oct 2009) | 1 line

  clarify buffer arg #7178
........
  r75589 | benjamin.peterson | 2009-10-21 21:26:47 -0500 (Wed, 21 Oct 2009) | 1 line

  whitespace
........
  r75590 | benjamin.peterson | 2009-10-21 21:36:47 -0500 (Wed, 21 Oct 2009) | 1 line

  rewrite to be nice to other implementations
........
  r75591 | benjamin.peterson | 2009-10-21 21:50:38 -0500 (Wed, 21 Oct 2009) | 4 lines

  rewrite for style, clarify, and comments

  Also, use the hasattr() like scheme of allowing BaseException exceptions through.
........
  r75657 | antoine.pitrou | 2009-10-24 07:41:27 -0500 (Sat, 24 Oct 2009) | 3 lines

  Fix compilation error in debug mode.
........
  r75742 | benjamin.peterson | 2009-10-26 17:51:16 -0500 (Mon, 26 Oct 2009) | 1 line

  use 'is' instead of id()
........
  r75868 | benjamin.peterson | 2009-10-27 15:59:18 -0500 (Tue, 27 Oct 2009) | 1 line

  test expect base classes
........
  r75952 | georg.brandl | 2009-10-29 15:38:32 -0500 (Thu, 29 Oct 2009) | 1 line

  Use the correct function name in docstring.
........
  r75953 | georg.brandl | 2009-10-29 15:39:50 -0500 (Thu, 29 Oct 2009) | 1 line

  Remove mention of the old -X command line switch.
........
  r75954 | georg.brandl | 2009-10-29 15:53:00 -0500 (Thu, 29 Oct 2009) | 1 line

  Use constants instead of magic integers for test result.  Do not re-run with --verbose3 for environment changing tests.
........
  r75955 | georg.brandl | 2009-10-29 15:54:03 -0500 (Thu, 29 Oct 2009) | 1 line

  Use a single style for all the docstrings in the math module.
........
  r75956 | georg.brandl | 2009-10-29 16:16:34 -0500 (Thu, 29 Oct 2009) | 1 line

  I do not think the "railroad" program mentioned is still available.
........
  r75957 | georg.brandl | 2009-10-29 16:44:56 -0500 (Thu, 29 Oct 2009) | 1 line

  Fix constant name.
........
  r76057 | benjamin.peterson | 2009-11-02 09:06:45 -0600 (Mon, 02 Nov 2009) | 1 line

  prevent a rather unlikely segfault
........
  r76105 | georg.brandl | 2009-11-04 01:38:12 -0600 (Wed, 04 Nov 2009) | 1 line

  #7259: show correct equivalent for operator.i* operations in docstring; fix minor issues in operator docs.
........
  r76139 | benjamin.peterson | 2009-11-06 19:04:38 -0600 (Fri, 06 Nov 2009) | 1 line

  spelling
........
  r76143 | georg.brandl | 2009-11-07 02:26:07 -0600 (Sat, 07 Nov 2009) | 1 line

  #7271: fix typo.
........
  r76162 | benjamin.peterson | 2009-11-08 22:10:53 -0600 (Sun, 08 Nov 2009) | 1 line

  discuss how to use -p
........
  r76223 | georg.brandl | 2009-11-12 02:29:46 -0600 (Thu, 12 Nov 2009) | 1 line

  Give the profile module a module directive.
........
2009-11-13 02:25:08 +00:00
Benjamin Peterson aeaa592516 Merged revisions 76230 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76230 | benjamin.peterson | 2009-11-12 17:39:44 -0600 (Thu, 12 Nov 2009) | 2 lines

  fix several compile() issues by translating newlines in the tokenizer
........
2009-11-13 00:17:59 +00:00
Skip Montanaro ba1e0f46ab Issue 7147 - remove ability to attempt to build Python without complex number support (was broken anyway) 2009-10-18 14:25:35 +00:00
Benjamin Peterson 0289b15820 Merged revisions 73004,73439,73496,73509,73529,73564,73576-73577,73595-73596,73605 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r73004 | jeffrey.yasskin | 2009-05-28 22:44:31 -0500 (Thu, 28 May 2009) | 5 lines

  Fix nearly all compilation warnings under Apple gcc-4.0.  Tested with OPT="-g
  -Wall -Wstrict-prototypes -Werror" in both --with-pydebug mode and --without.
  There's still a batch of non-prototype warnings in Xlib.h that I don't know how
  to fix.
........
  r73439 | benjamin.peterson | 2009-06-15 19:29:31 -0500 (Mon, 15 Jun 2009) | 1 line

  don't mask encoding errors when decoding a string #6289
........
  r73496 | vinay.sajip | 2009-06-21 12:37:27 -0500 (Sun, 21 Jun 2009) | 1 line

  Issue #6314: logging.basicConfig() performs extra checks on the "level" argument.
........
  r73509 | amaury.forgeotdarc | 2009-06-22 14:33:48 -0500 (Mon, 22 Jun 2009) | 2 lines

  #4490 Fix sample code run by "python -m xml.sax.xmlreader"
........
  r73529 | r.david.murray | 2009-06-23 13:02:46 -0500 (Tue, 23 Jun 2009) | 4 lines

  Fix issue 5230 by having pydoc's safeimport check to see if the import
  error was thrown from itself in order to decide if the module can't be
  found.  Thanks to Lucas Prado Melo for collaborating on the fix and tests.
........
  r73564 | amaury.forgeotdarc | 2009-06-25 17:29:29 -0500 (Thu, 25 Jun 2009) | 6 lines

  #2016 Fix a crash in function call when the **kwargs dictionary is mutated
  during the function call setup.

  This even gives a slight speedup, probably because tuple allocation
  is faster than PyMem_NEW.
........
  r73576 | benjamin.peterson | 2009-06-26 18:37:06 -0500 (Fri, 26 Jun 2009) | 1 line

  document is_declared_global()
........
  r73577 | benjamin.peterson | 2009-06-27 09:16:23 -0500 (Sat, 27 Jun 2009) | 1 line

  link to extensive generator docs in the reference manual
........
  r73595 | ezio.melotti | 2009-06-27 18:45:39 -0500 (Sat, 27 Jun 2009) | 1 line

  stmt and setup can contain multiple statements, see #5896
........
  r73596 | ezio.melotti | 2009-06-27 19:07:45 -0500 (Sat, 27 Jun 2009) | 1 line

  Fixed a wrong apostrophe
........
  r73605 | georg.brandl | 2009-06-28 07:10:18 -0500 (Sun, 28 Jun 2009) | 1 line

  Remove stray pychecker directive.
........
2009-06-28 17:22:03 +00:00
Brett Cannon e3944a5e1e The BDFL has retired! Long live the FLUFL (Friendly Language Uncle For Life)! 2009-04-01 05:08:41 +00:00
Benjamin Peterson f5b52246ed ignore the coding cookie in compile(), exec(), and eval() if the source is a string #4626 2009-03-02 23:31:26 +00:00
Kristján Valur Jónsson 19288c247a Fix an issue in the tokenizer, where a file is opened by fd, but the underlying PyFileIO object wasn created with the closefd attribute true.
Also fix error handling for close() int _fileio.c .  It was incorrect, looking for a negative refcount, and so errors weren't raised.  This is why this issue wasn't caught.
There is a second reason why it isn't seen:  Class IOBase in io.py has a try:/except: around the close() funtion in the __del__() method.  This also masks these error conditions.

This issue was discovered by removing the _set_invalid_parameter_handler() fiddling, thus enabling the C runtime checks on windows.
2008-12-18 17:15:54 +00:00
Brett Cannon da78043237 Latin-1 source code was not being properly decoded when passed through
compile(). This was due to left-over special-casing before UTF-8 became the
default source encoding.

Closes issue #3574. Thanks to Victor Stinner for help with the patch.
2008-10-17 03:38:50 +00:00
Amaury Forgeot d'Arc cf8016a8d6 Issues #2384 and #3975: Tracebacks were not correctly printed when the source file
contains a ``coding:`` header: the wrong line was displayed, and the encoding was not respected.

Patch by Victor Stinner.
2008-10-09 23:37:48 +00:00
Amaury Forgeot d'Arc 9252287f2c Follow-up of #3773: In PyTokenizer_FindEncoding, remove the call to PyErr_NoMemory when PyMem_MALLOC() fails.
It is not stritly necessary, the function may already return NULL without an exception set,
for example when the file cannot be opened.

Discussed with Benjamin Peterson.
2008-09-04 22:53:19 +00:00
Amaury Forgeot d'Arc 1b933ed50a #3773: Check for errors around the use of PyTokenizer_FindEncoding().
reviewed by Brett Cannon.
2008-09-04 22:34:09 +00:00