Issue #18103: Update README.txt and test_idle to describe and run gui tests.

This commit is contained in:
Terry Jan Reedy 2013-06-29 18:22:02 -04:00
parent c4565a9524
commit e4e530e7e8
2 changed files with 70 additions and 28 deletions

View File

@ -1,14 +1,19 @@
README FOR IDLE TESTS IN IDLELIB.IDLE_TEST
1. Test Files
The idle directory, idlelib, has over 60 xyz.py files. The idle_test
subdirectory should contain a test_xyy.py for each one. (For test modules,
make 'xyz' lower case.) Each should start with the following cut-paste
template, with the blanks after after '.'. 'as', and '_' filled in.
subdirectory should contain a test_xyy.py for each. (For test modules, make
'xyz' lower case, and possibly shorten it.) Each file should start with the
something like the following template, with the blanks after after '.' and 'as',
and before and after '_' filled in.
---
import unittest
import idlelib. as
from test.support import requires
import idlelib. as
class Test_(unittest.TestCase):
class _Test(unittest.TestCase):
def test_(self):
@ -21,43 +26,75 @@ Once test_xyy is written, the following should go at the end of xyy.py,
with xyz (lowercased) added after 'test_'.
---
if __name__ == "__main__":
from test import support; support.use_resources = ['gui']
import unittest
unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False)
---
In Idle, pressing F5 in an editor window with either xyz.py or test_xyz.py
loaded will then run the test with the version of Python running Idle and
tracebacks will appear in the Shell window. The options are appropriate for
developers running (as opposed to importing) either type of file during
development: verbosity=2 lists all test_y methods; exit=False avoids a
spurious sys.exit traceback when running in Idle. The following command
lines also run test_xyz.py
2. Gui Tests
Gui tests need 'requires' and 'use_resources' from test.support
(test.test_support in 2.7). A test is a gui test if it creates a Tk root or
master object either directly or indirectly by instantiating a tkinter or
idle class. For the benefit of buildbot machines that do not have a graphics
screen, gui tests must be 'guarded' by "requires('gui')" in a setUp
function or method. This will typically be setUpClass.
---
@classmethod
def setUpClass(cls):
requires('gui')
---
All gui objects must be destroyed by the end of the test, perhaps in a tearDown
function.
Support.requires('gui') returns true if it is either called in a main module
(which never happens on buildbots) or if use_resources contains 'gui'.
Use_resources is set by test.regrtest but not by unittest. So when running
tests in another module with unittest, we set it ourselves, as in the xyz.py
template above.
Since non-gui tests always run, but gui tests only sometimes, tests of non-gui
operations should best avoid needing a gui. Methods that make incidental use of
tkinter variables and messageboxes can do this by using the mock classes in
idle_test/mock_tk.py.
3. Running Tests
Assume that xyz.py and test_xyz.py end with the "if __name__" statements given
above. In Idle, pressing F5 in an editor window with either loaded will run all
tests in the test_xyz file with the version of Python running Idle. The test
report and any tracebacks will appear in the Shell window. The options in these
"if __name__" statements are appropriate for developers running (as opposed to
importing) either of the files during development: verbosity=2 lists all test
methods in the file; exit=False avoids a spurious sys.exit traceback that would
otherwise occur when running in Idle. The following command lines also run
all test methods, including gui tests, in test_xyz.py. (The exceptions are that
idlelib and idlelib.idle start Idle and idlelib.PyShell should (issue 18330).)
python -m idlelib.xyz # With the capitalization of the xyz module
python -m unittest -v idlelib.idle_test.test_xyz
python -m idlelib.idle_test.test_xyz
To run all idle tests either interactively ('>>>', with unittest imported)
or from a command line, use one of the following.
To run all idle_test/test_*.py tests, either interactively
('>>>', with unittest imported) or from a command line, use one of the
following. (Notes: unittest does not run gui tests; in 2.7, 'test ' (with the
space) is 'test.regrtest '; where present, -v and -ugui can be omitted.)
>>> unittest.main('idlelib.idle_test', verbosity=2, exit=False)
python -m unittest -v idlelib.idle_test
python -m test -v -ugui test_idle
python -m test.test_idle
python -m test test_idle
The idle tests are 'discovered' in idlelib.idle_test.__init__.load_tests,
The idle tests are 'discovered' by idlelib.idle_test.__init__.load_tests,
which is also imported into test.test_idle. Normally, neither file should be
changed when working on individual test modules. The last command runs runs
changed when working on individual test modules. The third command runs runs
unittest indirectly through regrtest. The same happens when the entire test
suite is run with 'python -m test'. So it must work for buildbots to stay green.
suite is run with 'python -m test'. So that command must work for buildbots
to stay green. Idle tests must not disturb the environment in a way that
makes other tests fail (issue 18081).
To run an individual Testcase or test method, extend the
dotted name given to unittest on the command line.
To run an individual Testcase or test method, extend the dotted name given to
unittest on the command line. (But gui tests will not this way.)
python -m unittest -v idlelib.idle_test.text_xyz.Test_case.test_meth
To disable test/test_idle.py, there are at least two choices.
a. Comment out 'load_tests' line, no no tests are discovered (simple and safe);
Running no tests passes, so there is no indication that nothing was run.
b.Before that line, make module an unexpected skip for regrtest with
import unittest; raise unittest.SkipTest('skip for buildbots')
When run directly with unittest, this causes a normal exit and traceback.

View File

@ -10,5 +10,10 @@ idletest = import_module('idlelib.idle_test')
load_tests = idletest.load_tests
if __name__ == '__main__':
# Until unittest supports resources, we emulate regrtest's -ugui
# so loaded tests run the same as if textually present here.
# If any Idle test ever needs another resource, add it to the list.
from test import support
support.use_resources = ['gui'] # use_resources is initially None
import unittest
unittest.main(verbosity=2, exit=False)