From 031d0640cde8c25229cfd536c61ea1c7450b10a9 Mon Sep 17 00:00:00 2001 From: Terry Jan Reedy Date: Sat, 16 May 2015 14:23:39 -0400 Subject: [PATCH] Issue #23964: Update idlelib/idle_test/README.txt. first patch by Al Sweigart. --- Lib/idlelib/idle_test/README.txt | 130 +++++++++++++++++++------------ 1 file changed, 79 insertions(+), 51 deletions(-) diff --git a/Lib/idlelib/idle_test/README.txt b/Lib/idlelib/idle_test/README.txt index f6b6a21ad5a..2339926ef33 100644 --- a/Lib/idlelib/idle_test/README.txt +++ b/Lib/idlelib/idle_test/README.txt @@ -1,14 +1,24 @@ README FOR IDLE TESTS IN IDLELIB.IDLE_TEST +0. Quick Start + +Automated unit tests were added in 2.7 for Python 2.x and 3.3 for Python 3.x. +To run the tests from a command line: + +python -m test.test_idle + +Human-mediated tests were added later in 2.7 and in 3.4. + +python -m idlelib.idle_test.htest + 1. Test Files The idle directory, idlelib, has over 60 xyz.py files. The idle_test -subdirectory should contain a test_xyy.py for each. (For test modules, make -'xyz' lower case, and possibly shorten it.) Each file should start with the -something like the following template, with the blanks after after '.' and 'as', -and before and after '_' filled in. ---- +subdirectory should contain a test_xyz.py for each, where 'xyz' is lowercased +even if xyz.py is not. Here is a possible template, with the blanks after after +'.' and 'as', and before and after '_' to be filled in. + import unittest from test.support import requires import idlelib. as @@ -18,33 +28,33 @@ class _Test(unittest.TestCase): def test_(self): if __name__ == '__main__': - unittest.main(verbosity=2, exit=2) ---- -Idle tests are run with unittest; do not use regrtest's test_main. + unittest.main(verbosity=2) + +Add the following at the end of xyy.py, with the appropriate name added after +'test_'. Some files already have something like this for htest. If so, insert +the import and unittest.main lines before the htest lines. -Once test_xyy is written, the following should go at the end of xyy.py, -with xyz (lowercased) added after 'test_'. ---- if __name__ == "__main__": import unittest unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False) ---- -2. Gui Tests -Gui tests need 'requires' from test.support (test.test_support in 2.7). A -test is a gui test if it creates a Tk root or master object either directly -or indirectly by instantiating a tkinter or idle class. For the benefit of -test processes that either have no graphical environment available or are not -allowed to use it, gui tests must be 'guarded' by "requires('gui')" in a -setUp function or method. This will typically be setUpClass. +2. GUI Tests + +When run as part of the Python test suite, Idle gui tests need to run +test.support.requires('gui') (test.test_support in 2.7). A test is a gui test +if it creates a Tk root or master object either directly or indirectly by +instantiating a tkinter or idle class. For the benefit of test processes that +either have no graphical environment available or are not allowed to use it, gui +tests must be 'guarded' by "requires('gui')" in a setUp function or method. +This will typically be setUpClass. + +To avoid interfering with other gui tests, all gui objects must be destroyed and +deleted by the end of the test. Widgets, such as a Tk root, created in a setUpX +function, should be destroyed in the corresponding tearDownX. Module and class +widget attributes should also be deleted.. -To avoid interfering with other gui tests, all gui objects must be destroyed -and deleted by the end of the test. If a widget, such as a Tk root, is created -in a setUpX function, destroy it in the corresponding tearDownX. For module -and class attributes, also delete the widget. ---- @classmethod def setUpClass(cls): requires('gui') @@ -54,49 +64,55 @@ and class attributes, also delete the widget. def tearDownClass(cls): cls.root.destroy() del cls.root ---- -Support.requires('gui') causes the test(s) it guards to be skipped if any of + +Requires('gui') causes the test(s) it guards to be skipped if any of a few conditions are met: - - The tests are being run by regrtest.py, and it was started without - enabling the "gui" resource with the "-u" command line option. + + - The tests are being run by regrtest.py, and it was started without enabling + the "gui" resource with the "-u" command line option. + - The tests are being run on Windows by a service that is not allowed to interact with the graphical environment. + - The tests are being run on Mac OSX in a process that cannot make a window manager connection. + - tkinter.Tk cannot be successfully instantiated for some reason. + - test.support.use_resources has been set by something other than regrtest.py and does not contain "gui". - -Since non-gui tests always run, but gui tests only sometimes, tests of non-gui -operations should best avoid needing a gui. Methods that make incidental use of -tkinter (tk) variables and messageboxes can do this by using the mock classes in -idle_test/mock_tk.py. There is also a mock text that will handle some uses of the -tk Text widget. + +Tests of non-gui operations should avoid creating tk widgets. Incidental uses of +tk variables and messageboxes can be replaced by the mock classes in +idle_test/mock_tk.py. The mock text handles some uses of the tk Text widget. -3. Running Tests +3. Running Unit Tests -Assume that xyz.py and test_xyz.py end with the "if __name__" statements given -above. In Idle, pressing F5 in an editor window with either loaded will run all -tests in the test_xyz file with the version of Python running Idle. The test -report and any tracebacks will appear in the Shell window. The options in these -"if __name__" statements are appropriate for developers running (as opposed to -importing) either of the files during development: verbosity=2 lists all test -methods in the file; exit=False avoids a spurious sys.exit traceback that would -otherwise occur when running in Idle. The following command lines also run -all test methods, including gui tests, in test_xyz.py. (The exceptions are that -idlelib and idlelib.idle start Idle and idlelib.PyShell should (issue 18330).) +Assume that xyz.py and test_xyz.py both end with a unittest.main() call. +Running either from an Idle editor runs all tests in the test_xyz file with the +version of Python running Idle. Test output appears in the Shell window. The +'verbosity=2' option lists all test methods in the file, which is appropriate +when developing tests. The 'exit=False' option is needed in xyx.py files when an +htest follows. -python -m idlelib.xyz # With the capitalization of the xyz module +The following command lines also run all test methods, including +gui tests, in test_xyz.py. (Both '-m idlelib' and '-m idlelib.idle' start +Idle and so cannot run tests.) + +python -m idlelib.xyz python -m idlelib.idle_test.test_xyz -To run all idle_test/test_*.py tests, either interactively -('>>>', with unittest imported) or from a command line, use one of the -following. (Notes: in 2.7, 'test ' (with the space) is 'test.regrtest '; -where present, -v and -ugui can be omitted.) +The following runs all idle_test/test_*.py tests interactively. + +>>> import unittest +>>> unittest.main('idlelib.idle_test', verbosity=2) + +The following run all Idle tests at a command line. Option '-v' is the same as +'verbosity=2'. (For 2.7, replace 'test' in the second line with +'test.regrtest'.) ->>> unittest.main('idlelib.idle_test', verbosity=2, exit=False) python -m unittest -v idlelib.idle_test python -m test -v -ugui test_idle python -m test.test_idle @@ -113,3 +129,15 @@ To run an individual Testcase or test method, extend the dotted name given to unittest on the command line. python -m unittest -v idlelib.idle_test.test_xyz.Test_case.test_meth + + +4. Human-mediated Tests + +Human-mediated tests are widget tests that cannot be automated but need human +verification. They are contained in idlelib/idle_test/htest.py, which has +instructions. (Some modules need an auxiliary function, identified with # htest +# on the header line.) The set is about complete, though some tests need +improvement. To run all htests, run the htest file from an editor or from the +command line with: + +python -m idlelib.idle_test.htest