Commit Graph

276 Commits

Author SHA1 Message Date
Julian Oes 9764c32b55 mavsdk_tests: only apply workaround in GitHub CI 2020-04-08 16:56:52 +02:00
Julian Oes 54fe3c5fd5 mavsdk_tests: remove waitforgz helper, add sleep
This is an attempt to workaround the gz spawn timeout in CI.
Let's just hard-code a sleep before spawning the model.
2020-04-08 16:56:52 +02:00
Julian Oes be9f125c3d mavsdk_tests: raise timeout for czspawn
And provide better output about what happened.
2020-04-03 11:45:35 +02:00
Julian Oes 9c04f66f06 mavsdk_tests: raise overall timeouts
This might prevent some false positives when CI is running slowly or
intermittently.
2020-04-03 11:45:35 +02:00
Julian Oes 28f4dc10ae mavsdk_tests: make sure all log output is printed
This fixes the issue where the last lines of the log output was not
printed in case of error or on the verbose setting. This meant that
essentially the actual test error was not printed.

The fix involves two parts:
1. Firstly collect the output again even if a process has exited.
2. Collect all lines at once and not one line per iteration.
2020-04-03 11:45:35 +02:00
Julian Oes 2a37321293 mavsdk_tests: wait a bit longer for gazebo command
Otherwise this triggers in CI sometimes.
2020-03-31 14:37:37 +02:00
Julian Oes 0edc2aa334 mavsdk_tests: don't stall on timeout 2020-03-31 14:37:37 +02:00
JaeyoungLim 315135c07e Fix indentation in run script and update submodule 2020-03-23 10:55:13 +01:00
Julian Oes 103bfd0d30 mavsdk_tests: fix wait script output 2020-03-23 10:55:13 +01:00
Julian Oes 29c102b205 mavsdk_tests: wait for gz to run for model spawn 2020-03-23 10:55:13 +01:00
JaeyoungLim b1d3b95ebf Add gzmdoelspawning to mavsdk tests 2020-03-23 10:55:13 +01:00
Julian Oes 6d654cb50b mavsdk_tests: improve usage text
These are improvements based on the review by @hamishwillee
in https://github.com/PX4/Devguide/pull/982.
2020-03-19 08:53:21 +01:00
Julian Oes 89599f345f mavsdk_tests: c++ test_runner is now mavsdk_tests 2020-03-18 17:57:41 +00:00
Julian Oes ade39481ad mavsdk_tests: only env vars that are set
Otherwise this raises a KeyError.
2020-03-18 17:57:41 +00:00
Julian Oes 759e3d6a04 mavsdk_tests: pass PX4_HOME_ env variables on
This way tests can be run at different locations.
2020-03-18 17:57:41 +00:00
Julian Oes 4a91d791b1 mavsdk_tests: remove commented out code 2020-03-18 17:57:41 +00:00
Julian Oes 9d124c2b1a mavsdk_tests: a timeout of 5min should be enough 2020-03-18 17:57:41 +00:00
Julian Oes eb1141bbea mavsdk_tests: remove unused import 2020-03-18 17:57:41 +00:00
Julian Oes c6160d4e3f mavsdk_tests: fix error for Python < 3.8 2020-03-18 17:57:41 +00:00
Julian Oes 41341cd985 mavsdk_tests: name files consistently 2020-03-18 17:57:41 +00:00
Julian Oes b5048a3414 mavsdk_tests: add header and fix style 2020-03-18 17:57:41 +00:00
Julian Oes 4905ac5e2c mavsdk_tests: add filter for cases 2020-03-18 17:57:41 +00:00
Julian Oes 79387428c6 mavsdk_tests: fix missing space after number 2020-03-18 17:57:41 +00:00
Julian Oes 3f1990b083 mavsdk_tests: print where to find logfiles 2020-03-18 17:57:41 +00:00
Julian Oes 188177cef6 mavsdk_tests: print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes ab2186f126 mavsdk_tests: improve and fix colors/result 2020-03-18 17:57:41 +00:00
Julian Oes b1ceef5b93 mavsdk_tests: add combined log 2020-03-18 17:57:41 +00:00
Julian Oes 814d79cb32 mavsdk_tests: improve log file folder structure 2020-03-18 17:57:41 +00:00
Julian Oes 8283d19682 mavsdk_tests: print overall results 2020-03-18 17:57:41 +00:00
Julian Oes a208129921 mavsdk_tests: satisfy mypy type checks 2020-03-18 17:57:41 +00:00
Julian Oes 12252c97c5 mavsdk_tests: further argument cleanup, some types 2020-03-18 17:57:41 +00:00
Julian Oes ff64d87ae6 mavsdk_tests: import naming, easier coloring 2020-03-18 17:57:41 +00:00
Julian Oes 11a28840f3 mavsdk_tests: split up Python script
Otherwise, the script will get a tangled mess.
2020-03-18 17:57:41 +00:00
Julian Oes b281d87b4a mavsdk_tests: initialize Tester class in ctor 2020-03-18 17:57:41 +00:00
Julian Oes e2c80e546d mavsdk_tests: properly exit on Ctrl+C
With threads we need to manually take care of it, otherwise it gets
messy and we need to press Ctrl+C multiple times.
2020-03-18 17:57:41 +00:00
Julian Oes 2c2e314ffe mavsdk_tests: improve output, allow log and stdout
This makes it possible to write to logfiles and at the same time print
everything to console in verbose mode.
2020-03-18 17:57:41 +00:00
Julian Oes d7eb600b59 mavsdk_tests: improve test name/description 2020-03-18 17:57:41 +00:00
Julian Oes b2e300ad1f mavsdk_tests: improve test output readability 2020-03-18 17:57:41 +00:00
Julian Oes 54645268ed mavsdk_tests: only print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes 911cdc8774 mavsdk_tests: output less verbose unless needed 2020-03-18 17:57:41 +00:00
Julian Oes bfef243dba mavsdk_tests: write output to logs folder
And create folder if it doesn't exist already.
2020-03-18 17:57:41 +00:00
Julian Oes 5ba28d417a mavsdk_tests: try using tags as intended
The tags should enable use to assemble the tests in various ways. This
will probably require some iterations though.
2020-03-18 17:57:41 +00:00
Julian Oes 512faa6ebe mavsdk_tests: use CHECK if we don't need to abort 2020-03-18 17:57:41 +00:00
Julian Oes 821f7c3cd9 mavsdk_tests: remove unused/not working test
We can add it again once fixedwing is supported. Until then, this is
just confusing.
2020-03-18 17:57:41 +00:00
Julian Oes fb5b05ec39 mavsdk_tests: add missing unit 2020-03-18 17:57:41 +00:00
Julian Oes a5d1ce91e4 mavsdk_tests: consolidate level of abstraction
This moves the CHECK/REQUIRE inside of AutopilotTester.
2020-03-18 17:57:41 +00:00
Julian Oes 70eb444f17 mavsdk_tests: improve naming of argument
We abort when we already failed so this makes more sense.
2020-03-18 17:57:41 +00:00
Julian Oes 41b2362dcf mavsdk_tests: add more config into JSON 2020-03-18 17:57:41 +00:00
Julian Oes c61cd10db5 mavsdk_tests: without speedup the VTOL takes time 2020-03-18 17:57:41 +00:00
Julian Oes b4916fdecb mavsdk_tests: move config into json file 2020-03-18 17:57:41 +00:00
Julian Oes 44a556717d mavsdk_tests: remove plane tests
We can put them back once it's supported.
2020-03-18 17:57:41 +00:00
Julian Oes ae2032d29f mavsdk_tests: use the min of the speed factors
Facepalm.
2020-03-18 17:57:41 +00:00
Julian Oes 86f24f5121 mavsdk_tests: remove opt_flow test
It doesn't actually seem to work.
2020-03-18 17:57:41 +00:00
Julian Oes b7e8837562 mavsdk_tests: add max speed factor
This is required because some tests don't work at more than 1x.
2020-03-18 17:57:41 +00:00
Julian Oes d8b6c864cd mavsdk_tests: PEP8 fixes 2020-03-18 17:57:41 +00:00
Julian Oes e8e8b79322 mavsdk_tests: MAVSDK is required
Without the library installed, we can't build and run the tests.
Silently ignoring this just leads to confusion.
2020-03-18 17:57:41 +00:00
Julian Kent 5bf2648c1f Add debugger options for mavsdk test runner 2020-02-25 17:15:40 +01:00
Lorenz Meier eb50e89d87
Add flow tests to MAVSDK tests (#14039)
* workflows: remove unneeded pip dep and comments

* mavsdk_tests: simple test using flow/offboard

This adds a first proof of concept of a test using flow and offboard.

* Flow MAVSDK test: check against ground truth

* CleanUp

* workflows: use latest docker image with psutil

* mavsdk_tests: fix PEP8 issues

* Add VTOL CI tests back

* switch to non-rendering flow mockup

* workflows: install mavsdk from GitHub release .deb

* Add vision test and disable VTOL temporarlly

* Fist draft of combining test coverage and test in one workflow

* Add VTOL mavsdk tests back in CI

Co-authored-by: Julian Oes <julian@oes.ch>
Co-authored-by: kritz <kritz@ethz.ch>
2020-01-30 11:57:48 +01:00
Julian Oes a2e95f171f mavsdk_tests: fix model name of plane
However, the test does not pass yet, so I'm leaving it commented out.
2020-01-28 20:35:25 +01:00
Julian Oes bb1c08fdd1 mavsdk_tests: enable VTOL, fix TEST_CASE name
We can't use commas in the TEST_CASE name. With the comma, the test did
not run and displayed:

Filters: Takeoff, transition and RTL
No test cases matched 'Takeoff'
No test cases matched 'transition and RTL'
==========================================
No tests ran
2020-01-28 20:35:25 +01:00
Julian Oes 01f29a51a8 mavsdk_tests: DISPLAY env for gazebo, --verbose 2020-01-28 20:35:25 +01:00
Lorenz Meier 7b44f5d069 Tests: Add VTOL test
This is a first initial VTOL test.
2020-01-28 08:59:25 +01:00
Lorenz Meier 480e67fcd4 Plane tester: Complement test
Prepare wider plane test.
2020-01-28 08:59:25 +01:00
Lorenz Meier de8fadf1f5 Test runner: Terminate all px4 apps correctly
Without this change an aborted run leaves zombie processes running.
2020-01-28 08:59:25 +01:00
Julian Oes f64a0982c7 mavsdk_tests: fix build instructions 2020-01-24 12:04:47 +01:00
Julian Oes 0f77e3ba44 mavsdk_tests: actually use chosen model
This does not work for plane yet, and VTOL only acts as multicopter for
now, so doesn't make much sense just yet.
2020-01-24 12:04:47 +01:00
Julian Oes 5589812104 mavsdk_tests: refactor/cleanup
This splits the main function which has become way too long.
2020-01-24 12:04:47 +01:00
Julian Oes 3387c9599c mavsdk_tests: don't constrain speed factor to int
Speed factors slower than 1 should also be possible.
2020-01-17 10:32:21 +01:00
TSC21 dd11a1c5ba don't specifically export PYTHONPATH 2020-01-13 21:48:35 +00:00
TSC21 ed974d83f5 run mission_test over python2 2020-01-13 21:48:35 +00:00
TSC21 e789010295 rostests: add PYTHONPATH export path so to find Python 2 packages 2020-01-13 21:48:35 +00:00
Julian Oes 9e239e47c4 mavsdk_tests: fix PEP8 issues 2020-01-06 11:15:43 -05:00
Lorenz Meier b0cb29ed82 MAVSDK tests: Add minimal plane example
This will add coverage for baseline plane controllers.
2020-01-04 12:17:12 -05:00
Daniel Agar 1edfee4fdd mavsdk_tests: build should be independant of px4 2020-01-01 22:05:29 -05:00
Daniel Agar 7775dfbc56 mavsdk_test_runner.py remove verbose gazebo output 2020-01-01 21:55:42 +01:00
Lorenz Meier 26d01b876c GH Actions: Enable grind option
This allows to run the script in grind mode to find CI failures that are triggered through e.g. race conditions.
2019-12-26 10:16:58 +01:00
Lorenz Meier 73edc21667 Tests: Add further instrumentation 2019-12-26 10:16:58 +01:00
Lorenz Meier 663436fd47 Integration tests: Add Makefile target
This enables unfamiliar users to run the tests quickly without having to memorize all commandsline options.
2019-12-26 10:16:58 +01:00
Julian Oes 3a228622b9 mavsdk_tests: fix verbose output 2019-12-26 10:16:58 +01:00
Julian Oes 99fbadac70 mavsdk_tests: remove output to logfile for now 2019-12-26 10:16:58 +01:00
Julian Oes 6e0cb7859e mavsdk_tests: add option for gzclient GUI 2019-12-26 10:16:58 +01:00
Julian Oes 698731e4f5 mavsdk_tests: don't use SIGINT, just terminate 2019-12-26 10:16:58 +01:00
Julian Oes f5cc6b7dcf mavsdk_tests: end the logfiles with .log 2019-12-26 10:16:58 +01:00
Julian Oes d32e3919dd mavsdk_tests: return error if not ready 2019-12-26 10:16:58 +01:00
Julian Oes b6e6d66616 mavsdk_tests: env vars need to be strings 2019-12-26 10:16:58 +01:00
Julian Oes 7ff4bd8289 mavsdk_tests: use subprocess instead of sys stdout 2019-12-26 10:16:58 +01:00
Julian Oes eef57f93ae mavsdk_tests: fix bugs in readyness checks 2019-12-26 10:16:58 +01:00
Julian Oes 580ea06fb7 mavsdk_tests: restart SITL after each test case 2019-12-26 10:16:58 +01:00
Julian Oes c363a15ad8 mavsdk_tests: enable VTOL tests as well 2019-12-26 10:16:58 +01:00
Julian Oes cc249f980b mavsdk_tests: check if build everything is built 2019-12-26 10:16:58 +01:00
Julian Oes 0a3152786b mavsdk_tests: use SIGINT before SIGTERM 2019-12-26 10:16:58 +01:00
Julian Oes 79a4bde155 mavsdk_tests: rename C++ executable
The executable contains all tests, so the name was a poor choice.
2019-12-26 10:16:58 +01:00
Julian Oes 90c79493f2 mavsdk_tests: print overall result 2019-12-26 10:16:58 +01:00
Julian Oes dde7530df6 mavsdk_tests: properly handle timeout 2019-12-26 10:16:58 +01:00
Julian Oes ce9aa119ff mavsdk_tests: option for speed factor
And we also need to communicate the speed factor for the px4 executable,
otherwise it can't adapt the timeouts accordingly.
2019-12-26 10:16:58 +01:00
Julian Oes 22bac037ed mavsdk_tests: better process management
We now error if px4 or gzserver are already running on startup, and we
also make sure to terminate what we started if interrupted.
2019-12-26 10:16:58 +01:00
Julian Oes 2bbe4dac25 mavsdk_tests: run test runner, speed up sim 2019-12-26 10:16:58 +01:00
Julian Oes 728a0b033e mavsdk_tests: add termination/kill, custom log dir 2019-12-26 10:16:58 +01:00
Julian Oes 09d63f5531 mavsdk_tests: added test runner 2019-12-26 10:16:58 +01:00
Julian Oes 956bd1b7b1 Move mavsdk_tests into test 2019-12-26 10:16:58 +01:00
Martina Rivizzigno 123f769306 enable safe landing test 2019-10-14 10:53:13 -04:00
Martina Rivizzigno 7f4ddde378 enable avoidance test 2019-10-14 10:53:13 -04:00
Martina Rivizzigno 959a35b3bd add safe_landing_planner to CI 2019-08-02 09:12:02 -04:00
Martina Rivizzigno 14501dfdf2 update local_planner ci to avoidance stable release 0.3.0 2019-08-02 09:12:02 -04:00
Martina Rivizzigno f0ce0646a3 rostest_avoiance_run: bump to avoidance stable release 0.2.0 2019-04-10 16:36:18 -04:00
Martina Rivizzigno 09b795161e Obstacle Avoidance testing in CI (#10780) 2019-03-15 00:37:23 -04:00
Anthony Lamping a9fc04b2f9 CI: SITL test run: select ROS version based on env variable 2019-03-09 16:26:37 -05:00
Anthony Lamping 4821885b03 CI: test: alphabetize args 2019-01-31 09:16:18 -05:00
Anthony Lamping 0fafa9dea7 CI: test: set verbose true 2019-01-31 09:16:18 -05:00
Daniel Agar abb3817d31 boards new split VENDOR_MODEL naming convention 2018-11-26 14:40:14 -08:00
Daniel Agar f692ad04d0 boards organization 2018-11-26 14:40:14 -08:00
Beat Küng d418a09e1d module_schema.yaml: move from test/ to validation/
It doesn't really fit to test/, and also no other existing place.
2018-09-25 07:53:29 +02:00
Beat Küng e4d4192f9c module_schema.yaml: add parameter definitions 2018-09-25 07:53:29 +02:00
Beat Küng 52967bd654 Tools/validate_yaml: add schema validation for module yaml config files 2018-09-25 07:53:29 +02:00
Daniel Agar e356fd89b0
CI mission tests add FW and cleanup (#10250)
* jenkins: decrease VTOL mission alt and FW mission use simple land wps
* startup: decrease min airspeed scaling factor for landing
* jenkins: FW mission include loiter_to_alt cmd
2018-08-17 13:02:14 -04:00
Beat Küng 5363aff879 fixes for integration and unit tests
Move ros logs dir so that .ulg files are not in the same directory
(mission_test.py:: get_last_log() fails otherwise)
2018-08-08 21:09:39 +02:00
Beat Küng b66f0f36a5 ros tests: add an interactive flag & disable the PX4 shell for ROS tests (#9672) 2018-06-14 12:37:47 -04:00
Anthony Lamping e91275bcb4 CI: test launch files: setup for different vehicles, some reordering 2018-03-31 17:00:54 -04:00
Anthony Lamping 171a65c1cb CI: on mavros failure restart it instead of killing test 2018-03-31 17:00:54 -04:00
Anthony Lamping 752d43d94c lengthen offboard tests
* land after offboard flying complete
* lengthen rostest time limit for tests (5 min ea)
2018-03-30 18:54:04 -04:00
Anthony Lamping 1ef7b351be CI: assmue plan file format 2018-03-01 12:21:22 -05:00
Anthony Lamping b1f08a9640 ROS launch: cleanup
* remove non-functional gazebo headless arg
* remove unused namespace args
* simplify mavros launch, use mavros's px4.launch instead
* fix single_vehicle_spawn spelling
* formatting with xmllint: remove empty lines, set intent to 4 spaces, add xml tag for editors, add comments to break up spections
* remove old and deprecated launch files
2018-02-11 15:16:35 +01:00
Anthony Lamping 86ae744266 CI: allow Gazebo to restart on crash (#8817)
* add respawn_gazebo arg to be used with empty_world.launch
* catch rospy sleep method's exceptions
* fix copy-paste mistake in land state failure message
2018-02-06 15:11:09 -05:00
Anthony Lamping d375880c4b improve mavros SITL tests (#8652)
-created a test base class to centralize redundant methods among the different tests
-added mission waypoint list topic listener (this also helps make sure the simulation is ready)
-check number of mission waypoints in FCU against mission
-increase time for mavros topics to be ready from 30 to 60 seconds
-reduce position check loop rates
-clean up logging
-support QGC plan for mission file format, see #8619
-vehicle is an arg for mission test launch file, working toward other airframes
-Jenkins: fix vtol vehicle arg value
-get MAV_TYPE param and use FW radius for pure fixed-wing mission position check
-remove unused vehicle arg from test in multiple tests launch, clearing runtime warning
2018-01-14 21:13:45 -05:00
Daniel Agar 6bdc18df6d sitl launch default ekf2 everywhere 2018-01-02 01:35:57 +01:00
Daniel Agar 63deb40a76 ROS tests move to test/ and new Jenkins 2018-01-02 01:35:57 +01:00