Commit Graph

276 Commits

Author SHA1 Message Date
Julian Oes a5a577a6c4 mavsdk_tests: use speed factor, increase timeouts
We had not actually properly adjusted the timeout to the lockstep speed
factor. Once we did that, we had to increase the timeouts quite a bit to
have the tests pass.
2020-09-16 12:51:56 -04:00
Julian Oes 4eb1ea10f0 mavsdk_tests: don't block inside of callback
This is required with MAVSDK v0.29.0 and later!
2020-09-16 12:51:56 -04:00
Julian Oes 8d3f1e63f0 mavsdk_tests: use all os env vars by default 2020-09-16 12:51:56 -04:00
Julian Oes 55d2bdc438 mavsdk_tests: adapt to changed inject API
We now need to say which instance, for now it's all instances.
2020-09-16 12:51:56 -04:00
Julian Oes ed357cf356 mavsdk_tests: move math things to separate header 2020-09-16 12:51:56 -04:00
Julian Oes 1eaceb17dc mavsdk_tests: adjust timeouts by lockstep speed up 2020-09-16 12:51:56 -04:00
Julian Oes 20621e6744 mavsdk_tests: add tests for GPS as height source 2020-09-16 12:51:56 -04:00
Julian Oes c31246e13a mavsdk_tests: add test for stuck mag 2020-09-16 12:51:56 -04:00
Julian Oes afcfa2fbe0 mavsdk_tests: check for stuck baro 2020-09-16 12:51:56 -04:00
Julian Oes a05ad9c64c mavsdk_tests: add wildcards for case filtering 2020-09-16 12:51:56 -04:00
Julian Oes c4efcbf895 mavsdk_tests: check for baro failure in mission 2020-09-16 12:51:56 -04:00
Julian Oes 8d6ebf15b6 mavsdk_tests: test for losing mag during mission 2020-09-16 12:51:56 -04:00
Julian Oes ef7e4531f9 mavsdk_tests: enable GPS lost test for VTOL
This is in multicopter mode only for now though.
2020-09-16 12:51:56 -04:00
Julian Oes 3c499fc3a2 mavsdk_tests: add first GPS lost failsafe test 2020-09-16 12:51:56 -04:00
Beat Küng 055fa768d8 posix: enable ROMFSROOT
This will copy the ROMFS into the build directory under etc/, and thus
needs a change of the PX4 startup parameters and mixer paths.
2020-09-15 09:30:16 +02:00
Julian Kent 435c2fe438 Fix rarely flakey test on Github Actions 2020-07-15 09:50:58 -04:00
Julian Oes 5f2abb66a4 mavsdk_tests: add workaround to prevent failsafe
This workaround should fix the test failure where we disarm before
taking off because we accidentally switched to failsafe mode right
before taking off because we were still in Manual mode and not Hold yet.
2020-07-15 09:38:06 -04:00
Julian Oes 4ad7ea6c1a mavsdk_tests: increase the poll time resolution
With only 10 steps for e.g. 60 seconds we are likely to miss updates.
2020-07-15 09:38:06 -04:00
Julian Kent 78ff7c608f Expand tracking error for straight line test 2020-07-14 11:34:14 +02:00
Julian Kent 14e2603bc4 Pull out matrix math primitives into anon functions 2020-07-14 11:34:14 +02:00
Julian Kent 6ce2769342 Pull out point-to-line distance calc, getting local mission coords 2020-07-14 11:34:14 +02:00
Julian Kent ff4be5d815 Set rate when subscribing to velocity, specify units 2020-07-14 11:34:14 +02:00
Julian Kent e6c5771c59 Add 'corridor check' which makes sure 3D tracking is along mission 2020-07-14 11:34:14 +02:00
Julian Kent db361d7a59 Add straight-line MAVSDK test 2020-07-14 11:34:14 +02:00
Matthias Grob 480cd5ed4e mavsdk_tests: load all os environment variables for gazebo client
such that OpenGL options are also used for mavsdk_tests when the GUI
is enabled.
2020-07-09 16:43:34 +02:00
Julian Oes 620916e4c1 mavsdk_tests: add test back in
This failed with other frames but seems to pass with iris and
standard_vtol.
2020-06-24 10:44:56 -04:00
Julian Oes 5e21eb3b78 mavsdk_test: remove flaky tests
Tests for both of these models did not pass for me locally. Therefore,
I'm removing them for now. It's up to the stakeholders of these models
to add them again.
2020-06-24 10:44:56 -04:00
Daniel Agar 43fdc770c7 mavsdk_tests: temporarily disable offboard position control test 2020-06-22 13:04:39 -04:00
TSC21 bc2b118e48 mavsdk_tests: add iris_dual_gps tests 2020-06-22 12:46:40 -04:00
JaeyoungLim 59c70a5811 Add plane_catapult test for testing hand/catapult launch for fw
This adds the plane_catapult model to the test for testing fw hand /catapult takeoff
2020-06-21 11:10:33 +01:00
Daniel Agar 3092d9e1f9 github actions run SITL tests in parallel
- temporarily disable iris dual gps
2020-06-20 17:44:57 -04:00
TSC21 24ae8f601d mavsdk_tests: add iris_dual_gps to the test pipeline 2020-06-16 13:03:44 +01:00
Julian Oes be4e253e63 mavsdk_tests: bump library from v0.24.0 to v0.27.0 2020-06-11 15:18:57 +02:00
Julian Oes 9b2527549b mavsdk_tests: relax timeout for offboard goto
This probably makes sense because we ask for more accuracy, so it will
take longer to reach a position.
2020-05-29 11:46:14 +02:00
Julian Oes bc1960ef2b mavsdk_tests: make offboard tests more accurate
By making the acceptance radius smaller we should potentially also be
more accurate while landing and fail less.
2020-05-29 11:46:14 +02:00
Julian Oes dd95498379 mavsdk_tests: at normal speed RTL can take a while 2020-05-29 11:46:14 +02:00
Julian Oes 76b7d462ae mavsdk_tests: relax requirement to land at home
Hopefully, this will fix CI.
2020-05-29 11:46:14 +02:00
Julian Oes d43e4d88c3 mavsdk_tests: wait until home is set properly
Otherwise we might have NaN stored as home.
2020-05-29 11:46:14 +02:00
Julian Oes 73dda496ef mavsdk_tests: more verbose output on fail 2020-05-29 11:46:14 +02:00
Julian Oes 160aca9207 mavsdk_tests: more debug info 2020-05-29 11:46:14 +02:00
Julian Oes 1a25cdf7b2 mavsdk_tests: remove intermittently failing tests
We can bring them back when they work consistently.
2020-05-29 11:46:14 +02:00
Julian Oes a1bf1bdd9d mavsdk_tests: prevent exception if log_fd is None 2020-05-29 11:46:14 +02:00
Julian Oes 262341455b mavsdk_tests: map stderr to stdout
That way we don't need to collect it separately.
2020-05-29 11:46:14 +02:00
Julian Oes eba1cf08f8 mavsdk_tests: also use sleep workaround locally
However, don't sleep as long as in CI.
2020-05-29 11:46:14 +02:00
Julian Oes 76750fc8a6 mavsdk_tests: collect buffered up stdout output
Somehow only subprocess.stdout.readline() works at a time. In order not
to miss out on some of the stdout output, we need to collect it all at
the end. Also, we can stop using readline() for processes that have quit
already.
2020-05-29 11:46:14 +02:00
kamilritz 5bd0df422d Speed up mavsdk sitl tests 2020-05-06 09:25:03 +02:00
kamilritz af1a445031 Vision test can be speed up 2020-05-06 09:25:03 +02:00
kritz 33dc0fd146
Add vision velocity integration test (#14818) 2020-05-05 11:14:30 +02:00
Pedro Roque 3ea52fecd3 Added Roll Pitch Yawrate Thrust offboard test. Testing... 2020-04-30 17:43:25 -04:00
Julian Oes cc0b834d15 mavsdk_tests: don't fail if DISPLAY is not set 2020-04-10 14:55:18 +02:00
Julian Oes 9764c32b55 mavsdk_tests: only apply workaround in GitHub CI 2020-04-08 16:56:52 +02:00
Julian Oes 54fe3c5fd5 mavsdk_tests: remove waitforgz helper, add sleep
This is an attempt to workaround the gz spawn timeout in CI.
Let's just hard-code a sleep before spawning the model.
2020-04-08 16:56:52 +02:00
Julian Oes be9f125c3d mavsdk_tests: raise timeout for czspawn
And provide better output about what happened.
2020-04-03 11:45:35 +02:00
Julian Oes 9c04f66f06 mavsdk_tests: raise overall timeouts
This might prevent some false positives when CI is running slowly or
intermittently.
2020-04-03 11:45:35 +02:00
Julian Oes 28f4dc10ae mavsdk_tests: make sure all log output is printed
This fixes the issue where the last lines of the log output was not
printed in case of error or on the verbose setting. This meant that
essentially the actual test error was not printed.

The fix involves two parts:
1. Firstly collect the output again even if a process has exited.
2. Collect all lines at once and not one line per iteration.
2020-04-03 11:45:35 +02:00
Julian Oes 2a37321293 mavsdk_tests: wait a bit longer for gazebo command
Otherwise this triggers in CI sometimes.
2020-03-31 14:37:37 +02:00
Julian Oes 0edc2aa334 mavsdk_tests: don't stall on timeout 2020-03-31 14:37:37 +02:00
JaeyoungLim 315135c07e Fix indentation in run script and update submodule 2020-03-23 10:55:13 +01:00
Julian Oes 103bfd0d30 mavsdk_tests: fix wait script output 2020-03-23 10:55:13 +01:00
Julian Oes 29c102b205 mavsdk_tests: wait for gz to run for model spawn 2020-03-23 10:55:13 +01:00
JaeyoungLim b1d3b95ebf Add gzmdoelspawning to mavsdk tests 2020-03-23 10:55:13 +01:00
Julian Oes 6d654cb50b mavsdk_tests: improve usage text
These are improvements based on the review by @hamishwillee
in https://github.com/PX4/Devguide/pull/982.
2020-03-19 08:53:21 +01:00
Julian Oes 89599f345f mavsdk_tests: c++ test_runner is now mavsdk_tests 2020-03-18 17:57:41 +00:00
Julian Oes ade39481ad mavsdk_tests: only env vars that are set
Otherwise this raises a KeyError.
2020-03-18 17:57:41 +00:00
Julian Oes 759e3d6a04 mavsdk_tests: pass PX4_HOME_ env variables on
This way tests can be run at different locations.
2020-03-18 17:57:41 +00:00
Julian Oes 4a91d791b1 mavsdk_tests: remove commented out code 2020-03-18 17:57:41 +00:00
Julian Oes 9d124c2b1a mavsdk_tests: a timeout of 5min should be enough 2020-03-18 17:57:41 +00:00
Julian Oes eb1141bbea mavsdk_tests: remove unused import 2020-03-18 17:57:41 +00:00
Julian Oes c6160d4e3f mavsdk_tests: fix error for Python < 3.8 2020-03-18 17:57:41 +00:00
Julian Oes 41341cd985 mavsdk_tests: name files consistently 2020-03-18 17:57:41 +00:00
Julian Oes b5048a3414 mavsdk_tests: add header and fix style 2020-03-18 17:57:41 +00:00
Julian Oes 4905ac5e2c mavsdk_tests: add filter for cases 2020-03-18 17:57:41 +00:00
Julian Oes 79387428c6 mavsdk_tests: fix missing space after number 2020-03-18 17:57:41 +00:00
Julian Oes 3f1990b083 mavsdk_tests: print where to find logfiles 2020-03-18 17:57:41 +00:00
Julian Oes 188177cef6 mavsdk_tests: print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes ab2186f126 mavsdk_tests: improve and fix colors/result 2020-03-18 17:57:41 +00:00
Julian Oes b1ceef5b93 mavsdk_tests: add combined log 2020-03-18 17:57:41 +00:00
Julian Oes 814d79cb32 mavsdk_tests: improve log file folder structure 2020-03-18 17:57:41 +00:00
Julian Oes 8283d19682 mavsdk_tests: print overall results 2020-03-18 17:57:41 +00:00
Julian Oes a208129921 mavsdk_tests: satisfy mypy type checks 2020-03-18 17:57:41 +00:00
Julian Oes 12252c97c5 mavsdk_tests: further argument cleanup, some types 2020-03-18 17:57:41 +00:00
Julian Oes ff64d87ae6 mavsdk_tests: import naming, easier coloring 2020-03-18 17:57:41 +00:00
Julian Oes 11a28840f3 mavsdk_tests: split up Python script
Otherwise, the script will get a tangled mess.
2020-03-18 17:57:41 +00:00
Julian Oes b281d87b4a mavsdk_tests: initialize Tester class in ctor 2020-03-18 17:57:41 +00:00
Julian Oes e2c80e546d mavsdk_tests: properly exit on Ctrl+C
With threads we need to manually take care of it, otherwise it gets
messy and we need to press Ctrl+C multiple times.
2020-03-18 17:57:41 +00:00
Julian Oes 2c2e314ffe mavsdk_tests: improve output, allow log and stdout
This makes it possible to write to logfiles and at the same time print
everything to console in verbose mode.
2020-03-18 17:57:41 +00:00
Julian Oes d7eb600b59 mavsdk_tests: improve test name/description 2020-03-18 17:57:41 +00:00
Julian Oes b2e300ad1f mavsdk_tests: improve test output readability 2020-03-18 17:57:41 +00:00
Julian Oes 54645268ed mavsdk_tests: only print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes 911cdc8774 mavsdk_tests: output less verbose unless needed 2020-03-18 17:57:41 +00:00
Julian Oes bfef243dba mavsdk_tests: write output to logs folder
And create folder if it doesn't exist already.
2020-03-18 17:57:41 +00:00
Julian Oes 5ba28d417a mavsdk_tests: try using tags as intended
The tags should enable use to assemble the tests in various ways. This
will probably require some iterations though.
2020-03-18 17:57:41 +00:00
Julian Oes 512faa6ebe mavsdk_tests: use CHECK if we don't need to abort 2020-03-18 17:57:41 +00:00
Julian Oes 821f7c3cd9 mavsdk_tests: remove unused/not working test
We can add it again once fixedwing is supported. Until then, this is
just confusing.
2020-03-18 17:57:41 +00:00
Julian Oes fb5b05ec39 mavsdk_tests: add missing unit 2020-03-18 17:57:41 +00:00
Julian Oes a5d1ce91e4 mavsdk_tests: consolidate level of abstraction
This moves the CHECK/REQUIRE inside of AutopilotTester.
2020-03-18 17:57:41 +00:00
Julian Oes 70eb444f17 mavsdk_tests: improve naming of argument
We abort when we already failed so this makes more sense.
2020-03-18 17:57:41 +00:00
Julian Oes 41b2362dcf mavsdk_tests: add more config into JSON 2020-03-18 17:57:41 +00:00
Julian Oes c61cd10db5 mavsdk_tests: without speedup the VTOL takes time 2020-03-18 17:57:41 +00:00
Julian Oes b4916fdecb mavsdk_tests: move config into json file 2020-03-18 17:57:41 +00:00
Julian Oes 44a556717d mavsdk_tests: remove plane tests
We can put them back once it's supported.
2020-03-18 17:57:41 +00:00
Julian Oes ae2032d29f mavsdk_tests: use the min of the speed factors
Facepalm.
2020-03-18 17:57:41 +00:00
Julian Oes 86f24f5121 mavsdk_tests: remove opt_flow test
It doesn't actually seem to work.
2020-03-18 17:57:41 +00:00
Julian Oes b7e8837562 mavsdk_tests: add max speed factor
This is required because some tests don't work at more than 1x.
2020-03-18 17:57:41 +00:00
Julian Oes d8b6c864cd mavsdk_tests: PEP8 fixes 2020-03-18 17:57:41 +00:00
Julian Oes e8e8b79322 mavsdk_tests: MAVSDK is required
Without the library installed, we can't build and run the tests.
Silently ignoring this just leads to confusion.
2020-03-18 17:57:41 +00:00
Julian Kent 5bf2648c1f Add debugger options for mavsdk test runner 2020-02-25 17:15:40 +01:00
Lorenz Meier eb50e89d87
Add flow tests to MAVSDK tests (#14039)
* workflows: remove unneeded pip dep and comments

* mavsdk_tests: simple test using flow/offboard

This adds a first proof of concept of a test using flow and offboard.

* Flow MAVSDK test: check against ground truth

* CleanUp

* workflows: use latest docker image with psutil

* mavsdk_tests: fix PEP8 issues

* Add VTOL CI tests back

* switch to non-rendering flow mockup

* workflows: install mavsdk from GitHub release .deb

* Add vision test and disable VTOL temporarlly

* Fist draft of combining test coverage and test in one workflow

* Add VTOL mavsdk tests back in CI

Co-authored-by: Julian Oes <julian@oes.ch>
Co-authored-by: kritz <kritz@ethz.ch>
2020-01-30 11:57:48 +01:00
Julian Oes a2e95f171f mavsdk_tests: fix model name of plane
However, the test does not pass yet, so I'm leaving it commented out.
2020-01-28 20:35:25 +01:00
Julian Oes bb1c08fdd1 mavsdk_tests: enable VTOL, fix TEST_CASE name
We can't use commas in the TEST_CASE name. With the comma, the test did
not run and displayed:

Filters: Takeoff, transition and RTL
No test cases matched 'Takeoff'
No test cases matched 'transition and RTL'
==========================================
No tests ran
2020-01-28 20:35:25 +01:00
Julian Oes 01f29a51a8 mavsdk_tests: DISPLAY env for gazebo, --verbose 2020-01-28 20:35:25 +01:00
Lorenz Meier 7b44f5d069 Tests: Add VTOL test
This is a first initial VTOL test.
2020-01-28 08:59:25 +01:00
Lorenz Meier 480e67fcd4 Plane tester: Complement test
Prepare wider plane test.
2020-01-28 08:59:25 +01:00
Lorenz Meier de8fadf1f5 Test runner: Terminate all px4 apps correctly
Without this change an aborted run leaves zombie processes running.
2020-01-28 08:59:25 +01:00
Julian Oes f64a0982c7 mavsdk_tests: fix build instructions 2020-01-24 12:04:47 +01:00
Julian Oes 0f77e3ba44 mavsdk_tests: actually use chosen model
This does not work for plane yet, and VTOL only acts as multicopter for
now, so doesn't make much sense just yet.
2020-01-24 12:04:47 +01:00
Julian Oes 5589812104 mavsdk_tests: refactor/cleanup
This splits the main function which has become way too long.
2020-01-24 12:04:47 +01:00
Julian Oes 3387c9599c mavsdk_tests: don't constrain speed factor to int
Speed factors slower than 1 should also be possible.
2020-01-17 10:32:21 +01:00
TSC21 dd11a1c5ba don't specifically export PYTHONPATH 2020-01-13 21:48:35 +00:00
TSC21 ed974d83f5 run mission_test over python2 2020-01-13 21:48:35 +00:00
TSC21 e789010295 rostests: add PYTHONPATH export path so to find Python 2 packages 2020-01-13 21:48:35 +00:00
Julian Oes 9e239e47c4 mavsdk_tests: fix PEP8 issues 2020-01-06 11:15:43 -05:00
Lorenz Meier b0cb29ed82 MAVSDK tests: Add minimal plane example
This will add coverage for baseline plane controllers.
2020-01-04 12:17:12 -05:00
Daniel Agar 1edfee4fdd mavsdk_tests: build should be independant of px4 2020-01-01 22:05:29 -05:00
Daniel Agar 7775dfbc56 mavsdk_test_runner.py remove verbose gazebo output 2020-01-01 21:55:42 +01:00
Lorenz Meier 26d01b876c GH Actions: Enable grind option
This allows to run the script in grind mode to find CI failures that are triggered through e.g. race conditions.
2019-12-26 10:16:58 +01:00
Lorenz Meier 73edc21667 Tests: Add further instrumentation 2019-12-26 10:16:58 +01:00
Lorenz Meier 663436fd47 Integration tests: Add Makefile target
This enables unfamiliar users to run the tests quickly without having to memorize all commandsline options.
2019-12-26 10:16:58 +01:00
Julian Oes 3a228622b9 mavsdk_tests: fix verbose output 2019-12-26 10:16:58 +01:00
Julian Oes 99fbadac70 mavsdk_tests: remove output to logfile for now 2019-12-26 10:16:58 +01:00
Julian Oes 6e0cb7859e mavsdk_tests: add option for gzclient GUI 2019-12-26 10:16:58 +01:00
Julian Oes 698731e4f5 mavsdk_tests: don't use SIGINT, just terminate 2019-12-26 10:16:58 +01:00
Julian Oes f5cc6b7dcf mavsdk_tests: end the logfiles with .log 2019-12-26 10:16:58 +01:00
Julian Oes d32e3919dd mavsdk_tests: return error if not ready 2019-12-26 10:16:58 +01:00
Julian Oes b6e6d66616 mavsdk_tests: env vars need to be strings 2019-12-26 10:16:58 +01:00
Julian Oes 7ff4bd8289 mavsdk_tests: use subprocess instead of sys stdout 2019-12-26 10:16:58 +01:00
Julian Oes eef57f93ae mavsdk_tests: fix bugs in readyness checks 2019-12-26 10:16:58 +01:00
Julian Oes 580ea06fb7 mavsdk_tests: restart SITL after each test case 2019-12-26 10:16:58 +01:00
Julian Oes c363a15ad8 mavsdk_tests: enable VTOL tests as well 2019-12-26 10:16:58 +01:00
Julian Oes cc249f980b mavsdk_tests: check if build everything is built 2019-12-26 10:16:58 +01:00
Julian Oes 0a3152786b mavsdk_tests: use SIGINT before SIGTERM 2019-12-26 10:16:58 +01:00
Julian Oes 79a4bde155 mavsdk_tests: rename C++ executable
The executable contains all tests, so the name was a poor choice.
2019-12-26 10:16:58 +01:00
Julian Oes 90c79493f2 mavsdk_tests: print overall result 2019-12-26 10:16:58 +01:00
Julian Oes dde7530df6 mavsdk_tests: properly handle timeout 2019-12-26 10:16:58 +01:00
Julian Oes ce9aa119ff mavsdk_tests: option for speed factor
And we also need to communicate the speed factor for the px4 executable,
otherwise it can't adapt the timeouts accordingly.
2019-12-26 10:16:58 +01:00
Julian Oes 22bac037ed mavsdk_tests: better process management
We now error if px4 or gzserver are already running on startup, and we
also make sure to terminate what we started if interrupted.
2019-12-26 10:16:58 +01:00
Julian Oes 2bbe4dac25 mavsdk_tests: run test runner, speed up sim 2019-12-26 10:16:58 +01:00
Julian Oes 728a0b033e mavsdk_tests: add termination/kill, custom log dir 2019-12-26 10:16:58 +01:00
Julian Oes 09d63f5531 mavsdk_tests: added test runner 2019-12-26 10:16:58 +01:00
Julian Oes 956bd1b7b1 Move mavsdk_tests into test 2019-12-26 10:16:58 +01:00