Commit Graph

554 Commits

Author SHA1 Message Date
Pierre Kancir 9550721ae4 Tools: add initial_mode testing 2020-12-16 17:27:07 +09:00
Peter Barker 9ad2961664 autotest: allow regex for matching wait_statustext 2020-12-11 07:35:02 +11:00
Peter Barker cecdc798ca autotest: present strings for unexpected results in run_cmd_get_ack 2020-12-09 14:09:44 +11:00
Peter Barker 9207bd8a8a autotest: remove armability check from compass calibration test
The new prearm-checking code has shown that we're not actually armable
after all!

Even the new prearm bit won't tell you if you can arm - you have to arm
to test that!
2020-12-07 11:21:30 +11:00
Peter Barker 087300749c autotest: ensure we have position without being armable for compass cal check 2020-12-07 11:21:30 +11:00
Peter Barker 38e324f141 autotest: do not check prearm bit in dataflash-over-mavlink
We won't be armable - but the other checks we make will ensure we get
the error we are looking for

Also use a more reliable way of draining the data from the autopilot
2020-12-07 11:21:30 +11:00
Peter Barker 5707d8ecc0 autotest: check new MAV_SYS_STATUS_PREARM_CHECK in wait_ready_to_arm 2020-12-07 11:21:30 +11:00
Peter Barker 17f806cdb9 autotest: add test for execution of button-bound functions 2020-12-01 10:06:57 +11:00
Peter Barker 12f001066b autotest: split out a wait_armed function 2020-12-01 10:06:57 +11:00
Peter Barker 3f6b398a01 autotest: include EK2,EK3 and DAL LogStructures in code-defined messages 2020-11-24 15:33:37 +11:00
Peter Barker 29b913cd41 autotest: avoid using undefined fmt variable 2020-11-12 21:00:04 +11:00
Peter Barker 58bdae1b52 autotest: get Valgrind closer-to-working
This gets us past the reboot problem for the most part, where Valgrind doesn't like you exec'ing
2020-11-11 18:22:58 +11:00
Andrew Tridgell 403275beea autotest: added RBOH message 2020-11-10 16:15:45 +11:00
Andrew Tridgell 4a06641048 autotest: added RWOH message 2020-11-10 16:15:45 +11:00
Andrew Tridgell 7f0113b82d autotest: added REPH and REVH 2020-11-10 16:15:45 +11:00
Andrew Tridgell ef5baaf482 autotest: added ROFH msg 2020-11-10 16:15:45 +11:00
Andrew Tridgell 8ba067608b autotest: cope with LogStructure.h in multiple places
and allow for no docs in replay msgs
2020-11-10 16:15:45 +11:00
Andrew Tridgell 161f2a8d2f autotest: improved error msg for missing doc fields
Co-authored-by: Peter Barker <pbarker@barker.dropbear.id.au>
2020-11-10 16:15:45 +11:00
bugobliterator c72c96fa9c Tools: setup CI test for CANBus 2020-11-04 19:59:40 +11:00
Peter Barker 18330d16de autotest: add test for MAVlite 2020-10-27 10:17:53 +11:00
Buzz 586e9f80d5 AutoTest: break copter autotests into smaller "under 10 minute" chunks to speedup github actions CI 2020-10-21 18:01:47 +11:00
Peter Barker 937d7bf335 autotest: add quiet option to drain_mav 2020-10-15 16:31:41 +11:00
Peter Barker 49fd762f26 autotest: fix and re-enable button test
A recent commit to fix the setting-of-pullup-resistors in SITL makes it
possible to re-enable this.

Closes #15259
2020-10-15 08:34:08 +11:00
Peter Barker 8f1e9ebc7f autotest: reset camera feedback rate if exception is raised 2020-10-12 13:48:15 +11:00
Peter Barker d13f6acc66 autotest: quieten mav draining 2020-10-08 17:21:35 +09:00
Peter Barker 1d0735f206 autotest: add a poll_message method 2020-10-07 11:54:02 +11:00
Peter Barker 669622c614 autotest: improve use of SITL-is-running
ATM if the SITL binary dies we don't catch it anywhere.  This might
become part of the solution for that, as well as some changes on
run_one_test.
2020-09-22 21:11:57 +10:00
Peter Barker 1075e13352 autotest: stop set_parameter from using get_parameter
Setting a parameter on ArduPilot causes the autopilot to emit the new
value in a PARAM_VALUE message, so we do not need to fetch it ourselves.

Fetching it ourselves causes subtle problems for the autotest suite
where a PARAM_VALUE is currently in the uart buffer - so the autotest
set_parameter has consumed one PARAM_VALUE message (the auto-emitted
one).  If it immediately does a set_parameter then the next
PARAM_VALUE it sees will be the OLD value as the value comes out of the
uart.
2020-09-15 19:27:36 +10:00
Peter Barker 3e882fcad8 autotest: loosen mag-percent-complete threshold to 95%
So 98% wasn't loose enough...
2020-09-11 15:31:45 +10:00
Peter Barker 0f5f040380 autotest: send all progress messages to autopilot
These will be logged into dataflash and into the telemetry log
2020-09-09 23:47:28 +10:00
bugobliterator d997ca8779 autotest: pass calibration test even if 98% complete before success 2020-09-09 23:46:23 +10:00
Peter Barker 51b54d3740 autotest: avoid leaving message hooks around
Can lead to some very weird and confusing output
2020-09-09 17:59:17 +10:00
Peter Barker 9228f47dc9 autotest: create collections-on-context to record selected messages 2020-09-08 14:19:08 +10:00
Peter Barker f3866e5f39 autotest: disable button test 2020-09-08 12:31:15 +10:00
Andrew Tridgell 7c5a0a6a43 Tools: disable LOG_REPLAY by default in autotest
LOG_REPLAY prevents us logging when we have sensor failures, which
makes many tests hard to debug
2020-09-08 11:01:14 +10:00
Peter Barker 65268ee3d4 autotest: more button-test fixes 2020-09-06 16:49:16 +10:00
bugobliterator 6203e0685e autotest: disable MAG orientation test 2020-09-06 11:21:28 +10:00
Andy Piper 0b9d9fc284 autotest: use StringIO correctly 2020-09-06 09:12:59 +10:00
Andy Piper 6ebc7622fe autotest: comprehensive dataflash test 2020-09-05 10:20:39 +10:00
Peter Barker 2ea41b7997 autotest: spit out actual simulated mag orientations as part of debug 2020-09-03 13:08:07 +10:00
Peter Barker 5a8b1639d0 autotest: fix nmea output test 2020-09-02 10:05:26 +10:00
Peter Barker 23ba2002e8 autotest: dump final mag cal report verbosely 2020-09-01 14:11:57 +10:00
Peter Barker 74c47827d9 autotest: drain stdout of SITL binary
The new rate messages were filling up the stdout buffer, causing the
process to block....
2020-08-30 08:38:35 +10:00
Peter Barker 397e134106 autotest: quieten RTL 2020-08-30 08:38:35 +10:00
Peter Barker 065d03dd30 autotest: add a timeout in the button test 2020-08-30 08:38:35 +10:00
Peter Barker c01b26ca62 autotest: fix race condition in EKF type 10
EKF type 10 is always able to give you a position - but the arming
checks require you to have a good GPS position, and that's something
else again.
2020-08-30 08:38:35 +10:00
Peter Barker 43cdceba8f autotest: tweak wait_ready_to_arm 2020-08-30 08:38:35 +10:00
Peter Barker b7641345e1 autotest: set and get parameters directly using mavlink message 2020-08-30 08:38:35 +10:00
Pierre Kancir 936511a720 Autotest: common: correct some text format 2020-08-29 10:44:19 +10:00
Pierre Kancir 5b72de25e0 Autotest: update wait_distance_to_home
Autotest: copter: allow to pass min_distance for RTL

Autotest: rover: allow to pass min distance for RTL
2020-08-29 10:44:19 +10:00
Pierre Kancir 16b499d0f6 Autotest: allow to select which GPS flags we want to wait for 2020-08-27 09:11:13 +10:00
Peter Barker 80bee19bc9 autotest: correct race condition in button test
A BUTTON_CHANGE may have been emitted after we send the parameter but before it is processed by the autopilot
2020-08-26 13:22:14 +10:00
Pierre Kancir a181fcf8a2 Autotest: common: put generated parameter and LogMessage files into buildlogs directory 2020-08-25 20:01:27 +10:00
Peter Barker a523868b33 autotest: fix Beacon test 2020-08-25 17:21:50 +10:00
Peter Barker 69d246d63e autotest: add test for priority-based compass reordering 2020-08-25 08:30:04 +10:00
Peter Barker bb697dd4b9 autotest: avoid using mavproxy for testing message intervals 2020-08-24 19:33:37 +10:00
Peter Barker 00bfe04539 autotest: remove duplicate recovery code from set-message-interval test 2020-08-24 14:00:38 +10:00
Peter Barker 03d0eda7f1 autotest: use a passthrough progress function for Telem objects 2020-08-24 13:24:41 +10:00
Pierre Kancir 8b92af732a Autotest: common: add wait_heartbeat to change_mode to clean up msg and give time to status to update 2020-08-21 13:35:51 +10:00
Peter Barker cddb5fb1cc autotest: use message hook when waiting for statustext
Eliminates a race condition when waiting for statustexts.  Getting the
current sim time slurps messages.
2020-08-21 12:26:12 +10:00
Pierre Kancir b7f46462f8 Autotest: common: reduce timeout on ArmFeature test
This is unnessary to wait for 60s for failure, 15s is enough
2020-08-21 09:13:32 +10:00
Pierre Kancir acc4a1db3f Autotest: remove special case for vicon and default vicon for copter to false 2020-08-17 21:17:26 +10:00
Peter Barker 82dd7ece6c autotest: fix frsky bitop on float issue
Crops up in Python 3.8:

    gpi_lat = self.tf_encode_gps_latitude(gpi.lat)
  File "/home/pbarker/rc/ardupilot/Tools/autotest/common.py", line 6183, in tf_encode_gps_latitude
    value = ((abs(lat)/100)*6) | 0x40000000
TypeError: unsupported operand type(s) for |: 'float' and 'int'
2020-08-15 10:42:47 +10:00
Pierre Kancir 388fef23d9 Autotest: common: remove use of some raw mavproxy cmd for rc 2020-08-15 09:16:24 +10:00
Pierre Kancir 30fa433faa Tools: don't add duplicate param to context 2020-08-14 18:16:47 +10:00
Mark Whitehorn d0ff26e782 autotest: expand quadplane AirMode autotest 2020-08-13 08:47:27 +10:00
Peter Barker b9dc7118d4 autotest: add do_timesync_roundtrip to do a timesync against SITL
On the assumption that ArduPilot processes mavlink packets
synchronously (or at least in order), after we have run a timesync
roundtrip we can reasonably expect any mavlink command we have sent
prior to the roundtrip to have been processed, and we should be able to
see the results in the mavlink stream.
2020-08-05 13:24:15 +10:00
Peter Barker 4640c24872 autotest: correct checking for no-logs string 2020-08-04 15:12:08 +10:00
Peter Barker b2748f19f6 autotest: correct validator for Distance wait-and-maintain 2020-08-04 15:12:08 +10:00
Willian Galvani 0f992ac6a3 Autotest: add test for Sub position hold 2020-08-03 20:19:28 -07:00
Peter Barker 66ffb4a17b autotest: retry parameter fetch in MIS_TOTAL test
There's a race condition with MAVProxy; it's fetching parameters here
and seems to ignore our requests.

Also correct named parameter from retry to attempts
2020-07-24 09:38:39 +10:00
Peter Barker ccff006517 autotest: correct time-spent-waiting-to-arm for zero case 2020-07-23 18:10:06 +10:00
Peter Barker f1fc61cb41 autotest: improve compass test diagnostics 2020-07-23 15:11:11 +10:00
Peter Barker b69130d352 autotest: require uptime of 5s before declaring GPS definitely not present
Until we detect a GPS we mark it as "not present", even if the
parameters say that there should be one present.
2020-07-23 11:17:42 +10:00
Peter Barker 3b0524a7e1 autotest: add diagnostics to frsky passthrough test 2020-07-21 11:26:52 +10:00
Peter Barker 4acb66cd13 autotest: correct check of frsky home-relative altitude
GLOBAL_POSITION_INT.relative_alt is in mm
2020-07-20 13:31:17 +10:00
Peter Barker ac827cf45b autotest: allow much more time for frsky test to finish
parameters don't seem to come down very fast at all
2020-07-20 13:31:17 +10:00
Peter Barker a49580e0af autotest: improve debug on frsky tests 2020-07-20 13:31:17 +10:00
Peter Barker d9aa334489 autotest: ensure cached sim time moves forward in frsky test 2020-07-20 13:31:17 +10:00
Peter Barker 6eb1d289b8 autotest: add trivial test for CRSF in copter 2020-07-07 18:48:06 +10:00
Samuel Tabor bde9ce10e3 autotest: Add wipe as argument. 2020-07-07 11:46:32 +10:00
Peter Barker 6c9d756f11 autotest: validate context nesting 2020-07-06 14:21:55 +10:00
yaapu 0cbff6ac66 Tools: autotest FRSkyPassThrough increased timeout to let the test complete 2020-07-02 21:53:06 +10:00
yaapu 75b621a09f Tools: autotest added frsky serial_protocol 4 and 10 testing against mavlink telemetry 2020-07-02 21:53:06 +10:00
Peter Barker 53b6d1a639 autotest: wait for heartbeat when opening connection
# need to wait for a heartbeat to arrive as then mavutil will
        # select the correct set of messages for us to receive in
        # self.mav.messages.  You can actually recieve messages with
        # recv_match and those will not be in self.mav.messages until
        # you do this!

Also, wait_heartbeat ignores heartbeats from e.g. MAVProxy
2020-07-01 14:38:21 +10:00
Peter Barker 3737c13784 autotest: allow message rates to be set by message name, not just ID
Simply for convenience.
2020-06-24 18:09:28 +10:00
Peter Barker cd1198bf49 autotest: drain mav before attempting to run a command
We're losing an arbitrary amount of data while getting the ACK, so it's
fine to lose whatever is in the buffer.
2020-06-24 18:09:28 +10:00
Peter Barker d0b97b89e7 autotest: add an assert_receive_message method 2020-06-24 18:09:28 +10:00
Peter Barker 77e5236278 autotest: drain mavlink queue to avoid failing on slow MISSION_COUNT
It was observed from a log of a failed CI test that the ACK from
clearing the rally items took 6 wallclock seconds to arrive.

We were not waiting for that ACK to arrive before sending the request
for the mission item count, but if it has taken more than six seconds
for the ACK to arrive it is reasonable to assume that MISSION_COUNT
could very well take more than the 10 seconds we allow for it.

If we drain the mav before sending the request for the mission count we
should remove any signficiant timing problem due to a backlog of mavlnk
messages, but the amount of traffic here is problematic.

Also drain in lots of other places where we might be spending way too
long parsing messages.
2020-06-11 20:53:50 +10:00
Peter Barker dc19dfaed8 autotest: fix several race conditions in RCOverride test 2020-06-11 08:30:32 +10:00
Peter Barker e207bbdf2a autotest: fix SYSTEM_TIME stream detection function
MAVProxy's output is read by autotest via pexpect.

SITL's output is read by MAVProxy.

If we don't read MAVProxy's stdout then it doesn't read SITL and
everything stops.

Also, since we need to drain pexpects as part of reboot, and applying
parameter files requires rebooting... we need to append the expect
objects to the global list before we apply parameter files.  So move
that call.
2020-06-10 22:00:01 +10:00
bugobliterator 0528daecd8 autotest: increase timeout for compass calibration 2020-06-08 07:25:51 +10:00
bugobliterator a5e9da337a Tools: autotest: fix FixedYawCalibration 2020-06-05 09:38:40 +10:00
Pierre Kancir 2c7fc65fbd Tools: common.py: add generic compass calibration test 2020-06-05 09:38:40 +10:00
Pierre Kancir b72a17533c Tools: common: add a method to try arming and wait a result 2020-06-05 09:38:40 +10:00
Pierre Kancir 59de90d97f Tools: common.py: make max_delta a paramter for verify_parameter_values 2020-06-05 09:38:40 +10:00
Peter Barker 6490a57179 autotest: move check_logs to be an AutoTest-class function
This makes the AutoTest instance cognizant of the binary log files it is creating.  This will be useful for checking the contents of the log files created.
2020-06-04 20:15:31 +10:00
Peter Barker c474edbfbc autotest: attempt to make parameter download more reliable
something is stopping parameters from being fetched.  Give it time to
clear.
2020-05-30 10:11:07 +10:00
Peter Barker b0916231b2 autotest: add tests for log download 2020-05-26 19:32:49 +10:00
Peter Barker 605672b631 autotest: round frsky altitude rather than truncate
Closes https://github.com/ArduPilot/ardupilot/issues/14431
2020-05-23 17:40:24 +10:00
Peter Barker 1f1383ab64 autotest: add quadplane test for qassist 2020-05-20 18:39:42 +10:00
Peter Barker f9c86bbfd0 autotest: ensure cached timestamp is up-to-date for run_cmd_get_ack
If a long-running process drains the mavlink stream rather than parsing
it then the cached timestamp can be very, very out-of-date.  When we
next receieve a timestamp, then, there can be a signficant change in
time when we weren't expecting it.

run_cmd_get_ack can't use get_sim_time() as it might swallow the ack it
is looking for.
2020-05-19 19:12:41 +10:00
Peter Barker b857002158 autotest: drain mav before checking EKF values
May be getting stale values from this function
2020-05-19 19:12:41 +10:00
Peter Barker d51d6752e5 autotest: drain mav before delay
We're seeing suspiciously low wall-clock delays.
2020-05-19 19:12:41 +10:00
Peter Barker dec615bfb5 autotest: drain SITL pexpect when waiting for parameter
This is important when rebooting as the ArduPilot process can block on
sending to stdout, which pexpect is reading from.  While rebooting we're
waiting for a parameter to be reset to a different value in this loop,
which could take quite some time.
2020-05-15 16:02:09 +10:00
Peter Barker f6b121ad87 autotest: add tests for logging 2020-05-15 16:02:09 +10:00
Peter Barker b312260be3 autotest: add timestamp to progress message 2020-05-05 11:09:43 +10:00
Peter Barker ebc781ba3a autotest: switch to looking for barometer update string for passthrough test
MAVProxy can fetch parameters using FTP rather than normal param
mechanism.  Currently this does not emit any of our statustext
information - like the text we're looking for in this test
2020-05-04 10:07:38 +10:00
Peter Barker 864717310d autotest: allow specification of EKF type to use on commandline 2020-04-24 13:37:47 +10:00
Peter Barker c6cf809041 autotest: correct some parsing-out-of-logging-message 2020-04-22 21:10:55 +10:00
Samuel Tabor 913e5a23fe autotest: Add method to get default params for model. 2020-04-22 10:01:09 +10:00
Samuel Tabor e30f0a9173 autotest: Move soaring test into plane. 2020-04-19 11:09:16 +10:00
Peter Barker efb5ad2f1c autotest: correct checking of downloaded file
First time we've tried to round-trip a mission file with comments it it

New frames used by new mission files need stuff to be ignored
2020-04-15 16:24:40 +10:00
Peter Barker 10e8f3f3d7 autotest: canonicalise on defaults_filepath rather than defaults_file
This naming better represents the fact this isn't just a filename.
2020-04-14 15:09:17 +10:00
Samuel Tabor c4e255fa5f autotest: Fix key error 2020-04-14 15:09:17 +10:00
Peter Barker 691ae58a79 Tools: rename APMrover2 to Rover 2020-04-14 09:50:34 +09:00
Peter Barker 9511fa59ec autotest: permit specification of frame as part of customing cmdline 2020-04-12 20:34:26 +10:00
Peter Barker e0380fc9be autotest: improve debug information for missing fields 2020-04-12 08:36:48 +10:00
Pierre Kancir 5b26df5780 Tools: put Mission files into right directories 2020-04-09 09:01:03 +10:00
Peter Barker 2d171d0cb4 autotest: add sanity check for duplicate labels in LogWrite calls 2020-04-08 11:17:41 +10:00
Samuel Tabor a4c5e12fac Tools: Make soaring test a subclass of plane test. 2020-04-08 09:11:54 +10:00
Peter Barker 52bb7760bc autotest: add debug for COMMAND_LONG commands 2020-04-06 10:07:26 +10:00
Peter Barker 53ced35f9d autotest: correct bytes decoding for Python3 in Logger documentation 2020-03-29 14:44:36 +11:00
Peter Barker b8dba9886f autotest: Python3 fixes for LTM tests 2020-03-29 10:00:08 +11:00
Pierre Kancir 2d65cbd884 Tools: Common : create generic wait_and_maintain function, upgrade all wait functions to use it 2020-03-27 22:13:52 +11:00
Peter Barker 6206cbd1fa autotest: add sanity checking for undocumented/overdocumented logger messages 2020-03-22 18:35:14 +11:00
Peter Barker e671a0912e autotest: logger_metadata: understand , in @LoggerMessage
This means multiple messages use exactly the same meaning in the fields

Also retain ordering of fields
2020-03-21 09:34:43 +11:00
Peter Barker 28c3ba32ae autotest: add test for AP_Logger documentation generation 2020-03-20 13:05:46 +11:00
Peter Barker 741d3e1435 autotest: make verify_parameter_values more verbose 2020-03-20 13:05:15 +11:00
Peter Barker 403950f61f autotest: add autotests for CPU failsafe 2020-03-20 12:19:21 +11:00
Peter Barker 92d2186b95 autotest: don't use self.mav.motors_disarmed_wait as it blocks forever 2020-03-17 16:31:29 +11:00
Peter Barker 7038d522b8 autotest: run GetCapabilities test on all vehicles 2020-03-03 09:44:39 +11:00
Peter Barker cb8d623624 autotest: add test for fixed yaw calibration 2020-02-29 16:25:30 +11:00
Peter Barker 82f53abee2 autotest: restore fly.ArduCopter as running all tests
the test list for ArduCopter is made up of those from tests1 and tests2
2020-02-22 16:51:39 +11:00
Pierre Kancir e1d8148cc0 autotest: Python correctness fixes
Tools: common: correct copy shadowing in set_rc_from_map

Tools: common: comment out unused code

Tools: common: remove iteritem usage for py3

Tools: common: remove duplicate key in FRSkySPort id description for GPS

Tools: rover: correct type shadowing

Tools: rover: make expected_distance_epsilon float

Tools: rover: comment out unused code

Tools: rover: correct variable shadowing on my_message_hook

Tools: copter: correct variable shadowing

Tools: common: use Bool for port.setblocking

Tools: common: fix variable shadowing
2020-02-22 10:37:29 +11:00
Peter Barker b2d05c9a85 autotest: make download_parameters more robust 2020-02-19 15:09:16 +11:00
Peter Barker e3e152ae3e autotest: make all-parameter-documentation-working a full autotest 2020-02-19 15:09:16 +11:00
Peter Barker a86dc33235 autotest: improve debugging of original location discrepancy 2020-02-18 12:29:00 +11:00
Peter Barker 14ad9a58bd autotest: don't blat main test exception with still-armed exception 2020-02-14 16:36:12 +11:00
Peter Barker fa33719da6 autotest: improve distance-from-home debug 2020-02-14 16:36:12 +11:00
Peter Barker 6843d0e628 autotest: improve debug on parameter download failure 2020-02-13 21:31:20 +11:00
Peter Barker be7ae6fbc9 autotest: add --validate-parameters autotest option
After each reboot, the complete list of parameters is downloaded and
checked against parameter documentation generated from the source code.
2020-02-13 21:31:20 +11:00
Yaapu b0e5bfc32d autotest:FRSkySPort: added support for VARIO 0x30 and GPS 0x800 sensor types
This adds support for the VARIO sensor 0x30 and GPS sensor 0x800 to the autotest suite
2020-02-09 20:52:58 +11:00
Peter Barker fd8088f1e5 autotest: add Copter tests for altitude change on arm 2020-02-08 11:06:03 +11:00
Peter Barker ac904085d6 autotest: add statistics on time taken to arm 2020-02-08 11:06:03 +11:00
Peter Barker b0aceaea70 autotest: add tests for LTM telemetry output 2020-02-08 10:38:34 +11:00
Peter Barker 62f289b8e6 autotest: drain_mav_unparsed gets a quiet option 2020-02-08 10:38:34 +11:00
Peter Barker b75dd36f71 autotest: create Telem baseclass for telem clients 2020-02-08 10:38:34 +11:00
Peter Barker 212bc6538b autotest: improve output on wait_altitude 2020-02-07 15:02:35 +11:00
Peter Barker 441f43103a autotest: add send_debug_trap to send a mavlink trap to the autopilot 2020-01-29 10:27:01 +11:00