Commit Graph

72 Commits

Author SHA1 Message Date
Andrew Tridgell 48acb1a2f7 AP_Logger: added LOG_DARM_RATEMAX
this sets the logging rate max when disarmed. In combination with
LOG_DISARMED=3 it gives a very nice setup to get always on logging
with very little addition to the log sizes. It is particularly useful
in combination with LOG_REPLAY=1
2023-04-14 07:48:24 +10:00
Andrew Tridgell 63e45a3910 AP_Logger: added LOG_DISARMED=3
when LOG_DISARMED is set to 3 then we log while disarmed but if we
reboot without ever arming the log is discarded. This allows for using
LOG_DISARMED without filling the microSD.
2023-04-14 07:48:24 +10:00
Andy Piper 38bc9bfab1 AP_Logger: on example sketches there is always time to write out FMT messages 2022-11-09 19:01:14 +11:00
Andy Piper f9451e36f5 AP_Logger: control PrepForArming in UNKNOWN builds 2022-11-09 19:01:14 +11:00
Peter Barker 8e06d5cd36 AP_Logger: include required AP_Vehicle_Type header 2022-11-02 18:35:48 +11:00
Henry Wurzburg a395b3d2a8 AP_Logger: add AUX switch to pause logging of streamed entries 2022-08-02 10:15:08 +10:00
murata 6102e094be AP_Logger: Console output can be disabled 2022-05-17 09:53:06 +10:00
Andrew Tridgell 43a947f390 AP_Logger: fixed a bug in wrapped log count
fixes #20687
2022-05-06 06:56:33 +10:00
Andrew Tridgell ed1a5c086b AP_Logger: removed annoying message on missing logs 2022-04-19 11:45:26 +10:00
Andrew Tridgell 93b55e367e AP_Logger: fixed log listing with gap, and EKF error on log list
this fixes two issues:

The first issue that if we are missing a log file in the middle of the
list then it was not possible to download recent logs, as we get the
incorrect value for total number of logs. This happened for me with
107 logs, with log62 missing from the microSD. It would only show 45
available logs, so the most recent logs could not be downloaded.

The second issue is that get_num_logs() was very slow if there were a
lot of log files in a directory. This would cause EKF errors and ESC
resets. Using a opendir/readdir loop is much faster (approx 10x faster
in my testing with 107 logs on a MatekH743).
2022-04-19 11:45:26 +10:00
Pierre Kancir 89e5e7f5c5 AP_Logger: fix clang-tidy warning about non correct check
strncmp return both positive and negative value, therefore is it better to implicitly compare to 0
2022-01-06 09:13:54 +11:00
Pierre Kancir 4add35355e AP_Logger: use nullptr 2022-01-06 09:13:54 +11:00
Peter Barker e95aedd748 AP_Logger: tidy construction of backends 2021-11-09 10:18:10 +11:00
Andrew Tridgell e8926f8518 AP_Logger: avoid all opens of log files in main thread
this fixes a problem with sdcards where file open is very slow. It can
trigger a watchdog if it is slow enough. Peter and I hit this issue on
a pixracer today with a new sd card
2021-11-01 20:32:38 +11:00
Buzz d4e2828540 AP_Logger: esp32 takes longer to init its SD card after boot.
see libraries/AP_HAL_ESP32/README.md for more.

Author: Charles Villard <charlesvillard10@gmail.com>
Author: Buzz <davidbuzz@gmail.com>
2021-11-01 17:40:31 +11:00
Andrew Tridgell f148c3fc44 AP_Logger: fixed stuck logging with LOG_DISARMED=1
this prevents a stuck write closing logging when LOG_DISARMED=1
2021-09-28 10:47:48 +10:00
bugobliterator 7832f81a84 AP_Logger: change class name from SITL::SITL to SITL::SIM 2021-08-05 07:25:31 +10:00
Andrew Tridgell 662f7a0191 AP_Logger: added LOG_FILE_RATEMAX parameter
this allows for a global "maximum rate in Hz for streaming messages"
setting
2021-08-03 10:23:36 +10:00
Andrew Tridgell 421faa0ada AP_Logger: fixed log creation on forced arm
when we force arm we need to ask the IO thread to create the log, not
create it ourselves
2021-07-01 15:30:11 +10:00
Andrew Tridgell ece3cf5241 AP_Logging: update EKF while waiting for log open
this prevents the EKF getting unhappy when we arm
2021-06-21 20:15:17 +10:00
Peter Barker 151fe6facf AP_Logger: rearrange PrepForArming stuff 2021-06-15 16:42:02 +10:00
Andrew Tridgell 806fdd9389 AP_Logger: moved log file open to logger thread
this prevents a watchdog if the filesystem takes a long time to
respond to a file open
2021-06-15 16:42:02 +10:00
Andrew Tridgell 1279772d92 AP_Logger: disallow log creation in main thread when armed
this fixes an issue where the sd card fails in flight and then
re-mounts. When that happens the logging backend can trigger a new log
open. That causes filesystem operations in the main thread while
flying. That can cause long delays or even a watchdog.

Thanks to Giacomo for noticing this on his flying wing
2021-06-09 15:08:28 +10:00
Peter Barker b681dc940a AP_Logger: add support for AP_Logger into AP_Periph 2021-06-08 09:57:55 +10:00
Peter Barker 13dc5bee25 AP_Logger: fix memory leak when finding last log 2021-04-15 19:33:15 +10:00
Andrew Tridgell c886fd7025 AP_Logger: make log file erase async
this prevents clagging up the main loop on a big erase. With an erase
of large enough files we could trigger a disarmed watchdog.
2021-04-06 10:40:40 +10:00
Peter Barker f261045a2d AP_Logger: multiply IO thread timeout according to SITL speedup
Intended to solve issues we're now getting on CI where the IO thread is
blocking-on-write and the main thread is calling it dead.
2021-03-24 12:54:20 +11:00
Peter Barker 7d42f473fc AP_Logger: allow each backend to be compiled out 2021-01-08 12:48:55 +11:00
Peter Barker e8d24a1f68 AP_Logger: initialise logger instance variables in header 2021-01-07 10:42:13 +11:00
Peter Barker 80e8688888 AP_Logger: remove perf counters 2021-01-07 09:20:00 +11:00
Peter Barker ef0b860a48 AP_Logger: remove Prep and NeedPrep functions
These were only being called directly after Init(), so just tacked them
onto the end of those functions.

The checks in NeedPrep turned out to be mostly redundant.
2021-01-06 18:25:21 +11:00
Andy Piper 779c2a9e5f AP_Logger: move logger IO to a separate thread 2021-01-02 22:30:08 +11:00
Andy Piper 048deeb43e AP_Logger: check is_system_initialized() when determining liveness 2020-12-30 18:16:59 +11:00
Andrew Tridgell 6bf7f9e864 AP_Logger: use load_file()
save a bit of flash
2020-11-18 09:11:36 +11:00
Andrew Tridgell 71e4d72fe3 AP_Logger: removed 64k limitation on log buffer
the reason for the limit is gone with AP_FileSystem
2020-11-10 16:15:45 +11:00
Andrew Tridgell 2945de4796 AP_Logger: use WITH_SEMAPHORE in _WritePrioritisedBlock
this prevents a 1ms delay when waiting for the semaphore
2020-11-10 16:15:45 +11:00
Andrew Tridgell 09aff03edc AP_Logger: support new replay system
added allow_start_ekf and block write method for replay

Co-authored-by: Peter Barker <pbarker@barker.dropbear.id.au>
2020-11-10 16:15:45 +11:00
Andrew Tridgell 66596dcf25 AP_Logger: allow for retry of log open with LOG_DISARMED=1
if booting without a microSD and with LOG_DISARMED=1 then this allows
for a retry of opening the log every 5s
2020-10-27 20:35:56 +11:00
Patrick José Pereira 258083a6f2 AP_Logger: AP_Logger_File: Set custom log directory in init
`_log_directory` was used before `ensure_log_directory_exists`,
to fetch the log path

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
2020-10-20 09:31:49 +11:00
Andy Piper ccb583d092 AP_Logger: make block logger conform to mavlink expectations of numbering
add support for log time to block logger
refactor rotation into backed. Don't start logs when erasing
correct log start logic
separate read and write points so that requesting log information does not corrupt the current log
when starting a new log stop logging first
clear the write buffer when starting a new log
insert utc time when requesting info for the current log
stop logging and request formats again when starting a new log
cope with erase happening while we are logging
keep pushing out startup messages even when format messages are done
don't log to the gcs in the io thread
don't start new logs in the io thread
don't validate logs while erasing
flush logs when stopping logging
account for page header when calculating logs sizes
don't return data when asked for more data than in the log
optimize locking and use separate semaphore to mediate ring buffer access
stop logging when the chip is full and send a notification
calculate logs sizes correctly even when they wrap
read log data correctly even when it wraps
add stats support to block logger
reset dropped when starting a new log
fail logging when the chip is full
refactor critical bufferspace checks
increase messagewriter budget to 250us and to 300us for FMT
2020-09-05 10:20:39 +10:00
Peter Barker 0c5d31f40f AP_Logger: ensure log directory exists before opening log
This will help with starting logs when inserting a fresh SD card which
doesn't have the ardupilot log directory on it
2020-06-16 12:03:21 +10:00
Peter Barker 66f8359c6b AP_Logger: factor out an ensure_log_directory_exists method 2020-06-16 12:03:21 +10:00
Iampete1 fee5b998bf AP_Logger: add min MB free param 2020-06-09 09:23:25 +10:00
Peter Barker 115751833b AP_Logger: process pending rotate on arming
If the user arms the vehicle during the logging-persist-timeout we
should rotate the log immediately.
2020-05-15 16:02:09 +10:00
Peter Barker ca4af94833 AP_Logger: prevent potential infinite recursion in log-open codepath
If anything in start_new_log did logging (for example, by sending a
statustext), we end up infinitely recursing.

With the patch:

diff --git a/libraries/AP_Logger/AP_Logger_File.cpp b/libraries/AP_Logger/AP_Logger_File.cpp
index 69b8ef0431..eb422d10f8 100644
--- a/libraries/AP_Logger/AP_Logger_File.cpp
+++ b/libraries/AP_Logger/AP_Logger_File.cpp
@@ -778,6 +778,7 @@ void AP_Logger_File::PrepForArming()
  */
 void AP_Logger_File::start_new_log(void)
 {
+    gcs().send_text(MAV_SEVERITY_WARNING, "Starting new log");
     stop_logging();

     start_new_log_reset_variables();
pbarker@bluebottle:~/rc/ardupilot(master)$

We see:

    at ../../libraries/AP_Logger/AP_Logger_File.cpp:781
    this=0x555555ad9d30, pBuffer=0x7fffff8209d0, size=75, is_critical=true)
    at ../../libraries/AP_Logger/AP_Logger_Backend.cpp:372
    this=0x555555ad9d30, pBuffer=0x7fffff8209d0, size=75)
    at ../../libraries/AP_Logger/AP_Logger_Backend.h:32
    this=0x555555ad9d30, message=0x7fffff820b10 "Starting new log")
    at ../../libraries/AP_Logger/LogFile.cpp:466
    this=0x555555a6d758 <copter+11384>,
    message=0x7fffff820b10 "Starting new log")
    at ../../libraries/AP_Logger/AP_Logger.cpp:752
    this=0x555555a6e708 <copter+15400>, severity=MAV_SEVERITY_WARNING,
    fmt=0x5555557d64d0 "Starting new log", arg_list=0x7fffff820be0,
    dest_bitmask=1 '\001') at ../../libraries/GCS_MAVLink/GCS_Common.cpp:1847
    this=0x555555a6e708 <copter+15400>, severity=MAV_SEVERITY_WARNING,
    fmt=0x5555557d64d0 "Starting new log", arg_list=0x7fffff820be0)
    at ../../libraries/GCS_MAVLink/GCS.cpp:53
    this=0x555555a6e708 <copter+15400>, severity=MAV_SEVERITY_WARNING,
    fmt=0x5555557d64d0 "Starting new log")
    at ../../libraries/GCS_MAVLink/GCS.cpp:60
    at ../../libraries/AP_Logger/AP_Logger_File.cpp:781
    this=0x555555ad9d30, pBuffer=0x7fffff820dc0, size=75, is_critical=true)
    at ../../libraries/AP_Logger/AP_Logger_Backend.cpp:372

I'm not aware of any instances in the code where this will actually
happen - but it could easily sneak in.
2020-05-11 15:12:36 +10:00
Peter Barker 7befe84f0c AP_Logger: create and use INTERNAL_ERROR macro so we get line numbers 2020-04-30 13:21:36 +10:00
Andrew Tridgell b5cc1f236a AP_Logger: use new DirHandle for directory listings 2020-03-28 15:28:03 +11:00
murata f6595a41ae AP_Logger: Change the type of a method 2020-01-22 12:45:27 +11:00
Andrew Tridgell 25f5069b58 AP_Logger: removed old NuttX hack 2020-01-16 22:16:30 +11:00
Andrew Tridgell 3ebc69320c AP_Logger: fixed LOG_FILE_DSRMROT with delayed log stop
now that we persist logging for 15s after disarm we need to also delay
the log rotation if LOG_FILE_DSRMROT=1. Otherwise we will put the log
data into the next log, which defeats the purpose of the 15s
persistance
2019-10-03 07:17:16 +10:00