Script to update include paths:
for i in $(grep -rl 'include <px4_work_queue' src platforms); do sed -i 's/#include <px4_work_queue/#include <px4_platform_common\/px4_work_queue/' $i; done
* Add kdevelop to gitignore
* Add test stubs
* Rename px4_add_gtest to px4_add_unit_gtest
* Add infrastructure to run functional tests
* Add example tests with parameters and uorb messages
* Fix memory issues in destructors in uORB manager and CDev
* Add a more real-world test of the collision prevention
The current SEGV handling on posix is not useful: as soon as our handler
is left, it's triggered again, infinitely.
This patch changes to restore the original handler, so the OS can create
a core dump, etc.
This moves the hysteresis test out of the systemlib and makes it its own
small library. Since it still depends on hrt_absolute_time this does not
link yet. My attempt to get all link dependencies together failed.
This makes sure we add the lockstep_scheduler_test even if the
ENABLE_LOCKSTEP_SCHEDULER is not set to yes. This means the
lockstep_scheduler is not used for SITL but the CMakeLists.txt file
still used and the test added.
It turns out that we can fix the unit tests of the lockstep_scheduler
just by checking if `passed_lock` is not `nullptr`.
Without this check, the unit tests segfaulted.
This solves a potential dead-lock when trying to shutdown: a call to exit()
stops all threads and calls all destructors for static objects.
The destructor of LockstepScheduler takes a lock. However this is not
safe, as the lock could already be taken (by any thread).
Some threads do not exit and are still running when
trying to exit SITL running under Windows in Cygwin.
This problem was introduced with:
- posix shell #10173 because of strating a child process
for the startup script and mixing up the signal handling
(only Ctrl+C broken)
- lockstep #10648 because of simulator threads not
stopping gracefully anymore with timing race conditions
(no graceful exit possible anymore)
I leave the SIGINT handler on its default implementation for
Cygwin which kills the process and all its threads when pressing
Ctrl+C.
This hotfix at least allows the usage of Ctrl+C again instead
of forcing the user to use the task manager to shut down
px4.exe going crazy on CPU load instead of exiting
everytime.
- use a linked-list instead of std::vector. Insertion and removal are now
O(1)
- avoid malloc and use a thread_local instance of TimedWait.
It gets destroyed when the thread exits, so we have to add protection
in case a thread exits too quickly. This in turn requires a fix to the
unit-tests.
The API of cond_timedwait was wrong. It used return -1 and set errno
instead of returning the error as specified for pthread_cond_timedwait
which it tries to mock.
It seems that the hysteresis test fails every now and then, presumably
due to timing issues. The tests needs some improvements, e.g. isolating
it from the system time.
@bkueng found that the old implementation was likely to wrap-around
given seconds is only a uint32_t. We now cast it directly to uint64_t
and therefore should fix this problem.
This uses the "fake" px4_sem based on mutex and condition_variable on
all POSIX system, not just macOS and Cygwin. This means that we can
change px4_sem_timedwait under the hood and inject the simulated time.
This integrates the lockstep_scheduler, so that the system time is set
by the mavlink HIL_SENSOR message.
This means that the speed factor is removed and the speed is entirely
given by the simulator.
These contains some rough changes trying to get SITL to speed up by a
SPEED_FACTOR.
This platform time code probably requires some more thought and refactor
but this gets a demo at 4x working.
You can now add `DYNAMIC` as an option to `px4_add_module`, which will
cause that module to no longer be compiled into the px4 executable, but
instead produce a separate shared library file, which can be loaded and
executed with the new `dyn` command:
pxh> dyn ./hello.px4mod start
This will load the shared object file `hello.px4mod` if it wasn't
already loaded, and execute its main function with the given arguments.
The threads running commands for clients through the Posix daemon used
to write to a char buffer through snprintf (etc.) which was then written
directly to the file descriptor, whereas in the other case printf
(etc.) was used to write to stdout (FILE*). Both versions used some
macro's and repeated code to have the same output.
This change unifies these two cases by using a FILE* in both cases. The
(line) buffering is done by the standard C library's implementation
(just like with stdout), and px4_log.c now uses the same code in all
cases (using fprintf, etc.) for printing (colored) output.
Because it was always failing from the beginning on and
we want to make sure no other tests break in the meantime
by running the currently passing tests also on Windows CI.
Unlike pipes, unix sockets provide bi-directional
communication with each connected client.
- No need to generate a unique uuid per client anymore.
- The client doesn't have to create its own pipe anymore.
- Since there is no risk of multiple client's writes getting mixed up,
messages don't need to fit in a single write anymore, removing the
limit on command length.
- Since the server can detect a connection closing, the client no longer
needs to handle signals. When the client is killed, the connection is
automatically closed, which will cause the server to kill the related
px4 thread.
Since this does not rely on handling signals and the client sending an
additional message, this is much more reliable.
- Client is no longer a singleton.
- The protocol is simplified. Standard output is directly written to the
socket back to the client, without wrapping it in any protocol
message.
- Because of the simple protocol, one could now even use netcat to run a
px4 command:
$ echo hello | netcat -UN /tmp/px4-sock-0
Also removes a few race conditions.
Fixes these invalid format strings:
- A `%d` for a pointer (replaced it by `%p`)
- A 0x%08x (and a 0x%0x8!) for a pointer (replaced by %p)
- 2 cases of `%d` for a `ssize_t` (replaced it by `%zi`)
- 1 case of a %u for an `int` (replaced by %i)
- 3 cases of %d for a `long` (replaced by %ld)
- 19 cases of `%d`, `%i`, `%u` or `%lu` for a `size_t` (replaced it by `%zu`)
- An unused formatting argument (removed it)
- A missing `%d` (added it)
- A missing `%s` (added it)
- 2 cases of `%llu` for a `uint64_t` (replaced it by `"%" PRIu64`)
- 6 cases of giving a string directly as format string (replaced it by `("%s", string)`)
- 2 cases of %*-s, which should probably have been %-*s.
(Looks like NuttX accepts (the invalid) %*-s, but other platforms don't.)
- A %04x for a `uint32_t` (replaced by "%04" PRIx32)
This uses the systems default shell:
- Ubuntu: dash
- Fedora: bash
Since bash is invoked via /bin/sh, it operates in POSIX mode:
https://tiswww.case.edu/php/chet/bash/POSIX
- remove '# Ignore the expand_aliases command in zshell.'
Not needed because the shell operates in POSIX mode
- [[ is bashism -> use [
- autostart_files=( $autostart_file_match )
is not supported in dash, so use 'ls'
- shellcheck runs the dash flavor, since dash is a minimalistic shell.
Tested on dash & bash.
- Environment variables for building with asan
- Docker argument for leak sanitizer
- Regex for tests to fail on sanitizer error
Signed-off-by: Lukas Woodtli <woodtli.lukas@gmail.com>
The FIFO was created in the server thread, and the PX4 main thread could
already have continued and started to execute the bash script.
In that case the client tried to open the FIFO but it did not exist yet.
Client error:
ERROR [px4_daemon] pipe open fail
ERROR [px4_daemon] Could not send commands
By reading sizeof(packet), it could have happened that 2 packets at once
were read, and the second one was then ignored.
This makes sure that only one packet at a time is read.
The other 2 changes are for efficiency.
It was not enough to lock individual accesses to the maps.
For example it could happen that a thread was started and exited very
quickly, before the pthread_t item was inserted into the map, such
that when the cleanup method was called, the thread and pipe fd were not
found (and fd=0=stdin was closed).
- move posix-configs/SITL/init/{rcS,10016_iris,6011_typhoon_h480} to ROMFS/px4fmu_common/init.d-posix
allows for easier unification, only one symlink is required.
- rcS: add AUTOCNF support. Update scripts to match with behavior of PX4
master (parameter values, some sitl driver got removed)
- add alias to allow 'set variable value' syntax in scripts to px4-alias.sh
- use px4_getopt
- use separate argument for the test_data directory
- append PATH from within the px4 binary: this simplifies the usage
w/o the sitl_run.sh script.
- add 'source px4-alias.sh' to all existing sitl startup scripts
- move sitl mixers to ROMFS/px4fmu_common/mixers-sitl
makes it easier to use existing mixers and sitl-specific ones.
- remove unused rcS_gazebo_delta_wing
squashed & rebased version, not including:
- listener changes
- src/firmware renaming
Commits:
tag_to_version.py: fix Python3 error
subprocess.communicate returns bytes instead of a str which is not the
same for Python3. Therefore, we need to decode the bytes.
cmake: remove folder src/firmware
The folder src/firmware was not intuitive. Why would the binaries for
SITL be inside a src and why even inside a src/firmware folder. Also,
the rootfs was put there which made it even more confusing.
The CMakeLists.txt files are moved into cmake/ and get now called from
the main CMakeLists.txt.
qshell: support for return value
Instead of just sending commands, qshell will now also wait until
the command has finished on QURT and sent back a return value. This will
allow all modules on the DSP side to be spawned from the Linux side
meaning that we only need one config/startup file instead of two.
adb_upload: create folders before pushing
Previously the script failed if the folder on the destination was not
already existing. This therefore makes pushing easier.
posix: spawn PX4 modules in bash
This adds the possibility to spawn PX4 modules out of bash. Basically,
the main executable can now be started as a server/daemon or as a
client.
The server replaces the existing functionality of the main exe with
the pxh shell, however, it also opens a pipe that clients can talk to.
Clients can run or spawn PX4 modules or commands by connecting to the
server over the pipe. They clients will get the stdout and return value
of their commands via a client specific pipe back.
This work will allow to start all modules using a bash script similar to
the way it is done in NuttX where the NuttShell scripts the startup
scripts and starts the modules.
SITL: use new client shell in SITL
This is a first step to use the new shell capabilities for SITL.
The new startup bash script rcS merges (and therefore replaces) the two
existing scripts rcS_gazebo_iris and rcS_jmavsim_iris.
More cleanup will be necessary for the rest of the SITL startup scripts.
Snapdragon: use new shell to start all modules
Instead of different mainapp.config and px4.config files, we can now use
a unified rcS bash script which starts all the modules based on
parameters, mainly the SYS_AUTOSTART param.
Snapdragon: fix the airframe description
pxh: argv needs to end with a nullptr
The comment was wrong that argv needs an additional 0 termination.
Instead it needs a nullptr at the end.
px4_posix_tasks: variable cleanup
The px4_task_spawn_cmd function got a cleanup while debugging, however,
no functional changes.
Snapdragon: move some drivers to 4100 config
These drivers are supported by the community, so they go into the 4100
config.
Snapdragon: update 210qc platform
px4_daemon: use doxygen comments
apps.h_in: fix string printf: use .c_str()
px4_daemon: \b -> \n in printf
px4_daemon: handle error in generate_uuid (close the file on error)
posix main: some clarifications in comment (it's the symlinks not the script aliases)
cmake: remove new install command again
This one was probably wrong and untested. Installing needs revisiting.
POSIX: remove argument USES_TERMINAL
POSIX: copy init and mixer files for SITL
Instead of using non-working install commands, the mixer and startup
files are now copied as part of the build in cmake.
adb_upload.sh: remove leftover commented printf
POSIX main: just the pointer instead of memmove
POSIX main: remove chroot
chroot is removed because it hasn't been used anywhere and seems
untested.
px4_daemon: remove client pipe when cleaning up
px4_daemon: fail if the client pipe already exists
The client pipe is supposed to be specific (by UUID), so the path
shouldn't exist already.
history: limit the number of history entries
This is a protection to avoid filling the memory if we are entering a
lot of commands (e.g. auto-generated).
px4_daemon: add a threadsafe map and use it
px4_daemon: whitespace
px4_daemon: fix client parsing
Sometimes the client ends up reading more than one packet in one read.
The parsing is not made for this and would require a (ring)buffer for
it.
The solution of this commit just reads as much as needed from the pipe
which avoids having to do buffering and parsing.
posix: changes sitl_run.sh and main.cpp cleanup
This changes the paths in sitl_run.sh quite a bit to allow the px4
binary to run in the rootfs directory which should make it convenient
and very close to the NuttX variant.
Also main.cpp got a big cleanup after the big rebase with some
conflicts. Quite some functionality was removed but it has yet to be
seen if it needs to be re-added.
px4_log: cleanup log levels, now they make sense
Before DEBUG and INFO log levels where inverted which didn't make much
sense in my eyes.
dataman: fix path for bash shell
logger: fix paths for bash shell
mavlink: fix paths for bash shell
param: fix path for bash shell
inav: fix paths for bash shell
sdlog2: fix paths for bash shell
ROMFS: add forgotten mixer to list
SITL init: more models, more options
- Support for different models using the unified startup
script rcS.
- Support to choose the estimator by setting the environment variable
PX4_ESTIMATOR.
- Support to choose the logger by setting the environment variable
PX4_LOGGER.
rcS: fix string comparison
listener: use template file
Instead of having all of the C++ code inside the Python file it is
nicer to have a separate template file with the C++ headers, etc.
px4_log: add PX4_INFO_RAW for raw printfs
This allows to do custom formatting but is still transported over
sockets to clients.
topic_listener: use PX4_INFO_RAW instead of printf
commander: use PX4_INFO_RAW for status
listener: rewrite to classes and factory
posix: fix some argument warnings
generate_listener.py: by accident changed shebang
listener: big refactor of the generator
Hopefully this makes it easier to read and change in the future.
rcS: manually take over rebase changes
listener: remove leftover try
listener: properly clean up topic instance
rcS: take over some vehicle specific changes
posix-configs: vehicle specifics to separate files
posix-configs: remove leftover lines
uORBDevices: new PX4_INFO_RAW instead of printf
px4_log: just use printf on NuttX
listener: use less binary space, strip on NuttX
generate_listener.py: remove commented code
cmake: fix syntax error from merge
px4_daemon: fixes after rebase of apps.h/cpp fix
px4_daemon: namespace missing
posix: only create stub for fsync on QURT
unitests: reduce dependencies of param test
This makes the unit test compile and link again after the bash changes.
QURT: some compile fixes after a rebase
SITL: arg change for sitl_run.sh to use rcS_test
This allows to use a custom startup file for testing.
SITL: add the folder test_data
SITL: implement shutdown command as systemcmd
The shutdown command needs to be a proper systemcmd, otherwise the alias
and symlink generation doesn't work and we end up calling shutdown of
the host computer which is to be avoided.
px4fmu_test: same IO_pass mixer as px4fmu_default
px4fmu_test: use normal quad x mixer
There is no good reason to use a specific test mixer, except more cmake
code around it. Therefore just use the same mixer as default, and at
some point px4fmu_test and px4fmu_default can get merged
POSIX: cleanup, dir and symlink fixes
This cleans up the logic behind the symlinking and creating directories.
POSIX: correct arg order in usage info
tests: fix paths for SITL tests
POSIX: printf fix
sitl_run.sh: try to make this run on Mac as well
cmake: try to make jenkins happier
Path cleanup, the bin is no longer in src/firmware
POSIX: fix symlink logic
SITL: prefix all exported env variables
cmake: fix path for ROS tests
integrationtests: fix log path
launch: try to make tets with ROS working again
px4_defines: fix after wrong merge deconflicting
px4_defines: get paths for POSIX correct
cmake: fix cmake arguments
This was fine with cmake 3.6 but did not work with cmake 3.2.2
cmake: use cp instead of cmake -E copy
cmake -E copy does not support copying multiple files with versions <
3.5. Therefore, just use cp for now.
ROMFS: fix build error after rebase
cmake: fix paths in configs
launch: use `spawn_model` again
cmake: various fixes after big rebase
param: path fixes after rebase
posix platform: fixes after rebase
test_mixer: fix screwed up rebase
Most of the incompatitbilities are luckily similar to the darwin build.
- New target OS __PX4_CYGWIN added because in other build environments on Windows defines will very likely be completely different
- added all necessary exeptions to the defines
- disabled priorities completely because on Windows they are defined 1-32 and with all the arbitrary +40 -40 priority settings there were a lot of problems
not only did some threads/"virtual tasks" not start because of out of bound priorities but also the resulting scheduling was totally random and inadequate
with default priorities it ran toally fine during my first tests, should be rethought when windows is used onboard in the future
Daniel Agar
Beat Küng
roangel
Julian Kent
Anthony Lamping
Julian Oes
Matthias Grob
Nico van Duijn
Lorenz Meier
karliss
TSC21
James Goppert
Kārlis Seņko
Mara Bos
dkang
David Sidrane
Bart Slinger
Lukas Woodtli
Roman
Bram Strack van Schijndel
Bob-F
Dennis Mannhart
vkotaru
Elia Tarasov
Nicolas de Palezieux