Leopard Imaging LI-M021C-MIPI Stereo-Optic Cameras

Cameras setup

The camera sensors should be conencted to a ConnectTech's Elroy board.

Features

• V4L2 Kernel Driver Version 2.0 supported on L4T32.2.1
• V4l2 controls
  • test pattern
  • individual gains
  • vertical/horizontal flip
  • flash control
• LibArgus and nvarguscamerasrc
• Resolution supported: 1280x720 @ 60fps
• Gain, exposure, and framerate controls
• Camera synchronization

Capture Tests

Frame-rate Tests

• Set the framerate to 60fps and the driver will configure the sensor:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 aelock=true awblock=true ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! fakesink

• Set the framerate to 45fps and the driver will configure the sensor:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 aelock=true awblock=true ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)45/1' ! fakesink

UDP Streaming Test

Sender Endpoint

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! omxh264enc control-rate=2 bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! rtph264pay mtu=1400 ! udpsink host=$HOST_IP port=5000 sync=false async=false

Receiver Endpoint

gst-launch-1.0 udpsrc port=5000 ! "application/x-rtp,media=(string)video,payload=(int)96,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! queue ! avdec_h264 ! xvimagesink sync=true async=false

Set Controls Test

Run a pipeline, then set gain and exposure controls using v4l2-ctl:

v4l2-ctl -d /dev/video1 -c exposure=14000
v4l2-ctl -d /dev/video1 -c gain=100

Dual Synchronized Capture Test

First run the master pipeline and then the slave pipeline:

Master Pipeline

gst-launch-1.0 nvarguscamerasrc sensor-id=0 aelock=true awblock=true ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! fakesink

Slave Pipeline

gst-launch-1.0 nvarguscamerasrc sensor-id=1 aelock=true awblock=true ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! fakesink

It is not recommended to start both streams at the same time, because nvarguscamerasrc will fail if no buffers arrive on a defined timeout.

V4l2 Capture Test

Master

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap

Slave

v4l2-ctl -d /dev/video1 --set-fmt-video=width=1280,height=720,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap

AE Synchronized

First run the master pipeline and then the slave pipeline:

Master Pipeline

gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! fakesink

Slave Pipeline

gst-launch-1.0 nvarguscamerasrc sensor-id=0 aelock=true awblock=true ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12,framerate=(fraction)60/1' ! fakesink

Run script

./script_to_control_gain_exposure.sh &

Appends

Auto Exposure

The AE controls realays in the feedback provided by the Driver's camera sensor to the nvarguscamerasrc libraries, and the custom DTSIs with the sensor parameters definitions. As the AE has interdependency with the digital gain. This gain it's and operation on the Driver, and follows this setup:

  /*
   * Digital gain equation:
   *
   *  RANGE: 1x, 7.97x
   *  GAIN: VAL / STEPS;
   *  STEPS: 1/32
   *
   *  SCALE FACTOR = 100.000
   *
   *  min_gain_val = 100.000
   *  max_gain_val = 797.000
   *  step_gain_val = 3125
   */

   MT9M021 sensor datasheet:
   * <a href="https://files.niemo.de/aptina_pdfs/MT9M021-M031_Developer_Guide.pdf">MT9M021 Developer Guide</a>

Where, the min_gain_val, max_gain_val, step_gain_val are part of the cameras' DTSI (tegra186-tx2-spiri-camera.dtsi) fixed parameters as per the datasheet. The step_gain_val consists of the steps (1/32 for the register) scaled to a value that makes this fraction become an significant in integer.

Kernel Changes

The Driver for the MT9M021 cameras consists on the following structure, that adds the DTB and Kernel sources, along with its Makefiles that lead its portability to a Kernel source.

. 
├── hardware
│   └── nvidia-spiri
│       └── platform
│           └── t18x
│               ├── common
│               │   └── kernel-dts
│               │       └── t18x-common-platforms
│               │           ├── tegra186-tx2-spiri-camera-base.dtsi
│               │           └── tegra186-tx2-spiri-camera.dtsi
│               └── quill
│                   └── kernel-dts
│                       ├── Makefile
│                       ├── tegra186-tx2-spiri-base.dts
│                       ├── tegra186-tx2-spiri-mPCIe.dts
│                       ├── tegra186-tx2-spiri-revF+.dts
│                       └── tegra186-tx2-spiri-USB3.dts
├── kernel
│   ├── kernel-4.9
│   │   └── arch
│   │       └── arm64
│   │           └── configs
│   │               └── tegra_defconfig
│   └── nvidia-spiri
│       ├── drivers
│       │   └── media
│       │       ├── i2c
│       │       │   ├── Kconfig
│       │       │   ├── Makefile
│       │       │   ├── mt9m021.c
│       │       │   └── mt9m021_mode_tbls.h
│       └── include
│           └── media
│               └── mt9m021.h
└── README.md

In order to add the Driver to the Kernel, the following reference Kernel files are patched for adding custom controls that the camera implements.

• kernel/nvidia/drivers/media/platform/tegra/camera/camera_common.c
• kernel/nvidia/drivers/media/platform/tegra/camera/tegracam_ctrls.c
• kernel/nvidia/include/media/camera_common.h
• kernel/nvidia/include/media/tegra-v4l2-camera.h

Documentation

• CSI2 adapter board guide
• Camera module data sheet
• Camera sensor data sheet
• Camera sensor development guide
• MIPI bridge