Intel® RealSense™ Cross Platform API
Intel Realsense Cross-platform API
|
librealsense2 provides flexible model for frame management and synchronization. The document will overview frame memory management, passing frames between threads and synchronization.
The core C++ abstraction when dealing is the rs2::frame
class and the rs2::device::start
method. All other management and synchronization primitives can be derived from those two APIs.
Once you call start, the library will start dispatching new frames from selected device into the callback you provided. The callback will be invoked from the same thread handling the low-level IO ensuring minimal latency. Any object implementing void operator()(rs2::frame)
can be used as a callback. In particular, you can pass an anonymous function (lambda with capture) as the frame callback:
As a side-note, rs2::device::stop
will block until all pending callbacks return. This way within callback scope you can be sure the device object is available.
rs2::frame
is a smart reference to the underlying frame - as long as you hold ownership of the rs2::frame
the underlying memory is exclusively yours and will not be modified or freed.
rs2::frame::get_data
will provide a direct pointer to the buffer provided by the underlying driver stack. No extra memory copies are performed in this case.RS2_FORMAT_RGB8
it is likely librealsense will do the conversion from YUY
format internally) librealsense will store the processing output in an internal buffer, and rs2::frame::get_data
will point to it.rs2::frame
object by moving it out of the callback into some global, thread-safe, data structure. (See below) Moving rs2::frame
does not involve a mem-copy of its content.rs2::frame_queue
but not to rs2::syncer
primitive)rs2::frame
objects in less then the 1000 / fps
milliseconds, you will likely encounter frame drops. These events will be visible in the log, if you decrease the severity to DEBUG level.Callbacks are invoked from an internal thread to minimize latency. If you have a lot of processing to do, or simply want to handle the frame in your main event loop, librealsense provides rs2::frame_queue
primitive to move frames from one thread to another in a thread-safe fashion:
Since rs2::frame_queue
implements operator()
you can also pass the queue directly to start
:
You could also have a separate queue for each stream type:
This is particularly handy if you want to set-up different processing pipeline for each stream type.
There are two common types of applications of the streaming API:
librealsense provides some degree of control over this trade-off using RS2_OPTION_FRAMES_QUEUE_SIZE
option. If you increase this number, your application will consume more memory and some frames might potentially wait in line more time, but frame drops will be less likely to happen. On the flip side, if you decrease this number, you will get frames faster, but if new frame will arrive while you are busy it will get dropped.
Often the input to an image processing application is not simply a frame, but rather a coherent set of frames, preferably taken at the same time. librealsense provides rs2::syncer
primitive to help with this problem:
rs2::syncer
to synchronize between devices, assuming it makes sense.