Main MRPT website > C++ reference
MRPT logo
Public Types | Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes

mrpt::hwdrivers::CKinect Class Reference


Detailed Description

A class for grabing "range images", intensity images and other information from an Xbox Kinect sensor.

Configuration and usage:


Data is returned as observations of type mrpt::slam::CObservation3DRangeScan (and mrpt::slam::CObservationIMU for accelerometers data). See those classes for documentation on their fields.

As with any other CGenericSensor class, the normal sequence of methods to be called is:

Calibration parameters


For an accurate transformation of depth images to 3D points, you'll have to calibrate your Kinect, and supply the following threee pieces of information (default calibration data will be used otherwise, but they'll be not optimal for all sensors!):

Coordinates convention


The origin of coordinates is the focal point of the depth camera, with the axes oriented as in the diagram shown in mrpt::slam::CObservation3DRangeScan. Notice in that picture that the RGB camera is assumed to have axes as usual in computer vision, which differ from those for the depth camera.

The X,Y,Z axes used to report the data from accelerometers coincide with those of the depth camera (e.g. the camera standing on a table would have an ACC_Z=-9.8m/s2).

Some general comments


Converting to 3D point cloud


You can convert the 3D observation into a 3D point cloud with this piece of code:

Then the point cloud mrpt::slam::CColouredPointsMap can be converted into an OpenGL object for rendering with mrpt::slam::CMetricMap::getAs3DObject() or alternatively with:

    mrpt::opengl::CPointCloudColouredPtr gl_points = mrpt::opengl::CPointCloudColoured::Create();
    gl_points->loadFromPointsMap(&pntsMap);

Raw depth to range conversion


At construction, this class builds an internal array for converting raw 10 or 11bit depths into ranges in meters. Users can read that array or modify it (if you have a better calibration, for example) by calling CKinect::getRawDepth2RangeConversion(). If you replace it, remember to set the first and last entries (index 0 and KINECT_RANGES_TABLE_LEN-1) to zero, to indicate that those are invalid ranges.

kinect_depth2range_10bit.png

R(d) = k3 * tan(d/k2 + k1);
k1 = 1.1863, k2 = 2842.5, k3 = 0.1236

Platform-specific comments


For more details, refer to libfreenect documentation:

Format of parameters for loading from a .ini file


  PARAMETERS IN THE ".INI"-LIKE CONFIGURATION STRINGS:
 -------------------------------------------------------
   [supplied_section_name]
    sensorLabel     = KINECT       // A text description
    preview_window  = false        // Show a window with a preview of the grabbed data in real-time

    device_number   = 0           // Device index to open (0:first Kinect, 1:second Kinect,...)

    grab_image      = true        // Grab the RGB image channel? (Default=true)
    grab_depth      = true        // Grab the depth channel? (Default=true)
    grab_3D_points  = true        // Grab the 3D point cloud? (Default=true) If disabled, points can be generated later on.
    grab_IMU        = true        // Grab the accelerometers? (Default=true)

    // Calibration matrix of the RGB camera:
    rgb_cx        = 328.9427     // (cx,cy): Optical center, pixels
    rgb_cy        = 267.4807
    rgb_fx        = 529.2151     // (fx,fy): Focal distance, pixels
    rgb_fy        = 525.5639

    // Calibration matrix of the Depth camera:
    d_cx          = 339.3078     // (cx,cy): Optical center, pixels
    d_cy          = 242.7391
    d_fx          = 594.2143     // (fx,fy): Focal distance, pixels
    d_fy          = 591.0405

    // The relative pose of the RGB camera wrt the depth camera.
    //  (See mrpt::slam::CObservation3DRangeScan for a 3D diagram of this pose)
    relativePoseIntensityWRTDepth  =  [0 -0.02 0 -90 0 -90]   //  [x(m) y(m) z(m) yaw(deg) pitch(deg) roll(deg)]

    pose_x=0    // Camera position in the robot (meters)
    pose_y=0
    pose_z=0
    pose_yaw=0  // Angles in degrees
    pose_pitch=0
    pose_roll=0

More references to read:

Definition at line 194 of file CKinect.h.

#include <mrpt/hwdrivers/CKinect.h>

Inheritance diagram for mrpt::hwdrivers::CKinect:
Inheritance graph
[legend]

List of all members.

Public Types

typedef float TDepth2RangeArray [KINECT_RANGES_TABLE_LEN]
 A typedef for an array that converts raw depth to ranges in meters.

Public Member Functions

 CKinect ()
 Default ctor.
 ~CKinect ()
 Default ctor.
virtual void initialize ()
 Initializes the 3D camera - should be invoked after calling loadConfig() or setting the different parameters with the set*() methods.
virtual void doProcess ()
 To be called at a high rate (>XX Hz), this method populates the internal buffer of received observations.
void getNextObservation (mrpt::slam::CObservation3DRangeScan &out_obs, bool &there_is_obs, bool &hardware_error)
 The main data retrieving function, to be called after calling loadConfig() and initialize().
void getNextObservation (mrpt::slam::CObservation3DRangeScan &out_obs, mrpt::slam::CObservationIMU &out_obs_imu, bool &there_is_obs, bool &hardware_error)
 This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
virtual void setPathForExternalImages (const std::string &directory)
 Set the path where to save off-rawlog image files (this class DOES take into account this path).
Sensor parameters (alternative to \a loadConfig ) and manual control
void open ()
 Try to open the camera (set all the parameters first!) - users may also call initialize(), which in turn calls this.
bool isOpen () const
 Whether there is a working connection to the sensor.
void close ()
 Close the conection to the sensor (not need to call it manually unless desired for some reason, since it's called at destructor)
void setDeviceIndexToOpen (int index)
 Set the sensor index to open (if there're several sensors attached to the computer); default=0 -> the first one.
int getDeviceIndexToOpen () const
void setTiltAngleDegrees (double angle)
 Change tilt angle.
double getTiltAngleDegrees ()
void enablePreviewRGB (bool enable=true)
 Default: disabled.
void disablePreviewRGB ()
bool isPreviewRGBEnabled () const
void setPreviewDecimation (size_t decimation_factor)
 If preview is enabled, show only one image out of N (default: 1=show all)
size_t getPreviewDecimation () const
double getMaxRange () const
 Get the maximum range (meters) that can be read in the observation field "rangeImage".
size_t getRowCount () const
 Get the row count in the camera images, loaded automatically upon camera open().
size_t getColCount () const
 Get the col count in the camera images, loaded automatically upon camera open().
const mrpt::utils::TCameragetCameraParamsIntensity () const
 Get a const reference to the depth camera calibration parameters.
void setCameraParamsIntensity (const mrpt::utils::TCamera &p)
const mrpt::utils::TCameragetCameraParamsDepth () const
 Get a const reference to the depth camera calibration parameters.
void setCameraParamsDepth (const mrpt::utils::TCamera &p)
void setRelativePoseIntensityWrtDepth (const mrpt::poses::CPose3D &p)
 Set the pose of the intensity camera wrt the depth camera.
const mrpt::poses::CPose3DgetRelativePoseIntensityWrtDepth () const
TDepth2RangeArraygetRawDepth2RangeConversion ()
 Get a reference to the array that convert raw depth values (10 or 11 bit) into ranges in meters, so it can be read or replaced by the user.
const TDepth2RangeArraygetRawDepth2RangeConversion () const
void enableGrabRGB (bool enable=true)
 Enable/disable the grabbing of the RGB channel.
bool isGrabRGBEnabled () const
void enableGrabDepth (bool enable=true)
 Enable/disable the grabbing of the depth channel.
bool isGrabDepthEnabled () const
void enableGrabAccelerometers (bool enable=true)
 Enable/disable the grabbing of the inertial data.
bool isGrabAccelerometersEnabled () const
void enableGrab3DPoints (bool enable=true)
 Enable/disable the grabbing of the 3D point clouds.
bool isGrab3DPointsEnabled () const

Protected Member Functions

virtual void loadConfig_sensorSpecific (const mrpt::utils::CConfigFileBase &configSource, const std::string &section)
 Loads specific configuration for the device from a given source of configuration parameters, for example, an ".ini" file, loading from the section "[iniSection]" (see utils::CConfigFileBase and derived classes)

Exceptions:
Thismethod must throw an exception with a descriptive message if some critical parameter is missing or has an invalid value.

Protected Attributes

mrpt::poses::CPose3D m_sensorPoseOnRobot
bool m_preview_window
 Show preview window while grabbing.
size_t m_preview_window_decimation
 If preview is enabled, only show 1 out of N images.
size_t m_preview_decim_counter_range
size_t m_preview_decim_counter_rgb
mrpt::gui::CDisplayWindowPtr m_win_range
mrpt::gui::CDisplayWindowPtr m_win_int
mrpt::utils::TCamera m_cameraParamsRGB
 Params for the RGB camera.
mrpt::utils::TCamera m_cameraParamsDepth
 Params for the Depth camera.
mrpt::poses::CPose3D m_relativePoseIntensityWRTDepth
 See mrpt::slam::CObservation3DRangeScan for a diagram of this pose.
double m_maxRange
 Sensor max range (meters)
int m_user_device_number
 Number of device to open (0:first,...)
bool m_grab_image
bool m_grab_depth
bool m_grab_3D_points
bool m_grab_IMU
 Default: all true.

Private Member Functions

void calculate_range2meters ()
 Compute m_range2meters at construction.

Private Attributes

std::vector< uint8_t > m_buf_depth
std::vector< uint8_t > m_buf_rgb
 Temporary buffers for image grabbing.
TDepth2RangeArray m_range2meters
 The table raw depth -> range in meters.

Member Typedef Documentation

typedef float mrpt::hwdrivers::CKinect::TDepth2RangeArray[KINECT_RANGES_TABLE_LEN]

A typedef for an array that converts raw depth to ranges in meters.

Definition at line 199 of file CKinect.h.


Constructor & Destructor Documentation

mrpt::hwdrivers::CKinect::CKinect ( )

Default ctor.

mrpt::hwdrivers::CKinect::~CKinect ( )

Default ctor.


Member Function Documentation

void mrpt::hwdrivers::CKinect::calculate_range2meters ( ) [private]

Compute m_range2meters at construction.

void mrpt::hwdrivers::CKinect::close ( )

Close the conection to the sensor (not need to call it manually unless desired for some reason, since it's called at destructor)

void mrpt::hwdrivers::CKinect::disablePreviewRGB ( ) [inline]

Definition at line 271 of file CKinect.h.

virtual void mrpt::hwdrivers::CKinect::doProcess ( ) [virtual]

To be called at a high rate (>XX Hz), this method populates the internal buffer of received observations.

This method is mainly intended for usage within rawlog-grabber or similar programs. For an alternative, see getNextObservation()

Exceptions:
Thismethod must throw an exception with a descriptive message if some critical error is found.
See also:
getNextObservation

Implements mrpt::hwdrivers::CGenericSensor.

void mrpt::hwdrivers::CKinect::enableGrab3DPoints ( bool  enable = true) [inline]

Enable/disable the grabbing of the 3D point clouds.

Definition at line 317 of file CKinect.h.

void mrpt::hwdrivers::CKinect::enableGrabAccelerometers ( bool  enable = true) [inline]

Enable/disable the grabbing of the inertial data.

Definition at line 313 of file CKinect.h.

void mrpt::hwdrivers::CKinect::enableGrabDepth ( bool  enable = true) [inline]

Enable/disable the grabbing of the depth channel.

Definition at line 309 of file CKinect.h.

void mrpt::hwdrivers::CKinect::enableGrabRGB ( bool  enable = true) [inline]

Enable/disable the grabbing of the RGB channel.

Definition at line 305 of file CKinect.h.

void mrpt::hwdrivers::CKinect::enablePreviewRGB ( bool  enable = true) [inline]

Default: disabled.

Definition at line 270 of file CKinect.h.

const mrpt::utils::TCamera& mrpt::hwdrivers::CKinect::getCameraParamsDepth ( ) const [inline]

Get a const reference to the depth camera calibration parameters.

Definition at line 291 of file CKinect.h.

const mrpt::utils::TCamera& mrpt::hwdrivers::CKinect::getCameraParamsIntensity ( ) const [inline]

Get a const reference to the depth camera calibration parameters.

Definition at line 287 of file CKinect.h.

size_t mrpt::hwdrivers::CKinect::getColCount ( ) const [inline]

Get the col count in the camera images, loaded automatically upon camera open().

Definition at line 284 of file CKinect.h.

int mrpt::hwdrivers::CKinect::getDeviceIndexToOpen ( ) const [inline]

Definition at line 263 of file CKinect.h.

double mrpt::hwdrivers::CKinect::getMaxRange ( ) const [inline]

Get the maximum range (meters) that can be read in the observation field "rangeImage".

Definition at line 279 of file CKinect.h.

void mrpt::hwdrivers::CKinect::getNextObservation ( mrpt::slam::CObservation3DRangeScan out_obs,
mrpt::slam::CObservationIMU out_obs_imu,
bool &  there_is_obs,
bool &  hardware_error 
)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Note:
This method also grabs data from the accelerometers, returning them in out_obs_imu
void mrpt::hwdrivers::CKinect::getNextObservation ( mrpt::slam::CObservation3DRangeScan out_obs,
bool &  there_is_obs,
bool &  hardware_error 
)

The main data retrieving function, to be called after calling loadConfig() and initialize().

Parameters:
out_obsThe output retrieved observation (only if there_is_obs=true).
there_is_obsIf set to false, there was no new observation.
hardware_errorTrue on hardware/comms error.
See also:
doProcess
size_t mrpt::hwdrivers::CKinect::getPreviewDecimation ( ) const [inline]

Definition at line 276 of file CKinect.h.

TDepth2RangeArray& mrpt::hwdrivers::CKinect::getRawDepth2RangeConversion ( ) [inline]

Get a reference to the array that convert raw depth values (10 or 11 bit) into ranges in meters, so it can be read or replaced by the user.

If you replace it, remember to set the first and last entries (index 0 and KINECT_RANGES_TABLE_LEN-1) to zero, to indicate that those are invalid ranges.

Definition at line 301 of file CKinect.h.

const TDepth2RangeArray& mrpt::hwdrivers::CKinect::getRawDepth2RangeConversion ( ) const [inline]

Definition at line 302 of file CKinect.h.

const mrpt::poses::CPose3D& mrpt::hwdrivers::CKinect::getRelativePoseIntensityWrtDepth ( ) const [inline]

Definition at line 296 of file CKinect.h.

size_t mrpt::hwdrivers::CKinect::getRowCount ( ) const [inline]

Get the row count in the camera images, loaded automatically upon camera open().

Definition at line 282 of file CKinect.h.

double mrpt::hwdrivers::CKinect::getTiltAngleDegrees ( )
virtual void mrpt::hwdrivers::CKinect::initialize ( ) [virtual]

Initializes the 3D camera - should be invoked after calling loadConfig() or setting the different parameters with the set*() methods.

Exceptions:
Thismethod must throw an exception with a descriptive message if some critical error is found.

Reimplemented from mrpt::hwdrivers::CGenericSensor.

bool mrpt::hwdrivers::CKinect::isGrab3DPointsEnabled ( ) const [inline]

Definition at line 318 of file CKinect.h.

bool mrpt::hwdrivers::CKinect::isGrabAccelerometersEnabled ( ) const [inline]

Definition at line 314 of file CKinect.h.

bool mrpt::hwdrivers::CKinect::isGrabDepthEnabled ( ) const [inline]

Definition at line 310 of file CKinect.h.

bool mrpt::hwdrivers::CKinect::isGrabRGBEnabled ( ) const [inline]

Definition at line 306 of file CKinect.h.

bool mrpt::hwdrivers::CKinect::isOpen ( ) const

Whether there is a working connection to the sensor.

bool mrpt::hwdrivers::CKinect::isPreviewRGBEnabled ( ) const [inline]

Definition at line 272 of file CKinect.h.

virtual void mrpt::hwdrivers::CKinect::loadConfig_sensorSpecific ( const mrpt::utils::CConfigFileBase configSource,
const std::string &  section 
) [protected, virtual]

Loads specific configuration for the device from a given source of configuration parameters, for example, an ".ini" file, loading from the section "[iniSection]" (see utils::CConfigFileBase and derived classes)

Exceptions:
Thismethod must throw an exception with a descriptive message if some critical parameter is missing or has an invalid value.

Implements mrpt::hwdrivers::CGenericSensor.

void mrpt::hwdrivers::CKinect::open ( )

Try to open the camera (set all the parameters first!) - users may also call initialize(), which in turn calls this.

Raises an exception upon error.

Exceptions:
std::exceptionA textual description of the error.
void mrpt::hwdrivers::CKinect::setCameraParamsDepth ( const mrpt::utils::TCamera p) [inline]

Definition at line 292 of file CKinect.h.

void mrpt::hwdrivers::CKinect::setCameraParamsIntensity ( const mrpt::utils::TCamera p) [inline]

Definition at line 288 of file CKinect.h.

void mrpt::hwdrivers::CKinect::setDeviceIndexToOpen ( int  index) [inline]

Set the sensor index to open (if there're several sensors attached to the computer); default=0 -> the first one.

Definition at line 262 of file CKinect.h.

virtual void mrpt::hwdrivers::CKinect::setPathForExternalImages ( const std::string &  directory) [virtual]

Set the path where to save off-rawlog image files (this class DOES take into account this path).

An empty string (the default value at construction) means to save images embedded in the rawlog, instead of on separate files.

Exceptions:
std::exceptionIf the directory doesn't exists and cannot be created.

Reimplemented from mrpt::hwdrivers::CGenericSensor.

void mrpt::hwdrivers::CKinect::setPreviewDecimation ( size_t  decimation_factor) [inline]

If preview is enabled, show only one image out of N (default: 1=show all)

Definition at line 275 of file CKinect.h.

void mrpt::hwdrivers::CKinect::setRelativePoseIntensityWrtDepth ( const mrpt::poses::CPose3D p) [inline]

Set the pose of the intensity camera wrt the depth camera.

See also:
See mrpt::slam::CObservation3DRangeScan for a 3D diagram of this pose

Definition at line 295 of file CKinect.h.

void mrpt::hwdrivers::CKinect::setTiltAngleDegrees ( double  angle)

Change tilt angle.

Note:
Sensor must be open first.

Member Data Documentation

std::vector<uint8_t> mrpt::hwdrivers::CKinect::m_buf_depth [private]

Definition at line 371 of file CKinect.h.

std::vector<uint8_t> mrpt::hwdrivers::CKinect::m_buf_rgb [private]

Temporary buffers for image grabbing.

Definition at line 371 of file CKinect.h.

Params for the Depth camera.

Definition at line 361 of file CKinect.h.

Params for the RGB camera.

Definition at line 360 of file CKinect.h.

Definition at line 368 of file CKinect.h.

Definition at line 368 of file CKinect.h.

Definition at line 368 of file CKinect.h.

Default: all true.

Definition at line 368 of file CKinect.h.

Sensor max range (meters)

Definition at line 364 of file CKinect.h.

Definition at line 343 of file CKinect.h.

Definition at line 343 of file CKinect.h.

Show preview window while grabbing.

Definition at line 341 of file CKinect.h.

If preview is enabled, only show 1 out of N images.

Definition at line 342 of file CKinect.h.

The table raw depth -> range in meters.

Definition at line 372 of file CKinect.h.

See mrpt::slam::CObservation3DRangeScan for a diagram of this pose.

Definition at line 362 of file CKinect.h.

Definition at line 339 of file CKinect.h.

Number of device to open (0:first,...)

Definition at line 366 of file CKinect.h.

Definition at line 344 of file CKinect.h.

Definition at line 344 of file CKinect.h.




Page generated by Doxygen 1.7.3 for MRPT 0.9.4 SVN: at Sat Mar 26 06:16:28 UTC 2011