Step 3: (Soution 1) C++ Pipes

Through hours of testing and experimentation, with the much need interjection of knowledge by my advising professor Dr. Gordon Erlebacher, I was finally able to get the libfreenect and PCL/OpenNI libraries to work together in the same program. This feat was achieved through the use of the pipe() and fork () functions in C++.

A pipe is used for one-way communication of a stream of bytes. The command to create a pipe is pipe(), which takes an array of two integers. It fills in the array with two file descriptors that can be used for low-level I/O. A common use of pipes is to send data to or receive data from a program being run as a subprocess. One way of doing this is by using a combination of pipe() (to create the pipe), fork() (to create the subprocess), dup2 (to force the subprocess to use the pipe as its standard input or output channel), and exec (to execute the new program). For more information and examples on pipes, you can visit this websites:

 http://www.unix.com/showthread.php?t=58138
http://www.cs.sunysb.edu/~cse533/asgn1/pipes.html
http://www.ecst.csuchico.edu/~hilzer/csci372/pdf/Chapter7.pdf
http://stackoverflow.com/questions/4812891/fork-and-pipes-in-c
http://www.gnu.org/software/libc/manual/html_node/Creating-a-Pipe.html#Creating-a-Pipe

In my program, I first executed the KinectMotor header file which uses the libfreenect library to move the Kinect Motor to the appropriate angle. The angle is entered through the command line by the user. Once the Kinect motor is moved to the appropriate angle, the angle is then "piped" through the child process to the parent process, where the OpenNISaveFrame header file is constructed and called to control the Kinect's camera and save point clouds from the camera into a .pcd file.Once the point cloud is saved into a file, the process is looped until the user pressed CTRL + C to end the program.


This method has been reliable so far in the tests, but i need to add conditional statements to handle bad data being entered into the program by the user such as the Kinect angle (can only be from -30 to 30 degrees) and filenames (as the angle is the filename of the point cloud .pcd file). Most importantly, for some reason, the OpenNISaveFrame function is executed multiple times per function call. I haven't been able to figure out why this is, but with the naming convention, this is not a major problem as I will be able to get one frame per angle as the new frames will have the same names effectively overwriting itself. Still, this is a major feat for my work thus far and I am proud of this work (for now).

Step 3: Moving the Kinect Motor & Saving Point Clouds to a file using the Microsoft Kinect

Through hours and testing and experimentation, I have come to find out that I can get the Kinect motor to move using the libfreenect library and save point clouds to a file using the OpenNI and PCL libraries. I also found out out that the way to achieve this is very brute force and is a bit inconsistent.

The steps are:

1. Move the Kinect motor to the appropriate angle

2. Save the point cloud to file (run this code 2-3 times to ensure that the connection is established between the OpenNI/PCL code and the Kinect)


The reason that the OpenNI/PCL code does not establish a connection with the Kinect has to do with the signal:

// make callback function from member function
        boost::function<void (const pcl::PointCloud<pcl::PointXYZRGBA>::ConstPtr&)> f =
        boost::bind (&OpenNISaveFrame::save, this, _1);
        // connect callback function for desired signal. In this case its a point cloud with color values
        boost::signals2::connection c = interface->registerCallback (f);

        // start receiving point clouds
        interface->start ();

        // stop the grabber
        interface->stop ();

If the Kinect already has an established signal (which is does with the libfreenect library) the connection never registers a callback to the function, which causes the save function in my code not to be executed. Through testing, I have found out that the execution of the save code multiple times (at least 2-3 times) guarantees that the save function is called at least once. I need to test this more to see why that is the case, but, for now,  I can take point clouds from the environment at different angles using the Kinect.

Step 2: Save a Point Cloud from a Kinect

Because the Point Cloud Library website provides an easy implementation for saving point clouds to a file, which can be found here, creating a class for to accomplish this was very trivial. In my project, I call this class OpenNISaveFrame, which saves the point cloud captured by the Kinect into a binary compressed format and as a .pcd (Point Cloud Data) file.

The part of this step that was not trivial was trying to combine these two classes into one workable program that can be implemented rather quickly. My first attempts at this have fail as the libreenect and OpenNI "fight" over control of the Kinect with their function calls. More technically, when the program wants to move the Kinect's motor, the code establishes a signal to the Kinect in the code via libfreenect functions and moves the Kinect accordingly. If the program wants to save point clouds from the Kinect, the code establishes a different connection via PCL and OpenNI functions and the Kinect responds to that signal accordingly. The problem occurs when both connections are being established in the same program. One of two different complications occurred while testing this problem:

  - The libfreenect connection is established, but the OpenNI/PCL connection throws an Exception as the OpenNI/PCL cannot establish a connection with the Kinect since the libfreenect connection is still sending signals to the Kinect, causing the program to end prematurely.

- The libfreenect connection is established and manually shutdown by the code, which causes a segmentation fault to occur in the program.

Since I believe there is a way to get this two libraries to work concurrently in the same program. I will continue to experiment with the classes and the program and will update this blog accordingly.

*** Update: through more testing, I have come to find out that freenect_sync.h and freenect_sync.c provide a disconnection in the thread between the signal and the Kinect. I need to create a class and immediately implement it. (12/1/13) ***

Step 1: Moving the Kinect Motor

The first step for my project is to get the Kinect Motor to tilt so that it is possible to take point clouds of an environment from different angles. Unfortunately, this is not possible using the Point Cloud Library and through a series of tests, it is also not very effective using the implementation provided by OpenNI in the Linux environment. Based on this observations, I have turn to a older C++ library called libfreenect developed by the OpenKinect community. With this library, I was able to create a C++ class called KinectMotor.h. For the purpose of my project, this class will deal with any adjustments needed to the hardware components of the Kinect such as moving and tilting the Kinect angle and changing the LED light on the Kinect.

Based on some experimentations, I have come to find out that the Kinect can only tilt between -31 and 31 degrees, with 0 as the reset angle. I should conduct some timing on the code to see how long it takes for the commands to take over the Kinect and move the device accordingly.

Next, I need to combine the C++ libraries OpenNI and libfreenect in the same program, as I will need the libfreenect library to move the Kinect to the appropriate angle and the OpenNI library to save the point clouds to a file to be processed later.

Research Overview

My name is Brandon Johnson. I am a first year graduate student in the Department of Scientific Computing at Florida State University. My purpose for creating this blog is to keep an up-to-date development log on my research involving the Microsoft Kinect and the Point Cloud Library. I plan on presenting this research at the campus-wide expo called DIGITECH and the department's Computational Expo. I also plan on publishing a paper on my work done from now to April or May 2014.