Grabbing 3D Data

The necessary steps to get x, y and z coordinates in mm for all pixels are as follows:

• Open the stereo camera by running the Open command

• Grab an image pair by calling Capture

• Perform depth computation via ComputeDisparityMap

• Convert the disparity values into x, y and z coordinates in mm by calling ComputePointMap

• (Close the camera by calling Close)

After the call to ComputePointMap the PointMap node of the camera holds metric 3D point coordinates in world coordinates.

Code Examples

C++Halcon/HDevelop

struct XYZ {
	float x, y, z;
};
NxLibItem root; // References the tree's root item at path "/"

// replace "1234" with your camera's serial number:
NxLibItem camera = root[itmCameras]["1234"]; // References the camera's item at path "/Cameras/BySerialNo/1234"

// Call the Open command to open the camera
NxLibCommand open(cmdOpen);
// Note: the variable name 'open' does not matter; the constant 'cmdOpen'
// specifies the name of the command to be executed when '.execute()' is called
// on the object.
// Set parameters for the command
open.parameters()[itmCameras] =
    "1234"; // Insert the serial number of the camera to be opened into the Cameras parameter of Open
open.execute();

// Capture a single image, compute the x,y,z coordinates of all pixels and find their average Z value
// Execute the Capture  command with default parameters
NxLibCommand capture(cmdCapture);
capture.execute();

// Now we can compute the disparity map and point cloud. Computing the disparity map is computationally the most demanding
// part. All other computations (such as the generation of the x,y,z data via
// ComputePointMap) are based on the computed
// DisparityMap. It is thus necessary to call ComputeDisparityMap first before calling ComputePointMap.
NxLibCommand computeDisparity(cmdComputeDisparityMap);
computeDisparity.execute();
NxLibCommand computePoints(cmdComputePointMap);
computePoints.execute();
int width, height, channels, bytesPerElement;
bool isFloat;
double timestamp;

// query information about the binary data
camera[itmImages][itmPointMap].getBinaryDataInfo(
    &width, &height, &channels, &bytesPerElement, &isFloat, &timestamp);
assert(
    width > 0 && height > 0 &&
    channels == 3); // the binary data format is fixed, but let's be safe and do some basic checks
assert(isFloat && bytesPerElement == sizeof(float));
std::vector<XYZ> points; // This is where the point cloud data will be copied to
camera[itmImages][itmPointMap].getBinaryData(
    points, &timestamp); // points will now contain x, y and z coordinates for each image pixel
// You can now compute the average z value of all pixels
double z = 0;
int numberOfValidPixels = 0;
for (int y = 0; y < height; y++) {
	for (int x = 0; x < width; x++) {
		float currentZ = points[y * width + x].z;
		if (!std::isnan(currentZ)) {
			// this is a valid pixel
			z += currentZ;
			numberOfValidPixels++;
		} else {
			// depth data could not be reconstructed for this pixel; x, y and z are set to NaN
		}
	}
}
z = z / numberOfValidPixels;