Deferred 3D Processing

It is possible to defer the 3D processing of the images. This allows time critical applications to capture stereo image pairs quickly and do the computationally demanding stereo matching later. The procedure is as follows:

Image Acquisition

• Use the Capture or Trigger and Retrieve commands to acquire an image pair

• Retrieve the binary image data from the Images/Raw nodes and store them in your application’s memory or file.

• Retrieve the calibration data from the Calibration node and store it with the image pair into memory or file.

• Start over to capture the next image until your entire sequence is recorded

Stereo Processing

There are two possible ways of processing the saved images. Either you load the images back from memory, or you create a file camera from the saved images, which is preferred.

• From memory:

  • Load a stored image pair from your application’s memory into the Images/Raw nodes of the camera.

  • Restore the matching calibration data into the Calibration node.

  • Compute disparity and point maps using ComputeDisparityMap and ComputePointMap.

  • Start over to continue with the next stored image pair.

• From a file camera:

  • create a file camera out of the saved file with the CreateCamera Command.

  • operate with the file camera.

Note

Although the camera calibration is fixed it is necessary to store the calibration data with every image in order to obtain accurate reconstructions. This is necessary to correctly compensate temperature deformations of the camera. Dynamic calibration effects are currently tracked within the Dynamic node of the camera’s calibration parameters.

Note

By storing the entire Calibration subtree you are safe to obtain the very same reconstructions offline as online, also in future software revisions. Currently it is sufficient to save the entire Calibration subtree once, and only store the Dynamic parameter subtree for every image pair. Before processing the image pairs you can then restore the Calibration subtree (or leave it as it is, if you’re using the same camera as for capturing), and then set the Dynamic parameters for each image pair before doing the stereo matching.

Code Examples

The C++ sample shows the preferred way. Saving the images as files and create a file camera out of it. The Halcon Script shows how to set the Images/Raw nodes from the images stored in memory.

C++Halcon/HDevelop

NxLibItem root; // References the tree's root item at path "/"

// replace "1234" with your camera's serial number:
NxLibItem camera =
    root[itmCameras][itmBySerialNo]["1234"]; // References the camera's item at path "/Cameras/BySerialNo/1234"

// (1) Capture 10 images and store them together with their calibration data into files.

int NumberOfImagesToCapture = 10;
for (int i = 0; i < NumberOfImagesToCapture; i++) {
	// Execute the Capture command with default parameters
	NxLibCommand capture(cmdCapture);
	capture.parameters()[itmSerialNumber] = "1234";
	capture.execute();

	// Create filenames to save the binary data and their calibration data
	std::string imageLeftFileName = std::to_string(i) + "_left.png";
	std::string imageRightFileName = std::to_string(i) + "_right.png";
	std::string calibrationFileName = std::to_string(i) + "_calib.json";

	// Invoke the save image command to save binary data into files
	NxLibCommand saveImage(cmdSaveImage);

	// Saving the left image
	saveImage.parameters()[itmFilename] = imageLeftFileName;
	saveImage.parameters()[itmNode] = camera[itmImages][itmRaw][itmLeft].path;
	saveImage.execute();

	// Saving the right image
	saveImage.parameters()[itmFilename] = imageRightFileName;
	saveImage.parameters()[itmNode] = camera[itmImages][itmRaw][itmRight].path;
	saveImage.execute();

	// Save the calibration data as json formatted std::string into a file
	std::ofstream calibrationFile(calibrationFileName);
	if (!calibrationFile) {
		std::cerr << "Could not open or create file: " << calibrationFileName << std::endl;
		break;
	}
	calibrationFile << camera[itmCalibration].asJson();
}

// (2) Store all the files within a folder and open the folder as a file camera. You can now work with the file
// camera created from the saved raw images and their corresponding calibration files.

// ...

Note

Some subnodes are read-only so we need to specify the onlyWriteableNodes parameter of setJson as true so that the function will not fail on read-only nodes but just continue to restore all writable parameters.