The functionality resides in cvaux library. To use it in your application, place #include "cvaux.h" in your source files and:
Calculates disparity for stereo-pair
cvFindStereoCorrespondence( const CvArr* leftImage, const CvArr* rightImage, int mode, CvArr* depthImage, int maxDisparity, double param1, double param2, double param3, double param4, double param5 );
The function cvFindStereoCorrespondence
calculates disparity map
for two rectified grayscale images.
Example. Calculating disparity for pair of 8-bit color images
/*---------------------------------------------------------------------------------*/ IplImage* srcLeft = cvLoadImage("left.jpg",1); IplImage* srcRight = cvLoadImage("right.jpg",1); IplImage* leftImage = cvCreateImage(cvGetSize(srcLeft), IPL_DEPTH_8U, 1); IplImage* rightImage = cvCreateImage(cvGetSize(srcRight), IPL_DEPTH_8U, 1); IplImage* depthImage = cvCreateImage(cvGetSize(srcRight), IPL_DEPTH_8U, 1); cvCvtColor(srcLeft, leftImage, CV_BGR2GRAY); cvCvtColor(srcRight, rightImage, CV_BGR2GRAY); cvFindStereoCorrespondence( leftImage, rightImage, CV_DISPARITY_BIRCHFIELD, depthImage, 50, 15, 3, 6, 8, 15 ); /*---------------------------------------------------------------------------------*/
And here is the example stereo pair that can be used to test the example
Calculates scanlines coordinates for two cameras by fundamental matrix
void cvMakeScanlines( const CvMatrix3* matrix, CvSize img_size, int* scanlines1, int* scanlines2, int* lengths1, int* lengths2, int* line_count );
The function cvMakeScanlines
finds coordinates of scanlines for two images.
This function returns the number of scanlines. The function does nothing except
calculating the number of scanlines if the pointers scanlines1
or scanlines2
are
equal to zero.
Rectifies image
void cvPreWarpImage( int line_count, IplImage* img, uchar* dst, int* dst_nums, int* scanlines );
The function cvPreWarpImage
rectifies the image so that the scanlines in the
rectified image are horizontal. The output buffer of size
max(width,height)*line_count*3
must be allocated before calling the function.
Retrieves scanlines from rectified image and breaks them down into runs
void cvFindRuns( int line_count, uchar* prewarp1, uchar* prewarp2, int* line_lengths1, int* line_lengths2, int* runs1, int* runs2, int* num_runs1, int* num_runs2 );
The function cvFindRuns
retrieves scanlines from the rectified image and breaks
each scanline down into several runs, that is, series of pixels of almost the
same brightness.
Finds correspondence between two sets of runs of two warped images
void cvDynamicCorrespondMulti( int line_count, int* first, int* first_runs, int* second, int* second_runs, int* first_corr, int* second_corr );
The function cvDynamicCorrespondMulti
finds correspondence between two sets of
runs of two images. Memory must be allocated before calling this function.
Memory size for one array of correspondence information is
max( width,height )* numscanlines*3*sizeof ( int )
.
Calculates coordinates of scanlines of image from virtual camera
void cvMakeAlphaScanlines( int* scanlines1, int* scanlines2, int* scanlinesA, int* lengths, int line_count, float alpha );
(0.0 - 1.0)
.
The function cvMakeAlphaScanlines
finds coordinates of scanlines for the virtual
camera with the given camera position.
Memory must be allocated before calling this function. Memory size for the array
of correspondence runs is numscanlines*2*4*sizeof(int)
. Memory size for the
array of the scanline lengths is numscanlines*2*4*sizeof(int)
.
Morphs two pre-warped images using information about stereo correspondence
void cvMorphEpilinesMulti( int line_count, uchar* first_pix, int* first_num, uchar* second_pix, int* second_num, uchar* dst_pix, int* dst_num, float alpha, int* first, int* first_runs, int* second, int* second_runs, int* first_corr, int* second_corr );
(0.0 - 1.0)
.
The function cvMorphEpilinesMulti
morphs two pre-warped images using information
about correspondence between the scanlines of two images.
Warps rectified morphed image back
void cvPostWarpImage( int line_count, uchar* src, int* src_nums, IplImage* img, int* scanlines );
The function cvPostWarpImage
warps the resultant image from the virtual camera by
storing its rows across the scanlines whose coordinates are calculated by
cvMakeAlphaScanlines.
Deletes moire in given image
void cvDeleteMoire( IplImage* img );
The function cvDeleteMoire
deletes moire from the given image. The post-warped
image may have black (un-covered) points because of possible holes between
neighboring scanlines. The function deletes moire (black pixels) from the image
by substituting neighboring pixels for black pixels. If all the scanlines are
horizontal, the function may be omitted.
The section discusses functions for tracking objects in 3d space using a stereo camera. Besides C API, there is DirectShow 3dTracker filter and the wrapper application 3dTracker. Here you may find a description how to test the filter on sample data.
Simultaneously determines position and orientation of multiple cameras
CvBool cv3dTrackerCalibrateCameras(int num_cameras, const Cv3dTrackerCameraIntrinsics camera_intrinsics[], CvSize checkerboard_size, IplImage *samples[], Cv3dTrackerCameraInfo camera_info[]);
The function cv3dTrackerCalibrateCameras
searches for a checkerboard of the specified size in each
of the images. For each image in which it finds the checkerboard, it fills
in the corresponding slot in camera_info
with the position
and orientation of the camera
relative to the checkerboard and sets the valid
flag.
If it finds the checkerboard in all the images, it returns true;
otherwise it returns false.
This function does not change the members of the camera_info
array
that correspond to images in which the checkerboard was not found.
This allows you to calibrate each camera independently, instead of
simultaneously.
To accomplish this, do the following:
valid
flags before calling this function the first time;valid
flags after each call.
When all the valid
flags are set, calibration is complete.
valid
flags and
use the return value to decide when calibration is complete.
Determines 3d location of tracked objects
int cv3dTrackerLocateObjects(int num_cameras, int num_objects, const Cv3dTrackerCameraInfo camera_info[], const Cv3dTracker2dTrackedObject tracking_info[], Cv3dTrackerTrackedObject tracked_objects[]);
tracked_objects
.)
const Cv3dTracker2dTrackedObject tracking_info[num_cameras][num_objects]
.
The id
field of any unused slots must be -1. Ids need not
be ordered or consecutive.
The function cv3dTrackerLocateObjects
determines the 3d position of tracked objects
based on the 2d tracking information from multiple cameras and
the camera position and orientation information computed by
3dTrackerCalibrateCameras.
It locates any objects with the same id
that are tracked by more
than one camera.
It fills in the tracked_objects
array and
returns the number of objects located. The id
fields of
any unused slots in tracked_objects
are set to -1.
The functions described in this section do PCA analysis and compression for a set of 8-bit images that may not fit into memory all together. If your data fits into memory and the vectors are not 8-bit (or you want a simpler interface), use cvCalcCovarMatrix, cvSVD and cvGEMM to do PCA
Calculates covariance matrix for group of input objects
void cvCalcCovarMatrixEx( int object_count, void* input, int io_flags, int iobuf_size, uchar* buffer, void* userdata, IplImage* avg, float* covar_matrix );
IplImage
input objects or to the read
callback function according to the value of the parameter ioFlags
.
The function cvCalcCovarMatrixEx
calculates a covariance matrix of the input
objects group using previously calculated averaged object. Depending on ioFlags
parameter it may be used either in direct access or callback mode. If ioFlags
is
not CV_EIGOBJ_NO_CALLBACK
, buffer must be allocated before calling the
function.
Calculates orthonormal eigen basis and averaged object for group of input objects
void cvCalcEigenObjects( int nObjects, void* input, void* output, int ioFlags, int ioBufSize, void* userData, CvTermCriteria* calcLimit, IplImage* avg, float* eigVals );
IplImage
input objects or to the read
callback function according to the value of the parameter ioFlags
.
NULL
.
The function cvCalcEigenObjects
calculates orthonormal eigen basis and the
averaged object for a group of the input objects. Depending on ioFlags
parameter
it may be used either in direct access or callback mode. Depending on the
parameter calcLimit
, calculations are finished either after first
calcLimit.max_iter
dominating eigen objects are retrieved or if the ratio of the
current eigenvalue to the largest eigenvalue comes down to calcLimit.epsilon
threshold. The value calcLimit -> type
must be CV_TERMCRIT_NUMB,
CV_TERMCRIT_EPS
, or CV_TERMCRIT_NUMB | CV_TERMCRIT_EPS
. The function returns
the real values calcLimit->max_iter
and calcLimit->epsilon
.
The function also calculates the averaged object, which must be created previously. Calculated eigen objects are arranged according to the corresponding eigenvalues in the descending order.
The parametereigVals
may be equal to NULL
, if eigenvalues are not needed.
The function cvCalcEigenObjects
uses the function
cvCalcCovarMatrixEx.
Calculates decomposition coefficient of input object
double cvCalcDecompCoeff( IplImage* obj, IplImage* eigObj, IplImage* avg );
The function cvCalcDecompCoeff
calculates one decomposition coefficient of the
input object using the previously calculated eigen object and the averaged
object.
Calculates all decomposition coefficients for input object
void cvEigenDecomposite( IplImage* obj, int eigenvec_count, void* eigInput, int ioFlags, void* userData, IplImage* avg, float* coeffs );
IplImage
input objects or to the read
callback function according to the value of the parameter ioFlags
.
The function cvEigenDecomposite
calculates all decomposition coefficients for the
input object using the previously calculated eigen objects basis and the
averaged object. Depending on ioFlags
parameter it may be used either in direct
access or callback mode.
Calculates object projection to the eigen sub-space
void cvEigenProjection( void* input_vecs, int eigenvec_count, int io_flags, void* userdata, float* coeffs, IplImage* avg, IplImage* proj );
IplImage
input objects or to a
callback function, depending on io_flags
.
The function cvEigenProjection
calculates an object projection to the eigen
sub-space or, in other words, restores an object using previously calculated
eigen objects basis, averaged object, and decomposition coefficients of the
restored object. Depending on io_flags
parameter it may be used either in direct
access or callback mode.
In order to support embedded models the user must define structures to represent 1D HMM and 2D embedded HMM model.
Embedded HMM Structure
typedef struct _CvEHMM { int level; int num_states; float* transP; float** obsProb; union { CvEHMMState* state; struct _CvEHMM* ehmm; } u; } CvEHMM;
level ==0
, HMM is most external. In 2D HMM
there are two types of HMM: 1 external and several embedded. External HMM has
level ==1
, embedded HMMs have level ==0
.
(num_state×num_state )
.
For representation of observations the following structure is defined:
Image Observation Structure
typedef struct CvImgObsInfo { int obs_x; int obs_y; int obs_size; float** obs; int* state; int* mix; } CvImgObsInfo;
obs_x*obs_y
.
Creates 2D embedded HMM
CvEHMM* cvCreate2DHMM( int* stateNumber, int* numMix, int obsSize );
stateNumber [0]+1
.
The function cvCreate2DHMM
returns the created structure of the type CvEHMM with
specified parameters.
Releases 2D embedded HMM
void cvRelease2DHMM(CvEHMM** hmm );
The function cvRelease2DHMM
frees all the memory used by HMM and clears the
pointer to HMM.
Creates structure to store image observation vectors
CvImgObsInfo* cvCreateObsInfo( CvSize numObs, int obsSize );
CV_COUNT_OBS( roi, dctSize, delta, numObs )
, where roi,
dctSize, delta, numObs
are the pointers to structures of the type CvSize . The
pointer roi
means size of roi
of image observed, numObs
is the output parameter
of the macro.
The function cvCreateObsInfo
creates new structures to store image observation
vectors. For definitions of the parameters roi, dctSize
, and delta
see the
specification of The function cvImgToObs_DCT
.
Releases observation vectors structure
void cvReleaseObsInfo( CvImgObsInfo** obsInfo );
The function cvReleaseObsInfo
frees all memory used by observations and clears
pointer to the structure CvImgObsInfo .
Extracts observation vectors from image
void cvImgToObs_DCT( IplImage* image, float* obs, CvSize dctSize, CvSize obsSize, CvSize delta );
The function cvImgToObs_DCT
extracts observation vectors, that is, DCT
coefficients, from the image. The user must pass obsInfo.obs
as the parameter
obs
to use this function with other HMM functions and use the structure obsInfo
of the CvImgObsInfo type.
Calculating Observations for HMM
CvImgObsInfo* obs_info; ... cvImgToObs_DCT( image,obs_info->obs, //!!! dctSize, obsSize, delta );
Performs uniform segmentation of image observations by HMM states
void cvUniformImgSegm( CvImgObsInfo* obsInfo, CvEHMM* hmm );
The function cvUniformImgSegm
segments image observations by HMM states uniformly
(see Initial Segmentation for 2D Embedded HMM for 2D embedded HMM with 5
superstates and 3, 6, 6, 6, 3 internal states of every corresponding
superstate).
Initial Segmentation for 2D Embedded HMM
Segments all observations within every internal state of HMM by state mixture components
void cvInitMixSegm( CvImgObsInfo** obsInfoArray, int numImg, CvEHMM* hmm );
The function cvInitMixSegm
takes a group of observations from several training
images already segmented by states and splits a set of observation vectors
within every internal HMM state into as many clusters as the number of mixture
components in the state.
Estimates all parameters of every HMM state
void cvEstimateHMMStateParams( CvImgObsInfo** obsInfoArray, int numImg, CvEHMM* hmm );
The function cvEstimateHMMStateParams
computes all inner parameters of every HMM
state, including Gaussian means, variances, etc.
Computes transition probability matrices for embedded HMM
void cvEstimateTransProb( CvImgObsInfo** obsInfoArray, int numImg, CvEHMM* hmm );
The function cvEstimateTransProb
uses current segmentation of image observations
to compute transition probability matrices for all embedded and external HMMs.
Computes probability of every observation of several images
void cvEstimateObsProb( CvImgObsInfo* obsInfo, CvEHMM* hmm );
The function cvEstimateObsProb
computes Gaussian probabilities of each observation
to occur in each of the internal HMM states.
Executes Viterbi algorithm for embedded HMM
float cvEViterbi( CvImgObsInfo* obsInfo, CvEHMM* hmm );
The function cvEViterbi
executes Viterbi algorithm for embedded HMM. Viterbi
algorithm evaluates the likelihood of the best match between the given image
observations and the given HMM and performs segmentation of image observations
by HMM states. The segmentation is done on the basis of the match found.
Segments observations from all training images by mixture components of newly assigned states
void cvMixSegmL2( CvImgObsInfo** obsInfoArray, int numImg, CvEHMM* hmm );
The function cvMixSegmL2
segments observations from all training images by mixture
components of newly Viterbi algorithm-assigned states. The function uses
Euclidean distance to group vectors around the existing mixtures centers.