Video tutorial of OpenCV tracking API, the simplest code ever
This tutorial focus on single target tracking in OpenCV 4 contribution tracking API. The code is included and contains under 60 simple lines and described in the 7 minutes of a youtube video as well. You will learn to use OpenCV tracking API to follow single object in a video sequence.
Let me know what do you think in the comment. Feel free to try the code down here.
Opencv tracking API example goal
The goal of this example is to use a mouse to select the region in the image. This region will use the selected region to initialize the tracker. The selected region will be tracked by the tracker available in the OpenCV tracking module. This module is available in opencv_contrib libraries.
Environment Opencv tracking contribution modules
It is important to install opencv with contribution modules. You can found this tutorial for windows here opencv 4 contrib modules . You need to exclude Gstreamer. This is not needed for Tracking API. This will enable you to use functionality under opencv2/tracking. This library, you can find various single target tracking algorithms like TLD, MIL, Online boosting, Tracking by matching as well as implementation for Kalman filter.Tracking API code description
Opencv tracking API code description
The main tool here is Ptr<Tracker> tracker; . This is class with the implementation of the various tracking algorithms. I am using this Pointer to the class tracker as a global variable. It is a simple way how to share the Pointer to Tracker between function. In other words, I can use the tracker in the whole scope of my program.
Init of Ptr<Tracker>
The global pointer is initialized in the main function by the following code. You can use the initialization of all of these available trackers. Some of them need some input parameters, but some of them are initialized without any parameters to the defaults.
tracker = TrackerCSRT::create();
tracker = TrackerBoosting::create()
tracker = TrackerGOTURN::create();
tracker = TrackerKCF::create()
tracker = TrackerMedianFlow::create()
tracker = TrackerMIL::create()
tracker = TrackerMOSSE::create()
tracker = TrackerTLD::create()
SelectedRoi.displayRoi = 0;
SelectedRoi.trackerReady = 0;
SelectedRoi.initTracker = 0;
Control of the program states
The following code will handles the events by mouse over the window, which display the video called Video.
namedWindow("Video", WINDOW_AUTOSIZE);
setMouseCallback("Video", CallBackF, 0);
CallBackF is able to catch once the right or left mouse button event occured. I will reduce a scope to describe just most important events and concepts.
EVENT_LBUTTONDOWN
Update initial values of the region to be tracked and display ROI by push the left button of the mouse down.
SelectedRoi.initX = x;
SelectedRoi.initY = y;
SelectedRoi.displayRoi = 1;
EVENT_LBUTTONUP
Calculate width and height of the secelted roi based on actial position mines the initial position. Now we receive full information about Rectangle x y width and height to track. We can init tracker.
SelectedRoi.finalWidth = x- SelectedRoi.initX ;
SelectedRoi.finalHeight = y-SelectedRoi.initY ;
SelectedRoi.initTracker = 1;
More about mouse callback can be found in this poset
In this state, the main loop is jumped into condition, where the tracker is initialized with the rectangle selected by mouse Rect2d. The state of the program is now switched to the tracker is ready.
if (SelectedRoi.initTracker == 1) {
tracker->init(img, Rect2d(SelectedRoi.initX, SelectedRoi.initY,
SelectedRoi.finalWidth, SelectedRoi.finalHeight));
SelectedRoi.trackerReady = 1;
}
if (SelectedRoi.trackerReady == 1) {
Rect2d track;
tracker->update(img, track);
rectangle(img, track, Scalar(0, 0, 255), 4, 8, 0);
Mat roi = img(track);
if (roi.cols > 0) {
resize(roi, roi, Size(320, 460));
roi.copyTo(img(Rect(1, 1, 320, 460)));
}
This is main pricnciple.
Code of opencv video tracking tutorial
#include <opencv2/dnn.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio/videoio.hpp>
#include <opencv2/imgcodecs/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/core.hpp>
#include <opencv2/core/utility.hpp>
#include <opencv2/ml.hpp>
#include <opencv2/tracking/kalman_filters.hpp>
#include <opencv2/tracking/tracking.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include "opencv2/video/tracking.hpp"
#include <iostream>
using namespace cv;
using namespace cv::ml;
using namespace std;
Ptr<Tracker> tracker;
struct initRoi {
//initial coordination based on EVENT_LBUTTONDOWN
int initX;
int initY;
// actual coordination
int actualX;
int actualY;
// final coordinates
int finalX;
int finalY;
int finalWidth;
int finalHeight;
int displayRoi;
int trackerReady;
int initTracker;
}SelectedRoi;
static void CallBackF(int event, int x, int y, int flags, void* img) {
Mat& imge = *((Mat*)img);
if (event == EVENT_RBUTTONDOWN) {
cout << "right button " << endl;
return;
}
if (event == EVENT_LBUTTONDOWN) {
SelectedRoi.initX = x;
SelectedRoi.initY = y;
SelectedRoi.displayRoi = 1;
cout << "left button DOWN" << endl;
return;
}
if (event == EVENT_LBUTTONUP) {
SelectedRoi.finalWidth = x- SelectedRoi.initX ;
SelectedRoi.finalHeight = y-SelectedRoi.initY ;
SelectedRoi.initTracker = 1;
cout << "left button UP" << endl;
return;
}
if (event == EVENT_MOUSEMOVE) {
cout << "event mouse move"<< endl;
SelectedRoi.actualX = x;
SelectedRoi.actualY = y;
return;
}
}
void main(){
//VideoCapture cap("tr.mov");
VideoCapture cap(0);
SelectedRoi.displayRoi = 0;
SelectedRoi.trackerReady = 0;
SelectedRoi.initTracker = 0;
tracker = TrackerCSRT::create();
// write output to file
VideoWriter outputVideo;
outputVideo.open("video.vmw", VideoWriter::fourcc('W', 'M', 'V', '2'),
cap.get(CAP_PROP_FPS), Size(1024, 800), true);
for (;;)
{
if (!cap.isOpened()) {
cout << "Video Capture Fail" << endl;
break;
}
else {
Mat img;
cap >> img;
resize(img, img, Size(1024, 800));
namedWindow("Video", WINDOW_AUTOSIZE);
setMouseCallback("Video", CallBackF, 0);
if (SelectedRoi.displayRoi != 0) {
rectangle(img, Rect(SelectedRoi.initX, SelectedRoi.initY, SelectedRoi.actualX
- SelectedRoi.initX, SelectedRoi.actualY - SelectedRoi.initY), Scalar(255, 255, 255), 4, 8, 0);
}
if (SelectedRoi.initTracker == 1) {
tracker->init(img, Rect2d(SelectedRoi.initX, SelectedRoi.initY,
SelectedRoi.finalWidth, SelectedRoi.finalHeight));
SelectedRoi.trackerReady = 1;
}
if (SelectedRoi.trackerReady == 1) {
Rect2d track;
tracker->update(img, track);
rectangle(img, track, Scalar(0, 0, 255), 4, 8, 0);
Mat roi = img(track);
if (roi.cols > 0) {
resize(roi, roi, Size(320, 460));
roi.copyTo(img(Rect(1, 1, 320, 460)));
}
}
outputVideo << img;
imshow("Video", img);
int key2 = waitKey(20);
}
}
}
Nice Post. I like your blog. Thanks for Sharing.
Machine Learning Course in Noida