| Home Introduction User's Guide Download

| developer guide → video processing : |  |

| The background and practical application of video processing 1. Video processing background 2. Reading and writing AVI files 3. Common methods for detection of moving objects 4. Detecting the moving maritime objects

### 1. Video processing background

Nowadays, the design of automated surveillance systems has become so hot in the computer vision field. Furthermore the prevalence of surveillance cameras in banks, stores and residential areas makes it both feasible and meaningful. It would just be amazing that we could use computers to alarm those security problems and make useful statistics through those video cameras. However a powerful surveillance system should fix most of the complex cases, which leads to a lot of expensive and repeated experiments. Packing those frequently used functions and providing a visual tool would make those work more efficient and interesting, meanwhile there is a free scientific software package - Scilab which is well done with matrix operation. Thus it's our dream to implement those video processing functions with Scilab. And we tried our best to do as much work as we could in the past two months.

top

### 2. Reading and writing AVI files

First of all, it's important to provide an interface for reading and writing video files under Scilab. We choose AVI files as our supported video format. In general, AVI files contain multiple streams of different types of data. Most AVI sequences use both audio and video streams. What we are really concerned is video streams.

RIFF ('AVI '
LIST ('hdrl'
'avih'(<Main AVI Header>)
LIST ('strl'
'strh'(<Stream header>)
'strf'(<Stream format>)
[ 'strd'(<additional header data>)]
[ 'strn'(<Stream name>)]
...
)
…
)

LIST ('movi'
{SubChunk | LIST ('rec '
SubChunk1
SubChunk2
…
)
…
}
…
)
['idx1'<AVI Index>]
)

More details about AVI files you can find below:

> http://dev.csdn.net/article/26/26870.shtm

Now we figure out the general AVI file format. Finally we made transplantable codes for reading and writing AVI files by C language, and then we wrote interface program to make them available as Scilab functions. Those codes are finished mainly according to the AVI file format. And in order to seek the frames in the video streams randomly, we make sure that, when the AVI file is opened, the index data is stored in the memory, so that we can quickly choose any position of video frames anytime．

There're some limits about reading the AVI files successfully and effectively.

• The video stream should be an uncompressed one.
• It should have the AVI index.
• Its storage type should be 24.

However, those are both the recommended and widely used characteristics, and it is OK for those temporal experiments.

top

### 3. Common methods for detection of moving objects

Since moving object detection is quite important in surveillance system. Without good moving object detection, everything others are not available. Always we think that moving objects are the most dangerous things which might leads to disaster or insecurity. Here are the common methods for detection of moving object:

1) Temporal differencing is adaptive to changeable environment. But it's hard to detect the moving object integrallty.
2) Background subtraction can detect the moving object integrallty but it's hard to apply this method to changeable environment without adaptive background update.
3) Optical flow can detect the moving objects in the moving background. But it costs so much time that it's always not so practically.

So we focus our design on temporal differencing and background subtraction.
(1) Temporal differencing

stands for the th image ， stands for th image, stands for the subtraction of the two images:

And then choose a threshold ， thresholding ， determine the background pixels and forground pixels as following:

1 stands for forground data while 0 stands for background data.

We pack the temporal differencing in function framesub.

subframe = framesub(newframe , interval , thre)

Because the previous t frames should be recorded for the subtraction, so a list is needed.

Function framesub first inserts the newframe to the frame list which is declared as a global variable, chooses the previous ‘interval' frame, calculates the subtraction of the two frames, and then thresholding it according to the thre. Furthermore, if the number of elements of the frame list is less than ‘interval', then a matrix of zeros will be returned. Otherwise, if the number of elements of the frame list is equal to t, then the ‘oldest' one in the frame list would be deleted. So it's to say, the frame list keeps the ‘interval' number of elements generally.

(2) background subtraction

stands for the th image , b(x,y) stands for the background image, stands for the subtraction of the two images:

And then choose a threshold ， thresholding ， determine the background pixels and forground pixels as following:

1 stands for forground data while 0 stands for background data.

Function OnSubtract can be used for background subtraction.

sub_img = OnSubtract( Img1, Img2, thre)

You can either choose a frame as a background frame or update the background frame by your own background update equation, and pass it to the parameter Img2 to get the binary result.

top

### 4. Detecting the moving maritime objects

Here is a complex example to indicate the useful application of video processing.

(1) Background

Detecting the moving maritime objects is so hard for the sake of the wavelet noise and the moving waves.

First, it's quite hard to find an effective way for telling the man-made objects from the waves in the static image. The effective methods always cost a lot of time, for example, analyzing the texture or fractal geometry of the objects and the waves.

Second, those prominent objects, for example, the ships, are not so outstanding from those moving waves with the common detecting methods of purely temporal differencing or background subtraction.

I would like to introduce you an effective method for detecting those moving maritime objects. However, you should refer to Qiying's graduate design paper –‘Detecting and Tracking of Moving Maritime Objects in Video Sequences' in details.

(2) Main ideas and flowcharts

In the first step, we remove the wavelet waves and find the regions of interests in all the video frames. Here comes the flowchart of the processing:

The experiment results are below:

Here comes the detection of regions of interests. But it's only the result of static processing. Those still objects such as the mountains and rocks are also included. We have to figure out those moving objects by dynamic processing.

Typical temporal differencing method is not so sensitive to the changeable of the maritime background. But it's hard to detect the entire moving object, especially those slow moving ships. Here is an experiment to see those disadvantages:

The 1st frame of the video 　 　 　 　 　 　 The 1500th frame of the video

The result (interval = 3）　 　 　 　 　 　 The result (interval = 5）

However, the intensity difference between different frames reflects the moving characteristic, so an improved temporal differencing method is applied.

The main idea is that determine the moving object by the region not the pixel.

We store the region of interest by its minimum enclosing rectangle. And a modified temporal differencing is used for removing those still regions. That is we count the number of the moving pixels of the region and calculate the proportion of the region.

Here are the rules for determination.

• Dif: The moving points of the region. It's the number of the forground pixels after the temporal differencing in the region.

• S: the area of the region. Known the description of the minimum enclosing rectangle ,then S is calculated:

• Then the moving coefficient is calculated by the formula as following:

,

then we can judge the moving region by thresholding the m ：

L(0<L<1) stands for the moving coefficient threshold. B 1 is the Boolean value. 1 stands for moving region, 0 stands for static region.

Experiment result

From the result, we can see that the modified temporal differencing is quite efficient! The moving maritime objects are well marked.

##

Entry
Home Introduction User's Guide
Function Reference
SIP Functions New Image Func Video Functions
Developer's Guide
Graphical Display Video Processing Modified Functions GUI
Demo
Image Processing Video Processing
Support
Future Work Known Bugs FAQ Acknowledgement About Us

|  |  | About Us | Contact Us | Our University Copyright © 2004-2005 Scilab group of Xiamen University, China |  |