Research
▼ Integrated Algorithms for HFR-Video Processing
Real-Time High-Frame-Rate Optical Flow Estimation
In this study, we develop a high-frame-rate (HFR) vision system that can estimate the optical flow in real time at 1000 fps for 1024x1024 pixel images via the hardware implementation of an improved optical flow detection algorithm on a high-speed vision platform.
Based on the Lucas-Kanade method, we adopt an improved gradient-based algorithm that can adaptively select a pseudo-variable frame rate according to the amplitude of the estimated optical flow to accurately detect the optical flow for objects moving at high and low speeds in the same image. In the improved algorithm, product sums of only the spatial partial derivatives (Sxx, Sxy, and Syy) are calculated in the same manner as that in the Lucas-Kanade method, whereas product sums of the spatial and temporal partial derivatives (Sxt and Syt) are calculated by accumulating the product sums in a variable interval n, which is automatically adjusted according to the speed of the objects being observed; n is small for objects moving rapidly and large for objects moving slowly. 
We implemented the improved algorithm on a high-speed vision platform H3 Vision and realized 1000 fps real-time optical flow detection for gray-level 1024x1024 pixel images. In this study, we divided a 1024x1024 pixel image into block regions of 32x32 pixels and calculated the optical flows at 1000 fps for all the divided 1024 blocks.
The performance of our developed HFR optical flow system was verified through experimental results for high-speed movements such as a top's spinning motion and a human's pitching motion.
Based on the Lucas-Kanade method, we adopt an improved gradient-based algorithm that can adaptively select a pseudo-variable frame rate according to the amplitude of the estimated optical flow to accurately detect the optical flow for objects moving at high and low speeds in the same image. In the improved algorithm, product sums of only the spatial partial derivatives (Sxx, Sxy, and Syy) are calculated in the same manner as that in the Lucas-Kanade method, whereas product sums of the spatial and temporal partial derivatives (Sxt and Syt) are calculated by accumulating the product sums in a variable interval n, which is automatically adjusted according to the speed of the objects being observed; n is small for objects moving rapidly and large for objects moving slowly.
We implemented the improved algorithm on a high-speed vision platform H3 Vision and realized 1000 fps real-time optical flow detection for gray-level 1024x1024 pixel images. In this study, we divided a 1024x1024 pixel image into block regions of 32x32 pixels and calculated the optical flows at 1000 fps for all the divided 1024 blocks.
The performance of our developed HFR optical flow system was verified through experimental results for high-speed movements such as a top's spinning motion and a human's pitching motion.
 |
MPEG movie(2.2M) A human's pitching motion |
 |
MPEG movie(4.5M) A top's rotation |
Reference
- Idaku Ishii, Taku Taniguchi, Kenkichi Yamamoto, and Takeshi Takaki : High-Frame-Rate Optical Flow System, IEEE Trans. on Circuits and Systems for Video Technology, Vol.22, No.1, pp.105-112 (2012)
- Idaku Ishii, Taku Taniguchi, Kenkichi Yamamoto, and Takeshi Takaki : 1000 fps Real-Time Optical Flow Detection System, Proc. SPIE-IS&T Electronics Imaging 2010 meeting, Vol.7538, pp.75380M-1--75380M-11, 2010.