Post
Originally shared by Andrew DavisonWe have what we think are breakthrough results in reconstructing natural looking mosaics from a hand-held event camera (DVS) and no additional sensing.
An event camera is a silicon retina which has no global shutter. Instead, each pixel is an independent and asynchronous brightness sensor which reports an event or spike whenever it measures a threshold change in log intensity along with the microsecond-precise timing of that change.
Our approach to recovering intensity mosaics from this output is an essential SLAM one of interleaved probabilistic filters to track the pure rotation pose of the camera (incrementally, event by event) while reconstructing a gradient map of the scene --- since each event, given an estimate of camera position and velocity relative to the estimated mosaic, improves our estimate of the component of the gradient in the mosaic in the instantaneous direction of motion. The gradient map can then be upgraded to an intensity map using Poisson reconstruction. We think that essentially the same method should extend to 3D depth map estimation and visual odometry in the near future, or other sorts of motion assumption up to generic optical flow estimation.
We can reconstruct mosaics with both higher resolution than the input event stream and huge dynamic range (the video shows log intensity mosaics and gradient maps), and we should be able to track extremely rapid motion against these. We think this really proves that the DVS device is doing what it is supposed to and capturing the really important information in a moving scene --- the changes --- at a hugely reduced data rate compared to standard video, but in such a way that the whole scene can be reconstructed if needed.
This work was led by +Hanme Kim with help from +Ankur Handa (now in Cambridge) and collaboration with Sio-Hoi Ieng and Ryad Benosman from the Institut de Vision in Paris, and will be published at BMVC 2014. Here is the paper: http://www.doc.ic.ac.uk/~ajd/Publications/kim_etal_bmvc2014.pdf
An event camera is a silicon retina which has no global shutter. Instead, each pixel is an independent and asynchronous brightness sensor which reports an event or spike whenever it measures a threshold change in log intensity along with the microsecond-precise timing of that change.
Our approach to recovering intensity mosaics from this output is an essential SLAM one of interleaved probabilistic filters to track the pure rotation pose of the camera (incrementally, event by event) while reconstructing a gradient map of the scene --- since each event, given an estimate of camera position and velocity relative to the estimated mosaic, improves our estimate of the component of the gradient in the mosaic in the instantaneous direction of motion. The gradient map can then be upgraded to an intensity map using Poisson reconstruction. We think that essentially the same method should extend to 3D depth map estimation and visual odometry in the near future, or other sorts of motion assumption up to generic optical flow estimation.
We can reconstruct mosaics with both higher resolution than the input event stream and huge dynamic range (the video shows log intensity mosaics and gradient maps), and we should be able to track extremely rapid motion against these. We think this really proves that the DVS device is doing what it is supposed to and capturing the really important information in a moving scene --- the changes --- at a hugely reduced data rate compared to standard video, but in such a way that the whole scene can be reconstructed if needed.
This work was led by +Hanme Kim with help from +Ankur Handa (now in Cambridge) and collaboration with Sio-Hoi Ieng and Ryad Benosman from the Institut de Vision in Paris, and will be published at BMVC 2014. Here is the paper: http://www.doc.ic.ac.uk/~ajd/Publications/kim_etal_bmvc2014.pdf
Hanme Kim, Ankur Handa, Ryad Benosman, Sio-Hoi Ieng, and Andrew J. Davison. Simultaneous Mosaicing and Tracking with an Event Camera. In Proceedings of the B...
Shared with: Public
Reshared by: Vikram Dhillon
This post was originally on Google+