Andrew Davison: Research

Real-Time Spherical Mosaicing using Whole Image Alignment (PDF format),
Steven Lovegrove and Andrew J. Davison, ECCV 2010.

September 2010: Adrien Angeli presented our paper at BMVC on live feature clustering. This is work in the direction of understanding how appearance and geometry information can be used in a more unified way in visual SLAM than in current systems where place recognition and loop closure detection is done with bag-of-words type approaches which discard geometry when looking up features which represent a place. Can we embed appearance information in the 3D world model in a more integrated way? Here we show a step towards that, be demonstrating that we can cluster the 3D features obtained from visual SLAM into meaningful clusters in real-time, where cluster membership depends both on appearance similarity and geometrical proximity. These clusters may or may not correspond well to objects in the scene, but certainly represent repeatable structure which should have good properties of viewpoint-invariance and we plan to move on to using them for flexible and efficient place recognition, as well as potentially investigating their use for semantic labelling.

Live Feature Clustering in Video Using Appearance and 3D Geometry (PDF format),
Adrien Angeli and Andrew J. Davison, BMVC 2010.

August 2010: There is currently a funded PhD studentship available under the joint supervision of Prof. Paul Kelly and myself on software tools for heterogeneous multicore/manycore/GPU architectures applied to real-time vision and mapping. This position (which could start soon if we find the right student) is funded by AMD. If you are interested, please contact me or Prof. Kelly directly for more details.

June 2010: Dyson, with whom I've been working recently on robotics and vision applications, are currently looking for a Robotics System Engineer to work at their R&D centre in Malmesbury, UK. Ideally this will be someone with PhD level experience in computer vision or robotics but get in touch for more details. See the advertisement here.

June 2010: I attended Robotics: Science and Systems, which was as usual a very stimulating conference, and particularly enjoyable this year as it was hosted by close colleagues and friends of mine in Zaragoza. We presented a paper by Hauke Strasdat, J. M. M. Montiel and myself called Scale Drift-Aware Large Scale Monocular SLAM. In this work, we have applied an optimisation approach right through the monocular SLAM problem, and studied the particular issues that arise due to the use of a single camera. Since monocular vision cannot measure metric scale, the maps it generates are subject to scale drift which becomes experimentally apparent when large loops are closed. We show how to take account of this during loop optimisation. This work relates closely to the RobotVision software library for SLAM optimisation that we have recently released open source, which includes code to replicate the results in the paper.

Scale Drift-Aware Large Scale Monocular SLAM (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, RSS 2010.

June 2010: We attended CVPR and successfully presented our two papers (see the details below). Richard Newcombe also gave a live demo of his live dense reconstruction system. Here are a couple of example images of single camera reconstructions he did during the demo (click on the images to see large versions):

June 2010: We have just released a new open source (LGPL) software package for vision-based SLAM. The RobotVision library is available from openslam.org and offers various essential elements of a visual SLAM back-end, including bundle adjustment, feature initialisation, pose-graph optimisation and 2D/3D visualisation. This release features software used in our new paper to be presented at RSS 2010 and is largely thanks to the hard work of Hauke Strasdat, supported by Steven Lovegrove and the rest of my group.

May 2010: I am very happy that the paper "Real-Time Monocular SLAM: Why Filter?", which I wrote with Hauke Strasdat and J. M. M. Montiel, has been awarded Best Vision Paper at ICRA 2010.

Real-Time Monocular SLAM: Why Filter? (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, ICRA 2010 (Winner, Best Vision Paper).

April 2010: We have two papers accepted for CVPR 2010.

Richard Newcombe and I have developed a breakthrough method for live dense reconstruction. Our system, which requires just a single standard hand-held video camera attached to a powerful PC, is capable of automatic reconstruction of a desktop-scale scene within just a few seconds as the live camera browses the scene. Without markers or any other infrastructure, the camera's pose is tracked and a 3D point reconstruction made in real-time using monocular SLAM (currently we use Klein and Murray's PTAM). Automatically selected bundles of video frames with pose estimates are then used to generate highly detailed depth maps via dense matching and triangulation, and these are then joined into a globally registered dense mesh reconstruction of the scene's surfaces. This reconstruction is of a level of detail and quality which has not previously been approached by any similar real-time system. The system works in standard everyday scenes and lighting conditions, and is able to reconstruct the details of objects of arbitrary geometry and topology and even those with remarkably low levels of surface texture. The main technical novelty is to use an approximate but smooth base mesh generated from point-based SLAM to enable `view-predictive optical flow': this permits the FlowLib library from Graz University of Technology, running on the GPU, to obtain high precision dense correspondence over the baseline we need for detailed reconstruction. In the video linked below we demonstrate advanced augmented reality where augmentations interact physically with the 3D scene and are correctly clipped by occlusions. We plan to demonstrate the system live at CVPR, and further information is available on Richard's page about the work.

AVI

Live Dense Reconstruction with a Single Moving Camera (PDF format),
Richard A. Newcombe and Andrew J. Davison, CVPR 2010.

And Ankur Handa, Margarita Chli, Hauke Strasdat and I will present new work which extends our previous concept of Active Matching and presents algorithms which make it now applicable as the matching engine in real-time systems aiming at robust matching of hundreds of features per frame. In the Active Matching paradigm, feature matching (within SLAM or similar systems) is performed in a guided, one-by-one manner, where a joint prior on correspondence locations is used to guide incremental image processing and the search for matching consensus. Each feature match improves the predictions of all subsequent feature searches, and decisions are guided by information theoretic measures rather than the random sampling and fixed thresholds of RANSAC or similar. The original AM algorithm (Chli and Davison, ECCV 2008, see below) maintained a joint covariance over the current estimated correspondence locations, and the necessary update of this representation after every feature match limited scalability to a few tens of features per frame in real-time. The new development in our Scalable Active Matching work is to use a general graph-theoretic model of the structure of correspondence priors, and then use graph sparsification either to a tree or a tree of subsets in order to dramatically speed up computation in new algorithms called CLAM and SubAM. The video illustrates the methods in operation to match hundreds of features per frame in the context of our new keyframe SLAM system.

AVI

Scalable Active Matching (PDF format),
Ankur Handa, Margarita Chli, Hauke Strasdat and Andrew J. Davison, CVPR 2010.

February 2010: I only just realised that this is online... a talk I gave when I visited Microsoft Research in Redmond in September 2008.

February 2010: A new paper in collaboration with Hauke Strasdat and J. M. M. Montiel has been accepted for publication at ICRA 2010, and will be presented in one of the prestigious "50 Years of Robotics" special sessions. We think that this is important work which makes a rigorous analysis of the relative merits of real-time monocular SLAM methods based either on filtering (like MonoSLAM or similar systems), or on keyframes and repeated optimisation (like Klein and Murray's PTAM). Which approach provides the best local building block for a monocular SLAM map? Our analysis is based on a measure of accuracy in motion estimation relative to computational cost. It turns out that at current computation levels there is a clear winner: the optimisation method is preferable. This is because real accuracy in motion estimation comes from the use of a large number of feature correspondences, which is computationally much more feasible in the optimisation approach. Filtering on the other hand is better at incorporating information from a large number of camera poses, but this has relatively little effect on accuracy. So you can expect to see much more work based on optimisation coming from my group in the future... though we still think that filtering or hybrid approaches may have an important role to play in high uncertainty situations such as in bootstrapping camera tracking.

Real-Time Monocular SLAM: Why Filter? (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, ICRA 2010 (Winner, Best Vision Paper).

December 2009: Well done to Margarita who passed her PhD viva in great style last week! She is going to stay in my group as a post-doc for the current time.

November 2009: I am very proud that thanks to a lot of hard work, my group managed to submit a fat sheaf of nice papers to CVPR 2010. Fingers crossed for some good reviews come February...

November 2009: I've been thinking a lot over the last couple of years, and discussed with my group and others, that there seems to be a set of common `Quality' principles and methods behind any algorithms in computer vision and robotics that really seem to work. I have tried to write down an initial idea of what those might be and gradually refine it, with the idea that this might one day form the basis of a book.... I have put my current list, in a very raw form, up on this Google site so that it's possible for people to take a look and add comments --- I would be very interested to hear feedback on this!

August 2009: New results in collaboration with Javier Civera and J. M. M. Montiel on applying MonoSLAM-type methods to visual odometry will be presented at IROS 2009 in October. Essentially, this is a "forgetting filter" from which features are deleted once they pass from the field of view. The results approach those obtainable from sliding window bundle adjustment, though presumably will never be quite as accurate. What we do get from filtering is quite nice automatic management of features and track lengths, and the ability to do fully probabilistic outlier rejection.

AVI

1-Point RANSAC for EKF-Based Structure from Motion (PDF format),
Javier Civera, Oscar G. Grasa, Andrew J. Davison and J. M. M. Montiel, IROS 2009.

May 2009: Margarita Chli presented our paper on using image-based mutual information measures to infer the correlation structure of visual maps at ICRA 2009 in Kobe, and Javier Civera presented work on sequential camera self-calibration. See the movies and papers below.

AVI

AVI

Automatically and Efficiently Inferring the Hierarchical Structure of Visual Maps (PDF format),
Margarita Chli and Andrew J. Davison, ICRA 2009.

Camera Self-Calibration for Sequential Bayesian Structure From Motion (PDF format),
Javier Civera, Diana R. Bueno, Andrew J. Davison and J. M. M. Montiel, ICRA 2009(Finalist, Best Vision Paper)

April 2009: A small article (mostly sensible!) about my research appeared this week in The Economist.

November 2008: The Special Issue on Visual SLAM of IEEE Transactions on Robotics which I recently guest-edited with José Neira and John Leonard should now have been published. The full contents are available online here, but you or your institution will need to be an IEEE Xplore subscriber to download the full PDFs for the papers from this site. The guest editorial by José, myself and John is available here:

Guest Editorial Special Issue on Visual SLAM (PDF format),
José Neira, Andrew J. Davison and John J. Leonard, IEEE T-RO 2008.

August 2008: Active Matching: a new algorithm developed with Margarita Chli for efficient probabilistic matching of sets of features in images. It combines the advantages of step-by-step active search guided by information theory with a multiple-hypothesis, mixture of Gaussians model able to cope fully with matching ambiguity. We show large reductions in the number of image processing operations required to achieve consensus matching of sets of features during camera tracking when compared with "match first, then resolve outliers" algorithms such as RANSAC and JCBB.

AVI

Active Matching (PDF format),
Margarita Chli, Andrew J. Davison, ECCV 2008.

June 2008: New results on real-time tracking of motion with extreme dynamics using MonoSLAM with a gigabit ethernet camera at 200Hz, published at RSS 2008 in collaboration with Peter Gemeiner and Marcus Vincze from Vienna University of Technology. We show results of smooth real-time 3D tracking of a vigourously shaken hand-held camera, and one thrown from hand to hand, and present repeatable validation using a robot arm. A constant acceleration motion model is shown to improve performance at this extreme frame-rate.

AVI

Improving Localization Robustness in Monocular SLAM Using a High-Speed Camera (PDF format),
Peter Gemeiner, Andrew J. Davison and Markus Vincze, RSS 2008.

May 2008: One post-doc position and two fully funded PhD candidate positions on high performance visual SLAM now available: please see this advert for details.

April 2008: My work with Javier Civera, Juan Magallón and J. M. M. Montiel (University of Zaragoza) on visual mosaicing using SLAM-like techniques (following on from the `Visual Compass' method published at ICRA 2006) has recently been accepted for publication in the Internional Journal of Computer Vision. The advantage of this technique is that it can build consistent and drift-free mosaics over a full sphere in seamless real-time, thanks to the backbone of a SLAM map of points at infinity. The video below shows a new results sequence captured outside the Royal Albert Hall near to Imperial College.

AVI

Drift-Free Real-Time Sequential Mosaicing (PDF format),
Javier Civera, Andrew J. Davison, J. A. Magallón and J. M. M. Montiel, IJCV 2008 (accepted for publication).

March 2008: Singularity Indicators: a blog-like personal view.

February 2008: New results to be published at ICRA 2008 demonstrating online model selection within monocular SLAM in collaboration with Javier Civera and J. M. M. Montiel (University of Zaragoza). Using an interacting multiple model (IMM) framework, we show that Bayesian model selection can be applied sequentially during real-time monocular SLAM to improve accuracy by automatically detecting periods when the camera is stationary, purely rotating or accelerating with different dynamics. This is particularly useful when tracking starts up without any known objects in the scene and avoids the underestimation of feature depths during low parallax motion.

AVI

Interacting Multiple Model Monocular SLAM (PDF format),
Javier Civera, Andrew J. Davison and J. M. M. Montiel, ICRA 2008.

December 2007: I have been lucky enough to be awarded an ERC Starting Grant from the European Research Council. This is a new Europe-wide scheme of individual grants to fund five year research projects. The focus of my project is highly efficient visual SLAM algorithms able to function at high frame-rates and track dynamic motion. See this article about my project and those of the three other successful candidates from Imperial College. Note that this means that it is a particularly good time time contact me if you are interested in coming to work at Imperial as either a PhD student or post-doc!

November 2007: The deadline for the IEEE T-RO Special Issue on Visual SLAM which I am guest-editing has been extended to December 15th 2007.

September 2007: I gave a tutorial on Visual SLAM at BMVC 2007 at the University of Warwick, together with Andrew Calway and Walterio Mayol from the University of Bristol. Resources from the tutorial are available from this page in Bristol.

August 2007: I have just accepted an invitation to serve as Publicity Chair for Robotics: Science and Systems 2008. This will be the fourth time this prestigious single track international conference will be held, and in 2008 it will move from the US to Europe for the first time to be held in Zurich, Switzerland. The deadline for full paper submissions is January 15th 2008. RSS is a new type of robotics conference with the most comprehensive reviewing process I have encountered at any conference in robotics or computer vision and very high quality papers so please submit your best work!

July 2007: I will be guest-editing a Special Issue on Visual SLAM of IEEE Transactions on Robotics together with José Neira from the University of Zaragoza and John Leonard from MIT. The aim is to produce a comprehensive special issue which showcases the amazing recent advances in this research area. We are currently seeking good quality submissions and the closing date is December 15th 2007. Please see the official call for papers.

June 2007: Some new results published at RSS 2007 on extending the range of monocular SLAM using a hierarchical submapping approach, in collaboration with Laura Clemente, José Neira and Mingo Tardós from the University of Zaragoza and Ian Reid from Oxford. High quality submaps containing around 100 features are built using MonoSLAM with inverse depth feature parameterisation and the Joint Compatibility test to reject outliers. Submaps are chained together to form a large map and loop closures are detected with a map matching test. This approach allows mapping of large outdoor loops with a single hand-held camera, and all the steps of the algorithm will in principle run in real-time with current hardware. The results shown here are for Keble College quad in Oxford, a loop of at least 200m.

June 2007: Javier Civera, J. M. M. Montiel (University of Zaragoza) and I have been working on Dimensionless Monocular SLAM: a formulation of MonoSLAM which uses only dimensionless quantities and makes explicit the fact that a single camera builds maps with undetermined scale. In this formulation, the previous metric tuning parameters of Inverse Depth MonoSLAM (camera acceleration, frame-rate, expected scene depth, etc.) are combined into dimensionless constants which have an appealing intuitive interpretation in image space. This work is a further step towards being able to track any type of image sequence with a MonoSLAM-style probabilistic filtering approach.

Dimensionless Monocular SLAM (PDF format),
Javier Civera, Andrew J. Davison and J. M. M. Montiel, IBPRIA 2007.

November 2006: Excuse me for slipping into blog mode temporarily, but are superhuman AI and the technological singularity really coming in the near future (i.e. the next 20--30 years)? As I see the incredible recent progress in computer processing power, computer vision algorithms, search engines, miniaturisation, nanotechnology and so on it seems more and more feasible to me that they might be. It is hard to even start thinking about all of the implications of this, positive and negative, but I recommend Kurzweil's book and I think this is something that serious scientists and the general public should all be talking about.

September 2006: Real-time MonoSLAM using straight lines as well as point features, in collaboration with Paul Smith and Ian Reid from Oxford. Here efficiently-detected straight line segments in the image are parameterised as 3D lines and included in the SLAM state vector. When used in combination with point features lines can increase robustness and the quality of scene representation.

August 2006: I co-organised the SLAM Summer School 2006, held in Oxford, together with Paul Newman. This was a big success with more than 60 PhD students from around the world attending and a great group of invited lecturers. See the website for more information --- full lecture and practical resources and photos from the event are now available to all online.

May 2006: A new release (confidently labelled 1.0) of the open source SceneLib library for real-time SLAM, which now includes full support for real-time MonoSLAM, is now available from my software page.

May 2006: Together with J. M. M. Montiel and Javier Civera from the University of Zaragoza, I have been working on a new unified parameterisation for EKF-based monocular SLAM which permits the smooth initialisation of features over much wider depth ranges than my previous particle method, and even copes with features potentially "at infinity". The results of this work were presented at the Robotics: Science and Systems conference (RSS 2006) in August. The following movie shows the method at work initialising features during a challenging outdoor motion with some features at very large depths. Some of these features retain very high depth uncertainties throughout the sequence. Note that Matlab-based code implementing MonoSLAM and the inverse depth concept was part of one of the practical exercises at the SLAM Summer School 2006 and can be downloaded from the website.

May 2006: The results of my work with J. M. M. Montiel from the University of Zaragoza, Spain, on using SLAM techniques for real-time rotation estimation and mosaicing were presented at ICRA 2006. In this work, we show that straightforward adoption of an EKF SLAM approach with sparse mapping of the directions of visual features permits fully consistent, real-time pose estimation for a rotating camera over a spherical field of view (including seamless loop closures). The following video shows the use of this technique for mesh-based mosaicing from a freely rotating hand-held camera (the scene is in front of the Royal Albert Hall, very close to Imperial College in London).

October 2005: I gave an oral presentation of my paper "Active Search for Real-Time Vision" at ICCV 2005 in Beijing. I am very excited about this work, which tackles the problem of active measurement selection in real-time tracking (relevant to Single Camera SLAM or other more general tracking scenarios), using information theory to analyse the value and computational cost of measurements.

June 2005: I gave a tutorial on Real-Time Motion and Structure Estimation from Moving Cameras at CVPR in San Diego, together with Dr. David Nister from the University of Kentucky. David's slides in Powerpoint format are available from his website here. My part of the tutorial is available in HTML form here. The movies in my presentation should work in your browser if you have the correct plugins installed, though it may be inconvenient to view it all online because of the large size of the movie files: if this is the case you can download the whole talk as a bundle to unpack on your local computer here. Also available now is a video of my hour-long presentation (note the very large 116Mb size of this file --- please only download if required!) Thank you to Jason Meltzer of UCLA for providing this video.

April 2005: New results in using MonoSLAM perform real-time visual SLAM with the humanoid robot HRP-2. This work was done in collaboration with Olivier Stasse during my visit to the Joint Japanese-French Research Lab (JRL), AIST, Japan. Image capture was via a wide-angle firewire camera fitted to the robot (the robot's standard trinocular rig was not used due to its limited field of view), and vision and SLAM processing was on-board using the robot's internal Linux PC. Real-time GUI and graphical output were achieved over a wireless CORBA link to a separate workstation. The videos below show external and SLAM views of a motion where the robot walked in a small circle around the lab, detecting and mapping natural point features autonomously and re-detecting early features to close the loop at the end of the trajectory (a substantial correction to the map is seen at this stage). The swaying motion of the robot during walking can be clearly seen in the trajectory recovered by SLAM. Improved loop-closing performance was achieved by also integrating output from the three-axis gyro in the robot's chest to reduce orientation uncertainty.

MPEG

MPEG

February 2005: My work on augmented reality with Ian Reid in Oxford was featured in an article from New Scientist.

September 2004: new results in recovering the surface orientations of planar patches in the world and using hand gestures for augmenting the SLAM map, both during real-time single camera SLAM. In the surface orientation work, performed in collaboration with Nick Molton and Ian Reid, an initial hypothesized orientation estimate for each feature patch is refined sequentially during real-time tracking by measuring its warped appearance change using an image alignment technique. The two movies show wire-frame rectangles at the initial orientation estimate and textured patches at the refined estimates, for a planar outdoor scene and a multiplanar indoor scene. Estimating the orientation of a patch in the world means that it becomes more useful as a feature in SLAM because its appearance can be predicted from a wide range of viewpoints and therefore matched robustly. In addition a more comprehensive scene description is built up. In the hand gesture work, done with Walterio Mayol, Ben Tordoff, and David Murray, colour segmentation is used to detect hand gestures so that a user with a wearable camera can annotate a SLAM map in real-time.

March 2004: a big step forward in the quality of results and presentation demonstrated in Real-Time Augmented Reality and Personal Localisation using Single Camera SLAM. This video, produced in collaboration with Nick Molton, Ian Reid, Ben Tordoff and Walterio Mayol, demonstrates the general operation of our single camera localisation and mapping technique and its application to augmented reality and personal localisation. Two new developments beyond the system presented at ICCV2003 here are Nick Molton's algorithm for feature patch transformation (meaning that features can now be matched over larger camera motions; even when upside-down), and the use of a wide-angle lens with non-perspective projection characteristics. The augmented reality demonstration shows virtual "kitchen fitting" as furniture is added interactively to live video (by attaching it with mouse-clicks to automatically-mapped scene features) while the camera continues to move. We also show personal localisation with a passive wearable camera mount built by Walterio Mayol. All image and SLAM processing and rendering runs at 30Hz on a Pentium M 1.6GHz laptop and this video was made by direct screen capture from the system running in real-time. Besides a laptop, the only equipment required is a cheap IEEE1394 "Firewire" webcam.

DivX (80Mb)

Jan 2004: much improved Single Camera SLAM results using a wide-angle lens, in collaboration with Yolanda Gonzalez Cid and Nobuyuki Kita. The camera used was the wide-angle version of the Unibrain Fire-i which has a field of view of just over 90 degrees. The camera was pre-calibrated via a perspective + one parameter radial distortion model. The wide field of view means that features can be seen through much larger ranges of camera movement, and mapping can be sparser and more efficient --- extending the range of application from the desk-top scale to a small room. Camera motion estimation results are noticeably more stable and close to ground truth. As before, all processing here is at 30Hz on a standard laptop. These results will be published at IAV2004 (see paper below).

MPEG Real-Time 3D SLAM with Wide-Angle Vision (PDF format),
Andrew J. Davison, Yolanda Gonzalez Cid and Nobuyuki Kita, IAV 2004.

April 2003: new applications of real-time single camera localisation and SLAM. On the left: real-time localisation with a known map of features using a single camera with a fish-eye lens (joint work with Nobuyuki Kita and Francois Berenger at AIST Japan). This lens has a field of view of around 150 degrees with a spherical projection curve such that image coordinate is proportional to incoming ray angle. Our camera localisation method is easily adapted to this case with a new measurement model. Camera position estimation actually works better using this than a normal perspective lens since the same set of features is visible during larger motions. 30Hz operation, all processing on a 2GHz laptop (this work was demoed at ICCV 2003). On the right: real-time SLAM for a wearable active vision robot built by Walterio Mayol and David Murray. The robot has a miniature IEEE1394 camera with a perspective lens. Output from real-time visual SLAM is used to localise the robot and control its fixation point automatically: the robot's camera can be directed to fixate on any of the feature points in its map as the wearer moves around freely. The wearable results were presented at ISMAR2003 and ISRR2003.