# Andrew Davison: Research

I hold the position of Professor of Robot Vision at the Department of Computing, Imperial College London, and lead the Robot Vision Research Group. I am working in computer vision and robotics: specifically my main research over the past few years has been on SLAM (Simultaneous Localisation and Mapping) using vision, with a particular emphasis on methods that work in real-time with a single camera which can be classed as Real-Time Structure from Motion, or as I would rather call it Single Camera SLAM or MonoSLAM (vision-based, monocular, bearing-only SLAM, in full 3D and without inertial sensing). This is technology that could provide cheap real-time localisation for domestic robots, humanoid robots, wearable sensors, game interfaces or other devices. This work derives from my early research at the University of Oxford where I developed one of the very first real-time SLAM systems for robot navigation, and the first using vision as its primary sensor (ECCV 1998).

Currently my main interests are in improving the performance in terms of dynamics, scale and detail level of real-time visual localisation and mapping. Imperial College Robotics; Imperial College Computer Vision: please contact me if you are interested in coming to Imperial for PhD studies or post-doctoral research as I am always looking for excellent new team members. I apologise that I am not always able to reply to all messages but you can be sure that your message will catch my eye if you have top level mathematics and programming skills and experience which is relevant to the Robot Vision Group's research. Unfortunately we do not normally have positions for interns or short-term visitors.

### Current PhD Students

Ankur Handa: Efficient Image Processing for High Performance SLAM (started October 2008)
Renato Salas-Moreno: Future Computing Architectures for Computer Vision and SLAM (started October 2010; co-supervised with Prof. Paul Kelly)
Jan Jachnik: Light Field SLAM (started October 2011)
Jacek Zienkiewicz: Dense Vision for Mobile Robot Guidance (started April 2012)
Robert Lukierski: Rapid Room Understanding using Wide Angle Vision (started July 2012)
Lukas Platinsky: Dense SLAM Algorithms (TBC!) (joined Robot Vision Group April 2013)
Akis Tsiotsios: Underwater Vision and Photometric Stereo (joined Robot Vision Group November 2012; co-supervised with Dr. T-K Kim)
Ping-Lin Chang: 3D Dense Vision / Surgical Guidance (joined Robot Vision Group January 2013; co-supervised with Dr. Eddie Edwards)

### Current RA

Hanme Kim: Project TBC (started December 2008)

## Some recent research and news

Please see the Robot Vision Research Group Webpage for news on my group's work, or my Google+ page, as I now put updates there.

June 2013 (READVERTISED!! We are still looking for an excellent candidate...): I have a Post-Doc/RA position available in my Robot Vision Group at Imperial College London. It's on an exciting new collaborative project which has the aim of an integrated attack on the software, compiler, runtime/operating systems and architecture challenges in manycore computer systems, driven by 3D scene understanding. Our aim is to use real-time computer vision as a way of pushing the frontiers of the practical architectures and system software at the heart of future mass-market devices --- building the foundation for future applications based on full awareness of the three-dimensional environment. The project is a collaboration with experts all the way through a vertical stack from application to architecture, led by Prof. Paul Kelly at Imperial College, Prof. Michael O'Boyle in Edinburgh and Prof. Steve Furber in Manchester (co-designer of the original ARM microprocessor). Please see this advert for full details; forward to anyone who might be interested and get in touch with any questions.

December 2010: I am transferring more content to our Robot Vision Research Group Webpage, including now a full set of downloadable publications and most recently a videos page where there is a link to the imperialrobotvision channel on youtube.

November 2010: Dyson, with whom I've been working recently on robotics and vision applications, are still expanding their robotics team and are looking for another Robotics System Engineer and also a Robotics Software Engineer, a to work at their R&D centre in Malmesbury, UK. Ideally the Robotics System Engineer will be someone with post-graduate level research experience in computer vision or robotics. Go to this link for more details, or get in touch with me or them if you have questions.

October 2010: Our new Robot Vision Research Group Webpage has now gone online, and I will gradually transfer most of the research content from this page to that site. In the meantime, please take a look.

October 2010: After completing a PhD and one year post-doc here, Margarita Chli has now left my group and joined the Autonomous Systems Lab at ETH Zurich as a Senior Researcher: see her new webpage here. We wish her the best of luck in her new job!

September 2010: At ECCV Steven Lovegrove and I presented our paper on live spherical mosaicing and we also gave a live demo of the system. In this work, we show how to make a mosaicing system which operates sequentially in real-time but also generates globally consistent mosaics over a whole sphere. As video is gathered live from a purely rotating camera held in the hand or on a tripod, two parallel processing threads operate: one tracks the camera pose at frame-rate relative to the current mosaic, stored as a set of registered keyframes, while the second repeatedly globally optimises the keyframe poses and also refines an estimate of camera intrinsics. Both processes are based on whole image alignment on a pyramid rather than feature matching, and Steve's implementation makes heavy use of GPU acceleration throughout the pipeline. Click on the image below for an example high resolution mosaic (note that this image does not necessarily imply that we are Arsenal fans, we just thought it looked good!)

Real-Time Spherical Mosaicing using Whole Image Alignment (PDF format),
Steven Lovegrove and Andrew J. Davison, ECCV 2010.

September 2010: Adrien Angeli presented our paper at BMVC on live feature clustering. This is work in the direction of understanding how appearance and geometry information can be used in a more unified way in visual SLAM than in current systems where place recognition and loop closure detection is done with bag-of-words type approaches which discard geometry when looking up features which represent a place. Can we embed appearance information in the 3D world model in a more integrated way? Here we show a step towards that, be demonstrating that we can cluster the 3D features obtained from visual SLAM into meaningful clusters in real-time, where cluster membership depends both on appearance similarity and geometrical proximity. These clusters may or may not correspond well to objects in the scene, but certainly represent repeatable structure which should have good properties of viewpoint-invariance and we plan to move on to using them for flexible and efficient place recognition, as well as potentially investigating their use for semantic labelling.

Live Feature Clustering in Video Using Appearance and 3D Geometry (PDF format),
Adrien Angeli and Andrew J. Davison, BMVC 2010.

August 2010: There is currently a funded PhD studentship available under the joint supervision of Prof. Paul Kelly and myself on software tools for heterogeneous multicore/manycore/GPU architectures applied to real-time vision and mapping. This position (which could start soon if we find the right student) is funded by AMD. If you are interested, please contact me or Prof. Kelly directly for more details.

June 2010: Dyson, with whom I've been working recently on robotics and vision applications, are currently looking for a Robotics System Engineer to work at their R&D centre in Malmesbury, UK. Ideally this will be someone with PhD level experience in computer vision or robotics but get in touch for more details. See the advertisement here.

June 2010: I attended Robotics: Science and Systems, which was as usual a very stimulating conference, and particularly enjoyable this year as it was hosted by close colleagues and friends of mine in Zaragoza. We presented a paper by Hauke Strasdat, J. M. M. Montiel and myself called Scale Drift-Aware Large Scale Monocular SLAM. In this work, we have applied an optimisation approach right through the monocular SLAM problem, and studied the particular issues that arise due to the use of a single camera. Since monocular vision cannot measure metric scale, the maps it generates are subject to scale drift which becomes experimentally apparent when large loops are closed. We show how to take account of this during loop optimisation. This work relates closely to the RobotVision software library for SLAM optimisation that we have recently released open source, which includes code to replicate the results in the paper.

Scale Drift-Aware Large Scale Monocular SLAM (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, RSS 2010.

June 2010: We attended CVPR and successfully presented our two papers (see the details below). Richard Newcombe also gave a live demo of his live dense reconstruction system. Here are a couple of example images of single camera reconstructions he did during the demo (click on the images to see large versions):

June 2010: We have just released a new open source (LGPL) software package for vision-based SLAM. The RobotVision library is available from openslam.org and offers various essential elements of a visual SLAM back-end, including bundle adjustment, feature initialisation, pose-graph optimisation and 2D/3D visualisation. This release features software used in our new paper to be presented at RSS 2010 and is largely thanks to the hard work of Hauke Strasdat, supported by Steven Lovegrove and the rest of my group.

May 2010: I am very happy that the paper "Real-Time Monocular SLAM: Why Filter?", which I wrote with Hauke Strasdat and J. M. M. Montiel, has been awarded Best Vision Paper at ICRA 2010.

Real-Time Monocular SLAM: Why Filter? (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, ICRA 2010 (Winner, Best Vision Paper).

April 2010: We have two papers accepted for CVPR 2010.

Richard Newcombe and I have developed a breakthrough method for live dense reconstruction. Our system, which requires just a single standard hand-held video camera attached to a powerful PC, is capable of automatic reconstruction of a desktop-scale scene within just a few seconds as the live camera browses the scene. Without markers or any other infrastructure, the camera's pose is tracked and a 3D point reconstruction made in real-time using monocular SLAM (currently we use Klein and Murray's PTAM). Automatically selected bundles of video frames with pose estimates are then used to generate highly detailed depth maps via dense matching and triangulation, and these are then joined into a globally registered dense mesh reconstruction of the scene's surfaces. This reconstruction is of a level of detail and quality which has not previously been approached by any similar real-time system. The system works in standard everyday scenes and lighting conditions, and is able to reconstruct the details of objects of arbitrary geometry and topology and even those with remarkably low levels of surface texture. The main technical novelty is to use an approximate but smooth base mesh generated from point-based SLAM to enable `view-predictive optical flow': this permits the FlowLib library from Graz University of Technology, running on the GPU, to obtain high precision dense correspondence over the baseline we need for detailed reconstruction. In the video linked below we demonstrate advanced augmented reality where augmentations interact physically with the 3D scene and are correctly clipped by occlusions. We plan to demonstrate the system live at CVPR, and further information is available on Richard's page about the work.

AVI

Live Dense Reconstruction with a Single Moving Camera (PDF format),
Richard A. Newcombe and Andrew J. Davison, CVPR 2010.

And Ankur Handa, Margarita Chli, Hauke Strasdat and I will present new work which extends our previous concept of Active Matching and presents algorithms which make it now applicable as the matching engine in real-time systems aiming at robust matching of hundreds of features per frame. In the Active Matching paradigm, feature matching (within SLAM or similar systems) is performed in a guided, one-by-one manner, where a joint prior on correspondence locations is used to guide incremental image processing and the search for matching consensus. Each feature match improves the predictions of all subsequent feature searches, and decisions are guided by information theoretic measures rather than the random sampling and fixed thresholds of RANSAC or similar. The original AM algorithm (Chli and Davison, ECCV 2008, see below) maintained a joint covariance over the current estimated correspondence locations, and the necessary update of this representation after every feature match limited scalability to a few tens of features per frame in real-time. The new development in our Scalable Active Matching work is to use a general graph-theoretic model of the structure of correspondence priors, and then use graph sparsification either to a tree or a tree of subsets in order to dramatically speed up computation in new algorithms called CLAM and SubAM. The video illustrates the methods in operation to match hundreds of features per frame in the context of our new keyframe SLAM system.

AVI

Scalable Active Matching (PDF format),
Ankur Handa, Margarita Chli, Hauke Strasdat and Andrew J. Davison, CVPR 2010.

February 2010: I only just realised that this is online... a talk I gave when I visited Microsoft Research in Redmond in September 2008.

February 2010: A new paper in collaboration with Hauke Strasdat and J. M. M. Montiel has been accepted for publication at ICRA 2010, and will be presented in one of the prestigious "50 Years of Robotics" special sessions. We think that this is important work which makes a rigorous analysis of the relative merits of real-time monocular SLAM methods based either on filtering (like MonoSLAM or similar systems), or on keyframes and repeated optimisation (like Klein and Murray's PTAM). Which approach provides the best local building block for a monocular SLAM map? Our analysis is based on a measure of accuracy in motion estimation relative to computational cost. It turns out that at current computation levels there is a clear winner: the optimisation method is preferable. This is because real accuracy in motion estimation comes from the use of a large number of feature correspondences, which is computationally much more feasible in the optimisation approach. Filtering on the other hand is better at incorporating information from a large number of camera poses, but this has relatively little effect on accuracy. So you can expect to see much more work based on optimisation coming from my group in the future... though we still think that filtering or hybrid approaches may have an important role to play in high uncertainty situations such as in bootstrapping camera tracking.

Real-Time Monocular SLAM: Why Filter? (PDF format),
Hauke Strasdat, J. M. M. Montiel and Andrew J. Davison, ICRA 2010 (Winner, Best Vision Paper).

December 2009: Well done to Margarita who passed her PhD viva in great style last week! She is going to stay in my group as a post-doc for the current time.

November 2009: I am very proud that thanks to a lot of hard work, my group managed to submit a fat sheaf of nice papers to CVPR 2010. Fingers crossed for some good reviews come February...

November 2009: I've been thinking a lot over the last couple of years, and discussed with my group and others, that there seems to be a set of common `Quality' principles and methods behind any algorithms in computer vision and robotics that really seem to work. I have tried to write down an initial idea of what those might be and gradually refine it, with the idea that this might one day form the basis of a book.... I have put my current list, in a very raw form, up on this Google site so that it's possible for people to take a look and add comments --- I would be very interested to hear feedback on this!

August 2009: New results in collaboration with Javier Civera and J. M. M. Montiel on applying MonoSLAM-type methods to visual odometry will be presented at IROS 2009 in October. Essentially, this is a "forgetting filter" from which features are deleted once they pass from the field of view. The results approach those obtainable from sliding window bundle adjustment, though presumably will never be quite as accurate. What we do get from filtering is quite nice automatic management of features and track lengths, and the ability to do fully probabilistic outlier rejection.

AVI

1-Point RANSAC for EKF-Based Structure from Motion (PDF format),
Javier Civera, Oscar G. Grasa, Andrew J. Davison and J. M. M. Montiel, IROS 2009.

May 2009: Margarita Chli presented our paper on using image-based mutual information measures to infer the correlation structure of visual maps at ICRA 2009 in Kobe, and Javier Civera presented work on sequential camera self-calibration. See the movies and papers below.

AVI AVI

Automatically and Efficiently Inferring the Hierarchical Structure of Visual Maps (PDF format),
Margarita Chli and Andrew J. Davison, ICRA 2009.

Camera Self-Calibration for Sequential Bayesian Structure From Motion (PDF format),
Javier Civera, Diana R. Bueno, Andrew J. Davison and J. M. M. Montiel, ICRA 2009(Finalist, Best Vision Paper)

April 2009: A small article (mostly sensible!) about my research appeared this week in The Economist.

November 2008: The Special Issue on Visual SLAM of IEEE Transactions on Robotics which I recently guest-edited with José Neira and John Leonard should now have been published. The full contents are available online here, but you or your institution will need to be an IEEE Xplore subscriber to download the full PDFs for the papers from this site. The guest editorial by José, myself and John is available here:

Guest Editorial Special Issue on Visual SLAM (PDF format),
José Neira, Andrew J. Davison and John J. Leonard, IEEE T-RO 2008.

August 2008: Active Matching: a new algorithm developed with Margarita Chli for efficient probabilistic matching of sets of features in images. It combines the advantages of step-by-step active search guided by information theory with a multiple-hypothesis, mixture of Gaussians model able to cope fully with matching ambiguity. We show large reductions in the number of image processing operations required to achieve consensus matching of sets of features during camera tracking when compared with "match first, then resolve outliers" algorithms such as RANSAC and JCBB.

AVI

Active Matching (PDF format),
Margarita Chli, Andrew J. Davison, ECCV 2008.

June 2008: New results on real-time tracking of motion with extreme dynamics using MonoSLAM with a gigabit ethernet camera at 200Hz, published at RSS 2008 in collaboration with Peter Gemeiner and Marcus Vincze from Vienna University of Technology. We show results of smooth real-time 3D tracking of a vigourously shaken hand-held camera, and one thrown from hand to hand, and present repeatable validation using a robot arm. A constant acceleration motion model is shown to improve performance at this extreme frame-rate.

AVI

Improving Localization Robustness in Monocular SLAM Using a High-Speed Camera (PDF format),
Peter Gemeiner, Andrew J. Davison and Markus Vincze, RSS 2008.

May 2008: One post-doc position and two fully funded PhD candidate positions on high performance visual SLAM now available: please see this advert for details.

April 2008: My work with Javier Civera, Juan Magallón and J. M. M. Montiel (University of Zaragoza) on visual mosaicing using SLAM-like techniques (following on from the `Visual Compass' method published at ICRA 2006) has recently been accepted for publication in the Internional Journal of Computer Vision. The advantage of this technique is that it can build consistent and drift-free mosaics over a full sphere in seamless real-time, thanks to the backbone of a SLAM map of points at infinity. The video below shows a new results sequence captured outside the Royal Albert Hall near to Imperial College.

AVI

Drift-Free Real-Time Sequential Mosaicing (PDF format),
Javier Civera, Andrew J. Davison, J. A. Magallón and J. M. M. Montiel, IJCV 2008 (accepted for publication).

March 2008: Singularity Indicators: a blog-like personal view.

February 2008: New results to be published at ICRA 2008 demonstrating online model selection within monocular SLAM in collaboration with Javier Civera and J. M. M. Montiel (University of Zaragoza). Using an interacting multiple model (IMM) framework, we show that Bayesian model selection can be applied sequentially during real-time monocular SLAM to improve accuracy by automatically detecting periods when the camera is stationary, purely rotating or accelerating with different dynamics. This is particularly useful when tracking starts up without any known objects in the scene and avoids the underestimation of feature depths during low parallax motion.

AVI

Interacting Multiple Model Monocular SLAM (PDF format),
Javier Civera, Andrew J. Davison and J. M. M. Montiel, ICRA 2008.

December 2007: I have been lucky enough to be awarded an ERC Starting Grant from the European Research Council. This is a new Europe-wide scheme of individual grants to fund five year research projects. The focus of my project is highly efficient visual SLAM algorithms able to function at high frame-rates and track dynamic motion. See this article about my project and those of the three other successful candidates from Imperial College. Note that this means that it is a particularly good time time contact me if you are interested in coming to work at Imperial as either a PhD student or post-doc!

November 2007: The deadline for the IEEE T-RO Special Issue on Visual SLAM which I am guest-editing has been extended to December 15th 2007.

September 2007: I gave a tutorial on Visual SLAM at BMVC 2007 at the University of Warwick, together with Andrew Calway and Walterio Mayol from the University of Bristol. Resources from the tutorial are available from this page in Bristol.

August 2007: I have just accepted an invitation to serve as Publicity Chair for Robotics: Science and Systems 2008. This will be the fourth time this prestigious single track international conference will be held, and in 2008 it will move from the US to Europe for the first time to be held in Zurich, Switzerland. The deadline for full paper submissions is January 15th 2008. RSS is a new type of robotics conference with the most comprehensive reviewing process I have encountered at any conference in robotics or computer vision and very high quality papers so please submit your best work!

July 2007: I will be guest-editing a Special Issue on Visual SLAM of IEEE Transactions on Robotics together with José Neira from the University of Zaragoza and John Leonard from MIT. The aim is to produce a comprehensive special issue which showcases the amazing recent advances in this research area. We are currently seeking good quality submissions and the closing date is December 15th 2007. Please see the official call for papers.

June 2007: Some new results published at RSS 2007 on extending the range of monocular SLAM using a hierarchical submapping approach, in collaboration with Laura Clemente, José Neira and Mingo Tardós from the University of Zaragoza and Ian Reid from Oxford. High quality submaps containing around 100 features are built using MonoSLAM with inverse depth feature parameterisation and the Joint Compatibility test to reject outliers. Submaps are chained together to form a large map and loop closures are detected with a map matching test. This approach allows mapping of large outdoor loops with a single hand-held camera, and all the steps of the algorithm will in principle run in real-time with current hardware. The results shown here are for Keble College quad in Oxford, a loop of at least 200m.

Mapping Large Loops with a Single Hand-Held Camera (PDF format),
Laura A. Clemente, Andrew J. Davison, Ian Reid, José Neira and Juan D. Tardós, RSS 2007.

June 2007: Javier Civera, J. M. M. Montiel (University of Zaragoza) and I have been working on Dimensionless Monocular SLAM: a formulation of MonoSLAM which uses only dimensionless quantities and makes explicit the fact that a single camera builds maps with undetermined scale. In this formulation, the previous metric tuning parameters of Inverse Depth MonoSLAM (camera acceleration, frame-rate, expected scene depth, etc.) are combined into dimensionless constants which have an appealing intuitive interpretation in image space. This work is a further step towards being able to track any type of image sequence with a MonoSLAM-style probabilistic filtering approach.

Dimensionless Monocular SLAM (PDF format),
Javier Civera, Andrew J. Davison and J. M. M. Montiel, IBPRIA 2007.

November 2006: Excuse me for slipping into blog mode temporarily, but are superhuman AI and the technological singularity really coming in the near future (i.e. the next 20--30 years)? As I see the incredible recent progress in computer processing power, computer vision algorithms, search engines, miniaturisation, nanotechnology and so on it seems more and more feasible to me that they might be. It is hard to even start thinking about all of the implications of this, positive and negative, but I recommend Kurzweil's book and I think this is something that serious scientists and the general public should all be talking about.

September 2006: Real-time MonoSLAM using straight lines as well as point features, in collaboration with Paul Smith and Ian Reid from Oxford. Here efficiently-detected straight line segments in the image are parameterised as 3D lines and included in the SLAM state vector. When used in combination with point features lines can increase robustness and the quality of scene representation.

AVI
Real-Time Monocular SLAM with Straight Lines (PDF format),
Paul Smith, Ian Reid and Andrew J. Davison, BMVC 2006.

August 2006: I co-organised the SLAM Summer School 2006, held in Oxford, together with Paul Newman. This was a big success with more than 60 PhD students from around the world attending and a great group of invited lecturers. See the website for more information --- full lecture and practical resources and photos from the event are now available to all online.

May 2006: A new release (confidently labelled 1.0) of the open source SceneLib library for real-time SLAM, which now includes full support for real-time MonoSLAM, is now available from my software page.

May 2006: Together with J. M. M. Montiel and Javier Civera from the University of Zaragoza, I have been working on a new unified parameterisation for EKF-based monocular SLAM which permits the smooth initialisation of features over much wider depth ranges than my previous particle method, and even copes with features potentially "at infinity". The results of this work were presented at the Robotics: Science and Systems conference (RSS 2006) in August. The following movie shows the method at work initialising features during a challenging outdoor motion with some features at very large depths. Some of these features retain very high depth uncertainties throughout the sequence. Note that Matlab-based code implementing MonoSLAM and the inverse depth concept was part of one of the practical exercises at the SLAM Summer School 2006 and can be downloaded from the website.

AVI
Unified Inverse Depth Parametrization for Monocular SLAM (PDF format),
J. M. M. Montiel, Javier Civera and Andrew J. Davison, RSS 2006.

May 2006: The results of my work with J. M. M. Montiel from the University of Zaragoza, Spain, on using SLAM techniques for real-time rotation estimation and mosaicing were presented at ICRA 2006. In this work, we show that straightforward adoption of an EKF SLAM approach with sparse mapping of the directions of visual features permits fully consistent, real-time pose estimation for a rotating camera over a spherical field of view (including seamless loop closures). The following video shows the use of this technique for mesh-based mosaicing from a freely rotating hand-held camera (the scene is in front of the Royal Albert Hall, very close to Imperial College in London).

AVI
A Visual Compass based on SLAM (PDF format),
J. M. M. Montiel and Andrew J. Davison, ICRA 2006.

October 2005: I gave an oral presentation of my paper "Active Search for Real-Time Vision" at ICCV 2005 in Beijing. I am very excited about this work, which tackles the problem of active measurement selection in real-time tracking (relevant to Single Camera SLAM or other more general tracking scenarios), using information theory to analyse the value and computational cost of measurements.

Active Search for Real-Time Vision (PDF format), Andrew J. Davison, ICCV 2005.

June 2005: I gave a tutorial on Real-Time Motion and Structure Estimation from Moving Cameras at CVPR in San Diego, together with Dr. David Nister from the University of Kentucky. David's slides in Powerpoint format are available from his website here. My part of the tutorial is available in HTML form here. The movies in my presentation should work in your browser if you have the correct plugins installed, though it may be inconvenient to view it all online because of the large size of the movie files: if this is the case you can download the whole talk as a bundle to unpack on your local computer here. Also available now is a video of my hour-long presentation (note the very large 116Mb size of this file --- please only download if required!) Thank you to Jason Meltzer of UCLA for providing this video.

April 2005: New results in using MonoSLAM perform real-time visual SLAM with the humanoid robot HRP-2. This work was done in collaboration with Olivier Stasse during my visit to the Joint Japanese-French Research Lab (JRL), AIST, Japan. Image capture was via a wide-angle firewire camera fitted to the robot (the robot's standard trinocular rig was not used due to its limited field of view), and vision and SLAM processing was on-board using the robot's internal Linux PC. Real-time GUI and graphical output were achieved over a wireless CORBA link to a separate workstation. The videos below show external and SLAM views of a motion where the robot walked in a small circle around the lab, detecting and mapping natural point features autonomously and re-detecting early features to close the loop at the end of the trajectory (a substantial correction to the map is seen at this stage). The swaying motion of the robot during walking can be clearly seen in the trajectory recovered by SLAM. Improved loop-closing performance was achieved by also integrating output from the three-axis gyro in the robot's chest to reduce orientation uncertainty.

MPEG MPEG

February 2005: My work on augmented reality with Ian Reid in Oxford was featured in an article from New Scientist.

September 2004: new results in recovering the surface orientations of planar patches in the world and using hand gestures for augmenting the SLAM map, both during real-time single camera SLAM. In the surface orientation work, performed in collaboration with Nick Molton and Ian Reid, an initial hypothesized orientation estimate for each feature patch is refined sequentially during real-time tracking by measuring its warped appearance change using an image alignment technique. The two movies show wire-frame rectangles at the initial orientation estimate and textured patches at the refined estimates, for a planar outdoor scene and a multiplanar indoor scene. Estimating the orientation of a patch in the world means that it becomes more useful as a feature in SLAM because its appearance can be predicted from a wide range of viewpoints and therefore matched robustly. In addition a more comprehensive scene description is built up. In the hand gesture work, done with Walterio Mayol, Ben Tordoff, and David Murray, colour segmentation is used to detect hand gestures so that a user with a wearable camera can annotate a SLAM map in real-time.

MPEG MPEG MPEG
Locally Planar Patch Features for Real-Time Structure from Motion (PDF format),
Nicholas Molton, Andrew J. Davison and Ian Reid, BMVC 2004.
Interaction Between Hand and Wearable Camera in 2D and 3D Environments (PDF format),
Walterio Mayol, Andrew J. Davison, Ben Tordoff, Nick Molton and David Murray, BMVC 2004.

March 2004: a big step forward in the quality of results and presentation demonstrated in Real-Time Augmented Reality and Personal Localisation using Single Camera SLAM. This video, produced in collaboration with Nick Molton, Ian Reid, Ben Tordoff and Walterio Mayol, demonstrates the general operation of our single camera localisation and mapping technique and its application to augmented reality and personal localisation. Two new developments beyond the system presented at ICCV2003 here are Nick Molton's algorithm for feature patch transformation (meaning that features can now be matched over larger camera motions; even when upside-down), and the use of a wide-angle lens with non-perspective projection characteristics. The augmented reality demonstration shows virtual "kitchen fitting" as furniture is added interactively to live video (by attaching it with mouse-clicks to automatically-mapped scene features) while the camera continues to move. We also show personal localisation with a passive wearable camera mount built by Walterio Mayol. All image and SLAM processing and rendering runs at 30Hz on a Pentium M 1.6GHz laptop and this video was made by direct screen capture from the system running in real-time. Besides a laptop, the only equipment required is a cheap IEEE1394 "Firewire" webcam.

DivX (80Mb)

Jan 2004: much improved Single Camera SLAM results using a wide-angle lens, in collaboration with Yolanda Gonzalez Cid and Nobuyuki Kita. The camera used was the wide-angle version of the Unibrain Fire-i which has a field of view of just over 90 degrees. The camera was pre-calibrated via a perspective + one parameter radial distortion model. The wide field of view means that features can be seen through much larger ranges of camera movement, and mapping can be sparser and more efficient --- extending the range of application from the desk-top scale to a small room. Camera motion estimation results are noticeably more stable and close to ground truth. As before, all processing here is at 30Hz on a standard laptop. These results will be published at IAV2004 (see paper below).

MPEG Real-Time 3D SLAM with Wide-Angle Vision (PDF format),
Andrew J. Davison, Yolanda Gonzalez Cid and Nobuyuki Kita, IAV 2004.

April 2003: new applications of real-time single camera localisation and SLAM. On the left: real-time localisation with a known map of features using a single camera with a fish-eye lens (joint work with Nobuyuki Kita and Francois Berenger at AIST Japan). This lens has a field of view of around 150 degrees with a spherical projection curve such that image coordinate is proportional to incoming ray angle. Our camera localisation method is easily adapted to this case with a new measurement model. Camera position estimation actually works better using this than a normal perspective lens since the same set of features is visible during larger motions. 30Hz operation, all processing on a 2GHz laptop (this work was demoed at ICCV 2003). On the right: real-time SLAM for a wearable active vision robot built by Walterio Mayol and David Murray. The robot has a miniature IEEE1394 camera with a perspective lens. Output from real-time visual SLAM is used to localise the robot and control its fixation point automatically: the robot's camera can be directed to fixate on any of the feature points in its map as the wearer moves around freely. The wearable results were presented at ISMAR2003 and ISRR2003.

MPEGMPEG
Real-Time Localisation and Mapping with Wearable Active Vision (PDF format),
Andrew J. Davison, Walterio Mayol and David W. Murray, ISMAR 2003