Welcome to the PRoVisG Mars 3D Challenge!

The Mars 3D Challenge aims at testing and improving the state of the art in visual odometry and 3D terrain reconstruction in planetary exploration.

The task of the challenge is to reconstruct depth, camera trajectory and 3D map of Mars landscape observed by the Mars Exploration Rovers (MER). Results of the challenge will help to develop technology for future planetary mission such as ESA ExoMars.

The challenge consists of three stages. The results of the challenge were evaluated by a the PRoVisG Project consortium including the JPL NASA operating the MERs. Results of the challenge are presented at a ICCV 2011 workshop "CVVT:E2M - Computer Vision in Vehicle Technology: From Earth to Mars" and published in a follow-up journal paper.

The winners of this challenge are members of the IMAGINE Group. IMAGINE is a joint project of the École des Ponts ParisTech (ENPC) and the French Scientific and Technical Centre for Building (CSTB). IMAGINE is now part of the Center for Visual Computing (CVC), in association with the École Centrale de Paris (ECP), and it is part as well of the Computer Science lab (LIGM) of University Paris Est (UPE). Some participants to the challenge are also partly supported by Mikros Image.

IMAGINE has been working for several years on dense multi-view stereovision. The main focus of the group has been on high precision 3D surface reconstruction from images, targeting large-scale data sets taken under uncontrolled conditions.

IMAGINE currently hold the best results worldwide on the Strecha et al. reference benchmark, with the most complete and the most precise reconstructions. One of the key and original components of the IMAGINE pipeline is a variational mesh refinement that reprojects mesh hypotheses into the original images to improve photo consistency. This expertise and software has been recently transferred to the startup company Acute3D, powering Autodesk's 123D Catch (formerly project Photofly), a web service to create 3D models from photographs.

IMAGINE is currently working on improving calibration using statistical methods, e.g., in the framework of the Callisto project, also in association with the CNES in the context of the MISS project. IMAGINE is interested in other sensors too, such as lasers and Kinect, as well semantization, e.g., for high-level building model reconstruction.

For more details, please visit http://imagine.enpc.fr/.

Here are a few reconstruction samples:

One key step in the development of equipment and technology to be used for robotic planetary exploration is their test in relevant environments. To this effect, field trials will be conducted in the El Teide National Park in Tenerife (Spain) during September 2011. Its unique features, combining a flat landscape and crater-like dips with scarce vegetation make it an ideal location to test the imaging equipment and processing techniques that have been developed in the framework of the ProvisG programme.

Live broadcasting and more information about the event can be found here.

A press day is scheduled for the 16^th of September. Please contact Dan Mosely for attendance: daniel.mosely (at) astrium.eads.net; +44 (0) 1488778180.

For further information please contact the PRoVisG Management: gerhard.paar (at) joanneum.at

The simultaneously held PRoVisG Summer School in Berlin will give students the unique opportunity to obtain and process data directly from the field trial.<-->

Compact catadioptrical cameras and Time-Of-Flight (TOF) cameras show a high potential for autonomous vehicle navigation. For the European Project "PRoVisG", CSEM has developed a miniature omniview camera with a horizontal field of view of 360° and a vertical field of view of 70°.  With two such cameras stacked vertically, the overlapping vertical field of view can be used to calculate distance maps from stereo vision. The stereo omniview camera system and an additional 3D-TOF camera were successfully integrated on a rover during first field trials on Earth.

Read more here.

Read more here.

(image credit: European Commission, Enterprise and Industry)

In the mid-term of the current 7th Research Framework Programme, the European Commission, supported by the Hungarian EU Presidency, organised an international Conference on Space Research to provide the opportunity for Community financed research consortia to present their achievements from first years of space research under FP7.
Lets embrace space will take place on 12-13 May 2011 at the Marriott in Budapest, and will cover all the topics covered by the FP7 Space Work Programme.

PRoVisG co-ordinator Gerhard Paar from Joanneum Research / Graz, Austria will present the PRoVisG Project in the frame of the afternoon session on Space exploration and Space Science on Thursday 12, 2011.

The PRoVisG consortium is proud to present the first of a series of quarterly newsletters. The EC FP7-SPACE Project PRoVisG brings together major EU and US research institutions and stakeholders involved in space robotic vision and navigation to develop a unified approach to robotic vision ground processing. PRoVisG will build a unified European framework for Robotic Vision Ground Processing. State-of-art computer vision technology will be collected inside and outside Europe to better exploit the image data gathered during future robotic space missions to the Moon and the Planets. This will lead to a significant enhancement of the scientific, technologic and educational outcome of such missions. For details please visit our webpage www.provisg.eu.

The PRoVisG partners are:

• Joanneum Research, Graz, Austria (coordinator)
• Aberystwyth University, United Kingdom
• German Aerospace Center, Berlin, Germany
• Czech Technical University, Prague, Czech
• SciSys, Bristol, United Kingdom
• EADS Astrium, Stevenage, United Kingdom
• Technical University of Berlin, Berlin, Germany
• University College London, United Kingdom
• Ohio State University, Columbus, United States
• University of Surrey, Guildford, United Kingdom
• University of Nottingham, Nottingham, United Kingdom
• CSEM, Neuchatel, Switzerland
• CNES, Toulouse, France

Associated:

• JPL, Pasadena, United States
• University of Leicester, United Kingdom

Science and Technology

Field Test, Data Collection and 3D Vision Processing at Clarach Bay

From July 7-9, a field test on the 'eyes' for new Mars rovers was conducted by PRoVisG partners at Clarach Bay, a sunny beach in Aberystwyth. The Bridget prototype rover was put through her paces testing sophisticated camera hardware and software and collecting 3D vision data.

In the field test, a new camera sensor suite, designed and built by CSEM, was introduced. The sensor suite consisted of two miniature omni-view cameras for stereo vision and one 3D-time-of-flight (TOF) camera. The stereovision pair of cameras was capable of capturing 1024x1024 images at 37fps. The 3D-TOF camera was based on the latest MESA Imaging technology capable of accurately sensing surrounding objects up to a few meters. The sensor suite was mounted on Bridget for 3D data acquisition of motion sequences in real time.

Other experiments and 3D data acquisitions include: image acquisition for AU PanCam color calibration, imaging and 3D reconstruction of panoramas from Bridget and tripod, SLAM image data acquisition from webcam, rehearsal of AMASE campaign operations, imaging of a motion sequence using AU WAC pair on Bridget, and DLR HRC image acquisition on 3 scientifically interesting targets (landslide, rocks, and layered cliff).

Aberystwyth University contributes to Exomars

A BEACH in West Wales doubled for the surface of Mars today as a state-of-the-art robot was put through its paces.Clarach Bay was chosen as the place to test the vision of the new mars rover 'Bridget' which is being developed by academics at nearby Aberystwyth University.The university is leading the development of the Panoramic Camera (PanCam) for the robot, which is due to be sent to the red planet in a joint ESA/ NASA ExoMars mission scheduled for 2018. Professor David Barnes said the field trials went according to plan.

See the press release on http://www.walesonline.co.uk/news/wales-news/2010/07/14/mars-rover-bridget-put-through-its-paces-on-welsh-beach-91466-26854943/.

Seven years of Mars Express - unusual structures at Magellan Crater

In the southwest of the Tharsis volcanic region on Mars is the large impact crater Magellan, named after the Portuguese explorer Ferdinand Magellan (1480-1521). The High Resolution Stereo Camera (HRSC), carried by ESA's Mars Express orbiter and operated by the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR), acquired images of unusual structures on the southern edge of the crater. The process by which these structures developed is not fully understood.

For more details go to http://www.dlr.de/en/desktopdefault.aspx/tabid-1/86_read-25233/.  

MSSL designs 'eyes' of robot scientist bound for Mars

Professor Andrew Coates from the Mullard Space Science Laboratory (MSSL) is leading the team designing the Panoramic Camera, or PanCam, aboard the ExoMars rover. The 'robotic scientist' will search for evidence of past and present life and study the local environment of the Red Planet to understand when and where conditions that could have supported the development of life may have prevailed.

http://www.physorg.com/news195834060.html

Spectacular Mars images reveal evidence of ancient lake

The evidence comes from NASA's Mars Reconnaissance Orbiter (MRO) which spied a series of depressions linked by what look like drainage channels. Recent research, led by a team from Imperial College London and University College London (UCL), suggests that during the Hesperian Epoch, approximately 3 billion years ago, Mars had lakes made of melted ice, each around 20km wide, along parts of the equator. Earlier research had suggested that Mars had a warm and wet early history but that between 4 billion and 3.8 billion years ago, before the Hesperian Epoch, the planet lost most of its atmosphere and became cold and dry. In the new study, the researchers analysed detailed images from NASA's Mars Reconnaissance Orbiter, which is currently circling the red planet, and concluded that there were later episodes where Mars experienced warm and wet periods. For more information, please visit http://www.mssl.ucl.ac.uk/general/news/Mars_Lakes_20100104/marslakes_20100104.htm

3D Reconstruction Service on the Web

Dr. Tomas Pajdla and his team at the Center for Machine Perception, Czech Technical University provide a remote access web service to their 3D reconstruction systems. The service includes useful jobs like camera calibration, camera model conversion, and sparse and dense 3D reconstruction. The service can be accessed through a web browser based interface or by a command line interface based utility. Details of the service are available at: http://ptak.felk.cvut.cz/sfmservice

Conferences, Workshops and Public Outreach

EPSC Workshop on Planetary Robotics and Vision Processing

A Workshop on Planetary Robotics and Vision Processing was held during the European Planetary Science Congress 2010 from September 19-25 in Rome, Italy. The workshop was organised by PRoVisG partners and covered key robotics and vision processing areas that must be addressed for future planetary exploration ambitions. More information can be found here.

PRoVisG Representation at the International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS) 2010

Dr. Mark Woods from Scisys and Ms Karin Shala from the University of Surrey presented papers on PRoVisG related work at i-SAIRAS, from August 29 to September 1, 2010 in Sapporo, Japan. Dr. Woods gave a talk on High-Level Autonomy for Exploration Robotics and Ms. Shala presented results obtained from a Comparative Analysis of Localisation and Mapping Techniques for Planetary Rovers. For more information please visit http://robotics.jaxa.jp/i-sairas2010/index.htm.

ECCV 2010 Workshop on reconstruction and modeling of large-scale 3D virtual environments

http://graphics.cs.ucy.ac.cy/RMLE/

Professor Ron Li from Ohio State University presents PRoVisG at Stanford University

More information can be found here: http://thelaunchpad.xprize.org/2010/07/multi-million-euro-research-grants-for.html.

ACCV 2010 Workshop on Computer Vision in Vehicle Technology: from Earth to Mars

Today, computer vision technique provides methodology to assist a long-distance explorer project using visual sensing system such as mars rover project. A Workshop on Computer Vision in Vehicle Technology: from Earth to Mars, will be held during the 10^th Asian Conference on Computer Vision 2010 from November 8-12 in Queenstown, New Zealand. The workshop is jointly organised by PRoVisG partners and will cover important areas in computer vision vehicle technology like tracking, reconstruction, and prediction. The official page of the workshop is:  http://www.media.imit.chiba-u.jp/CVVT2010/.

Related Projects

PRoViScout - a new European project on long range scouting/exploration on a terrestrial planet

ProViScout will demonstrate the combination of vision-based autonomous sample identification & sample selection with terrain hazard analysis for a long range scouting/exploration mission on a terrestrial planet. It brings together major European groups currently working on planetary robotic vision, supported by NASA-JPL from USA.

Most robotic planetary space missions performing in situ exploration of the surface and atmosphere for any planetary object outside the Earth involve a means of mobility provided by either a surface vehicle ( rover ) or by aerial vehicles (balloons, aerobots etc.).

Mobile systems are among the most critical of all space missions in requiring a rapid and robust on-site processing and preparation of scientific data to allow efficient operations for a maximum use of their limited lifetime. In future the number and variety of such platforms will require more autonomy than is feasible today, particularly in the autonomous on-site selection of and access to scientific and mission-strategic targets. ProViScout will provide the robotic vision building blocks on board of such future autonomous exploration systems.

PRoViScout Participating Institutions:

• JOANNEUM RESEARCH, Graz, A, Institute of Digital Image Processing
• Aberystwyth University, UK, Department of Computer Science
• DLR, Berlin, D, Institute of Planetary Research
• Czech Technical University, Prague, CZ, Center for Machine Perception
• SciSys Ltd., Bristol, UK 
• University College London, UK, MSSL Planetary Science Group
• CSEM, Neuchatel, CH
• University of Leicester, UK, Dept. of Physics and Astronomy
• GMV, ES
• University of Strathclyde, UK
• TraSys, BE

For more information please visit http://3dvision.joanneum.at/3DVision/projects/proviscout/overview/ and http://www.proviscout.eu

-------------------------------------------------------------

The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 218814 "PRoVisG".

If you wish to receive this PRoVisG newsletter also in the future, please subscribe on http://www.provisg.eu/.

To unsubscribe from this newsletter, please send an e-mail with subject "unsubscribe" to admin@provisg.eu.

Perspective view, LROC image, and DEM [NASA/GSFC/Ohio State University].

The area covered by this stereo pair shows the topography seen by the Apollo 16 astronauts, including rolling plains with many impact craters.  The 3-D DEM is colored showing elevations, ranging over about 1 km. South Ray Crater (A) in the perspective view is 750m in diameter and approximately 70m deep. The surrounding craters range from several meters in diameter to 1,200 m. The tallest mountain in the southwest corner (lower right) of the DEM is 880m above the plain. The DEM area [NAC images M102064759 and M102057602; centered at 9° S, 15.4° E] is 50km long and 7km wide. The LROC images were taken on July 12, 2009 during LRO orbit 218 near the Apollo 16 landing site. Images were processed using software developed by the Mapping and GIS Laboratory at the Ohio State University. Click on the image for a more detailed view.

Can we measure the size and shape of equipment and objects of the Apollo missions using today’s orbital images from LROC cameras? As an example to demonstrate the precise 3D measurement capability of LROC NAC stereo imagery, three objects (including the Lunar Module, ALSEP equipment, and Turtle Rock) and the astronauts’ traverse at the Apollo 14 landing site are measured and their 3D models are reconstructed. Such 3D measurements and models can be used for mission planning and space human research analysis for future landed missions [NASA/GSFC/Arizona State University/The Ohio State University].

Using two high-resolution (.5 m/pixel) LROC NAC images (M114064206 and M111708164) taken from two separate orbits (1943 & 1596, respectively), we can form a stereo image pair for 3D measurements at the Apollo 14 landing site. Thanks to the high image resolution of LROC NAC cameras,  among the visible objects are the lunar module (Antares), ALSEP (Apollo Lunar Surface Experiments Package, west of Antares), a rock nicknamed Turtle Rock (north of Antares), and multiple astronaut traverse lines indicated by disturbed soils. Photogrammetric data processing methods can be used to identify the objects and measure their sizes and shapes. Such information can then be employed to reconstruct 3D models of the objects, which are displayed on the digital terrain model (DTM) of the site that is automatically generated from the same data set.  Additionally, Cone Crater is at the northeast end of the astronaut traverse.

In the zoomed image below, the lunar module can be identified by its top (red points) and the shadow (green lines). These points are identified and measured in the two stereo images and their corresponding 3D ground coordinates are computed. Note that the shadow analysis uses different times and sun angles of the two images. In addition, the nearby terrain is measured at the selected points on the ground (green points) as a reference. From these measurements, we can compute the height and diameter of the lunar module. As the result, the height of the lunar module (descent stage) is estimated as 3.0 m, compared to the design specification of 3.2 m. On the other hand, the shadow analysis resulted in a height of the lunar module of 3.2 m. Furthermore, using a least squares fitting to a circle the diameter of the lunar module is computed as 4.4 m, compared to the design data of 4.2 m.

ALSEP and Turtle Rock are relatively small in the vertical direction. The LROC NAC image resolution and the imaging geometry would not be able to resolve for the height that is less than 1 m. However, we can measure their horizontal dimensions: 2 m x 1 m for ALSEP (yellow points), and 1.5 m x 1 m for Turtle Rock (green points).

Based on the above 3D photogrammetric measurements and the additional information from the design specifications, we created 3D models of the selected objects of the landing site in the AutoCAD system. We then imported these 3D object models into ArcGIS to combine the 3D models with the DTM of the landing site. Generally, a CAD package for 3-D modeling (e.g., AutoCAD or Google Sketchup) and a GIS package (e.g., Autodesk 3D Map or ArchGIS) can be employed for the visualization purpose in this case.

http://news.bbc.co.uk/1/hi/technology/8271733.stm

PRoVisG contributed to this year's Arctic Mars Analogue Svalbard Expedition (AMASE) 2009 in the Svalbard archipelago. Within an international team consisting of representatives from Agencies (ESA, NASA-JPL) and space Instrumens teams (e.g. ExoMars PanCam), on-site 3D Vision processing abilities were demonstrated, and a huge amount of representative test data was acquired.

PRoVisG Team member Arnold Bauer: "Our AMASE contribution was generally successful for PRoVisG and also the ExoMars PanCam Team. Great progress since 2008 was recognized. PanCam had a Team of 3 people, coming with quite different scope, well complementing one another. A large data set, extremely useful for PRoVisG, was collected, and large part of it was already on-site processed.

The target of the expedition from our understanding (to bring together space community and launch discussions and joint projects / undertakings during the expedition and in future) was fulfilled."

For more information click here.