Searching for a Quantitative Proxy for Rover Science Effectiveness

Abstract

During two weeks of study in September and October of 2004, a science team directed a rover and explored the arid Atacama Desert in Chile. The objective of the mission was to search for life. Over the course of the mission the team gained experience with the rover and the rover became more reliable and autonomous. As a result, the rover/operator system became more effective. Several factors likely contributed to the improvement in science effectiveness including increased experience, more effective search strategies, different science team composition, different science site locations, changes in rover operational capabilities, and changes in the operation interface. However, it is difficult to quantify this effectiveness because science is a largely creative and unstructured task. This study considers techniques that quantify science team performance leading to an understanding of which features of the human-rover system are most effective and which features need further development. Continuous observation of the scientists throughout the mission led to coded transcripts enumerating each scientific statement. This study considers whether six variables correlate with scientific effectiveness. Several of these variables are metrics and ratios related to the daily rover plan, the time spent programming the rover, the number of scientific statements made and the data returned. The results indicate that the scientists created more complex rover plans without increasing the time to create the plans. The total number of scientific statements was approximately equal (2187 versus 2415) for each week. There was a 50% reduction in bytes of returned data between the two weeks resulting in an increase in scientific statements per byte of returned data ratio. Of the original six, the most successful proxies for science effectiveness were the time to program each rover task and the number of scientific statements related to data delivered by the rover. Although both these measures have face validity and were consistent with the results of this experiment, their ultimate empirical utility must be measured further.