BRIEF PROJECT OVERVIEW

The study of marine ecosystems is vital for understanding environmental effects, such as climate change and the effects of pollution, but is extremely difficult because of the inaccessibility of data. Undersea video data is usable but is tedious to analyse (for both raw video analysis and abstraction over massive sets of observations), and is mainly done by hand or with hand-crafted computational tools. Fish4Knowledge will allow a major increase in the ability to analyse this data: 1) Video analysis will automatically extract information about the observed marine animals which is recorded in an observation database. 2) Interfaces will be designed to allow researchers to formulate and answer higher level questions over that database.

The project will investigate: information abstraction and storage methods for reducing the massive amount of video data (from 10E+15 pixels to 10E+12 units of information), machine and human vocabularies for describing fish, flexible process architectures to process the data and scientific queries and effective specialised user query interfaces. A combination of computer vision, database storage, workflow and human computer interaction methods will be used to achieve this.

The project will use live video feeds from 10 underwater cameras as a testbed for investigating more generally applicable methods for capture, storage, analysis and querying of multiple video streams. We will collate a public database from 2 years containing video summaries of the observed fish and associated descriptors. Expert web-based interfaces will be developed for use by the marine researchers themselves, allowing unprecedented access to live and previously stored videos, or previously extracted information. The marine researcher interface will also allow easy formulation of new queries. Extensive user community evaluations will be carried out to provide information on the accuracy, ease and speed of retrieval of information.

For a longer overview of the planned project: click here

PROJECT OBJECTIVES

• Detecting targets in noisy environments.

• Characterising interactions between the targets.

• Recognising fish species by integrating multiple 2D perspectively distorted views over time.

• Exploiting ontologies to interpret user queries.

• Exploiting ontologies to convert queries into workflow sequences.

• Storing and accessing massive amounts of video and RDF data in a timely manner.

• Integration of the research in a publically usable web tool.

• Creation of a fish database suitable for behavioural and environmental studies.

• Training of staff in cross-disciplinary methods (computer vision with database and workflow scientists, computer scientists with biologists).

Fish4Knowledge is funded by the European Union Seventh Framework Programme [FP7/2007-2013] under grant agreement 257024, addressing Objective ICT-2009.4.3: Intelligent Information Management, Challenge 4: Digital Libraries and Content. NARL/NCHC is funded by the Taiwan National Science Council under grant agreement NSC 101-2923-I-492-002-MY2. We would also like to thank the Taiwan Power Company, Taiwan Ocean Research Institute and Kenting National Park for sharing their video datasets collected from their underwater observatories at Nanwan, Lanyu and Houbi Lake of Taiwan, which enabled this research. The project runs from October 1, 2010 through September 30, 2013.