AI used to recognise primate behaviours in the wild

Scientists at the University of Oxford have developed new artificial intelligence (AI) models to recognise behaviours of chimpanzees in the wild. The method will allow researchers and wildlife conservationists to significantly cut back on time and resources spent analysing animal behaviour in video footage, according to the new paper published today in Science Advances.

The new computer model was trained using videos from two populations of wild chimpanzees in West Africa from Bossou in Guinea, and Cantanhez National Park in Guinea-Bissau, to capture several behaviours: nut-cracking, eating, and buttress drumming. The tool is the first of its kind to automatically recognise behaviour in wild primates using both audio and video, and builds on previous work which developed an automatic method for tracking and identifying individuals using face recognition.

"We use methods from deep learning with networks that are able to ingest both the audio and the visual stream of information from a video, crucial in the wild where an animal behaviour might be heard but not seen"

"Our models can be applied to thousands of hours of video recordings of chimpanzees in their natural habitat from camera traps or archival footage", says Max Bain, a DPhil student in computer vision and machine learning in the Visual Geometry Group supervised by Professor of Computer Vision Engineering Andrew Zisserman. "We use methods from deep learning with networks that are able to ingest both the audio and the visual stream of information from a video, crucial in the wild where an animal behaviour might be heard but not seen (e.g. cracking nuts behind a tree) or inaudible actions."

With the widespread deployment of camera trap databases and use of long-term video archives in field research, capturing the sheer volume of animal behaviours using human researchers is becoming increasingly difficult to scale.

"For species such as chimpanzees which have remarkable behavioural complexity, cross-site comparisons of video datasets using AI presents an exciting opportunity to capture subtle variation between groups and the evolution of behaviour over time at a scale and depth not previously possible," says Daniel Schofield, researcher and DPhil student at Oxford University's Primate Models Lab, Institute of Human Sciences.

"Our method is not restricted to chimpanzees, and can be trained to recognise any behaviour"

"The method is novel by combining individual identification from previous approaches with audio-visual behaviour recognition to gain richer insight into the complexity of animals’ lives. Our method is not restricted to chimpanzees, and can be trained to recognise any behaviour," says Bain. "We hope that other teams and researchers can apply our cutting edge methods to other species."

Monitoring vast amounts of data from field research of wild species is a crucial component of conservation efforts tackling the biodiversity crisis. The use of behaviour recognition using AI has potential to capture novel behavioural indicators that can more accurately measure the viability of threatened populations.

"Ultimately this can help us examine the social and ecological drivers of behaviour, as well as monitor how these communities are responding to environmental pressures"

"Ultimately this can help us examine the social and ecological drivers of behaviour, as well as monitor how these communities are responding to environmental pressures caused by climate change and habitat degradation from human activities," says Schofield.

The international collaboration also featured scientists from other institutions in the UK (University of Exeter), Japan (Chubu Gakuin University, Japan Monkey Centre) and the US (University of Rochester, California Institute of Technology).

Footage credits: • For the camera trap footage (drumming): Cantanhez Chimpanzee Project • For the archival footage (nut-cracking): Kyoto University, Primate Research Institute.