Skip to main content

Research Studentship in Multimodal learning and analysis

Research Studentship in Multimodal learning and analysis

Project: Multimodal learning and analysis for healthcare

3.5-year D.Phil. studentship 
Supervisor: Prof Alison Noble

This project is offered in association with the new EPSRC-funded computer vision programme grant called Visual AI.

One of the themes of the Visual AI programme grant is multi-modal data learning and analysis. The multi-modal data we are interested in modelling is full-length ultrasound video of fetal scanning with synchronized recorded eye tracking data of the person scanning (the sonographer), probe movements, and audio of the sonographer describing the scanning as they do it. The multi-modal data is a digital record of a series of known but un-ordered tasks recorded in a clinic.

The doctoral thesis will investigate how to build meaningful multi-modal models of the dataset using the latest ideas from computer vision deep learning, but also appreciating the design constraints of working with real-world clinical data. These include the need to accommodate real world data imbalance, to validate methods with meaningful domain-specific metrics, and to consider how methods are proven to be trustworthy in the domain of interest.  The models we are currently building from this dataset aim to understand how sonographers scan by answering questions like, “can we get a computer to look where an expert looks in a scan?”, or “can a computer describe a video clip as well as an expert?”, “can a computer tell a user how to move the probe towards a target”. We have elementary understanding of how to answer these questions today. This doctoral thesis will develop and evaluate new multi-modal learning and analysis deep learning architectures motivated by these questions.   

To find out more about related work we are doing on this dataset refer to the PULSE project website.


This studentship is funded through the UK Engineering and Physical Sciences Research Council (EPSRC) Doctoral Training Partnership and is open to UK students (full award – fees plus stipend).  Full details of the EPSRC eligibility requirements can be found here.

There is very limited flexibility to support international students. If you are an international student and want to apply for this studentship please contact the supervisor to see whether the flexibility might be available for you.

Award Value

Course fees are covered at the level set for UK students (c. £8290 p.a.). The stipend (tax-free maintenance grant) is c. £15590 p.a. for the first year, and at least this amount for a further two and a half years. 

Candidate Requirements

Prospective candidates will be judged according to how well they meet the following criteria:

  • Have a very good first degree in engineering or computer science (or have completed one before starting the PhD),
  • Have strong computational and mathematical skills,
  • Have experience in theoretical and practical image processing and analysis.
  • Provide evidence of ability to conduct high quality research commensurate with experience (for example a first degree dissertation). This will be discussed at interview (do not send a dissertation as part of your application).
  • be an excellent communicator (oral and written).
  • provide evidence of ability to work in a team.
  • Interest in healthcare applications of computational methods.

The following skills are desirable:

  • Knowledge of machine learning.
  • Knowledge of computer vision.
  • Knowledge of basic principles of medical imaging.

Application Procedure

Informal enquiries are encouraged and should be addressed to Prof Alison Noble FRS FREng (

Candidates must submit a graduate application form and are expected to meet the graduate admissions criteria.  Details are available on the course page of the University website.

Please quote 21ENGBM_AN in all correspondence and in your graduate application.

Application deadline:  noon on 22 January 2021

Start date: October 2021