Oxford Science Blog | University of Oxford

Oxford Science Blog

Scottish Higher results are out today

By Professor Louise Hayward, Department of Education, University of Glasgow and Professor Jo-Anne Baird, Department of Education, University of Oxford

We are not living in normal times.  In common with many countries around the world, Scotland has had to cancel their examinations and to rely on teacher judgement. The young people and their teachers who have worked so hard and who have achieved so much in such difficult circumstances are to be applauded.  The pressure such sudden changes have put on examination boards also has to be recognised.  They have had to generate, trial and test then implement a new system in a matter of months when normally such changes take many years.

Today young people across Scotland will receive their examination results. They are anxious because these results are high stakes.  Future opportunities, for example access to particular university or college courses or to employment are limited and the examination results represent the way that Scotland chooses to open doors to these opportunities.  It may not be much comfort to this year’s young people but for previous generations the stakes were even higher. Not performing well in examinations meant that doors closed.  The education system in Scotland is now much more flexible and offers alternative routes such as the wide range of opportunities offered by Scotland’s colleges. 

Every year in Scotland the examination results prove contentious.  If the results improve, standards must be falling; if the results decline, the education system is failing.   Rarely is either true 

However, every year in Scotland the examination results prove contentious.  If the results improve, standards must be falling; if the results decline, the education system is failing.   Rarely is either true.  However, any approach to assessment where the implications are so significant for individuals and their families should, and do, come under significant scrutiny.  The nature of any system that seeks to ration opportunities will always lead to a situation where some are delighted and others disappointed.  That happened last year and it will happen this year.

Teacher assessment has much to commend it - the teacher is able to gather evidence about more of the curriculum, knows the young person, is able to collect evidence over time rather than a single snapshot.  Yet, particularly when the stakes are high, teacher assessment also faces challenges.   Having teacher judgements be consistent, not only within a school, but across every school in the country is a major challenge for countries internationally.  For the system to be fair, teacher judgements need to be consistent nationally. 

The moderation of teachers’ professional judgements is part of a national quality assurance system to ensure parity.  Without moderation, this year’s results might not only have been inconsistent, they might have been far too generous.  This is what we know from research on teachers’ estimates.  The consequences of that for this year’s young people could have been very damaging, if this had become known as the year that the results were inflated. There is research evidence to point to differences in teacher judgement that relate to gender, class and ability. It is crucial that due attention is paid to those to ensure that young people are treated fairly.

No method of assessment is perfect.  External examinations suit some young people who thrive on the pressure - others find the pressure unbearable and struggle to cope.  Have an off day on the exam day and your results can vary significantly.

No method of assessment is perfect.  External examinations suit some young people who thrive on the pressure - others find the pressure unbearable and struggle to cope.  Have an off day on the exam day and your results can vary significantly

Examinations are limited in what they can measure and commonly internationally lead to narrowing of the curriculum and limited approaches to learning and teaching.   Significant amounts of time in 4th, 5th and 6th years are spent rehearsing for examinations and can become little more than past papers, prelims and endless tests leaving young people dissatisfied and disinterested.  The Examinations become an end in themselves and learning only matters if it is in the examination.

Learning in the 21st century demands more than that.  What matters for young people in future depends as much on their ability to collaborate, to be creative, to add value to society as it does on their ability to solve a quadratic equation.  It is not a question of either/or.  Both matter.  Our qualifications system needs to change to respond to what matters for our future citizens. Teacher professional judgement will play a crucial role in that new system.

This year, as in previous years, some young people will not receive the results they so hoped for.   Recognising that the system has had to change more quickly than might have been desired to respond to COVID-19, the SQA will offer a more open appeal process than has been the case in previous years.  This is just.  The Appeal process offers a space where evidence related to each case can be reconsidered and decisions reviewed.  Where further evidence points to the need to change a grade, changes will be made. SQA should and has committed to do this.

This year, as in previous years, some young people will not receive the results they so hoped for.   Recognising that the system has had to change more quickly than might have been desired to respond to COVID-19, the SQA will offer a more open appeal process than has been the case in previous years.  This is just

However, as in previous years, even after Appeal some young people will remain disappointed. That is the nature of a system where future opportunities are competitive because they are limited in number. Public confidence in the qualification system is crucially important to this year’s pupils.  As a nation, we owe it to our young people to value their efforts and their achievements and not to seek to ask more of this year's qualification system than we have asked of the systems in previous years.  To do so would put at risk the credibility of the awards of the very young people we seek to support.

Professors Hayward and Baird are independent academics (Professors of Education) and do not speak on behalf of their institutions.  They are members of a number of advisory groups, including SQA’s Qualifications Committee but are not employees and do not speak on behalf of SQA.

Doctor putting on a glove to prevent coronavirus contamination

Infectious disease and clinical machine learning experts at the University of Oxford have developed an Artificial Intelligence test that can rapidly screen for COVID-19 in patients arriving in emergency departments, and a preprint paper has been published on its effectiveness. 

The ‘CURIAL’ AI test assesses data routinely collected during the first hour in emergency departments, such as blood tests and vital signs, to determine the chance of a patient testing positive for Coronavirus.

Currently, testing for Covid-19 is by a molecular analysis of a nose and throat swab, called a Polymerase Chain Reaction (PCR). However, this typically has a turnaround time of 12-48 hours and requires specialist equipment and staff.

The new Oxford study has developed AI that gives a near real-time prediction of a patient’s Covid-19 status.

The team is led by Dr Andrew Soltan, an NIHR Academic Clinical Fellow at the John Radcliffe Hospital, joining with the ‘AI for Healthcare’ lab of Professor David Clifton within Oxford’s Institute of Biomedical Engineering, and Professor David Eyre of the Oxford Big Data Institute

The study has been running since March and began by developing machine learning algorithms trained on data from confirmed cases and pre-pandemic controls to detect subtle differences. It was hoped these algorithms would allow the level of risk of having the illness to be determined.

The early clinical course of SARS-CoV-2 infection can be difficult to distinguish from other undifferentiated medical presentations to hospital

The preprint article explains: ‘The early clinical course of SARS-CoV-2 infection can be difficult to distinguish from other undifferentiated medical presentations to hospital, however viral specific real-time polymerase chain reaction (RT-PCR) testing has limited sensitivity and can take up to 48 hours for operational reasons.

‘In this study, we develop two early-detection models to identify COVID-19 using routinely collected data typically available within one hour (laboratory tests, blood gas and vital signs) during 115,394 emergency presentations and 72,310 admissions to hospital.’

Once trained, the algorithms had to be assessed for their accuracy, and the two early-detection models were put to the test in a real hospital setting. The results have now been published in the preprint.

Useful tool for the NHS

Researchers are working hard to rapidly trial the CURIAL AI as a clinically useful tool for the NHS.

Dr Soltan said that identifying Covid-19 early in a hospital admission is essential for maintaining infection control and delivering timely care to patients.

‘Until we have confirmation that patients are negative we must take additional precautions for patients with coronavirus symptoms, which are very common. The CURIAL AI is optimised to quickly give negative results with high confidence, safely excluding Covid-19 at the front door and maintaining flow through the hospital.

‘The blood tests patients receive on admission are usually available in one hour, and have a large number of data points that algorithms can use to distinguish Covid-19 from a wide variety of other illnesses. Because of the advanced electronic records systems at Oxford University Hospitals (OUH), we’ve been able to train our algorithms using rich clinical data from 115,000 visits to A&E.

The blood tests patients receive on admission are usually available in one hour, and have a large number of data points that algorithms can use

‘When we tested the CURIAL AI on data for all patients coming to OUH’s emergency departments in the last week of April and the first week of May, it correctly predicted patients’ Covid status more than 90% of the time.’

The AI test was validated by applying it to all patients presenting to A&E and admitted across the four OUH hospital sites between the 20 April and 6 May.

Results showed that the Emergency Department Model correctly predicted the COVID-19 status of patients 92.3% of the time, across the 3,326 patients coming to A&E in the two week test period, and the Admissions Model was correct 92.5% of the time for the 1,715 patients admitted.

Dr Soltan continues: ‘The next steps are to deploy our AI in to the clinical workflow and assess its role in practice.

‘A strength of our AI is that it fits within the existing clinical care pathway and works with existing lab equipment. This means scaling it up may be relatively fast and cheap.

‘I hope that our AI may help keep patients and staff safer while waiting for results of the swab test.’

David Clifton, Professor of Clinical Machine Learning at the Department of Engineering Science, added: ‘With many of our clinical colleagues working on the front lines to fight COVID-19, data scientists in Healthcare AI have a supporting role to play by constructing tools to help care for patients. The unique ecosystem at Oxford between hospitals and clinical AI teams gives us a great opportunity to contribute to the international effort against Coronavirus. 

The unique ecosystem at Oxford between hospitals and clinical AI teams gives us a great opportunity to contribute to the international effort against Coronavirus

‘This project initiated by clinical collaborator Dr Andrew Soltan is a great example of what can be done, and at very great pace, to fast-track the development of technologies to help in the current pandemic – and to increase the resilience of the country’s healthcare system for any future events.’

It is hoped the development of these techniques will also inform clinical teams in the early stages of future pandemics, and expedite implementation of appropriate public health measures.

Microscope

A new study from S:CORT demonstrates an easy, cheap way to determine colorectal cancer molecular subtype using AI deep-learning digital pathology technology.

Understanding the molecular subtype of a cancer is becoming an importance part of the diagnostic process as it helps a doctor better understand a patient’s prognosis, determine the best course of action for treatment and helps researchers devise new, more-efficient, precision therapies.

Colorectal cancer (CRC) currently has four known molecular subtypes which are identified on the basis of its RNA expression profile, using RNA analysis. But the process of RNA analysis is costly, technically challenging and it requires a specialist to interpret data to determine the subtype. In order to more efficiently and cheaply determine the molecular subtype of a patient’s CRC, there is a need to use more easily-acquired data on a tumour and categorise its subtype based on an automated technique.

One prevalent source of data in cancer diagnostics is images, such as histological images of the microscopic anatomy of a cancer. Almost all CRC patients already have a sample of their tumour inspected by a pathologist as part of their standard care.

Tumour images under microscope

Caption: Image information can be used to predict the molecular classification of patient tumor samples in colour, using state-of-the-art deep learning models in pathology.

A new paper published by Dr Korsuk Sirinukunwattana, Dr Enric Domingo, Prof Viktor H Koelzer & Prof Jens Rittscher from the Stratification in Colorectal Cancer consortium (S:CORT), has demonstrated the potential for using these images to determine molecular subtype.

Through the training of a deep-learning neural network, using over 1,000 tumour samples, the team discovered that AI could be trained to distinguish between the four molecular subtypes using images alone.

Usually, histological images are a modest indicator of how colorectal cancers have progressed, and there is currently no way to determine the molecular subtype of a tumour by the human eye alone, without doing a gene profile on tumour samples. However, this new study demonstrates that technology can be trained to identify the minute morpho-molecular differences between molecular subtypes.

This new concept, called Image-based Consensus Molecular Subtyping (imCMS), will allow images to be interpreted through association of tile-level predictions with morphology, molecular features and outcome data.

Furthermore, this study also demonstrated that AI is able to classify the molecular subtype of samples that were previously unclassifiable by the standard process of RNA expression profiling. It also provides information for the spatial distribution of each subtype within a tumour.

This AI technology will have huge implications on the standardisation of classifying molecular subtypes of CRC. It opens new opportunities to create simple, cheap and reliable ways to quickly stratify a patient’s colorectal cancer, and decreases the demand for specialist input that is not always immediately available.

It has potential to reveal new avenues for the translation of the classification of CRC subtypes into clinical practice and increase availability of molecular stratification in low-resource settings, helping to make accurate cancer diagnoses and appropriate treatment more accessible to everyone.

About S:CORT and the researchers

For more information about the Stratification in Colorectal Cancer consortium (S:CORT) see here.

Samples for this study were collected by the S:CORT program and The Cancer Genome Atlas (TCGA). The S:CORT consortium is a Medical Research Council stratified medicine consortium jointly funded by the MRC and CRUK. This work was further supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre.

This study is a joint venture conceived by Prof Tim Maughan (Dept. of Oncology), Prof Ian Tomlinson (Institute of Genetics and Molecular Medicine, Edinburgh), Prof Jens Rittscher (Institute of Biomedical Engineering) and Prof Viktor H Koelzer (University Hospital and University of Zurich; Honorary Senior Clinical Researcher, Dept. of Oncology, University of Oxford;)

S:CORT consists of a team of clinicians, healthcare professionals, academics and scientists across the UK, lead from Oxford by Prof. Tim Maughan, and seeks to apply cutting edge molecular diagnostic and histopathological technologies to improve and tailor the treatment of colorectal cancer patients.

Contact tracing App on phone sitting on wall

By Dr Michelle Kendall, senior researcher, Oxford University’s Nuffield Department of Medicine

A Test and Trace (TT) programme was rolled out across the UK in May 2020, first on the Isle of Wight (5 May) and then nationwide (18-28 May). The Isle of Wight’s programme included version 1 of the NHS contact tracing app. We looked to see if the Isle of Wight’s epidemic changed after the TT launch and if the island fared any differently from comparable areas of the UK.

Using Public Health England’s daily “case counts” (positive swabs) we developed a method to estimate the number of new infections per day - there is a lag because people rarely get swabbed the day they are infected. The graph below shows Pillar 1 (hospital testing) case counts on the Isle of Wight in green, and our estimated number of new infections per day in red. New infections were decreasing from mid-April. There was a slight increase at the end of April, possibly an artefact of increased testing on 5th May, but then a noticeable decrease in infections despite the increased testing shortly after the TT launch.

App 1

Figure 1. Isle of Wight Pillar 1 data used to back-calculate estimation of incidence of new infections (red) from COVID-19 case data (green)

Similarly, the reproduction number R - the average number of further infections each infected person causes - decreased rapidly on the island following the TT launch as shown by the red line below:

Figure 2

Figure 2. Isle of Wight Pillar 1 data (red line) used to estimate R, together with confidence intervals (grey shade)

But how did the Isle of Wight compare to other areas? We used a variety of methods which all led to the same conclusion: something quite different happened on the Isle of Wight. The incidence and R rate dropped more rapidly than for other areas over comparable time periods. The graph below is a “nowcast” of the expected number of new hospital cases in the near future. The island had a sizeable epidemic in April-May, positioned “middle of the pack” in comparison with other areas; by June-July it led the way with case counts down to less than one a week, even when we included Pillar 2 (community tested) cases. Using a Maximum-Likelihood method we found that from March-May the Isle of Wight was ranked amongst the worst R rates in England (147th out of 150) but by mid-June it was positioned 10th best.

Figure 3

Figure 3. Pillar 1 data showing a simple "nowcast" for 150 local authorities to quantify the combined effects of R and incidence; the Isle of White shown in red

We have made our analyses available in this interactive EpiNow-C19 tool which updates daily. It allows you to view the epidemic trend in each of the 150 local authorities to identify successes and hotspots, including the most recent surges in Leicester, Herefordshire and Blackburn with Darwen.

The success on the Isle of Wight is striking, and the course of its epidemic was different from other areas - a statistically significant difference. To prove that the TT programme was the cause of the success would require ruling out all other possible causes, for which we would need more data. The island’s epidemic control certainly warrants further investigation because it might translate to other local and national strategies. 

If TT did have an impact we will also need to disentangle which aspects had the greatest effect. Was it the huge advertising campaign at the launch? Did people self-isolate more carefully after a positive test result? Was the contact tracing programme getting ahead of the virus, advising people to quarantine before they infected others? And if so, was that primarily human contact tracing or via the app? Top level results suggest 163 people were traced through the human tracing service and 1,188 through the app between 6-26th May. A key piece of the puzzle would be: of the people who tested positive, how many were already self-isolating because they had been contact-traced? We hope that more data will soon be made available so that we can answer these important questions.

COVID-19 incidence and R decreased on the Isle of Wight after the launch of the Test and Trace programme. Kendall et al. Available as a preview ahead of pre-publication on MedRxiv

Woman shopping in a supermarket wearing a face mask, as new regulations requiring face coverings are announced

Face coverings in shops will be made compulsory from 24 July, and the prohibition may be extended to other indoor spaces. The move follows last week’s COVID-19 face coverings study from Oxford’s Leverhulme Centre for Demographic Science on behalf of the Royal Society and British Academy.

The announcement was made this morning, and comes after Prime Minister Boris Johnson was pictured over the weekend publicly wearing a face covering. The Prime Minister described face coverings yesterday as offering a ‘great deal of value’ in controlling the spread of the coronavirus and giving people confidence to go back to shops and workplaces. But this morning came the news that, from 24 July, shoppers face a £100 fine, if they enter a shop without a face covering.

There is a general assumption that countries such as the UK, which have no culture or history of mask wearing, will not rapidly adopt them or that people will be put off going to shops if they have to wear a face covering. But this just doesn’t reflect the data from similar countries

Professor Melinda Mills, Director of the Leverhulme Centre and lead author of the study on the effectiveness of face coverings, says, ‘It is great news that face coverings are to be made compulsory in shops in England. More than 120 countries around the world have introduced recommendations or regulations and the report last week contains clear evidence that face coverings are effective in respect of COVID-19.’

She maintains, ‘There is a general assumption that countries such as the UK, which have no culture or history of mask wearing, will not rapidly adopt them or that people will be put off going to shops if they have to wear a face covering. But this just doesn’t reflect the data from similar countries. As of late April, mask-wearing was up to 84% in Italy, 66% in the US and 64% in Spain, which by June was over 90%, increasing almost immediately after clear policy recommendations and advice was given to the public.’ 

As early as April, a report in the British Medical Journal was advising people to wear face coverings.  Professor Trisha Greenhalgh, Oxford’s professor of primary health care sciences, was among experts who urged the adoption of face coverings, 'It’s time to encourage people to wear face masks as a precautionary measure on the grounds that we have little to lose and potentially something to gain.'

They maintained in their report that, despite limited evidence, masks ‘could have a substantial impact on transmission with a relatively small impact on social and economic life’. 

Face coverings have not been widely worn in the UK, in spite of the pandemic. The Leverhulme-led study shows cloth face coverings, even homemade masks of the correct material, are effective in reducing the spread of the virus – both for the wearer and those around them

Speaking today about the announcement, Professor Greenhalgh says, 'I very much welcome this news. It made no sense to mandate face coverings on public transport while not requiring them in other crowded and poorly-ventilated indoor spaces such as shops and supermarkets. What we now urgently need is proactive use of behavioural science techniques to inform a public information campaign to overcome weeks of mixed messages and shift the ethos of this policy from something that is 'enforced' and 'policed' to an exercise in common sense and social solidarity. Masking is a symbolic practice as well as a public health intervention. My own team and also Melinda Mills in Demography (among many others) have done research on this topic; we’d be happy to help make this policy a success.'

Face coverings have not been widely worn in the UK, in spite of the pandemic. The Leverhulme-led study shows cloth face coverings, even homemade masks of the correct material, are effective in reducing the spread of the virus – both for the wearer and those around them. And the study reveals that, once the WHO announced there was a pandemic in mid-March, within days many countries around the world had already recommended wearing face masks. Nations including South Korea, Japan and a series of African countries, experienced in handling previous epidemics including SARs and Ebola, have experienced very low numbers of deaths and transmission.

It is ‘far more invasive’ to tell people to stay at home than to wear a mask

Professor Mills, insists that face coverings should be recognised as a public health issue as part of a package of policies alongside hand hygiene and social distancing, not a matter of civil liberties.

She maintains, it is ‘far more invasive’ to tell people to stay at home than to wear a mask, with the report revealing that in order to increase compliance of the general public, it is important that they understand how the virus and coverings works to protect both themselves and others. The study reveals, ‘Next to hand washing and social distancing, face masks and coverings are one of the most of widely adopted non-pharmaceutical interventions for reducing the transmission of respiratory infections.’

Refuting claims that there is no or weak scientific evidence, Professor Mills finds that there has been an over-reliance on medical studies and specifically ‘randomised control trials’ – while high quality behavioural studies have been discounted. But she says, ‘The evidence is clear that people should wear face coverings to reduce virus transmission and protect themselves, with most countries recommending long ago that the public should wear them, particularly in enclosed indoor spaces and crowded areas.’

The evidence is clear that people should wear face coverings to reduce virus transmission and protect themselves, with most countries recommending long ago that the public should wear them, particularly in enclosed indoor spaces and crowded areas

The study calls for clear and consistent policies and public messaging on the wearing face masks and coverings by the general public. Professor Mills says, ‘The public is confused about wearing face coverings because they have heard the scientific evidence is inconclusive and advice from the WHO and others has changed.

 ‘But where are randomised control trials about coughing into your elbow or social distancing?’ And she adds ‘People need to know what to wear, when and where to wear it, how to wear it and who cannot wear it.’

 Professor Mills says, ‘What to wear is high quality multi-layer cloth coverings and not surgical masks or respirators and people need to ensure there are no gaps. For instance, combining cotton and silk or flannel provide over 95% filtration, so wearing a mask can protect others.

The public is confused about wearing face coverings because they have heard the scientific evidence is inconclusive...But where are randomised control trials about coughing into your elbow or social distancing?

 ‘When and where includes crowded shops, where you cannot often keep social distance, are places where face coverings would be valuable, but you don’t need to wear one while outside, walking or jogging, in a socially distanced way. And in restaurants, you need to wear one, until you are sitting down at your table. We also need to acknowledge who cannot wear them such as those with disabilities, breathing difficulties or young children.

Just as the WHO gathered evidence and changed its advice in early June, it is now time to re-evaluate the evidence and make clear recommendations to the public and those who need to re-open their businesses

'By learning from mask-wearing experiences from previous epidemics, such as SARS, H1N1 and MERS, the systematic review revealed key behavioural factors underpinning the public’s compliance to wearing a mask. People need to understand virus transmission and how masks protect them and others. And they need to understand how to reduce the barriers of wearing them related to practical aspects such as comfort or what to do in a restaurant or shop.’

She says, ‘We learned from previous pandemics that individuals underestimate their own risks of contracting the virus or transmitting it to others and think that ‘it won’t happen to me’.’

But, she adds, ‘Just as the WHO gathered evidence and changed its advice in early June, it is now time to re-evaluate the evidence and make clear recommendations to the public and those who need to re-open their businesses.’

 The full text of ‘Face masks and coverings for the general public: Behavioural knowledge, effectiveness of cloth coverings and public messaging’, on behalf of the Royal Society and British Academy can be found here.

See also

COVID Conversations

Face masks work!

Join Professor Melinda Mills to hear more about the effectiveness of different face mask types and coverings and ask any questions you have.

Wednesday 22 July, 1.15pm–1.45pm 

This talk will be broadcast here on YouTube, and on our main Facebook & Twitter channels.