Scientists prove reliability of quantum simulations for materials design

Findings critical at a time of rapid technological advances

Current and former TYC members, Professors Keith Refson and Chris Pickard, have been part of an international effort to demonstrate the reliability of computer simulations in the sciences, achieving a strong proof of this in the subject of computational materials simulation.

The findings are critical at a time of rapid technological change with increasing demand for new materials to make better batteries for mobile devices, catalysts, photocells for energy conversion and much more. Computer simulations at the atomic scale play an increasingly important role in designing them.

The international team asked the question: just how reliable are these simulations?  If we are to make the rapid technological advances we all seek, we need to be sure that researchers and engineers across the world can rely on getting the same results.  Such reproducibility is a corner stone of science: independent yet identical experiments should produce identical results. Only in this way can science identify ‘laws’, which lead to new insight and new technologies. However, several recent studies have pointed out that such reproducibility cannot be taken for granted.

Over the last few years there has been growing alarm in the scientific community that some results cannot be reproduced.  Even predictions of the same physics by different software packages (“computer codes”) require caution, since the way in which theoretical models are implemented may affect simulation results.

For the study and design of materials, for instance, there are several independent software packages available based on quantum physics. Codes including CASTEP, the UK’s leading materials simulation package, are being used increasingly often in automated procedures with limited human supervision in the new field of “materials informatics”. It is therefore essential to know to what extent predicted materials properties depend on the code that was used.

Despite the need for reliable predictions of the properties of materials, the reproducibility of quantum simulations had not been investigated systematically before now.  Professor Keith Refson from Royal Holloway’s Department of Physics and Professor Chris Pickard, University of Cambridge, therefore joined forces with more than 60 colleagues in the UK and worldwide, bringing together the know-how of over 30 prominent institutions.

In their study, which appears in Science, the researchers investigated 40 different software packages and variants to describe the influence of pressure in 71 different crystals. Due to the international make-up of the team, discussions and collaboration were conducted using online tools – similarly to the way people collaborate to write Wikipedia. The team can now demonstrate that, although a few of the older methods disagree noticeably among themselves, predictions by recent codes are almost identical. They moreover define a quality benchmark that allows the verification of future software developments against their extensive database. New test data are continuously added to a publicly available website ( The researchers hope that this work will lead to higher standards for materials property simulations, and that it will ease the development of improved simulation codes and methods.

Journal link: Reproducibility in density functional theory calculations of solids.  DOI: 10.1126/science.aad3000

Image credit: Science.

Follow @tyc_london for updates from the Thomas Young Centre.