Embedded CSE (eCSE) support

Through a series of regular calls, Embedded CSE (eCSE) support provides funding to the ARCHER user community to develop software in a sustainable manner to run on ARCHER.

Details of any current eCSE calls and their remit can be found on the eCSE calls page

Useful Links

The eCSE programme provides tangible software enhancements to the communities exploiting software on ARCHER. This in turn has led to significant scientific advancements and both economic and social benefits to society.

Reports from completed eCSE projects

Red fox skull

VOX-FE for biomechanical modelling - new functionality for new communities

Understanding the mechanics of biological structures such as bones is very challenging, and experimental measurements can be extremely difficult. Computational modelling offers an alternative approach for studying the biomechanics of living and extinct animals, from insects to dinosaurs. In this project, researchers at the Universities of Hull and Edinburgh have significantly extended the VOX-FE modelling software, which can be used in predictive biology and virtual experimentation to reduce animal experiments. Potential applications include high resolution modelling of whole bones to understand normal and pathological bone biomechanics (e.g. in osteoporosis), and in silico design and testing of new dental and orthopaedic implants.

Illustration of Hilbert space-filling curve

Optimisation of the EPOCH laser-plasma simulation code

The EPOCH laser-plasma simulation code is a mature community code with a large international user base. The code uses particle-in-cell (PIC) techniques. The purpose of this project was to both improve the performance of EPOCH and futureproof the code for next generation multi-core architectures. As a result of the improvements made to the code, a speed-up of up to 3.58 has been achieved, and EPOCH can now be used for a wider variety of simulations that feature non-uniform particle distributions.


Image: Illustration of Hilbert space-filling curves for 3D spaces: William Gilbert, University of Waterloo

Ocean simulation

NEMO Regional Configuration Toolbox for ocean modelling

NEMO is a state-of-the-art ocean modelling framework used by the Met Office, NERC and many other UKHEIs to simulate the ocean from the global scale down to that of estuaries. This project has developed a unique set of tools - the NEMO Regional Configuration Toolbox (NRCT) - to aid users to set up lateral boundary conditions for near real-time regional ocean simulations within the NEMO environment. The NRCT provides access to remote datasets, and the tool effectively translates data from a parent model source into a format suitable to run a regional NEMO ocean simulation. A GUI is also provided to allow the user to define the regional ocean model domain.

Nektar code logo

Improving the performance of Nektar++ using communication and I/O masking

Nektar++ is an open-source spectral/hp element framework for numerical solution of partial differential equations in complex geometries using high-order discretisations. As a result of the work done in this project, the code now makes more efficient use of computational resources and scales to higher core counts, allowing simulations to run in a shorter period of time. This benefits a wide range of application areas including transient aerodynamics simulation (F1, aircraft design and analysis), aero-acoustic simulation (aircraft engines), modelling of patient-specific cardiac electrophysiology (atrial fibrillation) and biological flow simulations for understanding disease formation and development (atherosclerosis).

Contour plot of the relative free energy density (summed enthalpy and entropy) for hydration of the cucurbit[7]uril system.

Algorithmic Enhancements to the Solvaware Package for the Analysis of Hydration

The Solvaware package is a workflow that runs and analyses molecular dynamics (MD) trajectories to estimate hydration free energies by computing the contribution of different subvolumes around a solute. The accurate calculation of hydration free energies is a vital goal in computational modelling of biological and engineered aqueous systems. This project has enhanced the Solvaware package by improving the efficiency of key algorithms in the workflow, increasing its functionality, and widening its impact by making it available to the ARCHER community.

Shape of a GS2 flux tube in MAST

Optimising GS2 plasma physics simulation code used by fusion energy researchers

This project improved the performance at high core counts of the GS2 plasma physics simulation code. With a speed-up of up to 5 times (for short simulations) or up to 2 times (for longer simulations), this will allow plasma simulations to be carried out in much more detail, leading to a better understanding of the turbulence features in the plasma. Understanding, and controlling, turbulence in plasma is one of the key requirements for building a usable fusion reactor, and codes like GS2 can help in the design and understanding of such reactors. Fusion energy has the potential to provide a clean source of electricity for future generations.

3D Domain Decomposition

Developing highly scalable 3-D Incompressible Smoothed Particle Hydrodynamics

Smoothed Particle Hydrodynamics (SPH) is a novel computational technique that is fast becoming a popular methodology to tackle industrial problems with violent flows. Recently, accurate incompressible SPH (ISPH) codes have become more popular now that the numerical stability can be ensured, particularly for free-surface flows. In this eCSE project, researchers from the University of Manchester and STFC Daresbury Laboratory have developed a scalable, 3-D ISPH engineering toolkit.

strand of DNA

Efficient computation of two-electron integrals in a mixed Gaussian/B-spline basis

The aim of this project was to develop efficient parallel routines to calculate two-particle integrals in a mixed basis of Gaussian Type Orbitals (GTOs) and B-spline orbitals. The object-oriented library developed in the project has contributed to a significant improvement in the capability of the UKRmol+ suite of codes, designed to study electron and positron collisions with molecules. This will allow the study of electron interaction with bigger molecules than ever before, which could lead, for example, to a better understanding of how radiation damages cell constituents, in particular DNA.


Image: DNA ©iStock.com/nechaev-kon

graph

Optimising van der Waals simulations with CASTEP code

CASTEP is a general purpose code for the quantum simulation of properties of materials. Researchers have improved the performance of the van der Waals (vdW) interactions as implemented by CASTEP by replacing the previous simple sum method with a modified Ewald scheme. The code has also been parallelised using MPI. As a result of these two changes, for large calculations on ARCHER the code now runs many times faster.

reconstructed three-dimensional luminal surface ready to be used for HemeLB simulation

Microvasculature blood flow using HemeLB

The work done in this project by researchers at UCL represents a substantial leap forward for the simulation of blood flow in microvasculature and enables for the first time the theoretical study of advanced aspects of haemorheology, oxygen transport, and cell trafficking in realistic vascular networks, and of the collective dynamics of dense red blood cell suspensions in complex vessel geometries.

Zacros software package logo

Pushing the Frontiers of Kinetic Monte Carlo Simulation in Catalysis - Zacros Software Package Development

The importance of heterogeneous catalysis in applications that enhance the quality of life cannot be overstated. Iron-based catalysts for the production of ammonia by the Haber process is a notable example, the importance of which is shown by the fact that fertilisers generated from ammonia produced in this way are responsible for sustaining one third of the Earth's population. Scientists at UCL have employed kinetic Monte Carlo methodology and their own software package, Zacros, to contribute to the unravelling of complex phenomena on catalytic surfaces. This knowledge can be used to improve existing catalysts or design novel materials with optimal activity and selectivity for bespoke applications.

Visualisation of convergence for Poisson mother problem

Scalable automated parallel PDE-constrained optimisation for dolfin-adjoint

The dolfin-adjoint package enables the automated optimisation of problems constrained by partial differential equations (PDEs). These problems are ubiquitous in engineering, eg the design of wings to maximise lift, or the cheapest bridge that will support the required load. Prior to this project, dolfin-adjoint was limited to serial optimisation libraries with no concept of parallel linear algebra. The work done in this project has eliminated the performance penalty associated with gathering the data and lifted the upper bound on the size of problems which can be considered.

Section of Crambin unit cell

Hybrid OpenMP and MPI within the CASTEP code

First-principles simulations of materials have had a profound and pervasive impact on science and technology, from physics, chemistry and materials science to diverse areas such as electronics, geology and medicine. CASTEP is a widely-used, UK-developed program for the quantum mechanical modelling of materials. This project has added a new level of parallelism to CASTEP, laying the foundations for a version of the code that will work well on future exascale supercomputers. The net result is a new science capability, allowing the study of larger and more complex systems than before, in less time.

test systems modelled by the software

Tuning FHI-Aims for complex simulations on Cray HPC platforms

The main objective of this project was to tune FHI-aims, a quantum mechanical electronic structure code, for optimal efficiency on ARCHER's hardware, for the benefit of members of the Materials Chemistry Consortium (MCC) and other users. The overarching target of this work by scientists at University College London was to improve scalability for a wide range of applications to enable new science, using current and emerging UK computer facilities, in a wide range of fields including materials and life sciences, chemistry, physics, and engineering.

image of alizarin in implicit solvent

Calculating Excited States of Extended Systems in LR-TDDFT

Electronic structure theory is a hugely important contributor to understanding properties of materials. Large-scale Density Functional Theory (DFT) codes such as ONETEP allow us to predict ground-state properties for large systems such as complex biomolecules. In this project, scientists at the University of Cambridge aimed to enhance, extend and improve the implementation in ONETEP of Linear Response Time-Dependent Density-Functional Theory (LR-TDDFT), the method of choice for computing optical properties of large systems.

Climate simulation image

Porting and enabling use of the Community Earth System Model (CESM) climate model on ARCHER

The Community Earth System Model (CESM) is a state-of-the-art coupled climate model for simulating the earth's climate system. Composed of four separate sub-models simultaneously simulating the earth's atmosphere, ocean, land surface and sea-ice, and one central coupler component, CESM allows researchers to conduct fundamental research into the earth's past, present and future climate states. Scientists at the University of Edinburgh have ported to ARCHER and optimised two versions of CESM, and these are now both readily usable by the UK climate research community. This is expected to generate a further growth in interest in the model.

image of solvent cavity for the T4 lysozyme protein L99A/M102Q (2600 atoms) as produced by the ONETEP solvation model

A pinch of salt in ONETEP's solvent model

Chemical reactions, drug-protein interactions, and many chemical and physical processes on surfaces are examples of technologically important processes that happen in the presence of solvents. The inclusion of electrolytes (salt) in solvents such as water is crucial for biomolecular simulations, as most processes (e.g. protein-protein or protein-drug interactions or DNA mutations) take place in saline solutions. This project aimed to develop the capability to model electrolyte-containing solvents in quantum-mechanical simulations of materials from first principles. Using a linear-scaling code such as ONETEP enables simulations to be performed on entire biomolecules or catalysts that typically involve hundreds or thousands of atoms.

Visualisation of wavefunction density at the end of an RMT calculation

Performance enhancement in RMT codes in preparation for application to circular polarised light fields

One of the grand challenges in physics and chemistry is to understand what actually happens during a chemical reaction. The nuclei in molecules move on the femtosecond (10-15 s) timescale, but the electrons in the molecules move on the attosecond (10-18 s) timescale. The R-matrix with time dependence (RMT) code is a leading code for the description of ultra-fast processes in general atoms and molecules. Scientists at Queen's University Belfast have been working on the RMT code, increasing its speed by up to a factor of 5 and reducing the amount of memory required by one or more orders of magnitude.

Cross-section of the proximal end of a human femur

Understanding how bones develop and respond to disease and the use of implants

Scientists at the University of Hull have developed their simulation software to utilise ARCHER to model complete bones or large sections of bones. This offers the exciting opportunity to model skeletal development and adaptation. The potential benefits are enormous, ranging from a better understanding of both the fundamental biomechanics of bone and the cause and effects of musculoskeletal conditions, to better implant design.

Gas pipelines on the sea floor

TPLS: Optimised Parallel I/O and Visualisation

TPLS (Two Phase Level Set) is a powerful 3D Direct Numerical Simulation (DNS) solver that is able to simulate multi-phase flows at unprecedented detail, speed and accuracy with applications in energy (oil/gas pipeline flows, microelectronic cooling via phase-change), environment (carbon capture and cleaning) and health (flows in retinal capillaries). Scientists at the University of Edinburgh have been working on the code, converting the previously serial I/O to a scalable parallel implementation. This results in a halving of the time taken for each simulation from set-up to complete analysis.

Topological connectivity diagram

Scalable and interoperable I/O for Fluidity

At Imperial College London, scientists have been working with the Fluidity CFD code to improve file I/O and the performance and scalability of the underlying PETSc library. The resulting performance increase is then utilised to improve the mesh initialisation performance of the Fluidity CFD code through run-time distribution and more efficient data migration. In addition to performance improvements the capabilities of PETSc's DMPlexDistribute interface have been extended to include load-balancing and re-distribution of parallel meshes, as well as the ability to generate multiple levels of partition overlap. Furthermore, support for additional mesh file formats has been added to DMPlex and Fluidity, including binary Gmsh, Fluent-Case.