BASE-II Work Packages

Leicester, UCL, ANL, & SDSC worked towards accelerating HPC with AI, using surrogate modelling and optimising software. This work package was connected to WP1 and WP5. Having contributed by developing software for managing workflows and targeting the accuracy of ab-initio simulations as the priority, below are output summaries from this work package: 

Surrogate models in cosmological simulations (Craig Bower (Leicester), Azam Khan (Leicester), Corentin Houpert (Leicester), Martin Bourne (Hertfordshire), Debora Sijacki (Cambridge), Ashiq Anjum (Leicester), Mark Wilkinson (Leicester))

Supermassive black holes that live in the centres of galaxies can release vast amounts of energy as they grow through accretion. Known as active galactic nuclei (AGN) feedback, this energy in the form of radiation, winds and jets directly impacts the evolution of the galaxy itself. State-of-the-art cosmological simulations of galaxy formation rely on the interaction between AGN feedback and their host galaxies to reproduce observed rates of star formation. Modelling feedback mechanisms, for example, in the form of powerful jets, is a formidable task in the context of galaxy formation simulations, given the large density contrasts, high velocities, limited resolution (~kpc) and huge dynamic range involved (> 14 orders of magnitude from the black hole event horizon to the cosmic web!). 

In order to model AGN jet evolution accurately, simulations are performed at high resolution. However, these are far too expensive to model over cosmic timescales. Cosmological simulations of galaxy formation face challenges in accurately modelling AGN jets because of their coarse resolution.

We have developed a machine learning framework called DeepOJet, demonstrating that the high-dimensional spatio-temporal dynamics of AGN jets can be encoded as a low-dimensional representation. DeepOJet uses a hypernetwork architecture that enables geometric, temporal and parametric complexity to be embedded into a resolution-agnostic prediction. 

Training DeepOJet on high-resolution images of AGN jets allows for simulations at much lower resolution without the loss of known physical behaviour when running simulations of the same jet at low resolution using state-of-the-art numerical simulations. DeepOJet also enables simulations of varying parametric values at any desired resolution.

The real strength of applying Deep Learning surrogate models to such multi-scale physics simulations is the efficiency with which predictions can be generated. It is now possible to include predictions from DeepOJet in large-scale cosmological simulations in real time.

Energy-aware job scheduling for HPC clusters  (Ali Zahir (Leicester), Ashiq Anjum, Mark Wilkinson)

 This research enhances energy efficiency in high-performance computing (HPC) data analysis workflows by integrating energy-aware scheduling and machine learning-based optimization. By categorizing workflows and utilizing Variational Autoencoders (VAEs) for synthetic data generation, the study predicts optimal operational configurations, reducing energy consumption by up to 10% while maintaining minimal impact on turnaround time. The findings provide valuable insights into sustainable computing and optimizing resource utilization in large-scale distributed environments.