This wiki service has now been shut down and archived

Theme 23

From ESIWiki

Jump to: navigation, search

Return to the Main page

Welcome to the Wiki space for the eSI minitheme "Strategy for the UK Research Computing Ecosystem", which will be used to collect material related to the theme.


Final version of the report - message from Peter Coveney

I am pleased to be able to attach herewith the final version of the "Strategy for the UK Research Computing Ecosystem" (1.4 MB). Following the 9th September Community Meeting in Oxford, a final round of comments were sought by 16 September. These comments have been integrated into the document which is now in its final form.

This is most timely, as I now have to provide this to the Willetts' Editorial Board which meets at BIS on Wednesday 21st September at 2.00pm (with a wider selection of invited participants) in order to discuss the draft "UK e-Infrastructure Strategy for Science and Business" Report now being prepared at the Minister's behest under the leadership of Dominic Tildesley. Our Strategy for the UK Research Computing Ecosystem document has already been heavily plundered in the initial drafting of that Report, while Dominic has himself enjoyed a few minutes in the presence of David Cameron to underscore the importance of this initiative.

I think, therefore, that we can justifiably feel that we have made a substantial impact with our work thus far, and I am most grateful both to everyone who attended the meetings at UCL and OeRC for their input and to those who provided written comments.

We plan to make a glossy version of the Strategy for the UK Research Computing Ecosystem in the very near future. It will also appear on the eSI and Town Meeting wikis. I commend the report to you and would ask you to disseminate it as widely as possible amongst your colleagues, friends and academic communities.

Best regards,


20th September 2011



  • 23rd August 2011: Prof Peter Coveney has accepted an invitation to join an Editorial Board being assembled by UK Government Department of Business, Innovation & Skills to prepare a Report at the behest of the Right Honourable David Willetts MP, Minister for Science & Universities. This report is intended to lay out a grand strategy for e-infrastructure across the whole of UK academe, industry and commerce and aims to influence UK government policy and thinking from the autumn onward. The Board will be chaired by Dominic Tlildesley, VP for the Discover Platform at Unilever. Prof Coveney is the sole academic member of the Board, others being drawn from funding agencies and industry.
  • 3rd October 2011 A £145 million boost to improve Britain’s e-infrastructure was announced by the Department for Business, Innovation and Skills

Background Information

Based on input from a broad spectrum of applications scientists, computer scientists and numerical algorithms experts, a strategy plan will be developed for HPC and e-Science infrastructures to support provision of and access to compute and data-intensive research infrastructures in the UK.

Access to a growing range of computing and data services that integrates sensor networks, data repositories and the largest supercomputers, e-infrastructure in short, is essential for the UK to continue to play a leading part in many areas of economic and societal importance, ranging from weather forecasting, climate and energy to health and well being. The research conducted using e-infrastructure as an experimental environment has profound impact on society, as researchers now analyse a wide range of physical events and test a wide range of prototypes without a costly physical testing environment. These developments have resulted in rapid and accurate predictions of weather events, climate change and natural disasters such as floods, as well as numerous advances and industrial prototypes in economic areas such as healthcare, engineering, traffic management and energy. In addition, modelling and simulation using supercomputers and discovery through searching diverse datasets are of crucial importance to the scientific community.

As the size and power of high end computers advances towards exascale, their integration into the broader e-infrastructure "landscape" becomes more pressing than ever. Vast quantities of data need to be shipped to these resources, and large quantities of data produced by simulations then need to be transferred to other infrastructures for many forms of post-processing and analysis. Due to the inherent data and network intensive nature of modern supercomputer usage, the time when we could even consider using supercomputers as "islands" operating independently from everything else in the digital landscape has now firmly ended.

The general picture that emerges today is one of the computational scientist requiring access to a whole range of resources, from the "lowest" desktop machine to University-provided clusters to the most powerful supercomputers, including access to large scale databases and storage systems, all connected through high performance networks.

The scientist's requirement for a coordinated and integrated compute infrastructure is difficult to achieve with the existing fragmented organization of these infrastructures within the UK.

The current fragmented organization of UK e-infrastructure organisation and management prevents users from efficiently involving the most powerful supercomputers into their research and development. Denied access to the higher end and without an effective resource-sharing model at the mid-range, the largest and most challenging scientific problems become inaccessible to UK scientists and their ability to compete, let alone lead the world in these most pressing of issues is fully lost. However, we can prevent this lapse in scientific progress by working to unify the UK e-infrastructure, aiming to create a research environment where the compute, storage and networking facilities at national facilities and each university are integrated and accessible and where the HPC resources seamlessly interoperate with the high-quality networking, storage and post-processing resources that modern users require. Such a unified approach requires a nationwide strategic plan for HPC and e-Science infrastructures, which this mini-theme aims to create by engaging all of the stakeholders. A subsequent outcome will hopefully be a body that can represent all stakeholders and speak with one voice when working with the Government and Research Councils to plan and deliver future provision for science.

Planned outcomes

In this mini-theme we will work out a strategic plan for HPC and e-Science Infrastructures within the UK. This plan will propose how to proceed in order to ensure that the UK-based e-Infrastructure provides its users the ability to operate on the frontiers of science and industrial development. It will describe the actions required to forge a unified and seamlessly interoperating environment, and to ensure that the potential offered by the e-infrastructure in the UK is used in full.

Development of the national strategic plan for HPC and e-Science Infrastructures will take place in close cooperation with the scientific user communities, as they are ultimately responsible for converting the potential offered by e-infrastructures into scientific progress, and thereby for ensuring the success of e- Science in the UK.

The Collaborative Computational Projects (CCP) Steering Panel, chaired by Peter Coveney, represents the leading expertise in the key fields of computational research within the UK. The CCPs organize regular meetings and workshops and more than 300 research groups participate directly within the CCPs themselves. During the last CCP Steering Panel meeting, it became clear that there is an urgent need from the research community to develop a national and unified vision for HPC and e-Science infrastructures. Aside from the CCPSP, we will also closely work together with the scientists in UK-based HPC consortia, both those focussed on using HECToR and DiRAC Tier-1 systems and those predominantly supported by university Tier-2 services (coordinated through the HPC-SIG). These researchers are highly dependent on a combination of national (EPSRC/STFC) and local (HEFCE) funding. If this funding is to end, then these consortia will disappear and years of research effort and code development will be wiped out. Both the CCPSP and the RC-funded consortia will be consulted regularly during the development of the strategy plan. In addition, as we work out the strategy for HPC and e-Science infrastructures, we will also liaise with the UK Government Department of Business, Innovation and Skills (BIS). It is important that BIS is aware of the urgent need of a unified strategy for HPC and e-Science infrastructures, but likewise we will require knowledge of the political constraints to ensure that the strategic plan contains goals that can realistically be accomplished.

We intend to complete the UK Strategic Plan for HPC and e-Science Infrastructures by early July so that a draft will be available for consultation of the user community and for discussion by them at the Town Meeting in UCL on 8 July 2011.


  • 18th May 2011 - Minutes
  • 1st June 2011 - Minutes
  • 8th July 2011 - Town Meeting to discuss the Future of e-Science and HPC Infrastructures and Applications in the UK
    • Notes of the Town Meeting
  • 13th July 2011 - From science to sales - taking time and money out of the innovation pipeline: seminar organised by BIS
    • Letter from Professor Stephen Hawking to Rt. Hon. David Willetts MP regarding a Strategic Plan for UK Research Computing PDF (11KB)
    • Peter Coveney's presentation - e-enabled Science (PDF - 1.8MB)
  • 9th September 2011: UK Research Computing Ecosystem - September Community Forum

Reference documents

29th August 2011: latest version of the Strategic Plan for the UK Research Computing Ecosystem

Other useful documents:

Twitter record (237KB) of the meeting (compiled by Steven Young)

Letter from Professor Stephen Hawking to Rt. Hon. David Willetts MP regarding a Strategic Plan for UK Research Computing PDF (11KB)

Useful links

  • HPCWire Summary of IDC HPC Survey at ISC Tom Tabor gives a summary of the latest IDC HPC Market Update, which has some relevant comments on European HPC. The IDC PowerPoint is also available from this page.
This is an archived website, preserved and hosted by the School of Physics and Astronomy at the University of Edinburgh. The School of Physics and Astronomy takes no responsibility for the content, accuracy or freshness of this website. Please email webmaster [at] ph [dot] ed [dot] ac [dot] uk for enquiries about this archive.