|
Astronomers used to worry that there would not be a supercomputer big enough to process data from the SKA. But high performance computing has continued to grow exponentially, and today there are multiple commercial providers who operate computers larger than the ngVLA & SKA requirements. We are already on the way to having a cheap and plentiful source of HPC for radio astronomy. But can we as radio astronomers take advantage of this resource? How easy or hard is it to use a ’bare metal’ provider like DUG or a ’commercial cloud’ provider like Amazon? Over the last four years I’ve been finding out via a set of research projects involving heavy data processing at commercial providers. In this talk I’ll cover my FRB search using the MWA telescope, and high spatial & frequency resolution imaging of HI gas in the Magellanic Clouds using ASKAP. I also got to experience the world of social science by collecting and analysing other researchers’ experiences with commercial supercomputing. In this talk I will summarise the good and bad of commercial HPC, and give you some tips on the difficult decision about how much effort to put in to optimising your code. |
|