This year we are honoured to have Professor Andreas Wicenec, Professor for Data Intensive Research at The University of Western Australia, present on the topic

"Astronomical (size) Data Management and Processing in the Era of the Square Kilometre Array".

One of the biggest science projects ever, the Square Kilometre Array (SKA), is being constructed as we speak. The SKA will consist of two radio telescopes, one here in Western Australia and the other one in South Africa. Every day new antennas are being build, tested, shipped and installed on the two sites in South Africa and Western Australia. Here, in the Murchison Shire, a total of up to 131,072 dipole antennas will be installed, clustered in 512 stations across an area of about 60km in diameter. In South Africa there will be 196 parabolic dishes distributed across an area of 190km in diameter.

The science goals for these telescopes are extremely broad and range from observations in our direct vicinity, inside the solar system to studies of the very early universe some 13 billion light-years away. Constructing and operating such delicate scientific instruments in very remote areas is obviously a big challenge, but all of that effort is worth nothing, if we can't analyse and scientifically exploit the data produced by them.

Fundamentally, radio antennas are measuring voltages in an extremely high cadence and across many frequency channels and dual polarizations. These measurements are almost immediately digitized and then transferred over fibre links across Wide Area Network links from the observatory sites in South Africa and Western Australia to dedicated High Performance Computing centers in Cape Town and Perth, respectively. The data rates at this stage of the signal chain are truly eye-watering and reach up to several Terabytes per second (TB/s). This data is directly ingested into bespoke, FPGA based correlators, which perform some averaging and calculate the correlation of the signals from each pair of antennas. The correlators reduce the data rates by about a factor of 10, but the resulting data rates are still of the order of 0.5 TB/s. That data is directly consumed by dedicated High Performance Computing (HPC) clusters.

This is where the job of the so-called Science Data Processor starts and where the contributions and the expertise of my team and myself are concentrated. Correlated voltage measurements are still quite some ways from something like an image, which typically is the starting point for actual scientific investigations. Getting from those raw inputs to calibrated data products, ready to be analysed by scientists requires compute clusters with several hundred Petaflops (PFLOPs) performance (1 PFLOP is 10**15 floating point operations per second) and is thus far 18 out of reach of any normal installation. This compute capacity has to be available continuously, since the telescopes can observe the whole day long, every day and the whole year. The resulting data products are still massive and tick in at up to 1 Petabyte (PB) for every observation taking between 6 and 12 hours. One PB is the equivalent of 1000 typical laptop storage drives today and thus still far too big to consume on a normal, single computer. In this talk I will present the computational and data management challenges and some of the solutions to maximize the scientific output of the SKA and other large-scale astronomical facilities and outline the limitations and opportunities along the way to support achieving transformational science results and potential Nobel prizes.

Since 2012, to commemorate fifty years of digital computing in Western Australia, the WA Branch of the ACS has invited a distinguished scholar and researcher with a connection to WA to present a lecture on the leading edge of an important and emerging area of information and computer technology.

Press the play button to watch the short video of past WA Dennis Moore Orators and the topics presented