Embedded Files
AGU, Washington DC & NeurIPS, Vancouver 2024
AGU, Washington DC & NeurIPS, Vancouver 2024
It was a busy week for Emily O'Riordan before Christmas, who presented our DeepWeather work at AGU (American Geophysical Union) in Washington DC, and then at NeurIPS (Neural Information Processing Systems) in Vancouver. There was lots of interest in what we’ve achieved so far, and Emily has brought back all she’s learnt from the amazing talks and presentations at these two events.
MetSoc 2024
MetSoc 2024
Greg, Emily and Lexi attended the annual Meteorological Society of New Zealand conference in Auckland in November 2024. Greg gave a lightning talk outlining the DeepWeather AI-NWP model structures that we are currently experimenting with. Emily discussed some of the technical details of the AI models, including some preliminary results of the DeepSensor model. Lexi gave a presentation on the Duplexity Python package currently in development (and available on Github here: https://github.com/lexixu19/Duplexity) which provides users with an easy-to-use framework for validating weather forecast models.
Several papers regarding this work are currently in development, and we look forward to sharing further DeepWeather results in the near future.
Project Update
Project Update
November 2024
November 2024
We are finalising the deep learning models which are being used to create the high resolution target data. Using convolutional neural processes, we are combining WRF forecasts with station observations to create downscaled forecasts that are adjusted by station values. This has been completed for a number of surface variables, including temperature, pressure and precipitation.
The outputs convolutional neural process models will be used as training data, and the models themselves will be used to create initial high resolution fields to be fed into the overall DeepWeather model at inference time.
A paper is in development regarding this work, and the open access GitHub is available here.
STEM Careers Event August 2024
STEM Careers Event August 2024
On 19 August 2024, the Mātaki Marangai project team held a 1-day STEM Careers event in Gisborne.
Ākonga from Gisborne Boys’ High, Lytton High, Campion College, Ngata Memorial College, Tolaga Bay Area School, Hatea-a-rangi School and Te Kura Kaupapa Māori o Taonga Tuturu ki Tokomaru attended the event. The students from across Tairāwhiti came together at the Mātai Medical Research Institute to hear from five local speakers, who shared their experience and journey from high school to where they are in their STEM career today.
We'd like to thank Mātai Medical Research Institute for hosting the event, and to our speakers: educationist Dr Wayne Ngata, StardustME co-founder Stu Potter, marine biologist Te Aomihia Walker, Rocket Lab launch technician Adam Bartlett, and Mātai Research Institute chief executive and medical physicist Samantha Holdsworth. After hearing from our speakers, everyone tucked into a delicious lunch from Puku Ora, and enjoyed a tour around the Mātai Medical Research Institute.
Read more about the event and what the students thought in The Gisborne Herald's article here.
Project Update
July 2024
Project Update
July 2024
WRF is now successfully up and running on NeSI (https://www.nesi.org.nz/)!
We are now testing how well WRF shepherds the neural network we've built, and setting up the coupling between the neural network and WRF.
Initial results are looking good. This graphic shows temperature data from a 1 January 2019 WRF run, showing how WRF generates a lot of spatial detail over and above what is seen in the 25km resolution ERA5 field. Stay tuned for further updates...
Mātaki Marangai project
Mātaki Marangai project
In the DeepWeather team's previous visit to Tairāwhiti, we visited schools and communities in the region to discuss the research being undertaken by the DeepWeather project. As noted in a previous update, Tairāwhiti has been subject to devastating weather events in the past few years, making weather a central focus of the daily lives of East Coast communities.
Along with teachers and community members, we are now running a school-based science project called 'Mātaki Marangai', which translates to 'watching the rain' in te reo Māori. This project has seen eight automatic weather stations (AWS) installed at eight kura (schools) between Tolaga Bay and Ruatoria. Live readings of these weather stations can be seen on the MetService website: https://www.metservice.com/maps-radar/weather-stations/nz.
The maps on the left show the distribution of AWS in Tairāwhiti before (top) and after (bottom) the commencement of the Mātaki Marangai project!
This project also provides rain gauges to 100 students across our eight kura, which they take home and produce readings from every day. The readings can be used for science projects in schools, as well as for personal and community interest. We are working with ākonga-kaimahi (student-staff) who are employed by the project to ensure that students are supported, and to hopefully increase engagement and interest in the young scientists of Tairāwhiti. To read more about the project, visit https://www.mataki-marangai.com/about.
In relation to DeepWeather, the data received through the Mātaki Marangai project will be used to produce a detailed, spatially and temporally high-resolution dataset over Tairāwhiti. The DeepWeather forecasting model will be validated on this dataset, to ensure it performs well for one of the regions most severely impacted by extreme weather events in Aotearoa New Zealand.
American Geophysical Union Conference
December 2023
American Geophysical Union Conference
December 2023
Emily attended the American Geophysical Union Fall Meeting in December 2023, held in San Francisco. The conference had over 25,000 attendees from over 100 countries, spanning a wide range of geophysical disciplines. There were many sessions dedicated to machine learning and AI at the conference, creating ample opportunity for discussions and networking with others working in the intersection of AI and weather forecasting.
Emily presented a poster on the DeepWeather project, which outlined the aims of the project, as well as some initial model results and some future research directions. Click here to view the poster presented.
British Antarctic Survey visit
November 2023
British Antarctic Survey visit
November 2023
Emily recently visited our collaborators Scott Hosking and Risa Ueno at the British Antarctic Survey (BAS) in Cambridge, UK. Emily and Risa are working together to create high-resolution fields over New Zealand using deep learning methods which combine several datasets, including reanalysis data and weather station observations. These datasets will be used to train the DeepWeather model, and will hopefully be of use for other research in the future. Many other researchers at BAS are working on similar projects (mainly focused on Antarctica of course!), so a lot of overlaps and potential collaborations for the future were identified during the visit.
Emily also presented her work on the DeepWeather project for the Artificial Intelligence for Environmental Research group, which lead to a lot of interesting discussions. Many thanks to BAS and University of Cambridge for hosting Emily for the week!
Tairāwhiti Hui
June 2023
Tairāwhiti Hui
June 2023
DeepWeather project members Greg Bodeker, Mark Schwarz, Tui Warmenhoven and Emily O’Riordan visited several communities in Tairāwhiti at the end of June 2023, a trip that has been rescheduled several times due to extreme weather events in the district. Tairāwhiti has been severely impacted by heavy rainfall events and cyclones over the past few years, and faces a future with more frequent extreme weather exacerbated by climate change.
The purpose of the trip was not only to introduce the DeepWeather project to its end-users, but to seek input as to how this project can be developed to best meet their needs. The team met with the Ruatoria Civil Defence Emergency Management (CDEM) group, of which Tui is a member, to discuss how recent weather events have affected them, both individually and as a community. The hui was the start of a working relationship with the CDEM group, whereby the DeepWeather team will seek to regularly engage with the community to share ideas and knowledge.
The team met with communities that have been particularly impacted by recent weather events, including the residents of Makarika and Anaura Bay. Both communities commented on a lack of communication from decision-makers during extreme weather events, as well as the need for funding to mitigate the impacts of such events. The need for reliable energy sources to prevent power outages was discussed, and the possibility of investigating funding for solar panels and gas was raised. Such conversations will be explored by the DeepWeather team and fed back to government officials in our MBIE reporting.
The trip also included outreach at local schools. The team first attended Tolaga Bay Area School, where students gave their kōrero about recent weather events, and asked questions about the weather, artificial intelligence and science in general. At Ngata Memorial College, an interactive session explained how a warm front moves through the atmosphere, with students acting as grid cells within a weather model. At both schools, the idea to equip volunteering students with their own rain gauges to track local rainfall was raised, allowing students to track rainfall at their house or school and compare with others in the area. This could make for an engaging science project, provide useful data for the DeepWeather project, and possibly kickstart an interest in becoming a weather scientist!
The DeepWeather team are extremely grateful to the individuals and communities who gave their time to help us understand the current situation and concerns in Tairāwhiti. Special thanks go to Tui Warmenhoven for her mahi in coordinating this hui.
DeepWeather Project Update
June 2023
DeepWeather Project Update
June 2023
The DeepWeather project team have been collecting and curating appropriate data sources to initialise and train the artificial neural network model. Thanks to MetService and University of Canterbury, we have access to a number of forecast realisations from different models and at different scales. MetService also have access to observation data, such as gauge-corrected rain radar data.
An initial basic model has been developed using XGBoost, taking Weather Research and Forecasting (WRF) model forecasts from MetService as input. Initial tests using this basic model have shown promising results using only a limited source of data; see the figure below for examples of using this model to downscale coarse-resolution precipitation and temperature fields. The downscaled images on the right show better resolved fields at 4km resolution over the whole of New Zealand. The DeepWeather project aims to downscale to higher resolutions than this, but this initial XGBoost model allows us to build a pipeline between incoming data and model outputs. Using this model and domain expertise of the team, we have also been able to select key variables which, when using only a data-driven approach, best predict our selected target variables.
Initial outputs from a simple XGBoost model. A 20km resolution precipitation field (top-left) has been downscaled to a 4km resolution precipitation field (top-right) using the XGBoost model. A 20km resolution temperature field (bottom-left) has been downscaled to a 4km resolution temperature field (bottom-right).
The immediate goals for the near future are to start building a more complex and comprehensive model, including considerations such as physics constraints as well as physical features such as topography and land use. We will investigate the outcomes of different architectures, as well as the effectiveness of transfer learning to allow for computationally efficient training of the model.
12th International Conference on Climate Informatics
April 2023
12th International Conference on Climate Informatics
April 2023
DeepWeather's partner organisation, British Antarctic Survey, co-hosted the 12th International Conference on Climate Informatics at the University of Cambridge in April. DeepWeather team members Scott Hosking, Tom Andersson, and Emily O'Riordan attended what was a fantastic conference, which brought together international researchers and users in climate science, data science, and computer science, to share state-of-the-art developments in climate data and informatics. The aim of the conference series is to accelerate the rate of discovery in climate science and adaptation of climate applications.
To learn more about Climate Informatics, please visit: http://www.climateinformatics.org/
This was also a wonderful opportunity for members of our team in the UK and NZ to meet in-person for the first time!
Stakeholder Meeting
14 December 2022
Stakeholder Meeting
14 December 2022
Thank you to everyone who attended our first stakeholder meeting. All the feedback and suggestions provided were very useful in shaping the programme to generate hyperlocal weather forecasts to suit your needs. The slides from the meeting are available on the left, in case you missed anything.
A report summarising the findings from the meeting has been made. If you would like a copy of the report, please email annabel@bodekerscientific.com so we can review your request.
Page updated
Report abuse