ARM | ARM Priorities

Task Id	Title	State	Target Completion
ENG0004847	Automation and Integration of Epochs in ARM Data Discovery	In Progress	2026-09-30
ENG0004950	Develop an AI assistant for discovering, understanding, and ordering ARM data	In Progress	2026-09-30
ENG0005042	Develop AI-assisted tool for historical reprocessing backlog planning and organization	In Progress	2026-03-31
ENG0005079	ATLAS (Agentic Tooling and Large Language Model (LLM) Augmentation Stack)	In Progress	2026-09-30
ENG0005089	Ops Tools Framework	In Design	2026-09-30

Task Id	Title	State	Target Completion
ENG0004533	AMF3 BNF: Developing a Network Node for Spatially-Distributed Aerosol Measurements for Deployment of the AMF3 to the Southeast United States	In Progress	2026-12-31
ENG0004539	Procure third generation C-band precipitation radar	In Progress	2026-12-31
ENG0004574	AMF3 BNF: Aerosol Flux Measurements on the Tower - CPCs	In Progress	2027-12-31
ENG0004877	Setting up SMPS and CPC gold-standard calibration capabilities for ARM AOS instruments	In Progress	2026-12-31
ENG0004964	Adaptive Scanning Capabilities for ARM	In Progress	2026-06-30
ENG0004972	2025 ARM Lidar Intercomparison	In Progress	2026-12-31
ENG0005082	Energy Impacts: Portable Atmospheric Profiling Systems	In Design	2027-04-30
ENG0005085	Aerosol Extinction Lidar for Mobile Facility Deployments	In Progress	2026-12-31

Task Id	Title		State	Target Completion
ENG0005026	Development of an AI-based MASC Particle Classification VAP		In Design	2026-09-30
ENG0005049	LARS: LLM-Assisted Radar Scene classification using semi-supervised learning		In Progress	2026-09-30

Task Id	Title		State	Target Completion
ENG0004938	ARM Website: AI-Powered Search Enhancements		In Progress	2026-08-31

Automation and Integration of Epochs in ARM Data Discovery

The identification and creation of Data epochs (selected QCd datastream(s) over periods of scientific interest) are currently done manually and added to the ARM database for Data Discovery use. These epochs represent signatures of "special events" at ARM sites (e.g., Hurricane Hilary at EPCAPE, Tonga volcano blast at ENA) or atmospheric phenomena and states of interest. "Special events" are sparse and unique and, therefore, require manual identification, tagging, and linking to relevant datastreams. However, identification and tagging of commonly known and widely examined atmospheric phenomena and states of interest such as surface precipitation, new particle formation, and cloudiness state, can be generalized and automated, often with one or more high-level (QCd) ARM datastreams. This generalization and automation thereby reduce overhead, streamlining the creation-to-publication of epoch workflow, and improving the user experience.

The purpose of this ENG is to develop, implement, and automate epoch creation and publication of commonly known atmospheric phenomena and states of interest that will:

1. Improve data utilization by end users in Data Discovery.

2. Support active field campaign PIs in finding days of interest in near-realtime through the ARM Field Campaign Dashboard.

3. Provide additional epoch-based features to the ARM workbench.

4. Facilitate the basis for setting virtual field campaigns per ARM's Decadal Vision.

Develop an AI assistant for discovering, understanding, and ordering ARM data

The ADC is uniquely positioned to take full advantage of the recent AI and ML breakthroughs. Projects such as a data recommendation pipeline, a Primary Measurement Type (PMT) classifier, an anomaly detection deep-learning model, an epoch detection machine learning model, metadata abstract generation for datastreams using LLMs and automated metadata generation for datasets in collaboration with USGS, along with various other machine learning projects, showcase the capabilities of the ADC's AI Innovations Team. Collaboration with the ESIP AI Data Readiness and the USGS FAIR Assessment for AI Readiness working groups, along with the various AI initiatives at ORNL—the world's premier research institution—provide ARM with cutting-edge AI and ML research. Current ADC AI initiatives have been recognized by the Emerging Tech and Computing Group and Artificial Intelligence Program at ORNL as candidate projects for testing and leveraging Forerunner, an ongoing project with the goal of helping AI projects easily build ML pipelines with ORNL's available computing resources. Several members of the ADC have presented multiple AI- and ML-focused posters during various events and conferences, including ORNL's AI Expo and the ARM/ASR PI Meeting. These initiatives have placed the ADC at the forefront of emerging AI and ML technologies and have opened the door to a plethora of revolutionary possibilities going forward.

Improving the usability of Data Discovery has remained a high priority for the ADC AI Innovations Team. Data Discovery is designed to fulfill two major functions: helping users locate data and then providing access to those data. It becomes apparent that the first function is the most complex, as it requires users to meet several preconditions before Data Discovery can provide the data. These preconditions include, but are not limited to: determining which data are suitable for a given line of research; becoming familiar with the Data Discovery interface; learning the hierarchy of ARM metadata; understanding ARM jargon and acronyms; assessing ARM Data Quality Reports (DQRs); determining the ideal data format for placing data orders (such as file concatenation, file formats, DQR filtering); and understanding the different delivery methods.

The solution is the ARM Data Advisor (ADA): a revolutionary new way for users to understand, navigate, and obtain ARM data. The name is inspired by Ada Lovelace, recognized as the first computer programmer. Users will interact with ADA through a conversational interface, expressing their needs via sentence prompts. For example, a user might ask ADA, "I'm a researcher new to ARM; where can I find latent heat flux measurements?" In response, ADA could provide information such as, "I recommend you begin by reviewing latent heat flux measurements collected by the 30ebbr instrument at ARM's Southern Great Plains (SGP) site. The SGP site is the longest running permanent ARM site, consisting of 81 total facilities and 1540 individual data products." ADA is anticipated to enhance the user's experience by providing additional information about relevant and related data products, including data epochs, data quality information, visualizations, locations on a map, related publications, links to arm.gov, and more insights from Data Discovery. ADA could also incorporate functionalities that assist users in exploring various metadata ontologies. Users might wonder how measurement descriptions from other metadata ontologies, such as NASA's GCMD (Global Change Master Directory) Keywords, SWEET (Semantic Web for Earth and Environmental Terminology), or Data Catalog (DCAT), align with ARM's hierarchical measurements. ADA will be equipped to facilitate this exploration by using ML to semantically match and translate these external metadata ontologies, allowing users to uncover how well these measurements are classified within ARM's framework. The interaction would continue as users prompt ADA with additional requests to refine their search for data products. Once ADA has generated a list of data products appropriate for the user, ordering the data would be as easy as clicking a button. Results would not be limited to routine ARM data but could include Intensive Operational Period (IOP) data as well, and even data from external data repositories such as Next-Generation Ecosystem Experiments (NGEE) Arctic, Environmental Molecular Sciences Laboratory (EMSL), and Aerosol, Clouds and Trace Gases Research Infrastructure (ACTRIS) to further promote the principles of FAIR (Findable, Accessible, Interoperable, Reproducible) data. ADA could potentially replace the guided search in Data Discovery or serve as a standalone application to assist and educate less experienced users.

Develop AI-assisted tool for historical reprocessing backlog planning and organization

I am responsible for working through a sizeable historical backlog of reprocessing jobs. My approach differs from past attempts: rather than processing jobs sequentially, I cycle between executing work and researching the backlog to identify and group related jobs for more efficient processing.

A primary bottleneck is research. Information about related issues is scattered across multiple ServiceNow tickets spanning different sites, facilities, and time periods. Critical details are often buried deep in lengthy comment threads - a note in one old ticket may affect several other jobs, but I must manually read through all conversations to find these connections. This manual research is extremely time-consuming.

Proposed Solution:

Develop a personal productivity tool using existing AI services (ChatGPT/Claude APIs) to accelerate my backlog research process. The tool would aggregate information from ServiceNow tickets and ARM tool APIs, then present organized summaries, identify relationships between tickets, and provide planning recommendations based on domain-specific knowledge.

For example: instructions for using a recalibration tool needed for several jobs might be buried deep in one ticket's comment thread. Currently, I must manually read through all conversations to find this information. The tool could surface and connect this across related jobs.

ATLAS (Agentic Tooling and Large Language Model (LLM) Augmentation Stack)

To support ARM AI/ML initiatives with the necessary resources and the capability at scale, and to consolidate and operationalize emerging AI capabilities within the ARM ecosystem, the ARM Data Center (ADC) introduces ATLAS, the Agentic Tooling and Large Language Model (LLM) Augmentation Stack. ATLAS represents a cohesive, agentic foundational stack designed to enable secure, scalable, context-aware agentic workflows by providing a standardized and extensible framework for accessing the advanced computational infrastructure at the ADC. The ATLAS implementation plan includes provisioning high-performance GPUs, the installation of high-performance LLMs onto ADC resources, exploring existing DOE leadership computing facility avenues for inference and compute power, implementation of standardized interaction protocols, including Model Context Protocol (MCP), Agent-to-Agent (A2A), and Agent-User interaction (AG-UI) enabling multi-agent coordination and Agentic Retrieval-Augmented Generation (A-RAG), a centralized vector store for shared contextual memory layer and persistent embeddings, and a templated approach to agentic tooling integration.

Several emerging AI/ML initiatives across the ARM program are leveraging the compute and inference capabilities provided by ATLAS, including the upcoming AskARM ChatBot (ENG0004938), the ARM Data Advisor (ADA) (ENG0004950), and the Workflow Automation Service Platform (WASP). Preliminary ATLAS documentation is provided for initiatives wishing to contribute to and interface with ATLAS at https://atlas-docs.svcs.arm.gov.

A large portion of the ATLAS architecting and development progress was encompassed within the ADA initiative, and ENG0004950 includes monthly summaries covering the instantiation of the various ATLAS modules. Please refer to the notes section of this ENG to find summaries of ATLAS progress from ENG0004950. We are creating this ATLAS ENG to better capture ATLAS-related progress moving forward.

Ops Tools Framework

The ADC Operational Tools team upgraded Ops Dashboard to the Ops Portal which led to huge improvements across the applications within it. This ENG (Previously code named WASP) is for the next round of improvements to bring even more capabilities to the ARM Portal. The natural progression of this tool requires more complex workflows, contextual relations, and advanced AI capabilities. The Ops Platform addresses these needs by adding a workflow engine capable of integrating with all of ARM's databases, services and systems to greatly improve business logic implementation and management. The additional contextual capabilities being added across the codebase and workflows will significantly improve development time and enable a new capability for rapid high fidelity prototypes. Lastly, a new assistant within the Ops Portal will be added with tool capabilities for the apps within the Ops Portal. This will give users a new way to interact with tools and perform the actions needed for their daily activities.

AMF3 BNF: Developing a Network Node for Spatially-Distributed Aerosol Measurements for Deployment of the AMF3 to the Southeast United States

Within a global climate grid cell, aerosols exhibit high spatial variability due to their disparate sources, short atmospheric lifetimes, strong coupling to the land-surface heterogeneity via surface heat, mass, and momentum fluxes, as well as the nonlinear, interdependent physical and chemical processes that control their properties. Land-surface controls on aerosol sources (e.g., vegetative BVOC emissions, anthropogenic SO2/H2SO4 emissions), sinks (e.g., wet/dry deposition), and transformations (e.g., aerosol uptake of atmospheric water vapor) tend to vary significantly over short distances, leading to complex interactions between atmospheric aerosol evolution and the dominant spatial patterns of relevant surface fluxes. Aerosol advection further obscures characterization of land-surface aerosol controls due to simultaneous mixing and aerosol processing during transport. Land-surface controls on aerosol present a significant observational and modeling challenge, limiting our ability to understand and predict aerosol effects on radiation, precipitation, and thus weather and climate in heterogeneous regions, as exemplified by the agricultural, forest, rural, and urban atmospheric environments impacting the deployment of the AMF3 in northern Alabama. Historically, complex measurements of the range of aerosol properties have been made at a single point, which informs our understanding of the relationship among aerosol physical, chemical, and optical properties. To address the current knowledge gaps however, spatially distributed aerosol measurements are critically needed to understand land-surface controls on the variability in aerosol-climate impacts, particularly in the Southeast United States.

While heavily-instrumented "point" measurements (e.g., AOS) have proven immensely useful for process-level science, they may not necessarily be representative of the larger atmospheric environment. However, it will be challenging both financially and operationally to deploy an AOS network due to the costs associated with such comprehensive instrumentation, infrastructure, and labor, as well as the operational complexities associated with siting and installing an AOS (e.g., power, communications, land-use agreements). Instead, we propose to develop an aerosol measurement network that meets both the measurement requirements needed to characterize aerosol-climate impacts, as well as the operational requirements needed to support a lower-cost, lower-complexity system.

Procure third generation C-band precipitation radar

The CSAPR2 was obtained to provide a deployable precipitation measurement capability (ENG0000906). The radar was originally designed to operate on a trailer and without a radome; however, as use cases were explored, it became clear that this configuration would not permit operation in kind of conditions (rain, wind) that were of most interest. Therefore, a system was reconfigured to include a radome and a platform that could support the pedestal+radome configuration. That system was first deployed to Argentina for CACTI and is currently deployed to Houston for TRACER for its second mission. Both of these campaigns have a strong focus on deep convection and associated precipitation, and the CSAPR2 is a great tool for probing those systems. The next deployment for the CSAPR2 will be an extended-term deployment to the southeast United States. This deployment is expected to last at least five years so the CSAPR2 will not be available for deployment with other AMF1 or 2 deployments for the foreseeable future.

The purpose for this ENG is to purchase a new deployable radar that can take the place of the CSAPR2 for upcoming mobile deployments and that could also potentially be deployed at the SGP or elsewhere.

AMF3 BNF: Aerosol Flux Measurements on the Tower - CPCs

As part of the AMF3 tower measurements, there is a need from the science team to measure aerosol fluxes at multiple heights. The science team has proposed deploying 4 CPCs at different heights (Top, Above, Below, 10,) to measure aerosol concentrations and combine them with 3D-winds from the TOWERMET system to produce an aerosol flux product. This aerosol flux product will be handled in a separate ENG at a later time once the process is refined. An inlet and housing for the CPC will need to be designed for implementation on the tower.

For each CPC, we will collect and process it like a normal CPC datastream to a1 and b1 levels which will result in 4 sets of datastreams with facilities of S10, S11, S12, S13.

Setting up SMPS and CPC gold-standard calibration capabilities for ARM AOS instruments

The main goal of the Center for Aerosol Measurement Science (CAMS) is to provide regular field instrument calibrations to ARM AOS instruments using standardized equipment and traceable methods to ensure uniformity, transparency, and quantification of instrument calibrations and operations.

In FY25, the AOS team will initiate calibrations of particle size and number concentration instruments with the goal of achieving calibration of two ARM SMPSs (SGP unit and the transfer standard) and the SGPs CPCs by the end of the fiscal year. Following the development and approval of a gold-standard aerosol instrumentation calibration plan and processes for particle size and number concentration, we hope to expand these protocols to other AOS measured aerosol properties (i.e., composition, optical properties, hygroscopicity, coarse mode aerosol).

The procurement of calibration instrumentation for the establishment of a gold-standard reference for size distribution (SMPS) and number concentration measurements (CPC) started in FY24 (see ENG0004592). This equipment will be installed at the ARM's calibration lab, located at the Brookhaven National Laboratory (BNL) CAMS. These standards will then be compared against World Calibration Center for Aerosol Physics (WCCAP) standards every year.

The milestone-deliverable is included in Table 1 (attached) and it is to assist in tracking progress towards the goals identified above.

Adaptive Scanning Capabilities for ARM

Adaptive scanning of ARM's radars and lidars is an important capability that could further enhance the scientific impact of these systems. Previous efforts like that during TRACER have showcased the feasibility of adaptive scanning while utilizing ARM's scanning radar systems (CSAPR2). There is a need to develop an open, consistent, science rules driven, and reusable framework in ARM for implementing these requests with the CSAPR2 (BNF, DUSTIEAIM) and eventually other instruments.

It is proposed that ARM dedicate resources towards the development of this framework for interacting with ARM's CSAPR2 radars in an open and modular fashion to allow extension to other instruments as well. While the current method to allow PI's access to change configuration files has worked for TRACER, it introduces concerns related to security and potential damage to the radar.

While the new CSAPR2 should have storm tracking software that can operate by itself, it would also be beneficial for ARM to develop an internal system for cell-tracking and relaying scan requirements to the CSAPR2s to allow cell-tracking capabilities for users that may not have the capabilities themselves to allow for a more robust and flexible tracking system.

This effort is broken into two phases of development that can be worked in parallel.

Full google document can be found here: https://docs.google.com/document/d/1oEuIVDKguSkb2pmo_7ucmr387qEk-8099A5djL8ZNB4/edit?usp=sharing

Phase 1: API for Interfacing with the CSAPR

-------------------------------------------------------

The goal is to demonstrate the capability to adaptively scan the new CSAPR2 at DUSTIEAIM. While the radar will be different from the BNF CSAPR2, we propose to initially start testing and the development of a "plug-in" for interacting with the BNF system in the summer of FY25 when there are no chances of precipitation.

The first task (1.1.1) is to have base scripts that can send a sequence of scans to the BNF CSAPR2 by September 30, 2025. This will require the developers gaining access to the CSAPR computers and working with SDS and the engineering team on how to provide instructions for scanning. For TRACER, this included modifying a configuration file. Existing code from TRACER to interact with the radar can be found in https://code.arm.gov/instrument/CSAPR_Cell_Tracking (Java code). The second part of this (1.1.2) is to develop the "plug-in" for the DUSTIEAIM CSAPR once installed and verified as operational.

The second task (1.2) of this phase is to develop a middleware layer or API to translate scan requests from PI's, mentors, or other users as needed into valid radar instructions. This layer would verify the user against a known list of users authorized to access the radar and during which periods, and verify that the proposed scan strategy will stay within the engineering limits defined by the mentor before sending instructions to the radar.

This middle layer system would take in a variety of input such as

ARM username and token (data discovery) to verify approval to operate. We should only allow one PI at a time to direct the CSAPR with overrides for the radar mentor team

Code for the instrument system to apply to (BNFCSAPR, SGPXSAPR, etc..)

Scan Metadata (Further defined by task 1.1)

Center of azimuth sector

Limits of azimuth sector

Elevation limits

Sequence of PPI elevations or RHI azimuths

Ideally within a few seconds, the inputs should be verified and the scan information sent to the radar. The system should also be able to default back to an approved operational scan strategy once the input has timed out. This may require further discussions with the radar mentor team on possibilities that do not require manual intervention with the radar.

This system should be modular to allow for the easy addition of additional instruments such as the scanning cloud radars (SACR) or scanning doppler lidars as "plug-ins". The API should be completed by December 2025 to allow for testing with the DUSTIEAIM radar prior to DUSTIEAIM.

Phase 2: ARM Cell Tracking

----------------------------------

The second phase (which can be developed in parallel with phase 1) is to develop cell-tracking capabilities to provide input into the API, similar to what was done in TRACER. This framework should include lessons learned from existing frameworks and work with existing open-source cell-tracking software. The first task (2.1) will be to engage with the scientific community around potential needs for this framework. It is anticipated that it should be able to run solely off ARM's real-time CSAPR2 data flow (as SDS set up for TRACER) but could also operate with real-time external data such as NEXRAD. The second task (2.2) is to review existing cell-tracking software and choose a system to base the initial ARM-Track system on taking into account feedback from the community.

The goal is for this system to analyze near-real-time data from the CSAPR, identify cells of interest based on a set of default requirements as defined by ARM and scientific requirements defined by PIs, and provide the necessary information to the ARM-Scan API.

Task 3 (2.3) will be the creation of this framework around a cell-tracking software to allow for varied inputs for tracking but also varied requirements from ARM and the science community. This system should take into account the needs of the radar mentor team for routine PPIs and scans over the main ARM site to aid in calibration and quality control. All metadata from the tracking should be saved and processed into an ARM-standard product for the users.

After verifying the consistent output from the tracking software, it should be tested with one of the CSAPR2's (2.4) during either maintenance mode with the new CSAPR2 prior to DUSTIEAIM or with the BNF CSAPR2 during a day free of precipitation.

The final task of phase 2(2.5) would be the development of a GUI interface. This interface would display real-time data from the radars and allow the ability to select cells based on the real-time data and have the radar track the cell selected to the end of life or upon new input from the user. The interface should allow the display of all variables through a selectable interface. Additional settings such as azimuth width could be entered by the user but also would need to be calculated from the cell information.

---------------------------

Depending on the speed of this entire system, edge computing may need to be explored. All these systems should be modular enough to operate on different systems as may be required.

2025 ARM Lidar Intercomparison

ARM deploys a wide variety of lidar systems across the fixed and mobile facilities including ceilometers (CL-31), micropulse lidars, doppler lidars, raman lidars, and high-spectral resolution lidars. Additionally, ARM is looking to procure a CL-61 ceilometer, water vapor lidar, aerosol profiling lidar, and perform a buy-back with the vendor to trade-in a MPL for a new miniMPL UHD. With this growing inventory of lidar systems it is important to understand the performance characteristics and assess which systems meet the scientific objectives of ARM's users. To support the development of a strategy for ARM's lidar systems, it is proposed that ARM perform a lidar intercomparison at SGP, under a variety of atmospheric conditions, to benchmark performance, assess measurement consistency, and identify opportunities for improvement.

The goal of ARM's lidar strategy is to ensure a robust, well-characterized, and scalable set of lidar capabilities that support cloud, aerosol, and water vapor observations across ARM sites. This strategy balances operational reliability, measurement performance, and calibration stability, while also guiding the evolution of ARM's lidar network to meet future scientific and modeling needs.

Timeline will be dependent on delivery of the new lidars systems but is intended to run for 1-3 months, depending on conditions and potentially more for systems that are not intended to be deployed at an AMF.

In order to better understand and compare the lidars, it is proposed to analyze the following metrics.

Instrument Specification Comparison (temporal/vertical resolution; min/max ranges, mean time between failure if known)

Measurement Accuracy and Precision including

SRN during day and night

Backscattered signal sensitivity to cloud/rain/ice

Backscatter consistency in highly reflective layers

Low-level backscatter consistency

LDR consistency

Cloud base height detection consistency across different ranges (0-5km, 5-10km, 10-15km; thin vs thick clouds; CEIL/MPL) including day/night

Cloud base height and signal sensitivity vs. height

Aerosol extinction profiles (MPL/HSRL/RL)

Water vapor profiles (RL/DiAL)

Spatial/Temporal Consistency including

Coverage (Are they seeing the same layers) -

Variability across deployments. More generally, differences between two instruments of the same model

Day/night cycle agreement

Detection timing (Agreement when features are detected)

Performance under adverse conditions (rain, fog, etc)

Energy Impacts: Portable Atmospheric Profiling Systems

To support emerging energy-related research identified through the recent ARM Energy Workshops, ARM will develop two portable atmospheric profiling systems designed to provide high-quality observations of boundary layer structure, thermodynamics, winds and precipitation processes. Improved understanding of these processes is intended to enhance the predictability of high-impact events relevant to energy infrastructure, such as winter weather and severe storms.

These profiling systems will include a mix of existing and new-to-ARM instrumentation including microwave radiometer (MWR), atmospheric emitted radiance interferometer (AERI), doppler lidar (DL), surface meteorology (METWXT), laser disdrometer (LDIS), and an micro-rain radar (MRR). Additional instrumentation will be considered in conjunction with future AMF deployment needs. Together these will be deployed on infrastructure such as a trailer to make it easy to bring to locations.

This ENG will focus on designing and implementing a standardized, portable platform that houses the instrument suite in a system capable of reliable operations across a range of environments and conditions including the ability to deploy off-grid. The effort will leverage lessons learned from previous mobile and deployable systems conducted by labs, universities, and/or other organizations.

Aerosol Extinction Lidar for Mobile Facility Deployments

Characterizing the vertical distribution of aerosols and their microphysical properties is important for aerosol research related to new particle formation, aerosol optical properties, and aerosol-cloud interactions. ARM's long-term observatories have or will have advanced lidar such as HSRL and Raman Lidar to help constrain the extinction measurement. There is an also an HSRL that deploys with the 2nd ARM Mobile Facility. This ENG is to procure a lidar that is capable of measuring aerosol extinction and portable enough to deploy with the an AMF. ARM previously in 2025 released an RFP to procure and aerosol extinction lidar but the proposals did not meet ARM's specifications.

Development of an AI-based MASC Particle Classification VAP

The Multi-Angle Snowflake Camera (MASC) Particle Classification algorithm from Schmitt et al. 2025, applies modern artificial intelligence and machine learning (AI/ML) techniques to automatically classify hydrometeor shapes and characteristics from MASC imagery collected at the ARM North Slope of Alaska (NSA) site. It is proposed that ARM incorporate this work into a standard ARM VAP.

This effort directly supports ARM's AI/ML Triennial AI/ML Plan by operationalizing an already existing AI/ML workflow within ARM and will enhance ARM's capability to characterize winter precipitation microphysics by providing objective, scalable classifications of particle habit. These classifications, along with quality indicators will yield higher-level insights into precipitation processes and their variability across seasons and storm types which is key in supporting the NSA precipitation group that is currently meeting to discuss how we can advance the science at NSA.

LARS: LLM-Assisted Radar Scene classification using semi-supervised learning

Introduction

The impact of severe convection on energy infrastructure strongly depends on storm morphology. In particular, supercells are more favorable for producing large hail and violent tornadoes that cause more localized, extreme damage while mesoscale convective systems more commonly produce widespread wind damage and weak tornadoes. Given that the ARM Facility is looking to respond to the Administration's priorities on energy security, producing a convective scene classification product from ARM Radars will assist in this task by organizing the hundreds of TB of ARM scanning radar data by their storm morphology. Having such classes helps users navigate ARM's scanning radar products, finding cases that would be most useful for studying the impacts of extreme weather on energy infrastructure. We will develop LARS, an LLM-Assisted Radar Scene classification product that provides the scene classification as a metadata attribute. To create LARS, we will leverage multimodal LLMs (mLLMs) for automation of scene labelling to create a labelled training dataset for semi-supervised learning. Using this labelled dataset, we will fine-tune various pre-trained computer vision (CV) models to classify radar scenes.

Methodology

First, we will create a training dataset of labelled CSAPR2 0.5 degree PPI scan images using all of the BNF CSAPR2 CMAC data from 2025. First, we will hand-label 300 CSAPR2 PPI scans into 6 categories: no precipitation, stratiform rain, snow, scattered convection, linear convection, and supercells. We will then use these hand labels to develop prompts for various mLLMs (GPT-5, LLAMA, etc.). Every prompt will be the same multiple choice question:

"What is the morphology of the precipitation in this radar image? Please choose between these seven choices:

No precipitation

Stratiform rain

Snow

Scattered convection

Linear convection

Supercells

Unknown"

The hand labels provide the answers for these prompts.

Using Argonne LDRD funds, ANL created a workflow called Nepho to ingest atmospheric data plots into multimodal LLMs (mLLMs) and evaluate the performance of mLLMs. Using a wide array of atmospheric data plots and prompts, GPT-5.0 answered 66% of the prescribed questions for various ARM and other atmospheric datastreams correctly to describe a wide array of atmospheric datasets for 144 data plots. This therefore shows promise for mLLMs for automating the labelling of radar images for developing training datasets.

LARS will build upon Nepho by expanding its testing dataset to include about 300 radar 0.5 degree reflectivity images from BNF. We will use Nepho to prompt the mLLMs the above given prompt for each image in the hand-labelled dataset and evaluate the performance of each mLLM for this hand-labelled dataset. While mLLMs can categorize radar images, for operationalization of radar classifications, mLLMs will be far too slow to operate over hundreds of thousands of radar images. Deep learning models, on the other hand, are most appropriate for scaling to hundreds of thousands of scans. Therefore, we will use Nepho to automatically label 4000 images for training deep learning computer vision models. For each automatic label, all storms labelled as "Unknown" will then be hand-labelled into the other six categories when developing this training dataset.

We will then fine tune deep learning computer vision models (ResNet50, ResNet101, ViT) using workflows developed under LDRD funding by Bobby Jackson and Seongha Park from ANL for deep learning models that predict the location of a lake breeze on the KLOT NEXRAD radar available here (https://github.com/rcjackson/adam). This workflow was also used for ARM Doppler lidar images by Jackson et al. (2023). We will modify this workflow to perform the classification task over CSAPR2 data. We will use this workflow to fine-tune and evaluate pre-trained ImageNet models and ViTs over the training dataset. Finally, we will release the best performing fine-tuned ImageNet models to develop LARS as a Python with continuous integration and continuous delivery (CI/CD), using HuggingFace for model storage and GitHub for code storage. We will then release the classification product as a c2-level time series product for BNF, allowing for users to download a reduced dataset amenable for both manual exploration and RAG pipeline integration.

Scope and Deliverables

Personnel

Bobby Jackson: Machine Learning Engineer, will develop entire pipeline

Scott Collis: ARM Radar Translator, mentoring

References:

Jackson, R. C., and Coauthors, 2023: ARMing the Edge: Designing Edge Computing–Capable Machine Learning Algorithms to Target ARM Doppler Lidar Processing. Artif. Intell. Earth Syst., 2, 220062, https://doi.org/10.1175/AIES-D-22-0062.1.

ARM Website: AI-Powered Search Enhancements

The ARM website serves as the primary gateway for new users seeking information about ARM, offering access to a vast array of content, including scientific publications, databases, WordPress-embedded resources, technical reports, and news articles. A robust search tool is essential for navigating this diverse content efficiently. However, the current search functionality faces challenges in delivering highly relevant results, as it relies on a traditional keyword-based approach that lacks contextual understanding and semantic relevance. Keyword-based search solutions rely on exact term matching and simple relevance scoring, which often leads to suboptimal results, especially in a large and diverse document repository like ARM's. Users may struggle to find relevant content due to issues like synonym mismatching, lack of contextual understanding, and difficulty handling complex queries. Additionally, keyword-based search does not account for semantic relationships between words, leading to irrelevant or incomplete results. AI-powered search, leveraging technologies like natural language processing (NLP) and vector-based retrieval, can significantly improve search quality by understanding user intent, ranking results based on contextual relevance, and providing more intelligent recommendations. AI can also enhance search with features like query expansion, auto-suggestions, and summarization, ensuring that users can quickly and accurately find the most relevant content within the repository.

This ENG outlines a plan to integrate AI into the ARM.gov search tools to enhance result relevancy and help users find information more efficiently. As part of the ENG for the ARM.gov website redesign (ENG0004709), we leveraged a pipeline developed by the PNNL team for another EERE-funded project to prototype a chatbot and summarization tool. However, initial testing revealed that the large and diverse nature of ARM data necessitates additional AI techniques beyond those used in the prototype to achieve more comprehensive results. This ENG breaks this effort into two phases. Phase I focuses on adding semantic search capabilities to the arm.gov search tools to improve the relevancy of results and adding a summarization tool for a Google-like search experience. Phase II involves expanding search capabilities to enable more complex and nuanced question answering by leveraging AI techniques to query multiple data sources, and developing a user-facing chatbot that utilizes these advanced search workflows to provide comprehensive answers about ARM.

As part of this effort, we will develop reusable code templates, enabling these AI-driven techniques to be quickly integrated into other ARM tools for broader applicability and impact.

ARM Priorities

Related ARM Documents

Data Services & System Engineering

Instrument Engineering

Science Products

Technical Coordination

Follow Us:

Keep up with the Atmospheric Observer

ARM User Profile

Data

About

Resources