Leveraging Machine Learning to Survey Solar Energetic Particles

This page details my 2024 SURF @ Caltech project with the Space Radiation Laboratory. Special thanks to my mentors Allan and Ashish for providing advice throughout the summer. As this project is ongoing, the codebase is not public; if you have any questions feel free to ask via the email on my homepage

Update 10/9/2024: My work for this project has been accepted as part of a presentation at the AGU 2024 Conference!

Update 12/1/2024: Code for the DMDT analysis portion of the project, as well as a detailed writeup has been published on my Github page!

Update 6/10/2025: My work for this project has been accepted as part of a presentation at the ICRC 2025 Conference!

Introduction and Motivation

During the 11-year solar cycle , the Sun occasionally produces coronal mass ejections (CMEs), releasing bursts of Solar Energetic Particles (SEPs) . These events are composed of intense abundances of electrons/ions at energies up to 100s MeV/nuc, which can not only disrupt critical infrastructure on Earth but also produce atmospheric events.

Understanding SEPs includes measuring their ionic charge states , which indicate the temperature of their source. For example, a source at 5,000-6,000K results in an iron charge state of 10-12, while a hotter source at 10-20 million K produces a charge state of 20+. This analysis helps scientists understand the conditions where SEPs originate .

An airglow over rural Colorado

Although instruments like NASA’s Solar Isotope Spectrometer (SIS) aboard the Advanced Composition Explorer (ACE) do not directly measure ionic charge states, these states can be inferred if an energy-dependence exists within the time decay constants of SEP event profiles. These profiles can also yield info on the types of particle accelerations present. (Sollitt et al. (2008)).

Current methods for extracting decay profiles for charge state inference involve manual and semi-automated algorithms that are time-consuming and prone to implicit bias . To solve this, we leverage machine learning techniques , specifically Multilayer Perceptrons (MLP) combined with time-series analysis, to automate the identification of decay profiles and constants.

Understanding Charge States from Decay Patterns

Identified exponential decay phases are fitted in SEP time-intensity profiles to obtain energy-dependent decay times \(\tau\). For each element, we can write the decay constant (which is \(\tau_X\)) as:

\[ \frac{1}{\tau_X} = \frac{1}{\tau_Y} + W \left(\frac{A_X}{Q_X}\right)^{2\alpha - 1} E^\alpha \]

Here, \(\tau_Y\) is a low-energy decay time due to convection and adiabatic cooling, \(W\) is a normalization factor, \(A_X\) is the atomic mass of the element, \(Q_X\) is the charge state of the element, \(E\) is the energy per nucleon, and \(\alpha\) describes how the mean free path relates to the particle's rigidity.

By fitting this equation to decay data at different energy levels, we can determine the relative charge states (\(Q_X\)) of the elements, using a known calibration element for comparison.

\[ \beta_X = \left(\frac{Q_{\text{cal}}}{A_{\text{cal}}}\right) \left(\frac{A_X}{Q_X}\right)^{2\alpha - 1} \]

Where \(Q_{\text{cal}}\) and \(A_{\text{cal}}\) are the charge and mass of the calibration element, and \(A_X\) and \(Q_X\) are the mass and charge of the element \(X\). This method helps us estimate the charge states at various energies.

Understanding the Dataset

NASA's SIS instrument is capable of measuring the flux intensities of element ions across a range of energies on a 256-second, hourly, or daily basis. It uses a silicon-based detector plate that ionizes incoming charged particles and creates an electrical signal proportional to the energy of the particle. These intensities are expressed in particles/(cm² Sr sec MeV/nucleon)

Sample flux intensity data for Helium over a 5 year period with a calculated noise threshold.

To optimize between detail and time resolution, we opted for hourly averages of flux intensities for the most significant elements. These include C, Fe, He, N, O, and Si across the lowest 4 energies, and spanning 10 years (2014-2024) . Lower energy bins were chosen because they generally have higher flux intensities, reducing noise impact and improving signal-to-noise ratio.

Methodology

Algorithmic Event Detection and Dataset Creation

• In machine learning, datasets are typically split into two subsets. Training data is used to train the ML algorithm, and testing data is used to evaluate the accuracy of the trained algorithm.
• To create the datasets for the machine to learn from, we perform complex data analysis to extract candidate decay events. The datasets will include examples, as well as non-examples of decays that we want our models to learn from.
• The statistical noise threshold defined by 3 · Q1 of the dataset was applied, and a log transformation was applied to linearize any exponential decay patterns in flux data.
• The most abundant element, Helium, was chosen as the reference element for event cutoffs.
• A sliding window algorithm was developed which performs iterative linear regression within each window tick to detect periods of exponential decay in the reference He flux, indicating the presence of a decay.

Sliding Window Algorithm

Interestingly, similar algorithms are used in time series forecasting for the stock market, which involves dividing historical data into windows, training a model (usually LSTMs) on each window, and forecasting the next period's stock prices.

Our algorithm for SEPs uses a similar fixed-size window (specified by window_size ) that slides across the logarithmic flux data along the x-axis. For each position of the window, linear regression is performed on the data within the window. The regression line's slope is computed, and if this slope is less than a predefined threshold ( slope_threshold ), it indicates a significant downward trend, characteristic of an exponential decay . The start and end times of the window are recorded as a decay segment if the slope condition is met.

After identifying all potential decay segments , it proceeds to merge overlapping or adjacent segments . This step ensures that a continuous decay period is not fragmented into multiple segments. The merging process checks if the start time of a new segment is within the current segment being considered. If the segments are close enough (considering the flux increase threshold ), they are merged into a single continuous segment. This merging is done iteratively to cover all identified decay periods.

Here is an interactive graph of all flux data with decay windows graphed. Warning : this can be viewed only on a desktop and is heavily resource intensive.

Further Analysis

Further work was done to analyze all events identified within the data range by filtering each event by the number of elements decaying within the time window. A decay identification algorithm similar to the one used in the sliding window approach was applied to each element within the event window, and the number of elements exhibiting an exponential decay (coefficient of correlation "R" > 0.8) was considered to be a decaying element.

Feature Extraction

• Events with decay phases in 4 or more elements were chosen, resulting in 1260 individual decay profiles.
• Key features were extracted from the identified SEP decays, including decay slopes, event duration, peak intensities, and element ratios across different energy channels.
• Features are also calculated for the dataset of non-decay events, and corresponding target labels (1 or 0) were assigned .

Model Training

The extracted feature set of 1260 manually labeled profiles was trained on a Multilayer Perceptron (MLP) as well as a Random Forest Classifier , both with a grid search to optimize hyperparameters.

Results and Evaluation

Trained Model Performance

• Post-fine tuning via Grid search, we were able to observe that both the MLP and Random Forest classifiers performed admirably, with the RF classifier outperforming slightly due to more robust feature selection.
• The Random Forest achieved a Test Accuracy of 97.62% , and a Training Accuracy of 99.31% , while the MLP model achieved a Test accuracy of 96.83% and a Training accuracy of 98.61% .

Although both models exhibit slight overfitting, this is expected given the complexity and variability of SEP events. The primary goal is to maximize the model's ability to identify decay patterns, and a small degree of overfitting suggests that the models capture important features that contribute to accurate classification without significantly compromising their generalization abilities.

Generalization and Testing

• To assess the generalization capabilities of our model, we test it by using it to analyze unseen data from a different solar cycle. The data used was flux data collected by SIS from 1998-2014 .
• Similar to how the training datasets were created, the new data was preprocessed and evaluated using our Random Forest model to identify SEP decays.
• Results were consistent; however, hallucinations did occur; the model correctly identified 2150/2238 decay traces, amounting to roughly 96%.

Sample decays identified using our Random Forest classifier.

Conclusions, Future Work, and References

Using SEP flux data from NASA's SIS instrument, we have integrated MLP and Random Forest (RF) classifiers , effectively automating the identification and classification of SEP decay profiles for charge state analysis, with models achieving 96.83% and 97.62% test accuracies respectively.

Future plans include:

• Using ML techniques to better handle decay constant fitting, as well as further classification (power-law, irregular).
• Developing automated pipelines to perform charge state analysis using real-time data, as well as data pipelines to adapt other missions.
• Exploring more complex deep learning techniques, such as a DMDT technique used for light curves, or an LSTM analysis.

References

• L.S. Waters, et al. (2007) "The Advanced Composition Explorer". Space Science Reviews , 56, 199-217
• J.C. Kline, et al. (2023) "Analyzing Solar Energetic Particles Using Machine Learning". Astrophysical Journal , 150, 25