The work below was conducted as part of a semester-long project for the 

Advanced Topics in Human/Machine Intelligence (CSCE 6290)

 course at the University of North Texas during the Spring 2025 semester. Most of the descriptions included here were written for a term paper summarizing the results of this project.

The increasing ubiquity of artificial intelligence (AI) tools in modern life highlights a growing need for AI systems that are understandable for ordinary people. However, public opinion on the ubiquity of AI tools differs relative to one’s level of expertise in the technical capabilities and limitations of AI 

, including feelings of optimism and anxiety about AI. Non-experts, for example, are more likely to believe that AI will cause harm to humanity in the future 

, generally describes the ease at which humans can comprehend how an AI agent generates predictions 

. Sonification, which can be understood as the transformation of data into sound 

, is posited to be one way of enhancing AI explainability 

This project examines the ability of existing large language models (LLMs) to generate and comprehend a mapping of words encoded as audio frequencies for use in sonification. The results indicate that LLM outputs can be used for sonification, but models struggle to translate sentences using encoded languages without exposure to the encoded language during training. Finally, a hybrid ensemble architecture is proposed as a means for capturing and generating audio from LLM outputs.

, were tested for their ability to receive encoded prompts, interpret the prompts as language, and generate encoded responses that can be recreated as audio. ChatGPT was also tested, but the results are difficult to reproduce consistently and are excluded here.

Since LLMs are typically designed to interact with and generate textual data, sonification requires formatting audio-based model inputs such that they can be decoded as text, and formatting model outputs such that they can be used to create digital audio. Additional tools, such as software synthesizers, can then be used to produce sound. 

The first step in this project involved creating a character mapping suitable for converting text to audio frequencies, generating sonifications of LLM outputs, and translating sonifications back into text. The character mapping was created by encoding every letter, number, and punctuation mark in the English language as a set of unique numbers representing audio frequencies that fall within the range of human hearing (i.e., 20– 20,000 Hz). Words are formatted as nested lists of floating-point values. Word boundaries are contained within single brackets and sentence boundaries are contained within double brackets (Figure 1).

Simulating LLM conversations using a LSTM translator

An initial test of the sonification pipeline involved simulated conversation loops between two models, LLM-A and LLM-B, in which plain-text sentences are encoded as audio frequencies, translated back into text, and used as prompts. The simulation was achieved by training a long short-term memory (LSTM) model to predict textual characters associated with audio frequencies, then using the LSTM model as a translator between two DistilGPT2 models fine-tuned on the openwebtext dataset. A diagram of the simulation flow is shown in in Figure 2 above.

The LSTM model was created by following a TensorFlow tutorial on audio recognition 

. Using a supervised learning approach, each frequency from the encoded dataset was exported as a WAV file and placed in a folder labeled with the respective decoded character. During model training, WAV files are converted to spectrograms, which the model uses for predicting text labels associated with audio characteristics (Figures 4 and 5). The model was intentionally overfit for testing purposes and not intended to generalize beyond the test set (Figure 6).


The simulated conversation is initiated by running a Python script in a terminal window on a PC. During the simulation, each character of an LLM response is encoded as an audio frequency, exported as a WAV file, and fed to the LSTM for inference. As the model makes predictions, spectrograms for each predicted character are visible on screen. Finally, audio of the predicted response is generated using Python Synthesizer 

The DistilGPT2 model used for this simulation was only trained well enough to be usable for demonstration purposes, but additional hyperparameter adjustments during training might produce less absurd conversations. An example simulation is shown in Figure 7 below and a demo video is available 

NanoGPT and DistilGPT2 models were also tested for their ability to generate responses after being trained on a fully encoded subset of the Open Assistant Conversation Dataset (oasst2) 

. The dataset is structured such that rows alternate between prompts and responses, where every even row (and index 0) is a prompt, and every odd row is a response. 

To encode the oasst2 dataset, each character of a word is encoded using the character mapping and then reformatted. Since the full oasst2 dataset includes 161,443 prompt or response messages in 35 languages, but the original character mapping only includes English characters, only the English subset of the oasst2 dataset was used for model training. The final encoded dataset includes 61,278 rows for model training and 3,235 rows for validation.

Most of the hyperparameters used to train nanoGPT were the defaults provided in the original repository. DistilGPT2 offers different hyperparameter options, but a comparable selection of hyperparameters is shown in Figure 8 below for both models.

The nanoGPT model required ~2 hours and 43 minutes to complete training. Training and validation loss were nearly identical for each step. Loss improved rapidly at the beginning of training then remained flat, offering preliminary evidence that learning occurred. 

The DistilGPT2 model took ~1 hour to train. Like nanoGPT, loss improved throughout training, but validation loss began to worsen by epoch 8 and continued deteriorating throughout the remaining epochs. Plots of the training results for both models are shown in Figure 9 below.

A selection of ten prompts from the validation set were used to test model performance (Figure 10). The plain-text prompts are encoded, tokenized, and provided to the trained model. Model responses are then cleaned, formatted, translated, and evaluated.

The responses from DistilGPT2 after training on an encoded dataset do not resemble anything that could be considered language. Surprisingly, the nanoGPT model appeared to learn patterns from the encoded dataset and was able to generate coherent responses. 

Both models struggled to format responses properly, requiring substantial data cleaning and processing. Hallucinated patterns or incomplete number sequences appear in many responses. It is likely that additional hyperparameter adjustments will improve the performance of LLMs trained to generate encoded text that can be used for sonification. Figures 11 and 12 show the responses for each model.

Hybrid ensemble pipeline for LLM sonification using streamed audio

A second iteration of the simulated conversation was designed to use the LSTM model for predicting characters from a microphone input, rather than generating predictions from WAV files. It is assumed that such an adjustment could make LSTM suitable for inclusion in a hybrid ensemble by reducing the latency between encoded exchanges with LLMs. An example of such a pipeline is shown in Figure 13 above.

To test this idea, a Seeed Studio XIAO ESP32S3 Sense microcontroller (MCU) fitted with a digital microphone was used to for capturing audio. The MCU captures samples at a rate of 16 kHz and streams data to the serial port of an attached PC every 0.1 seconds. The PC connects to the MCU using pySerial 

, decodes the incoming data, and stores the data as chunks (~1,600 samples) in a buffer to prevent blocking. Data from the buffer is then accessed iteratively, converted to a tensor, reshaped, and used to create a spectrogram for model inference.

The LSTM model is highly accurate at predicting character strings from WAV files exported directly from the software synthesizer, but requires perfectly formatted data when invoked for live audio. Additionally, the sample rate necessary for capturing streaming audio exceeds the speed at which the model could be invoked on the hardware used for testing. Nonetheless, an initial test using the MCU indicates that the LSTM model can predict text from live audio using spectrograms in the same way it predicts text from WAV files (Figure 14). Further refinements to the full LLM sonification pipeline are needed before additional testing can occur, which were beyond the scope of work completed for this project. 

It is no secret that LLMs are challenging to train. Even the worst- performing models evaluated here required several hours of training time at 100% GPU utilization on a dedicated server. Training multiple LLMs for optimizing hyperparameter configurations was not feasible in the time available for this project, but should be explored in future work. Additionally, these models use different architectures and were trained using hyperparameter configurations that may not be comparable. As such, the evaluations here may not fully capture potential model performance on these tasks. For example, DistilGPT2 was trained using a transfer learning approach and likely retained patterns learned from previous training data. This may explain why it failed to learn and use new patterns from the encoded dataset. In contrast, nanoGPT likely performed well because it was trained from scratch and retained only patterns from the new training data. Future work should strive evaluate models based on their performance across equivalent hyperparameter configurations to form more accurate comparisons.

The subset of the oasst2 dataset used for training contained numerous mislabeled prompts and responses, including text in languages other than English. A more focused and refined dataset might improve the results observed in small-scale experiments like those described in this work.

Since AI models, including LLMs, will almost certainly see increased deployment on embedded systems and other devices with limited processing power, a secondary goal of this project was to simulate LLM conversations using a Raspberry Pi 5. However, a TensorFlow Lite conversion of the DistilGPT2 model was too large (~120M parameters) to be performant on a Raspberry Pi 5, requiring ~14 seconds to generate a single response. Given the model size and signal processing challenges encountered during testing, the encoded audio-to-text pipeline in this work remains incomplete. As such, it was not possible to evaluate the impact of sonification for enhancing explainability of LLM outputs with human subjects in this scope of work. In the future, researchers should continue exploring the ways in which sonification impacts human understanding of LLM outputs, including radically optimized models for enhancing AI explainability on embedded systems.

Despite the challenges experienced when attempting to use LLMs to generate data that can be used for sonification, nanoGPT was successfully trained on an encoded dataset and capable of generating encoded responses that could be translated into sentences. Additionally, sequences of audio like those produced in the simulated conversations described here might be useful aids for explaining natural language processing (NLP) and how LLMs generate responses. Researchers should continue experimenting with sonification as a means for enhancing explainability of LLMs, and seek to identify sonification techniques that compliment, rather than obfuscate, LLM outputs.

Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, Vince I. Madai, and the Precise4Q consortium. 2020. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making 20, 1 (November 2020), 310. https://doi.org/10.1186/s12911-020-01332-6

Thomas Hermann, Andy Hunt, and John G. Neuhoff (Eds.). 2011. The sonification handbook. Logos Verlag, Berlin.

Hugging Face. 2024. distilgpt2. Retrieved May 1, 2025 from https://huggingface.co/distilbert/distilgpt2

Hugging Face. 2025. OpenAssistant. Retrieved May 1, 2025 from https://huggingface.co/datasets/OpenAssistant/oasst2

Andrej Karpathy. 2025. nanoGPT. Retrieved May 1, 2025 from https://github.com/karpathy/nanoGPT

Colleen McClain, Brian Kennedy, Jeffrey Gottfried, and Monica Anderson. 2025. How the U.S. Public and AI Experts View Artificial Intelligence. Retrieved from https://www.pewresearch.org/internet/2025/04/03/how-the-us-public-and-ai-experts-view-artificial-intelligence/

Yuma Mihira. 2025. Python synthesizer. Retrieved May 1, 2025 from https://github.com/yuma-m/synthesizer

Björn W. Schuller, Tuomas Virtanen, Maria Riveiro, Georgios Rizos, Jing Han, Annamaria Mesaros, and Konstantinos Drossos. 2021. Towards Sonification in Multimodal and User-friendlyExplainable Artificial Intelligence. In Proceedings of the 2021 International Conference on Multimodal Interaction, October 18, 2021. ACM, Montréal QC Canada, 788– 792. https://doi.org/10.1145/3462244.3479879

Simple audio recognition: Recognizing keywords | TensorFlow Core. TensorFlow. Retrieved May 1, 2025 from https://www.tensorflow.org/tutorials/audio/simple_audio

pySerial — pySerial 3.4 documentation. Retrieved May 1, 2025 from https://pyserial.readthedocs.io/en/latest/pyserial.html

Sonification for Explainability in Large Language Models

This work was conducted as part of a final project for the 

 course at the University of North Texas during the Fall 2024 semester. It was a privilege to collaborate with fellow PhD student, 

, on this project. Most of the descriptions here were written for a term paper summarizing these results.

The rising quantity of objects in orbit increases the probability that conjunctions will occur, posing risks to spaceflight operations, astronaut crews, and the general public. Existing conjunction assessments rely on complex modeling of orbital dynamics that are not standardized, which is expected to become problematic as the number of spaceflight operators continues to grow. Our work explores alternative methods for predicting conjunctions using features extracted from publicly available satellite data. The results show how synchronization time is a significant predictor of the probability of collision between satellite pairs.

The United States Space Command (USSPACECOM) provides public access to satellite data via Space-Track [1], including details like the country of origin and telemetry information of different objects in orbit. This data is essential for estimating whether orbital fluctuations reduce the distance between satellites (i.e., conjunctions) such that collisions are possible. 

The number of objects in orbit increased significantly since 2018 and is expected to continue rising [2], [3]. Consequently, the number of collisions that generate space debris is also expected to increase [2], potentially introducing new safety and operational hazards to humans working aboard space stations and to the general public on the ground. Existing techniques for assessing these kinds of risks emerged in 1986 after the Space Shuttle Challenger accident, and are described in a handbook released by the National Aeronautics and Space Administration (NASA) in 2020 [3]. The handbook offers guidance for assessing the risks associated with closely orbiting objects (i.e., conjunction assessment). A comprehensive guide for conjunction assessment is offered by USSPACECOM [4].

In general, conjunction assessment relies on extensive catalogs of orbiting objects, such as the U.S. Space Surveillance Network (SSN), to track orbital fluctuations and predicting close approaches [4]. This process involves daily screening of object positions (i.e., 

), conducting risk assessments, and reporting via a Conjunction Data Message (CDM) when the probability of collision (PC) meets or exceeds a threshold. Objects in Low Earth Orbit, for example, are reported when the PC is greater than or equal to 0.0009. The resulting data are shared with operators in standard formats to allow operators to enact collision avoidance maneuvers when necessary. 

Two Line Elements (TLEs) provide telemetry information used to predict satellite orbits. Data includes an identifier, current time (epoch) in days and years, mean motion with first and second derivatives, BSTAR (drag), inclination, right ascension of the node, eccentricity, argument of perigee, mean anomaly, mean motion, and the total revolutions made by the body [5]. This data offers information about the satellite’s history and orbit path. TLEs are currently the most common format for tracking satellites larger than ∼ 10 cm. A variety of software libraries, such as PyEphem, can parse TLE entries and such that satellite positions can be calculated.

We use statistical modeling, astrophysics software libraries, and open source APIs to identify relationships between variables in publicly available satellite data. The goal of this work is to inform the creation of models that may be used to predict PC scores in a more streamlined and accessible manner than existing methods. We anticipate that the factors leading to conjunctions will be easily extracted from publicly available satellite data.

Though existing methods for predicting conjunctions are expected to be highly accurate, they were not originally designed for assessing the risk of conjunctions [3]. We apply a utilitarian ethic in this work, as we hope to modernize this process for the benefit of a wide range of stakeholders and contribute to the democratization of methods used to conduct conjunction assessments.

 Are easily obtained features of satellites useful predictors of the probability of collision?

 Country of origin and object type can predict the probability of collision. 

Satellite conjunction data was obtained using the space- track REST API. Each row of the dataset features the probability of collision (PC) score associated with pairs of objects in orbit. Object types include debris, rocket bodies, payloads, and unknown objects. Satellite names from the conjunction report were used to obtain TLE sets for each object by querying the N2YO REST API [7] with a Python script. The TLEs allowed us to calculate the approximate position of each object within a limited time frame and plot their orbits using Matplotlib and example code found online [6].

We searched for additional conjunction datasets on Kaggle and NASA’s Open Data Portal using the keywords 

satellite AND (conjunction OR conjunction analysis)

. With the exception of datasets derived from a competition sponsored by the European Space Agency (ESA) in 2020 [8], few other relevant datasets were identified in our search (Table 1). The retrieved datasets were processed using standard analysis and visualization libraries, including Pandas, NumPy, Scikit-learn, Matplotlib, and Seaborn. All project code is stored in a GitHub repository.

We conducted an exploratory data analysis (EDA) to identify the countries associated with highest-risk satellite pairs in the conjunction report. While the conjunction report does not explicitly name the countries of origin for each satellite, the space-track API features a 

 dataset that displays the total count of orbital object types and their associated countries. We used the box score dataset to infer countries of origin without manually pairing each satellite name with its respective country, although such pairing would be necessary if country of origin relates to PC scores in a meaningful way.

Of all objects known to be in orbit, 13,614 are debris, 13,117 are payloads, and 2,353 are rocket bodies. The United States (USA), Russia (CIS), and China (PRC) are each associated with similar amounts of orbital debris (

 = 217.92). The USA is responsible for approximately half as many debris as payloads, whereas Russia and China are each responsible for more than twice as many debris as payloads (Fig. 2).

We then analyzed the object types in the conjunction report to better understand the qualities of satellite pairs in the conjunction report. Approximately 72% of all objects with a probability of collision greater than 0 are debris (Table 2).

The mean and median PC scores are 0.0007 and 0.00026 (

 = 0.00197), respectively, indicating that most objects in the dataset have a PC of 0.00026 or lower (Fig. 3). The pairs of objects with both the highest (0.06) and lowest (0.0001) PC scores are fragments from a CZ-6A rocket body belonging to the People’s Republic of China (PRC).

Spaceflight Safety Handbook for Satellite Operators

 notes how object sizes are factored into PC calculations to inform the creation of a conjunction plane [4, p. 46]. We used linear regression analysis to examine correlations between object types and the PC score for each object pair in the dataset to see if this feature could be extracted. There is a slight slope visible in the regression analysis, indicating that object type is somewhat correlated with collision probability (Fig. 4).

Given that most of the objects in the conjunction dataset are debris, we expected a stronger correlation to exist between object type and PC score. These results suggest that object type alone may not be a useful predictor of PC scores or, by extension, the actual likelihood a satellite may experience a conjunction.

Linear regression was also used to determine if relationships exist between PC and variables not present in the dataset. Specifically, we compared PC scores with the amount of time the orbits of two objects are synchronized. In other words, we assumed that if two satellites orbit in close proximity to one another, their risk of collision may be lower when their orbits are synchronized and higher when their orbits are not synchronized.

Sync times were determined using the TLE for 1,460 satellite pairs in our conjunction dataset, allowing us to calculate orbital periods for each satellite pair using PyEphem [11] and estimate the amount of time orbits are synchronized. We hypothesized that higher sync times correlate with lower PC scores and vice versa.

When two satellites share the exact same orbit and orbital period, they face virtually no risk of collision. However, when the orbital period varies, the orbits are desynchronized and the satellites face a higher risk of collision. In our exploratory data analysis, we found that satellite pairs with the highest PC have low sync times, whereas those with the lowest PC have high sync times. We describe 

 as the time required for two orbital frequencies to synchronize, where 

In the conjunctions dataset, the frequency is provided by the satellite’s TLE as mean motion. Angular speed is calculated using revolutions per day. In circumstances where satellites share similar orbits and their orbital periods are not synchronized, it is reasonable to assume that they are more likely to collide and should have higher PC scores.

This formula can also be used if frequency is unavailable and only the orbital period is known. In such cases, the inverse of the period can be used to determine frequency. Since we compute the inverse difference of two frequencies for use in machine learning (ML), this detail is not expected to be problematic. Models should be able to extrapolate sync time from a difference in either period or frequency. Similar frequencies will have similar periods, and dissimilar frequencies will have dissimilar periods.

Linear regression analysis indicates that sync time has some impact on PC scores. However, there appears to be a fairly nonlinear relationship between PC scores and sync time. The majority of scores fall below the average score (∼ 0.0007) and small clusters of outlier PC scores exist at the high and low ends of the range. Both of these factors are likely modulating the regression coefficient. These results suggest that linear regression may not be a useful method for predicting the likelihood a satellite may experience a conjunction.

We conducted ANOVA twice to identify the extent of variations between groups with low, average, and high PC scores relative to sync time. In the first ANOVA, we examined equal distributions of scores that are below average, average, or above average. In the second ANOVA (Table 3), we specified the low, average, and high ranges based on the distribution of PC scores in the dataset (Figure 5). In both cases, the resulting p-values (

 < 1×10−100) are much lower than 0.05, indicating highly statistically significant differences between groups.

Since there are currently no documented cases of injury caused by space debris returning to Earth, it could be argued that the safety risks of conjunctions may be minimal for the general public [12]. It is unclear if this trend is expected to persist as the number of objects in orbit continues to surge.

Since TLE pairs are only accurate for several weeks, long simulations spanning months or years may not give precise results. This limitation might be overcome by comparing older TLE pairs with newer TLE pairs to evaluate the extent to which the orbits change. If a satellite has a more predictable orbit, it may also have a lower PC.

Our work relies on publicly available data provided by USSPACECOM, which may not include details of objects managed by other countries. The extent to which PC estimates vary between governmental space programs and commercial operators was not immediately clear to us. As such, predictive models trained using this data may reflect a bias towards objects tracked by the United States. Future work for PC estimation should account for variances that may exist between scoring methods and available data.

We demonstrated how since sync time appears to be a strong predictor of PC scores. Future work could use sync time to train different machine learning (ML) models for PC prediction and compare the accuracy with existing methods used to calculate the probability of collision. Additionally, satellites rotating in the same direction should have conjunctions half as frequently as satellites moving in the same direction. Exploring the impact of orbital direction on PC may further improve PC predictions.

Investigating features that are strong indicators of potential collisions could help identify more satellite pairs that are likely to collide, as well as the causes of potential collisions. Our ANOVA test demonstrates how the sync time between satellite orbits is a significant factor in calculating the probability of collision, showing promise as a feature that may inform the creation of more advanced models of PC estimation.

Most objects with a probability of collision greater than 0 are space debris. While the USA is more efficient (orbital 

payloads / orbital debris) than other space programs and commercial entities, it still generates a similar amount of debris. Legislation could be passed to ensure government organizations and commercial operators are more mindful of what is sent into space.

As the number of satellites in orbit continues to rise, accurate predictions of conjunctions will become increasingly vital for securing the safety of humans working in space, as well as those on the ground. Identifying the factors leading to conjunctions is paramount to developing accurate conjunction assessments and mitigating the risks posed by the increasing number of satellites in orbit.

https://www.space-track.org/documentation#/odr

H. G. Lewis and G. Skelton, “Safety considerations for large constellations of satellites,” Journal of Space Safety Engineering, Aug. 2024, [Online; accessed 2024- 10-16], ISSN: 2468-8967. DOI: 10.1016/j.jsse.2024.08.001. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2468896724001150.

NASA Spacecraft Conjunction Assessment and Collision Avoidance Best Practices Handbook, Feb. 2023. [Online]. Available: https://nodis3.gsfc.nasa.gov/ OCE docs/OCE 51.pdf.

Spaceflight Safety Handbook for Satellite Operators, Apr. 2023. [Online]. Available: https://www.space-track.org/documents/SFS Handbook For Operators V1.7.pdf.

D. A. Vallado and P. J. Cefola, “Iac-12-A6.6.11 TWO- LINE ELEMENT SETS – PRACTICE AND USE,”

RP, Plot Satellites’ Real-time Orbits with Python’s Matplotlib — In Plain English, en-GB, Nov. 2023. [Online]. Available: https://plainenglish.io/blog/plot-satellites-real-time-orbits-with-python-s-matplotlib (visited on 12/04/2024).

N2YO.com API, https://www.n2yo.com/api/, [Online; accessed 2024-11-25]. [Online]. Available: https://www.n2yo.com/api/.

Kelvins Collision Avoidance Challenge Data, https://kelvins.esa.int/collision-avoidance-challenge/data/, [Online; accessed 2024-11-25]. [Online]. Available: https://kelvins.esa.int/collision-avoidance-challenge/data/.

Collision Avoidance Challenge, [Online; accessed 2024-
11-25]. [Online]. Available: https://www.kaggle.com/datasets/shadmanrohan/collisionavoidancechallenge. 

NASA Open Data Portal, https://data.nasa.gov/, [Online; accessed 2024-11-25]. [Online]. Available: https://data.nasa.gov/.

PyEphem Home Page, https://rhodesmill.org/pyephem/,
[Online; accessed 2024-11-28]. [Online]. Available:
https://rhodesmill.org/pyephem/.

ARES — Orbital Debris Program Office — Frequently Asked Questions. [Online]. Available: https://orbitaldebris.jsc.nasa.gov/faq/ (visited on 12/01/2024).


Predictive Factors of Satellite Conjunctions

As part of my PhD coursework, I had the opportunity to take the

Embedded Hardware/Software Design (CSCE 5612)

during the Spring 2025 semester. The course offered hands-on practice with microcontrollers and a semester-long project where students design and develop an embedded system integrating sensors and machine learning models. I was fortunate to work with 

 as a lab partner for these projects throughout the semester. I also opted to continue working on my final project after completing the course, which will likely become my doctoral thesis project. I will share more details as soon as I am permitted to do so.  

Our weekly assignments for the course were submitted as posts on individual project websites we each created. The links below route to pages on my project website (a subdomain of this website) that include more details for a few of my favorite projects from this course.  

e collected data for five activities and trained a machine learning (ML) model for on-device activity recognition. The activities were ascending stairs, descending stairs, walking, standing, and sitting.

Finite State Machine and Object Detection

In this lab, we used the OV2640 camera module on a 

 to perform object recognition and control a finite state machine. Our dataset included ~20 images for each of 5 different toy cars (~100 images total). The images feature a variety of angles, background elements, and degrees of motion blur to provide the model a diversity of data for training. After collecting a sufficient number of images, we used 

 model for object detection. Finally, we created a finite state machine (FSM) that is represented as a traffic light, which detects the presence of toy cars and responds accordingly.

In this lab, we used the XIAO ESP32-S3 to create a musical keyboard by generating a sine wave and modulating its characteristics using button presses. The key design detail of the circuit is the use of the binary weighted method to create a digital-to-analog converter (DAC), which essentially uses a series of resistors to change the voltage of each output pin. I was able to connect the device to my guitar rig and experiment with different effects.

Embedded Systems Projects - Spring 2025

Activity recognition

Finite State Machine and Object Detection

DAC Synthesizer

Sonification for Explainability in Large Language Models

Predictive Factors of Satellite Conjunctions