It seems every time a phone is turned on, a software application needs an update. Many of these updates are  integrating artificial intelligence (AI) models of some sort into crucial functions of the application.

Music retrieval apps, for example, employ AI-powered search algorithms. While an update may technically improve the app by decreasing the average error rate of music retrieval, the subtle changes to the retrieval mechanism can cause a subjective performance degradation for the user.

This is known as software regression.

A Georgia Tech School of Electrical and Computer Engineering research team of postdoctoral research Fellow Mohit Prabhushankar, recent Ph.D. graduate Ryan Benkert, and Professor Ghassan AlRegib were the runners-up for the Best Paper Award at the 2024 IEEE International Conference on Big Data in Washington, D.C. Their research focused on minimizing subjective software regression from updates while simultaneously continuing to decrease objective errors.

The paper, “Targeting Negative Flips in Active Learning using Validation Sets,” aimed to train and update AI models using the Negative Flip Rate (NFR) metric.

NFR is a measure used in machine learning to quantify the rate at which a new model makes incorrect predictions on data points that were correctly predicted by a previous model.

The goal for the research is to have low NFR within the models with high algorithm accuracy. They were able to identify certain subsets in the training data that achieved this. The next step is creating a method that can proactively select these subsets.

The paper ultimately proposed to construct a Regression-ordered Subset Estimation (RoSE) using the data to consistently ensure accuracy of the AI model is high while reducing NFR.

Prabhushankar received his Ph.D. in electrical engineering from Georgia Tech in 2021. He is currently a postdoctoral research fellow in the Omni Lab for Intelligent Visual Engineering and Science (OLIVES). He is working in the fields of image processing,
machine learning, active learning, healthcare, and robust and explainable AI.

Benkert received his Ph.D. in electrical engineering from Georgia Tech in 2024. He is currently a deep learning software engineer at NVIDIA in Santa Clara, CA. His research interests are at the intersection of active learning, uncertainty estimation, and performance consistency in neural network learning. Prior to Georgia Tech, he received his B.Sc and M.Sc from the RWTH Aachen University in Germany.

Professor AlRegib is currently the John and Marilu McCarty Chair Professor ECE and directs OLIVES. He and his group work on robust and interpretable machine learning algorithms, uncertainty and trust, and human in the loop algorithms. The group has demonstrated their work on a wide range of applications such as Autonomous Systems, Medical Imaging, and Subsurface Imaging. The group is interested in advancing the fundamentals as well as the deployment of such systems in real-world scenarios. He has been issued several U.S. patents and invention disclosures. He is a Fellow of the IEEE.