Abstract: Most practically relevant low rank (LR) matrix learning problems - LR Column-wise Sensing, LR matrix completion, Robust PCA and LRMC are instances of “partly decoupled” optimization problems.
These find important applications in parameter efficient fine-tuning of LLMs, multi-task and few-shot learning, federated sketching, and MRI. In many of these applications, the data is either available in a federated fashion, or it is available centrally, but we need to distribute it to speed up computing. In both settings, there is a need for communication-efficient solutions. Modern distributed and federated learning systems are vulnerable to various kinds of adversarial attacks. Byzantine attacks are one of the most difficult attacks to deal with. Alternating GD and minimization (AltGDmin) is a novel solution
framework, introduced in our recent work. It provides a provable faster and communication-efficient alternative to both Alternating Minimization (AltMin) and Factored GD for “partly-decoupled’’ optimization problems, while needing the same sample complexity and outlier tolerance. “Partlydecoupled’’ refers to problems in which the set of optimization variables can be split into two, or more,
subsets/blocks such that the optimization over one block, keeping the other fixed, is decoupled for all but one block (slow block). After careful initialization, it alternates between a GD step for the slow block and full minimization over the decoupled (and hence fast) blocks. Since it only involves only one GD step per iteration, an efficient Byzantine-resilient modification can be easily designed by directly adapting existing Byzantine GD literature. A third advantage of AltGDmin is that it needs very few parameters
making it easily generalizable for multiple MRI applications. We will show real-data experimental results on the advantage (speed and generality) of AltGDmin-based methods over the existing state-of-the-art within dynamic MRI. I will end the talk with a brief description of CyMath, which is a low-cost ISU STEM faculty and student driven K-12 Math Tutoring and Enrichment program that I have been leading since 2020. It is a a scalable and low-cost approach to developing two generations of the STEM workforce
simultaneously: K–12 students gaining foundational math skills, and university STEM students gaining teaching, communication, leadership, and mentoring experience. Given NSF’s new 30M investment in K-12 math excellence, I am looking to find partners to try to replicate this model at other large universities.
Bio: Namrata Vaswani is a Professor of Electrical and Computer Engineering, and the Anderlik Professor of Engineering at Iowa State University. She also holds a courtesy professorship in the department of Mathematics. She received a Ph.D. in 2004 from the University of Maryland, College Park and a B.Tech. from Indian Institute of Technology (IIT-Delhi) in 1999. Her research is in statistical ML and signal processing, and in imaging (MRI and video analytics). Vaswani is also the director of the CyMath K-12 math tutoring and support program at Iowa State. She is a recipient of the IEEE Signal Processing Society (SPS) Best Paper Award (2014), the University of Maryland ECE Distinguished Alumni Award (2019), and the Iowa State Mid-Career Achievement in Research Award (2019). Vaswani is an AAAS Fellow (class of 2023) and an IEEE Fellow (class of 2019).
Faculty Host: Selin Aviyente
Zoom Meeting ID: 992 8348 7443
Passcode: eceseminar