In this talk, we consider two fundamental problems in signal processing and machine learning: (convolutional) dictionary learning and deep network training. For both problems, we provide the first global nonconvex landscape analysis of the learned representations, which will in turn provide new guiding principles for better model/architecture design, optimization, robustness, and in both supervised and unsupervised scenarios. More specifically, the first part of the talk will focus on (convolutional) dictionary learning (aka shallow representation learning) in the unsupervised setting, we show that a nonconvex  L_4 loss over the sphere has no spurious local minimizers. This further inspires us to design efficient optimization methods for convolutional dictionary learning, with applications in imaging sciences. Second, we study the last-layer representation in deep learning, where recent seminal work by Donoho et al. showed a prevalence empirical phenomenon during the terminal phase of network training - neural collapse. By studying the optimization landscape of the training loss under an unconstrained feature model, we provide the theoretical justification for this phenomenon, which could have broad implications for network training, design, and beyond.

Problems of algorithmic bias are often framed in terms of lack of representative data or formal fairness optimization constraints to be applied to automated decision-making systems. However, these discussions sidestep deeper issues with data used in AI, including problematic categorizations and the extractive logics of crowdwork and data mining. This talk will examine two interventions: first by reframing of data as a form of infrastructure, and as such, implicating politics and power in the construction of datasets; and secondly discussing the development of a research program around the genealogy of datasets used in machine learning and AI systems. These genealogies should be attentive to the constellation of organizations and stakeholders involved in their creation, the intent, values, and assumptions of their authors and curators, and the adoption of datasets by subsequent researchers.

Predicting life outcomes is a challenging task even for advanced machine learning (ML) algorithms. At the same time, accurately predicting these outcomes has important implications in providing targeted assistance and in improving policy making. Recent studies based on Fragile Families and Child Wellbeing Study dataset have shown that…

Deep learning has disrupted nearly every major field of study from computer vision to genomics. The unparalleled success of these models has, in many cases, been fueled by an explosion of data. Millions of labeled images, thousands of annotated ICU admissions, and hundreds of hours of transcribed speech are common standards in the literature. Clinical neuroscience is a notable holdout to this trend. It is a field of unavoidably small datasets, massive patient variability, and complex (largely unknown) phenomena. My lab tackles these challenges across a spectrum of projects, from answering foundational neuroscientific questions to translational applications of neuroimaging data to exploratory directions for probing neural circuitry. One of our key strategies in this data-starved regime has been to blend the structure and interpretability of classical methods with the representational power of deep learning.

Measuring timely high-resolution socioeconomic outcomes is critical for policy-making and evaluation, but hard to reliably obtain. With the help of machine learning and cheaply available data such as social media and nightlight, it is now possible to predict such indices in fine granularity. This paper demonstrates an adaptive way to measure the time trend and spatial distribution of housing activeness with the help of multiple easily-accessible datasets.

Meta-learning or learning to learn has been shown to be a powerful tool for fast learning over unseen tasks by efficiently extracting the knowledge from a range of observed tasks. Such empirical success thus highly motivates theoretical understanding of the performance guarantee of meta-learning, which will serve to guide the better design of meta-learning and further expand its applicability. In this talk, I will present our recent studies of meta-learning based on optimization theory. I will focus on a popular meta-learning approach, the model-agnostic meta-learning (MAML) type of algorithms, which have been widely used in practice due to their simplicity and effectiveness. I will first present the convergence guarantee and the computational complexity that we establish for the vanilla MAML algorithm. I will then talk about our result on a more scalable variant of MAML, called the almost no inner loop (ANIL) algorithm. We characterize the performance improvement of ANIL over MAML as well as the impact of the loss function landscape on the overall computational complexity. I will finally present the experimental validations of our theoretical findings and discuss a few future directions on the topic.

While recent advances in natural language processing (NLP) have greatly enhanced our ability to analyze online text, distilling broad social-oriented research questions into tasks concrete enough for NLP models remains challenging. In this work, we develop state-of-the-art NLP models grounded in frameworks from social theory in order to analyze two social media movements: online media coverage of the #MeToo movement in 2017-2018 and tweets about #BlackLivesMatter protests in 2020.

Google Quantum Artificial Intelligence Lab

Website Jarrod McClean Website

Thursday, Sep. 30, 2021
4:00pm

Zoom Meeting

Host - Haw Yang

More information and abstract forthcoming.

The Princeton Dialogues in AI and Work is a research agenda investigating what algorithmic and predictive data-driven tools mean to stakeholders across society. Building on prior work in the Dialogues in AI and Ethics case study series, the current phase of research takes an empirical, sociotechnical focus on how the different communities will interact with, be represented by, and be implicated by, algorithmic technologies in different ways.

Whether operating in a manufacturing plant or assisting within the home, many robotic tasks require safe and controlled interaction with a complex and changing world. However, state-of-the-art approaches to both learning and control are most effective when this interaction either occurs in highly structured settings or at slow speeds unsuitable for real-world deployment. In this talk, I will focus broadly on our most recent efforts to model and control complex, multi-contact motions. Even given a known model, current approaches to control typically only function if the contact mode can be determined or planned a priori. Our recent work has focused on real-time feedback policies using tactile sensing and ADMM-style algorithms to adaptively react to making and breaking contact or stick-slip transitions. For dynamic impacts, like robotic jumping, tactile sensing may not be practical; instead, I will show how impact-invariant strategies can both be robust to uncertainty during collisions while preserving control authority.