CSE Colloquia: Lipstick on a Pig: Using Language Models as Few-Shot Learners

Abstract: Language models provide representations that can be adapted to many NLP tasks with minimal effort. In particular, language modeling has provided exceptional few-shot natural language understanding and reasoning performance by framing these tasks as prompts. These capabilities improve significantly with larger models and datasets, making the direct use of language models the dominant approach for NLP applications. However, the goals of language modeling are not precisely the same as what we need from few-shot learners, and it is vital to understand this gap.

In this talk, we will discuss the basics of neural networks, the text corpus, and the training pipeline that enables language models to behave as these general-purpose AI agents. However, we will show how this very paradigm of language modeling also introduces fundamental limitations in this technology. We will characterize these vulnerabilities in language models and discuss how they affect end-use applications. The results presented suggest language modeling may not be sufficient to learn robust reasoners and that we need to take the pretraining data into account when interpreting few-shot evaluation results.

Bio: Dr. Sameer Singh is an Associate Professor of Computer Science at the University of California, Irvine (UCI) and a Cofounder of Spiffy AI. He is working primarily on the robustness and interpretability of machine learning algorithms and models that reason with text and structure for natural language processing. He has been named the Kavli Fellow by the National Academy of Sciences, received the NSF CAREER award, UCI Distinguished Early Career Faculty award, the Hellman Faculty Fellowship, and was selected as a DARPA Riser. His group has received funding from Allen Institute for AI, Amazon, NSF, DARPA, Adobe Research, Hasso Plattner Institute, NEC, Base 11, and FICO. Sameer has published extensively at machine learning and natural language processing venues and received numerous paper awards, including at KDD 2016, ACL 2018, EMNLP 2019, AKBC 2020, ACL 2020, and NAACL 2022. (https://sameersingh.org/)


Share this event

facebook linked in twitter email

Media Contact: Timothy Zhu



The School of Electrical Engineering and Computer Science was created in the spring of 2015 to allow greater access to courses offered by both departments for undergraduate and graduate students in exciting collaborative research fields.

We offer B.S. degrees in electrical engineering, computer science, computer engineering and data science and graduate degrees (master's degrees and Ph.D.'s) in electrical engineering and computer science and engineering. EECS focuses on the convergence of technologies and disciplines to meet today’s industrial demands.

School of Electrical Engineering and Computer Science

The Pennsylvania State University

207 Electrical Engineering West

University Park, PA 16802


Department of Computer Science and Engineering


Department of Electrical Engineering