Humans of SymSys: Allen Nie
/
Allen Nie (MS '17) works on NLP and AI research with Professor James Zou in the Biomedical Data Science department at the Stanford School of Medicine.
What drew you to the SymSys master's program? Why did you pick SymSys as opposed to other programs or related fields?
I've always been very interested in interdisciplinary fields. SymSys (unlike other cognitive science programs) has a strong focus on computation. It studies artificial intelligence and its surrounding fields without the need to focus on a particular approach (such as a cognitive science approach, symbolic/logic approach, statistical approach, etc.). It allows the students to understand the merits of all approaches and delve into the one they can believe in. In the age of artificial intelligence, I would like to see Symbolic Systems grow and become an actual department or institution with full-time researchers studying the mechanisms and fundamentals of intelligent systems and their behaviors.
What’s your favorite SymSys-related class that you’ve taken?
LING 130A with Chris Potts and EE 376A with David Tse. LING 130A focuses on understanding language using tools of logic and the later one focused on probabilistic systems and information theory.
Both classes teach a tool that allows students to understand aspects of the world. LING 130A teaches logic and EE 376A teaches probabilistic operators such as entropy and mutual information. Without getting into a philosophical abyss that is epistemology, the human scientific exploration focuses on featurizing the world, using models (sometimes our own brain) to find meaningful signals, discretizing the finding into digestible patterns or rules or “discoveries”, and call it knowledge. In order to study a field of interest, an analytical tool must be acquired. Both classes provide tools that allow us to model our hypotheses of the world and verify them.
What is one piece of advice you'd like to offer to younger students?
Find a field that you'd like to study. It can be music, language, cognition, nervous systems, molecules, networks of humans, and think about the best tool you can find to study them.
What's something cool you recently worked on?
Universal sentence encoding, building interpretable NLP models, and ML safety where the model has the chance to abstain/reject when it is not confident about its decision.
What underlying questions and issues do you hope to tackle/learn more about?
I've been working on universal sentence representation and I'm applying these representations to low-resource domains (a problem with very limited training data) such as analyzing clinical text. My other area of interest is to make NLP models more interpretable. In the past decades, we haven't had models that can truly match human-level performance on language tasks. But now we do. Is there hidden knowledge that we can learn from these high-performing predictive models? This is what statisticians have been doing in the past -- using a predictive model to conduct feature selection, and a clinician will jump out and spit out “definitive proof" about wine drinking and living long age. With the seriousness of correlation and causation aside, which may very well be part of a larger philosophical debate, are we able to discover valuable patterns on language from a high-performing predictive model with minimal inductive biases? Would those patterns contradict current linguistic theories? Sam Bowman (a Linguistics professor at NYU) has published some amazing work on this topic last summer.
As a diverse program with a lot of flexibility, many students struggle to find continuity across their coursework. (How) do you address this?
I don't think SymSys program provides continuity. Instead, each student needs to define the continuity from their own interests. If you are interested in language, you can follow the path of studying language and acquiring various tools, but you need to find the common threads among the classes you take. A professor in a statistics class won't tell you that Andrey Markov was thinking about fitting the vowels and consonants in Pushkin's poetry after he developed Markov Chain, or Charles Spearman, who invented Spearman correlation, is a highly revered Psychologist and regarded his own statistics work as secondary to his quests on Psychology (human intelligence). In order to find continuity in your work, you should first find out your personal interest and construct your curriculum and research around it.
If you could go back in time and be a Stanford student again, what would you have done differently and why?
There have been classes that I'm very interested in but was not able to take, such as MKTG 355: Designing for Happiness or CS 181: Computers, Ethics, and Public Policy — especially considering the Uber incident, who is to blame and to punish when a software that seemingly possesses agency kills someone? And since machine learning is marching into healthcare, is a line such as “our system on average only makes mistakes and kills 1000 people (less than 5000 people in a human-based setting) in your hospital each year” defendable in the public scrutiny? I imagine I might have gotten an answer if I took CS 181 :)