Vassar CMPU 366: Natural Language Processing

Monday	1:30–2:45 p.m.
Wednesday	1:30–2:45 p.m.

New England 105

Professor Gordon

↓ Current

1	Introduction
	Sep. 1
Sep. 3	Computers and language	Core reading: Syllabus Further exploration: J&M (2e) 1: Introduction Weizenbaum, 1966: ELIZA – a computer program for the study of natural language communication between man and machine
2	Text and words
	Sep. 8	Processing text	Core reading: J&M 2: Regular Expressions, Tokenization, Edit Distance Further exploration: Zentgraf, 2015: What every programmer absolutely, positively needs to know about encodings and character sets to work with text Erad, 2017: How the appetite for emojis complicates the effort to standardize the world’s alphabets Coldeway, 2019: How Japan’s new imperial era broke the internet in a very tiny way
	Sep. 10	Tokens and normalization Assignment 1
3	Modeling language
	Sep. 15	Language models	Core reading: J&M 3: N-gram Language Models (through §3.6) Further exploration: Michel et al., 2011: Quantitative analysis of culture using millions of digitized books Jurgens et al., 2017: Incorporating dialectical variability for socially equitable language identification
	Sep. 17	Model evaluation and smoothing (Notebook) Assignment 2
4	Machine learning
	Sep. 22	Logistic regression	Core reading: J&M 4: Logistic Regression and Text Classification
	Sep. 24	Gradient descent Assignment 3
5	Geometry and meaning
	Sep. 29	Vector semantics	Core reading: J&M 5: Embeddings Further exploration: Goldberg & Levy, 2014: Word2vec explained Bolukbasi et al., 2016: Man is to computer programmer as woman is to homemaker? debiasing word embeddings Gonen & Goldberg, 2019: Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them Hamilton et al., 2018: Diachronic word embeddings reveal statistical laws of semantic change
	Oct. 1	Word2Vec Assignment 4
6	Deep learning
	Oct. 6	Neural networks	Core reading: J&M 6: Neural Networks
	Oct. 8	Deep learning with PyTorch Assignment 5
7	Midterm
	Oct. 13	Ethics & bias in NLP	Core reading: Blodgett et al., 2020: Language (Technology) is Power: A Critical Survey of ‘Bias’ in NLP
	Oct. 15	Midterm Vote for special topics	This is Simchat Torah; make-up exams will be on Thursday or Friday.
8	October break
	Oct. 20	🍁
	Oct. 21	🍂
9	Large language models
	Oct. 27	Large language models, part 1	Core reading: J&M 7: Large Language Models
	Oct. 29	Large language models, part 2 (Notebook) Sign up for special topic presentations
10	Transformers
	Nov. 3	Attention and transformers	Core reading: J&M 8: Transformers
	Nov. 5	Transformers and LLMs Project pitch (due Nov. 11)
11	Special topics and project work
	Nov. 10	Special topics: Machine translation Computational pragmatics	Machine translation: WSJ, 2024: How Google Translate Turns 134 Languages Into Math (video) Xu et al., 2019: A Study of Multilingual Neural Machine Translation Briakou et al., 2023: Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM’s Translation Capability Computational pragmatics: Crash Course Linguistics: Pragmatics (video) Jurafsky, 2003: Pragmatics and Computational Linguistics Hu et al., 2023: A fine-grained comparison of pragmatic language understanding in humans and language models
	Nov. 12	Project work: Topics and getting started Full proposal (due Nov. 16) Draft of related work (due Nov. 19)
12	Special topics and project work
	Nov. 17	Special topics: Evaluation in NLP Automatic summarization	Evaluation in NLP: Bear Brown, 2024: Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMs, p. 6–13 and 17–23 Belinkov & Glass, 2019: Analysis Methods in Neural Language Processing: A Survey, p. 1–11 Automatic summarization: Murel & Kavlakoglu: Text summarization defined Wolfe, 2024: Summarization and the Evolution of LLMs
	Nov. 19	Project work: Peer review Draft of related work due in class
13	Special topics and project work
	Nov. 24	Special topics: Interpretability and explainability Mitigating bias	Interpretability and explainability: A Data Odyssey: Interpretable vs Explainable Machine Learning Nostalgebraist, 2020: Interpreting GPT: the logit lens Mitigating bias: Huang et al., 2025: Bias Testing and Mitigation in LLM-based Code Generation Borah & Mihalcea, 2024: Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
	Nov. 26	Project work
14	Special topics and project work
	Dec. 1	Special topics: Topic modeling Natural language processing ECS	Topic modeling: Liu et al., 2016: An overview of topic modeling and its current applications in bioinformatics Sear et al., 2022: Dynamic Topic Modeling Reveals Variations in Online Hate Narratives
	Dec. 3	Project presentations M. Medina, J. Zuckerman M. Pajouh I. Borgsted, S. Jaman, E. Kogan A. Wang, R. Yao J. Chidiac, C. Peterson A. Aziz, R. Aziz
15	Project presentations
	Dec. 8	Project presentations D. Allen, W. Miller, N. Phelan L. Chen, M. K. Hu, C. Núñez S. Llanes, A. Maza D. Arreola J. Bergman, W. Metcalf D. Beaucamp, S. Shelly
		Project due December 12; late deadline December 14.

Rosencrantz: What are you playing at?
Guildenstern: Words, words. They’re all we have to go on.

Tom Stoppard, Rosencrantz and Guildenstern are Dead, 1966

1

Introduction

2

Text and words

3

Modeling language

4

Machine learning

5

Geometry and meaning

6

Deep learning

7

Midterm

8

October break

9

Large language models

10

Transformers

11

Special topics and project work

12

Special topics and project work

13

Special topics and project work

14

Special topics and project work

15

Project presentations