Vassar CMPU 366: Natural Language Processing

Monday	1:30–2:45 p.m.
Wednesday	1:30–2:45 p.m.

New England 105

Professor Gordon

↓ Current

1	Introduction
	Sep. 1
Sep. 3	Computers and language	Core reading: Syllabus Further exploration: J&M (2e) 1: Introduction Weizenbaum, 1966: ELIZA – a computer program for the study of natural language communication between man and machine
2	Text and words
	Sep. 8	Processing text	Core reading: J&M 2: Regular Expressions, Tokenization, Edit Distance Further exploration: Zentgraf, 2015: What every programmer absolutely, positively needs to know about encodings and character sets to work with text Erad, 2017: How the appetite for emojis complicates the effort to standardize the world’s alphabets Coldeway, 2019: How Japan’s new imperial era broke the internet in a very tiny way
	Sep. 10	Tokens and normalization Assignment 1
3	Modeling language
	Sep. 15	Language models	Core reading: J&M 3: N-gram Language Models (through §3.6) Further exploration: Michel et al., 2011: Quantitative analysis of culture using millions of digitized books Jurgens et al., 2017: Incorporating dialectical variability for socially equitable language identification
	Sep. 17	Model evaluation and smoothing Assignment 2
4	Machine learning
	Sep. 22	Logistic regression	Core reading: J&M 4: Logistic Regression and Text Classification
	Sep. 24	Gradient descent Assignment 3
5	Geometry and meaning
	Sep. 29	Vector semantics	Core reading: J&M 5: Embeddings Further exploration: Goldberg & Levy, 2014: Word2vec explained Bolukbasi et al., 2016: Man is to computer programmer as woman is to homemaker? debiasing word embeddings Gonen & Goldberg, 2019: Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them Hamilton et al., 2018: Diachronic word embeddings reveal statistical laws of semantic change
	Oct. 1	Word2Vec Assignment 4
6	Deep learning
	Oct. 6	Neural networks	Core reading: J&M 6: Neural Networks
	Oct. 8	Deep learning with PyTorch Assignment 5
7	Midterm
	Oct. 13	Case studies: Ethics & bias in NLP Vote for special topics	Core reading: Blodgett et al., 2020: Language (Technology) is Power: A Critical Survey of ‘Bias’ in NLP
	Oct. 15	Midterm	This is Simchat Torah; make-up exams will be on Thursday or Friday.
8	October break
	Oct. 20	🍁
	Oct. 21	🍂
9	Large language models
	Oct. 27	Large language models, part 1 Sign up for special topic presentations	Core reading: J&M 7: Large Language Models
	Oct. 29	Large language models, part 2
10	Transformers
	Nov. 3	Attention Project pitch	Core reading: J&M 8: Transformers
	Nov. 5	Transformers
11	Post-training
	Nov. 10	Transfer learning, prompt engineering, and alignment	Core reading: J&M 9: Post-training: Instruction Tuning, Alignment, and Test-Time Compute
	Nov. 12	Project work: Topics and getting started
12	Special topics and project work
	Nov. 17	Special topics
	Nov. 19	Project work: Peer review Draft of related work due in class
13	Special topics and project work
	Nov. 24	Special topics
	Nov. 26	Project work
14	Special topics and project work
	Dec. 1	Special topics
	Dec. 3	Project presentations
15	Natural language processing
	Dec. 8	Natural language processing ECS
		Project due December 12; late deadline December 14.

Rosencrantz: What are you playing at?
Guildenstern: Words, words. They’re all we have to go on.

Tom Stoppard, Rosencrantz and Guildenstern are Dead, 1966

1

Introduction

2

Text and words

3

Modeling language

4

Machine learning

5

Geometry and meaning

6

Deep learning

7

Midterm

8

October break

9

Large language models

10

Transformers

11

Post-training

12

Special topics and project work

13

Special topics and project work

14

Special topics and project work

15

Natural language processing