Vassar CMPU 366: Natural Language Processing

Tuesday	1:30–2:45 p.m.
Thursday	1:30–2:45 p.m.

Sanders Classroom 006

Professor Gordon

↓ Current

1	Introduction
	Sep. 1	Computers and language	Core reading: Syllabus J&M 1: Introduction Further exploration: Weizenbaum, 1966: ELIZA – a computer program for the study of natural language communication between man and machine Zentgraf, 2015: What every programmer absolutely, positively needs to know about encodings and character sets to work with text Erad, 2017: How the appetite for emojis complicates the effort to standardize the world’s alphabets Coldeway, 2019: How Japan’s new imperial era broke the internet in a very tiny way
	Sep. 3	Text as data? Assignment 0
2	Words
	Sep. 8	Searching text	Core reading: J&M 2: Words and Tokens
	Sep. 10	Tokens and normalization Assignment 1
3	Modeling language
	Sep. 15	Language models	Core reading: J&M 3: N-gram Language Models (through §3.6) Further exploration: Michel et al., 2011: Quantitative analysis of culture using millions of digitized books Jurgens et al., 2017: Incorporating dialectical variability for socially equitable language identification
	Sep. 17	Model evaluation and smoothing Assignment 2
4	Machine learning
	Sep. 22	Logistic regression	Core reading: J&M 4: Logistic Regression and Text Classification
	Sep. 24	Gradient descent Assignment 3
5	Geometry and meaning
	Sep. 29	Vector semantics	Core reading: J&M 5: Embeddings Further exploration: Goldberg & Levy, 2014: Word2vec explained Bolukbasi et al., 2016: Man is to computer programmer as woman is to homemaker? debiasing word embeddings Gonen & Goldberg, 2019: Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them Hamilton et al., 2018: Diachronic word embeddings reveal statistical laws of semantic change
	Oct. 1	Word2Vec Assignment 4
6	Deep learning
	Oct. 6	Neural networks	Core reading: J&M 6: Neural Networks
	Oct. 8	Deep learning with PyTorch Assignment 5
7	Midterm
	Oct. 13	Ethics & bias in NLP	Core reading: Blodgett et al., 2020: Language (Technology) is Power: A Critical Survey of ‘Bias’ in NLP
	Oct. 15	Midterm Vote for special topics
8	October break
	Oct. 20	🍁
	Oct. 22	🍂
9	Large language models
	Oct. 27	Large language models, part 1	Core reading: J&M 7: Transformers and Pretraining
	Oct. 29	Large language models, part 2 Sign up for special topic presentations
10	Transformers
	Nov. 3	Attention and transformers	Core reading: J&M 8: Post-training
	Nov. 5	Transformers and LLMs Project pitch
11	Special topics and project work
	Nov. 10	Special topics
	Nov. 12	Project work: Topics and getting started Full proposal Draft of related work
12	Special topics and project work
	Nov. 17	Special topics
	Nov. 19	Project work: Peer review Draft of related work due in class
13	Special topics and project work
	Nov. 24	Special topics
	Nov. 26	🥧
14	Special topics and project work
	Dec. 1	Special topics ECS
	Dec. 3	Project presentations
15	Project presentations
	Dec. 8	Project presentations

Rosencrantz: What are you playing at?
Guildenstern: Words, words. They’re all we have to go on.

Tom Stoppard, Rosencrantz and Guildenstern are Dead, 1966

1

Introduction

2

Words

3

Modeling language

4

Machine learning

5

Geometry and meaning

6

Deep learning

7

Midterm

8

October break

9

Large language models

10

Transformers

11

Special topics and project work

12

Special topics and project work

13

Special topics and project work

14

Special topics and project work

15

Project presentations