Source: Willow

Course webpage for the NYU Spring 2021 Course: Introduction to Computer Vision. This course is aims to cover a broad topics in computer vision, and is not primarily a deep learning course. We will covert topics in traditional computer vision such as camera geometry, image formation, segmentation, object recognition, classification, and detection (see Syllabus).

Special Topics in Data Science, DS-GA 3001.004/.005

Logistics

• DS-GA 3001.004 (Lecture)
  Wednesdays 1pm-2:40pm
  Location: Global Center for Academic & Spiritual Life, 238 Thompson Street (GCASL 461)
  Zoom Link: please log in to NYU classes to get a meeting ID
  Final Presentations: Wednesday, May 12 2PM-3:50PM on this Zoom link.
  
  Please join the lecture in person or on Zoom. Slides will be available after the class. (see Schedule).

• DS-GA 3001.005 (Lab)
  Thursdays from 12pm-12:50pm
  Location: Silver Center for Arts & Science, 100 Washington Sq East (SILV_101A)
  Zoom Link: please log in to NYU classes to get a meeting ID
  
  Labs will be used to cover additional materials or to work through practical exercises with the TA.

• Office Hours:
  Jean Ponce: Tuesday, 2-3pm (please email for appointment)
  Elena Sizikova: Monday, 11-12pm (please email for appointment)

• Piazza Link: https://piazza.com/nyu/spring2021/dsga3001004
  Please try first to post any questions about course logistics and material, HWs and final project on Piazza. We also highly encourage you to help each other out (but please do not reveal answers). For additional questions, please email course staff.

Instructors

Jean Ponce (jean.ponce@inria.fr)
Elena Sizikova (es5223@nyu.edu)

TAs

Aryaa Singh (as13538@nyu.edu)
Manjusha Mishra (mam1974@nyu.edu)

Grading

Four programming assignments (60% of the grade) + final project (40% of the grade). Assignments should be submitted using the NYU class site.

• Excercise 1 on camera calibration (zip file). Due on Feb. 21.
• Excercise 2 on Canny edge detector (zip file). Skeleton code link. Due on Mar. 14.
• Excercise 3 on mean shift (zip file). Skeleton code link. Due on Apr. 4.
• Excercise 4 on neural networks (ipynb notebook). Due on May 2.
• Final project: list of suggested papers is available here. Send an email to Elena to validate a project by Mar. 7 at the latest.

Syllabus

• Introduction
• Camera geometry and calibration
• Filtering and feature detection
• Radiometry and color
• Texture and image segmentation
• Stereopsis
• Structure from motion and 3D models from images
• Object recognition - historical perspective
• CNNs for object classification and detection
• 3D CNNs, Applications in Medical Imaging
• Weakly-supervised and unsupervised approaches to image and video interpretation

References:

We do not require purchase of any textbooks and the course will be self-contained. You may wish to consult the resources below for additional material formalization.

• D.A. Forsyth and J. Ponce, “Computer Vision: A Modern Approach”, second edition, Pearson, 2011. (Link)
• R. Szeliski, “Computer Vision: Algorithms and Applications”. (PDF)
• R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press, 2004. (Link)

Schedule:

Note: lecture slides will be posted after each lecture.

Date	Lecture	Topic	Link
01/28	Lab: Course Intro	Introduction, Logistics	Intro [PPT] [PDF] Logistics [Key] [PDF]
02/03	Lecture 1	Examples of vision tasks, camera geometry and calibration	Lecture [PPT] [PDF]
02/04	Lab	A detour of sensing country, intrinsic and extrinsic parameters	Lecture [PPT] [PDF]
02/10	Lecture 2	Linear calibration, analytic photogrammetry, filtering	Lecture Part 1 [PPT] [PDF] Lecture Part 2 [Key] [PDF]
02/11	Lab	Image Stitching Exercise	Notebook Link
02/17	Lecture 3	Edge detection, keypoints and features, RANSAC, Hough transform	Lecture [Key] [PDF]
02/21	Exercise 1 DUE
02/24	Lecture 4	Texture, Segmentation	Lecture [Key] [PDF]
02/25	Lab	Edge Detection using Sobel Operator	Notebook Link
03/03	Lecture 5	Radiometry and Color, Part 1	Lecture [PPT] [PDF]
03/04	Lab	Radiometry and Color, Part 2	Lecture [PPT] [PDF]
03/10	Lecture 6	Color	Lecture [PPT] [PDF]
03/11	Lab	Canny Edge Detection Skeleton Code	Notebook Link
03/14	Exercise 2 DUE
03/17	Lecture 7	Stereopsis, Epipolar Geometry, Essential and Fundamental Matrices	Lecture [PPT] [PDF]
03/18	Lab	Fundamental Matrix Esimation	Notebook Link
03/24	Lecture 8	Eight-point Algorithm, Correlation-based stereo, more sophisticated methods	Lecture [Key] [PDF]
03/25	Lab	Eight-point Algorithm	Notebook Link
03/31	Lecture 9	Structure from Motion	Lecture [PPT] [PDF]
04/01	Lab
04/04	Exercise 3 DUE
04/07	Lecture 10	Affine and Projective SfM	Lecture [PPT] [PDF]
04/08	Lab
04/14	Lecture 11	SfM, Laser Scanning, Space Carving, Multiview Stereo	P1[PPT] P1[PDF] P2[Key] P2[PDF]
04/15	Lab	ICP Implementation	Notebook Link
04/21	Lecture 12	Visual Recognition, Intro to NN	P1[PPT] P1[PDF] P2[Key] P2[PDF]
04/22	Lab	VAE Implementation	Notebook Link
04/28	Lecture 13	CNN Recognition and Detection	Lecture [Key] [PDF]
04/29	Lab
05/02	Exercise 4 DUE
05/05	Lecture 14
05/06	Lab

Acknowledgements

Much of the material for this course relies on the Computer Vision course given at ENS Paris by Mathieu Aubry, Karteek Alahari, Ivan Laptev, and Josef Sivic. Many of the slides are taken from James Hays, Svetlana Lazebnik, and Derek Hoeim. Website was originally designed by Matthew Trager.