Intro to Computer Vision

NYU, Spring 2022


Source: Willow

Course webpage for the NYU Spring 2022 Course Special Topics in Data Science, DS-GA 3001.003/.004 (Introduction to Computer Vision). This course is aims to cover a broad topics in computer vision, and is not primarily a deep learning course. We will covert topics in traditional computer vision such as camera geometry, image formation, segmentation, object recognition, classification, and detection (see Syllabus).

Logistics

  • DS-GA 3001.003 (Lecture)
    Thursdays 4:55pm-6:35pm
    Location: Global Center for Academic & Spiritual Life, 238 Thompson Street (GCASL 361)

    The class will be in-person or via Zoom (see Brightspace for link). Slides will be available after the class on this webpage. (see Schedule).

  • DS-GA 3001.004 (Lab)
    Tuesdays 5:55pm-6:45pm
    Location: Global Center for Academic & Spiritual Life, 238 Thompson Street (GCASL 361)

    The labs will be in-person or via Zoom (see Brightspace for link). Labs will be used to cover additional materials or to work through practical exercises with the TA.

  • Office Hours:
    Office hours with each instructor will be available by appointment.

  • Campuswire Link: Link
    Please try first to post any questions about course logistics and material, HWs and final project on Campuswire. We also highly encourage you to help each other out (but please do not reveal answers). For additional questions, please email course staff.

Instructors

Jean Ponce (jean.ponce@inria.fr)
Alberto Bietti (alberto.bietti@nyu.edu)
Elena Sizikova (es5223@nyu.edu)

TAs and Graders

Irmak Guzey (ig2283@nyu.edu)
Aakash Bhattacharya (ab9541@nyu.edu)

Grading

Four programming assignments (50% of the grade) + final project (40% of the grade) + class participation and attendance (5%) + lab participation and attendance (5%). Assignments should be submitted using the Brightspace site.

  • Excercise 1 on camera calibration. Link Due on Feb. 16.
  • Excercise 2 on Canny edge detector. Link Due on Mar. 9.
  • Excercise 3 on mean shift. Link Due on Apr. 6.
  • Excercise 4 on neural networks. Link Due on May 4.
  • Final project: here is a list of suggested papers for the final project. Submit project abstract by March 11 here. Final presentations will be held during the last lecture (May 5) and the project report will be due May 5, 11.59PM EST.

Participation and Attendance

You are expected to attend and participate in classes and labs in person or via Zoom (see NYU Brightspace for link). Class attendance will count for 5% of your grade and lab attendance will count for 5% of your grade.

Syllabus

  • Introduction
  • Camera geometry and calibration
  • Filtering and feature detection
  • Radiometry and color
  • Texture and image segmentation
  • Stereopsis
  • Structure from motion and 3D models from images
  • Object recognition - historical perspective
  • CNNs for object classification and detection
  • 3D CNNs, Applications in Medical Imaging
  • Weakly-supervised and unsupervised approaches to image and video interpretation

References:

We do not require purchase of any textbooks and the course will be self-contained. You may wish to consult the resources below for additional material formalization.

  • D.A. Forsyth and J. Ponce, “Computer Vision: A Modern Approach”, second edition, Pearson, 2011. (Link)
  • R. Szeliski, “Computer Vision: Algorithms and Applications”. (PDF)
  • R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press, 2004. (Link)

Schedule:

Note: lecture slides will be posted after each lecture.

Date Lecture Topic Link
01/25 Lab: Course Intro Logistics Slides (Keynote, PDF)
01/27 Lecture 1 Introduction to Computer Vision Slides (PPTX, PDF)
02/01 Lab Calibration Exercise Slides (PPTX, PDF) Notebook (Jupyter)
02/03 Lecture 2 Camera geometry and calibration Slides (PPTX, PDF)
02/08 Lab 2 Homework 1 review Slides (PPTX, PDF) Notebook (Jupyter)
02/10 Lecture 3 Camera geometry and calibration Slides (PPTX, PDF)
02/15 Lab HW1 Q&A  
02/16 Exercise 1 DUE HW on Camera Calibration Link
02/17 Lecture 4 Image Filtering Slides (Keynote, PDF)
03/22 Lab 3 Image Filtering Notebook (Jupyter)
02/24 Lecture 5 Edge and Feature Detection Slides (Keynote, PDF)
03/01 Lab 4 Non-Max Suppression and RANSAC Notebook (Jupyter)
03/03 Lecture 6 Radiometry Slides (PPTX, PDF)
03/08 Lab HW2 Q&A  
03/09 Exercise 2 DUE HW on Canny Edge Detector Link
03/10 Lecture 7 Radiometry Slides (PPTX, PDF)
03/11 Project abstract DUE   Submission form
03/17 No class (spring break)  
03/22 Lab 5 Image Segmentation with K-Means Clustering Notebook (Jupyter)
03/24 Lecture 8 Texture and Segmentation Slides (Keynote, PDF)
03/29 Lab 6 Image Segmentation using Texture Notebook ( Jupyter )
03/31 Lecture 9 Stereopsis and two-view geometry Slides (PPTX, PDF)
04/05 Lecture 10 Epipolar Geometry Slides (PPTX, PDF)
04/06 Exercise 3 DUE HW on Mean Shift Link
04/07 Lecture 11 Reconstruction & Structure From Motion Slides (PPTX, PDF)
04/12 Lecture 12 & Lab 7 3D Processing, cont. Slides (PPTX, PDF), Lab Notebook (Jupyter)
04/14 Lecture 13 CNNs Intro Slides (Keynote, PDF)
04/19 Lab 8 Receptive Fields Size Slides (PPTX, PDF), Notebook (Jupyter)
04/21 Lecture 14 Transformers and Object Detection Slides (Keynote, PDF)
04/26 Lab 9 Object Detection with CNNs and YOLO Notebook (Jupyter)
04/28 Lecture 15    
05/03 Lab    
05/04 Exercise 4 DUE HW on Neural Networks Link
05/05 Lecture 14 Final Presentations  

Acknowledgements

Much of the material for this course relies on the Computer Vision course given at ENS Paris by Mathieu Aubry, Karteek Alahari, Ivan Laptev, and Josef Sivic. Many of the slides are taken from James Hays, Svetlana Lazebnik, and Derek Hoeim. Website was originally designed by Matthew Trager.