Introduction to Digital Signal Processing: Interactive Sound and Video
Instructor: Frank Spigner
This course will demonstrate, and put into practice, an interdisciplinary approach to computer-based signal processing for the purpose of creating real-time interactive media for use in visual, sonic, performative, and time-based arts practices. The course will provide an introduction to the history and theory of digital signal processing (DSP), aesthetic issues, contemporary practices, and will focus on Cycling ’74’s Max/MSP/Jitter, a visual programming environment first developed by Miller Puckett at IRCAM in the mid-80s and continuing today as being an industry standard and ‘lingua franca’ of interactive and multimedia artists and researchers.
- To introduce students to the history, materials, and techniques of computer-based digital signal processing.
- To introduce students to Cycling ’74’s Max/MSP/Jitter.
- To provide students with opportunities to develop proficiency in interactive and real-time media production using DSP.
- To orient students to a range of creative coding and hardware components to augment their use of DSP in Max/MSP/Jitter.
- To provide students with a framework to build upon after the completion of the course, as to allow them to further adapt Max/MSP/Jitter to their own specific needs and to be prepared to integrate new technologies into their creative practice as they emerge.
- To give students a resource for building custom tools in their own creative practice, such as creating Max for Live virtual instruments and effects, or as a platform for multichannel and interactive instruments, installations, or live processing.
This is a hybrid/lab course with in-class lectures and discussion with assignments to be completed outside of class. There are several demonstration Max patches for each week to demonstrate new concepts, which are available to students for download.
Grading is as follows:
10% class participation
20% in-class presentations and assignments
Students are expected to attend class with a personal computer running Windows or Mac with Max/MSP/Jitter installed (Educational discount subscription available on official Cycling ’74 website)
Introduction to digital signal processing and examples of DSP used in art and music.
Introduction to Cycling ’74’s Max/MSP/Jitter.
Examples of music and art that use digital signal processing, physical computing, etc.
Assignment: Make sure Max/MSP/Jitter is installed and working properly
– Max Interface – the console, the inspector window, locking and unlocking patch/edit mode, placing objects
– Max/MSP basic syntax and signal flow
– Message ordering
– Types of data used in Max/MSP/Jitter – Numerical data, MIDI, Signals, Jitter matrices
– Types of messages – Bangs, Numbers (int, float), Words (or symbols), Lists
Make a patch using the button, toggle, number, float, comment, and print objects. The patch should demonstrate understanding of signal flow, all the objects covered in class, and the ability to print messages and numerical data to the console.
– Max Tutorial 1 “Hello” and tutorial patch “01mHello” (Help>Reference>Max Tutorials>”Hello”>”Open Tutorial”)
– Max Tutorial 5 “Message Ordering” and tutorial patch “05mMessageOrdering” (Help>Reference>Max Tutorials>”Message Ordering”>”Open Tutorial”)
- Mathematical operators
-Basic video operation
-Camera live feed
-Manipulating video with sound
-Sending and receiving values
-More about subpatches
-Physical computing with camera feed
-Multiple camera feeds
-Intro to computer vision
-“Sketching” objects with open GL
-Interact with file system outside Max
-More about 3D rendering
-Image to sound
ATTENDANCE / PARTICIPATION
Attendance is mandatory. Absences for religious holidays & family emergencies must be cleared with the instructor in advance.
Absence due to illness will only be excused with a doctor’s note.
Arrival to class more than 5 minutes late or leaving early will be counted as ½ of an absence, subsequently affecting your participation grade.
If you miss an exercise presentation because of an unexcused absence, you will receive a grade of 0
Three absences will result in failure of the course, in keeping with Tisch policy.
TISCH POLICY ON ACADEMIC INTEGRITY
The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members. Any attempt to commit plagiarism or cheat is in violation of Tisch School of the Arts community standards. All confirmed violations of the Academic Integrity Policy will be reported to the department chair and the associate dean for student affairs and subject to disciplinary action. Penalties for violations of the Academic Integrity Policy may range from being required to redo an assignment to dismissal from the school. The full Academic Integrity policy can be found at:
Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange 212-443-9999. Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center 212-998-4980. Please let your instructor know if you need help connecting to these resources