Audio Processing

Course Title: Audio Processing

Course No: ENCT389

Nature of the Course: Theory + Lab

Semester: 6

Full Marks: 60 + 40 + 25

Pass Marks: 24 + 16 + 10

Credit Hours: 3

Course Description

This course develops competency in analyzing and processing audio signals in time and frequency domains. It covers fundamentals of acoustics, digital audio representation, spectral analysis and audio effects, along with practical skills in feature extraction and Music Information Retrieval (MIR). Hands-on use of Digital Audio Workstations (DAWs), MIDI, and plugins enables application of audio signal processing techniques in multimedia, speech processing, music technology and emerging digital audio applications.

Course Objectives

Upon completion, students will be able to:

Analyze and process audio signals in time and frequency domains
Apply fundamentals of acoustics and digital audio representation
Perform spectral analysis and implement audio effects
Extract features and apply Music Information Retrieval (MIR) techniques
Use Digital Audio Workstations (DAWs), MIDI, and plugins for audio signal processing in multimedia, speech processing, music technology and emerging digital audio applications

Course Contents

1. Fundamental of Sound and Audio Signal

7 hrs9 marks

1.1. Physical acoustic: Wave propagation, frequency, amplitude, phase

1.2. Psycho acoustic: Pitch, loudness, timbre, dynamics and intensity

1.3. Fundamental of digital audio

1.4. Audio file format: WAV, AIFF, FLAC, ALAC, MP3

1.5. Practical audio quality: Sample rates (44.1khz, 48khz, 96khz) and bit depths

2. Time Domain Audio Processing

8 hrs11 marks

2.1. Discrete time audio signal, basic signal operations, system in time domain

2.2. Convolution and impulse response

2.3. Analog and digital filters in audio systems

2.4. Temporal feature extraction

3. Spectral Representation

7 hrs9 marks

3.1. DFT and FFT: Derivation and computational complexity

3.2. Windowing function in audio

3.3. Short time Fourier transform: Frame size, hope size and overlap

3.4. Spectrograms: Reading and interpreting time-frequency representation

3.5. Mel spectrogram: Reading and interpreting; Mel frequency representation

3.6. Constant Q transform and its relevance to music analysis

4. Frequency Domain and Non-Linear Effects

10 hrs13 marks

4.1. Sinusoidal model

4.2. Deterministic plus residual model

4.3. Deterministic plus stochastic model

4.4. Basis of non-linear effects

4.5. Dynamic range control

Envelop follow

Compressor and limiting

Expansion and gating

ADSR

Modulation effect

4.6. Time segment process: Time scratching and pitch shifting

5. Music Information Retrieval

6 hrs8 marks

5.1. MIR overview: Types of music data, challenges, application

5.2. Spectral centroid and roll off

Basics of spectral centroid and spectral roll

Comparison between centroid and roll off

Application (Timbre analysis, music/ speech classification)

5.3. Mel frequency cepstral coefficient (MFCC)

Motivation for ceptral features

Steps in MFCC

Interpretation of coefficient

Application (Speech recognition, music classification)

5.4. Chroma

Concept of pitch classes (12- tone system) mapping

Chromagram representation

5.5. Onset and tempo and their applications

6. Audio system and DAW Integration

4 hrs6 marks

6.1. Digital audio workstation (DAW)

Role in audio production

History and evolution of DAW

6.2. DAW available in market, DAW interface, timeline and arrangement view

6.3. MIDI: Basics, MIDI message structure, VSTi basics

6.4. Common plugin: Equalizer, compressor, reverb, delay

7. Recent Trend

3 hrs4 marks

7.1. Machine learning in audio

7.2. Audio foundation models for bioacoustic and healthcare

7.3. Spatial audio

7.4. Audio DSP ICs

7.5. Generative AI in music

Laboratory Works

Practical sessions covering sinusoid generation and aliasing, audio file format comparison, temporal feature extraction, spectrogram and STFT, Mel spectrogram and CQT, MIR features including MFCC, and DAW integration with MIDI and VSTi implementation. (15 hours)

1.Generate sinusoids; Downsample to observe aliasing
2.Measure file size, SNR, and spectral differences between WAV, MP3 (128k/320k), and FLAC
3.Temporal features extraction
4.Spectrogram and STFT
5.Mel spectrogram and CQT
6.MIR features: MFCC
7.DAW: Implement basic features
8.DAW integration and MIDI, VSTi implementation

Text Books

1.Christensen, M. G. (2019). Introduction to audio processing. Springer.
2.Smith, J. O. (2011). Spectral audio signal processing. W3K Publishing.
3.Zölzer, U. (Ed.). (2011). DAFX: Digital audio effects. Wiley.
4.Müller, M. (2015). Fundamentals of music processing: Audio, analysis, algorithms, applications. Springer.
5.Lyons, R. G. (2011). Understanding digital signal processing. Pearson Education.

Notes:

This syllabus follows the official BCT curriculum of Tribhuwan University. In case of any doubt or revision, the university's published syllabus shall be considered authoritative. https://ioe.tu.edu.np/pages/computer-engineering-curriculum-structure-2635

Audio Processing

Audio Processing

Select University

Select Program

TabFlux . Audio Processing . TU . BCT-NEW