SynthMaker The Synthesis of Complex Audio Spectra by Means of Frequency Modulation
One sine wave wobbling another's pitch at audio speed: two oscillators make rich spectra that change through a note. This is FM synthesis.
Papers
Papers on sound: a reading list
The key research papers on how sound is made, modelled and processed, from FM synthesis and the plucked string to circuit simulation, reverb and neural audio. Each comes with its idea in one plain line and a link to the original. Papers marked "Explained with patches" have their own page, where each idea becomes a live patch you can play.
· 43 papers · The circuits course
Synthesis
How to make a sound from nothing: start with FM and the plucked string, then the models that rebuild sounds from sines and noise, then the tricks that keep digital waveforms clean.
Digital Synthesis of Plucked-String and Drum Timbres
A burst of noise looped through a short delay, softened a little on every pass, rings out like a plucked string.
Extensions of the Karplus-Strong Plucked-String Algorithm
The plucked-string loop made playable: exact tuning, control over how fast notes die away, where the string is picked, and how hard.
Speech Analysis/Synthesis Based on a Sinusoidal Representation
A voice rebuilt from a few dozen sine waves, each gliding in pitch and level over time.
Spectral Modeling Synthesis: A Sound Analysis/Synthesis System Based on a Deterministic plus Stochastic Decomposition
A sound split in two: sine waves for the pitched part, shaped noise for the breath and scrape. DDSP is built on this harmonic-plus-noise model.
Alias-Free Digital Synthesis of Classic Analog Waveforms
Digital sawtooth and square waves without the harsh, out-of-tune tones that fold back from too-high harmonics, using band-limited impulse trains.
Antialiasing Oscillators in Subtractive Synthesis
A survey of the tricks that keep classic synth waveforms clean at digital sample rates, so high notes stay free of stray, folded-back tones.
Physical modelling
Simulating the physics of a string or a tube rather than copying its sound: the waveguide idea first, then the fractional delays that keep it in tune.
Physical Modeling Using Digital Waveguides
A string or a tube becomes two delay lines carrying waves in opposite directions: physics cheap enough to play in real time.
Splitting the Unit Delay: Tools for Fractional Delay Filter Design
How to delay a sound by a fraction of a sample, which a physical model needs to play in tune: whole-sample delays only reach certain pitches.
Circuits and virtual analog
Making software behave like an electronic circuit. Read it in threads: the reference simulator and op-amp model, the two schools of real-time circuit modelling (state-space and wave digital), filters, distortion without aliasing, and the bucket-brigade delay chip.
SPICE2: A Computer Program to Simulate Semiconductor Circuits
The circuit simulator that others, ours included, are still checked against.
Macromodeling of Integrated Circuit Operational Amplifiers
A whole op-amp boiled down to a handful of parts that still keep its speed limits. Our op-amp models are based on it.
Simulation of the Diode Limiter in Guitar Distortion Circuits by Numerical Solution of Ordinary Differential Equations
Two guitar distortion circuits solved as equations, one sample at a time: what a diode clipper really does to a signal.
Digital Implementation of Musical Distortion Circuits by Analysis and Simulation
The DK method: a schematic turned into a small state-space system that runs live. Our circuits run on it.
A Generalized Method for the Derivation of Non-Linear State-Space Models from Circuit Schematics
Any schematic turned automatically into the state-space form a real-time simulator needs, with no working out by hand for each circuit.
Wave Digital Filters: Theory and Practice
Digital filters built from circuit parts that pass waves to each other: wave digital filters, the other big school of circuit modelling.
Wave Digital Filter Adaptors for Arbitrary Topologies and Multiport Linear Elements
Wave digital filters extended to circuits that aren't simple trees of series and parallel connections.
The Art of VA Filter Design
How to turn analog filters into digital ones that respond like the originals, even when the cutoff knob is swept fast.
Reducing the Aliasing of Nonlinear Waveshaping Using Continuous-Time Convolution
Distortion without the digital grit: integrate the distortion curve instead of sampling it, and the harsh folded-back tones drop away.
Virtual Analog Models of the Lockhart and Serge Wavefolders
Two classic wavefolders, one made of a transistor pair and one of diodes and an op-amp, written as exact formulas: why a louder note folds into brighter harmonics.
Explained with patches →Antiderivative Antialiasing for Memoryless Nonlinearities
The same idea, generalised and analysed: antiderivative antialiasing, a way to make distortion curves alias less.
The TR-808 Cymbal: a Physically-Informed, Circuit-Bendable, Digital Model
Six square waves at clashing pitches, two band-pass filters and clipping amplifiers: a classic analogue drum machine's cymbal, modelled block by block from its parts.
Explained with patches →Practical Modeling of Bucket-Brigade Device Circuits
The analog delay chip behind classic choruses and echoes, modelled block by block.
A Combined Model for a Bucket Brigade Device and its Input and Output Filters
Run the simulated delay chip at its own clock rate and let the circuit's own filters do the resampling.
Antialiasing in BBD Chips Using BLEP
Keep the delay chip's own lo-fi aliasing, part of its sound, and remove the extra aliasing the simulation adds.
Reverb and delay
How to put a room inside a computer, in the order the ideas arrived: echo loops, early reflections, feedback delay networks, convolution with real rooms, and a classic plate design.
Natural Sounding Artificial Reverberation
The first digital reverb: echoes circling in feedback loops (comb filters), smeared into a smooth tail by all-pass filters.
About This Reverberation Business
Early reflections and damped loops: what it takes to make a digital room sound real.
Digital Delay Networks for Designing Artificial Reverberators
Feedback delay networks: many delay lines fed back into each other through a mixing matrix, for a smooth reverb you can control.
Efficient Convolution without Input-Output Delay
Real rooms by convolution, live and with no wait: a long recorded impulse response split into blocks that grow along its tail.
Effect Design, Part 1: Reverberator and Other Filters
A classic studio plate reverb's structure, laid out so you can build it.
Neural audio synthesis
Networks that make sound. Read in order, they move from writing raw audio one sample at a time to models that drive a real synth and run live.
WaveNet: A Generative Model for Raw Audio
A network that writes audio one sample at a time, each predicted from all before it. Listeners rated its speech more natural than earlier systems.
Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
A network that learns the space between instruments, so you can morph from one timbre to another, plus a dataset of 300,000 notes.
GANSynth: Adversarial Neural Audio Synthesis
Whole notes generated at once, with pitch and timbre learned apart, and several orders of magnitude faster than writing them sample by sample.
DDSP: Differentiable Digital Signal Processing
The network drives a real synth (sines, filtered noise and reverb) instead of writing samples, and can turn a singing voice into a violin.
RAVE: A variational autoencoder for fast and high-quality neural audio synthesis
A neural sound model that makes 48 kHz audio 20 times faster than real time on a laptop's processor: timbre transfer you can play.
Real-Time Guitar Amplifier Emulation with Deep Learning
A small recurrent network learns a real guitar amplifier's sound and runs live.
Analysis and the DSP classics
The tools for looking inside a sound: the fast Fourier transform and windows first, then measuring noise and single frequencies, the physics of resistor hiss, and tracking a note's pitch.
An Algorithm for the Machine Calculation of Complex Fourier Series
The fast Fourier transform: a spectrum of 1,024 samples for about a hundredth of the work of direct calculation. Behind every spectrum analyser since.
On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform
Why a spectrum leaks energy into neighbouring frequencies, and which window shape to use against it: the reference.
The Use of Fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging over Short, Modified Periodograms
Measuring noise reliably by averaging many short spectra instead of trusting one long, jumpy one.
An Algorithm for the Evaluation of Finite Trigonometric Series
The strength of one frequency in a signal, worked out cheaply without a full spectrum. We use it to measure circuits for the course.
Thermal Agitation of Electricity in Conductors
Every resistor hisses, and the hiss grows with temperature: thermal noise, measured.
Thermal Agitation of Electric Charge in Conductors
The theory behind resistor hiss: the 4kTR formula, tying the noise to temperature and resistance. Our thermal noise follows it.
YIN, a Fundamental Frequency Estimator for Speech and Music
Finding a note's pitch from its waveform, robustly.
CREPE: A Convolutional Representation for Pitch Estimation
A network that tracks pitch straight from the waveform, matching or beating the best classic method. DDSP uses it to follow a performer's pitch.