Understanding Sound from First Principles: Part 2

Introduction

We have made it thus far. For starters, please check out Part 1, where we delved into the underlying principles of sound creation from physical systems that vibrate. It laid the foundation for this and subsequent parts of the series as we walk our way through the layers of abstraction from signals to AI audio systems. Today, we are briefly going to talk about the broad classification of signals (i.e., speech, electric current, electric voltage, ECG recordings, etc.). There has been a groundswell of interest in studying the underlying principles of signals in a plethora of STEM fields (even the social sciences are joining the bandwagon, so to speak). The goal of this intellectual sprint is to gain a deeper appreciation of the potential applications of AI in sound research by leveraging the already mature body of work that exists in the space of signal processing. It can be quite tempting to follow the adrenaline rush of pursuing end-to-end learning from audio signals, which can introduce phantom problems and less expressive (aka black-boxed) models. As you may have realized, using a principled, multiscale approach that leverages existing domain knowledge in signal processing helps tremendously in building AI models for speech signals. Enough with the motivational speech. Let's get technical!

Sound can be classified, in a very broad sense, into periodic and aperiodic components. If you are wondering why we are suddenly talking about signals in general, you have a point. It is because signal processing is the umbrella that covers all kinds of signals; imagine it as the mathematical language we use to describe changes in some physical quantity. But our focus here is to study sound, not all the bells and whistles of signals in general. There are tons of textbooks, tutorials, you name it, that more than eloquently explain these notions. Our goal is to connect the dots, build the intuition and lay bare the possibility that you could have even discovered some of these algorithms yourself. You may ponder, what is a periodic signal and does it have a connection with the period of a wave? The answer to the latter is yes. A continuous-time signal $X(t)$ is periodic if there exists some positive constant $T$ such that

for all values of $t$. Here, $T$ is called the period of the signal. In other words, after every $T$ seconds, the signal returns to the same state. This is precisely the idea of a repeating pattern that we introduced in Part 1. The smallest positive value of $T$ satisfying this condition is usually referred to as the fundamental period of the signal. And, as we already know from Part 1, the frequency is simply the inverse of the period:

This is where things become particularly interesting for sound. Consider a tuning fork vibrating at a particular frequency. Its vibration can produce a sound wave with a very regular repeating structure. A pure sinusoidal tone is an even simpler example. If we write it as

then the waveform repeats itself every $T_0 = 1/f_0$ seconds. For example, a 440 Hz sinusoid completes 440 cycles every second and therefore has a period of approximately 2.27 milliseconds. This is the familiar concert pitch A above middle C. Notice that we are not saying that all sounds are sinusoids. Far from it. A sinusoid is simply the simplest building block with which we can begin understanding more complicated signals.

Periodic Signals

So what does a periodic sound actually sound like? Think of a sustained musical note played by a flute, a violin, or even a tuning fork. The pressure variations produced by the vibrating source exhibit a degree of regularity. This regularity is what makes periodicity such a useful concept in audio analysis. In particular, voiced human speech sounds such as vowels often contain a strong periodic component because the vocal folds open and close quasi-periodically.

I deliberately used the word quasi-periodically. This is important. Real-world sound is rarely as obedient as the mathematical signals we draw on a piece of paper. A human voice does not produce exactly the same waveform cycle after cycle. The vocal-fold vibration varies slightly over time, and the shape of the vocal tract changes as we speak. Consequently, speech is often better described as locally or approximately periodic rather than perfectly periodic over an infinitely long duration.

This distinction is important because it gives us a glimpse into one of the recurring themes of signal processing: mathematical models are abstractions of physical phenomena. They do not need to reproduce every microscopic detail of reality to be useful. They need to capture the structure that matters for the problem we are trying to solve. This is one of the reasons signal processing remains so powerful even in the age of deep learning.

Aperiodic Signals

Now, what happens when the pattern does not repeat? You guessed it: we enter the world of aperiodic signals. An aperiodic signal does not possess a finite period $T$ for which $X(t)=X(t+T)$ for all $t$. In audio, aperiodic components are associated with irregular or noise-like excitation. Think about the sound produced when you pronounce the consonant /s/. There is no nice, repeating pattern like the one we observed with a sustained vowel. Instead, turbulent airflow generates a noise-like acoustic signal.

Another intuitive example is a hand clap. The acoustic waveform contains a very short, broadband transient. There is no obvious repeating cycle that we can point to and say, "there it is again." Similarly, the sound of a cymbal contains many complex, rapidly changing components that are not well described as a single periodic waveform.

This gives us a useful conceptual decomposition of speech. Voiced sounds tend to contain a strong periodic component, while unvoiced sounds tend to contain stronger aperiodic or noise-like components. Speech, however, is not simply a binary switch between the two. Real speech can contain mixtures of periodic and aperiodic components, and the balance between them changes continuously as we move from one phoneme to another.

If you listen carefully to a spoken sentence, you can actually hear this transition. Try saying "ssssss" and then "zzzzzz". The first sound is predominantly aperiodic, while the second contains a substantial periodic component because the vocal folds are vibrating. This simple observation turns out to be remarkably useful in speech processing. Before we throw a neural network at a speech signal, we can already reason about some of the physical processes that produced it.

From the Physical World to a Signal

So far, we have been comfortably living in the continuous physical world. Air pressure varies continuously in time, and the sound wave propagates through space. But computers are not particularly good at storing "continuous" things. Computers deal with numbers, and numbers have to be represented in a finite way. This is where the beautiful marriage between physics, mathematics, electronics and computing begins.

Suppose I speak into a microphone. My vocal folds create mechanical vibrations, these vibrations create pressure variations in the surrounding air, and those pressure variations reach the microphone. The microphone contains a transducer that converts the acoustic energy into an electrical signal. Depending on the microphone and circuitry, the resulting electrical signal is an analog voltage that varies continuously with time.

At this point, we have transformed the problem from acoustics into electrical signal processing. The pressure variation in air has been represented by a varying electrical quantity. This analog signal can then be converted into a digital representation that a computer can store, manipulate and eventually feed into our machine learning algorithms.

And this is where two very important concepts enter the stage: sampling and quantization.

Sampling

Sampling is the process of measuring a continuous-time signal at discrete points in time. Imagine taking a photograph of a moving object. The physical phenomenon continues to exist between the photographs, but we only retain information at particular moments. Sampling does something conceptually similar to a signal.

If the original continuous signal is $x(t)$ and we sample it every $T_s$ seconds, then the resulting discrete-time signal can be represented as

Here $T_s$ is called the sampling period and the sampling frequency is

So, if we record audio at 44.1 kHz, we are taking 44,100 measurements of the analog signal every second. Notice the subtle but important distinction: the sound itself is not "44.1 kHz." The sampling rate is 44.1 kHz. The signal may contain many different frequencies, and the sampling rate determines how frequently we measure that signal.

But then a natural question arises: how frequently do we need to sample a signal if we want to preserve its information? Surely, taking only two samples per second cannot possibly be enough to represent a signal oscillating thousands of times per second. This intuition leads us directly to one of the most important results in digital signal processing: the Nyquist-Shannon sampling theorem.

The Nyquist-Shannon Sampling Theorem

In its practical form, the theorem tells us that if a continuous-time signal is band-limited to frequencies below $f_{\max}$, then it can be perfectly reconstructed from its samples when the sampling frequency satisfies

The quantity $f_s/2$ is called the Nyquist frequency. It represents the highest frequency that can be represented without aliasing for a given sampling rate. For example, with a sampling rate of 44.1 kHz, the Nyquist frequency is 22.05 kHz. This is one of the reasons the 44.1 kHz sampling rate became historically important for digital audio: it can theoretically represent frequencies up to 22.05 kHz, which covers the commonly cited upper limit of human hearing under ideal conditions.

There is, however, a catch. Real signals are not perfectly band-limited. Components above the Nyquist frequency can fold back into the lower-frequency range during sampling. This phenomenon is known as aliasing. Once aliasing occurs, the original signal cannot simply be recovered from the samples because different continuous-time signals can produce the same sampled representation.

This is why practical digital audio systems typically use an anti-aliasing low-pass filter before the analog signal is sampled. The filter attenuates frequency components that could not be represented safely at the chosen sampling rate. In other words, before the computer starts taking photographs of our signal, we make sure that the signal does not contain details that the camera is incapable of capturing.

Quantization

We have solved one problem, but another one remains. Sampling tells us when we measure the signal. It does not yet tell us how precisely we represent the measured amplitude.

Remember that an analog signal can theoretically take on a continuous range of amplitude values. A computer, however, has finite precision. Therefore, each sampled amplitude must be mapped to one of a finite number of available numerical levels. This process is called quantization.

Suppose we use an $N$-bit system. The number of available quantization levels is

For an 8-bit system, this gives us $2^8 = 256$ possible levels. A 16-bit system provides $2^{16}=65,536$ levels, while a 24-bit system provides $2^{24}=16,777,216$ levels. More bits therefore allow us to represent the amplitude of the signal with finer precision.

But quantization comes with a price. The original amplitude may lie between two available quantization levels. We therefore have to choose the closest available level. The difference between the original value and the quantized value is called quantization error.

If we denote the original sampled signal by $x[n]$ and the quantized signal by $x_q[n]$, then the quantization error can be written as

This is another beautiful example of abstraction. We started with a physical pressure variation in the air. We transformed it into an electrical signal, sampled that signal at discrete points in time, and then approximated each measured amplitude using one of a finite number of numerical levels. What eventually reaches our machine learning model is therefore not the original physical sound wave itself, but a numerical representation of it.

Sampling + Quantization = Digital Audio

At this point, we can put the pieces together. The journey from a physical sound to digital audio can be thought of as a sequence of transformations:

Sound source $\rightarrow$ Air pressure variations $\rightarrow$ Microphone $\rightarrow$ Analog electrical signal $\rightarrow$ Sampling $\rightarrow$ Quantization $\rightarrow$ Digital audio

This is the bridge between the physical world and the computational world. The microphone performs the first major transformation by converting acoustic energy into an electrical representation. The analog-to-digital converter (ADC) then samples the electrical signal and quantizes its amplitude, producing a sequence of numbers that can be stored and processed by a computer.

Once the sound exists as a sequence of numbers, we can perform all sorts of mathematical operations on it. We can filter it, compress it, transform it into the frequency domain, visualize it as a spectrogram, extract features from it, or feed representations of it into machine learning models. But notice something important: every computational representation we create is another layer of abstraction on top of the original physical phenomenon.

This is precisely why I keep insisting on understanding the problem from first principles. If we jump directly from a waveform into a sophisticated neural network, it is easy to forget what the numbers actually represent. But if we understand the journey from vibration to pressure, pressure to voltage, voltage to samples, and samples to numerical representations, many of the decisions we make later in an AI pipeline become much more intuitive.

Putting It All Together

Let's take a moment to connect everything we have learned so far. In Part 1, we started with a vibrating physical system and established that sound is a mechanical wave carrying energy through a medium. We then introduced the sinusoidal waveform and its fundamental properties: amplitude, frequency, period and phase.

In this part, we moved one step further. We introduced periodic and aperiodic signals and saw how these concepts help us reason about real-world sounds, particularly speech. We then left the purely physical world and entered the digital domain. Sampling discretizes time, while quantization discretizes amplitude. Together, they allow a continuous analog signal to be represented as a finite sequence of numbers.

And here is the really interesting part. The sequence of numbers is not the end of the story. In fact, it is just the beginning. We can look at the signal in the time domain, where we observe how amplitude changes over time. But we can also ask a completely different question: what frequencies make up this signal?

That question takes us from the time domain into the frequency domain and opens the door to one of the most elegant ideas in signal processing: the Fourier Transform. Suddenly, a complicated waveform can be understood as a combination of simpler sinusoidal components. This idea is so fundamental that it appears almost everywhere in audio processing, from spectral analysis and filtering to speech recognition, music information retrieval and modern deep learning systems.

But let's not get ahead of ourselves. We have already travelled quite a distance from a vibrating object to a collection of numbers stored inside a computer. In the next post, we will take those numbers and look at them from another perspective. We will explore the relationship between the time domain and frequency domain, introduce the Fourier Transform and begin to understand why frequency is such a powerful lens through which to study sound.

See you in Part 3.