Sampling

🎯 Learning Objectives:

To simplify our understanding of the conversion from analog to digital signals we will start from the simplest periodic analog signal, the sine wave. The sine function is represented by following equation.

The equation is not really important in this context but what is important is that the sine wave is in a sense a one-dimensional function. What we mean here is it is actually a single valued spatial point at any given time, if we do not consider time as a dimension of course. As we progress time, we can imagine that the sine wave acquires a different value. How many unique values can it have? Infinitely many of course. The sine wave is said to be a continuous function over time meaning that no matter how close to time intervals are we would not be able to find a single break in the signal nor loss of resolution nor staggering of the shape of the wave function. The signal is continuous.

A clear advantage of digital signals over analog signals is its storage capabilities. How then are we supposed to store a signal which has infinitely many points. All the hard drives in the world could not accommodate the number of unique and distinct signal values that this one sine function can produce. Obviously for sanity's sake we would not want to store all the values, only enough values to closely resemble this function.

So what we want to do is to break apart the wave to make it discontinuous to provide values only at certain points in time. To convert it into a discrete signal the process of transforming an analog signal to a digital one begins here by asking the question how often do we want to measure the signal or how many samples do we want to have within a given interval of time in this example. As we see, in the Figure, we have 30 samples within one second interval of this one Hertz sine wave. We can also say that we have sampled this wave every point 0.03 seconds. This seems like a fairly accurate representation of the sine wave, right? We can also go lower by sampling it twenty times a second or eight or even two or go higher and sample at 44100 times a second.

Here are the questions:

1. What is the right number?
2. How many samples are enough to accurately represent this wave?
   Well, thankfully it is not as arbitrary as we think. Continuous to discrete signal conversion is governed by the Nyquist Shannon sampling theorem which states that:

Any band-limited continuous-time signal can be accurately converted to one from digital signals when sampled at a rate at least twice as high as the highest frequency component of the waveform.

Sounds complicated? Let us break it down. The sampling theorem states two important facts:

1. That to represent an analog signal in the digital domain the number of samples we would need for each second of the signal should be more than the highest frequency represented by the signal. If a signal has a 10 kKz frequency component the sampling rate needs to be more than 20 kKz
2. It also states that the analog signal needs to be band-limited to the highest frequency and cannot contain any frequencies about that.

Let us put a pin on the second point and just talk about the first in our case we had a 1Hz sine wave. In Figure below we have a constant 1Hz wave. On 1 Hz is the highest frequency that this wave function will ever produce. The sampling theorem states that to digitally represent this wave we need at least more than 2 samples. Let us say 3. Any more is fine as well but the minimum required is 3. We can say that the sample rate is 3 Hz. Figure shows how the 1 Hz wave could be sampled using 3 sample.

You might wonder at this point that three points are clearly not enough to represent this wave correctly without losing a lot of resolution. Means if this sample waveform would have to be converted back into the analog signal there is room for interpretation, right? The signal could be rendered in several different ways as is illustrated in the Figure below.

Surely, if the $1 Hz$ wave would be sampled at say $40 Hz$ sampling rate, there would be less room for divergence and we would get a more accurate representation, right? Well the short answer is NO. This is where a lot of misconceptions of higher sampling rate equal to better audio comes from.

Higher sampling rate ≠ Better audio

The absolute beauty of the sampling theorem is that it proves mathematically that both $3 Hz$ sampled waveform and $40 Hz$ sample waveform for $1 Hz$ wave, when reconverted back into its analog form does not just produce the same output signal but produces a signal which is indistinguishable from the original analog input signal before conversion.

The absolute beauty of the sampling theorem is that it proves mathematically that both $3 Hz$ sampled waveform and $40 Hz$ sample waveform for $1 Hz$ wave, when reconverted back into its analog form does not just produce the same output signal but produces a signal which is indistinguishable from the original analog input signal before conversion. The input and output signals would essentially be carbon copies of each other with no loss in resolution. This seems really strange and not intuitive because we are essentially stripping out information and discarding it when converting it into a digital signal and then when we convert it back. How is this missing information magically reappearing.
Well this is where the second point comes into effect:

The signal would need to be band-limited.

This means that an input signal going into the conversion process is stripped of all frequency components above the maximum for 1 Hz and the digital signal once reconverted is again stripped of all frequency components higher than the maximum of 1 Hz. And this can be accomplished by using a low-pass filter. A theoretical low-pass filter would only allow the passage of frequencies lower than a set threshold and would block all frequencies higher than the threshold. Here is the illustration in the animation below.

Examples:

The audacity is used to demonstrate some of the examples here. I changed the project sample rate to a lower value of $8000 Hz$ as a sample rate. So according to the sampling theorem we have learned that we can only hope to represent frequencies below half that rate. We can represent frequencies till $4000 Hz$ and no more than that. In the video below I have generated a sine wave of $1 kHz$. Play the video and you will find that it sounds like a $1 kHz$ signal. Also we can see from the spectrum.

Now let us explore the fringe of possible frequencies $4 KHz$ exactly half the sampling rate. Audacity let us do this but we can not see any signal. This is easy to illustrate. If we take a $4 KHz$ signal and only sample it twice a cycle we see that in the Figure below all sampled points unfortunately sit at the zero valued point of the signal so naturally this is interpreted as no signal at all. But increase the phase offset of the signal by 45o and suddenly the wave comes to life. So this is the reason while sampling theorem states that all frequencies lower than half the sampling rate can be represented accurately and does not say anything about exactly half the sample rate because it would not be an accurate representation for all scenarios.

Now what about $3999 Hz$. That is lower than half the sampling rate, right? Let us find out in the video below which I made in audacity. I have generated a sine tone of $3999 Hz$. Play the video below and you get a bizarre output, where the amplitude is modulated from high to low and back constantly. Surely enough, you can hear the $3999 Hz$ signal and the frequency plot says so as well except for the fact that the amplitude is all messed up.

Now let us select our sampling rates to all the next available values, such as $11025 Hz$, $44100 Hz$, $92000 Hz$ and $192000 Hz$ and listen them all in the video below.

Convinced yet that there is no difference? Audibly there is no difference but viewing the waveform its dodgy, alright? Now let us explore why this is the case. For one thing we have been viewing the waveform all wrong. We don not just linearly interpolate between two data points and put a straight line between them. This is an inaccurate representation. The accurate representation of the signal is as shown in the Figure below, i.e. stem-plot, because this is exactly what we have in the digital domain the discrete data points but nothing else.

Now this digital signal needs to be converted into an analogue signal for us to listen to it goes through a series of electronic circuits in the simplest case it goes through a resistor ladder circuit, and without unnecessarily complicating this topic, it produces a stair step pattern of an intermediate analog signal. So our data points are turned into an analog signal which looks like as show in the Figure below. It is not the final output signal yet.

In this form the analog signal has a lot of super high frequency components, which we can not even hear. How would we tell this? Well it is not clearly evident from the waveform what the exact frequencies are but when our time domain signal shifts almost instantaneously like in the Figure above, it means that the corner of the speaker would have to resonate and vibrate according to the signal. This kind of abrupt changes usually perceived as clicks and noise but if it tends to repeat periodically. We are likely to get a high-frequency tone and the frequency would depend on the slope and height of such shifts. In a complex real-world input signal we would end up getting a complex spectrum of high frequency components. In the intermediate signal the obvious thing to do would be cut off these excess frequencies and discard them and the most obvious place to start would be all the frequency about half the sample rate from which point on will be referred to as the Nyquist frequency. This is because we promised that the input signal would be band limited. When it first arrived at the analog-to-digital conversion stage by putting a low-pass filter on it with a cut-off as the Nyquist frequency clearly the output signal would need to adhere to the same rules on the same low-pass cutoff at the Nyquist frequency as well. And when the filter is applied the intermediate signal is transformed into the signal shown below, which is essentially a smoothed out version which intersects all the points that were previously sampled and this is where things get interesting.

Let us watch this animation to understand the whole procedure.

🧪 Interactive Demo

Nyquist-Shannon Sampling Simulation:

This interactive simulation shows the Nyquist-Shannon Sampling Theorem.

Signal frequency (f)

f = 20.0000 Hz

Sampling frequency (fs)

fs = 20.8750 Hz

Show alias overlay

Tip: reduce fs (sampling rate) below 2·f to see aliasing. If fs is zero the plot will show just the continuous waveform.

📝 Key Takeaways

🧠 Quick Quiz