1. Introduction
The movie ‘Iron Man’ comes with an artificial
intelligence computer "Jarvis" that perfectly comprehends human
language as shown in Fig. 1. 'Jarvis' can be commanded through a human voice
without an input device such as a separate keyboard or mouse. Speech
recognition is the most basic technology for realizing this 'Jarvis'. To create
a machine that recognizes human speech, it is necessary to understand the
principle of voice generation and the characteristics of voice signals. In this
article, we describe the most widely used MFCCs for analyzing the
characteristics of human speech signals.
 |
| Fig 1. Jarvis in the Iron Man movie |
2. The generation of speech sound
The generation of a speech sound is due to
vibrations of the vocal cords as shown in Fig. 2. During the human breathing
process, the air passes through the airway quickly. At this time, the vocal
cords are vibrated by the air flowing between the vocal cords due to the
Bernoulli’s effect. Between the pairs of vocal cords there is a glottis where
the sound waves are first generated. The average vibration frequency of vocal
cords for men is 120 Hz and the average vibration frequency of vocal cords for
women is 230 Hz. This frequency is the fundamental frequency and the harmonic
components are synthesized in multiples of the fundamental frequency.
 |
Fig 2. Structure of vocal organ
|
The sound waves, generated in the vocal cords
and glottis, pass through the vocal tract and are converted to a certain level
of speech signal as shown in Fig. 3. A vocal tract is a path of sound composed
of the neck, mouth, oral cavity, nasal cavity, tongue, and so on.
Depending on the shape of the vocal tract, the speech sound is produced as
syllable units. In terms of digital signal processing, the shape of a vocal
tract can be regarded as a kind of transfer function. Therefore, we can define
it as vocal tract transfer function. The excitation signal generated by the
glottis is a source of voice, but not important in terms of speech
recognition. Rather, recognizing the vocal tract that can estimate the
shape of the oral cavity and the position of the tongue is the main purpose of
speech recognition. In other words, it is essential to extract the frequency
envelope of the speech signal and to remove the excitation signal.
 |
| Fig 3. The process of speech sound generation |
3. Characteristics of speech signal
The speech sound is generated by applying an
excitation signal generated by vibrations of the vocal cords to the vocal tract
transfer function. In the speaker recognition field, the excitation signal can
be used as an important element of personality. However, in the field of speech
recognition, only the envelope of the speech signal expressed by the vocal
tract transfer function is important. Therefore, in order to recognize a
speech, it is necessary to extract the envelope of the speech signal and focus
on suppressing the characteristics of the individual as much as possible.
Figure 4 shows one frame
of the speech signal in the time domain and frequency domain. The frequency
domain of the speech signal is obtained by an FFT operation. As shown in Fig.
4, there are many ripples in the frequency domain signal. These ripples are due
to vibrations of the vocal cords and are considered noise components that
interfere with speech recognition. Therefore, it is necessary to focus on the
spectral envelope of the speech signal in order to increase the recognition
rate of the speech signal.
 |
| Fig 4. Speech signal in time domain and frequency domain |
4. Speech signal analysis using Cepstrum
First of all, a person's voice is generated through the vibration of the vocal cords (excitation signal) and is made into a speech expressing language as it passes through the vocal tract. From the viewpoint of digital signal processing, it can be assumed that the excitation signal of the vocal cords passes through a digital filter expressed by a vocal tract transfer function as shown in Fig. 5.
 |
| Fig 5. Speech generation from the perspective of digital filter |
Thus, the speech signal s(n)
is the result of the convolution operation of the excitation signal x(n) and
the transfer function h(n), and this is expressed in the frequency domain as a
multiplication of X(f) and H(f).
Here, h(n) is the object that we want to extract for human speech recognition purposes. The shape of the lips and the position of the tongue can be estimated through h(n). However, it is difficult to separate the two signals from S(f), which is the form of the product of the two signals. Therefore, it is meaningful to use cepstrum to convert the form of multiplication into the form of addition by taking logarithm on both sides. The two signals synthesized in an additive form can be easily separated. It can be seen that the excitation signal and spectral envelope can be successfully separated using cepstrum as shown in Fig. 6. Then, ‘liftering’, which is the same concept of ‘filtering’ in the general digital signal processing field, can be used to extract only envelope area in cepstrum field.
 |
| Fig 6. Separation of envelope and excitation signal by using cepstrum |
The entire
process of speech signal analysis using cepstrum is illustrated in Fig. 7. As
described above, this process consists of the 'hamming windowing', 'DFT (FFT)'
for frequency domain transformation, and log operations and so on. Finally, we
can get cepstrum coefficients that can be used to pattern match or categorize.
 |
| Fig 7. Cepstrum process of speech signal |
5. Speech signal analysis using Mel-Frequency Cepstrum
Coefficients (MFCC)
We have already seen that the cepstrum is capable of
recognizing the speech signal by itself. Nevertheless, why do we need the Mel-frequency
cepstrum coefficients (MFCC)? The only difference between cepstrum and MFCC is
that there is a Mel-filter bank in the process.
The Mel-filter bank is
designed to mimic sound scale perception of the human ear. Therefore, the Mel-filter
bank allocates fewer sub-filters in the high-frequency range and a larger
number of sub-filters in the lower-frequency range as shown in Fig. 8. A
detailed description of the Mel-Scale and Mel-filter banks is available at the
following link:
 |
| Fig 8. Structure of Mel-filter bank |
Therefore,
the entire process of MFCC is illustrated in Fig. 9. As already mentioned above,
the MFCC process is almost identical to the cepstrum process but only the Mel-filter
bank. During the MFCC process, the 'IDFT (IFFT)' step can be replaced by 'Discrete Cosine Transform (DCT)'. When the IDFT or the IFFT is performed, there is a disadvantage that the imaginary term is generated again in the resultant value. On the contrary, when using the DCT, the result is composed of only real number coefficients and has the advantage of faster processing time. In addition, because the DCT coefficient values are arranged in the order of low frequency to high frequency, the envelope components concentrated in the low frequency region can be effectively separated using this feature.
 |
| Fig 9. Entire process of MFCC |
Finally, what are the advantages of
using MFCC in speech recognition? Thanks to the Mel-filter bank in the MFCC
process, it can be assumed that subtle differences in pronunciation according
to individual characteristics are eliminated. For human speech recognition, we
need to find common characteristics of everyone who uses same language. Speech
recognition becomes difficult when the characteristics of pronunciation of each
individual are reflected. In other words, by using the Mel-filter bank, the
subtle pronunciation characteristics of each individual can be eliminated. In
addition, since the noise in the high-frequency range is suppressed, the effect
of suppressing the surrounding high-frequency noise that can be included when
recording the voice through the microphone can be also obtained. Therefore,
cepstrum can be used for human speech recognition, but MFCC is used in more application.
6. Conclusion
1. The speech sound
is generated by applying an excitation signal generated by vibrations of the
vocal cords to the vocal tract transfer function.
2. Both cepstrum
and MFCC can be used to human speech recognition.
3. The only difference
between cepstrum and MFCC is that there is a Mel-filter bank in the MFCC process.
4. Due to the
Mel-filter bank in the MFCC process, subtle differences in pronunciation
according to individual characteristics could be eliminated. In addition, it
can also suppress the high frequency band noise that can be included in the
voice recording process. Because of this feature, MFCC can increase speech
recognition rate dramatically.
End.
