Character Animation with Direct3D- P13:This book is primarily aimed at teaching indie and hobby game developers how to create character animation with Direct3D. Also, the seasoned professional game developer may find some interesting things in this book. You will need a solid understanding of the C++ programming language as well as general object-oriented programming skills. | 226 Character Animation with Direct3D fairly educated guess as to which phoneme is being spoken. Figure shows the waveform from the previous speech sample together with the spectrograph of the same sample a spectrograph shows the frequency and amplitude of a signal . FIGURE Waveform and spectrograph of a voice sample. As you can see in Figure distinct patterns can be seen in the spectrograph as the phonemes are spoken. As a side note speech-to-text applications take this analysis one step further and use Hidden Markov Models HMM to figure out which exact word is being spoken. Luckily we don t need to dive that deep in order to create reasonable lip-syncing. T P If you are interested in analyzing speech and making your own phoneme extractor you ll need to run the speech data through a Fourier Transform. This will give you the data in the frequency domain which in turn will help you build the spectrogram and help you classify the phonemes. Check out for a Fast-Fourier-Transform library in C. Analyzing speech and extracting phonemes is a rather CPU-intense process and is therefore pre-processed in all major game engines. However some games in the past have used a real-time lip-syncing system based simply on the current amplitude of the speech Simpson04 . With this approach the voice lines are evaluated just a little ahead of the playback position to determine which mouth shape to use. In the coming sections I will look at a similar system and get you started on analyzing raw speech data. lease purchase PDF Split-Merge on to remove this watermark. Chapter 10 Making Characters Talk 227 Sound Data Before you can start to analyze voice data I ll need to go off on a tangent and cover how to actually load some raw sound data. No matter which type of sound format you will actually use once the sound data has been decompressed and decoded the raw sound data will be the same across all sound formats. In this chapter I ll just use .
