Investigating the visual lombard effect with gabor based features

Waito Chiu; Yan Xu; Andrew Abel; Chun Lin; Zhengzheng Tu

doi:10.21437/Interspeech.2020-1291

Investigating the visual lombard effect with gabor based features

Waito Chiu, Yan Xu, Andrew Abel, Chun Lin, Zhengzheng Tu

Department of Computing

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

2 Citations (Scopus)

Abstract

The Lombard Effect shows that speakers increase their vocal effort in the presence of noise, and research into acoustic speech, has demonstrated varying effects, depending on the noise level and speaker, with several differences, including timing and vocal effort. Research also identified several differences, including between gender, and noise type. However, most research has focused on the audio domain, with very limited focus on the visual effect. This paper presents a detailed study of the visual Lombard Effect, using a pilot Lombard Speech corpus developed for our needs, and a recently developed Gabor based lip feature extraction approach. Using Kernel Density Estimation, we identify clear differences between genders, and also show that speakers handle different noise types differently.

Original language	English
Title of host publication	Interspeech 2020
Publisher	International Speech Communication Association
Pages	4606-4610
Number of pages	5
ISBN (Print)	9781713820697
DOIs	https://doi.org/10.21437/Interspeech.2020-1291
Publication status	Published - 2020
Event	21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020 - Shanghai, China Duration: 25 Oct 2020 → 29 Oct 2020

Publication series

Name	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Volume	2020-October
ISSN (Print)	2308-457X
ISSN (Electronic)	1990-9772

Conference

Conference	21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020
Country/Territory	China
City	Shanghai
Period	25/10/20 → 29/10/20

Keywords

Gabor Features
Lip Features
Lombard Effect

Access to Document

10.21437/Interspeech.2020-1291

Cite this

Chiu, W., Xu, Y., Abel, A., Lin, C., & Tu, Z. (2020). Investigating the visual lombard effect with gabor based features. In Interspeech 2020 (pp. 4606-4610). (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH; Vol. 2020-October). International Speech Communication Association. https://doi.org/10.21437/Interspeech.2020-1291

@inproceedings{ca99a96a72f84adfb0dff533813867f5,

title = "Investigating the visual lombard effect with gabor based features",

abstract = "The Lombard Effect shows that speakers increase their vocal effort in the presence of noise, and research into acoustic speech, has demonstrated varying effects, depending on the noise level and speaker, with several differences, including timing and vocal effort. Research also identified several differences, including between gender, and noise type. However, most research has focused on the audio domain, with very limited focus on the visual effect. This paper presents a detailed study of the visual Lombard Effect, using a pilot Lombard Speech corpus developed for our needs, and a recently developed Gabor based lip feature extraction approach. Using Kernel Density Estimation, we identify clear differences between genders, and also show that speakers handle different noise types differently.",

keywords = "Gabor Features, Lip Features, Lombard Effect",

author = "Waito Chiu and Yan Xu and Andrew Abel and Chun Lin and Zhengzheng Tu",

note = "Publisher Copyright: Copyright {\textcopyright} 2020 ISCA; 21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020 ; Conference date: 25-10-2020 Through 29-10-2020",

year = "2020",

doi = "10.21437/Interspeech.2020-1291",

language = "English",

isbn = "9781713820697",

series = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

publisher = "International Speech Communication Association",

pages = "4606--4610",

booktitle = "Interspeech 2020",

}

Chiu, W, Xu, Y, Abel, A, Lin, C & Tu, Z 2020, Investigating the visual lombard effect with gabor based features. in Interspeech 2020. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 2020-October, International Speech Communication Association, pp. 4606-4610, 21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020, Shanghai, China, 25/10/20. https://doi.org/10.21437/Interspeech.2020-1291

Investigating the visual lombard effect with gabor based features. / Chiu, Waito; Xu, Yan; Abel, Andrew et al.
Interspeech 2020. International Speech Communication Association, 2020. p. 4606-4610 (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH; Vol. 2020-October).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Investigating the visual lombard effect with gabor based features

AU - Chiu, Waito

AU - Xu, Yan

AU - Abel, Andrew

AU - Lin, Chun

AU - Tu, Zhengzheng

PY - 2020

Y1 - 2020

N2 - The Lombard Effect shows that speakers increase their vocal effort in the presence of noise, and research into acoustic speech, has demonstrated varying effects, depending on the noise level and speaker, with several differences, including timing and vocal effort. Research also identified several differences, including between gender, and noise type. However, most research has focused on the audio domain, with very limited focus on the visual effect. This paper presents a detailed study of the visual Lombard Effect, using a pilot Lombard Speech corpus developed for our needs, and a recently developed Gabor based lip feature extraction approach. Using Kernel Density Estimation, we identify clear differences between genders, and also show that speakers handle different noise types differently.

AB - The Lombard Effect shows that speakers increase their vocal effort in the presence of noise, and research into acoustic speech, has demonstrated varying effects, depending on the noise level and speaker, with several differences, including timing and vocal effort. Research also identified several differences, including between gender, and noise type. However, most research has focused on the audio domain, with very limited focus on the visual effect. This paper presents a detailed study of the visual Lombard Effect, using a pilot Lombard Speech corpus developed for our needs, and a recently developed Gabor based lip feature extraction approach. Using Kernel Density Estimation, we identify clear differences between genders, and also show that speakers handle different noise types differently.

KW - Gabor Features

KW - Lip Features

KW - Lombard Effect

UR - http://www.scopus.com/inward/record.url?scp=85098112563&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2020-1291

DO - 10.21437/Interspeech.2020-1291

M3 - Conference Proceeding

AN - SCOPUS:85098112563

SN - 9781713820697

T3 - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

SP - 4606

EP - 4610

BT - Interspeech 2020

PB - International Speech Communication Association

T2 - 21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020

Y2 - 25 October 2020 through 29 October 2020

ER -

Investigating the visual lombard effect with gabor based features

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this