Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features

Andrew Abe; Chengxiang Gao; Leslie Smith; Roger Watt; Amir Hussain

doi:10.1109/SSCI.2018.8628931

Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features

Andrew Abe, Chengxiang Gao, Leslie Smith, Roger Watt, Amir Hussain

Department of Computing

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

9 Citations (Scopus)

Abstract

The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.

Original language	English
Title of host publication	Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018
Editors	Suresh Sundaram
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1033-1040
Number of pages	8
ISBN (Electronic)	9781538692769
DOIs	https://doi.org/10.1109/SSCI.2018.8628931
Publication status	Published - 2 Jul 2018
Event	8th IEEE Symposium Series on Computational Intelligence, SSCI 2018 - Bangalore, India Duration: 18 Nov 2018 → 21 Nov 2018

Publication series

Name	Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018

Conference

Conference	8th IEEE Symposium Series on Computational Intelligence, SSCI 2018
Country/Territory	India
City	Bangalore
Period	18/11/18 → 21/11/18

Keywords

barcodes
gabor
image processing
lip-reading
word features

Access to Document

10.1109/SSCI.2018.8628931

Cite this

Abe, A., Gao, C., Smith, L., Watt, R., & Hussain, A. (2018). Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. In S. Sundaram (Ed.), Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018 (pp. 1033-1040). Article 8628931 (Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SSCI.2018.8628931

Abe, Andrew ; Gao, Chengxiang ; Smith, Leslie et al. / Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. editor / Suresh Sundaram. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1033-1040 (Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018).

@inproceedings{b63f9a690ef349fbbf489570f1cad47b,

title = "Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features",

abstract = "The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.",

keywords = "barcodes, gabor, image processing, lip-reading, word features",

author = "Andrew Abe and Chengxiang Gao and Leslie Smith and Roger Watt and Amir Hussain",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 8th IEEE Symposium Series on Computational Intelligence, SSCI 2018 ; Conference date: 18-11-2018 Through 21-11-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/SSCI.2018.8628931",

language = "English",

series = "Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1033--1040",

editor = "Suresh Sundaram",

booktitle = "Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018",

}

Abe, A, Gao, C, Smith, L, Watt, R & Hussain, A 2018, Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. in S Sundaram (ed.), Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018., 8628931, Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018, Institute of Electrical and Electronics Engineers Inc., pp. 1033-1040, 8th IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, 18/11/18. https://doi.org/10.1109/SSCI.2018.8628931

Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. / Abe, Andrew; Gao, Chengxiang; Smith, Leslie et al.
Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. ed. / Suresh Sundaram. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1033-1040 8628931 (Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features

AU - Abe, Andrew

AU - Gao, Chengxiang

AU - Smith, Leslie

AU - Watt, Roger

AU - Hussain, Amir

PY - 2018/7/2

Y1 - 2018/7/2

N2 - The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.

AB - The extraction of relevant lip features is of continuing interest in the speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by glimpse based psychological research into racial barcodes. This allows for 3D geometric features to be produced using Gabor based image patches. This new approach can successfully extract lip features with a minimum of processing, with parameters that can be quickly adapted and used for detailed analysis, and with preliminary results showing successful feature extraction from a range of different speakers. These features can be generated online without the need for trained models, and are also robust and can recover from errors, making them suitable for real world speech analysis.

KW - barcodes

KW - gabor

KW - image processing

KW - lip-reading

KW - word features

UR - http://www.scopus.com/inward/record.url?scp=85062781039&partnerID=8YFLogxK

U2 - 10.1109/SSCI.2018.8628931

DO - 10.1109/SSCI.2018.8628931

M3 - Conference Proceeding

AN - SCOPUS:85062781039

T3 - Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018

SP - 1033

EP - 1040

BT - Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018

A2 - Sundaram, Suresh

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 8th IEEE Symposium Series on Computational Intelligence, SSCI 2018

Y2 - 18 November 2018 through 21 November 2018

ER -

Abe A, Gao C, Smith L, Watt R, Hussain A. Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features. In Sundaram S, editor, Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1033-1040. 8628931. (Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018). doi: 10.1109/SSCI.2018.8628931

Fast Lip Feature Extraction Using Psychologically Motivated Gabor Features

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this