Teacher-Student Training for Text-Independent Speaker Recognition

Raymond W.M. Ng; Xuechen Liu; Pawel Swietojanski

doi:10.1109/SLT.2018.8639564

Teacher-Student Training for Text-Independent Speaker Recognition

Raymond W.M. Ng
, Xuechen Liu
, Pawel Swietojanski

Emotech Labs
University of New South Wales

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

8 Citations (Scopus)

Abstract

This paper investigates text-independent speaker recognition using neural embedding extractors based on the time-delay neural network. Our primary focus is to explore the teacher-student (TS) training framework for knowledge distillation in a text-independent (TI) speaker recognition task. We report the results on both proprietary and public benchmarks, obtaining competitive results with 88-93% smaller models. Particularly, in clean testing conditions, we find TS training on neural-based TI systems achieved same or better performance than the i-vector based counterparts. Neural embeddings are less prone to short segment issues, and offer better performance particularly in the high-recall setting. They can also provide some additional insights about speakers, such as gender or how difficult a given speaker can be for recognition.

Original language	English
Title of host publication	2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1044-1051
Number of pages	8
ISBN (Electronic)	9781538643341
DOIs	https://doi.org/10.1109/SLT.2018.8639564
Publication status	Published - 2 Jul 2018
Externally published	Yes
Event	2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Athens, Greece Duration: 18 Dec 2018 → 21 Dec 2018

Publication series

Name	2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings

Conference

Conference	2018 IEEE Spoken Language Technology Workshop, SLT 2018
Country/Territory	Greece
City	Athens
Period	18/12/18 → 21/12/18

Keywords

Knowledge Distillation
Speaker Recognition
Teacher-Student training

Access to Document

10.1109/SLT.2018.8639564

Cite this

Ng, R. W. M., Liu, X., & Swietojanski, P. (2018). Teacher-Student Training for Text-Independent Speaker Recognition. In 2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings (pp. 1044-1051). Article 8639564 (2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SLT.2018.8639564

@inproceedings{6e6b4b4b548141299cf2a4e123534f00,

title = "Teacher-Student Training for Text-Independent Speaker Recognition",

abstract = "This paper investigates text-independent speaker recognition using neural embedding extractors based on the time-delay neural network. Our primary focus is to explore the teacher-student (TS) training framework for knowledge distillation in a text-independent (TI) speaker recognition task. We report the results on both proprietary and public benchmarks, obtaining competitive results with 88-93\% smaller models. Particularly, in clean testing conditions, we find TS training on neural-based TI systems achieved same or better performance than the i-vector based counterparts. Neural embeddings are less prone to short segment issues, and offer better performance particularly in the high-recall setting. They can also provide some additional insights about speakers, such as gender or how difficult a given speaker can be for recognition.",

keywords = "Knowledge Distillation, Speaker Recognition, Teacher-Student training",

author = "Ng, \{Raymond W.M.\} and Xuechen Liu and Pawel Swietojanski",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE Spoken Language Technology Workshop, SLT 2018 ; Conference date: 18-12-2018 Through 21-12-2018",

year = "2018",

month = jul,

day = "2",

doi = "10.1109/SLT.2018.8639564",

language = "English",

series = "2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1044--1051",

booktitle = "2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings",

}

Ng, RWM, Liu, X & Swietojanski, P 2018, Teacher-Student Training for Text-Independent Speaker Recognition. in 2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings., 8639564, 2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 1044-1051, 2018 IEEE Spoken Language Technology Workshop, SLT 2018, Athens, Greece, 18/12/18. https://doi.org/10.1109/SLT.2018.8639564

Teacher-Student Training for Text-Independent Speaker Recognition. / Ng, Raymond W.M.; Liu, Xuechen; Swietojanski, Pawel.
2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. p. 1044-1051 8639564 (2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Teacher-Student Training for Text-Independent Speaker Recognition

AU - Ng, Raymond W.M.

AU - Liu, Xuechen

AU - Swietojanski, Pawel

PY - 2018/7/2

Y1 - 2018/7/2

N2 - This paper investigates text-independent speaker recognition using neural embedding extractors based on the time-delay neural network. Our primary focus is to explore the teacher-student (TS) training framework for knowledge distillation in a text-independent (TI) speaker recognition task. We report the results on both proprietary and public benchmarks, obtaining competitive results with 88-93% smaller models. Particularly, in clean testing conditions, we find TS training on neural-based TI systems achieved same or better performance than the i-vector based counterparts. Neural embeddings are less prone to short segment issues, and offer better performance particularly in the high-recall setting. They can also provide some additional insights about speakers, such as gender or how difficult a given speaker can be for recognition.

AB - This paper investigates text-independent speaker recognition using neural embedding extractors based on the time-delay neural network. Our primary focus is to explore the teacher-student (TS) training framework for knowledge distillation in a text-independent (TI) speaker recognition task. We report the results on both proprietary and public benchmarks, obtaining competitive results with 88-93% smaller models. Particularly, in clean testing conditions, we find TS training on neural-based TI systems achieved same or better performance than the i-vector based counterparts. Neural embeddings are less prone to short segment issues, and offer better performance particularly in the high-recall setting. They can also provide some additional insights about speakers, such as gender or how difficult a given speaker can be for recognition.

KW - Knowledge Distillation

KW - Speaker Recognition

KW - Teacher-Student training

UR - https://www.scopus.com/pages/publications/85063104193

U2 - 10.1109/SLT.2018.8639564

DO - 10.1109/SLT.2018.8639564

M3 - Conference Proceeding

AN - SCOPUS:85063104193

T3 - 2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings

SP - 1044

EP - 1051

BT - 2018 IEEE Spoken Language Technology Workshop, SLT 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE Spoken Language Technology Workshop, SLT 2018

Y2 - 18 December 2018 through 21 December 2018

ER -

Teacher-Student Training for Text-Independent Speaker Recognition

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this