TY - GEN
T1 - Decoupled Learning for Long-Tailed Oracle Character Recognition
AU - Li, Jing
AU - Dong, Bin
AU - Wang, Qiu Feng
AU - Ding, Lei
AU - Zhang, Rui
AU - Huang, Kaizhu
N1 - Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2023
Y1 - 2023
N2 - Oracle character recognition has recently made significant progress with the success of deep neural networks (DNNs), but it is far from being solved. Most works do not consider the long-tailed distribution issue in oracle character recognition, resulting in a biased DNN towards head classes. To overcome this issue, we propose a two-stage decoupled learning method to train an unbiased DNN model for long-tailed oracle character recognition. In the first stage, we optimize the DNN under instance-balanced sampling, obtaining a robust backbone but biased classifier. In the second stage, we propose two strategies to refine the classifier under class-balanced sampling. Specifically, we add a learnable weight scaling module which can adjust the classifier to respect tail classes; meanwhile, we integrate the KL-divergence loss to maintain attention to head classes through knowledge distillation from the first stage. Coupling these two designs enables us to train an unbiased DNN model in oracle character recognition. Our proposed method achieves new state-of-the-art performance on three benchmark datasets, including OBC306, Oracle-AYNU and Oracle-20K.
AB - Oracle character recognition has recently made significant progress with the success of deep neural networks (DNNs), but it is far from being solved. Most works do not consider the long-tailed distribution issue in oracle character recognition, resulting in a biased DNN towards head classes. To overcome this issue, we propose a two-stage decoupled learning method to train an unbiased DNN model for long-tailed oracle character recognition. In the first stage, we optimize the DNN under instance-balanced sampling, obtaining a robust backbone but biased classifier. In the second stage, we propose two strategies to refine the classifier under class-balanced sampling. Specifically, we add a learnable weight scaling module which can adjust the classifier to respect tail classes; meanwhile, we integrate the KL-divergence loss to maintain attention to head classes through knowledge distillation from the first stage. Coupling these two designs enables us to train an unbiased DNN model in oracle character recognition. Our proposed method achieves new state-of-the-art performance on three benchmark datasets, including OBC306, Oracle-AYNU and Oracle-20K.
KW - Decoupled learning
KW - Knowledge distillation
KW - Long tail
KW - Oracle character recognition
UR - http://www.scopus.com/inward/record.url?scp=85173584986&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-41685-9_11
DO - 10.1007/978-3-031-41685-9_11
M3 - Conference Proceeding
AN - SCOPUS:85173584986
SN - 9783031416842
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 165
EP - 181
BT - Document Analysis and Recognition – ICDAR 2023 - 17th International Conference, Proceedings
A2 - Fink, Gernot A.
A2 - Jain, Rajiv
A2 - Kise, Koichi
A2 - Zanibbi, Richard
PB - Springer Science and Business Media Deutschland GmbH
T2 - 17th International Conference on Document Analysis and Recognition, ICDAR 2023
Y2 - 21 August 2023 through 26 August 2023
ER -