Training of multi-class linear Classifier with BFGS method

Xiaobo Jin*, Junwei Yu, Feng Chen, Pengfei Zhu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Linear Classifier is a basis model in the Classification learning field. BGFS is a classical unconstraint optimization technique. In this work, we applied BGFS methods into two representative linear Classifiers including logistic regression Classifier and SVM. The classical multi-class logistic regression Classifier uses Newton method to optimize its loss function and suffers the expensive computations and the un-stable iteration process. For SVM, we converted it into an unconstraint optimization problem in the primal form instead of the time-consuming SMO algorithm. The results on 25 datasets show that for logistic regression model BFGS achieves better Classification accuracy than the Newton method. Moreover, BFGS has the lower time complexity, in contrast with Newton method. Finally, we also observe that SVM with BFGS method demonstrates the comparable performance with the SVM Classifier but with the lower running time.

Original languageEnglish
Pages (from-to)251-258
Number of pages8
JournalJournal of Computational Information Systems
Issue number1
Publication statusPublished - 1 Jan 2014
Externally publishedYes


  • BFGS
  • Linear classifier
  • Logistic regression
  • Newton method
  • SMO
  • SVM

Cite this