Evaluating the Reliability of Machine Learning Predictors in m⁶A-SNP Association Analysis: A Comparative Study Using m⁶A-QTL Data

Introduction: N6-Methyladenosine (m⁶A) plays a crucial role in determining the fate of RNA after transcription. Understanding the downstream functions of individual m⁶A sites is of critical interest in epitranscriptomics. In published studies, two main approaches have been used to decipher the specific impact of m⁶A sites on gene expression and disease/traits: the m⁶A quantitative trait loci (m⁶A-QTL) and in-silico mutation prediction by Machine Learning (ML) models. However, earlier works still lack independent validation for the performance of ML-based methods. Methods: In this study, we use m⁶A-QTL as ground truth to evaluate the outcomes of in-silico mutation models. We benchmark both the newly trained machine learning models using genomic or sequence features and the existing model inference results published in in-silico mutation-dependent databases against m⁶A-QTL. Results: We found that the consistency between in-silico mutation and m⁶A-QTL is weak, regardless of the ML algorithms and predictive features used. This trend was also similar across multiple published databases based on in-silico mutation, including RMDisease2, m⁶AVar, and RMVar. Conclusion: These results emphasize the importance of critical empirical evaluations for ML models in future SNP-m⁶A association studies and suggest the need for more high-quality m⁶A-QTL experiments to guide model development.