UTR-DynaPro: a CNN–transformer multimodal language model for decoding 5′UTR regulatory mechanisms

The 5′ untranslated region (5′UTR) plays a pivotal role in controlling translation efficiency and protein synthesis. However, existing models often struggle to jointly capture local regulatory motifs and long-range dependencies while effectively integrating multimodal biological features. We present UTR-DynaPro, a multimodal language model that combines a parallel CNN–Transformer architecture with a k-mer–specific mixture-of-experts module and a dynamic fusion mechanism. The CNN branch extracts contiguous motif patterns, while the Transformer branch models hierarchical long-range interactions. To address the complexity of 5’UTR regulation, a dynamic fusion gate is employed to integrate sequence-derived embeddings with key biophysical and structural determinants, including minimum free energy, CDS length, AT ratio, G/C content and upstream open reading frames (uORFs). Across translation efficiency (quantified by mean ribosome loading) and expression level prediction tasks, UTR-DynaPro achieves up to 3.3%, 2.2%, and 2.4% improvements over state-of-the-art methods, respectively. Attention-based motif analysis further identifies both known and novel regulatory elements with consistent performance across cell types, offering a generalizable framework for decoding complex 5′UTR regulation and guiding the design of high-performance regulatory sequences.