Prediction of Epigenetic Protein using Deep Learning and Protein Large Language Models

Abstract

Epigenetic proteins (EPs) are essential in regulating gene activity without altering the DNA sequence. Their role in numerous diseases, including cancer, neurodegenerative, and autoimmune disorders, has rendered them significant targets in drug discovery. It’s important to be able to accurately predict EPs because they play such a key role. Unfortunately, experimental methods for prediction are often costly and take a long time and required domain experts. In this study, we introduced a new deep learning framework that uses protein sequence data to make predictions about EPs. We obtained protein sequence data from Uniprot database and used two powerful pre-trained large language models, ProtBERT and UniREP, to turn protein sequences into rich numerical representations. Then, these embeddings are used as inputs for three different deep learning models: a Multi-Headed Ensemble Residual Convolutional Neural Network (MERCNN), a Bi-Directional Gated Recurrent Unit (Bi-GRU), and an Encoder-Decoder model. We systematically evaluated each model’s performance using both embedding models to determine the best combinations for predicting EP. Our model (MERCNN-EP) achieved an impressive 96.89% accuracy on the training dataset and 91.12% on the testing dataset, which was obtained from the UniProt database to evaluate the generalization ability of the proposed model. These results outperform previously published studies on epigenetic protein prediction, demonstrating the effectiveness and improved predictive capability of the proposed framework. Our results show that the MERCNN architecture works best when combined with ProtBERT embeddings. This research creates a strong and scalable framework for high-capacity EP screening, which could have a big impact on improving targeted therapeutic strategies in precision medicine.