Self-supervised deep language modeling has shown unprecedented success across natural language ta... more Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme consists of masked language modeling combined with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to very large sequence lengths. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and outputs. ProteinBERT obtains state-of-the-art performance on multiple benchmarks covering diverse protein properties (including protein structure, post translational modifications and biophysical attributes), despite using a far smaller model than competing d...
A major bottleneck in our understanding of the molecular underpinnings of life is the assignment ... more A major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging. We conducted the second critical assessment of functional annotation (CAFA), a timed challenge to assess computational methods that automatically assign protein function. We evaluated 126 methods from 56 research groups for their ability to predict biological functions using Gene Ontology and gene-disease associations using Human Phenotype Ontology on a set of 3681 proteins from 18 species. CAFA2 featured expanded analysis compared with CAFA1, with regards to data set size, variety, and assessment metrics. To review progress in the field, the analysis ...
The amount of sequenced genomes and proteins is growing at an unprecedented pace. Unfortunately, ... more The amount of sequenced genomes and proteins is growing at an unprecedented pace. Unfortunately, manual curation and functional knowledge lag behind. Homologous inference often fails at labeling proteins with diverse functions and broad classes. Thus, identifying high-level protein functionality remains challenging. We hypothesize that a universal feature engineering approach can yield classification of high-level functions and unified properties when combined with machine learning approaches, without requiring external databases or alignment. In this study, we present a novel bioinformatics toolkit called ProFET (Protein Feature Engineering Toolkit). ProFET extracts hundreds of features covering the elementary biophysical and sequence derived attributes. Most features capture statistically informative patterns. In addition, different representations of sequences and the amino acids alphabet provide a compact, compressed set of features. The results from ProFET were incorporated in data analysis pipelines, implemented in python and adapted for multi-genome scale analysis. ProFET was applied on 17 established and novel protein benchmark datasets involving classification for a variety of binary and multi-class tasks. The results show state of the art performance. The extracted features' show excellent biological interpretability. The success of ProFET applies to a wide range of high-level functions such as subcellular localization, structural classes and proteins with unique functional properties (e.g. neuropeptide precursors, thermophilic and nucleic acid binding). ProFET allows easy, universal discovery of new target proteins, as well as understanding the features underlying different high-level protein functions. ProFET source code and the datasets used are freely available at https://github.com/ddofer/ProFET. michall@cc.huji.ac.ilSupplementary information: Supplementary data are available at Bioinformatics online.
Neuropeptides (NPs) are short secreted peptides produced in neurons. NPs act by activating signal... more Neuropeptides (NPs) are short secreted peptides produced in neurons. NPs act by activating signaling cascades governing broad functions such as metabolism, sensation and behavior throughout the animal kingdom. NPs are the products of multistep processing of longer proteins, the NP precursors (NPPs). We present NeuroPID (Neuropeptide Precursor Identifier), an online machine-learning tool that identifies metazoan NPPs. NeuroPID was trained on 1418 NPPs annotated as such by UniProtKB. A large number of sequence-based features were extracted for each sequence with the goal of capturing the biophysical and informational-statistical properties that distinguish NPPs from other proteins. Training several machine-learning models, including support vector machines and ensemble decision trees, led to high accuracy (89-94%) and precision (90-93%) in cross-validation tests. For inputs of thousands of unseen sequences, the tool provides a ranked list of high quality predictions based on the results of four machine-learning classifiers. The output reveals many uncharacterized NPPs and secreted cell modulators that are rich in potential cleavage sites. NeuroPID is a discovery and a prediction tool that can be used to identify NPPs from unannotated transcriptomes and mass spectrometry experiments. NeuroPID predicted sequences are attractive targets for investigating behavior, physiology and cell modulation.
The NeuroPID web tool is available at:
http:// neuropid.cs.huji.ac.il
ClanTox (classifier of animal toxins) was developed for identifying toxin-like candidates from co... more ClanTox (classifier of animal toxins) was developed for identifying toxin-like candidates from complete proteomes. Searching mammalian proteomes for short toxin-like proteins (coined TOLIPs) revealed a number of overlooked secreted short proteins with an abundance of cysteines throughout their sequences. We applied bioinformatics and data-mining methods to infer the function of several top predicted candidates. We focused on cysteine-rich peptides that adopt the fold of the three-finger proteins (TFPs). We identified a cluster of duplicated genes that share a structural similarity with elapid neurotoxins, such as α-bungarotoxin. In the murine proteome, there are about 60 such proteins that belong to the Ly6/uPAR family. These proteins are secreted or anchored to the cell membrane. Ly6/uPAR proteins are associated with a rich repertoire of functions, including binding to receptors and adhesion. Ly6/uPAR proteins modulate cell signaling in the context of brain functions and cells of the innate immune system. We postulate that TOLIPs, as modulators of cell signaling, may be associated with pathologies and cellular imbalance. We show that proteins of the Ly6/uPAR family are associated with cancer diagnosis and malfunction of the immune system.
Self-supervised deep language modeling has shown unprecedented success across natural language ta... more Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme consists of masked language modeling combined with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to very large sequence lengths. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and outputs. ProteinBERT obtains state-of-the-art performance on multiple benchmarks covering diverse protein properties (including protein structure, post translational modifications and biophysical attributes), despite using a far smaller model than competing d...
A major bottleneck in our understanding of the molecular underpinnings of life is the assignment ... more A major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging. We conducted the second critical assessment of functional annotation (CAFA), a timed challenge to assess computational methods that automatically assign protein function. We evaluated 126 methods from 56 research groups for their ability to predict biological functions using Gene Ontology and gene-disease associations using Human Phenotype Ontology on a set of 3681 proteins from 18 species. CAFA2 featured expanded analysis compared with CAFA1, with regards to data set size, variety, and assessment metrics. To review progress in the field, the analysis ...
The amount of sequenced genomes and proteins is growing at an unprecedented pace. Unfortunately, ... more The amount of sequenced genomes and proteins is growing at an unprecedented pace. Unfortunately, manual curation and functional knowledge lag behind. Homologous inference often fails at labeling proteins with diverse functions and broad classes. Thus, identifying high-level protein functionality remains challenging. We hypothesize that a universal feature engineering approach can yield classification of high-level functions and unified properties when combined with machine learning approaches, without requiring external databases or alignment. In this study, we present a novel bioinformatics toolkit called ProFET (Protein Feature Engineering Toolkit). ProFET extracts hundreds of features covering the elementary biophysical and sequence derived attributes. Most features capture statistically informative patterns. In addition, different representations of sequences and the amino acids alphabet provide a compact, compressed set of features. The results from ProFET were incorporated in data analysis pipelines, implemented in python and adapted for multi-genome scale analysis. ProFET was applied on 17 established and novel protein benchmark datasets involving classification for a variety of binary and multi-class tasks. The results show state of the art performance. The extracted features' show excellent biological interpretability. The success of ProFET applies to a wide range of high-level functions such as subcellular localization, structural classes and proteins with unique functional properties (e.g. neuropeptide precursors, thermophilic and nucleic acid binding). ProFET allows easy, universal discovery of new target proteins, as well as understanding the features underlying different high-level protein functions. ProFET source code and the datasets used are freely available at https://github.com/ddofer/ProFET. michall@cc.huji.ac.ilSupplementary information: Supplementary data are available at Bioinformatics online.
Neuropeptides (NPs) are short secreted peptides produced in neurons. NPs act by activating signal... more Neuropeptides (NPs) are short secreted peptides produced in neurons. NPs act by activating signaling cascades governing broad functions such as metabolism, sensation and behavior throughout the animal kingdom. NPs are the products of multistep processing of longer proteins, the NP precursors (NPPs). We present NeuroPID (Neuropeptide Precursor Identifier), an online machine-learning tool that identifies metazoan NPPs. NeuroPID was trained on 1418 NPPs annotated as such by UniProtKB. A large number of sequence-based features were extracted for each sequence with the goal of capturing the biophysical and informational-statistical properties that distinguish NPPs from other proteins. Training several machine-learning models, including support vector machines and ensemble decision trees, led to high accuracy (89-94%) and precision (90-93%) in cross-validation tests. For inputs of thousands of unseen sequences, the tool provides a ranked list of high quality predictions based on the results of four machine-learning classifiers. The output reveals many uncharacterized NPPs and secreted cell modulators that are rich in potential cleavage sites. NeuroPID is a discovery and a prediction tool that can be used to identify NPPs from unannotated transcriptomes and mass spectrometry experiments. NeuroPID predicted sequences are attractive targets for investigating behavior, physiology and cell modulation.
The NeuroPID web tool is available at:
http:// neuropid.cs.huji.ac.il
ClanTox (classifier of animal toxins) was developed for identifying toxin-like candidates from co... more ClanTox (classifier of animal toxins) was developed for identifying toxin-like candidates from complete proteomes. Searching mammalian proteomes for short toxin-like proteins (coined TOLIPs) revealed a number of overlooked secreted short proteins with an abundance of cysteines throughout their sequences. We applied bioinformatics and data-mining methods to infer the function of several top predicted candidates. We focused on cysteine-rich peptides that adopt the fold of the three-finger proteins (TFPs). We identified a cluster of duplicated genes that share a structural similarity with elapid neurotoxins, such as α-bungarotoxin. In the murine proteome, there are about 60 such proteins that belong to the Ly6/uPAR family. These proteins are secreted or anchored to the cell membrane. Ly6/uPAR proteins are associated with a rich repertoire of functions, including binding to receptors and adhesion. Ly6/uPAR proteins modulate cell signaling in the context of brain functions and cells of the innate immune system. We postulate that TOLIPs, as modulators of cell signaling, may be associated with pathologies and cellular imbalance. We show that proteins of the Ly6/uPAR family are associated with cancer diagnosis and malfunction of the immune system.
Uploads
Papers by Dan Ofer
The NeuroPID web tool is available at:
http:// neuropid.cs.huji.ac.il
The NeuroPID web tool is available at:
http:// neuropid.cs.huji.ac.il