Adversarial Transfer Learning for Biomedical Named Entity Recognition
Authors: 
Yue Wu, Jiana Meng, Dandan Zhao, Di Zhao, Wen SuAuthors Info & Claims
Yue Wu, Jiana Meng, Dandan Zhao, Di Zhao, Wen SuAuthors Info & ClaimsPages 183 - 189
Abstract
Biomedical Named Entity Recognition (BioNER) is one of the basic tasks of biomedical text mining. In reality, the labeled biomedical data is relatively limited, there is a lack of large enough training data to train a strong model, and manual labeling is expensive. To solve this problem, this paper proposes a network model based on deep transfer learning to improve the performance of entity recognition by learning text knowledge in the general domain (source resource) and migrating to the biomedical domain (target resource). In addition, in order to solve the problem of model training bias caused by the imbalance of data volume between the two domains and the large difference between the data, we construct an adversarial neural network model to extract domain-independent features to effectively alleviate the problem of negative migration. Without adding any artificial features, the proposed model is able to learn transferable feature representations better than existing methods and achieve better results on two biomedical field datasets.
References
[1]
Song B, Li F, Liu Y, Deep learning methods for biomedical named entity recognition: a survey and qualitative comparison. Briefings in Bioinformatics, 2021, 22(6): bbab282.
[2]
Zhou G, Zhang J, Su J, Recognizing names in biomedical texts: a machine learning approach. Bioinformatics, 2004, 20(7): 1178-1190.
[3]
Akhondi S A, Hettne K M, Van Der Horst E, Recognition of chemical entities: combining dictionary-based and grammar-based approaches. Journal of cheminformatics, 2015, 7(1): 1-11.
[4]
Leser U, Hakenberg J. What makes a gene name? Named entity recognition in the biomedical literature. Briefings in bioinformatics, 2005, 6(4): 357-369.
[5]
Campos D, Matos S, Oliveira J L. Biomedical named entity recognition: a survey of machine-learning tools. Theory and Applications for Advanced Text Mining, 2012, 11: 175-195.
[6]
Bikel D M, Schwartz R, Weischedel R M. An algorithm that learns what's in a name. Machine learning, 1999, 34(1): 211-231.
[7]
Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods. Cambridge university press, 2000.
[8]
Lafferty J, McCallum A, Pereira F C N. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001.
[9]
Zhu Q, Li X, Conesa A, GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics, 2018, 34(9): 1547-1554.
[10]
Korvigo I, Holmatov M, Zaikovskii A, Putting hands to rest: efficient deep CNN-RNN architecture for chemical named entity recognition with no hand-crafted rules. Journal of cheminformatics, 2018, 10(1): 1-10.
[11]
Dang T H, Le H Q, Nguyen T M, D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics, 2018, 34(20): 3539-3546.
[12]
Li L, Jiang Y. Biomedical named entity recognition based on the two channels and sentence-level reading control conditioned LSTM-CRF//2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2017: 380-385.
[13]
Mikolov T, Chen K, Corrado G, Efficient Estimation of Word Representations in Vector Space. Computer Science, 2013.
[14]
Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation//Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014: 1532-1543.
[15]
Devlin J, Chang M W, Lee K, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. 2018.
[16]
Wang X, Zhang Y, Ren X, Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics, 2019, 35(10): 1745-1752.
[17]
Hu J, Zhao H, Guo D, A Label-Aware Autoregressive Framework for Cross-Domain NER//Findings of the Association for Computational Linguistics: NAACL 2022. 2022: 2222-2232.
[18]
Huang X, Dong L, Boschee E, Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation//Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). 2019: 515-527.
[19]
Giorgi J M, Bader G D. Transfer learning for biomedical named entity recognition with neural networks. Bioinformatics, 2018, 34(23): 4087-4094.
[20]
Zhou J T, Zhang H, Jin D, Dual adversarial neural transfer for low-resource named entity recognition//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 3461-3471.
[21]
Cao P, Chen Y, Liu K, Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism//Proceedings of the 2018 conference on empirical methods in natural language processing. 2018: 182-192.
[22]
Yadav S, Ekbal A, Saha S, A unified multi-task adversarial learning framework for pharmacovigilance mining//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5234-5245.
[23]
Peng Q, Zheng C, Cai Y, Unsupervised cross-domain named entity recognition using entity-aware adversarial training. Neural Networks, 2021, 138: 68-77.
[24]
Leaman R, Lu Z. TaggerOne: joint named entity recognition and normalization with semi-Markov Models. Bioinformatics, 2016, 32(18): 2839-2846.
[25]
Habibi M, Weber L, Neves M, Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics, 2017, 33(14): i37-i48.
[26]
Yoon W, So C H, Lee J, Collabonet: collaboration of deep neural networks for biomedical named entity recognition. BMC bioinformatics, 2019, 20(10): 55-65.
[27]
Liu S, Sun Y, Li B, HAMNER: headword amplified multi-span distantly supervised method for domain specific named entity recognition//Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(05): 8401-8408.
[28]
Cho M, Ha J, Park C, Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition. Journal of biomedical informatics, 2020, 103: 103381.
[29]
Ma J, Ballesteros M, Doss S, Label Semantics for Few Shot Named Entity Recognition//Proceedings of the Association for Computational Linguistics. 2022:1956-1971.
[30]
Taiki Watanabe, Tomoya Ichikawa, Akihiro Tamura, Auxiliary Learning for Named Entity Recognition with Multiple Auxiliary Biomedical Training Data//In Proceedings of the 21st Workshop on Biomedical Language Processing. 2022:130–139.
Index Terms
- Adversarial Transfer Learning for Biomedical Named Entity Recognition
Index terms have been assigned to the content through auto-classification.
Recommendations
Learning multilingual named entity recognition from Wikipedia
We automatically create enormous, free and multilingual silver-standard training annotations for named entity recognition (ner) by exploiting the text and structure of Wikipedia. Most ner systems rely on statistical models of annotated data to identify ...
Unsupervised biomedical named entity recognition
Display Omitted BM-NER is approached by an unsupervised stepwise method.Noun phrase chunking is a good approximation of boundary detection.Distributional semantics works well in classifying entities.The system performs well on clinical and biological ...
Comments
Information & Contributors
Information
Published In
March 2023
212 pages
ISBN:9781450398398
DOI:10.1145/3594409
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 July 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIAI 2023
ICIAI 2023: 2023 the 7th International Conference on Innovation in Artificial Intelligence
March 3 - 5, 2023
Harbin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 31Total Downloads
- Downloads (Last 12 months)24
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format