Chinese Named Entity Recognition based on Transformer Encoder and BiLSTM

Abstract

Transformer encoder, which is often used for feature extraction, is ineffective in named entity recognition tasks due to the damage of word embedding, the loss of position information and direction information. In this paper, a method of Chinese named entity recognition at character level based on BiLSTM, Transformer encoder and CRF is proposed, which improves the use of position vector in Transformer, splices word embedding and position vector as character representation layer to avoid loss of word embedding information and position information. Extracting context features and incorporating direction information into position vector through BiLSTM. Besides, Transformer encoder is introduced to extract inter-word relationship features, and finally CRF is used to decode globally. Experiments on universal MSRA and Thangka datasets achieves 81.4% F1 value and 88.3% F1 value respectively, and show that the method effectively improves the performance of Chinese named entity recognition.