A deep neural network model for Chinese toponym matching with geographic pre-training model

ABSTRACT

Multiple tasks within the field of geographical information retrieval and geographical information sciences necessitate toponym matching, which involves the challenge of aligning toponyms that share a common referent. The multiple string similarity approaches struggle when confronted with the complexities associated with unofficial and/or historical variants of identical toponyms. Also, current state-of-the-art approaches/tools to supervised machine learning rely on labeled samples, and they do not adequately address the intricacies of character replacements either from transliterations or historical shifts in linguistic and cultural norms. To address these issues, this paper proposes a novel matching approach that leverages a deep neural network model empowered by geographic language representation model, known as GeoBERT, which stands for geographic Bidirectional Encoder Representations from Transformers (BERT). This model harnesses the groundbreaking capabilities of the GeoBERT framework by extending a generalized Enhanced Sequential Inference Model architecture and integrating multiple features to enhance the accuracy and robustness of the toponym matching. We present a comprehensive evaluation of the proposed method’s performance using three extensive datasets. The findings clearly illustrate that our approach outperforms the individual similarity metrics used in previous studies.

KEYWORDS:

• Toponym matching
• deep learning
• pre-training model
• geographic information retrieval
• natural language processing

1. Introduction

Address serves as a pivotal foundational asset for the intelligent development of urban areas. By means of thorough exploration and application of address data, it facilitates urban planners in achieving a more precise understanding of the city’s developmental trajectory and requirements. Consequently, this knowledge empowers them to optimize the spatial arrangement of the city and effectively distribute public services, thereby enhancing overall urban efficiency (Alsudais, Alotaibi, and Alomary 2022; Hu et al. 2022a; Ma 2022; Mauro, Ardissono, and Lucenteforte 2020; Qiu et al. 2022a; Santos, Murrieta-Flores, and Martins 2018a; 2018b; Zhang et al. 2022). Therefore, in various applications closely related to Geographic Information Retrieval (GIR) and Geographic Information System (GIS), place name matching has become an inherent challenge (Hu et al. 2022b; 2022c; Hu et al. 2023a; 2023b; Qiu et al. 2022b; Qiu et al. 2023). These include the merging of digital gazetteers or datasets related to points of interest, as well as the parsing of addresses in services for geocoding and map search, and toponym resolution concerning textual content, digitized maps, and digital library resources (Berkhin et al. 2015; Comber and Arribas-Bel 2019; Li et al. 2020a; 2020b; Moura, Davis Jr, and Fonseca 2017; Santos, Anastácio, and Martins 2015). Various tasks within the realm of GIR and the broader field of GIS necessitate the undertaking of toponym matching. This particular endeavor involves tackling the challenge of aligning toponyms that possess a shared referent, such as the instances where both ‘Wuhan’ and ‘Jiangcheng’ are employed to denote the city of Wuhan, China. Within the aforementioned array of tasks, toponym matching assumes a crucial role in mitigating ambiguity challenges inherent to the usage of different names to denote a singular geographical location. However, it is worth noting that alternative techniques are frequently investigated to address instances where identical names are employed for distinct places. Although the primary objective of toponym matching is to recognize various names that refer to the same entity, additional contextual details are frequently analyzed to resolve ambiguity when multiple instances of a particular place name are encountered.

A typical approach for matching toponyms predominantly relied on string matching techniques, approaching the problem by assessing character similarities (Santos, Murrieta-Flores, and Martins 2018a; Winkler 1990). Noteworthy examples include function-based algorithms, full-word matching methods, and fuzzy query approaches (Li et al. 2021; Li et al. 2022; Li, Feng, and Chiu 2023; Moreau, Yvon, and Cappé 2008; Recchia and Louwerse 2013; Varol and Bayrak 2012). Furthermore, some approaches of spatial geometry to explore place name matching techniques. These approaches involve constructing models that incorporate address features such as distance, location, and other spatial geometric characteristics. However, it should be noted that this approach necessitates numerous geometric operations, thereby resulting in relatively lower execution efficiency (Santos, Murrieta-Flores, and Martins 2018a). Presently, the prevailing research trend in the domain of place name matching algorithms predominantly revolves around the utilization of deep learning models.

Recent advancements in the field have brought forth a remarkable method for combining several string similarity measures, namely supervised machine learning techniques like deep learning. This methodology, as proposed by Santos et al. (2018a), has demonstrated its effectiveness in addressing the intricacies of toponym matching. However, it is important to acknowledge that even the combined application of diverse string similarity metrics may fall short in comprehensively tackling certain arduous challenges inherent in this domain. For example, effectively managing toponym modifications that are unofficial and/or historical, which may demonstrate significant differences, presents a notable challenge. Similarly, managing transliterations involving different languages and scripts presents another complex facet. The primary reason for this is the constraints of current string similarity metrics, which rely on common character substrings to establish semantic similarity. As a consequence, these metrics inadequately capture the substitutions commonly observed in transliterations and ignore the development of language use across time. Deep learning has witnessed a surge in its utilization within the domain of GIS. Firstly, deep learning facilitates the automatic modeling of address features, circumventing the need for manual design rules. Consequently, the reliance on predefined heuristics is significantly reduced. Secondly, deep learning excels in extracting semantic information, rendering it well-suited for diverse address structures, particularly those characterized by irregularities. As a result, deep learning techniques prove to be highly effective in addressing the challenges posed by such varied address formats.

This paper proposes GeoBERTTM, an innovative deep learning model with GeoBERT designed specifically for toponym matching within natural language texts in Chinese. The proposed deep learning approach in this manuscript entails a two-step process. Initially, the GeoBERT model is harnessed to facilitate the training and acquisition of word vectors pertaining to address elements, thereby effecting the transformation of input address records into their respective vector representations. Subsequently, the enhanced sequential inference model (ESIM), one of the state-of-the-art deep text-matching approaches, is leveraged to conduct local and global inferences between the compared address records in vector format, determining whether they match. The model leverages the groundbreaking geographic pre-training capabilities of the GeoBERT framework. By extending a generalized recurrent neural network model, our approach integrates multiple features to enhance the accuracy and robustness of the toponym matching process. Significantly, this model integrates innovative improvements to proficiently address the inherent irregularities encountered in texts composed in natural language.

Section 2 provides an extensive overview of fundamental concepts and prior research. It commences with a comprehensive introduction to widely adopted string similarity metrics, followed by an in-depth analysis of prior research addressing the challenge of toponym matching through machine learning and deep learning approaches. In Section 3, the proposed deep learning methodology is explicated, highlighting the intricate details of the neural network architecture employed for assessing the similarity between pairs of toponyms. Section 4 outlines the experimental evaluation protocol, encompassing the dataset utilized, general experimental procedures, and the evaluation metrics employed. Furthermore, the obtained results are presented and thoroughly discussed. Section 5 presents the detailed discussion. Section 6 offers a summary of the most noteworthy findings, along with potential directions for future research.

2. Related work

2.1. Previous reseach on string matching

Most of the previous studies on toponym matching have primarily depended on methodologies that revolve around calculating a similarity measure between the to-be-matched toponyms. Subsequently, a determination is made based on a predefined threshold applied to the resulting similarity score. Traditional approaches to toponym matching center around employing string similarity metrics like the Levenshtein edit distance (Levenshtein 1966) and the Jaro-Winkler metric (Winkler 1990), alongside the subsequent establishment of manual thresholds or tailored classifiers (Sun et al. 2013). These techniques primarily concentrate on assessing textual associations between address records, predominantly through simplistic word-for-word or character-for-character comparisons. Generally, these approaches can be categorized into three distinct groups: methods based on characters (e.g. Levenshtein 1966; Sun et al. 2013; Winkler 1990), methods based on vector space (e.g. Eidoon, Yazdani, and Oroumchian 2008; Li, Wang, and Mei 2010), and hybrid methods (e.g. Santos et al. 2017; Cheng, Liao, and Chen 2022). Character-based techniques primarily function by utilizing edit operations at the character level. Conversely, vector-space methods enable the conversion of strings into vector representations, on which similarity calculations are performed. Hybrid approaches ingeniously amalgamate both strategies to enhance the efficacy of matching names comprised of multiple tokens. Presently, deep learning has emerged as the prevailing method for toponym matching, superseding other alternatives.

Character-based methods involve the evaluation of character edit distances to measure similarity. Among these methods, measure for Levenshtein edit distance (1966) stands out as a widely recognized and esteemed approach. Buckles, Buckley, and Petry (1994) conducted a series of operations, namely stem extraction, alignment, and synonym substitution, on strings, subsequently employing fuzzy retrieval for address string matching at a predetermined precision level. Amir et al. (2009) presented an effective algorithm that addresses natural rearrangement errors by accommodating cases of matching with content consistency despite incorrectly positioned characters. However, these methods typically involve character-level comparisons, which can become inefficient when handling extensive datasets. Consequently, they are often combined with other techniques to enhance their performance.

In the realm of vector-space methodologies, a commonly employed strategy entails computing the cosine similarity metric between representations constructed using character n-grams, which encompass consecutive sequences of n characters. Eidoon, Yazdani, and Oroumchian (2008) developed an ontology model based on vector space representation and achieved ontology similarity estimation by comparing concept vectors. The successful application of this approach to ontology alignment assessment occurred in 2005. In a related study, Li, Wang, and Mei (2010) introduced a matching algorithm named Term Frequency-Inverse Document Frequency (TF-IDF), utilizing the commonly employed feature weight calculation method in the vector space model. To address the challenge of matching and rectifying addresses across distinct standardized address databases, they proposed a matching algorithm called Term-Weighted Dissimilarity. However, it is worth noting that when handling extensive datasets, the vector space model can potentially lead to issues such as the curse of dimensionality or matrix sparsity due to each dimension representing a word or feature.

Hybrid methodologies that integrate both of these approaches have emerged, offering a flexible treatment of word order and position while accommodating slight variations in word tokens. In their study, Santos et al. (2017) conducted experiments utilizing supervised machine learning to effectively merge multiple similarity metrics. This approach offers a notable advantage by eliminating the need for manual tuning of similarity thresholds. Similarly, Koumarelas et al. (2018) proposed a method that enriches address matching through a combination of geocoding, reverse geocoding, and similarity metrics. By augmenting each record with comprehensive address information and selecting suitable similarity metrics for individual address attributes, they aimed to enhance classifier performance in terms of F-values. Cheng, Liao, and Chen (2022) employed a multifaceted approach by integrating various similarity metrics, including string similarity, semantic similarity, and spatial similarity. This comprehensive strategy was pursued to enhance the accuracy of address matching.

While traditional string-based matching techniques have successfully addressed numerous challenges concerning toponymic matching in earlier stages, the current era of big data and the proliferation of heterogeneous data from multiple sources present new complexities. In this context, traditional methods often prove inadequate when confronted with non-standard or intricately structured data. Moreover, they fail to adequately capture toponymic variations caused by phonetic translations or alterations in language and culture over time (Santos et al. 2018a, 2018b; Qiu et al. 2022a; 2022b). Furthermore, although these methods have exhibited a certain level of effectiveness, they necessitate meticulous calibration of the similarity threshold. It has become evident that no single technique universally outperforms others.

2.2. Previous reseach on toponym matching based on machine learning

In recent years, the advancements in machine learning have generated notable breakthroughs in natural language processing (NLP), encompassing various tasks, including address matching. This task relies on fundamental principles within the realm of NLP. The deep learning models that serve as the foundation for address matching comprise Long short-term memory (LSTM) architecture (Cheng, Dong, and Lapata 2016), BERT framework (Devlin et al. 2018), and ESIM approach (Chen et al. 2016). These cutting-edge frameworks have demonstrated significant potential in addressing the intricacies of address matching and hold promise for further advancements in this field.

In order to facilitate embedding processes, the existing deep learning matching models need the extraction of semantic features related to addresses. This involves various techniques, such as utilizing word2vec for vectorizing address text to capture semantic information (Lin et al. 2020), or utilizing codec architectures to obtain the semantic vector representation for every address string (Shan et al. 2020). However, it is important to consider that these approaches may impact the comprehension of address semantics. To address this issue, Zhang et al. (2020) utilized a BERT model to capture contextual information within addresses, enabling effective processing of address features. They further incorporated a conditional random field (CRF) model to model the constraint relationships between labels, facilitating accurate prediction of matching outcomes. Chen et al. (2021) proposed an innovative address matching model, known as Attention-BiLSTM-CNN network (ABLC) with a comparative learning approach. By combining the contextual semantic learning capabilities of bidirectional LSTM (BiLSTM), the feature extraction advantages of convolutional neural network (CNN), and the dynamic weight assignment ability of attention mechanisms, the ABLC model significantly enhances matching performance compared to baseline models. Furthermore, Xu et al. (2022) introduced a deep migration learning approach to enhance the understanding of address semantics. Their method involves pre-training an address corpus and enabling the address semantic model (ASM) to learn unsupervised address context, thereby improving its capability to comprehend address semantics effectively.

As reported by Santos et al. (2018), who introduces an innovative approach for toponym matching, utilizing a deep neural network to discern whether pairs of toponyms correspond or not. The authors conduct an extensive evaluation of this method’s effectiveness, employing a substantial dataset sourced from the GeoNames gazetteer (https://www.geonames.org/). Remarkably, the findings demonstrate that the proposed approach yields notably superior results compared to individual similarity metrics employed in previous studies, as well as surpassing previous supervised machine learning methods for combining multiple metrics.

Supervised machine learning methods, especially those employing ensembles of decision trees, have demonstrated notable performance by effectively combining similarity metrics (Santos et al. 2017). Nevertheless, it is important to acknowledge that the complexity of Chinese addresses surpasses that of their English counterparts in terms of syntax and semantics. Unlike English, which relies on words and includes delimiters between them, Chinese addresses are character-based, lacking such delimiters. Moreover, due to rapid urbanization and unrestricted urban planning practices, address descriptions exhibit significant inconsistencies (Qin et al. 2016). For instance, a study investigating address discrepancies in Beijing identified 35 errors (Li et al. 2018), while Chongqing’s address data and model exhibited three distinct categories of mistakes. These challenges have necessitated the development of innovative approaches, as detailed in this article.

3. The proposed methodology

3.1. Overall framework

The overall framework of the methodology proposed in this paper is presented in Figure 1. This framework encompasses two distinct stages. Firstly, the BERT model undergoes pre-training on vast-scale geo-domain texts to acquire distributed embedding representations for characters (named GeoBERT). Subsequently, the text undergoes representation mapping using a character-to-feature mapping table, which includes Pinyin, Wubi, Radicals, and Strokes. The word2Vec model is then utilized to capture contextual information and generate distributed embedding representations for each feature. In the second stage, the semantic toponym matching training is conducted on ESIM using the feature vectors generated by BERT.

Figure 1. The overall framework of the methodology proposed in this paper, which includes two steps: obtaining vector representations of input toponym and deep learning model for toponym pairs matching. Step 1 trained the vectors of toponym elements derived from the processed toponym corpus using GeoBERT and transformed the labeled toponym dataset from text to vector format, and Step 2 we separated the dataset into three groups for training, development and test, finally we trained the ESIM to evaluate the accuracy of the best performing model.

3.2. Obtaining vector representations of toponym records

As mentioned in Section 1, the toponym pairs found in the annotated toponym dataset are exclusively given in textual format. Given that a toponym comprises a contiguous sequence of toponym elements, a common method for facilitating deep learning computations is to encode each component of the toponym using a distinct vector, thereby converting the toponym into its related vector representation (Figure 2). In this research, a toponym element includes the name of a toponym entity (e.g. ‘Wuhan’), a feature of the toponym model (e.g. ‘District’), or their various combinations.

Figure 2. Creating an address’s vector representation with combining multiple features (e.g. Wubi, Radicals, Pinyin and Strokes).

Word2vec facilitates the encoding of all conceivable toponym elements encompassed within the toponym dataset. Additionally, a fusion of word vectors obtained from word2vec can depict each toponym entry. However, it is crucial to acknowledge that word2vec captures the semantic significance of words according to their conventional contexts of usage. Consequently, it provides static representations of words and relies on training with a meticulously formatted text corpus. Consequently, numerous colloquial expressions may not be adequately covered by these embeddings. In such instances, vernacular terms are often represented by a generic unknown token embedding, leading to a potential loss of the precise semantics associated with the respective words.

In order to tackle this issue, we leveraged BERT, a neural network architecture, to train and obtain vector representations for toponym elements. BERT excels in its bidirectionality as it learns the contextual information of a word by considering its neighboring words (both left and right). To the existing BERT language model, general-domain corpora such as Wikipedia and the Book corpus are utilized. However, when working on domain-specific tasks, it is important to recognize that the target domain dataset’s data distribution may differ from that of the BERT model. Also, we further conducted pre-training on the BERT-base language model, which we refer to as GeoBERT, utilizing a geographic domain corpus. The objective was to capture the contextual significance of toponyms and improve the performance in the toponym matching task within the target domain. To enable the pre-training process for the BERT-base language model, we prepared a corpus consisting of 18,000,000 sentences extracted from social media texts and abstracts obtained from the training dataset. Prior to corpus preparation, we performed sentence segmentation and normalization using the Jieba library (https://github.com/fxsjy/jieba). Subsequently, we generated five different corpora through arbitrary processing for additional pre-training of the BERT model. Every generated document consists of tokens, segment IDs, masked labels, masked label positions, and a flag indicating the randomization of the next segment in the masked language model. GeoBERT provides a dynamic representation for each word by considering its usage within the corresponding sentence. This empowers the model to capture more profound semantic and syntactic implications of words in the target domain, thereby improving its performance in specific tasks within that domain.

Distinct from English, Chinese words are typically formed by characters, many of which can be further broken down into components like radicals. Characters and radicals possess abundant information and have the ability to convey semantic meanings of words. However, existing word embedding methods have not fully harnessed these resources. To address this limitation, multiple pre-trained word embeddings, including Pinyin, Wubi, Radicals, and Strokes, are employed to represent the words encompassed within a toponym. These pre-trained embeddings have undergone prior training and are capable of capturing the semantic essence of a word based on its typical co-occurrence patterns with other words.

Radicals have proven to be instrumental in extracting potential semantic relationships from unstructured textual data (Yin et al. 2016a, 2016b). This allows neural networks to acquire knowledge regarding the distinct boundaries of internal and external entities within the domain of geography by analyzing the structural composition of various Chinese characters. For instance, the character ‘氵’ bearing both semantic and structural resemblance to ‘River,’ exhibits a higher likelihood of being classified as such. To further refine the toponym classification process, we incorporate Wubi features that complement the radical features. Both Radicals and Wubi draw upon the hieroglyphic structure of Chinese characters. Additionally, variations in pronunciation significantly contribute to the semantic expression of these characters, prompting the inclusion of character Pinyin features to enhance the sequence-based attributes (Yu et al. 2017). In the geographic domain, Chinese characters with similar complexities in stroke formation tend to share common category labels (Cao et al. 2018; Su and Lee 2017). In this study, we adopt the positional embedding concept from BERT to expand the stroke count corresponding to each input sequence character. Subsequently, these features are employed alongside the word2vec model to train and obtain the pertinent characteristics of the text. Finally, we concatenate the acquired features belonging to the four aforementioned categories to construct the ultimate vector for the embedding layer.

3.3. Deep semantic matching of toponym pairs

Following the acquisition of vector representations for pairs of toponyms, for semantic toponym matching, we utilized the ESIM (Chen et al. 2016). The ESIM is a well-established deep learning model commonly employed for text matching using interaction-based methodologies (Fan et al. 2017; Lin et al. 2020). In our study, we applied the ESIM to perform local inference between toponym pairs and subsequently amalgamate this localized inference to generate an overall prediction. Throughout model training, this approach comprehensively considers interactions between related toponym items and their settings in the two toponym records under comparison. An overview the implementation of the ESIM can be found in Figure 3. Moreover, a comprehensive elucidation of each layer within the ESIM is provided in the subsequent section.

Figure 3. An overview of the ESIM (Chen et al. 2016). The ESIM includes four layers: input encoding layer for encoding the input toponym vectors and extracting higher-level representations for toponym records, local inference modeling layer for make local inference of a toponym pair, inference composition layer for making global inference between two compared toponym records, and prediction layer for predictive results of toponym pairs using MLP.

3.3.1. Input encoding layer

Let us consider that S_a (consisting of l_a elements), and S_b (consisting of l_b elements), are two toponyms from the labeled toponym dataset. In order to facilitate further analysis, we converted S_a and S_b into corresponding toponym vectors, referred to as a_la and b_lb, respectively.

Toponym with text format, being a distinctive form of sequential data, necessitates appropriate modeling to uncover semantic associations and interdependencies among its constituent address elements during the toponym parsing process. While an LSTM model (Hochreiter and Schmidhuber 1997) can capture geographic element dependencies following the reading order of toponym text, it fails to consider the semantic associations among toponym elements in the reverse reading order. To address this limitation, this study adopts a BiLSTM network model to extract potential semantic associations within address text. The BiLSTM network model comprises two LSTM layers, establishing connections with both the input and output layers. By training the model on input toponym character vectors using forward and backward sequences, it utilizes hidden states and gating units to capture contextual dependencies. Afterwards, the algorithm combines and calculates the ultimate hidden states by merging the outputs of the forward and backward LSTM neural networks in order to produce a fresh encoding vector $\overline{a_i}$ or $\overline{b_j}$, yielding character vectors arranged in both forward and backward sequences. Following this process, the score for each character in each label is obtained through concatenation and arithmetic operations.

3.3.2. Local inference modeling layer

Within this layer, we leverage a modified decomposable attention model to conduct local inference on a pair of toponyms, encompassing three pivotal stages. We perform a multiplication operation between a_i^T and b_j^T, resulting in an unnormalized attention weight matrix e_ij that captures the local inference between two compared toponym records. Subsequently, we normalize the attention weights through the softmax function and multiply them with b_j to generate the output represented by a_i. A parallel procedure is implemented to derive b_j by applying the reverse process. Lastly, the local inference of S_a with respect to S_b is aggregated by concatenating $\overline{a}$, $\tilde{a}$, their difference, and their dot product, resulting in m_a. Similarly, m_b is obtained by following the reverse process.

3.3.3. Inference composition layer

Within this layer, we leverage the prior local inferences of two compared toponym records to facilitate global inference. This process involves two crucial steps. Firstly, the BiLSTM model is used to derive higher representations of m_a and m_b, resulting in v_a,i and v_b,j as their respective outputs. Secondly, the v_a,i and v_b,j vectors are subjected to both maximum and average pooling procedures. The outcomes of these pooling methods are concatenated together, effectively summarizing the local inferences. Ultimately, a final vector v_final of fixed length is obtained, encapsulating the consolidated information from the pooling step.

3.3.4. Prediction layer

In this layer, a MLP is employed to generate predictive outcomes for toponym pairs, where the labels are assigned as either 0 or 1. The MLP utilized in this study comprises three fully connected layers, each equipped with activation functions such as ReLU, Tanh, and Softmax.

4. Experimental results, and discussion

In this section, we present the dataset employed to bolster the experimental analyses, outline the experimental setup implemented, and thoroughly discuss the resultant findings. The dataset plays a pivotal role in substantiating the validity and efficacy of our proposed methodologies, thereby facilitating comprehensive evaluations. By describing the experimental setup, we aim to provide readers with a clear understanding of the systematic procedures and parameters adopted throughout the study. Finally, the obtained results are critically analyzed and interpreted, shedding light on the implications and significance of our research findings.

4.1. Dataset

Our experiments rely on three datasets of 100,000 pairs of toponyms. The first dataset (named Alidataset) is from the CCKS2021 Chinese Address Correlation Academic Review Task (https://tianchi.aliyun.com/competition/entrance/531901/information), provided by Alibaba Dharma Institute. The dataset employed in this study comprises a collection of standardized addresses, addresses requiring matching, and corresponding labels. Notably, the address pairs within this dataset deviate from manually constructed conventional address pairs in several key aspects. Specifically, the address text exhibits variations, wherein no predetermined rewritten word list is implemented to construct heteronymous same-location addresses. Furthermore, the address specifications may not precisely align across different localities. Consequently, this dataset poses elevated expectations for the place name matching model in terms of its generalizability. By harnessing this unique dataset, subsequent experiments can intricately assess the model’s performance while effectively verifying its applicability in real-world scenarios. The utilization of this dataset enables comprehensive evaluations that encompass a wide range of address variations and challenges, thereby enriching the model’s overall robustness and practical feasibility.

The second original dataset is available from the Shenzhen Address Database (named SZdataset) (doi: 10.5281/zenodo.3477007) (Lin et al. 2020). The corpus utilized in this study encompasses an extensive collection of Chinese address records, totaling 57,253,694 entries, which were updated throughout in 2018. This comprehensive dataset effectively covers nearly all address data pertaining to the region of Shenzhen, China. To ensure the reliability and accuracy of the corpus, a series of rigorous data cleaning procedures were implemented. The data cleaning process encompassed three fundamental steps. Firstly, any duplicate address records were identified and subsequently eliminated from the corpus. Secondly, the removal of extraneous elements such as omitted letters and unique symbols (e.g. ‘.’,’`’, and ‘/’) was conducted to streamline the dataset further. Lastly, efforts were made to rectify instances where characters within the address records were incorrectly written. As a result of these meticulous data cleaning procedures, an exemplary dataset consisting of 84,474 address pairs and their corresponding labels was constructed.

We use the following steps to create a manually annotated dataset named HBdataset (see Table 1). We crawl the Hubei POI data from the web, and then pre-process these data. The detailed construction process of this dataset was as follows.

Table 1. The labeled address dataset examples of several data rows from HBdataset.

Toponym1	Toponym2	label
湖北省随州市曾都区东城街道交通大道285–8老双龙广场[285–8 Old Shuanglong Plaza, Dongcheng Street, Zengdu District, Suizhou City, Hubei Province]	湖北省随州市曾都区东城街道交通大道老双龙广场[Old Shuanglong Square, Dongcheng Street, Zengdu District, Suizhou City, Hubei Province]	1
湖北省宜昌市点军区点军街道夷桥路[Yiqiao Road, Doujun Street, Doujun District, Yichang City, Hubei Province]	湖北省黄冈市黄梅县濯港镇石仲寨[Shizhongzhai, Maughang Town, Huangmei County, Huanggang City, Hubei Province]	0

Step 1: Following the web data crawling process, it becomes essential to perform thorough data cleaning, which primarily encompasses two critical stages: noise processing and removal of excessively short address data. Noise processing involves the elimination of irrelevant characters and special symbols, such as spaces, ‘@’, ‘#’, and external links that do not contribute to the intended purpose of the data. This step effectively filters out extraneous noise present in the dataset. Additionally, eliminating excessively short address data entails evaluating the string length and establishing a predetermined threshold value to discard data entries that fall below this threshold. The address data obtained subsequent to the data cleaning procedure retains its primary characteristics, thereby mitigating the adverse impact of invalid information on the matching outcomes.

Step 2: A crucial aspect that necessitates attention involves the conversion of traditional Chinese characters within the crawled web address data. As these characters may not be contained within the existing vocabulary, they can potentially introduce semantic errors during the subsequent text vectorization process. To address this concern, the HanziConv library (https://pypi.org/project/hanziconv/) is employed to convert traditional Chinese text into simplified text, ensuring compatibility for subsequent vectorization processes.

Step 3: In order to facilitate the implementation of a supervised address matching algorithm, the generation of appropriately labeled training and test datasets becomes imperative. These datasets are composed of destination addresses (address1), addresses requiring matching (address2), and corresponding matching result labels. This systematic process ensures the availability of reliable and annotated data for algorithmic training and evaluation purposes.

Tables 2–4 show various statistical attributes pertinent to the annotated toponym dataset. The length difference metric quantifies the disparity in character count between pairs of toponyms. The Levenshtein distance (Levenshtein 1966) acts as a metric for quantifying the similarity between two toponym strings. A smaller distance value indicates a higher level of resemblance between pairs of toponyms. Furthermore, the Jaccard similarity coefficient (Jaccard 1908) is another vital measure for evaluating toponym similarity. A larger Jaccard similarity coefficient indicates fewer disparities between two toponym records. Also, following the metric proposed by Santos et al. (2018), in Figures 4–6, the distribution of toponym length is exhibited, measured in terms of both character count and word count. Moreover, Figure 7 portrays the distribution of the disparity in character count between matching and non-matching pairs of toponyms, which serves as an indicator of the level of difficulty encountered in the toponym matching task. Notably, pairs with a substantial difference in character count are expected to present higher challenges in classification.

Figure 4. The distribution of toponym lengths within the Alidataset.

Figure 5. The distribution of toponym lengths within the SZdataset.

Figure 6. The distribution of toponym lengths within the HBdataset.

Figure 7. Discrepancies observed in the character counts between matching and non-matching pairs across the three datasets.

Table 2. Statistical characteristics of the labeled address dataset from Alidataset.

Attribute	Value
Number of toponym pairs	2000
Number of matching toponym pairs	1000
Number of pairs with matching toponyms that are completely dissimilar	1000
Number of pairs with nonmatching toponyms that are completely dissimilar	1000
Average difference in number of characters per toponym pair	8.26
Average number of word tokens per toponym	13.64
Average number of characters per toponym	18.08
Number of characters in largest toponym	66
Average Levenshtein distance for all the toponym pairs	18.77
Average Levenshtein distance for matching toponym pairs	10.09
Average Levenshtein distance for unmatched toponym pairs	21.23
Average Jaccard similarity coefficient for all the toponym pairs	0.33
Average Jaccard similarity coefficient for matching toponym pairs	0.61
Average Jaccard similarity coefficient for unmatched toponym pairs	0.22

Table 3. Statistical characteristics of the labeled address dataset from SZdataset.

Attribute	Value
Number of toponym pairs	2000
Number of matching toponym pairs	1000
Number of pairs with matching toponyms that are completely dissimilar	1000
Number of pairs with nonmatching toponyms that are completely dissimilar	1000
Average difference in number of characters per toponym pair	6.55
Average number of word tokens per toponym	18.63
Average number of characters per toponym	24.51
Number of characters in largest toponym	46
Average Levenshtein distance for all the toponym pairs	9.77
Average Levenshtein distance for matching toponym pairs	5.33
Average Levenshtein distance for unmatched toponym pairs	14.11
Average Jaccard similarity coefficient for all the toponym pairs	0.38
Average Jaccard similarity coefficient for matching toponym pairs	0.61
Average Jaccard similarity coefficient for unmatched toponym pairs	0.27

Table 4. Statistical characteristics of the labeled address dataset from HBdataset.

Attribute	Value
Number of toponym pairs	16000
Number of matching toponym pairs	5030
Number of pairs with matching toponyms that are completely dissimilar	5030
Number of pairs with nonmatching toponyms that are completely dissimilar	970
Average difference in number of characters per toponym pair	5.82
Average number of word tokens per toponym	13
Average number of characters per toponym	18.3
Number of characters in largest toponym	48
Average Levenshtein distance for all the toponym pairs	9.17
Average Levenshtein distance for matching toponym pairs	5.46
Average Levenshtein distance for unmatched toponym pairs	15.66
Average Jaccard similarity coefficient for all the toponym pairs	0.49
Average Jaccard similarity coefficient for matching toponym pairs	0.71
Average Jaccard similarity coefficient for unmatched toponym pairs	0.32

4.2. Baselines

In order to assess the performance of our proposed model in a rigorous manner, we conducted an empirical comparison with models presented in ten prior approaches, thus establishing a comprehensive benchmark. To ensure fair comparisons, we utilized the publicly accessible source code provided by the aforementioned research endeavors. Moreover, we adopted the default parameter configurations as reported in those papers, ensuring consistency in the evaluation process. The comparison models employed in this study are succinctly summarized as follows:

• ABCNN uses a general Attention Based Convolutional Neural Network to determine if it matches or not (Yin et al. 2016).

• AlBERT, as proposed by Lan et al. (2019), introduces two innovative techniques for parameter reduction in BERT (Devlin et al. 2018). These techniques aim to mitigate memory consumption while simultaneously enhancing the efficiency of training procedures.

• BERT, is a novel language representation model introduced by Devlin et al. (2018). BERT effectively captures contextual information from both preceding and subsequent words in a sequence, thereby enabling enhanced understanding and representation of textual data.

• BIMPM represents a bilateral multi-perspective matching model based on the ‘matching-aggregation’ framework (Wang, Hamza, and Florian 2017).

• Decomposable Attention is a straightforward attention-based methodology for natural language inference that exhibits inherent parallelizability (Parikh et al. 2016).

• SiaGRU model is a variant of a bi-directional LSTM network for labeling data consisting of variable-length sequences (Huang et al. 2013).

• RoBERT, a refined iteration of the BERT model, demonstrates superior performance compared to all subsequent post-BERT methods, as highlighted by Liu et al. (2019).

• XLNet, as proposed by Yang et al. (2019), introduces a generalized autoregressive pretraining method that offers two key advancements in language modeling.

• DistillBERT, as presented by Sanh et al. (2019), offers a proficient approach to pre-training a compact and versatile language representation model. This model can subsequently be fine-tuned to achieve commendable performance across a diverse array of tasks, akin to its larger counterparts.

• ESIM is a model that integrates the application of BiLSTM and attention mechanisms (Chen et al. 2016).

4.3. Evaluation metrics

Within this study, we have incorporated accuracy, precision (P), recall (R) and F1 as evaluation metrics to validate the performance of the proposed model. Accuracy measures number of correctly matched samples as a percentage of the total. Precision calculates the percentage of correctly identified matched names (noted as True Positives, TP) among all the matched names predicted by the model, which combines both TP and False Positives (FP). Recall measures the percentage of correctly matched ones amongst all ground truth, which is the combination of TP and False Negatives (FN). F1-score is the harmonic mean of precision and recall, providing a comprehensive metric to evaluate model performance. Additionally, we have conducted a comprehensive analysis of cases wherein the method generated erroneous results, complementing the aforementioned metrics (Santos et al. 2018a).

These metrics afford a comprehensive evaluation framework, enabling a thorough assessment of the proposed method’s performance and shedding light on the specific areas where improvements are warranted. (1) $\mathrm{Accuracy}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$(1) (2) $\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$(2) (3) $\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$(3) (4) ${\mathrm{F}}_1={\displaystyle \frac{2{}^{\ast }\mathrm{Precisio}{\mathrm{n}}^{\ast }\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$(4)

To enhance the reliability of our experiments, we employed a twofold cross-validation methodology using the mentioned datasets. This method entailed dividing the toponym pairs into two distinct subsets, ensuring an equitable inclusion of matching and nonmatching pairs. The aim of this division was to effectively evaluate the performance of our proposed method while mitigating potential biases and ensuring generalizability.

4.4. Experiment configuration

In this study, the hyperparameters for the proposed model are detailed in Table 5. In order to investigate the most effective blend of hyperparameters for the proposed model, a systematic tuning procedure was conducted. This involved adjusting specific hyperparameters within their feasible value ranges (Bergstra and Bengio 2012). Following previous studies (Chen et al. 2016), the selection of these hyperparameters was based on their significant impact on the predictive accuracy of deep learning models in various scenarios. By evaluating the performance of different hyperparameter settings on the development set, we aimed to identify the most suitable configuration that maximizes the model’s predictive accuracy and generalization capability.

Table 5. Main hyperparameter settings for the proposed model in this paper.

Hyperparameter	Parameter description	Parameter setting
Learning rate	The interval for updating model parameters during each training epoch ranges from 0 to 1	0.0005 (Tuned and analysed in Section 4.6)
Dropout probability	The dropout technique involves randomly dropping out hidden nodes during the training process, and the probability of retaining each hidden node typically falls within the range of 0–1	0.2
Number of hidden nodes	The number of nodes in the hidden layers	300 (Tuned and analysed in Section 4.6)
Epoch	The number of training epochs	50 (Tuned and analysed in Section 4.6)
Batch size	The size of training samples employed in each epoch typically falls within the range of 10–100	50 (Tuned and analysed in Section 4.6)

4.5. The obtained results

To verify the proposed approach on the three toponym datasets, a series of initial experiments were carried out. These experiments effectively compare the performance of (1) distinct string similarity metrics, (2) supervised learning techniques that integrate these various string similarity metrics, and (3) the deep learning method. In addition to accuracy, precision, recall, and F1, and, we also provide the average processing time for applying each method to sets of 1000 records.

Table 6 presents the experimental results obtained from the Ali dataset and compares them with ten other deep learning methods discussed in Section 4.2, namely ABCNN, AlBERT, BERT, BIMPM, Decomposable Attention, SiaGRU, RoBERT, XLNet, Distillbert, and ESIM. The results, as shown in Table 6, indicate that our proposed method outperforms all other methods across all four metrics, achieving accuracy, precision, recall, and F1 values of 0.815, 0.945, 0.889, and 0.916, respectively. Furthermore, we also included the average runtime for each method on 1000 records, which amounts to 0.2978s for our proposed model. In comparison, the model employing ESIM achieves accuracy, precision, recall, and F1 values of 0.6802, 0.7754, 0.6802, and 0.7247, respectively. This performance difference can primarily be attributed to the fact that our proposed model is trained with word embedding representation models that encompass various word forms like Pinyin, Wubi, Radicals, and Strokes. As a result, it effectively captures the semantic features of words.

Table 6. Experimental results obtained with the different methods under consideration on the Ali dataset.

Model	Accuracy	Precision	Recall	F1	Time (s)
ABCNN	0.480	0.784	0.553	0.649	0.1082s
AlBERT	0.683	0.901	0.739	0.812	0.0360s
BERT	0.635	0.713	0.853	0.776	0.0355s
BIMPM	0.613	0.889	0.663	0.760	0.0470s
Decomposable Attention	0.522	0.889	0.558	0.686	0.0320s
SiaGRU	0.512	0.904	0.542	0.677	0.0106s
RoBERT	0.590	0.936	0.615	0.742	0.0353s
XLNet	0.744	0.725	0.876	0.793	0.0559s
Distillbert	0.500	0.799	0.499	0.615	0.0213s
ESIM	0.680	0.775	0.680	0.725	0.1564s
GeoBERTTM	0.815	0.945	0.889	0.916	0.2978s

Bold values represent the best statistically significant result for each metric.

Table 7 presents the experimental results obtained from the SZdataset dataset and compares them with ten other deep learning methods discussed in Section 4.2. A thorough comparison was conducted, and as indicated in Table 7, while the accuracy, precision, recall, and F1 values for models such as ALBERT, BERT, BIMPM, RoBERT, XLNet, and ESIM surpass 0.89, all of them fall short in performance compared to the GeoBERTTM model proposed in this paper. Our proposed model outperforms them, achieving accuracy, precision, recall, and F1 values of 0.978, 0.98, 0.967, and 0.973, respectively. The simple reason is that these pre-trained embeddings are specifically designed to accurately capture the semantic essence of the words. Additionally, the incorporation of the ESIM model enables the capturing of local features between texts through the LSTM and attention mechanism, resulting in more precise place name matching.

Table 7. Experimental results obtained with the different methods under consideration on the SZdataset.

Model	Accuracy	Precision	Recall	F1	Time (s)
ABCNN	0.672	0.944	0.700	0.803	0.0395s
AlBERT	0.949	0.948	0.899	0.923	0.0364s
BERT	0.937	0.944	0.904	0.923	0.0356s
BIMPM	0.926	0.929	0.877	0.902	0.0478s
Decomposable Attention	0.514	0.901	0.514	0.654	0.0320s
SiaGRU	0.500	0.970	0.500	0.660	0.0106s
RoBERT	0.930	0.964	0.936	0.950	0.0357s
XLNet	0.921	0.953	0.958	0.955	0.0559s
Distillbert	0.500	0.970	0.500	0.660	0.0214s
ESIM	0.952	0.970	0.952	0.961	0.1682s
GeoBERTTM	0.978	0.980	0.967	0.973	0.2215s

Bold values represent the best statistically significant result for each metric.

Taking inspiration from Lin et al. (2020), we assessed the accuracy of our deep learning method for toponym matching. To achieve this, we conducted a comparative analysis between our approach and several existing address matching methods, focusing on predictive accuracy using the test set. For our deep learning method, we utilized the optimal combination of hyperparameters as described in the previous section. Initially, we compared the proposed method with string similarity-based toponym matching methods. These methods involved measuring the relevance between two toponym records using the Levenshtein distance, Jaccard similarity coefficient, and Jaro similarity. Furthermore, we employed random forests (RF) and support vector machine (SVM) classifiers to determine if the address records matched. Additionally, we evaluated our method against a machine learning-based toponym matching approach that employed word2vec to convert address records into vectors and employed machine learning classifiers for predictions. The toponym matching accuracy results for these methods are presented in Table 8.

Table 8. Experimental results obtained by the proposed method on the HNDataset.

Strategy	Model	Precision	Recall	F1
S1	Levenshtein distance + RF	0.890	0.807	0.846s
S2	Levenshtein distance + SVM	0.827	0.833	0.830s
S3	Jaccard similarity coefficient + RF	0.905	0.891	0.898s
S4	Jaccard similarity coefficient + SVM	0.911	0.912	0.911s
S5	Jaro similarity + RF	0.913	0.908	0.910s
S6	Jaro similarity + SVM	0.936	0.781	0.852s
S7	SVM	0.831	0.818	0.824s
S8	RF	0.822	0.805	0.813s
S9	Word2vec + SVM	0.912	0.903	0.907s
S10	Word2vec + RF	0.921	0.923	0.922s
S11	GeoBERTTM	0.965	0.970	0.977s

Table 8 provides compelling evidence that the utilization of string similarity-based methodologies for address matching can yield a commendable level of predictive accuracy, particularly when employing the Jaccard similarity coefficient in conjunction with certain classifiers (S3 and S4). Furthermore, it was observed that, with fixed string similarity metrics, the implementation of RF as the classifier resulted in a notable surge in predictive accuracy (S1, S3, and S5). Notably, among all the string similarity-based methods, the Jaccard similarity coefficient + SVM demonstrated high precision, achieving precision, recall, and F1 values of 0.911, 0.912, and 0.911, respectively (S4). Nevertheless, our novel deep learning-based address matching approach showcased superior performance when compared to the S3 method. The precision, recall, and F1 scores attained by GeoBERTTM on the test set reached an impressive 0.965. Consequently, our proposed method surpassed the conventional text-matching techniques in terms of matching accuracy.

In conclusion, an extensive comparison was conducted between the direct implementation of word2vec embeddings for classification (S9 and S10) and our proposed method. While the precision of S10 demonstrated a noteworthy value of 0.92, it is crucial to highlight that upon integrating ESIM into address matching, all evaluation metrics achieved an impressive score of 0.96. These outcomes emphasize the significant enhancements in predictive accuracy on the test set, brought about by the deep text-matching model GeoBERTTM.

Table 9 presents the experimental results obtained from the HBdataset dataset and compares them with ten other deep learning methods discussed in Section 4.2. A comprehensive comparison was conducted to evaluate their performance. As depicted in Table 9, among the ten place name matching models, the GeoBERTTM model proposed in this paper demonstrates the highest performance. It achieves accuracy, precision, recall, and F1 values of 0.977, 0.977, 0.965, and 0.97, respectively, surpassing the performance of all other models by more than 2%, apart from the ABCNN model. The obtained experimental results unequivocally demonstrate that the model presented in this paper exhibits superior performance on our self-constructed HBdataset dataset, thereby confirming the effectiveness and validity of the dataset.

Table 9. Experimental results obtained with the different methods under consideration on the HBdataset dataset.

Model	Accuracy	Precision	Recall	F1	Time (s)
ABCNN	0.924	0.987	0.935	0.960	0.0926s
AlBERT	0.959	0.961	0.899	0.929	0.0355s
BERT	0.940	0.948	0.907	0.927	0.0348s
BIMPM	0.950	0.956	0.899	0.927	0.0529s
Decomposable Attention	0.941	0.951	0.931	0.941	0.0159s
SiaGRU	0.847	0.953	0.884	0.917	0.0105s
RoBERT	0.949	0.950	0.939	0.945	0.0348s
XLNet	0.949	0.946	0.929	0.938	0.0564s
Distillbert	0.838	0.940	0.838	0.886	0.0192s
ESIM	0.959	0.959	0.959	0.959	0.0130s
GeoBERTTM	0.977	0.977	0.965	0.970	0.0277s

Bold values represent the best statistically significant result for each metric.

4.6. Parameter sensitivity analysis

We investigate the impacts of several key hyper-parameters in GeoBERTTM, which are learning rate l, number of hidden nodes n, mini-batch size k, and epoch m for each observed check-in.

Figure 8 firstly presents the impacts of learning rate. The performance of GeoBERTTM increases with the learning rare increased. When the learning rate was l = 0.0005, and with the F1 score consistently hovering around 0.97, we have found that setting l = 0.0005 strikes a balance between recommendation performance and training times.

Figure 8. Impacts of learning rate on the HBdataset. P = Precision, R = Recall, and F1 is the harmonic mean of the precision and recall.

As depicted in Figure 9, the F1 score on the development set remained consistently above 0.97 when adjusting the number of hidden nodes (n) between 100 and 400. Notably, the model attained the highest predictive accuracy on the development set when n = 300. However, as indicated in Figure 9, increasing the number of hidden nodes to n = 350 resulted in a decline in predictive accuracy on the development set. This suggests that a larger number of hidden nodes (350 or 400) introduces training challenges and may cause overfitting. Consequently, we have chosen to set the number of hidden nodes to n = 300.

Figure 9. Impacts of number of hidden nodes on the HBdataset. P = Precision, R = Recall, and F1 is the harmonic mean of the precision and recall.

According to Figure 10, the performance of the model exhibits an upward-then-downward trend when varying the mini-batch size from 20 to 50 in increments of 5. The F1 scores on the development set hover slightly above 0.96 when the mini-batch size is either 25 or 50. However, when the mini-batch size is set to 60, the F1 scores display a relatively large fluctuation, indicating that this size is excessively large. Furthermore, Figure 10 reveals that the prediction accuracy on the development set is at its lowest when the mini-batch size is 25, suggesting that this value is too small. Consequently, we have opted for a mini-batch size of 50 in subsequent experiments.

Figure 10. Impacts of mini-batch size on the HBdataset. P = Precision, R = Recall, and F1 is the harmonic mean of the precision and recall.

The impacts of epoch m on the performance of GeoBERTTM are presented in Figure 11. We can observe that increasing epoch does provide performance improvements for toponym matching as m increases. GeoBERTTM achieves its best performance when m is 50 on the HBdataset dataset. However, the performance of GeoBERTTM drops with a larger m, and a larger m costs longer training times as well. So we set m = 50 for the HBdataset dataset.

Figure 11. Impacts of epoch on the HBdataset. P = Precision, R = Recall, and F1 is the harmonic mean of the precision and recall.

4.7. Case study and error analysis on the HBdataset

Lastly, the effectiveness of the proposed neural network architecture is showcased through the presentation of Table 10. This table exhibits instances of record pairs, encompassing both matching and nonmatching pairs, which were correctly or incorrectly classified by the model. While the table encompasses a limited number of examples, an extensive manual analysis further substantiated the superior performance of the proposed neural network architecture in such scenarios.

Table 10. A set of illustrative error examples for the results with the proposed approach on HBdataset.

Error type	Toponym 1	Toponym 2	Error percentage	Error description
Lack of prefix range information / large variations in string lengths	武汉市蔡甸区永安镇[Yongan Town, Caidian District, Wuhan]	永安镇[Yongan Town]	44%	Lack of prefix information at the city and district level leads to large differences in string lengths
Differences in details (e.g. house number/road number)	武汉市江夏区东湖开发区关南科技工业园光谷大道68号[No.68, Guanggu Avenue, Guannan Science and Technology Industrial Park, Donghu Development Zone, Jiangxia District, Wuhan, China]	武汉市江夏区东湖开发区光谷大道70号[No. 70, Guanggu Avenue, Donghu Development Zone, Jiangxia District, Wuhan, China]	15%	Differences in details between door number ‘68’ and ‘70’ lead to errors in judgement
Different names for the same place	福田区红荔西路7024号市政生活区20栋b-1506[B-1506, Block 20, Municipal Services Living Quarter, 7024 Hung Lai West Road, Futian District]	福田区红荔西路7024号天健市政大院20栋[Building 20, Tianjian Municipal Compound, No. 7024 Hongli West Road, Futian District]	3%	Geographical location of two names is the same but the expressions are different, leading to incorrect judgement
Oral expressions	光明路西角巷对面东成电动工具[Tung Shing Power Tools, opposite to Sai Kok Lane, Kwong Ming Road]	荔城街光明路42号对面荔城东成电动工具维修部[Opposite to No. 42 Kwong Ming Road, Lai Shing Street, Lai Shing Tung Shing Power Tools Repairing Department]	4%	The use of colloquial expressions such as ‘across the street’ in place names resulted in incorrect model matching
Same toponym but different range sizes (e.g. inclusive relationships)	吉林大学和平校区家属院经2栋3单元[Unit 3, Building 2, Heping Campus of Jilin University, Jilin, China]	西安大路5333号吉林大学和平校区[Heping Campus of Jilin University, No. 5333 Xi’an Avenue]	3%	Place names 1 and 2 both belong to the Heping Campus, but place name 1 is much smaller in scope and refers specifically to a particular building unit, leading to an error in judgement

We focused on the analysis of matching errors and summarized them. The mis-matches can be classified into seven categories:

1. 44% of the misclassifications were due to different toponym prefix ranges. The prefixes of some toponyms are classified at the district and city level, while the prefixes of some toponyms are classified at the road level, which leads to matching errors.

2. 17% of the errors are due to the lack of prefix information, in which case the model may not be able to accurately understand the content of the toponym, e.g. it is not possible to determine the mismatch of toponyms in different areas but with the same name.

3. 15% of the error classification occurs in the case of detail differences, such as the differences in the name of the district, door number, and building number can lead to matching errors.

4. 14% of the errors are caused by large differences in string length.

5. 3% of the errors are caused by not identifying the toponym of the containment relationship, the most common containment relationships such as a shopping mall and the shops in that mall, a community and the specific building or gateway in that community.

6. 3% of the errors are caused by unintelligible toponyms with different names in the same place, where the same toponym may be named differently due to factors such as history and application scenarios. In this case, the model may treat them as mismatched toponym pairs.

7. 4% of the errors were due to colloquialisms, which may be due to the use of informal or colloquial expressions in a toponym that the model cannot accurately understand.

5. Discussion

This research offers a new approach for automatically matching address from natural language texts, leveraging a deep neural network with geographic pre-training model to classify pairs of address as either matching or nonmatching. From an intellectual standpoint, this is the first attempt to enhance domain-specific address matching performance by combining a deep learning-based algorithm with a variety of embeddings (Pinyin, Wubi, Radicals, and Strokes). The suggested deep learning-based solution can decrease the amount of human intervention necessary during the address matching process. The adopted embeddings can provide rich semantics from the computational linguistics area and identify informative words in a sentence, allowing for a more in-depth comprehension of the text and improved capture of domain-specific aspects. From a practical standpoint, applying the suggested address matching technique might aid in streamlining the geocoding of urban data and advancing the geospatial administration of urban systems. As a result, integrating the suggested method’s use with other data, information, or knowledge or with other software systems already in place should be simple. A module to match a queried address with one in the address database, for instance, could be easily added to a number of applications/software related to geographical information retrieval and the geographical information sciences. This would enable the queried address to be converted into a geographical coordinate and, consequently, located on a map.

Three main limitations of the work are acknowledged, which point to four directions of future work. First, we do not consider the problem of duplicate place names in this paper because the address database used is limited to a single province (e.g. Hubei Province), and the ambiguity of place names is negligible. However, if our method is applied to a higher-level address database (e.g. a national address database), the model will not be able to resolve two cognitive place names with the same address elements but different locations in the training of word vectors. In addition, when training the address element vectors, we did not take into account the hierarchical relationship between them; that is, we considered address elements of different hierarchical levels as equally important and assigned the values of the word vectors based only on their word frequencies. Therefore, in our future work, we plan to adopt a word embedding approach that incorporates domain ontologies, incorporates more knowledge of place names, and converts address elements into vectors, which may help to solve this problem. Third, due to the data-hungry nature of deep learning, data availability and quality are inevitable topics to be discussed when it comes to large and complex deep learning models. It is common knowledge in the deep learning field that the larger the dataset, the better the generality and performance. However, large-scale, high-quality datasets mean that significant labor costs are required, and few-shot or zero-shot learning needs to be considered in the future. Fine-tuned language models can be zero-shot or few-shot learners, which means that these models can be applied directly to certain downstream tasks with little or no further training. This is because high-level language models can better capture the meaning of text. This is also demonstrated in this paper by the results of using BERT to enhance the module capabilities. Therefore, the inclusion of giant models such as GPT-3 and GPT-4 may lead to a new round of performance improvement.

6. Conclusions and future work

In this study, we propose a novel deep learning architecture that demonstrates efficacy in the realm of semantic toponym matching. Our approach encompasses two key steps. Firstly, we subject the BERT model to pre-training on an extensive corpus of geo-domain texts, which enables the acquisition of distributed embedding representations for characters, thereby resulting in GeoBERT. Subsequently, we utilize word2vec to convert the input toponym records, enriched with Pinyin, Wubi, Radicals, and Strokes features, into vector format. These vectors are then fed into the ESIM, a deep text-matching model, to compute the semantic similarity between compared toponym records and ascertain their degree of match. To validate our proposed method, we conducted experiments using three distinct toponym databases, including a carefully constructed toponym corpus. The experimental results demonstrate that the ESIM achieves optimal performance when configured with 300 hidden nodes, a learning rate of 0.0005, and a minibatch size of 50. Furthermore, our deep learning approach outperforms existing toponym matching methods in terms of matching accuracy, particularly when dealing with unstructured queried toponyms. Notably, our method excels at capturing the intricate character transliterations encountered in certain pairs of toponyms.

Although the paper presents interesting findings, it also presents numerous avenues for future research. For instance, exploring the utilization of advanced optimization techniques to train deep neural networks and employing systematic approaches to fine-tune hyperparameters would be worthwhile. Additionally, experimenting with diverse model architectures and knowledge representations, including models like ChatGPT, which draw inspiration from recent advancements in textual entailment and related NLP tasks, holds promise. In future work, we intend to introduce a strategy that assigns varying weights to toponym element vectors based on their hierarchical structure, thereby enhancing the accuracy of toponym matching.

Acknowledgments

We would like to express our great appreciation to the editors and two anonymous reviewers for constructive comments that helped improve the manuscript.

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability statements

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Additional information

Funding

This study was financially supported by the National Key R&D Program of China (No. 2022YFB3904200, 2022YFF0711601), the Natural Science Foundation of China (No. 42301492), the Open Fund of Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (No. 2022SDSJ04), and the Open Fund of Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources (No. KF-2022-07-014).