CN105095322A - Personnel name unit dictionary expansion method, personnel name language recognition method, personnel name unit dictionary expansion device and personnel name language recognition device - Google Patents

Abstract

Disclosed are a name unit dictionary expansion method, a name language recognition method and a device. The name unit dictionary expansion method includes: calculating the initial weight of each term in a person name unit dictionary in a predetermined plurality of languages; dividing the name into person name units , match the divided person name unit with the terms in each dictionary to determine the matching person name unit and the unmatched person name unit; Weights in each dictionary; Calculate the weight of unmatched person-name units in each dictionary based on the weights in each dictionary of all names that contain unmatched person-name units, and add unmatched person-name units to the dictionary; The weights of all the names of the person name unit in each dictionary are updated to match the weights of the person name unit in each dictionary; the above process is repeated until the predetermined conditions are met, so as to obtain the name unit dictionary with weights.

Description

Name unit dictionary expansion method, name language recognition method and device

technical field

The present disclosure relates to the technical field of information processing, and more specifically, to a method for expanding a dictionary of personal name units, a method and a device for recognizing language of personal names.

Background technique

Language recognition of personal names has been widely used in natural language processing fields such as text processing and machine translation, and it can effectively improve processing performance. In general, name language recognition can be considered as a classification problem, and classification performance is affected by training expectations and feature selection. Therefore, how to obtain a large amount of training corpus with high coverage and achieve effective feature extraction for short texts such as names is the focus of research in this field.

Specifically, the training corpus of names marked with names and countries needs to invest a lot of resources, and it is also difficult to make such a training corpus cover all possibilities of each language. Therefore, it is a challenge for those skilled in the art how to solve the problem of performing name language recognition without sufficient training corpus. On the other hand, since personal names are short texts, compared with ordinary texts, there are fewer features available for personal names, which makes the language recognition of personal names more difficult than that of ordinary texts. Therefore, how to construct classifier training features is also a big problem.

Contents of the invention

A brief summary of the present disclosure is given below in order to provide a basic understanding of some aspects of the present disclosure. It should be understood, however, that this summary is not an exhaustive summary of the disclosure. It is not intended to identify key or critical parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

In view of the above circumstances, the purpose of the present disclosure is to provide a name unit dictionary expansion method, a name language recognition method and a name language recognition device that can reduce the requirement for training corpus while ensuring performance.

According to one aspect of the present disclosure, there is provided a method for expanding a person-name unit dictionary, which may include: a term initial weight calculation step, based on the number of person-name unit dictionaries that each term appears, and calculating the weight of each term in a predetermined The initial weight in the name unit dictionary of a plurality of languages; The name unit matching step is divided into each name unit in a plurality of names of people as training samples, and the divided name unit is combined with the predetermined multiple languages The words in the name unit dictionary are matched, and the matched name unit is determined as the matching person name unit, and the unmatched person name unit is determined as the unmatched person name unit; the name weight calculation step, according to the matching person name unit Determine the weight of the person's name containing the matching person's name unit in each person's name unit dictionary according to the weight in the person's name unit dictionary of predetermined multiple languages; weights in individual name unit dictionaries, computes the weight of unmatched name units in individual name unit dictionaries, and adds unmatched name units as terms to matches in all names containing unmatched name units In the personal name unit dictionary where the personal name unit is located; the matching personal name unit weight update step is to update the matching personal name unit in each personal name unit dictionary according to the weight of all names in each personal name unit dictionary containing the matching personal name unit and repeating the processing in the person-name unit matching step, the person-name weight calculation step, the unmatched person-name unit processing step and the matching person-name unit weight update step, until all the terms in the person-name unit dictionary of a plurality of languages are predetermined Until the weight change of is less than a predetermined threshold, a dictionary of person-name units is obtained in which all terms are marked with weights.

According to another aspect of the present disclosure, there is also provided a method for recognizing the language of a person's name, which may include: a step of dividing a person's name into n-gram substrings, wherein the n-gram substrings represent the A unit of n consecutive characters, n is an integer greater than or equal to 2; the n-element substring weight calculation step is based on all the terms that include each n-element substring in the personal name unit dictionary marked with weight according to the present disclosure to calculate the weight of each n-element substring in each person's name unit dictionary; and the identification step, according to the weights of all n-element substrings in the person's name in each person's name unit dictionary, identify the language to which the name belongs .

According to another aspect of the present disclosure, there is also provided a personal name language recognition device, which may include: a personal name division unit configured to divide the input personal name into n-gram substrings, wherein the n-gram substrings represent A unit of n consecutive characters in a person's name, n is an integer greater than or equal to 2; an n-element substring weight calculation unit is configured to include all terms containing each n-element substring in the weighted The weights in the marked personal name unit dictionary are used to calculate the weights of each n-element substring in each personal name unit dictionary; The weight in identifies the language to which the person's name belongs.

According to another aspect of the present disclosure, there is also provided a storage medium, the storage medium includes machine-readable program code, and when the program code is executed on the information processing device, the program code causes the information processing device to perform the following steps: The item initial weight calculation step is based on the number of person-name unit dictionaries that appear in each term, and calculates the initial weight of each term in the person-name unit dictionary of predetermined multiple languages; the person-name unit matching step will be used as a training sample Divide each person name in the plurality of names of people into name units, match the divided name units with the terms in the name unit dictionary of predetermined multiple languages, and determine the matched name units as a list of matching people element, the unmatched personal name unit is determined as an unmatched personal name unit; the personal name weight calculation step determines that the personal name containing the matching personal name unit is in the The weight in each name unit dictionary; the unmatched person name unit processing step, according to the weight of all names in each person name unit dictionary containing the unmatched person name unit, calculate the unmatched person name unit in each person name unit dictionary , and add the unmatched person name unit as a term to the name unit dictionary where the matching person name unit in all names containing the unmatched person name unit is located; the matching person name unit weight update step, according to the inclusion of matching person The weight of all names of the name unit in each name unit dictionary, update the weight of the matching person name unit in each person name unit dictionary; and repeat the name unit matching steps, name weight calculation steps, and unmatched person name unit processing steps And match the processing in the person name unit weight update step until the weight changes of all the terms in the person name unit dictionaries of predetermined multiple languages are less than a predetermined threshold, so as to obtain a person name unit dictionary in which all terms have weight labels.

According to another aspect of the present disclosure, there is also provided a program product, the program product includes machine-executable instructions, and when the instructions are executed on the information processing device, the instructions cause the information processing device to perform the following steps: Calculation step, based on the number of person-name unit dictionaries that appear in each term, calculate the initial weight of each term in the person-name unit dictionary of predetermined multiple languages; Each person name in the name is divided into a person name unit, the divided person name unit is matched with the term in the person name unit dictionary of a predetermined plurality of languages, and the matched person name unit is determined as a matching person name unit, and the The unmatched personal name unit is determined as an unmatched personal name unit; the personal name weight calculation step determines that the personal name containing the matched personal name unit is included in each personal list according to the weight of the matched personal name unit in the personal name unit dictionary of a predetermined plurality of languages The weight in the meta-dictionary; the unmatched person-name unit processing step, according to the weight of all names in each person-name unit dictionary containing the unmatched person-name unit, calculate the weight of the unmatched person-name unit in each person-name unit dictionary, And the unmatched personal name unit is added as a term to the personal name unit dictionary where the matching personal name unit in the names of all the unmatched personal name units is located; the matching personal name unit weight update step, according to the inclusion of the matching personal name unit The weights of all names in each person name unit dictionary, update the weights of matching person name units in each person name unit dictionary; and repeat the person name unit matching step, the person name weight calculation step, the unmatched person name unit processing step and the matching person The processing in the step of updating the weight of the name unit is performed until the weight changes of all the terms in the person-name unit dictionaries of the predetermined multiple languages are less than a predetermined threshold, so as to obtain a person-name unit dictionary in which all terms are marked with weights.

According to another aspect of the present disclosure, there is also provided a storage medium, the storage medium includes machine-readable program code, and when the program code is executed on the information processing device, the program code causes the information processing device to perform the following steps: The dividing step is to divide the input person's name into n-element substrings, wherein, the n-element substrings represent units comprising consecutive n characters in the person's name, and n is an integer greater than or equal to 2; n-element substring weight calculations Step, calculate the weight of each n-element substring in each person's name unit dictionary according to the weights of all words containing each n-element substring in the person-name unit dictionary with weight labeling according to the present disclosure; and identify The step is to identify the language to which the name belongs according to the weights of all n-element substrings in the name in each name unit dictionary.

According to another aspect of the present disclosure, there is also provided a program product, the program product includes machine-executable instructions, and when the instructions are executed on the information processing device, the instructions cause the information processing device to perform the following steps: the step of classifying personal names, Divide the input person's name into n-element substrings, wherein, n-element substrings represent units that include consecutive n characters in the person's name, and n is an integer greater than or equal to 2; the n-element substring weight calculation step is based on Computing the weights of each n-element substring in each individual name unit dictionary according to the weights of all terms containing each n-element substring in the personal name unit dictionary marked with weight according to the present disclosure; and the recognition step, according to The weights of all n-gram substrings in the person's name in each person's name unit dictionary, identifying the language to which the person's name belongs.

Further aspects of embodiments of the present disclosure are given in the following descriptive section, wherein the detailed description serves to fully disclose preferred embodiments of the embodiments of the present disclosure without imposing limitations thereon.

Description of drawings

The present disclosure can be better understood by referring to the following detailed description given in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout to designate the same or similar parts. The accompanying drawings, together with the following detailed description, are incorporated in and form a part of this specification, and serve to further illustrate preferred embodiments of the present disclosure and explain principles and advantages of the present disclosure. in:

FIG. 1 is a flowchart illustrating a process example of a method for expanding a person-name unit dictionary according to an embodiment of the present disclosure;

2 is a block diagram showing an example of a functional configuration of a person-name unit dictionary expansion device according to an embodiment of the present disclosure;

FIG. 3 is a flow chart illustrating a process example of a method for identifying language of a person's name according to an embodiment of the present disclosure;

4 is a block diagram showing an example of a functional configuration of a personal name language recognition device according to an embodiment of the present disclosure; and

FIG. 5 is a block diagram of an example structure of a personal computer as an information processing device employable in an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in this specification. It should be understood, however, that in developing any such practical embodiment, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and those Restrictions may vary from implementation to implementation. Moreover, it should also be understood that development work, while potentially complex and time-consuming, would at least be a routine undertaking for those skilled in the art having the benefit of this disclosure.

Here, it should be noted that in order to avoid obscuring the present disclosure due to unnecessary details, only the device structure and/or processing steps closely related to the solution according to the present disclosure are shown in the drawings, and the Other details that are not materially relevant to the present disclosure are omitted.

Next, an embodiment of the present disclosure will be described with reference to FIGS. 1 to 5 .

First, a procedure example of a person-name unit dictionary expanding method according to an embodiment of the present disclosure will be described with reference to FIG. 1 . FIG. 1 is a flowchart illustrating a procedure example of a person-name unit dictionary expansion method according to an embodiment of the present disclosure.

As shown in FIG. 1 , the method for expanding a person-name unit dictionary according to an embodiment of the present disclosure includes a term initial weight calculation step S102, a person-name unit matching step S104, a person-name weight calculation step S106, an unmatched person-name unit processing step S108, and Step S110 for updating the weight of the matching person name unit.

First, in step S102 of calculating the initial weight of each term, the initial weight of each term in the personal-name unit dictionaries in predetermined languages is calculated based on the number of personal-name unit dictionaries in which each term appears.

It should be understood that the terms in the multilingual name unit dictionary are name units (ie, components of a person's name, such as last name, first name, etc.). The term initial weight calculation step is performed to disambiguate. This is because a term may appear in multiple person-name unit dictionaries at the same time, and since a person-name usually includes multiple person-name units, and these person-name units may also exist in different person-name-unit dictionaries, this is an important factor for the language of personal names. Recognition creates ambiguity.

Specifically, as an example, in step S102 of calculating the initial weight of a term, assuming that a term exists in three person-name unit dictionaries at the same time, the initial weight of the term in the three person-name unit dictionaries is 1/3 respectively .

Next, in the person name unit matching step S104, each person name in a plurality of person names as training samples is divided into a person name unit, and the divided person name unit is compared with the word in the person name unit dictionary of a predetermined plurality of languages Items are matched, and the matched person name unit is determined as a matched person name unit, and the unmatched person name unit is determined as an unmatched person name unit. It should be pointed out that the division method of the name unit is a well-known technology in the art, and will not be repeated here.

Then, in the name weight calculation step S106, according to the weights of the matching person name units in the person name unit dictionaries of predetermined multiple languages, the weights of the names including the matching person name unit in each person name unit dictionary are determined.

Specifically, it is assumed that the predetermined personal name unit dictionaries in multiple languages are a set D ^k = {d ^k ₁ , d ^k ₂ , d ^k ₃ , d ^k _j ,..., d ^k _n }, where d ^k _i represents The ith personal name unit dictionary, k represents the kth iteration, and n represents the number of personal name unit dictionaries. Furthermore, assume that each person-name cell dictionary is defined as where W _i,j represents the jth term in the ith personal name unit dictionary, k represents the kth iteration, and LEN(d _i ) represents the length of the dictionary d _i (i.e., the number of terms contained in ). Assume that the names of persons as training samples are p _m = {N ₁ ,..., N _t }, where p _m represents the m-th person name in the training sample, and N _t represents the t-th person list in the person name p _m Yuan.

Here, as an example, the weight distribution of the name p _m in all name unit dictionaries is defined as a vector ${\mathrm{vector}}^k\left(p_m\right)=<{v^k}_{d_1},{v^k}_{d_2},{v^k}_{d_3},...,{v^k}_{d_j},...,{v^k}_{d_{\mathrm{no}}}>,$ Each component of this vector is the weight of the person's name p _m in each person's unit dictionary, and the weight of the person's name p _m in the dictionary d _j at the kth iteration is defined as Among them, LEN(p _m ) represents the length of the person's name p _m (that is, the number of the person's name units included), and the person's name unit N here is the person's name unit that matches the term in the dictionary d _j (that is, ), and during the first iteration, weight (N) is the initial weight of the term that matches the name unit N calculated in the term initial weight calculation step S102.

Next, in the unmatched personal name unit processing step S108, according to the weights of the names of all persons containing the unmatched personal name unit in each personal name unit dictionary, calculate the weight of the unmatched personal name unit in each personal name unit dictionary, And add the unmatched person name unit as a term to the person name unit dictionary where the matching person name unit is located in all names containing the unmatched person name unit.

Specifically, as an example, assuming that the unmatched personal name unit is "kuai", which is contained in two personal names p ₁₂ and p ₄₃ , then ${\mathrm{vector}}^{k+1}\left(\mathrm{kuai}_{\&prime;\&prime;}^{\&prime;\&prime;}\right)=\frac{{\mathrm{vector}}^k\left(p_{12}\right)+{\mathrm{vector}}^k\left(p_{43}\right)}{\mathrm{len}\left(\right\{p_{12},p_{43}\left\}\right)}$ , wherein, vector ^k+1 ("kuai") represents a vector composed of the weights of the unmatched name unit "kuai" in each name unit dictionary, which can be composed of the name p ₁₂ containing the unmatched name unit "kuai" and p ₄₃ are determined by the weight distribution vector ^k (p ₁₂ ) and vector ^k (p ₄₃ ) in all name unit dictionaries. After calculating the weight of the unmatched personal name unit "kuai", it is added as a term to the personal name unit dictionary where the matching personal name units contained in the personal names p ₁₂ and p ₄₃ are located. For example, assuming that the person name p ₁₂ contains the matching person name unit "li", and the person name p ₄₃ contains the matching person name unit "song", then the unmatched person name unit "kuai" is added to the person where "li" and "song" are located in the unit dictionary.

It can be seen that by adding all the name units that appear in a person's name to the same name unit dictionary, the language to which the name belongs can be more accurately identified in the subsequent name recognition.

Next, in the weight updating step S110 of the matching person-name unit, the weight of the matching person-name unit in each person-name unit dictionary is updated according to the weights of the names of all persons including the matching person-name unit in each person-name unit dictionary.

Specifically, assuming that the matching name unit is "lin", which is contained in three names p ₁₂₃₄ , p ₄₃₅₆₇ and p ₈₉₃₅₂ respectively, the updated weight of the name unit "lin" is ${\mathrm{vector}}^{k+1}\left(\mathrm{the\; lin}_{\&prime;\&prime;}^{\&prime;\&prime;}\right)=\frac{\frac{{\mathrm{vector}}^k\left(p_{1234}\right)+{\mathrm{vector}}^k\left(p_{43567}\right)+{\mathrm{vector}}^k\left(p_{89352}\right)}{\mathrm{len}\left(\right\{p_{1234},p_{43567},p_{89352}\left\}\right)}+{\mathrm{vector}}^k\left(\mathrm{the\; lin}_{\&prime;\&prime;}^{\&prime;\&prime;}\right)}{2}.$ For example, assuming vector ^k+1 ("lin")=<0.344,0.112,0.092,...>, update the weight of "lin" in dictionary 1 to 0.344, and update the weight in dictionary 2 to 0.112, And the weight in dictionary 3 is updated to 0.092.

It should be understood that although the specific example formulas for weight calculations are given above, this is only an example rather than a limitation, and those skilled in the art can make changes to the above formulas according to the principles of the present disclosure, and these modifications should be considered to fall within the scope of this disclosure. within the scope of the public.

Next, repeat the processing in the above-mentioned personal name unit matching step, personal name weight calculation step, unmatched personal name unit processing step and matching personal name unit weight update step, until all words in the personal name unit dictionary of multiple languages are predetermined Until the change of the weight of an item is less than a predetermined threshold or repeats for a predetermined number of times, a person-name unit dictionary in which all terms are marked with weights is obtained.

It can be seen that, according to the name unit dictionary expansion method of the embodiment of the present disclosure, it is possible to construct a name unit dictionary in which each term has a weight label for more accurate recognition The language the person's name belongs to.

In addition, preferably, the name unit dictionary expansion method according to the embodiment of the present disclosure may also include a normalization step (shown in a dotted line box), in the normalization step, the name of the person calculated in the name weight step can be The weights in each individual unit dictionary are normalized. As an example, normalization can be done as follows: $\mathrm{the\; norm}\left({\mathrm{vector}}^k\left(p_m\right)\right)=<\mathrm{the\; norm}\left({v^k}_{d_1}\right),\mathrm{the\; norm}\left({v^k}_{d_2}\right),...,\mathrm{the\; norm}\left({v^k}_{d_{\mathrm{no}}}\right)>,$ in, It should be understood that this normalization algorithm is only an example and not a limitation, and other algorithms may be used to normalize the weights of names. Furthermore, it should also be understood that this normalization step is optional.

Preferably, in the step of processing the unmatched person name unit and the weight update step of the matched person name unit, the weight of the person name after normalization can be used for corresponding calculation.

It should be understood that the processing process of the method for expanding the personal name unit dictionary according to the embodiment of the present disclosure described above with reference to FIG. combination etc.

Next, a functional configuration example of a person-name unit dictionary expanding device according to an embodiment of the present disclosure will be described with reference to FIG. 2 . FIG. 2 is a block diagram showing an example of a functional configuration of a personal-name unit dictionary expansion device according to an embodiment of the present disclosure.

As shown in FIG. 2 , the apparatus 200 for expanding a person-name unit dictionary according to an embodiment of the present disclosure may include a term initial weight calculation unit 202 , a person-name unit matching unit 204 , a person-name weight calculation unit 206 , and an unmatched person-name unit processing unit. 208 , the step 210 and the control unit 212 for updating the weight of the matching person name unit.

The term initial weight calculation unit 202 may be configured to calculate the initial weight of each term in the personal-name unit dictionaries in predetermined multiple languages based on the number of personal-name unit dictionaries in which each term appears.

The personal name unit matching unit 204 may be configured to divide each of the multiple personal names used as training samples into a personal name unit, and compare the divided personal name units with the terms in the personal name unit dictionary of a predetermined plurality of languages. match, and determine the matching person name unit as a matching person name unit, and determine the unmatched person name unit as an unmatched person name unit.

The name weight calculation unit 206 may be configured to determine the weights of the names containing the matching name units in each name unit dictionary according to the weights of the matching name units in the name unit dictionaries of predetermined multiple languages.

The unmatched person name unit processing unit 208 may be configured to calculate the weight of the unmatched person name unit in each person name unit dictionary according to the weights of all names containing the unmatched person name unit in each person name unit dictionary, and The unmatched name unit is added as a term to the name unit dictionary containing the matching name unit in all names containing the unmatched name unit.

The matching person name unit weight updating unit 210 may be configured to update the weights of the matching person name unit in each person name unit dictionary according to the weights of all names containing the matching person name unit in each person name unit dictionary.

The control unit 212 may be configured to control the name unit matching unit 204, the name weight calculation unit 206, the unmatched name unit processing unit 208, and the matched name unit weight update unit 210 to repeatedly perform their respective processes until the predetermined number of languages Until the weight change of all terms in the person-name unit dictionary is less than a predetermined threshold or repeats for a predetermined number of times, a person-name unit dictionary in which all terms are marked with weights is obtained.

In addition, preferably, the name unit dictionary expansion device 200 can also include a normalization unit (shown in a dotted line box), the normalization unit can be configured to calculate the name weight calculation unit in each name unit The weights in the dictionary are normalized, and the unmatched person name unit processing unit and the matched person name unit weight update unit can use the normalized name weights to perform corresponding calculations.

It should be understood that the personal name unit dictionary expansion device described with reference to FIG. 2 is a device embodiment corresponding to the above-mentioned personal name unit dictionary expansion method, so for the content not described in detail in the device embodiment, you can refer to the corresponding method embodiment above. The description of the location will not be repeated here.

In addition, it should be pointed out that although an example of the functional configuration of the personal name unit dictionary expansion device according to the embodiment of the present disclosure is described above with reference to FIG. 2 , this is only an example rather than a limitation, and those skilled in the art can think Whereas the functional modules described in the above embodiments are combined and/or omitted and/or one or more functional modules are added, such modified examples should be deemed to fall within the scope of the present disclosure.

Next, a process example of the human name language recognition method according to the embodiment of the present disclosure will be described with reference to FIG. 3 . Specifically, how to perform the personal name language recognition process using the personal name unit dictionary in which terms are weighted annotated constructed as described above will be described.

FIG. 3 is a flow chart showing an example of a procedure of a human name language recognition method according to an embodiment of the present disclosure.

As shown in FIG. 3 , the name language recognition method according to the embodiment of the present disclosure includes a name division step S302 , an n-gram weight calculation step S304 and a recognition step S306 .

First, in the personal name division step S302, the input personal name is divided into n-gram substrings, wherein an n-gram substring represents a unit including n consecutive characters in the personal name, and n is an integer greater than or equal to 2.

Specifically, in the personal name division step S302, all possible n-gram substrings of English personal names (combined from spaces and 26 English letters) are constructed. In addition, preferably, for the characters at the beginning and end of the person's name, special characters (for example, "_") may be added in front and behind respectively for division. Here, taking the ternary substring (that is, n=3) as an example, assume that for the name "linshuhao", the divided ternary substring includes "_li", "lin", ..., " ao_".

Next, in the n-gram substring weight calculation step S304, each n-gram subcharacter is calculated according to the weights of all terms containing each n-gram substring in the above-mentioned weight-labeled personal name unit dictionary The weight of the string in each name unit dictionary.

Specifically, for example, suppose that in the dictionary d _j , the person name units with weight criteria are as follows: li: 0.8; lian: 0.9; liang: 1.0. Therefore, for the ternary substring "_li", its weight in the dictionary d _j can be calculated as 0.8+0.9+1.0=2.7. Similarly, the weights of all n-gram substrings included in the person name in each person name unit dictionary can be calculated. It should be understood that the weight calculation method is only an example rather than a limitation, and those skilled in the art may use other methods to determine the weight of the n-gram substring according to needs.

Then, in the identifying step S306, the language to which the name belongs is identified according to the weights of all n-gram substrings in the name in each individual name unit dictionary. Specifically, as an example, if the sum of the weights of all _n -gram substrings contained in the person's name in the dictionary dj is the largest, it can be considered that the person's name belongs to the language represented by the dictionary _dj .

In addition, preferably, the name language recognition method 300 according to the embodiment of the present disclosure may further include a sorting step S308.

In the sorting step S308, according to the weight of each n-gram substring in each person-name unit dictionary, determine the ordering of each n-gram substring in each person-name unit dictionary, and in the recognition step S306, according to the weight of each person-name unit dictionary The ordering of all n-gram substrings in each person's name unit dictionary is used to identify the language to which the person's name belongs.

Preferably, in the sorting step S308, the sorting of the n-gram substrings in each person-name unit dictionary can be determined according to the descending order of the weights of the n-gram substrings in each person-name unit dictionary. Specifically, the following Table 1 gives an example of determining the ranking according to the descending order of the weights.

Table 1

to sort Dictionary D1 Dictionary D2 Dictionary D3 Dictionary D4 1 _li: 9.3 in_: 10.2 om_: 9.9 mi_: 3.4 2 lin:4.5 _li: 7.2 imi: 0.4 in_: 2.2 3 in_: 2.1 lin: 3.5 lin: 0.2 lin:1.0

Preferably, in the recognition step S306, the sum of the ordering of all n-element substrings in the person's name in each person's name unit dictionary is calculated, and the language represented by the person's name unit dictionary corresponding to the smallest sum is determined as the name to which the person's name belongs language.

Specifically, still taking the person name "linshuhao" as an example, for dictionaries D1 to D4, the sum of the rankings of the person name in each dictionary is as follows:

Distance(D1,"linshuhao")=Order_D1(_li)+Order_D1(lin)+...Order_D1(ao_)

Distance(D2,"linshuhao")=Order_D2(_li)+Order_D2(lin)+...Order_D2(ao_)

Distance(D3,"linshuhao")=Order_D3(_li)+Order_D3(lin)+...Order_D3(ao_)

Distance(D4,"linshuhao")=Order_D4(_li)+Order_D4(lin)+...Order_D4(ao_)

In this example, since the sorting is performed in descending order of weight, the language to which the name "linshuhao" belongs should be the language represented by the dictionary whose sorting sum is the smallest. That is, Language=DxifDistance(Dx, "linshuhao")=Min(Distance(D1, "linshuhao"), Distance(D2, "linshuhao"), Distance(D3, "linshuhao"), Distance(D4, "linshuhao") ).

Alternatively, the method of calculating Distance can be changed to calculating the average value of all substrings included in the person name "linshuhao" in each dictionary, and the language represented by the dictionary with the smallest average value is used as the name language recognition result. Alternatively, the product of the normalized weight values of all substrings in the person's name in each dictionary can be calculated, and the language represented by the dictionary with the largest product value can be used as the language recognition result of the person's name.

It can be seen that if the language recognition of personal names is only performed directly based on the original weights, there may be errors in the recognition due to the weights in each dictionary not being normalized according to a unified standard. Therefore, according to the processing in the above-mentioned sorting step, the accuracy of recognition can be improved.

However, it should be understood that the above sorting algorithm is only an example rather than a limitation, and those skilled in the art can think of other algorithms for sorting according to the principle of the present disclosure. For example, they can also be sorted in ascending order of weights. In this case, the names of people can be determined as the language represented by the dictionary with the largest sorted sum among them.

In addition, it should be understood that the sorting step is optional, for example, instead of sorting, the weights in all name unit dictionaries can be normalized, so that the language recognition of names can be performed according to the normalized weights .

It should be pointed out that although the method for recognizing the language of a person's name according to the embodiment of the present disclosure is described above with reference to FIG. 3 , those skilled in the art can completely modify, combine, etc. the above processing procedures according to the principle of the present disclosure.

Next, an example of a functional configuration of a personal name language recognition device according to an embodiment of the present disclosure will be described with reference to FIG. 4 . FIG. 4 is a block diagram showing an example of a functional configuration of a personal name language recognition device according to an embodiment of the present disclosure.

As shown in FIG. 4 , the device 400 for recognizing the language of a person's name according to an embodiment of the present disclosure may include a name division unit 402 , an n-gram weight calculation unit 404 and a recognition unit 406 .

The personal name division unit 402 may be configured to divide the input personal name into n-gram substrings, where an n-gram substring means a unit including n consecutive characters in the personal name, and n is an integer greater than or equal to 2.

The n-gram substring weight calculation step 404 can be configured to calculate the weight of each n-gram substring in each person list according to the weights of all terms containing each n-gram substring in the above-mentioned person name unit dictionary with weight annotation. Weights in the meta dictionary.

The identification unit 406 may be configured to identify the language to which the person's name belongs according to the weights of all n-gram substrings in the person's name in each person's name unit dictionary.

Preferably, the apparatus 400 for identifying language of personal names may further include a sorting unit 408 .

The sorting unit 408 can be configured to determine the sorting of each n-gram substring in each person-name unit dictionary according to the weight of each n-gram substring in each person-name unit dictionary, and the recognition unit 406 can further according to the weight of each person-name unit dictionary. The ordering of all n-gram substrings in each person's name unit dictionary is used to identify the language to which the person's name belongs.

Preferably, the sorting unit 408 can further determine the ordering of the n-gram substrings in each person-name unit dictionary according to the descending order of the weights of the n-gram substrings in each person-name unit dictionary, and the recognition unit can further calculate the The sorted sum of all n-elements of substrings in each person name unit dictionary, and the language represented by the person name unit dictionary corresponding to the smallest sum is determined as the language to which the person name belongs.

It should be understood that the personal name language recognition device described with reference to FIG. 4 is a device embodiment corresponding to the above-mentioned personal name language recognition method, so for the content not described in detail in the device embodiment, you can refer to the description of the corresponding position of the above method embodiment, I won't repeat them here.

In addition, it should be pointed out that although an example of the functional configuration of the personal name language recognition device according to the embodiment of the present disclosure has been described above with reference to FIG. The functional modules described in the above embodiments are combined and/or omitted and/or one or more functional modules are added, and such modified examples should be deemed to fall within the scope of the present disclosure.

It should be understood that the machine-executable instructions in the storage medium and the program product according to the embodiments of the present disclosure can also execute the above-mentioned personal name unit dictionary expansion method and personal name language recognition method, so for the content not described in detail here, please refer to the previous corresponding The description of the location will not be repeated here.

Correspondingly, a storage medium for carrying the above-mentioned program product storing machine-executable instructions is also included in the disclosure of the present invention. The storage medium includes, but is not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

In addition, it should also be noted that the series of processes and devices described above may also be implemented by software and/or firmware. In the case of realization by software and/or firmware, a program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware configuration, such as a general-purpose personal computer 500 shown in FIG. , can perform various functions and so on.

In FIG. 5 , a central processing unit (CPU) 501 executes various processes according to programs stored in a read only memory (ROM) 502 or loaded from a storage section 508 to a random access memory (RAM) 503 . In the RAM 503 , data required when the CPU 501 executes various processes and the like is also stored as necessary.

The CPU 501 , ROM 502 , and RAM 503 are connected to each other via a bus 504 . The input/output interface 505 is also connected to the bus 504 .

The following components are connected to the input/output interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; The storage section 508 includes a hard disk and the like; and the communication section 509 includes a network interface card such as a LAN card, a modem, and the like. The communication section 509 performs communication processing via a network such as the Internet.

A drive 510 is also connected to the input/output interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary.

In the case of realizing the above-described series of processes by software, the programs constituting the software are installed from a network such as the Internet or a storage medium such as the removable medium 511 .

Those skilled in the art should understand that such a storage medium is not limited to the removable medium 511 shown in FIG. 5 in which the program is stored and distributed separately from the device to provide the program to the user. Examples of the removable medium 511 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including )) and semiconductor memory. Alternatively, the storage medium may be the ROM 502, a hard disk contained in the storage section 508, or the like, in which the programs are stored and distributed to users together with devices containing them.

It should also be pointed out that the steps for executing the above series of processes can naturally be executed in chronological order according to the illustrated order, but they do not need to be executed in chronological order. Certain steps may be performed in parallel or independently of each other.

Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the terms "comprising", "comprising" or any other variation thereof in the embodiments of the present disclosure are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also Including other elements not expressly listed, or also including elements inherent in such process, method, article or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

According to the embodiments of the present disclosure, the following additional notes are also disclosed:

1. A personal name unit dictionary expansion method, comprising:

The term initial weight calculation step is based on the number of personal name unit dictionaries that each term appears, and calculates the initial weight of each term in the personal name unit dictionary of predetermined multiple languages;

The personal name unit matching step divides each personal name in a plurality of personal names as training samples into a personal name unit, and matches the divided personal name unit with the terms in the personal name unit dictionary of the predetermined plurality of languages, And the matching person name unit is determined as a matching person name unit, and the unmatched person name unit is determined as an unmatched person name unit;

The name weight calculation step is to determine the weights of the names containing the matching name unit in each name unit dictionary according to the weights of the matching person name unit in the person name unit dictionaries of the predetermined multiple languages;

The unmatched personal name unit processing step is to calculate the weight of the unmatched personal name unit in each personal name unit dictionary according to the weight of all names containing the unmatched personal name unit in each personal name unit dictionary, and The unmatched person name unit is added as a term to the name unit dictionary where the matching person name unit in all names containing the unmatched person name unit is located;

The weight updating step of the matching personal name unit, updating the weight of the matching personal name unit in the respective personal name unit dictionaries according to the weights of all names containing the matching personal name unit in the respective personal name unit dictionaries; and

Repeating the processing in the name unit matching step, the name weight calculation step, the unmatched name unit processing step and the matched name unit weight update step until the list of people in the predetermined multiple languages Until the weight changes of all the terms in the meta-dictionary are less than a predetermined threshold, a person-name unit dictionary in which all terms are marked with weights is obtained.

2. According to the method described in Note 1, further comprising:

A normalization step, normalizing the weights of the names calculated in the name weight calculation step in each name unit dictionary,

Wherein, in the step of processing the unmatched person name unit and the step of updating the weight of the matched person name unit, corresponding calculations are performed using the weight of the normalized person name.

3. A name language recognition method, comprising:

The personal name division step divides the input personal name into n-element substrings, wherein the _n- element substrings represent units comprising consecutive n characters in the personal name, and n is an integer greater than or equal to 2;

The n-element substring weight calculation step is to calculate each n-element substring according to the weights of all terms containing each n-element substring in the person name unit dictionary with weight labeling described in Supplementary Note 1 or 2 weights in the individual name unit dictionaries; and

The identifying step is to identify the language to which the personal name belongs according to the weights of all n-gram substrings in the personal name in each personal name unit dictionary.

4. According to the method described in Note 3, further comprising:

Sorting step, according to the weight of each n yuan substring in each person's name unit dictionary, determine the ordering of each n yuan substring in each person's name unit dictionary,

Wherein, in the step of identifying, the language to which the name of the person belongs is identified according to the ordering of all n-gram substrings in the name of the person in each name unit dictionary.

5. according to the method described in supplementary note 4, wherein, in described sorting step, according to the descending order of the weight of described n-element substring in each personal name unit dictionary, determine that described n-element substring is in each person's name unit dictionary ordering in the personal name cell dictionary,

And wherein, in the recognition step, calculate the sum of the sorting of all n-element substrings in the name of the person in each name unit dictionary, and determine the language represented by the smallest sum corresponding to the name unit dictionary is the language to which the person's name belongs.

6. A name language recognition device, comprising:

The personal name division unit is configured to divide the input personal name into n-element substrings, wherein the n-element substrings represent units including consecutive n characters in the personal name, and n is an integer greater than or equal to 2 ;

The n-element substring weight calculation unit is configured to calculate each n-element according to the weights of all terms containing each n-element substring in the person name unit dictionary with weight labeling according to Supplementary Note 1 or 2 the weight of the substring in each person's cell dictionary; and

The identification unit is configured to identify the language to which the person's name belongs according to the weights of all n-gram substrings in the person's name in each person's name unit dictionary.

7. The device according to appendix 6, further comprising:

The sorting unit is configured to determine the ordering of each n-element substring in each person-name unit dictionary according to the weight of each n-element substring in each person-name unit dictionary,

Wherein, the recognition unit further recognizes the language to which the person's name belongs according to the ordering of all n-gram substrings in the person's name in each person's name unit dictionary.

8. The device according to Supplementary Note 7, wherein the sorting unit further determines that the n-gram substrings are included in each name list according to the descending order of the weights of the n-gram substrings in each personal name unit dictionary. ordering in the meta dictionary,

And wherein, the recognition unit further calculates the sum of the sorting sums of all n-element substrings in the person's name in each person's name unit dictionary, and determines the language represented by the person's name unit dictionary corresponding to the smallest sum as the The language of the person's name.

Claims (8)

1. A name unit dictionary expansion method comprises the following steps:

a term initial weight calculation step of calculating initial weights of respective terms in the name unit dictionaries of a predetermined plurality of languages based on the number of name unit dictionaries in which each term appears;

a person name unit matching step, namely dividing each person name in a plurality of person names serving as training samples into a person name unit, matching the divided person name unit with terms in a person name unit dictionary of the preset plurality of languages, determining a matched person name unit as a matched person name unit, and determining an unmatched person name unit as an unmatched person name unit;

a person name weight calculation step of determining weights of the person names including the matching person name units in the respective person name unit dictionaries, based on weights of the matching person name units in the person name unit dictionaries of the predetermined plurality of languages;

an unmatched name unit processing step, which is to calculate the weight of the unmatched name unit in each name unit dictionary according to the weight of all names containing the unmatched name unit in each name unit dictionary, and add the unmatched name unit as a lexical item to the name unit dictionary containing the matched name unit in all names containing the unmatched name unit;

a matching name unit weight updating step of updating the weight of the matching name unit in each name unit dictionary according to the weight of all names including the matching name unit in each name unit dictionary; and

and repeatedly executing the processing in the person name unit matching step, the person name weight calculation step, the unmatched person name unit processing step and the matched person name unit weight updating step until the weight change of all the terms in the person name unit dictionaries of the preset languages is less than a preset threshold value, so as to obtain the person name unit dictionary with all the terms having weight labels.

2. The method of claim 1, further comprising:

a normalization step of normalizing the weight of the name calculated in the name weight calculation step in each name unit dictionary,

and in the unmatched name unit processing step and the matched name unit weight updating step, the normalized weight of the name is used for corresponding calculation.

3. A person name language identification method, comprising:

a person name dividing step of dividing an input person name into n-gram substrings, wherein the n-gram substrings represent units including n consecutive characters in the person name, and n is an integer greater than or equal to 2;

an n-gram substring weight calculation step of calculating the weight of each n-gram substring in each name unit dictionary from the weights of all terms including each n-gram substring in the weight-annotated name unit dictionary according to claim 1 or 2; and

and a recognition step of recognizing the language to which the name belongs according to the weight of all the n-element substrings in the name in each name unit dictionary.

4. The method of claim 3, further comprising:

a sorting step of determining a sorting of each n-gram sub-character string in each name unit dictionary based on a weight of each n-gram sub-character string in each name unit dictionary,

wherein, in the identifying step, the language to which the person name belongs is identified according to the ordering of all n-gram substrings in the person name in each person name unit dictionary.

5. The method according to claim 4, wherein, in the sorting step, the sorting of the n-gram substrings in the respective name unit dictionaries is determined according to a descending order of weights of the n-gram substrings in the respective name unit dictionaries,

and wherein, in the recognizing step, the sum of the rankings of all the n-gram substrings in the name in the respective name unit dictionaries is calculated, and the language represented by the name unit dictionary corresponding to the smallest sum is determined as the language to which the name belongs.

6. A person name language recognition apparatus comprising:

a person name dividing unit configured to divide an input person name into n-gram substrings, wherein the n-gram substrings represent units including n consecutive characters in the person name, and n is an integer greater than or equal to 2;

an n-gram substring weight calculation unit configured to calculate a weight of each n-gram substring in each name unit dictionary from weights of all terms including each n-gram substring in the weight-annotated name unit dictionary according to claim 1 or 2; and

and the identification unit is configured to identify the language to which the name belongs according to the weight of all the n-element substrings in the name in each name unit dictionary.

7. The apparatus of claim 6, further comprising:

a ranking unit configured to determine a ranking of each n-gram sub-character string in each name unit dictionary based on a weight of each n-gram sub-character string in each name unit dictionary,

wherein the recognition unit further recognizes the language to which the name belongs according to the ordering of all n-gram substrings in the name in the respective name unit dictionaries.

8. The apparatus of claim 7, wherein the ranking unit further determines a ranking of the n-gram substrings in the respective name unit dictionaries according to a descending order of their weights in the respective name unit dictionaries,

and wherein the recognition unit further calculates a sum of the rankings of all the n-gram substrings in the name in the respective name unit dictionaries, and determines a language represented by the name unit dictionary corresponding to the smallest sum as a language to which the name belongs.