CN114692892A - Method for processing numerical characteristics, model training method and device - Google Patents
Method for processing numerical characteristics, model training method and device Download PDFInfo
- Publication number
- CN114692892A CN114692892A CN202210285951.2A CN202210285951A CN114692892A CN 114692892 A CN114692892 A CN 114692892A CN 202210285951 A CN202210285951 A CN 202210285951A CN 114692892 A CN114692892 A CN 114692892A
- Authority
- CN
- China
- Prior art keywords
- numerical
- feature
- value
- binary
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Analysis (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
- Error Detection And Correction (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
本说明书实施例描述了对数值特征进行处理的方法、模型训练方法和装置。根据实施例的方法，考虑对数值特征进行二进制编码，然后根据得到的二进制数值特征中的值可以确定出特征矩阵。如此实现了对数值特征可能值都进行了编码，能够降低特征信息损失的可能。进一步，对得到的特征矩阵进行数据域覆盖，能够使得利用该数值特征训练得到的模型在预测应用时，对未参与模型训练的数值输入值也能具有较好的输出。
The embodiments of this specification describe a method for processing numerical features, a model training method, and an apparatus. According to the method of the embodiment, the numerical feature is considered to be binary encoded, and then a feature matrix can be determined according to the value in the obtained binary numerical feature. In this way, all possible values of numerical features are encoded, which can reduce the possibility of loss of feature information. Further, the data domain coverage of the obtained feature matrix can enable the model trained by using the numerical feature to have a better output for numerical input values that are not involved in model training during predictive application.
Description
Technical Field
One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a method for processing numerical features, a model training method, and an apparatus.
Background
There are a large number of numerical features in various scenarios. Such as the amount of the financial transaction, a temperature value over a period of time, etc. Currently, model training by using these numerical features to predict some future phenomena is becoming a common technical means.
However, the numerical features need to be processed when training the model using the numerical features as sample data. In order to ensure that the reliability of the trained model is higher, the processed numerical features should reflect more feature information as much as possible. When the numerical features are processed at present, the situation that a large amount of feature information of the processed numerical features is lost often occurs, so that the numerical features cannot be efficiently utilized when model training is performed.
Disclosure of Invention
One or more embodiments of the present specification describe a method, a model training method, and an apparatus for processing a numerical feature, which can reduce information loss caused when the numerical feature is processed.
According to a first aspect, there is provided a method of processing a numerical feature, comprising:
obtaining numerical characteristics to be processed for model training;
carrying out binary coding on the numerical value characteristics to be processed to obtain binary numerical value characteristics;
determining a feature matrix according to values in the binary numerical feature;
covering the feature matrix in a data domain to obtain a feature vector of the numerical feature to be processed; and the data field is the value range of the input numerical value of the model during application.
In a possible implementation manner, the binary encoding of the to-be-processed numerical feature to obtain a binary numerical feature includes:
carrying out normalization processing on the numerical value features to be processed to obtain normalized numerical value features;
and carrying out binary conversion on the normalized numerical value characteristics to obtain the binary numerical value characteristics.
In a possible implementation manner, the normalizing the to-be-processed numerical feature to obtain a normalized numerical feature includes:
converting the numerical characteristic to be processed into a first numerical characteristic with an integer type;
determining a target numerical range to which the first numerical characteristic belongs from at least two numerical ranges; wherein the at least two numerical ranges are obtained by counting the numerical values of the at least two characteristics;
and determining the normalized numerical characteristic according to the first numerical characteristic and the target numerical range.
In a possible implementation manner, the converting the to-be-processed numerical feature into a first numerical feature of which the type is integer includes:
when the numerical value feature to be processed is a floating point type, multiplying the numerical value feature to be processed of the floating point type by a preset data precision truncation value; wherein, the data precision truncation value is not less than the precision value of the numerical value characteristic to be processed;
and acquiring an integer part in the obtained product to obtain the first numerical characteristic.
In a possible implementation manner, when the first numerical feature is not in any one of the at least two numerical ranges, the determining, from the at least two numerical ranges, a target numerical range to which the first numerical feature belongs includes:
calculating a difference between the first numerical characteristic and each boundary value of the at least two numerical ranges; wherein boundary values include the maximum and minimum values in the numerical range;
determining a first boundary value used when the minimum difference value is calculated;
and determining the numerical range to which the first boundary value belongs as the target numerical range.
In one possible implementation, the determining the normalized numerical characteristic according to the first numerical characteristic and the target numerical range includes:
determining a difference between the first numerical characteristic and a minimum value within the target numerical range as the normalized numerical characteristic when the first numerical characteristic is within the target numerical range;
determining 0 as the normalized numerical feature when the first numerical feature is less than a minimum value within the target numerical range;
when the first numerical characteristic is greater than the maximum value within the target numerical range, determining a difference between the maximum value and the minimum value within the target numerical range as the normalized numerical characteristic.
In one possible implementation, the determining a feature matrix according to the value in the binary-valued feature includes:
determining a binary length n of the binary numerical feature; wherein n is a positive number not less than 1;
determining n first feature vectors according to the value of each binary digit in the binary digit feature; wherein each first feature vector corresponds to a one-bit binary value;
splicing the n first eigenvectors into an n multiplied by m eigenvector matrix; wherein m is the length of the first feature vector.
In a possible implementation manner, the obtaining the feature vector of the numerical feature to be processed by performing data domain coverage on the feature matrix includes:
performing convolution on the feature matrix by using a preset convolution core to obtain at least one second feature vector;
and splicing the at least one second feature vector to obtain the feature vector.
In a possible implementation manner, the preset convolution kernels include at least two convolution kernels, the number of columns of a matrix corresponding to the at least two convolution kernels is the same as the number of columns of the characteristic matrix, and the number of rows of any two convolution kernels is different.
In a possible implementation manner, the stitching the at least one second feature vector to obtain the feature vector includes:
determining a feature representation value for each of said second feature vectors;
and combining the feature expression values of the second feature vectors into a vector, and determining the vector as the feature vector.
In a possible implementation manner, the determining one feature expression value of each of the second feature vectors includes:
for each second feature vector, determining the maximum value in the second feature vector as the feature representation value corresponding to the second feature vector;
and/or the presence of a gas in the atmosphere,
and determining the sum of all numerical values in the second feature vector as the feature representation value corresponding to the second feature vector aiming at each second feature vector.
According to a second aspect, there is provided a model training method comprising:
processing the numerical features for model training by using the method as defined in any one of the first aspect to obtain feature vectors of the numerical features;
and training the model by using the feature vector of the numerical characteristic.
According to a third aspect, there is provided an apparatus for processing a numerical feature, comprising:
the numerical characteristic acquisition module is configured to acquire numerical characteristics to be processed for model training;
the binary coding module is configured to perform binary coding on the numerical value feature to be processed acquired by the numerical value feature acquisition module to obtain a binary numerical value feature;
the characteristic matrix determining module is configured to determine a characteristic matrix according to the value in the binary numerical characteristic obtained by the binary coding module;
the data domain covering module is configured to perform data domain covering on the feature matrix determined by the feature matrix determining module to obtain a feature vector of the numerical feature to be processed; and the data field is the value range of the input numerical value of the model during application.
According to a fourth aspect, there is provided a model training apparatus comprising:
an input module configured to process the numerical features for model training by using the apparatus according to the third aspect, to obtain feature vectors of the numerical features;
and the training module is configured to train the model by using the feature vector of the numerical characteristic obtained by the input module.
According to a fifth aspect, there is provided a computing device comprising: a memory having executable code stored therein, and a processor that, when executing the executable code, implements the method of any of the first and second aspects described above.
According to the method and the device provided by the embodiment of the specification, when the numerical value features are processed, the numerical value features to be processed for model training are firstly obtained, and then binary coding is carried out on the numerical value features to be processed to obtain binary numerical value features. Further, a feature matrix may be determined from the values in the binary valued features. And then covering the data domain of the feature matrix to obtain the feature vector corresponding to the numerical feature to be processed. Therefore, the numerical value characteristics to be processed are coded through the binary system, the characteristic matrix is formed through the values in the binary system, the possible values are coded, and the possibility of characteristic information loss can be reduced. And by further covering the data domain of the characteristic matrix, the model can well express the numerical value which does not participate in the model training when in application, namely the processing method based on the scheme can effectively utilize the numerical characteristic used for the model training.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present specification, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow diagram of a method for processing numerical features provided in one embodiment of the present description;
FIG. 2 is a flow diagram of a method of binary encoding provided by one embodiment of the present description;
FIG. 3 is a flow diagram of a method of normalization processing provided by one embodiment of the present description;
FIG. 4 is a flow chart of a method of determining a target range of values provided by one embodiment of the present description;
FIG. 5 is a flow chart of a feature matrix determination method provided by one embodiment of the present description;
FIG. 6 is a flow diagram of a method of data domain overlay provided by one embodiment of the present description;
FIG. 7 is a schematic diagram of a data field overlay provided by one embodiment of the present description;
FIG. 8 is a flow chart of a method for feature vector determination provided by one embodiment of the present description;
FIG. 9 is a flow diagram of a model training method provided by one embodiment of the present description;
FIG. 10 is a schematic diagram of an apparatus for processing numerical features provided in one embodiment of the present description;
FIG. 11 is a schematic diagram of a model training apparatus provided in an embodiment of the present disclosure.
Detailed Description
As previously mentioned, there are a large number of numerical features in various scenarios. Such as transaction amount and transaction amount in a wind-controlled scenario, temperature and humidity in environmental predictions, mileage and speed in trajectory predictions, etc. In these scenarios, model training is often performed by using the numerical features, so as to predict some future phenomena in the scenario. When the numerical features are used for model training, in order to ensure that the numerical features can be effectively used and that the trained model has higher reliability, the numerical features need to be processed so that the processed numerical features can express more feature information.
At present, a large amount of feature information is lost when numerical features are processed. The numerical features are processed, for example, by normalization or normalization. Since the data is linearly changed to within the specified range, the effective range may be compressed due to some outliers. And if adjusted by a logarithm, this will result in a change of the distribution, and thus some characteristic information will be lost. Moreover, the input of the method into the neural network has only one dimension and limited expressive ability. For another example, when a numerical feature is processed by binning, if the score is thick, the differentiated value may be merged into one bin, thereby resulting in loss of feature information, and if the score is thin, the encoding may be too much.
Based on the technical scheme, the numerical value characteristics are considered to be subjected to binary coding, the number of codes can be greatly reduced by the characteristic that each binary bit occupies two codes, all possible values can be coded, and therefore the possibility of characteristic information loss is greatly reduced.
As shown in fig. 1, embodiments of the present description provide a method for processing a numerical feature, which may include the following steps:
step 101: obtaining numerical characteristics to be processed for model training;
step 103: binary coding is carried out on the numerical value characteristics to be processed to obtain binary numerical value characteristics;
step 105: determining a feature matrix according to values in the binary numerical features;
step 107: covering the data domain of the feature matrix to obtain a feature vector of the numerical feature to be processed; the data field is a value range of an input value of the model during application.
In this embodiment, when the numerical features are processed, to-be-processed numerical features for model training may be obtained first, and then binary coding may be performed on the to-be-processed numerical features to obtain binary numerical features. Further, a feature matrix may be determined from the values in the binary valued features. Therefore, the feature vector corresponding to the numerical characteristic to be processed can be obtained by covering the data domain of the feature matrix, so that the input numerical value of the model trained by utilizing the numerical characteristic can have a wider value range in application.
Therefore, the numerical value characteristics to be processed are coded through the binary system, and the characteristic matrix is generated according to the binary system values. That is, possible values of the numerical features are encoded, so that the possibility of loss of feature information can be reduced. And through binary coding, the number of codes can be greatly reduced by utilizing the characteristic that two codes occupy one binary bit. In addition, by further covering the data field of the feature matrix, the model can well express the numerical value which does not participate in the model training when in application, namely, the processing method based on the scheme can effectively utilize the numerical value feature used for the model training.
The steps in fig. 1 are described below with reference to specific examples.
In step 101, a to-be-processed numerical feature for model training is obtained.
As described above, there are a large number of numerical features in each scene, and these numerical features are often used for model training to predict some future phenomena in the scene. For example, in a wind control scenario, when a model for predicting risk transactions is trained, the numerical features used for model training may be transaction amount, transaction frequency, and the like in historical transactions. For another example, when training a model for predicting an environment that has not been used for a certain period of time, the numerical features used for model training may be temperature, humidity, and the like in the historical environmental information. As another example, in training a model for predicting certain motion profiles, the numerical characteristics may include mileage, speed, etc. in historical motion profile data.
In step 103, binary coding is performed on the numerical features to be processed to obtain binary numerical features.
The number of encodings may be large due to direct encoding with decimal values. Thus, the present embodiment contemplates encoding the numerical features to be processed using binary. In this way, the number of codes can be reduced, and all possible values of the numerical characteristic can be coded when the number of codes is low, thereby reducing the possibility of losing the characteristic information.
As shown in fig. 2, in one possible implementation, the step 103 may include the following steps when performing binary encoding:
step 201: carrying out normalization processing on the numerical value features to be processed to obtain normalized numerical value features;
step 203: and carrying out binary conversion on the normalized numerical characteristics to obtain binary numerical characteristics.
In this embodiment, when binary coding is performed on the numerical feature to be processed, firstly, normalization processing may be performed on the numerical feature to be processed to obtain a normalized numerical feature, and then binary conversion is performed on the normalized numerical feature to obtain a binary numerical feature. Therefore, by encoding the numerical value features to be processed by using the binary system, more complete information of the numerical value features to be processed can be expressed by lower encoding quantity.
In step 201, the numerical feature to be processed is normalized to obtain a normalized numerical feature.
It is contemplated that in many cases, there may be a wide variety of data types for the numerical features to be processed. For example, the types of numerical features may include: integer, real, floating point, etc. Also, the size of the numerical features may be large or small, which increases the complexity of the encoding. Therefore, in this step, normalization processing is performed on the numerical value features to be processed, and the numerical value features to be processed are expressed in a brief introduction manner.
For example, in one possible implementation, as shown in fig. 3, step 201 may include the following steps:
step 301: converting the numerical characteristics to be processed into first numerical characteristics of which the types are integer;
step 303: determining a target numerical range to which the first numerical characteristic belongs from at least two numerical ranges; wherein, the at least two numerical ranges are obtained by counting the numerical values of the at least two characteristics;
step 305: and determining the normalized numerical characteristic according to the first numerical characteristic and the target numerical range.
In this embodiment, when performing normalization processing on a to-be-processed numerical feature, firstly, the to-be-processed numerical feature is converted into a first numerical feature of which the type is integer, then a target numerical range to which the first numerical feature belongs is determined, and then the first numerical feature and the target numerical range are utilized to determine the normalized numerical feature. Therefore, the numerical value features to be processed are uniformly converted into integer, so that the numerical value features to be processed are more favorably encoded, and the encoding difficulty cannot be increased due to factors such as decimal points. Further, by determining the target numerical range of the first numerical characteristic, the target numerical range may be utilized to perform normalization processing on the size of the numerical characteristic to be processed. So that the complexity of the encoding is not increased by the numerical features being too large or too small, and the differences between the numerical values being too large.
In step 301, the numerical feature to be processed is converted into a first numerical feature of which the type is integer.
There are many types of data that take into account numerical characteristics, such as integer, real, floating point, etc. When processing various types of numerical features, the complexity of encoding the numerical features increases. For example, when the numerical features of the floating-point type are binary-coded subsequently, the binary is divided into two parts due to the existence of the decimal point, which undoubtedly increases the difficulty of subsequent processing. Based on this, in this step, the numerical feature to be processed is considered to be uniformly converted into the first numerical feature of which the type is integer, so that the integer numerical feature can be simpler in subsequent encoding and processing.
For example, when the numerical feature to be processed is an integer, the integer numerical feature to be processed is determined as the first numerical feature without being converted. For another example, when the to-be-processed numerical feature is a floating point type, it may be considered to multiply the to-be-processed numerical feature of the floating point type by a preset data precision cutoff value, and then obtain an integer part in the obtained product to obtain the first numerical feature.
It should be noted that the data precision cutoff value may be determined according to the precision value of the numerical feature to be processed, for example, the data precision cutoff value may be not less than the precision value of the numerical feature to be processed, so as to ensure the precision of the numerical feature to be processed. For example, float32 is used for common floating point type, i.e. the significant number is 6-7 bits, which can ensure 7 bits in total. So multiplied by 1 × 107Post truncation can ensure data accuracy. As another example, for some monetary classes of features, it is usually accurate to two decimal places, i.e., to "cents". For example, an amount of 45.23 yuan, it can be multiplied by 1 × 102I.e. it can be converted into an integer.
Of course, in some possible embodiments, when the numerical feature to be processed is multiplied by the data-precision cutoff value, the decimal point still exists, and it may be considered that the integer part obtained by the multiplication is directly determined as the first numerical feature, or the part after the decimal point is rounded to the integer.
In step 303, a target value range to which the first value characteristic belongs is determined from the at least two value ranges.
Before processing the numerical features, the data ranges of the respective features are counted to obtain at least two numerical ranges. Each different numerical range may correspond to a numerical characteristic. For example, the range of values corresponding to the characteristics of the transaction amount may be (0,100000), and the range of values corresponding to the characteristics of the rate class may be (5, 92). Thus, when the first numerical characteristic is determined, the numerical range in which the first numerical characteristic is located can be determined from the determined at least two numerical ranges.
However, since it is not guaranteed that all values of a feature are contained in determining a numerical range, there may be a case where the first numerical feature is not within at least two numerical ranges determined statistically in advance. Based on this, as shown in fig. 4, when the first numerical characteristic is not in any one of the at least two numerical ranges, step 303 may determine the target numerical range by:
step 401: calculating a difference between the first numerical characteristic and each boundary value of the at least two numerical ranges; wherein boundary values include the maximum and minimum values in the numerical range;
step 403: determining a first boundary value used when the minimum difference value is calculated;
step 405: and determining the numerical range to which the first boundary value belongs as a target numerical range.
When the first numerical characteristic is not within the predetermined at least two numerical ranges, first calculating a difference between the first numerical characteristic and each boundary value of the at least two numerical ranges, and then calculating a first boundary value used in calculating a minimum difference, and further determining the numerical range to which the first boundary value belongs as the target numerical range.
For example, the predetermined value ranges are (0,100000), (2, 94), (300,9000), (50,20000), and the first value characteristic is 500. Then, if the difference between the boundary values of the four numerical ranges and 500 is the smallest value of 300 of the third numerical range, the target numerical range of the first numerical characteristic is assumed to be (300,9000).
In step 305, a normalized numerical characteristic is determined based on the first numerical characteristic and the target numerical range.
After the first numerical characteristic and the target numerical range are determined, a normalized numerical characteristic may be determined based thereon. Of course, to ensure that outliers are not compressed and the distribution of values is not altered, normalization may be performed taking into account differences from boundary values in the target value range. For example, when the determined first numerical characteristic is within the target numerical range, the difference between the first numerical characteristic and the minimum value within the target numerical range may be determined as the normalized numerical characteristic. This ensures that outliers of the numerical features are not compressed and the distribution of the values is not altered. For example, if the target value range of a feature is (5,100), and the obtained plurality of first numerical features are 30, 6, 88, respectively, then by making a difference with the minimum value of the target value range, the normalized numerical features of the respective first numerical features are 25, 1, 83, respectively.
However, in some cases, the statistically derived range of values does not completely cover the individual values of the feature, and therefore the first possible numerical feature is not within the target range of values for the feature. The normalized numerical characteristic may be determined at this time in consideration of the relationship of the first numerical characteristic to the minimum value and the maximum value within the target numerical range.
For example, when the first numerical feature is smaller than the minimum value in the target numerical range, the first numerical feature may be assigned with the minimum value in the target numerical range, that is, the updated first numerical feature having a value equal to the minimum value in the target numerical range is obtained. And then, the first numerical characteristic is used for making a difference value with the minimum value in the target numerical range to obtain a normalized numerical characteristic, wherein the normalized numerical characteristic is 0 at the moment. For example, if the first numerical characteristic is 35 and the target numerical range to which the characteristic of the first numerical characteristic corresponds is (40,50), the first numerical characteristic may be assigned the minimum value 40 of the target numerical range and a normalized numerical characteristic of 0 may be obtained.
For another example, when the first numerical feature is greater than the maximum value within the target numerical range, the first numerical feature may be assigned with the maximum value within the target numerical range, i.e., the updated first numerical feature having a value equal to the maximum value within the target numerical range is obtained. And then, the first numerical characteristic is used for making a difference value with the minimum value in the target numerical range to obtain a normalized numerical characteristic. I.e. the difference between the maximum and minimum values within the target value range is determined as a normalized numerical characteristic. For example, if the first numerical characteristic is 52 and the target numerical range to which the characteristic of the first numerical characteristic corresponds is (40,50), the first numerical characteristic may be assigned the maximum value 50 of the target numerical range and the normalized numerical characteristic 10 may be obtained.
In step 203, the normalized numerical features are subjected to binary conversion to obtain binary numerical features.
In this step, the normalized numerical features are considered to be subjected to binary conversion. For example, when the normalized numerical feature is 5, the binary conversion is 0101; for another example, when the normalized numerical feature is 10, the binary conversion is 1010. Since the direct use of decimal values as codes results in an excessive number of codes, each binary bit takes up 2 codes after conversion to binary. For example, assuming 30 binary bits are used, 30 × 2-60 codes can express 2 × 230Approximately 21 hundred million different numbers are sufficient to satisfy the dynamic range of the generic feature. In practical application, the codes are combined by less than 60 codes in many cases, so that the reduction can be performed for different feature ranges.
In step 105, a feature matrix is determined from the values in the binary valued features.
Since the value of each bit in the binary system is composed of 0 and 1. Thus, before numerical feature processing, a different vector may be defined for whether each bit in the binary is a 0 or a 1. Thus, when the numerical features are processed, the corresponding vector can be selected according to the specific value of each bit. For example, for a 4-bit binary xxx, each bit corresponds to two vectors, and to 0 and 1 in the binary, respectively. I.e. the predefined matrix of the binary is:
a, B, C, D is the vector corresponding to each bit value being 1 in 4-bit binary system, and a, b, c, d are the vectors corresponding to each bit value being 0 in 4-bit binary system.
Thus, after the binary numerical value characteristic is obtained, the characteristic matrix can be determined according to the value of each bit of the specific binary system.
For example, as shown in fig. 5, in one possible implementation, step 105 may be implemented by:
step 501: determining a binary length n of a binary numerical feature; wherein n is a positive number not less than 1;
step 503: determining n first feature vectors according to the value of each binary digit in the binary digit feature; wherein each first feature vector corresponds to a one-bit binary value;
step 505: splicing the n first eigenvectors into an n multiplied by m eigenvector matrix; where m is the length of the first feature vector.
In this embodiment, when determining the feature matrix according to the value in the binary value feature, the binary length of the binary value feature may be determined, that is, how many bits of the binary value feature is. Based on the above, each bit corresponds to two vectors according to whether the bit is 0 or 1, and then the vector corresponding to each bit can be obtained according to the specific binary value characteristics. And further splicing the vectors corresponding to each binary system to obtain a feature matrix.
For example, in the above example, a 4-bit binary bit 0110, the vectors corresponding to each bit of the binary bit are: a. b, C, d are provided. Further, the obtained vectors corresponding to each binary digit are spliced to obtain a 4 x m matrix [ a, B, C, d ]]T。
It should be noted that m is the length of the vector corresponding to each binary bit, and can be set to be 8, 16, 32, etc.
In step 107, data domain coverage is performed on the feature matrix to obtain a feature vector of the numerical feature to be processed.
In consideration of the fact that the numerical features to be processed for model training may not cover all values, that is, after model training is performed by using the numerical features to be processed, the prediction output effect of some values may not be good enough when the model is used. Therefore, data coverage is carried out on the feature matrix corresponding to the numerical features to be processed, so that the trained model can have better expression on the numerical features without training.
As shown in fig. 6, when the step 107 performs data domain coverage on the feature matrix to obtain the feature vector of the numerical feature to be processed, the following steps may be implemented:
step 601: performing convolution on the feature matrix by using a preset convolution kernel to obtain at least one second feature vector;
step 603: and splicing the at least one second feature vector to obtain a feature vector.
In this embodiment, when the feature matrix is covered in the data domain, the feature matrix may be convolved by using a preset convolution kernel to obtain at least one second feature vector, and then the obtained second feature vectors are spliced to obtain the feature vector of the numerical feature to be processed. Thus, by convolving the feature matrix, smoothing is achieved. Namely, the values among the numerical characteristics can be more continuous, and the coverage of the data domain can be realized.
Step 601 will be explained.
In a possible implementation manner, when the feature matrix is convolved by using the preset convolution kernel in step 601, the convolution kernel may include at least two convolution kernels, the number of columns of the matrices corresponding to the at least two convolution kernels is the same as the number of columns of the feature matrix, and the number of rows of the matrices corresponding to any two different convolution kernels is different. For example, as shown in fig. 7, in the schematic diagram of covering the data field, the width of the convolution kernels 1 and 2 is equal to the width of the feature matrix, while the heights of the convolution kernels 1 and 2 are different, so that the smoothing effect can be improved by setting different heights of the convolution kernels.
Step 603 will be explained.
When the obtained at least one second feature vector is spliced to obtain the feature vector, one second feature vector can be represented by one feature representation value, so that the second feature vector corresponding to each digit can be simplified, and the binary system can be fully expressed. For example, in one possible implementation, as shown in fig. 8, step 603 may include the following steps:
step 801: determining a feature representation value for each second feature vector;
step 802: the feature expression values of the second feature vectors are combined into a vector, and the vector is determined as a feature vector.
For example, in step 801, when determining one feature expression value of each second feature vector, for each second feature vector, a maximum value in the second feature vector may be determined as a feature expression value corresponding to the second feature vector; for another example, for each second feature vector, the sum of each numerical value in the second feature vector may be determined as the feature expression corresponding to the second feature vector; for another example, for each second feature vector, an average value of respective numerical values included in the second feature vector may be determined as a performance value corresponding to the second feature vector.
Based on the schematic diagram of determining the feature vector shown in fig. 7, for each convolution kernel, a feature vector is sequentially slipped from the feature matrix, and each convolution kernel obtains a feature vector, such as a second feature vector 1, a second feature vector 2, a second feature vector 3, and a second feature vector 4 (illustrated as the second feature vector 1 and the second feature vector 4 in the figure). Further, for the second feature vector obtained by each convolution kernel, the maximum value in each second feature vector may be determined as the feature representation value corresponding to the second feature vector. And combining the characteristic expression values into a vector to obtain a characteristic vector of the numerical characteristic to be processed.
As shown in fig. 9, embodiments of the present specification provide a model training method, which may include the following steps:
step 901: processing the numerical features for model training by using the method for processing the numerical features provided by any one of the embodiments to obtain feature vectors of the numerical features;
step 903: and training the model by using the feature vector of the numerical feature.
In this embodiment, before performing model training, it may be considered that the numerical features are processed by using the method for processing numerical features provided in each embodiment of the present description, and then the model is trained by using the obtained feature vector corresponding to the numerical features. The numerical characteristic processing method provided by the scheme can reduce the loss of characteristic information, so that the numerical characteristic can be efficiently utilized in model training, and the reliability of the trained model is higher.
In addition, the method for processing the numerical characteristics provided by the scheme realizes the coverage of the data domain, so that the numerical characteristics which are not trained can be well expressed when the numerical characteristics are used as input to predict and output.
As shown in fig. 10, an embodiment of the present specification provides an apparatus for processing a numerical feature, which may include:
a numerical feature acquisition module 1001 configured to acquire numerical features to be processed for model training;
a binary coding module 1002, configured to perform binary coding on the to-be-processed numerical value feature acquired by the numerical value feature acquisition module 1001 to obtain a binary numerical value feature;
a feature matrix determination module 1003 configured to determine a feature matrix according to a value in the binary numerical feature obtained by the binary encoding module 1002;
a data domain covering module 1004 configured to perform data domain covering on the feature matrix determined by the feature matrix determining module 1003 to obtain a feature vector of the numerical feature to be processed; the data field is a value range of an input value of the model during application.
In one possible implementation, when binary-coding the to-be-processed numerical value feature to obtain the binary value feature, the binary-coding module 1002 is configured to perform the following operations:
carrying out normalization processing on the numerical value features to be processed to obtain normalized numerical value features;
and carrying out binary conversion on the normalized numerical characteristics to obtain binary numerical characteristics.
In one possible implementation, when the to-be-processed numerical feature is normalized to obtain a normalized numerical feature, the binary encoding module 1002 is configured to perform the following operations:
converting the numerical characteristics to be processed into first numerical characteristics of which the types are integer;
determining a target numerical range to which the first numerical characteristic belongs from at least two numerical ranges; wherein, the at least two numerical ranges are obtained by counting the numerical values of the at least two characteristics;
and determining the normalized numerical characteristic according to the first numerical characteristic and the target numerical range.
In one possible implementation, the binary encoding module 1002, when converting the numerical feature to be processed into the first numerical feature of type integer, is configured to perform the following operations:
when the numerical value feature to be processed is a floating point type, multiplying the numerical value feature to be processed of the floating point type by a preset data precision truncation value; wherein, the data precision truncation value is not less than the precision value of the numerical characteristic to be processed;
an integer portion of the resulting product is obtained, resulting in a first numerical characteristic.
In one possible implementation, when the first numerical feature is not in any one of the at least two numerical ranges, the binary encoding module 1002, when determining the target numerical range to which the first numerical feature belongs from the at least two numerical ranges, is configured to perform the following operations:
calculating a difference between the first numerical characteristic and each boundary value of the at least two numerical ranges; wherein boundary values include the maximum and minimum values in the numerical range;
determining a first boundary value used when the minimum difference value is calculated;
and determining the numerical range to which the first boundary value belongs as a target numerical range.
In one possible implementation, the binary encoding module 1002, in determining the normalized numerical characteristic from the first numerical characteristic and the target numerical range, is configured to perform the following operations:
when the first numerical characteristic is in the target numerical range, determining the difference value between the first numerical characteristic and the minimum value in the target numerical range as a normalized numerical characteristic;
when the first numerical characteristic is less than the minimum value within the target numerical range, determining 0 as a normalized numerical characteristic;
when the first numerical characteristic is greater than the maximum value within the target numerical range, the difference between the maximum value and the minimum value within the target numerical range is determined as a normalized numerical characteristic.
In one possible implementation, the feature matrix determination module 1003, when determining the feature matrix according to the values in the binary-valued feature, is configured to perform the following operations:
determining a binary length n of a binary numerical characteristic; wherein n is a positive number not less than 1;
determining n first feature vectors according to the value of each binary digit in the binary digit feature; wherein each first feature vector corresponds to a one-bit binary value;
splicing the n first eigenvectors into an n multiplied by m eigenvector matrix; where m is the length of the first feature vector.
In one possible implementation manner, when the data domain coverage module 1004 performs data domain coverage on the feature matrix to obtain a feature vector of the numerical feature to be processed, the data domain coverage module is configured to perform the following operations:
performing convolution on the feature matrix by using a preset convolution core to obtain at least one second feature vector;
and splicing the at least one second feature vector to obtain a feature vector.
In a possible implementation manner, when the data domain covering module 1004 performs convolution on the feature matrix by using a preset convolution kernel, the preset convolution kernel includes at least two convolution kernels, the number of columns of the matrix corresponding to the at least two convolution kernels is the same as the number of columns of the feature matrix, and the number of rows of any two convolution kernels is different.
In one possible implementation, when splicing the at least one second feature vector to obtain a feature vector, the data field covering module 1004 is configured to:
determining a feature representation value for each second feature vector;
the feature expression values of the second feature vectors are combined into a vector, and the vector is determined as a feature vector.
In one possible implementation, the data field covering module 1004, in determining one feature representation value for each second feature vector, is configured to perform the following operations:
for each second feature vector, determining the maximum value in the second feature vector as the feature representation value corresponding to the second feature vector;
and/or the presence of a gas in the atmosphere,
and determining the sum of all numerical values in the second feature vector as the feature representation value corresponding to the second feature vector aiming at each second feature vector.
As shown in fig. 11, embodiments of the present description further provide a model training apparatus, which may include:
an input module 1101 configured to process the numerical features used for model training by using the apparatus for processing numerical features provided in the foregoing embodiments, so as to obtain feature vectors of the numerical features;
a training module 1102 configured to train the model using the feature vectors of the numerical features obtained by the input module 1101.
The present specification also provides a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any of the embodiments of the specification.
The present specification also provides a computing device comprising a memory and a processor, the memory having stored therein executable code, which when executed by the processor, implements the method of any of the embodiments of the specification.
It should be understood that the schematic structure of the embodiments in this specification does not constitute a specific limitation to the apparatus for processing numerical features and the model training apparatus. In other embodiments of the specification, the means for processing numerical features and the model training means may comprise more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
For the information interaction, execution process, and other contents between the units in the apparatus, the specific contents may refer to the description in the method embodiment of the present specification because the same concept is based on the method embodiment of the present specification, and are not described herein again.
Those skilled in the art will recognize that in one or more of the examples described above, the functions described in this specification can be implemented in hardware, software, hardware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
The above-mentioned embodiments, the purpose, technical solutions and advantages described in the present specification are further described in detail, it should be understood that the above-mentioned embodiments are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (15)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285951.2A CN114692892B (en) | 2022-03-23 | 2022-03-23 | Method for processing numerical features, model training method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285951.2A CN114692892B (en) | 2022-03-23 | 2022-03-23 | Method for processing numerical features, model training method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114692892A true CN114692892A (en) | 2022-07-01 |
| CN114692892B CN114692892B (en) | 2023-08-29 |
Family
ID=82138984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210285951.2A Active CN114692892B (en) | 2022-03-23 | 2022-03-23 | Method for processing numerical features, model training method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114692892B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115660045A (en) * | 2022-10-17 | 2023-01-31 | 华为技术有限公司 | Model training method and device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0566513A1 (en) * | 1992-04-17 | 1993-10-20 | Innovatron S.A. | Method to debit a portable object with a number of amounts; portable object, transfer device and corresponding application |
| CN109002839A (en) * | 2018-06-22 | 2018-12-14 | 杭州电子科技大学 | An efficient feature selection method in multi-attribute feature environment |
| US20190303569A1 (en) * | 2017-06-16 | 2019-10-03 | Alibaba Group Holding Limited | Data type recognition, model training and risk recognition methods, apparatuses and devices |
| CN113159192A (en) * | 2021-04-26 | 2021-07-23 | 南京航空航天大学 | Multi-element time sequence retrieval method and system |
| WO2021204763A1 (en) * | 2020-04-07 | 2021-10-14 | Koninklijke Philips N.V. | Training a convolutional neural network |
| US20220083654A1 (en) * | 2019-01-09 | 2022-03-17 | British Telecommunications Public Limited Company | Anomalous behavior detection in a distributed transactional database |
-
2022
- 2022-03-23 CN CN202210285951.2A patent/CN114692892B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0566513A1 (en) * | 1992-04-17 | 1993-10-20 | Innovatron S.A. | Method to debit a portable object with a number of amounts; portable object, transfer device and corresponding application |
| US20190303569A1 (en) * | 2017-06-16 | 2019-10-03 | Alibaba Group Holding Limited | Data type recognition, model training and risk recognition methods, apparatuses and devices |
| CN109002839A (en) * | 2018-06-22 | 2018-12-14 | 杭州电子科技大学 | An efficient feature selection method in multi-attribute feature environment |
| US20220083654A1 (en) * | 2019-01-09 | 2022-03-17 | British Telecommunications Public Limited Company | Anomalous behavior detection in a distributed transactional database |
| WO2021204763A1 (en) * | 2020-04-07 | 2021-10-14 | Koninklijke Philips N.V. | Training a convolutional neural network |
| CN113159192A (en) * | 2021-04-26 | 2021-07-23 | 南京航空航天大学 | Multi-element time sequence retrieval method and system |
Non-Patent Citations (1)
| Title |
|---|
| 周俊;曹琦;王帅;: "基于虹膜特征的密钥生成研究", 计算机工程与应用, no. 21 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115660045A (en) * | 2022-10-17 | 2023-01-31 | 华为技术有限公司 | Model training method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114692892B (en) | 2023-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12225196B2 (en) | Image compression method and apparatus thereof | |
| CN108337000B (en) | Automatic method for converting to lower precision data formats | |
| CN107729989B (en) | A device and method for performing forward operation of artificial neural network | |
| CN109840589B (en) | Method and device for operating convolutional neural network on FPGA | |
| KR102774102B1 (en) | A method for processing artificial neural network and electronic device therefor | |
| US20190164043A1 (en) | Low-power hardware acceleration method and system for convolution neural network computation | |
| US11574239B2 (en) | Outlier quantization for training and inference | |
| CN110880038A (en) | System for accelerating convolution calculation based on FPGA and convolution neural network | |
| CN112085175A (en) | Data processing method and device based on neural network calculation | |
| EP3816866B1 (en) | Operation method and apparatus for network layer in deep neural network | |
| CN110874625B (en) | A data processing method and device | |
| CN111967608A (en) | Data processing method, device, equipment and storage medium | |
| CN118796471B (en) | Reasoning resource optimization method, device and electronic equipment | |
| US20230351180A1 (en) | Quantization recognition training method of neural network that supplements limitations of gradient-based learning by adding gradient-indipendent update | |
| CN110503182A (en) | Network layer operation method and device in deep neural network | |
| CN114730331B (en) | Data processing device and data processing method | |
| CN114692892A (en) | Method for processing numerical characteristics, model training method and device | |
| CN111126501B (en) | Image identification method, terminal equipment and storage medium | |
| US12488220B2 (en) | Data processing method and device, and computer-readable storage medium | |
| CN117273092B (en) | A model quantization method, apparatus, electronic device, and storage medium | |
| CN116841727A (en) | Resource allocation methods, processors, chips and electronic devices | |
| CN115841136A (en) | Neural network quantization method, device, chip, board card and equipment | |
| CN109308194B (en) | Method and apparatus for storing data | |
| CN116384445B (en) | Processing method and related device of neural network model | |
| CN116959489B (en) | Speech model quantification method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 310000 Zhejiang Province, Hangzhou City, Xihu District, Xixi Road 543-569 (continuous odd numbers) Building 1, Building 2, 5th Floor, Room 518 Patentee after: Alipay (Hangzhou) Digital Service Technology Co.,Ltd. Country or region after: China Address before: 310000 801-11 section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province Patentee before: Alipay (Hangzhou) Information Technology Co., Ltd. Country or region before: China |
|
| CP03 | Change of name, title or address |