Disclosure of Invention
In order to solve the problem of poor recognition accuracy of the existing plankton recognition model, the embodiment of the disclosure provides a plankton recognition model construction method and a plankton category recognition method.
In a first aspect, an embodiment of the present disclosure provides a method for constructing a plankton recognition model, including:
Extracting outlines of plankton sample images in sample data pairs to obtain outline features of planktons contained in the plankton sample images, wherein each plankton sample image comprises one type of planktons, and the sample data pairs comprise plankton type tags which are in one-to-one association with the plankton sample images in addition to the plankton sample images;
Clustering the profile features to obtain a plurality of profile feature clustering sets, and dividing the sample data pairs based on profile feature clusters corresponding to the plankton sample images to obtain sample data pairs corresponding to the profile feature clustering sets;
And respectively adopting sample data corresponding to the contour feature cluster sets to carry out model training to obtain plankton identification models corresponding to the contour feature cluster sets.
Optionally, before contour extraction of the plankton sample image in the sample data pair, the method further comprises:
Editing the plankton sample image to obtain an edited plankton sample image, and constructing a new sample data pair based on the edited plankton sample image and a corresponding plankton type label;
Wherein the editing process includes at least one of flipping, rotating, scaling within a set scale range, random cropping, random shifting, pixel occlusion, the edited plankton sample image still including pixel content characterizing plankton.
Optionally, the contour extraction of the plankton sample image in the sample data pair to obtain contour features of plankton contained in the plankton sample image includes:
Performing fuzzy processing on the plankton sample image to obtain a fuzzy processed sample image, wherein the fuzzy processing is used for eliminating high-frequency noise in the plankton sample image;
Performing convolution processing on the sample image subjected to the blurring processing by adopting a preset edge detection convolution check to obtain gradient characteristics of each pixel point of the plankton sample image;
plankton profile features are determined based on the gradient features.
Optionally, the extracted features obtained by performing convolution processing on the sample image subjected to the blurring processing include gradient magnitude and gradient direction;
the determining an image profile of the plankton sample image based on the gradient features includes:
performing non-maximum suppression based on the gradient magnitude and the gradient direction, and determining a local gradient maximum;
Processing the local gradient maximum value by adopting a preset double threshold value, and determining a definite maximum value and a candidate maximum value in the local gradient maximum value;
Screening the candidate maximum value based on the proximity relation between the candidate maximum value and the pixel corresponding to the confidence maximum value to obtain a reserved candidate maximum value;
And taking the pixel points corresponding to the confidence maximum and the retention candidate maximum as the image contour of the plankton sample image.
Optionally, the clustering processing is performed on the profile features to obtain a plurality of profile feature aggregation sets, including:
Performing feature analysis on the contour features to determine feature vectors corresponding to the contour features;
And carrying out cluster analysis on the feature vectors according to the preset cluster quantity, determining a cluster set corresponding to each feature vector, and taking the cluster set corresponding to the feature vector as a contour feature cluster set in which the contour features are located.
In a second aspect, an embodiment of the present disclosure provides a plankton class identification method, including:
extracting the outline of the plankton image to be identified to obtain outline characteristics of plankton;
determining a target feature cluster set from a plurality of profile feature cluster sets based on profile features of the plankton;
And processing the plankton image to be identified by adopting a plankton identification model associated with the target feature cluster set, and determining the category of plankton.
Optionally, the extracting the outline of the plankton image to be identified to obtain the outline characteristics of the plankton includes:
determining the number of plankton included in the plankton image to be identified;
Under the condition that the number of plankton is one, extracting the outline of the plankton image to be identified to obtain outline characteristics of the plankton;
In the case that the number of plankton is a plurality, cutting the plankton image to be identified to obtain a cut image, wherein the cut image is an image comprising single plankton, and
And extracting the contour of the segmented image to obtain the contour characteristics of plankton.
Optionally, the determining the target feature cluster set in a plurality of profile feature cluster sets based on the profile features of the plankton includes:
Performing feature analysis on the contour features to determine feature vectors corresponding to the contour features;
And obtaining the distance between the feature vector and the center point of the preset feature cluster sets, and selecting the feature distance set closest to the feature vector as the target feature cluster set.
In a third aspect, embodiments of the present disclosure provide a computing device comprising a processor and a memory for storing a computer program which, when loaded by the processor, causes the processor to perform a method of constructing a zoobiological recognition model as described above or a plankton class recognition method as described above.
In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform a method of constructing a zoobiological recognition model as described above or a plankton class recognition method as described above.
According to the embodiment of the disclosure, based on the problem that plankton of different types possibly have the same outline characteristics, the outline characteristics of the same plankton have different outline characteristics in different growth stages, and the outline characteristics of the plankton in a sample image can negatively influence the plankton identification model training, outline extraction and cluster analysis are firstly carried out on the plankton image in the sample data, the sample data pairs containing different plankton outline characteristics are divided into different cluster sets, and then model training is carried out by adopting sample data corresponding to each cluster set, so that a corresponding plankton identification model is obtained. Because the plankton profile similarity in the plankton sample image used for the plankton recognition model training is very high, the influence degree of the plankton profile on model parameters is reduced when a single plankton recognition model is trained, and the accuracy of each plankton recognition model is improved by adjusting parameters through other features except profile features.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment," another embodiment "means" at least one additional embodiment, "and" some embodiments "means" at least some embodiments. Related definitions of other terms will be given in the description below. Relational terms such as "first" and "second", and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
In order to solve the problem that the recognition accuracy of a plankton recognition model obtained by training is poor due to the fact that different appearance forms of plankton of different types are similar and different appearance forms exist in different growth stages of a single plankton, the embodiment of the disclosure provides a novel plankton recognition model construction method. The plankton identification model construction method provided by the embodiment of the disclosure is executed by the computing equipment.
Fig. 1 is a flowchart of a method for constructing a plankton recognition model according to an embodiment of the present disclosure. As shown in fig. 1, the plankton recognition model construction method provided by the embodiment of the present disclosure includes S110 to S130.
S110, contour extraction is carried out on the plankton sample image in the sample data pair, so that contour features of plankton contained in the plankton sample image are obtained.
The plankton sample image is an image obtained by photographing a water body sample containing plankton with a microscopic camera. In practical application, because the size of plankton is very small, a microscopic image obtained by shooting a water body sample at a time may contain a plurality of types of plankton samples, which causes trouble in labeling and subsequent classification of the plankton samples, and the plankton sample image can be subjected to segmentation processing to obtain a sample image with a smaller size containing only one plankton.
The sample data pair includes plankton type tags in one-to-one association with plankton sample images in addition to plankton sample images. The plankton type tag is a tag obtained by manually identifying a plankton sample image, and is used for labeling the type of plankton contained in the plankton sample image. In particular embodiments, plankton-type tags may include the name of the plankton, and may also include the family, genus, and species to which the plankton belongs.
After obtaining the plankton sample image, the computing device may perform contour extraction on the plankton sample image to obtain contour features including plankton therein. The outline features of plankton are features for representing the outline size and the outline shape of plankton, and the outline features of plankton can be represented by lines.
In some embodiments, the computing device may derive plankton profile features contained in the plankton sample image using S111-S113 as follows.
And S111, blurring processing is carried out on the plankton sample image, and a sample image subjected to blurring processing is obtained.
In particular implementations, the computing device may blur the plankton sample image using, for example, a gaussian blur algorithm, resulting in a blurred sample image. The blurring process is to eliminate high-frequency noise in the plankton sample image, so as to avoid the influence of high frequency on the subsequent image contour recognition, that is, avoid unnecessary (or wrong) contour edges from being recognized by the image noise.
And S112, carrying out convolution processing on the sample image subjected to the fuzzy processing by adopting a preset edge detection convolution check to obtain gradient characteristics of each pixel point of the plankton sample image.
The preset edge detection convolution kernel is a convolution kernel for identifying edge pixels in the blurred sample image. According to the principle of edge recognition, the edge of an image is a high-frequency variation component in the image, and a corresponding preset edge detection convolution kernel is a convolution kernel for realizing high-frequency feature recognition.
In specific implementation, the preset edge detection convolution kernel may be a Roberts operator convolution kernel, a Canny operator convolution kernel, a Prewitt operator convolution kernel, a Sobel operator convolution kernel, or the like, and the embodiment of the disclosure is not limited.
In one embodiment, the preset edge detection sees the convolution kernel using a Canny operator convolution kernel, and the preset edge detection sees the convolution kernel using the convolution kernel for the horizontal direction of the plankton sample imageAs an operator convolution kernel, aiming at the vertical direction of plankton sample images, adoptingAs an operator convolution kernel. Correspondingly, the lateral gradient determined for plankton sample images may be employedDetermining that a longitudinal gradient determined for an image of a plankton sample may be employedAnd (5) determining.
In some embodiments, after the foregoing horizontal gradient feature and vertical gradient feature are obtained, the computing device calculates the gradient feature of each pixel to determine the absolute value and gradient direction of the gradient of each pixel, and uses the gradient magnitude and attitude direction as the gradient feature for determining the plankton profile, where the gradient magnitude isThe gradient direction is。
Of course, in other embodiments, after the sample image is convolved with other types of operator convolution check, the resulting gradient features may be represented with other data, and are not limited to gradient magnitude and gradient direction.
S113, determining the outline characteristics of plankton based on the gradient characteristics.
After determining the gradient features corresponding to the blurred sample map, profile features of plankton may then be determined based on the gradient features.
In some embodiments, the computing device performs non-maximum suppression based on the gradient magnitude and gradient direction, determining local gradient maxima. The specific computing equipment performs proximity maximization according to the gradient direction, determines a proximity preset value of each gradient direction, and takes the direction of the proximity preset value as a standardized gradient direction.
The computing device may then perform non-maximum suppression based on the gradient magnitude and gradient direction, determining local gradient maxima. In specific implementation, the computing device compares the magnitude of the gradient intensity of a pixel point with the positive and negative directions of the gradient. If the gradient of the pixel is greater than the gradient of the adjacent pixel, the pixel gradient value is preserved, and if the gradient of the pixel is less than the gradient of the adjacent pixel, the pixel gradient value is suppressed (i.e., deleted). By the method, local gradient maximum values can be determined, and gradient values of the rest pixel points are deleted.
And then, the computing equipment adopts a preset double threshold to process the local gradient maximum value, and the definite maximum value and the candidate maximum value in the local gradient maximum value are determined. The preset double threshold is determined according to factors such as light characteristics of actual image edge recognition, and the like, and is an empirical value determined through experiments.
And finally, screening the candidate maximum value based on the proximity relation between the candidate maximum value and the pixel corresponding to the ensured maximum value to obtain the reserved candidate maximum value. Wherein if a certain candidate maximum is close to the confidence maximum, this candidate maximum is reserved as a reserved candidate maximum. If a candidate maximum is not close to the confidence maximum or the preferred maximum is retained, the candidate maximum is deleted.
After the confidence maxima and the retention candidate maxima are obtained, the pixel points corresponding to the confidence maxima and the retention candidate maxima can be used as the image contour of the plankton sample image.
By reasonably setting the preset double threshold values in the previous analysis, the image contour of the obtained plankton sample image can be made to be a closed contour as much as possible.
And S120, clustering the profile features to obtain a plurality of profile feature cluster sets, and dividing the sample data set based on the profile feature clusters corresponding to the plankton to obtain sub-sample data sets corresponding to the profile feature cluster sets.
In the embodiment of the disclosure, the purpose of clustering the profile features is to divide plankton sample images with the same profile features into one data set. For example, if the profile features include circular features, elliptical features, needle bar features, the goal of S120 is to separate the circular features, elliptical features, and needle bar features into one profile feature cluster set, respectively.
In a specific implementation, the computing device may perform clustering on the profile features to obtain a plurality of profile feature aggregation sets as follows in S121-122.
And S121, carrying out feature analysis on the contour features and determining feature vectors corresponding to the contour features.
The computing device may perform feature analysis on the contour features by using a principal component analysis direction or a line segment type matching method, to determine feature vectors corresponding to the contour features. In particular, because contours of different shapes have different shape features, and the different shape features correspond to elements at different positions of the feature vector, the feature vectors corresponding to different types of contour features may be greatly different.
S122, carrying out cluster analysis on the feature vectors according to the preset cluster quantity, determining a cluster set corresponding to each feature vector, and taking the cluster set corresponding to the feature vector as a contour feature cluster set where the contour features are located.
The preset number of clusters may be a number of clusters determined manually and empirically, or may be a number of clusters determined by previous cluster analysis, which is not limited in the embodiment of the present disclosure. If the number of clusters is determined by a prior cluster analysis, the computing device may determine the number of clusters using statistical methods such as elbow rules, profile coefficients, and the like. In the embodiment of the disclosure, the number of clusters is denoted by K.
After determining the number K of clusters, K sample points can be randomly determined as initial cluster centers, distances from each feature vector to the initial cluster centers are calculated, and each feature vector is distributed to a temporary cluster set where the nearest initial cluster center is located according to the distances. And then determining new cluster centers by utilizing the feature vectors in the temporary cluster set, and re-executing according to the method until the positions of the cluster centers are stable or the iteration times reach the set times.
After determining a plurality of contour cluster sets and determining the contour cluster set to which each feature vector belongs, the sample data set may be then partitioned based on the contour cluster set to which the feature vector belongs, and each sample data pair may be partitioned into a corresponding contour feature cluster set.
And S130, respectively adopting sample data corresponding to the contour feature cluster sets to carry out model training to obtain plankton identification models corresponding to the contour feature cluster sets.
After the sample data pairs corresponding to the contour feature cluster sets are obtained, training the initial models designed for the contour feature cluster sets respectively to obtain the corresponding plankton identification models.
In particular implementations, the initial model may be a convolutional neural network model, such as a neural network model, such as AlexNet architecture, VGG architecture, resNet architecture. The initial model includes at least a convolution layer and a full connection layer, and may include a pooling layer. The convolution kernel adopted by each convolution layer can be empirically set according to the characteristics of plankton sample images, and the output layer can adopt a softmax function, a ReLUctant function or a Tanh function as an activation function, wherein the softmax function is preferably adopted as the activation function.
In the model training process, super parameters such as learning rate, batch sample number, model iteration number and the like need to be preset, and a proper loss function is selected. Considering that plankton recognition models are multi-classification models in nature, cross entropy functions may be employed as the lost function. In the specific implementation, in the model training process, part of sample data pairs in each profile feature cluster set can be used as test data, model precision evaluation can be carried out on the trained plankton recognition model, and specifically, indexes such as recall rate, accuracy rate and the like can be calculated to evaluate the precision of the plankton recognition model.
According to the scheme of the embodiment of the disclosure, based on the problem that plankton of different types possibly have the same outline characteristics, the outline characteristics of the same plankton have different outline characteristics in different growth stages, and the outline characteristics of the plankton in a sample image can negatively influence the plankton identification model training, outline extraction and cluster analysis are firstly carried out on the plankton image in the sample data, the sample data pairs containing different plankton outline characteristics are divided into different cluster sets, and then model training is carried out by adopting sample data corresponding to each cluster set, so that a corresponding plankton identification model is obtained. Because the plankton profile similarity in the plankton sample image used for the plankton recognition model training is very high, the influence degree of the plankton profile on model parameters is reduced when a single plankton recognition model is trained, and the accuracy of each plankton recognition model is improved by adjusting parameters through other features except profile features.
In practical application, as the profile characteristics of plankton do not need to be considered when sample data pairs in the profile characteristic clustering data sets are trained, the loading degree of a single model can be reduced.
From another aspect, the embodiment of the disclosure uses plankton contours as classification basis, instead of conventional classification basis using plankton family as sample data pairs, that is, classification is performed on sample data pairs beyond the taxonomic hierarchy, so as to provide a new idea for model training and subsequent plankton recognition model use.
As previously analyzed, image acquisition, sample data pair construction are required to achieve model training, and the number of sample data pairs should be sufficient to ensure model quality. If image acquisition and sample data pair construction are performed manually, the early cost is too high, and the problem of insufficient coverage of sample features may occur. To solve the foregoing problem, in some embodiments of the present disclosure, the computing device may further perform S140 as follows, before performing S110 described above.
And S140, editing the plankton sample image to obtain an edited plankton sample image, and constructing a new sample data pair based on the edited plankton sample image and the corresponding plankton type label.
In a specific implementation, the editing processing of the plankton sample image comprises at least one of turning, rotating, scaling within a set proportion range, random clipping, random shifting and pixel shielding. In the implementation, for one plankton sample image, three rotation images can be obtained by respectively performing 90 DEG, 180 DEG and 270 DEG rotation treatment on the plankton sample image, a turnover image is obtained by performing turnover treatment on the rotation image and the turnover image, and then the plankton sample image after editing is obtained by performing random clipping, plankton shifting, pixel shielding and scaling treatment within a set proportion range on the rotation image and the turnover image.
After the edited plankton sample image is obtained, a label corresponding to the plankton sample image before editing can be used as a label corresponding to the plankton sample image after editing, and a new sample data pair can be constructed.
In the case of executing S140, S110 is to extract contours of plankton sample images in the original sample data pair and the new sample data pair.
In addition to providing the above-described plankton identification model construction method, embodiments of the present disclosure also provide a plankton type identification method of constructing a model based on the above-described method. Fig. 2 is a flowchart of a plankton class identification method provided by an embodiment of the present disclosure. As shown in fig. 2, the plankton class identification method includes S210 to S230.
And S210, extracting the outline of the plankton image to be identified to obtain outline characteristics of plankton.
Similar to the previous analysis, the plankton image to be identified is an image obtained by photographing the sample water body by a microscopic camera.
In a specific embodiment, the number of plankton included in the plankton image to be identified may be one or more. And if the number of plankton is plural, a subsequent recognition problem may be caused. To solve this problem, S210 may include S211 to S214 as follows.
S211, judging whether the number of plankton contained in the plankton image to be identified is one, if so, executing S212, and if not, executing S213.
S212, extracting contours of the plankton images to be identified to obtain contour features of plankton.
S213, slitting the plankton image to be recognized to obtain a slit image.
S214, carrying out contour extraction on the segmentation image to obtain contour features of plankton.
The aforementioned segmented image is an image including a single plankton. In particular implementations, existing techniques such as convolutional neural networks may be used to identify the number of plankton contained in the plankton image to be identified and to determine the selection box for each plankton (i.e., to determine the pixel range that includes each plankton). After the selection frame is obtained, the plankton image to be identified can be cut based on the selection frame, so that a cut image is obtained.
After the segmented image is obtained, contour selection is carried out on the segmented image, and then contour characteristics of plankton can be obtained.
In the embodiment of the present disclosure, the contour selection is performed on the plankton image to be identified or the segmented image, and the method for obtaining the contour features of the plankton is the same as the contour extraction process in the foregoing model construction method, which is not repeated herein, and may be specifically referred to the foregoing description.
S220, determining a target feature cluster set from a plurality of profile feature cluster sets based on profile features of plankton.
After obtaining the profile features of plankton, the computing device may perform transformation processing on the profile features of plankton according to the foregoing processing method, and determine feature vectors corresponding to the profile features.
After obtaining the feature vector, the computing device may calculate distances of the feature vector to a plurality of feature cluster centers, and select a feature cluster closest to the feature cluster as the target cluster. The characteristic aggregation sets are characteristic aggregation sets determined in the process of constructing the model, and each characteristic aggregation set is respectively associated with one plankton identification model.
S230, processing the plankton image to be identified by adopting a plankton identification model associated with the target feature cluster set, and determining the category of the plankton image.
After the target feature cluster set is obtained, the plankton image to be identified (of course, the segmented image may also be processed) may be processed by using the plankton identification processing model associated with the target feature cluster set, so as to determine the plankton category.
The embodiment of the disclosure also provides a device for constructing the plankton identification model. Fig. 3 is a schematic structural diagram of a device for constructing a plankton recognition model according to an embodiment of the present disclosure. As shown in fig. 3, the plankton recognition model construction device 300 provided by the embodiment of the present disclosure includes a contour extraction unit 301, a data classification unit 302, and a model training unit 303.
The contour extraction unit 301 is configured to perform contour extraction on plankton sample images in a sample data pair to obtain contour features of plankton contained in the plankton sample images, where each plankton sample image includes one type of plankton, and the sample data pair includes plankton type tags that are associated with the plankton sample images one by one in addition to the plankton sample images.
The data pair classification unit 302 is configured to perform clustering processing on the profile features to obtain a plurality of profile feature cluster sets, and divide the sample data pairs based on the profile feature clusters corresponding to the plankton sample images to obtain sample data pairs corresponding to the profile feature cluster sets.
The model training unit 303 is configured to perform model training by using sample data corresponding to the contour feature cluster set, so as to obtain plankton recognition models corresponding to the contour feature cluster sets.
In some embodiments, plankton identification model building device 300 further includes a preprocessing unit. The preprocessing unit is used for editing the plankton sample image to obtain an edited plankton sample image, and constructing a new sample data pair based on the edited plankton sample image and a corresponding plankton type label, wherein the editing process comprises at least one of turning, rotating, scaling within a set proportion range, random cutting, random shifting and pixel shielding, and the edited plankton sample image still comprises pixel content representing plankton.
In some embodiments, the contour extraction unit 301 performs a blurring process on the plankton sample image to obtain a blurred sample image, the blurring process is used for eliminating high-frequency noise in the plankton sample image, a convolution process is performed on the blurred sample image by adopting a preset edge detection convolution check to obtain gradient features of each pixel point of the plankton sample image, and the plankton contour features are determined based on the gradient features.
In some embodiments, the extracted features of convolving the blurred sample image include gradient magnitude and gradient direction. The corresponding contour extraction unit 301 performs non-maximum suppression based on the gradient magnitude and the gradient direction to determine a local gradient maximum, then processes the local gradient maximum by adopting a preset double threshold to determine a definite maximum and a candidate maximum in the local gradient maximum, screens the candidate maximum based on the proximity relation between the candidate maximum and the pixel corresponding to the definite maximum to obtain a reserved candidate maximum, and then uses the pixel points corresponding to the confirmed maximum and the reserved candidate maximum as the image contour of the plankton sample image.
In some embodiments, the classification unit 302 performs feature analysis on the profile features to determine feature vectors corresponding to the profile features, performs cluster analysis on the feature vectors according to a preset number of clusters, determines a cluster set corresponding to each feature vector, and uses the cluster set corresponding to the feature vector as the profile feature cluster set in which the profile features are located.
The embodiment of the disclosure also provides a plankton category identification device. Fig. 4 is a schematic structural diagram of a plankton recognition device according to an embodiment of the present disclosure. As shown in fig. 4, the plankton recognition device 400 provided by the embodiment of the present disclosure includes a contour extraction unit 401, a cluster set determination unit 402, and a model processing unit 403.
The contour extraction unit 401 is configured to perform contour extraction on an image of plankton to be identified, so as to obtain contour features of plankton. The gather determination unit 402 is configured to determine a target feature gather from a plurality of profile feature clusters based on profile features of plankton. The model processing unit 403 is configured to process the plankton image to be identified using the plankton identification model associated with the target feature cluster set, and determine a classification of plankton.
In some embodiments, the contour extraction unit 401 first determines the number of plankton included in the plankton image to be identified. In the case where the number of plankton is one, the contour extraction unit 401 performs contour extraction on the image of plankton to be recognized, resulting in contour features of plankton. In the case that the number of plankton is plural, the contour extraction unit 401 cuts the image of plankton to be recognized to obtain a cut image, the cut image being an image including a single plankton, and performs contour extraction on the cut image to obtain contour features of plankton
In some embodiments, the cluster set determining unit 402 performs feature analysis on the profile features to determine feature vectors corresponding to the profile features, obtains distances between the feature vectors and center points of a plurality of predetermined feature cluster sets, and selects a feature distance set closest to the feature distance set as a target feature cluster set.
The embodiment of the disclosure also provides a computing device for realizing the method. Fig. 5 is a schematic structural diagram of a computing device provided by an embodiment of the present disclosure. Referring now in particular to FIG. 5, a schematic diagram of a computing device 500 suitable for use in implementing embodiments of the present disclosure is shown. The computing device illustrated in fig. 5 is merely an example and should not be taken as limiting the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 5, the computing device 500 may include a processing means (e.g., a central processor, a graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with programs stored in a read-only memory ROM502 or loaded from a storage means 508 into a random access memory RAM 503. In the RAM503, various programs and data required for the operation of the computing device 500 are also stored. The processing device 501, the ROM502, and the RAM503 are connected to each other via a bus 504. An input/output I/O interface 505 is also connected to bus 504.
In general, devices may be connected to I/O interface 505 including input devices 505 such as touch screens, touch pads, cameras, microphones, etc., output devices 507 including Liquid Crystal Displays (LCDs), speakers, vibrators, etc., storage devices 508 including magnetic tapes, hard disks, etc., and communication devices 509. The communication means 509 may allow the computing device 500 to communicate wirelessly or by wire with other devices to exchange data. While fig. 5 illustrates a computing device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or from the storage means 508, or from the ROM 502. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 501.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable storage medium, a computer readable signal medium, or any combination of the two.
The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, computing devices may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with digital data communications (e.g., communication networks) in any form or medium. Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the computing device or may exist alone without being incorporated into the computing device.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the tester computer, partly on the tester computer, as a stand-alone software package, partly on the tester computer and partly on a remote computer or entirely on the remote computer or computing device. In the case of remote computers, the remote computer may be connected to the tester computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., connected through the internet using an internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases. The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic that may be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems-on-a-chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
The above is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.