CN111859985A

CN111859985A - AI customer service model testing method, device, electronic equipment and storage medium

Info

Publication number: CN111859985A
Application number: CN202010719768.XA
Authority: CN
Inventors: 宫雪
Original assignee: Ping An Puhui Enterprise Management Co Ltd
Current assignee: Shanghai Huaqi Information Technology Co ltd; Shenzhen Lian Intellectual Property Service Center
Priority date: 2020-07-23
Filing date: 2020-07-23
Publication date: 2020-10-30
Anticipated expiration: 2040-07-23
Also published as: CN111859985B

Abstract

The application relates to artificial intelligence, and provides an AI customer service model testing method, a device, an electronic device and a storage medium, which can acquire pre-configured standard corpora including input standard corpora and output standard corpora from a corpus, perform similarity analysis on the standard corpora based on a semantic similarity algorithm to obtain classified corpora, make subsequent tests more pertinent, expand the classified corpora based on a preset lexicon, generate a test sample including input data corresponding to the input standard corpora and expected data corresponding to the output standard corpora, make the coverage of the test sample more comprehensive, and effectively solves the problem of insufficient test data, inputs the input data into the AI customer service model to be tested to obtain the output data, and the configuration script calls the output data and the expected data to output the test result of the AI customer service model to be tested, thereby realizing the rapid automatic test of the AI customer service model. The application also relates to a block chain technology, and the test result can be stored in the block chain.

Description

AI customer service model testing method, device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an AI customer service model testing method, an AI customer service model testing device, electronic equipment and a storage medium.

Background

In the prior art, the accuracy of an output result of an AI (Artificial Intelligence) customer service model is generally evaluated in a manual marking manner, which takes a long time, and a hysteresis phenomenon may occur due to uncontrollable manual operation.

Disclosure of Invention

In view of the above, it is necessary to provide an AI customer service model testing method, device, electronic device and storage medium, which can perform targeted testing on an AI customer service model based on a semantic similarity algorithm, and the coverage of a test sample is more comprehensive, thereby effectively solving the problem of insufficient test data, and realizing rapid and automatic testing on the AI customer service model based on an artificial intelligence means.

An AI customer service model test method, the AI customer service model test method comprising:

when a test instruction for an AI customer service model to be tested is received, acquiring a pre-configured standard corpus from a corpus, wherein the standard corpus comprises an input standard corpus and an output standard corpus;

performing similarity analysis on the standard corpus based on a semantic similarity algorithm to obtain a classified corpus;

expanding the classified linguistic data based on a preset lexicon to generate a test sample, wherein the test sample comprises input data corresponding to the input standard linguistic data and expected data corresponding to the output standard linguistic data;

inputting the input data into the AI customer service model to be tested to obtain output data;

and calling the output data and the expected data by using a configuration script, and outputting a test result of the AI customer service model to be tested.

According to the preferred embodiment of the present invention, the analyzing the similarity of the standard corpus based on the semantic similarity algorithm to obtain the classified corpus comprises:

converting the standard corpus into semantic vectors based on natural language processing;

calculating cosine distances among word vectors in the semantic vectors by adopting a cosine similarity algorithm;

and classifying the word vectors according to the cosine distance to obtain the classified linguistic data.

According to a preferred embodiment of the present invention, the expanding the classified corpus based on the preset lexicon, and the generating the test sample includes:

for each target word vector in the classified corpus, calculating the similarity between the target word vector and the word vector in the preset word bank;

acquiring a word vector with the similarity greater than or equal to a preset similarity with the target word vector from the preset word bank as an expansion word vector of the target word vector;

and adding the expanded word vector of each target word vector to the classified corpus to obtain the test sample.

According to a preferred embodiment of the present invention, the invoking the output data and the expected data with the configuration script, and outputting the test result of the AI customer service model to be tested includes:

writing the output data and the expected data into Excel by adopting POI to generate an Excel file;

determining the file name of the excel file and the interface parameters of the designated interface;

modifying the configuration script by the file name and the interface parameter;

and calling the excel file by the modified configuration script, and outputting a test result of the AI customer service model to be tested.

According to the preferred embodiment of the present invention, the test result includes an accuracy, and the AI customer service model test method further includes:

acquiring a first number of data with similarity greater than or equal to a first similarity with the expected data in the output data;

determining a second amount of the output data;

calculating a quotient of the first quantity and the second quantity as the accuracy.

According to the preferred embodiment of the present invention, the test result further includes a recall rate, and the AI customer service model test method further includes:

for first output data in any category, determining first expected data corresponding to the first output data;

acquiring a third quantity of data with the similarity greater than or equal to a second similarity between the first output data and the first expected data;

determining a fourth amount of the first output data;

calculating a quotient of the third quantity and the fourth quantity as a recall ratio in the arbitrary category.

According to the preferred embodiment of the present invention, the AI customer service model testing method further comprises:

and when the AI customer service model to be detected is updated, re-executing the detection of the AI customer service model to be detected.

An AI customer service model test device, the AI customer service model test device comprising:

the acquisition unit is used for acquiring a pre-configured standard corpus from a corpus when a test instruction of the AI customer service model to be tested is received, wherein the standard corpus comprises an input standard corpus and an output standard corpus;

the analysis unit is used for carrying out similarity analysis on the standard corpus based on a semantic similarity algorithm to obtain a classified corpus;

the extension unit is used for extending the classified linguistic data based on a preset lexicon to generate a test sample, and the test sample comprises input data corresponding to the input standard linguistic data and expected data corresponding to the output standard linguistic data;

the input unit is used for inputting the input data into the AI customer service model to be tested to obtain output data;

and the test unit is used for calling the output data and the expected data by using a configuration script and outputting a test result of the AI customer service model to be tested.

According to a preferred embodiment of the present invention, the analysis unit is specifically configured to:

According to a preferred embodiment of the present invention, the extension unit is specifically configured to:

According to a preferred embodiment of the present invention, the test unit is specifically configured to:

According to a preferred embodiment of the present invention, the test result includes an accuracy, and the obtaining unit is further configured to obtain a first number of data, of the output data, whose similarity to the expected data is greater than or equal to a first similarity;

the AI customer service model test device further includes:

a determining unit for determining a second amount of the output data;

a calculating unit, configured to calculate a quotient of the first number and the second number as the accuracy.

According to a preferred embodiment of the present invention, the test result further includes a recall rate, and the determining unit is configured to determine, for first output data in any category, first expected data corresponding to the first output data;

the acquiring unit is further configured to acquire a third number of data, of the first output data, of which the similarity to the first expected data is greater than or equal to a second similarity;

the determining unit is further configured to determine a fourth amount of the first output data;

the calculating unit is further configured to calculate a quotient of the third quantity and the fourth quantity as a recall rate in the arbitrary category.

According to a preferred embodiment of the present invention, the testing unit is further configured to perform the detection of the AI customer service model to be detected again when it is detected that the AI customer service model to be detected is updated.

An electronic device, the electronic device comprising:

a memory storing at least one instruction; and

and the processor executes the instructions stored in the memory to realize the AI customer service model test method.

A computer-readable storage medium having stored therein at least one instruction, the at least one instruction being executable by a processor in an electronic device to implement the AI customer service model testing method.

According to the technical scheme, the invention can obtain the pre-configured standard corpus from the corpus when receiving the test instruction of the AI customer service model to be tested, the standard corpus comprises the input standard corpus and the output standard corpus, the similarity analysis is carried out on the standard corpus based on the semantic similarity algorithm to obtain the classified corpus, so that the subsequent test is more targeted, the classified corpus is further expanded based on the preset lexicon to generate the test sample, the test sample comprises the input data corresponding to the input standard corpus and the expected data corresponding to the output standard corpus, so that the generated test sample has more comprehensive coverage and can effectively solve the problem of insufficient test data, the input data is input into the AI customer service model to be tested to obtain the output data, and the output data and the expected data are called by the configuration script, and outputting a test result of the AI customer service model to be tested, thereby realizing rapid automatic test of the AI customer service model.

Drawings

FIG. 1 is a flow chart of the AI customer service model testing method according to the preferred embodiment of the invention.

FIG. 2 is a functional block diagram of an AI customer service model testing device according to a preferred embodiment of the invention.

Fig. 3 is a schematic structural diagram of an electronic device implementing the AI customer service model testing method according to the preferred embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a flow chart of a preferred embodiment of the AI customer service model testing method of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.

The AI customer service model test method is applied to one or more electronic devices, which are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.

The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.

The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a cloud computing (cloud computing) based cloud consisting of a large number of hosts or network servers.

The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.

And S10, when receiving a test instruction of the AI customer service model to be tested, acquiring a pre-configured standard corpus from the corpus.

In at least one embodiment of the present invention, the test instruction of the AI customer service model to be tested may be triggered by a relevant staff, or may be automatically triggered according to an actual use and a preset test time.

In at least one embodiment of the present invention, the corpus may be a preconfigured database having a plurality of corpora.

In at least one embodiment of the present invention, the standard corpus includes an input standard corpus and an output standard corpus.

Wherein, the AI (Artificial Intelligence) customer service model to be tested is used for automatically answering the questions of the customer.

In practical application, the questions asked by the user are various, and taking a loan scene as an example, the user asks questions of each link involved in the loan, such as: interest rate issues, deposit issues, repayment issues, etc., and may even include various cancy statements, etc.

For various classified scenes, firstly, during initial modeling, the scenes are classified and real questions of users are collected for standard answers of the models so as to ensure that the answers conform to external unified dialect of companies without legal risks and the like, and a testing stage can be started after modeling is completed.

Specifically, the AI customer service model can automatically answer the questions of the user in different scenes, and therefore the accuracy of the answer of the AI customer service model is particularly important, and the AI customer service model needs to be tested.

In at least one embodiment of the present invention, the input standard corpus refers to collected user-input standardized corpora.

In at least one embodiment of the present invention, the output standard corpus refers to a standard answer corpus of the input standard corpus.

And S11, performing similarity analysis on the standard corpus based on a semantic similarity algorithm to obtain a classified corpus.

Specifically, the analyzing the similarity of the standard corpus based on the semantic similarity algorithm to obtain the classified corpus includes:

further calculating cosine distances among word vectors in the semantic vectors by adopting a cosine similarity algorithm;

and the electronic equipment classifies each word vector according to the cosine distance to obtain the classified corpus.

Through the implementation mode, the standard corpora can be classified, so that the subsequent test is more targeted.

And S12, expanding the classified linguistic data based on a preset lexicon to generate a test sample, wherein the test sample comprises input data corresponding to the input standard linguistic data and expected data corresponding to the output standard linguistic data.

Specifically, the electronic device expands the classified corpus based on a preset lexicon, and generating the test sample includes:

for each target word vector in the classified corpus, the electronic equipment calculates the similarity between the target word vector and the word vector in the preset word stock;

For example: for the corpus "how much interest you are in companies", the corpus includes word vectors corresponding to the following words respectively: the electronic equipment can select the similar words such as the units of the company, the similar words such as the income and the interest rate of the interest, and the corpus such as the interest rate of your company can be expanded.

Through the embodiment, the electronic equipment can expand the classified corpora to generate a more comprehensive coverage of the test sample, and further can realize more accurate test on the AI customer service model to be tested.

In addition, the existing corpus is mainly based on production data, the data volume is limited, and the problem of insufficient test data can be effectively solved.

And S13, inputting the input data into the AI customer service model to be tested to obtain output data.

In this embodiment, the AI customer service model to be tested can output voice in response to the input data.

And the output data is an output result obtained by the AI customer service model to be tested responding to the input data, and belongs to the automatic solution of the model.

For example: when the input data is "what the income of the sub-loan is", the output data "the income of the sub-loan is 50 ten thousand" is obtained through automated processing such as analysis and matching of the to-be-tested AI customer service model.

And S14, calling the output data and the expected data by a configuration script, and outputting the test result of the AI customer service model to be tested.

Specifically, the calling the output data and the expected data by the configuration script, and outputting the test result of the to-be-tested AI customer service model includes:

the electronic equipment writes the output data and the expected data into Excel by adopting POI to generate an Excel file;

further, the electronic equipment determines the file name of the excel file and interface parameters of a specified interface;

furthermore, the electronic equipment modifies the configuration script by the file name and the interface parameter;

Wherein the interface parameters may include, but are not limited to: server IP (Internet Protocol), port, test URI (Uniform Resource Identifier), etc.

POI is the tool of reading and writing to the document of office, and the components (such as HSSF and XSS) in POI can read and write excel.

The configuration script can be pre-configured so as to be directly called after relevant parameters are modified, the operation efficiency is improved, and the development process is simplified.

Through the implementation mode, the excel file is generated by combining the POI technology, the generated excel file is further called by the modified configuration script, and the test result of the AI customer service model to be tested is output, so that the rapid automatic test can be realized.

In at least one embodiment of the invention, the test results include accuracy, the method further comprising:

determining a second amount of the output data;

further, the electronic device calculates a quotient of the first quantity and the second quantity as the accuracy rate.

For example: 10000 test samples in a certain test, 5000 correct output data, and 50% accuracy rate 5000/10000.

In at least one embodiment of the invention, the test results further include a recall, the method further comprising:

determining a fourth amount of the first output data;

further calculating a quotient of the third quantity and the fourth quantity as a recall ratio under the arbitrary category.

Through the implementation mode, the accuracy and the recall rate of the AI customer service model to be detected can be calculated, so that the AI customer service model to be detected can be effectively detected.

In at least one embodiment of the present invention, the AI customer service model testing method further includes:

In the optimization process of the model, the adjustment of the threshold value affects the conditions of each classification, for example, the accuracy of interest problems is increased, which may result in the decrease of the accuracy of the user operation class, but the accuracy of the high-frequency problems of the user needs to be in an increasing trend, so that the AI customer service model to be detected needs to be re-detected to ensure that the AI customer service model to be detected can make the best response to the question of the user.

It should be noted that, in order to improve the security and privacy of the data, the test result may be stored in the blockchain.

FIG. 2 is a functional block diagram of an AI customer service model testing device according to a preferred embodiment of the invention. The AI customer service model testing apparatus 11 includes an acquisition unit 110, an analysis unit 111, an extension unit 112, an input unit 113, a testing unit 114, a determination unit 115, and a calculation unit 116. The module/unit referred to in the present invention refers to a series of computer program segments that can be executed by the processor 13 and that can perform a fixed function, and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.

When receiving a test instruction for the AI customer service model to be tested, the obtaining unit 110 obtains the pre-configured standard corpus from the corpus.

The analysis unit 111 performs similarity analysis on the standard corpus based on a semantic similarity algorithm to obtain a classified corpus.

Specifically, the analyzing unit 111 performs similarity analysis on the standard corpus based on a semantic similarity algorithm, and obtaining the classified corpus includes:

the analysis unit 111 classifies each word vector according to the cosine distance to obtain the classified corpus.

Further, the expansion unit 112 expands the classified corpus based on a preset lexicon to generate a test sample.

The test sample comprises input data corresponding to the input standard corpus and expected data corresponding to the output standard corpus.

Specifically, the expanding unit 112 expands the classified corpus based on a preset lexicon, and generating a test sample includes:

for each target word vector in the classified corpus, the expansion unit 112 calculates the similarity between the target word vector and the word vector in the preset lexicon;

For example: for the corpus "how much interest you are in companies", the corpus includes word vectors corresponding to the following words respectively: "your", "company", "interest", and "how much", wherein the word vector "company" and the word vector "interest" are keywords, the expansion unit 112 may optionally extract the similar words "unit" of "company" and the like, the similar words "income", "interest rate" and the like of "interest" through similarity calculation, and then the corpus "how much your company interest is" may include the expansion corpus "how much your unit interest rate" and the like.

Through the above embodiment, the extension unit 112 can extend the classified corpus to enable the coverage of the generated test sample to be more comprehensive, so as to achieve more accurate test of the AI customer service model to be tested.

The input unit 113 inputs the input data into the to-be-tested AI customer service model to obtain output data.

Specifically, the test unit 114 calls the output data and the expected data by using a configuration script, and outputs a test result of the AI customer service model to be tested.

The test unit 114 calls the output data and the expected data by using a configuration script, and outputting a test result of the AI customer service model to be tested includes:

the test unit 114 writes the output data and the expected data into Excel by using POI to generate an Excel file;

further, the test unit 114 determines a file name of the excel file and an interface parameter of a designated interface;

further, the test unit 114 modifies the configuration script with the file name and the interface parameter;

In at least one embodiment of the present invention, the test result includes an accuracy, and the obtaining unit 110 obtains a first number of data having a similarity greater than or equal to a first similarity with the expected data in the output data;

the determining unit 115 determines the second amount of the output data;

further, the calculation unit 116 calculates a quotient of the first number and the second number as the accuracy.

In at least one embodiment of the present invention, the test result further includes a recall rate, and for the first output data in any category, the determining unit 115 determines first expected data corresponding to the first output data;

the acquiring unit 110 acquires a third number of data of which the similarity to the first desired data is greater than or equal to a second similarity in the first output data;

the determining unit 115 determines a fourth amount of the first output data;

the calculating unit 116 further calculates a quotient of the third number and the fourth number as a recall rate in the arbitrary category.

In at least one embodiment of the invention, when it is detected that the AI customer service model under test is updated, the test unit 114 re-executes the detection of the AI customer service model under test.

Fig. 3 is a schematic structural diagram of an electronic device implementing the AI customer service model testing method according to the preferred embodiment of the present invention.

The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as an AI customer service model test program, stored in the memory 12 and executable on the processor 13.

It will be understood by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation to the electronic device 1, the electronic device 1 may have a bus-type structure or a star-type structure, the electronic device 1 may further include more or less hardware or software than those shown in the figures, or different component arrangements, for example, the electronic device 1 may further include an input and output device, a network access device, and the like.

It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those that can be adapted to the present invention, should also be included in the scope of the present invention, and are included herein by reference.

The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, for example a removable hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 can be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of an AI customer service model test program, etc., but also to temporarily store data that has been output or is to be output.

The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., executing an AI customer service model test program, etc.) stored in the memory 12 and calling data stored in the memory 12.

The processor 13 executes an operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in the foregoing various embodiments of the AI customer service model testing method, such as the steps shown in fig. 1: s10, S11, S12, S13 and S14.

Alternatively, the processor 13, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example:

Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, an analysis unit 111, an extension unit 112, an input unit 113, a test unit 114, a determination unit 115, a calculation unit 116.

The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device) or a processor (processor) to execute parts of the AI customer service model testing method according to various embodiments of the present invention.

The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented.

Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).

Further, the computer-usable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.

Although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.

Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.

Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (organic light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.

It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.

Fig. 3 only shows the electronic device 1 with components 12-13, and it will be understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.

Referring to fig. 1, the memory 12 in the electronic device 1 stores a plurality of instructions to implement an AI customer service model test method, and the processor 13 can execute the plurality of instructions to implement:

Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.

In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.

The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.

Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An AI customer service model test method, characterized in that the AI customer service model test method comprises:

2. The AI customer service model testing method of claim 1, wherein the performing similarity analysis on the standard corpus based on a semantic similarity algorithm to obtain a classified corpus comprises:

3. The AI customer service model testing method of claim 1, wherein the expanding the corpus based on a preset lexicon to generate test samples comprises:

4. The AI customer service model testing method of claim 1, wherein the invoking the output data and the expectation data with a configuration script and outputting the test result for the AI customer service model to be tested comprises:

5. The AI customer service model testing method of claim 1, wherein the test results include an accuracy rate, the AI customer service model testing method further comprising:

determining a second amount of the output data;

6. The AI customer service model testing method of claim 1, wherein the test results further include a recall rate, the AI customer service model testing method further comprising:

determining a fourth amount of the first output data;

7. The AI customer service model testing method of claim 1, further comprising:

8. An AI customer service model test device, characterized in that the AI customer service model test device comprises:

9. An electronic device, characterized in that the electronic device comprises:

a memory storing at least one instruction; and

a processor executing instructions stored in the memory to implement the AI customer service model testing method of any of claims 1 to 7.

10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executable by a processor in an electronic device to implement the AI customer service model testing method of any of claims 1-7.