CN112015715A - Industrial Internet data management service testing method and system - Google Patents
Industrial Internet data management service testing method and system Download PDFInfo
- Publication number
- CN112015715A CN112015715A CN201910851128.1A CN201910851128A CN112015715A CN 112015715 A CN112015715 A CN 112015715A CN 201910851128 A CN201910851128 A CN 201910851128A CN 112015715 A CN112015715 A CN 112015715A
- Authority
- CN
- China
- Prior art keywords
- test
- data
- user
- management module
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
技术领域technical field
本发明涉及数据测试技术领域,尤其是一种面向多种数据类型的的工业互联网数据管理服务测试方法及系统。The invention relates to the technical field of data testing, in particular to an industrial Internet data management service testing method and system for multiple data types.
背景技术Background technique
工业互联网在工业企业中有众多应用场景,在工业互联网系统运行过程中,数据起到了重要的支撑连接作用,工业互联网数据包括工业产品生产全周期过程各个环节中产生的各种类型的数据,包括产品的设计、研发、生产、销售,以及订单管理、原料采购、库存管理、售后服务等过程。海量数据给工业产业发展带来巨大机遇的同时,也带来了对于数据安全和信息隐私性的挑战。因此,需要通过适当的数据库测试技术来确定工业互联网环境中数据管理服务产品是否能符合要求。Industrial Internet has many application scenarios in industrial enterprises. During the operation of industrial Internet systems, data plays an important supporting and connecting role. Industrial Internet data includes various types of data generated in all aspects of the production cycle of industrial products, including Product design, research and development, production, sales, as well as order management, raw material procurement, inventory management, after-sales service and other processes. While massive data brings great opportunities to the development of the industrial industry, it also brings challenges to data security and information privacy. Therefore, it is necessary to determine whether the data management service products in the industrial Internet environment can meet the requirements through appropriate database testing technology.
工业互联网数据管理系统的性能与系统架构、数据规模和基础硬件环境等多种因素有关,对同类型的不同工业互联网数据管理产品进行比较,需要在相同条件下对其进行性能测评。并充分考虑应用特点,进行有针对性的测试。测试计划的设计通常需要从多个角度考虑。在测试数据方面,测试数据集的大小、类型及其物理含义要符合工业互联网场景,测试数据的生成速度也要符合工业实际场景的要求。在测试指标方面,数据管理系统的基准测试可以从性能、可靠性等多个角度来进行综合评估。在应用场景方面,被测系统要与其在工业生产中的实际使用场景相对应。The performance of the industrial Internet data management system is related to various factors such as system architecture, data scale, and basic hardware environment. To compare different industrial Internet data management products of the same type, it is necessary to evaluate their performance under the same conditions. And fully consider the application characteristics, carry out targeted testing. The design of a test plan often requires consideration from multiple perspectives. In terms of test data, the size, type and physical meaning of the test data set must conform to the industrial Internet scenario, and the generation speed of the test data must also conform to the requirements of the actual industrial scenario. In terms of test indicators, the benchmark test of the data management system can be comprehensively evaluated from multiple perspectives such as performance and reliability. In terms of application scenarios, the system under test should correspond to its actual usage scenarios in industrial production.
常用的性能测试分为多种类型,典型的性能测试方式包括基准测试、负载测试等。不同测试类型从不同角度对数据库系统进行了性能评估。Commonly used performance tests are divided into various types. Typical performance testing methods include benchmark tests and load tests. Different test types evaluate the performance of database systems from different perspectives.
现有的测试系统存在以下不足:一是对多种数据类型的支持不够全面。以上测试工具主要面向一种或少数几种数据类型,而在实际工业场景中,多种类型的数据都是常见的,如果只使用某一种已有测试工具,不能满足对所有类型的数据进行测试的需求;二是缺少图形界面。不方便测试人员配置执行测试以及查看测试结果,例如:YCSB和BigDataBench没有对图数据库、消息队列数据库、二进制大对象等数据库类型提供直接的支持,并且没有向用户提供图形化的界面以进行测试的配置等操作,用户在启动测试以及查看测试结果时不够直观易用。LinkBench主要提供对图类型数据的增删改查等测试,不支持对其他数据类型的修改。The existing test system has the following shortcomings: First, the support for multiple data types is not comprehensive enough. The above test tools are mainly for one or a few data types. In actual industrial scenarios, many types of data are common. If only one existing test tool is used, the The need for testing; the second is the lack of a graphical interface. It is inconvenient for testers to configure and execute tests and view test results. For example, YCSB and BigDataBench do not provide direct support for database types such as graph databases, message queue databases, and binary large objects, and do not provide users with a graphical interface for testing. Configuration and other operations, users are not intuitive and easy to use when starting tests and viewing test results. LinkBench mainly provides tests such as adding, deleting, modifying, and checking graph type data, and does not support modification of other data types.
因此,现有技术需要改进。Therefore, there is a need for improvement in the prior art.
发明内容SUMMARY OF THE INVENTION
本发明实施例所要解决的一个技术问题是:提供一种工业互联网数据管理服务测试方法及系统,以解决现有技术存在的问题,A technical problem to be solved by the embodiments of the present invention is to provide an industrial Internet data management service testing method and system to solve the problems existing in the prior art,
基于本发明实施例的一个方面,公开一种工业互联网数据管理服务测试方法,包括:Based on an aspect of the embodiments of the present invention, an industrial Internet data management service testing method is disclosed, including:
根据测试系统中提出的测试用例要求,以及被测数据库提供的API,编写该被测数据库的测试用例函数以供测试服务器调用;According to the test case requirements put forward in the test system and the API provided by the database under test, write the test case function of the database under test for the test server to call;
通过用户界面模块选择被测数据库系统和测试用例,通过Django框架以表单提交的方式将用户选择的被测数据库系统以及测试用例ID传送给网站后台的测试管理模块;Select the database system under test and test case through the user interface module, and transmit the database system under test and test case ID selected by the user to the test management module in the background of the website by submitting a form through the Django framework;
测试管理模块接收到由前端提交的测试计划后,通过发送post请求的方式,将该测试计划根据其被测数据库类型发送给测试执行模块中对应数据类型的测试服务器;After the test management module receives the test plan submitted by the front end, it sends the test plan to the test server of the corresponding data type in the test execution module according to the type of the database under test by sending a post request;
测试服务器接收到测试计划后,根据用户指定的测试用例,调用测试用例函数,在被测系统服务器上进行测试,并记录下测试结果,将测试结果返回给测试管理模块;After the test server receives the test plan, it calls the test case function according to the test case specified by the user, performs the test on the system server under test, records the test result, and returns the test result to the test management module;
测试管理模块接收到测试服务器返回的测试用例运行结果,将相关结果数据返回给用户界面模块,用户界面模块使用AJAX技术将结果数据显示在页面中;The test management module receives the test case running result returned by the test server, and returns the relevant result data to the user interface module, and the user interface module uses AJAX technology to display the result data on the page;
用户界面模块根据用户选择的测试计划,将用户选择的数据库、测试用例以及运行的测试结果自动填充进Latex模板,并编译生成测试报告PDF文件,提供给测试人员下载。The user interface module automatically fills the database, test cases and running test results selected by the user into the Latex template according to the test plan selected by the user, and compiles and generates a PDF file of the test report, which is provided to the testers for download.
在基于本发明上述工业互联网数据管理服务测试方法的另一个实施例中,所述根据测试系统中提出的测试用例要求,以及被测数据库提供的API,编写该被测数据库的测试用例函数以供测试服务器调用包括:In another embodiment based on the above-mentioned industrial Internet data management service testing method of the present invention, the test case function of the tested database is written according to the test case requirements proposed in the test system and the API provided by the tested database for Test server calls include:
每种被测数据库系统具有单独的测试服务器,负责接收具体的测试请求并在被测数据库系统上执行测试。Each database system under test has a separate test server responsible for receiving specific test requests and executing tests on the database system under test.
基于本发明实施例的另一个方面,公开一种工业互联网数据管理服务测试系统,包括:Based on another aspect of the embodiments of the present invention, an industrial Internet data management service testing system is disclosed, including:
用户界面模块、测试管理模块、测试执行模块;User interface module, test management module, test execution module;
所述用户界面模块通过交互界面接收测试人员的测试相关要求和配置,作为测试计划提交给所述测试管理模块;The user interface module receives the test-related requirements and configuration of the tester through the interactive interface, and submits it to the test management module as a test plan;
所述测试管理模块根据测试计划的类型转发用户的测试计划给所述测试执行模块;The test management module forwards the user's test plan to the test execution module according to the type of the test plan;
所述测试执行模块对测试计划中的每条测试用例分别执行测试,将所有测试结果数据返回给所述测试管理模块;The test execution module performs tests on each test case in the test plan respectively, and returns all test result data to the test management module;
所述测试管理模块对所执行测试的结果数据进行统计计算,得到本条测试计划的测试结果,并转发给所述用户界面模块,由所述用户界面模块将对应测试的执行结果反馈给测试人员。The test management module performs statistical calculation on the result data of the executed test, obtains the test result of the test plan, and forwards it to the user interface module, and the user interface module feeds back the execution result of the corresponding test to the tester.
在基于本发明上述工业互联网数据管理服务测试系统的另一个实施例中,所述用户界面模块包括:In another embodiment of the above-mentioned industrial Internet data management service testing system based on the present invention, the user interface module includes:
用户登录界面、被测数据库选择界面、测试服务器配置界面、测试用例选择界面和测试结果展示界面;User login interface, database selection interface under test, test server configuration interface, test case selection interface and test result display interface;
通过所述用户登录界面,测试人员通过用户名和密码进行登录,所述用户界面模块向所述测试管理模块发送测试人员通过用户登录界面输入的用户名和密码信息,所述测试管理模块经过验证后将登录结果返回给用户登录界面;Through the user login interface, the tester logs in through the user name and password, and the user interface module sends the user name and password information entered by the tester through the user login interface to the test management module, and the test management module will The login result is returned to the user login interface;
通过被测数据库选择界面,用户从测试系统给定的数据库中选择测试服务器的数据库;Through the database selection interface under test, the user selects the database of the test server from the database given by the test system;
通过测试服务器配置界面,用户配置各个数据类型的测试执行模块的IP地址以及用户名和密码,以选择对应的测试执行模块中的测试服务器;Through the test server configuration interface, the user configures the IP address, user name and password of the test execution module of each data type to select the test server in the corresponding test execution module;
通过测试用例选择界面,用户从测试系统给定的测试用例列表中选择针对各个数据类型的测试用例,组合成为测试计划并执行测试;Through the test case selection interface, the user selects test cases for each data type from the test case list given by the test system, combines them into a test plan and executes the test;
测试结果展示界面将测试过程中输出的日志信息显示在用户界面模块中,在测试结束后,用户下载自动生成的测试报告。The test result display interface displays the log information output during the test in the user interface module. After the test, the user downloads the automatically generated test report.
在基于本发明上述工业互联网数据管理服务测试系统的另一个实施例中,所述测试管理模块包括:In another embodiment of the above-mentioned industrial Internet data management service test system based on the present invention, the test management module includes:
用户管理模块、规范符合性测试管理模块、性能测试管理模块;User management module, specification compliance test management module, performance test management module;
通过用户管理模块增加或删除有权登录用户界面模块的用户,并对用户及其权限进行管理,从而确保用户数据及被测系统的系统及数据安全;Add or delete users who have the right to log in to the user interface module through the user management module, and manage users and their permissions, so as to ensure the security of user data and the system and data of the system under test;
通过规范符合性测试管理模块对不同数据类型的数据管理系统进行规范;Standardize the data management system of different data types through the standard compliance test management module;
性能测试管理模块用于对被测试数据库系统的性能测试,包括数据迁移效率、数据查询效率、用户鉴权攻击和可信验证时延分析。The performance test management module is used to test the performance of the tested database system, including data migration efficiency, data query efficiency, user authentication attack and trusted verification delay analysis.
在基于本发明上述工业互联网数据管理服务测试系统的另一个实施例中,所述测试执行模块包括:In another embodiment of the above-mentioned industrial Internet data management service test system based on the present invention, the test execution module includes:
测试数据管理模块、测试结果管理模块、测试报告生成模块、测试服务器;Test data management module, test result management module, test report generation module, test server;
测试数据管理模块在执行测试时向被测系统提供对应数据类型的测试数据集,对于需要大量数据导入的被测系统测试,测试数据管理模块根据工业数据特征生成人工测试数据集;The test data management module provides a test data set corresponding to the data type to the system under test when executing the test. For the test of the system under test that requires a large amount of data import, the test data management module generates an artificial test data set according to the characteristics of industrial data;
测试结果管理模块对测试结果数据进行收集和计算,并反馈给对应的测试管理模块;The test result management module collects and calculates the test result data, and feeds it back to the corresponding test management module;
测试报告生成模块收集测试过程数据和测试结果信息,将测试的相关数据填充进测试报告模板,形成完整包含本次测试的测试环境、被测系统、测试用例、详细测试结果的测试报告;The test report generation module collects the test process data and test result information, fills the test related data into the test report template, and forms a test report that completely includes the test environment, system under test, test cases, and detailed test results of this test;
所述测试服务器包括多种,每一种类型的被测系统对应一种被测服务器,被测系统所在的测试服务器执行用户指定的测试计划,用户的测试计划由测试管理模块转发至具体数据类型的测试服务器后,测试服务器根据需要执行的测试用例进行测试。The test server includes a variety of types, each type of system under test corresponds to a server under test, the test server where the system under test is located executes the test plan specified by the user, and the test plan of the user is forwarded to the specific data type by the test management module. After the test server is installed, the test server performs the test according to the test cases that need to be executed.
在基于本发明上述工业互联网数据管理服务测试系统的另一个实施例中,所述测试用例通过使用被测数据库的API或者使用各语言的API库实现。In another embodiment of the above-mentioned industrial Internet data management service test system based on the present invention, the test case is implemented by using the API of the database under test or using the API library of each language.
在基于本发明上述工业互联网数据管理服务测试系统的另一个实施例中,所述测试数据管理模块包括:In another embodiment of the above-mentioned industrial Internet data management service test system based on the present invention, the test data management module includes:
关系数据、图数据、时序数据、文档数据、宽列数据、键值对数据、事件数据、消息队列数据、二进制大对象9种数据类型的数据集。Data sets of 9 data types including relational data, graph data, time series data, document data, wide column data, key-value pair data, event data, message queue data, and binary large objects.
与现有技术相比,本发明具有以下优点:Compared with the prior art, the present invention has the following advantages:
本发明提出了一种工业互联网数据管理服务测试方法及系统,可以对多种数据类型的数据管理服务进行测试,可以从安全性、可靠性、隐私性等方面对被测数据库的功能和性能进行测试,从而为工业企业在选择数据库产品时提供参考,并为数据库厂商进行产品评估提高提供参考。本发明提供了方便用户使用的用户交互界面,测试可以通过交互界面配置、发布测试任务,可以查看测试过程输出和测试结果。本发明包括关系数据、图数据、时序数据、文档数据、宽列数据、键值对数据、事件数据、消息队列数据、二进制大对象共9种数据类型的数据集,可以根据需要生成指定大小的测试数据集,根据执行的测试结果自动生成测试报告,显示被测系统相关信息以及所有测试用例测试结果。The invention proposes an industrial Internet data management service testing method and system, which can test data management services of various data types, and can test the functions and performance of the tested database from the aspects of security, reliability, privacy, etc. Test, so as to provide reference for industrial enterprises when selecting database products, and provide reference for database manufacturers to improve product evaluation. The invention provides a user interaction interface that is convenient for users to use, and the test can configure and publish the test task through the interaction interface, and can check the output of the test process and the test result. The present invention includes data sets of 9 data types including relational data, graph data, time series data, document data, wide column data, key-value pair data, event data, message queue data, and binary large objects. Test data set, automatically generate test reports based on the executed test results, display relevant information of the system under test and test results of all test cases.
下面通过附图和实施例,对本发明的技术方案做进一步的详细描述。The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.
附图说明Description of drawings
构成说明书的一部分的附图描述了本发明的实施例,并且连同描述一起用于解释本发明的原理。The accompanying drawings, which form a part of the specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
参照附图,根据下面的详细描述,可以更加清楚地理解本发明,其中:The present invention may be more clearly understood from the following detailed description with reference to the accompanying drawings, wherein:
图1为本发明的工业互联网数据管理服务测试系统的一个实施例的结构示意图;1 is a schematic structural diagram of an embodiment of an industrial Internet data management service testing system of the present invention;
图2为本发明的工业互联网数据管理服务测试方法的一个实施例的流程图。FIG. 2 is a flow chart of an embodiment of a method for testing an industrial Internet data management service of the present invention.
图中,1用户界面模块、11用户登录界面、12被测数据库选择界面、13测试服务器配置界面、14测试用例选择界面、15测试结果展示界面、2测试管理模块、21用户管理模块、22规范符合性测试管理模块、23性能测试管理模块、3测试执行模块、31测试数据管理模块、32测试结果管理模块、33测试报告生成模块、34测试服务器。In the figure, 1 user interface module, 11 user login interface, 12 tested database selection interface, 13 test server configuration interface, 14 test case selection interface, 15 test result display interface, 2 test management module, 21 user management module, 22 specification Compliance test management module, 23 performance test management module, 3 test execution module, 31 test data management module, 32 test result management module, 33 test report generation module, 34 test server.
具体实施方式Detailed ways
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到:除非另外具体说明,否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the invention unless specifically stated otherwise.
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论,但在适当情况下,所述技术、方法和设备应当被视为说明书的一部分。Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.
图1为本发明的工业互联网数据管理服务测试系统的一个实施例的结构示意图,如图1所示,所述工业互联网数据管理服务测试系统包括:1 is a schematic structural diagram of an embodiment of an industrial Internet data management service testing system of the present invention. As shown in FIG. 1 , the industrial Internet data management service testing system includes:
基于本发明实施例的另一个方面,公开一种工业互联网数据管理服务测试系统,包括:Based on another aspect of the embodiments of the present invention, an industrial Internet data management service testing system is disclosed, including:
用户界面模块1、测试管理模块2、测试执行模块3;
所述用户界面模块1通过交互界面接收测试人员的测试相关要求和配置,作为测试计划提交给所述测试管理模块2;The
所述测试管理模块2根据测试计划的类型转发用户的测试计划给所述测试执行模块3;The
所述测试执行模块3对测试计划中的每条测试用例分别执行测试,将所有测试结果数据返回给所述测试管理模块2;The
所述测试管理模块2对所执行测试的结果数据进行统计计算,得到本条测试计划的测试结果,并转发给所述用户界面模块1,由所述用户界面模块1将对应测试的执行结果反馈给测试人员。The
所述用户界面模块1包括:The
用户登录界面11、被测数据库选择界面12、测试服务器配置界面13、测试用例选择界面14和测试结果展示界面15;User login interface 11, tested database selection interface 12, test server configuration interface 13, test case selection interface 14 and test result display interface 15;
通过所述用户登录界面11,测试人员通过用户名和密码进行登录,所述用户界面模块1向所述测试管理模块2发送测试人员通过用户登录界面11输入的用户名和密码信息,所述测试管理模块2经过验证后将登录结果返回给用户登录界面11;Through the user login interface 11, the tester logs in through the user name and password, the
通过被测数据库选择界面12,用户从测试系统给定的数据库中选择测试服务器34的数据库;Through the tested database selection interface 12, the user selects the database of the test server 34 from the given database of the test system;
通过测试服务器配置界面13,用户配置各个数据类型的测试执行模块3的IP地址以及用户名和密码,以选择对应的测试执行模块3中的测试服务器34;Through the test server configuration interface 13, the user configures the IP address, user name and password of the
通过测试用例选择界面14,用户从测试系统给定的测试用例列表中选择针对各个数据类型的测试用例,组合成为测试计划并执行测试;Through the test case selection interface 14, the user selects test cases for each data type from the test case list given by the test system, combines them into a test plan, and executes the test;
测试结果展示界面15将测试过程中输出的日志信息显示在用户界面模块1中,在测试结束后,用户下载自动生成的测试报告。The test result display interface 15 displays the log information output during the test in the
测试人员登录后,可以看到被测数据库选择界面12、测试服务器配置界面13、测试用例选择界面14和测试结果展示界面15,测试人员可以通过勾选和输入等方式对测试服务器34进行配置,制定测试方案,并控制开始测试,相关测试数据传送到测试执行模块3后测试开始执行。相关测试结果数据和测试中间过程的日志信息将由测试管理模块2返回给用户界面模块1,并在测试结果展示界面15中显示。After the tester logs in, he can see the database under test selection interface 12, the test server configuration interface 13, the test case selection interface 14 and the test result display interface 15. The tester can configure the test server 34 by checking and inputting, etc. Formulate a test plan, and control the start of the test. After the relevant test data is transmitted to the
在一个具体的实施例中,给定的测试用例之间可能存在前置条件或包含关系,即要执行测试用例A必须先执行测试用例B,或者测试用例A的测试过程包含测试用例B。根据用户选择的测试用例,系统会自动检查前置条件,补充前置测试用例后生成测试计划,并传递给测试管理模块2。In a specific embodiment, there may be a precondition or an inclusion relationship between given test cases, that is, to execute test case A, test case B must be executed first, or the test process of test case A includes test case B. According to the test case selected by the user, the system will automatically check the preconditions, generate a test plan after supplementing the pre-test case, and pass it to the
测试管理模块2负责进行本测试系统所有测试计划的执行。所述测试管理模块2包括:The
用户管理模块21、规范符合性测试管理模块22、性能测试管理模块23;User management module 21, specification compliance test management module 22, performance test management module 23;
通过用户管理模块21增加或删除有权登录用户界面模块1的用户,并对用户及其权限进行管理,从而确保用户数据及被测系统的系统及数据安全;Add or delete users who have the right to log in to the
根据用户权限,系统可将用户分为管理员用户和普通用户。管理员用户可以新增用户或者删除已存在的用户,并可以对其他用户的密码进行修改;普通用户只有登录和修改自己密码的权限。According to user rights, the system can divide users into administrator users and ordinary users. Administrator users can add new users or delete existing users, and can modify the passwords of other users; ordinary users can only log in and modify their own passwords.
通过规范符合性测试管理模块22对不同数据类型的数据管理系统进行规范;在系统的测试用例管理中,对每条要求设计了多个测试用例供用户选择。用户进行测试时,可以选择单个或多个测试用例进行自由组合,形成测试计划,对被测系统进行规范符合性测试。The data management system of different data types is standardized through the specification compliance test management module 22; in the test case management of the system, multiple test cases are designed for each requirement for the user to choose. When testing, users can select single or multiple test cases to freely combine to form a test plan, and conduct specification compliance testing on the system under test.
性能测试管理模块23用于对被测数据库系统进行性能测试,包括数据迁移效率、数据查询效率、用户鉴权攻击和可信验证时延分析。对于应用于工业互联网系统中的数据库产品,不仅需要符合数据管理服务规范中的功能要求,而且在数据库性能上需要符合工业互联网系统的实际场景需求。The performance test management module 23 is used to perform performance tests on the database system under test, including data migration efficiency, data query efficiency, user authentication attack and trusted verification delay analysis. For database products used in industrial Internet systems, not only must they meet the functional requirements in the data management service specification, but also the database performance needs to meet the actual scenario requirements of the industrial Internet system.
由于不同数据类型的数据特征和常用数据操作不同,进行数据迁移效率和数据查询效率测试的测试用例也有区别。Due to the different data characteristics and common data operations of different data types, the test cases for data migration efficiency and data query efficiency tests are also different.
测试执行模块3接收来自测试管理模块2的用户测试计划,并在对应数据类型的测试服务器34上执行测试,并将测试报告等测试结果信息返回给测试管理模块2,在用户界面模块1上显示。The
测试管理模块2可以接收测试人员对测试服务器的配置、对被测数据库系统和测试用例的选择,并按照数据库类型转发给相应的测试执行模块。测试执行结束后,每条测试用例的测试结果数据,以及测试中间过程的日志信息将会返回给测试管理模块2。测试管理模块2经过计算处理后,将测试结果返回给用户界面模块1。The
所述测试执行模块3包括:The
测试数据管理模块31、测试结果管理模块32、测试报告生成模块33、测试服务器34;Test data management module 31, test result management module 32, test report generation module 33, test server 34;
测试数据管理模块31在执行测试时向被测系统提供对应数据类型的测试数据集,对于需要大量数据导入的被测系统测试,测试数据管理模块31根据工业数据特征生成人工测试数据集;The test data management module 31 provides a test data set of the corresponding data type to the system under test when performing the test, and for the test of the system under test that requires a large amount of data import, the test data management module 31 generates an artificial test data set according to the characteristics of the industrial data;
测试结果管理模块32对测试结果数据进行收集和计算,并反馈给对应的测试管理模块2;The test result management module 32 collects and calculates the test result data, and feeds it back to the corresponding
测试报告生成模块33收集测试过程数据和测试结果信息,将测试的相关数据填充进测试报告模板,形成完整包含本次测试的测试环境、被测系统、测试用例、详细测试结果的测试报告;The test report generation module 33 collects the test process data and the test result information, fills the relevant data of the test into the test report template, and forms a test report that completely includes the test environment, the system under test, the test case, and the detailed test results of this test;
所述测试服务器34包括多种,每一种类型的被测系统对应一种被测服务器,被测系统所在的测试服务器34执行用户指定的测试计划,用户的测试计划由测试管理模块2转发至具体数据类型的测试服务器34后,测试服务器34根据需要执行的测试用例进行测试。The test server 34 includes a variety of types, each type of system under test corresponds to a server under test, the test server 34 where the system under test is located executes the test plan specified by the user, and the test plan of the user is forwarded by the
测试执行模块3从测试管理模块2接收用户指定的测试用例等测试要求,通过测试服务器34控制被测数据库系统执行相应的测试,记录并计算相应的测试结果,并将测试结果通过测试管理模块2和用户界面模块1反馈给测试人员,同时可以自动生成当前测试的测试报告。对于性能测试,按照用户指定的测试用例,测试服务器34控制被测系统对数据导入效率、数据查询效率、用户鉴权能力等性能进行测试,并通过响应时间、出错率等进行结果反馈。The
所述测试用例通过使用被测数据库的API或者使用各语言的API库实现。例如安全性要求中的用户身份认证能力要求、用户权限分配能力要求,以及可靠性要求中的数据操作能力要求等。有的则可能需要单独编写代码对数据进行操作,例如隐私性要求中的信息嵌入能力要求等。The test case is implemented by using the API of the database under test or using the API library of each language. For example, the user identity authentication capability requirements in the security requirements, the user rights allocation capability requirements, and the data operation capability requirements in the reliability requirements, etc. Others may need to write code separately to operate on data, such as information embedding capability requirements in privacy requirements.
对于性能测试,测试执行模块3按照用户指定的测试用例进行测试,测试服务器34控制被测系统对数据导入效率、数据查询效率、用户鉴权能力等性能进行测试,并通过响应时间、出错率等进行结果反馈。For the performance test, the
所述测试数据管理模块31包括:The test data management module 31 includes:
关系数据、图数据、时序数据、文档数据、宽列数据、键值对数据、事件数据、消息队列数据、二进制大对象9种数据类型的数据集。Data sets of 9 data types including relational data, graph data, time series data, document data, wide column data, key-value pair data, event data, message queue data, and binary large objects.
图2为本发明的工业互联网数据管理服务测试方法的一个实施例的流程图,如图2所述,所述工业互联网数据管理服务测试方法包括:FIG. 2 is a flowchart of an embodiment of the industrial Internet data management service testing method of the present invention. As shown in FIG. 2 , the industrial Internet data management service testing method includes:
10,根据测试系统中提出的测试用例要求,以及被测数据库提供的API,编写该被测数据库的测试用例函数以供测试服务器34调用;例如,如果要对被测服务器上的Neo4j数据库的深度查询功能进行测试,则可以编写测试函数,当测试服务器调用该测试函数时,通过Neo4j提供的API与被测服务器系统进行连接后进行数据查询,并记录查询结果。10. According to the test case requirements put forward in the test system and the API provided by the database under test, write the test case function of the database under test for the test server 34 to call; To test the query function, you can write a test function. When the test server calls the test function, it connects with the server system under test through the API provided by Neo4j to query the data and record the query results.
20,通过用户界面模块1选择被测数据库系统和测试用例,通过Django框架以表单提交的方式将用户选择的被测数据库系统以及测试用例ID传送给网站后台的测试管理模块2;例如,测试人员在用户界面中选择了Neo4j服务器,并勾选了“同构图中基于出边类型的深度优先搜索”和“同构图中基于出边类型的广度优先搜索”两个测试用例。用户通过点击按钮,将以上测试计划通过提交表单方式传递给网站后台。20. Select the database system under test and test case through the
30,测试管理模块2接收到由前端提交的测试计划后,通过发送post请求的方式,将该测试计划根据其被测数据库类型发送给测试执行模块3中对应数据类型的测试服务器34;例如,后台接收到包含上述第2步中测试计划的表单数据后,向Neo4j测试服务器发送包含用户所勾选测试用例ID等的测试计划数据。30, after the
40,测试服务器34接收到测试计划后,根据用户指定的测试用例,调用测试用例函数,在被测系统服务器上进行测试,并记录下测试结果,将测试结果返回给测试管理模块2;例如,Neo4j测试服务器接收了包含用户所勾选测试用例ID的数据,分别调用对应“同构图中基于出边类型的深度优先搜索”和“同构图中基于出边类型的广度优先搜索”两个测试用例的测试函数,在测试服务器34上进行测试,并记录测试程序运行结果,将运行结果返回给测试管理模块2。40. After the test server 34 receives the test plan, according to the test case specified by the user, the test case function is called, and the test is performed on the system under test server, and the test result is recorded, and the test result is returned to the
50,测试管理模块2接收到测试服务器34返回的测试用例运行结果,将相关结果数据返回给用户界面模块1;50, the
60,用户界面模块1根据用户选择的测试计划,将用户选择的数据库、测试用例以及运行的测试结果自动填充进Latex模板,并编译生成测试报告PDF文件,提供给测试人员下载。60. The
所述根据测试系统中提出的测试用例要求,以及被测数据库提供的API,编写该被测数据库的测试用例函数以供测试服务器34调用包括:每种被测数据库系统具有单独的测试服务器34,负责接收具体的测试请求并在被测系统服务器上执行测试。Described according to the test case requirements proposed in the test system and the API provided by the database under test, writing the test case function of the database under test for the test server 34 to call includes: each database system under test has a separate test server 34, Responsible for receiving specific test requests and executing tests on the system under test server.
工业互联网数据管理系统在工业互联网环境下的功能和性能需要进行量化评估,包括数据操作能力、数据查询效率、数据迁移效率等。同时,数据管理系统可能存在系统安全性、数据隐私性等问题。工业互联网数据管理系统的性能与系统架构、数据规模和基础硬件环境等多种因素有关,对同类型的不同工业互联网数据管理产品进行比较,需要在相同条件下对其进行性能测评。测试人员可以通过测试系统对软件进行测试,通过测试系统,可以在同一条件下对被测系统进行测试,检验是否符合功能和性能要求。The functions and performance of the industrial Internet data management system in the industrial Internet environment need to be quantitatively evaluated, including data operation capabilities, data query efficiency, and data migration efficiency. At the same time, the data management system may have problems such as system security and data privacy. The performance of the industrial Internet data management system is related to various factors such as system architecture, data scale, and basic hardware environment. To compare different industrial Internet data management products of the same type, it is necessary to evaluate their performance under the same conditions. Testers can test the software through the test system, and through the test system, they can test the system under test under the same conditions to check whether it meets the functional and performance requirements.
测试人员可以对不同数据库管理系统从不同角度进行测试。Testers can test different database management systems from different angles.
首先,测试人员需要进行登录。通过用户登录界面11,测试人员输入用户名和密码。用户名及经过加密的密码传输至测试系统的测试管理模块2。测试管理模块2中的用户管理模块21验证用户名和密码是否正确后,将对应登录结果返回用户界面模块1,显示给用户。First, the tester needs to log in. Through the user login interface 11, the tester enters a user name and password. The user name and encrypted password are transmitted to the
测试人员登录成功后,通过系统的用户界面模块1,可以输入测试计划并对被测系统进行设置。测试计划指的是,用户在测试系统提供的所有测试用例中所选择的部分测试用例。用户对被测系统的设置包括指定被测系统所在服务器的IP地址,以及输入被测系统的用户名和密码。After the tester logs in successfully, through the
测试系统提供的测试用例可分为使能类测试用例、可信类测试用例以及性能测试用例等多种类型。使能类测试用例是指对被测的数据管理系统是否有能力完成数据管理的功能进行测试的测试用例。被测系统应当具备在一定时间、一定条件下可以完成必要的数据操作,执行指定功能,从而实现数据管理功能的能力,包括数据管理系统的数据写入能力、数据查询能力、数据修改能力、数据删除能力、数据导入和导出能力,以及多个服务实例之间的隔离能力。可信类测试用例是指对被测数据管理系统所提供的数据管理功能的稳定性与可靠性进行测试的测试用例。具体来说,被测系统应当具备拒绝非授权访问,防止系统被篡改或破坏,以及受到攻击后恢复正常功能的能力,从而保证系统所提供服务的安全与稳定;被测系统应当允许信息拥有者有权控制下列内容:与信息拥有者相关的信息可以向哪些人暴露、相关信息的使用授权,以及是否允许相关信息被收集、处理和存储,并可以在数据中隐式嵌入数据发布者或接收者的信息以进行溯源,从而保证用户隐私不被泄露;被测系统可以通过用户鉴权及用户权限控制等方式保证数据来源真实可靠以及数据传播范围稳定可控。性能测试用例是指对被测数据管理系统的性能进行测试的测试用例,包括数据导入/导出等数据迁移效率和数据查询效率等。The test cases provided by the test system can be divided into various types such as enabling test cases, trusted test cases and performance test cases. The enabling test case refers to the test case that tests whether the data management system under test is capable of completing the function of data management. The system under test should have the ability to complete necessary data operations and perform specified functions within a certain period of time and under certain conditions, thereby realizing data management functions, including data writing capabilities, data query capabilities, data modification capabilities, and data management capabilities of the data management system. The ability to delete, import and export data, and isolate between multiple service instances. Trusted test cases refer to test cases that test the stability and reliability of the data management functions provided by the data management system under test. Specifically, the system under test should have the ability to refuse unauthorized access, prevent the system from being tampered with or destroyed, and restore normal functions after being attacked, so as to ensure the security and stability of the services provided by the system; the system under test should allow information owners The right to control who the information about the owner of the information can be exposed to, the authorization to use the relevant information, and whether the relevant information is allowed to be collected, processed and stored, and can be implicitly embedded in the data publisher or receiver The user's information can be traced back to ensure that user privacy is not leaked; the system under test can ensure the authenticity and reliability of the data source and the stable and controllable data dissemination range through user authentication and user permission control. Performance test cases refer to test cases for testing the performance of the data management system under test, including data migration efficiency such as data import/export and data query efficiency.
测试管理模块2接收测试人员的测试计划,根据测试计划中的配置连接对应的测试服务器34,并根据数据类型调用测试计划中的测试用例所对应的各个实现接口,从而调用测试执行模块3中的测试服务器34执行对应测试。The
执行测试时,根据不同的测试用例类型,测试结果可能有不同形式。对于使能类测试用例和可信类测试用例,测试执行模块3在被测系统上尝试执行对应操作,根据能否正确完成相应操作得到该测试用例是否通过的测试结果;对于性能测试用例,测试执行模块3根据要求在被测系统上进行多次数据导入/导出/查询等操作,记录消耗的总时长,从而计算单次操作的平均时间开销,得到该测试用例的测试结果。所有测试用例执行完毕后,测试执行模块3记录所有测试结果,经过测试管理模块2处理后,将测试结果在用户界面模块1中显示给用户。When performing tests, the test results may take different forms depending on the type of test case. For enabling test cases and trusted test cases, the
本说明书中各个实施例均采用递进的方式描述,每个实施例重点说明的都是与其它实施例的不同之处,各个实施例之间相同或相似的部分相互参见即可。对于系统实施例而言,由于其与方法实施例基本对应,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may be referred to each other. As for the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for related parts, please refer to the partial description of the method embodiment.
本发明的描述是为了示例和描述起见而给出的,而并不是无遗漏的或者将本发明限于所公开的形式。很多修改和变化对于本领域的普通技术人员而言是显然的。选择和描述实施例是为了更好说明本发明的原理和实际应用,并且使本领域的普通技术人员能够理解本发明从而设计适于特定用途的带有各种修改的各种实施例。The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the invention to the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to better explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use.
Claims (8)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910452355 | 2019-05-28 | ||
CN2019104523557 | 2019-05-28 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112015715A true CN112015715A (en) | 2020-12-01 |
Family
ID=73506906
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910851128.1A Pending CN112015715A (en) | 2019-05-28 | 2019-09-10 | Industrial Internet data management service testing method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112015715A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112527878A (en) * | 2020-12-15 | 2021-03-19 | 普华基础软件股份有限公司 | Automatic tool for IO performance test of database |
CN115396896A (en) * | 2022-10-27 | 2022-11-25 | 中汽研软件测评(天津)有限公司 | Vehicle WiFi information safety testing method and device |
CN116192922A (en) * | 2023-04-23 | 2023-05-30 | 成都华兴汇明科技有限公司 | Issuing management method, device and system for test cases |
CN119854153A (en) * | 2025-03-24 | 2025-04-18 | 国汽(北京)智能网联汽车研究院有限公司 | Network and data security test system, method, device and readable storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20010054044A1 (en) * | 2000-06-14 | 2001-12-20 | Liu Yi Bo | Method for monitoring and browsing database of test system |
CN104133772A (en) * | 2014-08-13 | 2014-11-05 | 广东电网公司信息中心 | Automatic test data generation method |
CN107832226A (en) * | 2017-11-23 | 2018-03-23 | 中国平安人寿保险股份有限公司 | Report-generating method, device, equipment and computer media based on performance test |
CN108255712A (en) * | 2017-12-29 | 2018-07-06 | 曙光信息产业(北京)有限公司 | The test system and test method of data system |
CN109086216A (en) * | 2018-10-26 | 2018-12-25 | 苏州友教习亦教育科技有限公司 | Automatization test system |
-
2019
- 2019-09-10 CN CN201910851128.1A patent/CN112015715A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20010054044A1 (en) * | 2000-06-14 | 2001-12-20 | Liu Yi Bo | Method for monitoring and browsing database of test system |
CN104133772A (en) * | 2014-08-13 | 2014-11-05 | 广东电网公司信息中心 | Automatic test data generation method |
CN107832226A (en) * | 2017-11-23 | 2018-03-23 | 中国平安人寿保险股份有限公司 | Report-generating method, device, equipment and computer media based on performance test |
CN108255712A (en) * | 2017-12-29 | 2018-07-06 | 曙光信息产业(北京)有限公司 | The test system and test method of data system |
CN109086216A (en) * | 2018-10-26 | 2018-12-25 | 苏州友教习亦教育科技有限公司 | Automatization test system |
Non-Patent Citations (1)
Title |
---|
LIU XIAOCHUANG: ""Research on performance testing model of databases of different structure"", 《2019 2ND INTERNATIONAL CONFERENCE ON MECHANICAL, ELECTRONIC AND ENGINEERING TECHNOLOGY》, pages 132 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112527878A (en) * | 2020-12-15 | 2021-03-19 | 普华基础软件股份有限公司 | Automatic tool for IO performance test of database |
CN115396896A (en) * | 2022-10-27 | 2022-11-25 | 中汽研软件测评(天津)有限公司 | Vehicle WiFi information safety testing method and device |
CN116192922A (en) * | 2023-04-23 | 2023-05-30 | 成都华兴汇明科技有限公司 | Issuing management method, device and system for test cases |
CN116192922B (en) * | 2023-04-23 | 2023-08-11 | 成都华兴汇明科技有限公司 | Issuing management method, device and system for test cases |
CN119854153A (en) * | 2025-03-24 | 2025-04-18 | 国汽(北京)智能网联汽车研究院有限公司 | Network and data security test system, method, device and readable storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2126772B1 (en) | Assessment and analysis of software security flaws | |
CN112015715A (en) | Industrial Internet data management service testing method and system | |
WO2018113596A1 (en) | Method of processing application reviewing operation permission, device, and data storage medium | |
WO2019100576A1 (en) | Automated test management method and apparatus, terminal device, and storage medium | |
US10691822B1 (en) | Policy validation management | |
CN107977308A (en) | interface test method and device | |
US10379984B2 (en) | Compliance testing through sandbox environments | |
US20080235041A1 (en) | Enterprise data management | |
CN106445485A (en) | Software version consistency detection system and detection method thereof | |
CN116541372A (en) | Data asset management method and system | |
CN111522728A (en) | Method for generating automatic test case, electronic device and readable storage medium | |
CN111897721B (en) | Automatic testing method of API (application program interface) and storage medium | |
US20190138433A1 (en) | Evaluation of library test suites using mutation testing | |
WO2021129335A1 (en) | Operation monitoring method and apparatus, operation analysis method and apparatus | |
Li et al. | LogicScope: Automatic discovery of logic vulnerabilities within web applications | |
CN111694743A (en) | Service system detection method and device | |
CN113901476A (en) | A kind of vulnerability verification method, system, device and medium based on virtualization environment | |
CN106130830A (en) | The method of testing of safety equipment stability and test device | |
CN114595127A (en) | Log exception handling method, device, equipment and storage medium | |
CN110309062A (en) | Use case generation method, device, electronic device and storage medium | |
US12298995B1 (en) | Systems, methods, and computer-readable media for managing an extract, transform, and load process | |
CN114154161A (en) | To-be-audited contract auditing method and device, computer equipment and storage medium | |
CN113568838B (en) | Test data generation method, device, equipment, storage medium and program product | |
CN110659897A (en) | Method, system, computing device and medium for transaction verification | |
CN113868116B (en) | Method and device for generating test dependency data, server and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |