[go: up one dir, main page]

CN114048214A - Data analysis and acquisition method and device and terminal equipment - Google Patents

Data analysis and acquisition method and device and terminal equipment Download PDF

Info

Publication number
CN114048214A
CN114048214A CN202111364056.1A CN202111364056A CN114048214A CN 114048214 A CN114048214 A CN 114048214A CN 202111364056 A CN202111364056 A CN 202111364056A CN 114048214 A CN114048214 A CN 114048214A
Authority
CN
China
Prior art keywords
data
data collection
data acquisition
tool
acquisition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111364056.1A
Other languages
Chinese (zh)
Inventor
周仕波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Buer Automation Engineering Co ltd
Original Assignee
Anhui Buer Automation Engineering Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Buer Automation Engineering Co ltd filed Critical Anhui Buer Automation Engineering Co ltd
Priority to CN202111364056.1A priority Critical patent/CN114048214A/en
Publication of CN114048214A publication Critical patent/CN114048214A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention discloses a data analysis and acquisition method, a data analysis and acquisition device and terminal equipment, wherein the method comprises the steps of sequentially sending data acquisition requests to data acquisition equipment; the analysis protocol corresponding to the data acquisition request is preset with a protocol index according to the sending sequence of the data acquisition request; and receiving the collected data returned by the data collection equipment according to the data collection request, and setting data indexes for the collected data according to the return sequence. Has the advantages that: the data acquisition tool is independent, when the business system needs to acquire data, the data acquisition can be realized through the independent data acquisition tool, the system is prevented from being transformed and integrated for acquiring the data, and the problems of complex construction and deployment and high cost of projects are solved.

Description

Data analysis and acquisition method and device and terminal equipment
Technical Field
The invention relates to the technical field of data acquisition, in particular to a data analysis acquisition method, a data analysis acquisition device and terminal equipment.
Background
The information is an important resource of modern enterprises and is the basis of scientific management and decision analysis of the enterprises. Currently, most enterprises spend a lot of money and time to construct business systems for online transactions, which are used to record various relevant data of the transactions. In this context, data warehouses are used. The data warehouse is used for integrating the generated data together to discover the commercial value and provide decision support. The data warehouse is generated by selecting useful data from a large amount of source data, and storing the useful data into the data warehouse after a series of processing.
At present, the methods for collecting source data and processing data generally compile or order processing tools highly related to the required service data by a service data requiring party according to the service requirements of the business data requiring party, and integrate the processing tools into a service system of the business data requiring party.
Disclosure of Invention
The invention aims to provide a data analysis and acquisition method, a data analysis and acquisition device and terminal equipment.
The technical scheme of the invention is realized as follows:
a data analysis and acquisition method, a data analysis and acquisition device and terminal equipment comprise the steps of sequentially sending data acquisition requests to data acquisition equipment;
the analysis protocol corresponding to the data acquisition request is preset with a protocol index according to the sending sequence of the data acquisition request;
receiving the collected data returned by the data collection equipment according to the data collection request, and setting data indexes for the collected data according to the return sequence;
when an analysis instruction is received, determining a target protocol index through a data index of target acquisition data, acquiring a target analysis protocol according to the target protocol index, and analyzing the target acquisition data to obtain target analysis data;
the data acquisition request receives data acquisition setting information input by a user through a service interface and sends the data acquisition setting information to the data acquisition tool; acquiring data from a data acquisition terminal based on the data acquisition setting information through a universal interface of the data acquisition tool; and after receiving the information of data acquisition completion sent by the data acquisition tool, calling the acquired data from the database of the data acquisition tool.
Further, the data acquisition request and the corresponding resolution protocol are stored in a protocol list according to the sending sequence of the data acquisition request.
Further, after receiving the collected data returned by the data collecting device according to the data collecting request and setting a data index for each collected data according to the returning sequence, the method further includes: and storing the acquired data corresponding to the data acquisition request to a data list according to the return sequence.
Further, after sending the data collection setting information to the data collection tool, the method further comprises: and acquiring the state information of the data acquired by the data acquisition tool, and displaying the state information in real time through the service interface so as to facilitate a user to monitor and view the data acquisition state through the service interface.
A data acquisition device comprising: the display unit is used for displaying a service interface of a pre-established data acquisition tool after a preset instruction is acquired; the sending unit is used for receiving data acquisition setting information input by a user through the service interface and sending the data acquisition setting information to the data acquisition tool; the acquisition unit is used for acquiring data from a data acquisition end through a universal interface of the data acquisition tool based on the data acquisition setting information; and the calling unit is used for calling the acquired data from the database of the data acquisition tool after receiving the information of data acquisition completion sent by the data acquisition tool.
Further, the data acquisition device further comprises: and the acquisition unit is used for acquiring the state information of the data acquired by the data acquisition tool after the data acquisition setting information is sent to the data acquisition tool, and displaying the state information in real time through the service interface so as to facilitate a user to monitor and check the data acquisition state through the service interface.
A terminal device, comprising: memory, processor and computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 4 when executing the computer program.
The invention has the beneficial effects that: the corresponding protocol indexes are preset according to the sending sequence of the analysis protocols corresponding to the data acquisition requests of the data acquisition equipment, the data indexes are set according to the returning sequence of the analysis protocols, the data acquisition requests sent first return the corresponding acquisition data, and therefore the corresponding protocol indexes can be obtained according to the data indexes, the corresponding analysis protocols are obtained, the acquisition data are analyzed according to the analysis protocols, and the analysis data are obtained. In addition, the invention receives data acquisition setting information input by a user through a service interface which displays a pre-established data acquisition tool, and sends the data acquisition setting information to the data acquisition tool, acquires data from a data acquisition end through a universal interface of the data acquisition tool based on the data acquisition setting information, and calls the acquired data from a database of the data acquisition tool after receiving the information which is sent by the data acquisition tool and completes data acquisition The cost is high.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
According to the embodiment of the invention, a data analysis and acquisition method, a data analysis and acquisition device and terminal equipment are provided.
The data analysis and acquisition method, the data analysis and acquisition device and the terminal equipment provided by the embodiment of the invention comprise the steps of sequentially sending each data acquisition request to the data acquisition equipment;
the analysis protocol corresponding to the data acquisition request is preset with a protocol index according to the sending sequence of the data acquisition request;
receiving the collected data returned by the data collection equipment according to the data collection request, and setting data indexes for the collected data according to the return sequence;
when an analysis instruction is received, determining a target protocol index through a data index of target acquisition data, acquiring a target analysis protocol according to the target protocol index, and analyzing the target acquisition data to obtain target analysis data;
the data acquisition request receives data acquisition setting information input by a user through a service interface and sends the data acquisition setting information to the data acquisition tool; acquiring data from a data acquisition terminal based on the data acquisition setting information through a universal interface of the data acquisition tool; and after receiving the information of data acquisition completion sent by the data acquisition tool, calling the acquired data from the database of the data acquisition tool.
Specifically, the data acquisition requests are based on the received data acquisition requests, in the actual data communication process, the types and the number of the data acquisition requests are not unique, the data acquisition requests may be data acquisition requests for different data acquisition devices and different requested data lengths, or data acquisition requests for different data acquisition devices and the same requested data length, and for such data acquisition requests, due to the different types, corresponding comparison can be performed according to the corresponding request information and the data information of the acquired data, without setting an index. Therefore, the data acquisition request can be directly sent to the corresponding data acquisition equipment and acquired data acquired by the data acquisition equipment; further, when the analysis instruction is received, the corresponding analysis protocol can be directly determined according to the relevant information, and data analysis is completed.
Further, the data acquisition request and the corresponding resolution protocol are stored in a protocol list according to the sending sequence of the data acquisition request.
Further, after receiving the collected data returned by the data collecting device according to the data collecting request and setting a data index for each collected data according to the returning sequence, the method further includes: and storing the acquired data corresponding to the data acquisition request to a data list according to the return sequence.
Further, after sending the data collection setting information to the data collection tool, the method further comprises: and acquiring the state information of the data acquired by the data acquisition tool, and displaying the state information in real time through the service interface so as to facilitate a user to monitor and view the data acquisition state through the service interface.
A data acquisition device comprising: the display unit is used for displaying a service interface of a pre-established data acquisition tool after a preset instruction is acquired; the sending unit is used for receiving data acquisition setting information input by a user through the service interface and sending the data acquisition setting information to the data acquisition tool; the acquisition unit is used for acquiring data from a data acquisition end through a universal interface of the data acquisition tool based on the data acquisition setting information; and the calling unit is used for calling the acquired data from the database of the data acquisition tool after receiving the information of data acquisition completion sent by the data acquisition tool.
Further, the data acquisition device further comprises: and the acquisition unit is used for acquiring the state information of the data acquired by the data acquisition tool after the data acquisition setting information is sent to the data acquisition tool, and displaying the state information in real time through the service interface so as to facilitate a user to monitor and check the data acquisition state through the service interface.
A terminal device, comprising: the device comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the steps of the data analysis and collection method when executing the computer program.
Specifically, an 'automatic trigger' button and a 'trigger' button are preset on a service interface of the data acquisition tool; after the data acquisition setting information is sent to the data acquisition tool, before data is acquired from a data acquisition end through a general interface of the data acquisition tool based on the data acquisition setting information, a system detects whether an 'automatic trigger' button is pressed within a preset time; when the system detects that an 'automatic trigger' button is pressed, namely an automatic trigger instruction is obtained, the system immediately sends an instruction for starting a data conversion task to the data acquisition tool, and the data acquisition tool starts the data conversion task after receiving the instruction for starting the data conversion task sent by the service system; when the system does not detect that the 'automatic trigger' button is pressed, namely a manual trigger instruction is obtained, the service system monitors whether a user manually clicks the 'trigger' button, once the fact that the user clicks the 'trigger' button is monitored, the system sends an instruction for starting a data conversion task to the data acquisition tool, and the data acquisition tool starts the data conversion task after receiving the instruction for starting the data conversion task sent by the service system.
Further, acquiring data from a data acquisition terminal based on the data acquisition setting information through the universal interface of the data acquisition tool further includes: acquiring data according to the data acquisition setting information through a data conversion file in the data acquisition tool, performing escape processing on the acquired data through the data conversion file in the data acquisition tool, inserting the processed data into a database of the data acquisition tool, and specifically retrieving the acquired data from the database of the data acquisition tool as retrieving the processed data from the database of the data acquisition tool.
In practical application, when the dictionary at the data acquisition end and the database dictionary of the data acquisition tool are different in definition mode, after data are acquired, the acquired data need to be subjected to escaping processing, and the acquired data are escaped to be processed into data which can be identified by the database dictionary of the data acquisition tool, so that the acquired data can be stored in the database of the data acquisition tool. In an embodiment of the present invention, when data is collected from a data collection terminal based on the data collection setting information through a general interface of the data collection tool, the method further includes: and monitoring the error information in the data acquisition process, and generating an acquisition log from the error information, so that a user can conveniently check and analyze the error information in the data acquisition process.
In the actual data communication process, the types and the number of the data acquisition requests are not unique, and may be data acquisition requests for different data acquisition devices and different request data lengths, or data acquisition requests for different data acquisition devices and the same request data length. Therefore, the data acquisition request can be directly sent to the corresponding data acquisition equipment and acquired data acquired by the data acquisition equipment; further, when the analysis instruction is received, the corresponding analysis protocol can be directly determined according to the relevant information, and data analysis is completed.
The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.

Claims (7)

1.一种数据解析采集方法，其特征在于，包括1. a data analysis collection method, is characterized in that, comprises 将各个数据采集请求依次发送至数据采集设备；Send each data collection request to the data collection device in turn; 所述数据采集请求对应的解析协议按照所述数据采集请求的发送顺序预设有协议索引；The parsing protocol corresponding to the data collection request is preset with a protocol index according to the sending order of the data collection request; 接收所述数据采集设备根据所述数据采集请求返回的采集数据，并对各个所述采集数据按照返回顺序设置数据索引；Receive the collection data returned by the data collection device according to the data collection request, and set a data index for each of the collected data according to the return sequence; 当接收到解析指令时，通过目标采集数据的数据索引确定目标协议索引，根据所述目标协议索引获取目标解析协议对所述目标采集数据进行解析，获得目标解析数据；When the parsing instruction is received, the target protocol index is determined according to the data index of the target acquisition data, and the target parsing protocol is obtained according to the target protocol index, and the target acquisition data is analyzed to obtain the target parsing data; 数据采集请求通过服务界面接收用户输入的数据采集设置信息，并将所述数据采集设置信息发送给所述数据采集工具；通过所述数据采集工具的通用接口基于所述数据采集设置信息从数据采集端采集数据；在接收到所述数据采集工具发送的数据采集完成的信息之后，从所述数据采集工具的数据库中调取采集到的数据。The data collection request receives the data collection setting information input by the user through the service interface, and sends the data collection setting information to the data collection tool; The terminal collects data; after receiving the data collection completion information sent by the data collection tool, the collected data is retrieved from the database of the data collection tool. 2.根据权利要求1所述的一种数据解析采集方法，其特征在于，所述数据采集请求与对应的解析协议按照数据采集请求的发送顺序保存于协议列表中。2 . The data analysis and collection method according to claim 1 , wherein the data collection request and the corresponding analysis protocol are stored in a protocol list according to the sending order of the data collection request. 3 . 3.根据权利要求1所述的一种数据解析采集方法，其特征在于，接收所述数据采集设备根据所述数据采集请求返回的采集数据，并对各个所述采集数据按照返回顺序设置数据索引之后，还包括：将数据采集请求对应的采集数据按照所述返回顺序保存至数据列表。3. A data analysis and collection method according to claim 1, characterized in that, receiving the collection data returned by the data collection device according to the data collection request, and setting a data index for each of the collected data according to the return sequence Afterwards, the method further includes: saving the collected data corresponding to the data collection request in the data list according to the returned order. 4.如权利要求1所述的数据采集方法，其特征在于，在将所述数据采集设置信息发送给所述数据采集工具之后，所述方法还包括：获取所述数据采集工具采集数据的状态信息，并将所述状态信息通过所述服务界面实时显示，以便于用户通过所述服务界面监控和查看数据采集的状态。4. The data collection method according to claim 1, wherein after the data collection setting information is sent to the data collection tool, the method further comprises: acquiring the state of the data collected by the data collection tool information, and display the status information in real time through the service interface, so that the user can monitor and view the status of data collection through the service interface. 5.一种数据采集装置，其特征在于，包括：显示单元，用于在获取到预定指令后，显示预先创建的数据采集工具的服务界面；发送单元，用于通过所述服务界面接收用户输入的数据采集设置信息，并将所述数据采集设置信息发送给所述数据采集工具；采集单元，用于通过所述数据采集工具的通用接口基于所述数据采集设置信息从数据采集端采集数据；调取单元，用于在接收到所述数据采集工具发送的数据采集完成的信息之后，从所述数据采集工具的数据库中调取采集到的数据。5. A data collection device, comprising: a display unit for displaying a pre-created service interface of a data collection tool after obtaining a predetermined instruction; a sending unit for receiving user input through the service interface the data collection setting information, and send the data collection setting information to the data collection tool; the collection unit is used to collect data from the data collection end based on the data collection setting information through the general interface of the data collection tool; The retrieval unit is configured to retrieve the collected data from the database of the data collection tool after receiving the data collection completion information sent by the data collection tool. 6.根据权利要求5所述的一种数据采集装置，其特征在于，所述数据采集装置还包括：获取单元，用于在将所述数据采集设置信息发送给所述数据采集工具之后，获取所述数据采集工具采集数据的状态信息，并将所述状态信息通过所述服务界面实时显示，以便于用户通过所述服务界面监控和查看数据采集的状态。6 . The data acquisition device according to claim 5 , wherein the data acquisition device further comprises: an acquisition unit, configured to acquire the data acquisition setting information after sending the data acquisition setting information to the data acquisition tool. 7 . The data collection tool collects state information of the data, and displays the state information in real time through the service interface, so that the user can monitor and view the state of data collection through the service interface. 7.一种终端设备，其特征在于，包括：存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1至4任一项所述方法的步骤。7. A terminal device, characterized in that it comprises: a memory, a processor, and a computer program stored in the memory and running on the processor, the processor implementing the computer program as claimed when executing the computer program The steps of any one of claims 1 to 4 of the method.
CN202111364056.1A 2021-11-17 2021-11-17 Data analysis and acquisition method and device and terminal equipment Pending CN114048214A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111364056.1A CN114048214A (en) 2021-11-17 2021-11-17 Data analysis and acquisition method and device and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111364056.1A CN114048214A (en) 2021-11-17 2021-11-17 Data analysis and acquisition method and device and terminal equipment

Publications (1)

Publication Number Publication Date
CN114048214A true CN114048214A (en) 2022-02-15

Family

ID=80209985

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111364056.1A Pending CN114048214A (en) 2021-11-17 2021-11-17 Data analysis and acquisition method and device and terminal equipment

Country Status (1)

Country Link
CN (1) CN114048214A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105491038A (en) * 2015-12-07 2016-04-13 高新兴科技集团股份有限公司 Intelligent device protocol analysis method
CN109450922A (en) * 2018-11-29 2019-03-08 厦门科灿信息技术有限公司 A kind of communication data analytic method, device and relevant device
CN110019501A (en) * 2017-08-24 2019-07-16 深圳市金证科技股份有限公司 A kind of collecting method, device and terminal device
CN110222253A (en) * 2019-06-12 2019-09-10 北京睦合达信息技术股份有限公司 A kind of collecting method, equipment and computer readable storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105491038A (en) * 2015-12-07 2016-04-13 高新兴科技集团股份有限公司 Intelligent device protocol analysis method
CN110019501A (en) * 2017-08-24 2019-07-16 深圳市金证科技股份有限公司 A kind of collecting method, device and terminal device
CN109450922A (en) * 2018-11-29 2019-03-08 厦门科灿信息技术有限公司 A kind of communication data analytic method, device and relevant device
CN110222253A (en) * 2019-06-12 2019-09-10 北京睦合达信息技术股份有限公司 A kind of collecting method, equipment and computer readable storage medium

Similar Documents

Publication Publication Date Title
CN110245078B (en) Software pressure testing method and device, storage medium and server
CN109710615B (en) Database access management method, system, electronic device and storage medium
JP6823265B2 (en) Analytical instruments, analytical systems, analytical methods and analytical programs
US7913233B2 (en) Performance analyzer
CN110019349A (en) Sentence warning method, apparatus, device, and computer-readable storage medium
US20190197140A1 (en) Automation of sql tuning method and system using statistic sql pattern analysis
CN111367760B (en) Log collection method and device, computer equipment and storage medium
EP3384391B1 (en) Real-time change data from disparate sources
CN110515808A (en) Database monitoring method, device, computer equipment and storage medium
CN111352969A (en) Buried point data analysis method and device and electronic equipment
CN111124891A (en) Access state detection method and device, storage medium and electronic device
CN113392005A (en) Large file processing test method and system
CN109783284A (en) Information acquisition method, system and server, computer readable storage medium
CN107634878A (en) A performance test management platform
CN113283760B (en) Case flow analysis report generation method and system
CN114020580A (en) Method, system, device and storage medium for monitoring performance index of application system
CN110011845B (en) Log collection method and system
CN114048214A (en) Data analysis and acquisition method and device and terminal equipment
CN112948262A (en) System test method, device, computer equipment and storage medium
CN118747164A (en) A log-based risk management method and system
CN111506455A (en) Method and device for checking service release result
CN114064650A (en) A method of data analysis and collection
CN117349307A (en) Method, device, equipment and storage medium for acquiring associated data of slow query
CN114039878B (en) Network request processing method and device, electronic equipment and storage medium
CN117033125A (en) Application relation intelligent construction method based on probe, metadata acquisition method, medium and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination