CN105119758A - Data collection method and collection system - Google Patents

Abstract

The embodiment of the present invention discloses a data collection method and a collection system, which relate to the field of communication technology and can solve the problem of waste of interface resources caused by repeated acquisition of data from the same network interface during the data collection process. The method in the embodiment of the present invention includes: the sharing layer obtains the byte requirements of each application program in the application layer respectively, and the byte requirements are used to represent the relevant information of the data bytes required by the application program; According to the byte requirements of the program, the target data is obtained through a unified interface; the sharing layer sends the target data corresponding to each application to each application in the application layer. The invention is applicable to a collection system.

Description

A method and system for collecting data

technical field

The invention relates to the field of communication technology, in particular to a data collection method and a collection system.

Background technique

At present, in the process of data collection, it is usually possible to obtain data from different network interfaces according to the source of the data required by the application, and integrate the data obtained from different network interfaces to facilitate the use of the application, that is, the traditional " Chimney" collection.

With the development of communication technology, a variety of applications have emerged, and data collection needs to be performed according to the byte requirements of different applications. For example: application 1 needs to obtain data from a C/D interface and an SCP (Service Control Point, Service Control Point) interface, while application 2 only needs to obtain data from a C/D interface. In this way, data from the C/D interface needs to be acquired twice, resulting in repeated acquisition of data from the same network interface, resulting in waste of interface resources during the data acquisition process.

Contents of the invention

Embodiments of the present invention provide a data collection method and a collection system, which can solve the problem of waste of interface resources caused by repeated acquisition of data from the same network interface during the data collection process.

In order to achieve the above object, the embodiment of the present invention adopts the following technical solutions:

In a first aspect, an embodiment of the present invention provides a method for data collection, the method is used in a collection system, the collection system is provided with an application layer and a shared layer, and the method includes:

The sharing layer respectively obtains the byte requirement of each application program of the application layer, and the byte requirement is used to represent the relevant information of the data byte required by the application program;

The sharing layer obtains the target data through a unified interface according to the byte requirement of each application program of the application layer;

The sharing layer sends the target data corresponding to each application program to the each application program of the application layer.

In the second aspect, an embodiment of the present invention provides a collection system, the collection system is provided with an application layer and a sharing layer, and the sharing layer is used to separately obtain the byte requirements of each application program of the application layer, so The above byte requirements are used to represent the relevant information of the data bytes required by the application;

The sharing layer is further configured to obtain target data through a unified interface according to the byte requirement of each application program of the application layer;

The sharing layer is further configured to send the target data corresponding to each application program to the each application program in the application layer.

In the data collection method and collection system provided by the embodiments of the present invention, the sharing layer separately obtains the byte requirements of each application program in the application layer, and obtains the target data through a unified interface according to the byte requirements of each application program, The target data corresponding to each application is then sent to each application. Among them, the byte requirement is used to represent the relevant information of the data byte required by the application program. Compared with obtaining the data required by each application program from each network interface for different application programs in the prior art, the embodiment of the present invention can uniformly obtain target data according to the byte requirements of each application program. Therefore, when multiple applications need to obtain the data of the same network interface, because the target data is obtained through the unified interface, it avoids repeated collection of data when different applications call the same data, thereby solving the problem of data collection. , the problem of wasting interface resources caused by repeatedly obtaining data from the same network interface.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

FIG. 1 is a schematic diagram of a specific application scenario provided by an embodiment of the present invention;

Fig. 2 is a flow chart of a data acquisition method provided by an embodiment of the present invention;

FIG. 3 is a flow chart of another data collection method provided by an embodiment of the present invention;

FIG. 4 is a flow chart of another data collection method provided by an embodiment of the present invention;

Fig. 5 is a schematic structural diagram of an acquisition system provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The embodiment of the present invention can be used in an acquisition system. As shown in FIG. 1 , the acquisition system can be provided with an application layer, a sharing layer and an acquisition layer. The application layer is provided with N application programs, which are application program 1 to application program N respectively. Among them, the application program 1 to the application program N can be specifically various service platforms and business systems, and the business system can include at least one of the business systems such as government department security systems, network optimization systems, and online record query systems; the sharing layer can include data Sharing layer and data decoding layer. The data sharing layer can perform the association and analysis of communication protocols, and standardize and output the processing results, that is, the target data corresponding to each application. At the same time, it can also be responsible for the storage of target data and data decoding. The layer can perform functions such as decoding of network data packets, identification of communication protocols, and generation of bills. The decoding of network data packets can specifically obtain raw data from the target network interface; the acquisition layer can also be called the preprocessing layer, It is mainly used to complete the collection and preprocessing of the original data of the network interface, which may specifically include the collection, aggregation, and distribution of original signaling. In the embodiment of the present invention, the collection layer can be used to obtain raw data of other network interfaces such as C/D interface, CS (CircuitSwitched, circuit switched) domain Mc interface, SCP interface, etc. as shown in FIG. 1 . It should be noted that the acquisition layer can also be used to implement PS (PacketSwitched, packet switching) domain Gn, Gb/Iu-PS, S1_MME (MobilityManagementEntity, mobile management entity), S1_U, S11, S6a, SGs, S5/S8, firewall Access to the original data of the network interface, as well as collect and complete the analysis of the data. Wherein, the Mc interface includes the A/Iu-CS interface. Afterwards, the sharing layer can obtain the data parsed by the acquisition layer through the target interface. In the embodiment of the present invention, the acquisition layer can also realize the tracking of the original signaling of a single user or multiple users at the same time, and can transmit corresponding tracking files to the application layer.

The embodiment of the present invention provides a method for data collection, as shown in Figure 2, the method includes:

101. The sharing layer respectively acquires the byte requirement of each application program in the application layer.

Among them, the byte requirement is used to represent the relevant information of the data byte required by the application program.

In the embodiment of the present invention, each application program in the application layer, that is, various service platforms and business systems may have different byte requirements. For example: application 1 needs to obtain data from the C/D interface and SCP interface, and application 2 only needs to obtain data from the C/D interface.

102. The sharing layer obtains the target data through a unified interface according to the byte requirement of each application program in the application layer.

When application 1 in step 101 needs the first 3 bytes of the data from the C/D interface, application 2 needs the first 5 bytes of the data from the C/D interface, and the data from the C/D interface A total of 20 bytes are included. Therefore, in order to ensure the execution of application 1 and application 2, the first 5 bytes of data can be obtained directly from the 20 bytes of data from the C/D interface, so that application 1 and application 2 can use it .

103. The sharing layer sends the target data corresponding to each application program to each application program in the application layer.

It should be noted that the file type and file format of the target data corresponding to each application program are the same. In the embodiment of the present invention, after step 102 is executed, the 3-byte data processed by the shared layer may be sent to the application program 1, and the 5-byte data processed by the shared layer may be sent to the application program 2 at the same time.

In the data collection method provided by the embodiment of the present invention, the shared layer obtains the byte requirements of each application program in the application layer respectively, and obtains the target data through a unified interface according to the byte requirements of each application program, and then sends each Each application sends the target data corresponding to each application. Among them, the byte requirement is used to represent the relevant information of the data byte required by the application program. Compared with obtaining the data required by each application program from each network interface for different application programs in the prior art, the embodiment of the present invention can uniformly obtain target data according to the byte requirements of each application program. Therefore, when multiple applications need to obtain the data of the same network interface, because the target data is obtained through the unified interface, it avoids repeated collection of data when different applications call the same data, thereby solving the problem of data collection. , the problem of wasting interface resources caused by repeatedly obtaining data from the same network interface.

In an implementation of the embodiment of the present invention, on the basis of the implementation shown in Figure 2, it can also be implemented as the implementation shown in Figure 3, and the relevant information includes: the information of the target network interface of the data byte and the content of the target data, the shared layer in step 102 obtains the target data through a unified interface according to the byte requirements of each application program in the application layer, which can be specifically implemented as steps 1021 to 1023:

1021. The sharing layer determines a target network interface among all network interfaces according to the information of the target network interface.

1022. The collection layer obtains the original data from the target network interface determined by the sharing layer.

1023. The sharing layer acquires the target data in the original data through the target interface according to the content of the target data.

It should be noted that the target interface may be an XDR (External Data Representation, external data representation) shared interface, and the communication protocol adopted by the XDR shared interface is SDTP (Serial Data Transport Protocol, serial data transmission protocol). Specifically, the XDR sharing interface may be an IF1 interface, wherein the IF1 interface may support the distribution of real-time and non-real-time data, and may meet the transmission requirements of a large amount of data. In addition, the IF1 interface can provide data sharing functions such as data subscription, publishing, and delivery. In the embodiment of the present invention, the data content transmitted through the IF1 interface may include: data transmitted using SDTP or file transfer protocol, which may specifically be used to transmit single-interface XDR, original signaling or original service header corresponding to single-interface XDR Data, the original data corresponding to the specified user, device or specified XDRID. Data transmission through the IF1 interface can realize the unification of the transmitted file type and file format, so as to meet the collection requirements of various service platforms and business systems in the application layer for the corresponding target data.

In the method of data collection provided by the embodiment of the present invention, the sharing layer obtains the information of the target network interface and the content of the target data of the data bytes required by each application program of the application layer respectively, and determines all the data according to the information of the target network interface. The target network interface in the network interface, and obtain the original data from the target network interface, and then according to the content of the target data, obtain the target data in the original data from the target interface, and send the target data corresponding to each application to each application . Compared with obtaining the data required by each application program from each network interface for different applications in the prior art, the embodiment of the present invention can use the information of the target network interface and the data bytes required by each application program The content of the target data, and obtain the target data uniformly. Therefore, when multiple applications need to obtain data from the same network interface, since the target data is obtained through a unified interface, repeated collection of data when different applications call the same data is avoided. Moreover, by first determining the target network interface in all network interfaces, and then obtaining the target data in the original data obtained from the target network interface, it is possible to solve the problem caused by repeated acquisition of data from the same network interface during the data collection process. While solving the problem of waste of interface resources, improve the accuracy of obtaining target data.

In an implementation of the embodiment of the present invention, on the basis of the implementation shown in FIG. 2, it can also be implemented as the implementation shown in FIG. For the byte requirements of the program, after obtaining the target data through the unified interface, step 104 and step 105 can also be performed:

104. The sharing layer invokes the unified interface and stores the target data.

105. The sharing layer generates target data corresponding to each application program according to the target protocol and target data.

In the data acquisition method provided by the embodiment of the present invention, after the shared layer obtains the target data according to the byte requirements of each application program in the application layer, the shared layer can first call the unified interface to store the target data, and then according to the target Protocol and target data to generate target data corresponding to each application, and send target data corresponding to each application to each application. Among them, the byte requirement is used to represent the relevant information of the data byte required by the application program. Compared with obtaining the data required by each application program from each network interface for different application programs in the prior art, the embodiment of the present invention can uniformly obtain and store target data according to the byte requirements of each application program. Therefore, when multiple applications need to obtain the data of the same network interface, because the target data is obtained through the unified interface, it avoids repeated collection of data when different applications call the same data, thereby solving the problem of data collection. , the problem of wasting interface resources caused by repeatedly obtaining data from the same network interface.

An embodiment of the present invention provides a collection system 20, as shown in Figure 5, the collection system 20 is provided with an application layer 21 and a shared layer 22, and the shared layer 22 is used to obtain the byte requirements of each application program in the application layer 21 respectively , byte requirements are used to represent information about the data bytes required by the application.

The sharing layer 22 is also used to obtain target data through a unified interface according to the byte requirements of each application program in the application layer 21 .

The shared layer 22 is also configured to send target data corresponding to each application to each application of the application layer 21 .

In an implementation of the embodiment of the present invention, as shown in Figure 5, the collection system 20 is provided with a collection layer 23, and the relevant information includes: the information of the target network interface of the data bytes and the content of the target data, the sharing layer 22, specifically It is used to determine the target network interface among all network interfaces according to the information of the target network interface.

The collection layer 23 is configured to obtain raw data from the target network interface determined by the sharing layer 22 .

The sharing layer 22 is specifically used to obtain the target data in the original data through the target interface according to the content of the target data.

In an implementation of the embodiment of the present invention, after the sharing layer 22 obtains the target data through the unified interface according to the byte requirements of each application program in the application layer, the sharing layer 22 is also used to call the unified interface and store the target data. data.

The shared layer 22 is also used to generate target data corresponding to each application program according to the target protocol and target data.

It should be noted that the target interface is an external data representation XDR shared interface, and the communication protocol adopted by the XDR shared interface is the serial data transfer protocol SDTP; the file type and file format of the target data corresponding to each application are the same.

In an acquisition system provided by an embodiment of the present invention, the sharing layer separately obtains the byte requirements of each application program in the application layer, and obtains the target data through a unified interface according to the byte requirements of each application program, and then sends each application The program sends object data corresponding to each application. Among them, the byte requirement is used to represent the relevant information of the data byte required by the application program. Compared with obtaining the data required by each application program from each network interface for different application programs in the prior art, the embodiment of the present invention can uniformly obtain target data according to the byte requirements of each application program. Therefore, when multiple applications need to obtain the data of the same network interface, because the target data is obtained through the unified interface, it avoids repeated collection of data when different applications call the same data, thereby solving the problem of data collection. , the problem of wasting interface resources caused by repeatedly obtaining data from the same network interface.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to part of the description of the method embodiment.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM) and the like.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. A method for data collection, characterized in that, the method is used in a collection system, and the collection system is provided with an application layer and a shared layer, and the method comprises: The sharing layer respectively obtains the byte requirement of each application program of the application layer, and the byte requirement is used to represent the relevant information of the data byte required by the application program; The sharing layer obtains the target data through a unified interface according to the byte requirement of each application program of the application layer; The sharing layer sends the target data corresponding to each application program to the each application program of the application layer. 2. method according to claim 1, is characterized in that, described acquisition system is provided with acquisition layer, and described related information comprises: the information of the target network interface of described data byte and the content of described target data, so The sharing layer obtains the target data through a unified interface according to the byte requirements of each application program in the application layer, including: The sharing layer determines the target network interface among all network interfaces according to the information of the target network interface; The collection layer obtains raw data from the target network interface determined by the sharing layer; The sharing layer obtains the target data in the original data through the target interface according to the content of the target data. 3. The method according to claim 1, characterized in that, after the shared layer obtains the target data through a unified interface according to the byte requirement of each application program of the application layer, the method further comprises : The sharing layer invokes the unified interface and stores the target data; The sharing layer generates the target data corresponding to each application program according to the target protocol and the target data. 4. The method according to claim 2, wherein the target interface is an external data representation XDR sharing interface, and the communication protocol adopted by the XDR sharing interface is a serial data transfer protocol SDTP. 5. The method according to any one of claims 1 to 3, characterized in that the file type and file format of the target data corresponding to each application program are the same. 6. A collection system, characterized in that, the collection system is provided with an application layer and a shared layer, and the shared layer is used to respectively obtain the byte requirements of each application program of the application layer, and the byte Requirements are used to represent information about the data bytes required by the application; The sharing layer is further configured to obtain target data through a unified interface according to the byte requirement of each application program of the application layer; The sharing layer is further configured to send the target data corresponding to each application program to the each application program in the application layer. 7. acquisition system according to claim 6, is characterized in that, described acquisition system is provided with acquisition layer, and described relevant information comprises: the information of the target network interface of described data byte and the content of described target data, The sharing layer is specifically configured to determine the target network interface among all network interfaces according to the information of the target network interface; The collection layer is configured to obtain raw data from the target network interface determined by the sharing layer; The sharing layer is specifically configured to obtain the target data in the original data through the target interface according to the content of the target data. 8. The acquisition system according to claim 6, characterized in that, after the shared layer obtains the target data through a unified interface according to the byte requirement of each application program of the application layer, the shared layer , which is also used to call the unified interface and store the target data; The sharing layer is further configured to generate target data corresponding to each application program according to the target protocol and the target data. 9. The acquisition system according to claim 7, wherein the target interface is an external data representation XDR sharing interface, and the communication protocol adopted by the XDR sharing interface is a serial data transfer protocol SDTP. 10. The collection system according to any one of claims 6 to 8, characterized in that the file types and file formats of the target data corresponding to each application program are the same.