CN117608818A - Multithreading data processing method, device, equipment and medium - Google Patents
Multithreading data processing method, device, equipment and medium Download PDFInfo
- Publication number
- CN117608818A CN117608818A CN202311368059.1A CN202311368059A CN117608818A CN 117608818 A CN117608818 A CN 117608818A CN 202311368059 A CN202311368059 A CN 202311368059A CN 117608818 A CN117608818 A CN 117608818A
- Authority
- CN
- China
- Prior art keywords
- thread
- application program
- value
- network interface
- protocol stack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title description 14
- 238000000034 method Methods 0.000 claims abstract description 71
- 238000012545 processing Methods 0.000 claims abstract description 42
- 230000008569 process Effects 0.000 claims abstract description 36
- 230000007717 exclusion Effects 0.000 claims abstract description 16
- 230000000903 blocking effect Effects 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 16
- 230000004044 response Effects 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 9
- 230000006854 communication Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000003292 glue Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000036316 preload Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The disclosure relates to a method, a device, equipment and a medium for processing multithread data, wherein the method comprises the steps of controlling a user state protocol stack thread to obtain a first value of a preset key, and controlling an application program thread to obtain at least one second value of the preset key; the application program thread calls the standard network interface to create a socket according to the second value, binds and monitors the IO event of the socket, embeds the service logic under the programming framework of the user state protocol stack through the standard network interface, and the application program loads the user state protocol stack without modifying the service code; invoking a conditional mutex lock corresponding to the data packet to wake up an application program thread, so that the multithreading is orderly and efficiently scheduled, and the concurrency safety of the multithreading is protected; the user state protocol stack thread calls the network interface to receive the data packet according to the first value, and calls the conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread, so that the application program thread traverses and processes IO events, and the efficiency of multi-thread data processing is improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for processing multithreaded data.
Background
The user-mode protocol stack generally adopts the idea of 'glue adhesion', so that an interface in an operating system in the network protocol stack is 'stuck' with an application program, the number of system call times in the process of copying network data from a network card to the application program is reduced, and the data processing performance is improved.
In the prior art, the idea of a traditional user state protocol stack is adopted to process network data, only the receiving and transmitting of a message processed by a single thread in an RTC mode are supported, when the user state protocol stack is loaded, application program service codes are required to be modified in a large amount, and service logic can be embedded under a programming framework of the user state protocol stack so as to adapt to the programming model of the user state protocol stack.
Disclosure of Invention
In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, an apparatus, a device, and a medium for processing multithreaded data, so as to improve the efficiency of the multithreaded data processing.
In a first aspect, an embodiment of the present disclosure provides a method for processing multithreaded data, including:
controlling a user state protocol stack thread to acquire a first value of a preset key, and controlling an application program thread to acquire at least one second value of the preset key;
the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors IO events of the socket;
a preset function is rewritten, and a conditional mutual exclusion lock blocking IO operation is called in response to the application program thread blocking waiting;
and the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event.
In a second aspect, embodiments of the present disclosure provide a multi-threaded data processing apparatus comprising:
the control module is used for controlling the user state protocol stack thread to acquire a first value of a preset key and controlling the application program thread to acquire at least one second value of the preset key;
the monitoring module is used for the application program thread to call a standard network interface to create a socket according to the second value, bind and monitor the IO event of the socket;
the calling module is used for rewriting a preset function, responding to the application program thread blocking waiting, and calling a conditional mutual exclusion lock blocking IO operation;
and the processing module is used for calling a network interface to receive a data packet by the user state protocol stack thread according to the first value, calling a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread, so that the application program thread traverses and processes the IO event.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium having stored thereon a computer program for execution by a processor to implement the method of the first aspect.
In a fifth aspect, embodiments of the present disclosure also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement the method of the first aspect.
According to the multithreading data processing method, device, equipment and medium provided by the embodiment of the disclosure, the user state protocol stack thread is controlled to obtain the first value of the preset key, the application program thread is controlled to obtain at least one second value of the preset key, and the thread safety is protected through one-key multi-value; the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors the IO event of the socket, embeds service logic into the programming framework of the user mode protocol stack through the standard network interface, and the application program loads the user mode protocol stack without modifying service codes; invoking a conditional mutex lock corresponding to the data packet to wake up an application program thread, so that the multithreading is orderly and efficiently scheduled, and the concurrency safety of the multithreading is protected; and the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event, thereby realizing blocking type packet receiving logic and improving the efficiency of multithreaded data processing.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flow chart of a method for processing multithreaded data provided by an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a standard network interface package provided by an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a multi-threaded data processing apparatus according to an embodiment of the disclosure;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.
The user-mode protocol stack generally adopts the idea of 'glue adhesion', so that an interface in an operating system in the network protocol stack is 'stuck' with an application program, the number of system call times in the process of copying network data from a network card to the application program is reduced, and the data processing performance is improved.
In the prior art, the idea of a traditional user state protocol stack is adopted to process network data, only receive and dispatch of a message processed by a single thread in an RTC mode is supported, when the user state protocol stack is loaded, an application program service code needs to be modified in a large amount, and the service logic can be embedded under a programming framework of the user state protocol stack so as to adapt to the programming model of the user state protocol stack.
The defects of the prior art are as follows: under the single-thread programming framework of the user mode protocol stack, an application program data processing and data plane development suite (Data Plane Development Kit, dpdk) receives and transmits packets and is 'glued' in the context of one thread, and no thread lock and proper thread scheduling and awakening mechanism exist, so that the receiving and transmitting of the data packets and the data processing can be continuously pushed only by means of single-thread polling. The existing programming model cannot directly interface with a standard socket network interface, cannot realize the IO blocking operation of a select interface, and cannot use multithreaded application program service. In response to this problem, embodiments of the present disclosure provide a method for multithreaded data processing, which is described below in connection with particular embodiments.
Fig. 1 is a flowchart of a multithreaded data processing method according to an embodiment of the present disclosure. The method may be performed by a multi-threaded data processing apparatus, which may be implemented in software and/or hardware, and the multi-threaded data processing apparatus may be configured in an electronic device, such as a server or a terminal, where the terminal specifically includes a mobile phone, a computer, a tablet computer, or the like. In addition, the method can be applied to application scenes of multithreaded data processing, and it can be understood that the multithreaded data processing method provided by the embodiment of the disclosure can also be applied to other scenes.
The following describes the multi-threaded data processing method shown in fig. 1, which includes the following specific steps:
s101, controlling a user state protocol stack thread to acquire a first value of a preset key, and controlling an application program thread to acquire at least one second value of the preset key.
The user state protocol stack thread is controlled to obtain a first value of the preset key, the user state protocol stack thread can be expressed by thread0, the application program thread is controlled to obtain at least one second value of the preset key, the number of the application program threads is at least one, the at least one second value of the preset key is correspondingly obtained, and for convenience of description, the description will be carried out with one application program thread later, and the application program thread can be thread1.
Optionally, controlling the application thread to obtain at least one second value of the preset key includes: and the control application program thread calls a network interface function of the standard network interface and acquires at least one second value of the preset key through the global variable.
Specifically, the application thread (thread 1) invokes the network interface function vfus_pthread_create () of the standard network interface, sets proc0+thred1 to the second value of the preset key, and obtains at least one second value of the preset key in the application thread (thread 1) by the global variable curthread.
S102, the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors IO events of the socket.
Socket (Socket) is an abstraction of an endpoint that communicates bi-directionally between application processes on different hosts in a network. One socket is the end of the network where processes communicate, providing a mechanism for application layer processes to exchange data using network protocols. In terms of the position, the socket is connected with the application process in an upper mode, and the socket is connected with the network protocol stack in a lower mode, so that the socket is an interface for the application program to communicate through the network protocol, and is an interface for the application program to interact with the network protocol stack.
And the application program thread (thread 1) calls a standard network interface to create a socket according to a second value of the preset key, binds and monitors IO events of the socket.
Illustratively, the application thread (thread 1) invokes a standard network interface according to the second value of the preset key, where the standard network interface may be socket (), connect (), bind (), create a socket of the TCP server, obtain socket fd, bind socket fd, monitor an IO event related to the socket by using epoll technology, and block the application thread (thread 1) from waiting for a connect connection operation.
S103, rewriting a preset function, and calling a conditional exclusive lock to block IO operation in response to the application program thread blocking waiting.
Mutual exclusion lock: in a multitasking operation, multiple tasks running simultaneously may need to access the same resource, and there is such a lock in the thread that access to the shared resource is restricted. The mutual exclusive lock has two states, locking and unlocking. Atomic and unique emphasis is placed on the atomicity.
Conditional lock: conditional locks emphasize that conditional waiting, conditional locks block the current thread and do not continue to execute down until a certain condition is met.
The conditional mutex lock is a combination of the mutex lock and the conditional lock, and a plurality of tasks running simultaneously may need to access the same resource, so that a lock exists in a thread to limit the access to the shared resource, and the execution continues downwards when a certain condition is met.
The preset function, which may be an epoll_wait () function, is rewritten, and conditional mutex lock blocking IO operations are invoked in response to blocking wait of the application thread (thread 1).
Illustratively, the epoll_wait () function is rewritten, thereby rewriting the implementation of the epoll_wait () function such that the application thread (thread 1) invokes the conditional exclusive lock blocking IO operation while blocking the wait.
S104, the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event.
The user state protocol stack thread (thread 0) calls the network interface to receive the data packet according to the first value, and calls the conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread (thread 1) so that the application program thread (thread 1) traverses to process the IO event.
According to the embodiment of the disclosure, the user state protocol stack thread is controlled to obtain the first value of the preset key, the application program thread is controlled to obtain at least one second value of the preset key, and the thread safety is protected through one-key multi-value; the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors the IO event of the socket, embeds service logic into the programming framework of the user mode protocol stack through the standard network interface, and the application program loads the user mode protocol stack without modifying service codes; invoking a conditional mutex lock corresponding to the data packet to wake up an application program thread, so that the multithreading is orderly and efficiently scheduled, and the concurrency safety of the multithreading is protected; and the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event, thereby realizing blocking type packet receiving logic and improving the efficiency of multithreaded data processing.
In some embodiments, before controlling the user mode protocol stack thread to obtain the first value of the preset key and controlling the application program thread to obtain the at least one second value of the preset key, the method further includes: initializing a user state protocol stack, and creating a preset key according to the characteristic pthread_key_create of thread local storage, wherein access threads and values of the preset key are in one-to-one correspondence, and the access threads at least comprise user state protocol stack threads and application program threads.
Initializing a user mode protocol stack, and creating a preset key according to the characteristic pthread_key_create of the thread local storage (Thread Local Storage) in the initialization process of the vfus SDK user mode protocol stack, wherein the data pointed by the preset key is called thread private data. The access threads and the values of the preset key are in one-to-one correspondence, namely, the values (namely, thread private data) obtained by different threads accessing the preset key are different, and the access threads at least comprise the user mode protocol stack threads and the application program threads. The thread local store (Thread Local Storage) feature uses a one-key multi-value (i.e., a preset key corresponds to multiple values) technique, that is, when different threads access the same variable, they are essentially accessing different data.
Illustratively, when the user state protocol stack is initialized, the user state protocol stack thread is pulled up, the user state protocol stack thread0 is initialized, the preset key is set, the file system interface data (proc 0) +the user state protocol stack thread data (thread 0) is set as a parameter to be a first value of the preset key, and the global variable curthread takes the first value of the preset key in the user state protocol stack thread (thread 0). When the application thread (thread 1) is created, a second value of the preset key is set in a network interface function vfus_pthread_create () of the standard network interface created by the application thread, and the second value of the preset key is fetched by the global variable curthread in the application thread (thread 1).
According to the embodiment of the disclosure, the preset key is created through the thread local storage characteristic pthread_key_create, so that the influence of the curthread global variable on multithreading is solved, the cost is reduced, and the safety of multithreading data processing is improved.
In some embodiments, before the application thread invokes a standard network interface to create a socket according to the second value, the method further comprises: and encapsulating the network interface according to a preset standard to obtain the standard network interface.
And encapsulating the network interface according to a preset standard to obtain the standard network interface. As shown in fig. 2, the user state protocol stack uses a network interface on the left side, such as vfus_socket (), vfus_connect (), vfus_bind (), vfus_accept (), vfus_recv (); the application uses standard network interfaces on the right, such as socket (), connect (), bind (), accept (), recv (). And packaging and adapting the network interface on the left side according to a preset standard to obtain a standard network interface on the right side, wherein the standard network interface on the right side supports a model of an interface (Application Programming Interface, api) between standard programs.
According to the embodiment of the disclosure, the network interface is packaged according to the preset standard, so that the standard network interface is obtained, seamless switching is supported by a preload mode, dynamic loading of the software development tool package (Software Development Kit, sdk) is realized, and dynamic plug and discharge can be performed without modifying service codes when an application program loads a user state protocol stack.
In some embodiments, the user mode protocol stack thread calls a network interface to receive a data packet according to the first value, calls a conditional mutex lock corresponding to the data packet to wake up the application thread, so that the application thread traverses to process the IO event, and includes: the user state protocol stack thread calls a network interface to receive a connect data packet according to the first value, checks the data packet, stores the data packet in a preset position, and calls a conditional exclusive lock corresponding to the data packet to wake up the application program thread; when the application thread is awakened, the application thread traverses to process the IO event.
A user state protocol stack thread (thread 0) calls a network interface according to a first value to receive a connect data packet, verifies the data packet, stores the data packet in a preset position, wherein the preset position can be the buf of a socket, and calls a conditional mutex lock wake-up application program thread (thread 1) corresponding to the data packet; when the application thread (thread 1) wakes up, the application thread (thread 1) traverses to handle IO events.
When the client sends a connect data packet, the user state protocol stack thread (thread 0) detects the connect data packet, calls the network interface according to the first value to receive the connect data packet, verifies the data packet, stores the data packet in the buf of the socket, and calls the application program thread (thread 1) waiting for wakeup of the conditional exclusive lock corresponding to the data packet. When the application thread (thread 1) wakes up, the application thread (thread 1) traverses to handle IO events. The user state protocol stack thread (thread 0) continues to probe for packets.
Optionally, when the application thread is woken up, after the application thread traverses to process the IO event, the method further includes: and adding the target connection data to a monitoring queue based on the connection data packet as a target connection data packet.
Specifically, when the application thread (thread 1) determines that the connect data packet is a new connection data packet, determining that the connect data packet is a target connection data packet, and adding target connection data corresponding to the connect data packet to the listening queue. The application thread (thread 1) continues to wait to receive the IO event corresponding to the socket.
Optionally, the method further comprises: the user state protocol stack thread calls a network interface to receive data according to the first value, stores the data in a preset position, and calls a conditional mutex lock corresponding to the data to wake up the application program thread; when the application program thread is awakened, the application program thread traverses and processes the IO event, reads the data according to the type of the IO event and processes the data.
Specifically, the client continues to send data, and the user state protocol stack thread (thread 0) calls the network interface according to the first value to receive the data and stores the data in a preset position, wherein the preset position can be the buf of a socket, and calls a conditional mutex lock wake-up application thread (thread 1) corresponding to the data; when the application program thread (thread 1) is awakened, the application program thread (thread 1) traverses to process the IO event, reads data of a preset position according to the type of the IO event, processes the data and calls a standard network interface to send a response data packet to the client. The application thread (thread 1) continues to wait to receive the IO event corresponding to the socket.
According to the embodiment of the disclosure, the communication process of the client and the server is specifically described, so that the flow of the multithreading data processing method is clarified, and the multithreading data processing efficiency is improved.
Fig. 3 is a schematic structural diagram of a multithreaded data processing apparatus according to an embodiment of the disclosure. The multi-threaded data processing apparatus may be a terminal as described in the above embodiments or the multi-threaded data processing apparatus may be a part or component in the terminal. The multithreaded data processing apparatus provided in the embodiments of the present disclosure may execute the processing flow provided in the embodiment of the multithreaded data processing method, as shown in fig. 3, the multithreaded data processing apparatus 30 includes: the system comprises a control module 31, a monitoring module 32, a calling module 33 and a processing module 34; the control module 31 is configured to control a user state protocol stack thread to obtain a first value of a preset key, and control an application program thread to obtain at least one second value of the preset key; a monitoring module 32, configured to invoke a standard network interface to create a socket according to the second value by the application thread, bind and monitor an IO event of the socket; a calling module 33, configured to rewrite a preset function, respond to the application program thread blocking waiting, and call a conditional mutex blocking IO operation; and the processing module 34 is configured to invoke, by the user state protocol stack thread, a network interface to receive a data packet according to the first value, and invoke a conditional mutex lock corresponding to the data packet to wake up the application program thread, so that the application program thread traverses and processes the IO event.
Optionally, the multi-threaded data processing apparatus 30 further comprises: the initialization module 35 is configured to initialize a user state protocol stack, and create a preset key according to a characteristic pthread_key_create stored locally by a thread, where an access thread and a value of the preset key are in one-to-one correspondence, and the access thread at least includes the user state protocol stack thread and the application program thread.
Optionally, the control module 31 is further configured to control the application program thread to call a network interface function of the standard network interface, and obtain at least one second value of the preset key through a global variable.
Optionally, the multi-threaded data processing apparatus 30 further comprises: the encapsulation module 36 is configured to encapsulate the network interface according to a preset standard, so as to obtain a standard network interface.
Optionally, the processing module 34 is further configured to invoke, by the user mode protocol stack thread according to the first value, a network interface to receive a connect data packet, verify the data packet, store the data packet in a preset position, and invoke a conditional mutex lock corresponding to the data packet to wake up the application thread; when the application thread is awakened, the application thread traverses to process the IO event.
Optionally, the multi-threaded data processing apparatus 30 further comprises: the adding module 37 is configured to add the target connection data to the listening queue based on the connect data packet as a target connection data packet.
Optionally, the processing module 34 is further configured to invoke, by the user mode protocol stack thread, a network interface to receive data according to the first value, store the data in a preset position, and invoke a conditional mutex lock corresponding to the data to wake up the application thread; when the application program thread is awakened, the application program thread traverses and processes the IO event, reads the data according to the type of the IO event and processes the data.
The multithreaded data processing apparatus of the embodiment shown in fig. 3 may be used to implement the technical solution of the embodiment of the multithreaded data processing method, and its implementation principle and technical effects are similar, and are not repeated here.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure. The electronic device may be a terminal as described in the above embodiments. The electronic device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the multithreaded data processing method, as shown in fig. 4, the electronic device 40 includes: memory 41, processor 42, computer programs and communication interface 43; wherein the computer program is stored in the memory 41 and configured to be executed by the processor 42 for the multi-threaded data processing method as described above.
In addition, the embodiment of the present disclosure also provides a computer readable storage medium having stored thereon a computer program that is executed by a processor to implement the multi-threaded data processing method described in the above embodiment.
Furthermore, embodiments of the present disclosure provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a multi-threaded data processing method as described above.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
controlling a user state protocol stack thread to acquire a first value of a preset key, and controlling an application program thread to acquire at least one second value of the preset key;
the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors IO events of the socket;
a preset function is rewritten, and a conditional mutual exclusion lock blocking IO operation is called in response to the application program thread blocking waiting;
and the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event.
In addition, the electronic device may also perform other steps in the multi-threaded data processing method as described above.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of multithreaded data processing, the method comprising:
controlling a user state protocol stack thread to acquire a first value of a preset key, and controlling an application program thread to acquire at least one second value of the preset key;
the application program thread calls a standard network interface to create a socket according to the second value, binds and monitors IO events of the socket;
a preset function is rewritten, and a conditional mutual exclusion lock blocking IO operation is called in response to the application program thread blocking waiting;
and the user state protocol stack thread calls a network interface to receive a data packet according to the first value, and calls a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread so that the application program thread traverses and processes the IO event.
2. The method of claim 1, wherein prior to controlling the user mode protocol stack thread to obtain the first value of the preset key and controlling the application thread to obtain the at least one second value of the preset key, the method further comprises:
initializing a user state protocol stack, and creating a preset key according to the characteristic pthread_key_create of thread local storage, wherein access threads and values of the preset key are in one-to-one correspondence, and the access threads at least comprise user state protocol stack threads and application program threads.
3. The method of claim 1, wherein controlling an application thread to obtain at least one second value of the preset key comprises:
and the control application program thread calls a network interface function of the standard network interface and acquires at least one second value of the preset key through the global variable.
4. The method of claim 1, wherein before the application thread invokes a standard network interface to create a socket according to the second value, the method further comprises:
and encapsulating the network interface according to a preset standard to obtain the standard network interface.
5. The method of claim 1, wherein the user mode protocol stack thread invoking a network interface to receive a data packet according to the first value, invoking a conditional mutex lock corresponding to the data packet to wake the application thread to cause the application thread to traverse processing the IO event, comprising:
the user state protocol stack thread calls a network interface to receive a connect data packet according to the first value, checks the data packet, stores the data packet in a preset position, and calls a conditional exclusive lock corresponding to the data packet to wake up the application program thread;
when the application thread is awakened, the application thread traverses to process the IO event.
6. The method of claim 5, wherein when the application thread wakes up, the method further comprises, after the application thread traverses processing the IO event:
and adding the target connection data to a monitoring queue based on the connection data packet as a target connection data packet.
7. The method of claim 6, wherein the method further comprises:
the user state protocol stack thread calls a network interface to receive data according to the first value, stores the data in a preset position, and calls a conditional mutex lock corresponding to the data to wake up the application program thread;
when the application program thread is awakened, the application program thread traverses and processes the IO event, reads the data according to the type of the IO event and processes the data.
8. A multi-threaded data processing apparatus, the apparatus comprising:
the control module is used for controlling the user state protocol stack thread to acquire a first value of a preset key and controlling the application program thread to acquire at least one second value of the preset key;
the monitoring module is used for the application program thread to call a standard network interface to create a socket according to the second value, bind and monitor the IO event of the socket;
the calling module is used for rewriting a preset function, responding to the application program thread blocking waiting, and calling a conditional mutual exclusion lock blocking IO operation;
and the processing module is used for calling a network interface to receive a data packet by the user state protocol stack thread according to the first value, calling a conditional mutual exclusion lock corresponding to the data packet to wake up the application program thread, so that the application program thread traverses and processes the IO event.
9. An electronic device, comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-7.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311368059.1A CN117608818B (en) | 2023-10-20 | 2023-10-20 | Multithreading data processing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311368059.1A CN117608818B (en) | 2023-10-20 | 2023-10-20 | Multithreading data processing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117608818A true CN117608818A (en) | 2024-02-27 |
| CN117608818B CN117608818B (en) | 2024-09-27 |
Family
ID=89952293
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311368059.1A Active CN117608818B (en) | 2023-10-20 | 2023-10-20 | Multithreading data processing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117608818B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119025284A (en) * | 2024-10-24 | 2024-11-26 | 深圳南瑞科技有限公司 | A method and system for sequentially processing multi-threaded data in frequency regulation control |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958850A (en) * | 2018-06-29 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of log optimization method, device, equipment and computer readable storage medium |
| CN109617833A (en) * | 2018-12-25 | 2019-04-12 | 深圳市任子行科技开发有限公司 | The NAT Data Audit method and system of multithreading user mode network protocol stack system |
| CN111314311A (en) * | 2020-01-19 | 2020-06-19 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for improving performance of switch |
| CN112905357A (en) * | 2021-01-27 | 2021-06-04 | 清华大学 | Data reading method and system, electronic equipment and storage medium |
| CN114697194A (en) * | 2022-02-17 | 2022-07-01 | 清华大学 | Blocking type event notification method and device |
| WO2023284473A1 (en) * | 2021-07-14 | 2023-01-19 | 腾讯科技（深圳）有限公司 | Data management method and apparatus, computer device, and storage medium |
| WO2023030178A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Communication method based on user-mode protocol stack, and corresponding apparatus |
| CN116244004A (en) * | 2023-02-21 | 2023-06-09 | 北京火山引擎科技有限公司 | Resource management method, device, medium and electronic equipment |
| CN116383175A (en) * | 2023-03-21 | 2023-07-04 | 康键信息技术（深圳）有限公司 | Data loading method, device, equipment and computer readable medium |
| CN116708597A (en) * | 2023-08-04 | 2023-09-05 | 新华三技术有限公司 | Data processing method and device |
-
2023
- 2023-10-20 CN CN202311368059.1A patent/CN117608818B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958850A (en) * | 2018-06-29 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of log optimization method, device, equipment and computer readable storage medium |
| CN109617833A (en) * | 2018-12-25 | 2019-04-12 | 深圳市任子行科技开发有限公司 | The NAT Data Audit method and system of multithreading user mode network protocol stack system |
| CN111314311A (en) * | 2020-01-19 | 2020-06-19 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for improving performance of switch |
| CN112905357A (en) * | 2021-01-27 | 2021-06-04 | 清华大学 | Data reading method and system, electronic equipment and storage medium |
| WO2023284473A1 (en) * | 2021-07-14 | 2023-01-19 | 腾讯科技（深圳）有限公司 | Data management method and apparatus, computer device, and storage medium |
| WO2023030178A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Communication method based on user-mode protocol stack, and corresponding apparatus |
| CN114697194A (en) * | 2022-02-17 | 2022-07-01 | 清华大学 | Blocking type event notification method and device |
| CN116244004A (en) * | 2023-02-21 | 2023-06-09 | 北京火山引擎科技有限公司 | Resource management method, device, medium and electronic equipment |
| CN116383175A (en) * | 2023-03-21 | 2023-07-04 | 康键信息技术（深圳）有限公司 | Data loading method, device, equipment and computer readable medium |
| CN116708597A (en) * | 2023-08-04 | 2023-09-05 | 新华三技术有限公司 | Data processing method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119025284A (en) * | 2024-10-24 | 2024-11-26 | 深圳南瑞科技有限公司 | A method and system for sequentially processing multi-threaded data in frequency regulation control |
| CN119025284B (en) * | 2024-10-24 | 2025-03-07 | 深圳南瑞科技有限公司 | Method and system for sequentially processing multi-thread data in frequency adjustment control |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117608818B (en) | 2024-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106161537B (en) | Method, device and system for processing remote procedure call and electronic equipment | |
| US7080386B2 (en) | Architecture with digital signal processor plug-ins for general purpose processor media frameworks | |
| US7788668B2 (en) | System and method for implementing distributed priority inheritance | |
| CN100530107C (en) | Single process contents server device and method based on IO event notification mechanism | |
| US8483095B2 (en) | Configurable network socket retransmission timeout parameters | |
| US9553944B2 (en) | Application server platform for telecom-based applications using an actor container | |
| JP5516398B2 (en) | Multiprocessor system and method for sharing device between OS of multiprocessor system | |
| US9231995B2 (en) | System and method for providing asynchrony in web services | |
| EP1615129A2 (en) | Implementation of concurrent programs in object-oriented languages | |
| US9900275B2 (en) | Tracking object across processes | |
| CN117608818B (en) | Multithreading data processing method, device, equipment and medium | |
| CN113641410B (en) | A processing method and system for a high-performance gateway system based on Netty | |
| JP2004536382A (en) | Systems, methods, and articles of manufacture using replaceable components to select network communication channel components with replaceable quality of service features | |
| CN108073414B (en) | Implementation method for merging multithreading concurrent requests and submitting and distributing results in batches based on Jedis | |
| CN102955704A (en) | Software execution method and electronic device thereof | |
| CN113835904A (en) | Remote procedure call control method, device, equipment and storage medium | |
| CN104038390A (en) | Linux server cluster unified peripheral event monitoring method based on netlink | |
| CN102937893B (en) | A kind of software timing method of dynamic heavy duty timed length | |
| CN106445696B (en) | A kind of multi-process interactive processing method and system | |
| EP4475001A1 (en) | Network device-based data processing method and network device | |
| Yan et al. | Design and Analysis of the Network Software Stack of an Asynchronous Many-task System--The LCI parcelport of HPX | |
| US10313450B2 (en) | Method for transparently connecting augmented network socket operations | |
| US10379918B2 (en) | System and method for MPI implementation in an embedded operating system | |
| US7774779B2 (en) | Generating a timeout in a computer software application | |
| CN112749020A (en) | Microkernel optimization method of Internet of things operating system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |