CN119512783A

CN119512783A - Data transmission processing method, device, electronic device and storage medium

Info

Publication number: CN119512783A
Application number: CN202411545797.3A
Authority: CN
Inventors: 余蕾
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2024-10-31
Filing date: 2024-10-31
Publication date: 2025-02-25

Abstract

An embodiment of the present application provides a method, device, electronic device and storage medium for processing data transmission, and the method for processing data transmission includes: obtaining a memory space created by a second process through a first process; the first process is a process created by the second process; when the second process starts the first process, creating a memory pool through the first process based on the memory space; the memory pool is used to store data transmitted by the second process, and the first process and the second process communicate by sharing the address of the memory pool; executing a business through the second process based on the address of the memory pool; in the process of the second process executing the business, determining whether the second process exits normally; when the second process exits abnormally, closing the data transmission channel on the corresponding virtual function device through the first process based on the virtual function device information corresponding to the second process.

Description

Data transmission processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to, but not limited to, the field of communications technologies, and in particular, to a data transmission processing method, apparatus, electronic device, and storage medium.

Background

With the development of communication technology, after a process is started, a protocol (Internet Protocol, IP) library interconnected between networks is started, the process can establish a memory pool by itself, but after the process is abnormally exited, the IP library is not told to stop accessing the memory, the memory can be recovered by a kernel, and after the memory is recovered, an illegal address is accessed by the IP, so that the normal operation of a service is affected.

Disclosure of Invention

In view of the above, the embodiments of the present application provide a data transmission processing method, apparatus, electronic device, and storage medium.

The technical scheme of the embodiment of the application is realized as follows:

In a first aspect, an embodiment of the present application provides a processing method for data transmission, where the method includes acquiring, by a first process, a memory space created by a second process, where the first process is a process created by the second process, creating, by the second process, a memory pool based on the memory space by the first process when the second process starts the first process, where the memory pool is used to store data transmitted by the second process, where the first process and the second process communicate by sharing an address of the memory pool, executing a service by the second process based on the address of the memory pool, determining, by the second process in a process of executing the service, whether the second process exits normally, and closing, by the first process, a data transmission channel on a corresponding virtual function device based on virtual function device information corresponding to the second process when the second process exits abnormally.

In a second aspect, an embodiment of the present application provides a processing apparatus for data transmission, where the apparatus includes:

The system comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for acquiring a memory space created by a second process through a first process;

The first creating module is used for creating a memory pool based on the memory space through the first process when the second process starts the first process, wherein the memory pool is used for storing data transmitted by the second process, and the first process and the second process are communicated through sharing the address of the memory pool;

the execution module is used for executing the service based on the address of the memory pool through the second process;

The first determining module is used for determining whether the second process exits normally or not in the process of executing the service by the second process;

And the first closing module is used for closing a data transmission channel on the corresponding virtual function device through the first process based on the virtual function device information corresponding to the second process under the condition that the second process is abnormally exited.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program executable on the processor, and where the processor implements some or all of the steps of the above method when the processor executes the program.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which when executed by a processor performs some or all of the steps of the above method.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic implementation flow chart of a processing method for data transmission according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a system framework according to an embodiment of the present application;

Fig. 3 is a schematic implementation flow chart of another processing method for data transmission according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a process for implementing a kernel driver to create a VF and initialize PCIe resources in accordance with the embodiments of the present application;

fig. 5 is a schematic diagram of a composition structure of a processing device for data transmission according to an embodiment of the present application;

fig. 6 is a schematic diagram of a hardware entity of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solution of the present application will be further elaborated with reference to the accompanying drawings and examples, which should not be construed as limiting the application, but all other embodiments which can be obtained by one skilled in the art without making inventive efforts are within the scope of protection of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

It should be noted that the term "first\second\third" related to the embodiments of the present application is merely to distinguish similar objects, and does not represent a specific order for the objects, it being understood that the "first\second\third" may interchange a specific order or sequencing, where allowed, so that the embodiments of the present application described herein can be implemented in an order other than illustrated or described herein.

It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of the application belong unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The method provided by the embodiment of the application can be executed by electronic equipment, wherein the electronic equipment can be a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (such as a mobile phone, a portable music player, a personal digital assistant, a special message device and a portable game device) and other various types of terminals, and can also be implemented as a server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform.

The embodiment of the application provides a data transmission processing method, as shown in fig. 1, the data transmission processing method includes the following steps S101 to S105, where:

step S101, a memory space created by a second process is obtained through a first process, wherein the first process is a process created by the second process;

Here, a process in which a first process is created by a second process may refer to that in an operating system there is one process (referred to as a second process) that starts and creates another process (referred to as a first process), i.e., the first process is a slave process and the second process is a master process.

Here, the memory space may be created through some kind of virtualization technology (e.g., container technology, virtual machine technology, etc.) or an application programming interface (Application Programming Interface, API) provided by the operating system. The memory space may be stored in a designated shared directory, which is a known location in the file system that is accessible to both the first process and the second process.

In some embodiments, in the shared directory, the second process has created a file of the memory space, and the first process needs to open the file and read the content therein, so as to obtain the memory space created by the second process.

It should be noted that the file content may include the size of the memory space, the type of the memory space (e.g., 1 Gigabyte (GB) or 2 Megabytes (MB)), and the physical memory descriptor. The physical memory descriptor includes a segment base address, a segment limit, a segment attribute, etc., where the segment base address represents a starting physical address of a segment, the segment limit represents a length of memory occupied by a program segment or a data segment, and the segment attribute represents an access right of the segment.

Step S102, under the condition that the second process starts the first process, a memory pool is created by the first process based on the memory space;

Here, the memory pool is used to store data transmitted by the second process, and the first process and the second process communicate through the address of the shared memory pool, that is, the second process may write data into the memory pool, and the first process may read data from the memory pool, and vice versa.

In some embodiments, after the second process starts the first process, the first process uses the obtained memory space to create a memory pool, where the created memory pool is used to store data to be transferred to the first process by the second process.

Step S103, executing the service based on the address of the memory pool through the second process;

in some embodiments, the second process performs business logic based on the address of the memory pool, and the business may include writing, reading, or other operations of data.

Step S104, determining whether the second process exits normally or not in the process of executing the service by the second process;

in some embodiments, the system monitors the state of the second process, and in particular whether it is exiting normally, during execution of the service by the second process.

Step S105, in the case that the second process exits abnormally, closing, by the first process, a data transmission channel on the corresponding virtual function device based on the virtual function device information corresponding to the second process.

Here, an abnormal exit of the second process may refer to the second process not ending its execution in its preset normal flow or logic, which is generally related to the process being forced to terminate due to some kind of exception or error. The abnormal exit of the second process may include the second process crashing, being forcibly terminated, receiving an exception signal, etc.

Here, the virtual function device information corresponding to the second process generally refers to information of a virtual device associated with the second process and provided by a virtualization technology or a specific software framework. These virtual devices may be virtualized versions of hardware devices or devices that are purely emulated by software, designed to provide a functional interface in an operating system or application program that is similar to that of a real hardware device.

Specifically, the virtual function device information corresponding to the second process may include a device identifier (e.g., a bus device function (Bus Device Function, BDF) number), a device type, a device configuration, a device status, etc., where the device identifier is a unique identifier of a virtual device and is used to distinguish different virtual devices in the system, the BDF number is typically used to identify a peripheral component interconnect express bus (PERIPHERAL COMPONENT INTERCONNECT EXPRESS, PCIe) device, the device type is used to indicate a type of the virtual device, e.g., a network device, a storage device, an input/output device, etc., the device configuration may refer to configuration information of the virtual device, e.g., an IP address, a media access Control (MEDIA ACCESS Control, MAC) address, a read/write speed, etc., and the device status may refer to a current status of the virtual device, e.g., whether enabled, connected, whether erroneous, etc.

The data transmission channel on the virtual function device may refer herein to a logical connection or path established in the virtualized environment for efficient, secure transmission of data between different virtual function devices or between a virtual function device and a physical device. Such channels are built on top of the physical network infrastructure using virtualization technology to provide virtual function devices with communication functions similar to a private network.

In some embodiments, if the second process suddenly stops running due to an internal error (e.g., memory access violation, null pointer reference, etc.) or an external factor (e.g., hardware failure, power problem, etc.), i.e., the second process crashes, the first process may close the data transfer channel on the corresponding virtual function device based on the virtual function device information corresponding to the second process (e.g., information of some hardware device or software emulated device associated with the second process).

In some embodiments, if the second process is forced to end, i.e., the second process is forced to terminate, due to a request of a user or a system administrator, the first process may close a data transmission channel on the corresponding virtual function device according to the virtual function device information corresponding to the second process.

In some embodiments, if the second process receives an exception signal (e.g., an illegal command signal) that it cannot process, so that it cannot normally continue to execute, i.e., the second process receives the exception signal, the first process closes the data transmission channel on the corresponding virtual function device according to the virtual function device information corresponding to the second process.

In the embodiment of the application, on one hand, the first process and the second process communicate through the address of the shared memory pool, so that the complexity of inter-process communication is simplified, and the communication efficiency and reliability are improved. If the second process is not normally exited, the first process can close the corresponding data transmission channel according to the virtual function equipment information, so that continuity and stability of service execution are ensured.

In some embodiments, the implementation of step S102 "creating a memory pool based on the memory space through the first process" may include steps S1021 to S1022, where:

step S1021, mapping the address of the memory space to the first process through the first process, so as to obtain a mapped memory space;

In some embodiments, a first process obtains a memory space created by a second process, and then the first process maps the address of the memory space into its own address space. This means that the first process can now access this memory space through its own address space without having to go through additional system calls or network transmissions. This mapping process typically involves the involvement of the operating system kernel to ensure the correctness and security of the address mapping.

Step S1022, determining, by the second process, a part of the mapped memory space as the memory pool.

Here, the memory pool may be a block of memory area allocated and managed in advance for storing and transmitting data. The size and location of the memory pool can be configured and adjusted according to specific business requirements.

In some embodiments, after the memory space is mapped to the address space of the first process, the second process selects a portion of the mapped memory space as the memory pool. Thus, the first process and the second process can share the address of the memory pool, and efficient data transmission and inter-process communication are realized.

In the embodiment of the application, the address of the memory space is mapped to the first process, and then the second process determines part of the space as the memory pool, so that the second process can dynamically allocate and adjust the size of the memory pool according to actual requirements, and the flexibility of using the memory space is improved.

In some embodiments, the implementation of the step S105 "closing, by the first process, the data transmission channel on the corresponding virtual function device based on the virtual function device information corresponding to the second process" may include the following steps S1051 to S1054, where:

Step S1051, accessing a register file based on the virtual function device information through the first process, and determining a virtual address corresponding to the virtual function device, wherein the register file comprises a virtual address mapped by a physical address of a register of the virtual function device;

Here, the register file may be a high-speed storage unit inside the processor for storing temporary data such as operands, instruction addresses, etc. In a virtualized environment, a register file may contain information about registers of a virtual function device, e.g., physical addresses and mapped virtual addresses of registers of the virtual function device.

In some embodiments, first, a first process obtains virtual function device information, which may include a type, an identifier, configuration parameters, etc. of the virtual function device, the virtual function device information being a basis for determining a virtual address, and then, by accessing a register file, obtains detailed information about registers of the virtual function device. The register file may contain a plurality of entries, each entry corresponding to a register of a virtual function device, and finally, the first process may determine the virtual address to which the virtual function device corresponds by looking up the associated entry in the register file. This virtual address is the address that the software layer uses to access registers of the virtual function device.

It should be noted that the entries in the register file may contain mapping information of not only virtual addresses but also physical addresses. This is because the virtual address eventually needs to be translated to a physical address in order to perform the corresponding operation on the physical hardware. However, during this particular operation, it may only be of interest to determine the virtual address, and not to involve translation of the physical address.

Step S1052, determining, by the first process, a physical address of a register corresponding to the virtual address based on the virtual address and a mapping relationship, wherein the mapping relationship is used for representing a mapping relationship between the physical address of the register and a virtual address of a slave process;

Here, virtual addresses may refer to each virtual environment having its own address space, which are referred to as virtual addresses, which are addresses used by the software layer to reference resources such as memory and registers, in the virtualized environment. The physical address may refer to an actual address of a resource such as a memory and a register used by the hardware layer. The mapping relationship defines a rule for converting virtual addresses to physical addresses. In a virtualized environment, the operating system or virtualization software maintains such mappings to ensure that access requests in the virtualized environment are properly mapped onto physical hardware.

In some embodiments, the first process is capable of translating the virtual address into a physical address of a corresponding register according to the virtual address and the mapping relationship. In this way, the mapping relationship is not only used to translate virtual addresses to physical addresses, but also ensures that processes running in the virtual environment can properly access and control virtual function devices.

Step S1053, determining, by the first process, a reset register corresponding to the virtual function device based on the physical address and the address offset of the register;

Here, address offset is used to locate a particular memory location or register by adding an offset to a known base address. The reset register is a special register for controlling a reset operation of the electronic device. When the electronic device needs to be reset to its initial state, this can be done by writing a specific value to a reset register.

In some embodiments, by combining the physical address of the register with the address offset, the first process can calculate the exact physical address of the reset register, thereby determining the reset register corresponding to the virtual function device.

In step S1054, the first process resets the reset register, and closes the data transmission channel on the corresponding virtual function device.

Here, the reset operation may refer to performing a specific write operation on the reset register to trigger a reset procedure of the reset register.

In some embodiments, the data transfer channel on the corresponding virtual function device will be closed by the first process writing to the reset register, which means that the electronic device will stop sending and receiving data.

In the embodiment of the application, the register file is accessed through the first process, the virtual address is determined based on the virtual function equipment information, and then the physical address is found through the mapping relation, and finally the physical address is positioned to the reset register, so that the state of the virtual function equipment can be accurately controlled and managed, unauthorized access or operation can be prevented, and the safety of the system is improved.

In some embodiments, the implementation of step S104 "determine whether the second process exits normally" may include the following steps S1041 to S1043, where:

step S1041, developing a suite framework based on a data plane through the second process, and creating a configuration file;

Here, the data plane Development Kit (DATA PLANE Development Kit, DPDK) is a high-performance packet processing framework, which is commonly used for software Development of network devices. It provides direct access to hardware and optimized packet processing functions. Configuration files are typically used to store settings and parameters of an application or system.

In some embodiments, the second process creates a configuration file from the data plane development kit framework at runtime. This configuration file may contain the state of the application, configuration parameters, or other important information.

Step S1042, detecting the presence state of the configuration file by the callback function registered by the first process;

Here, a callback function may be a function called by a function pointer that allows a function to be automatically called when a specific event occurs.

In some embodiments, when a callback function is triggered, it will check if a configuration file exists. For example, if the configuration file exists and is expected (e.g., file size, content format, etc.), the callback function may return a status code or signal indicating success. If the configuration file is not present, corrupted, or otherwise inaccessible, the callback function may return an error status code or trigger an exception handling procedure.

Step S1043, determining whether the second process exits normally based on the presence status of the configuration file.

Here, the presence status of the profile includes the presence of the profile and the absence of the profile.

In some implementations, the first process can infer the state of the second process based on the presence state of the configuration file. If the configuration file exists, the second process normally exits, and if the configuration file does not exist, the second process abnormally exits.

In the embodiment of the application, the configuration file is created through the second process, and the callback function registered by the first process is used for detecting the existence state of the configuration file, which means that the code or logic of the second process does not need to be modified to add an additional monitoring function, thereby reducing unnecessary resource consumption.

In some embodiments, the implementation of step S1043 "determining whether the second process exits normally based on the presence status of the configuration file" may include the following steps S111 to S112, where:

step S111, determining that the second process is abnormally exited under the condition that the presence state representation of the configuration file does not exist;

In some implementations, when the presence status of the configuration file is absent, it means that the second process fails to create or update the configuration file as expected before exiting. This situation may arise from a number of reasons, such as the second process crashing, aborting, or somehow failing to execute to the code segment that created the configuration file. Thus, in this case, it can be determined that the second process is abnormally exited.

Step S112, determining that the second process normally exits under the condition that the presence state of the configuration file represents presence.

In some embodiments, if the presence status of the configuration file is present, meaning that the second process has successfully created the configuration file before exiting, and may have updated its contents as expected. Thus, in this case, it can be determined that the second process is normally exited.

In the embodiment of the application, the exiting state of the second process is determined by the existing state of the configuration file, so that erroneous judgment can be reduced to a certain extent. Because the presence of a configuration file is typically closely related to certain critical operations of a process (e.g., initialization, configuration updates, etc.), the presence status of a configuration file can more accurately reflect the running condition of a process.

In some embodiments, the processing method of data transmission may further include the following steps S121 to S124, wherein:

Step S121, adding a read-write lock to the configuration file through a first process, wherein the read-write lock is used for reading data in the configuration file by the first process and writing data in the configuration file by the second process, and the configuration file is used for judging whether the second process exits;

Here, a read-write lock is used to coordinate access to a configuration file by a first process and a second process. The first process uses a read lock when a configuration file needs to be read, and the second process uses a write lock when a configuration file needs to be updated.

In some embodiments, the first process adds a read-write lock to the configuration file, meaning that the first process will be responsible for controlling access to the configuration file to ensure that data inconsistencies do not occur when reading or writing data.

Step S122, adding a write lock to the configuration file through a callback function registered by the first process;

In some embodiments, when a first process needs to determine whether a second process is to exit, it will attempt to add a write lock to the configuration file via the registered callback function. This is because if the second process is running and possibly updating the configuration file, it should hold a write lock. Thus, if a first process is able to successfully acquire a write lock, this may mean that a second process has exited or no longer holds the write lock.

Step S123, determining that the second process exists in the case of a write lock failure;

In some implementations, if a first process fails in attempting to add a write lock (i.e., the write lock has been held by other processes), it may be determined that a second process is still present because the second process may be updating the configuration file using the write lock.

Step S124, determining that the second process exits in the case that the write lock is added successfully.

In some implementations, if the first process successfully increases the write lock, it can be inferred that the second process may have exited because the second process is no longer holding the write lock.

In the embodiment of the application, the callback function registered by the first process tries to add the write lock to the configuration file, so that whether the second process exists can be accurately judged. If the add write lock fails, indicating that the second process may be using the write lock for write operations, it may be determined that the second process exists, and if the add write lock is successful, it may mean that the second process has exited or no longer holds the write lock, and it may be inferred that the second process may have exited.

In some embodiments, the processing method of data transmission may further include the following steps S131 to S132, wherein:

step S131, under the condition that the second process normally exits, resetting the accelerator card hardware through the second process;

Here, an accelerator card is a hardware device that is generally used to accelerate a specific computing task, such as graphic processing, data processing, and the like. It may be a separate circuit board connected to the motherboard of the computer via a socket or may be a functional module integrated into the processor or other component.

In some embodiments, the second process may be responsible for performing a reset operation on the accelerator card when the second process exits normally. For example, by the second process sending a specific command or signal to the accelerator card to ensure that the accelerator card is in a clean, consistent state the next time it is used.

Step S132, after the accelerator card hardware completes the reset operation, the first process is closed by the second process.

In some embodiments, after the accelerator card hardware reset is completed, the second process may perform an operation to shut down the first process. The second process may close the first process for a number of reasons, such as the task of the first process has been completed, or the interaction of the first process with the accelerator card hardware has been completed, and no further operation of the first process is required. In addition, the first process may be closed to release system resources, so as to improve the overall performance of the system.

In the embodiment of the application, the second process is used for resetting the accelerator card hardware, so that the accelerator card hardware is ensured to be in a clean and consistent state when being used next time, and the reliability and stability of hardware resources are improved.

In the fifth generation mobile communication technology (5th Generation Mobile Communication Technology,5G) baseband processing Unit (Base Band Unit, BBU), the amount of data for the forward service and the backward service interaction is more than that of the fourth generation mobile communication technology (4th Generation Mobile Communication Technology,4G), so for realizing the data interaction with large bandwidth and low time delay, based on PCIe accelerator card hardware, a data plane Development Kit (DATA PLANE Development Kit, DPDK) framework is used, and the BBU and external data are interacted in a zero copy mode through user mode driving. It should be noted that, the forwarding service mainly carries mobile communication services, such as voice call, short message, and mobile data services (e.g., internet access, application use, etc.), and the data flow of the forwarding service is from the mobile device (e.g., mobile phone, tablet computer, etc.) to the mobile network. The backhaul service is mainly used for uplink data transmission, for example, a user uploads data to the internet or a cloud server, and the data flow of the backhaul service is from a mobile network to a core network or a cloud server. Zero copy means that when data is read from disk to kernel space, it can be transferred from kernel space to application's memory space without first copying to user space.

As shown in fig. 2, the system architecture provided in the embodiment of the present application includes a User space (User space) 21, a kernel space (KERNEL SPACE) 22, a peripheral component interconnect express bus 23 and Hardware (Hardware) 24, where the User space 21 includes a forwarding service software 211, a backhaul service software 212, a daemon (i.e., the first process described above) 213, a register file 214, a data plane development suite 215, the data plane development suite 215 includes a data plane development suite library (libdpdk) 2151, a Poll Mode Driver (PMD) 2152, the kernel space 22 includes a network device Driver (NETDEV DRIVER) 221 and a Driver (vfio-pci) 222, the Hardware 24 includes an accelerator card 241, and the accelerator card 241 includes a Base address register (Base ADDRESS REGISTER 0, bar0) 2411, a protocol core 2412 (Internet Protocol core, IP core) for interconnection between networks, a Field Programmable gate array (Field-Programmable GATE ARRAY, FPGA) 3 and an optical port (6 optical ports are shown in fig. 2).

Note that libdpdk is the memory that the service needs to use, and some interfaces are provided through libdpdk to connect PMD and drive hardware. The resources applied by the physical devices are independent, the resources applied by the virtual devices are shared, the resources of the same physical device can be shared by multiple virtual resources, and the ports can be widened.

Based on the system framework, the forward service software and the backward service software of the BBU can perform high-bandwidth and high-speed data interaction with the accelerator card. However, because the software business logic is complex and the tasks are many, it is difficult to ensure that the software does not exit abnormally. Because the DPDK user mode driver is used as a link library, the driver can also exit abnormally along with the software, so that PCIe accelerator card resources cannot be reset normally. The data channels of the corresponding services on the accelerator card cannot stop and continue to perform direct memory access (Direct Memory Access, DMA), so that corresponding memory information cannot be acquired when the IP core in the FPGA reads data, and PCIe completion timeout errors are generated. In addition, the ready signal (data transmission signal) of the FPGA may be pulled down for a long time, which causes the whole PCIe link to hang up, and the BBU cannot work after restarting.

After the software is started, the IP library is started, the software is abnormally exited, the IP library is not told to stop, and the memory can be mastered, but after the software is exited, the memory can be recovered by the kernel, and after the recovery, illegal addresses are accessed, so that PCIe acceleration card resources cannot be reset normally. The optical port in PCIe accelerator card hardware refers to a port with a high speed fiber optic network interface. The port takes optical ports of 10 gigabits per second (Gbps, G), 25G, 40G and 100G as main forces, different optical ports realize photoelectric signal conversion through an external hot-pluggable optical module, and then long-distance data transmission is carried out through an optical fiber.

Netdev provide a series of application programming interfaces (Application Programming Interface, APIs) and data structures that enable communication and interaction between drivers of network devices and operating system cores. In addition, netdev further includes a series of network protocol stacks and network function modules, which interact with the network device driver through the interface of netdev, so as to implement transmission, processing and routing of network data. vfio-pci is a driver in the Linux kernel that allows these devices to access hardware in the user state through virtual function input output (Virtual Function I/O, VFIO) drivers. In the kernel space, there are physical functions (Physical Function, PF) (i.e., PF0 through PF2 in fig. 2) and Virtual Functions (VF) (i.e., VF0 through VF3 in fig. 2).

In the related art, a delay detection circuit is added on a PCIe acceleration card, when software is abnormally exited, if IP (Internet protocol) read memory data is overtime, after the delay detection circuit detects a delay signal, PCIE RESET (reset) signals are sent to stop and reset the FPGA, so that the recovery of the whole PCIe link is realized when the software is abnormally. The method has the following defects that 1) in a delay detection circuit PCIe link scheme, although the function of PCIe link recovery is realized, after different VFs are created by adopting a Single I/O virtualization (Single Root I/OVirtualization, SR-IOV) function, the link recovery of a specific VF channel cannot be carried out, so that the services of other VF channels on the PCIe accelerator card are restarted. 2) PCIe link recovery involves resetting of FPGA hardware and restarting of traffic on the entire PCIe accelerator card, so that PCIe link recovery time is long, more traffic needs to be pulled up by the BBU, and fast recovery of traffic cannot be achieved in 5G systems. 3) The complexity of the hardware design and the cost of the hardware are increased.

In the related art, there is no slave process, and the master process creates a memory pool for storing a carrier for transmitting data information, and takes a memory buffer (mbuf) from the memory pool when transmitting data using one carrier. In the embodiment of the application, the secondary process applies for mbuf, so that the memories can be saved when the main process exits abnormally, the kernel is not recycled, and PCIe errors cannot occur. The slave process and the master process share a memory pool, the slave process does not do any business, the slave process creates the memory pool and gives the memory pool to the master process for use, but after the master process abnormally exits, the memory pool cannot be released immediately, the slave process can exit together with the master process after the slave process is reset and exits, and finally the memory pool is released.

The embodiment of the application provides another processing method for data transmission, which is characterized in that a daemon (slave process) is pulled up when service software (master process) is started, the slave process is based on a DPDK frame, VF device information and a large page memory space initialized by the master process are shared, the slave process is used for creating a memory pool based on the memory space and transmitting an address of the memory pool to the master process, the master process is used for creating an information carrier mbuf based on the memory pool when DMA (direct memory access) interacts service data, the slave process is used for keeping the memory pool space to exist, a timer callback function is registered, and the callback function periodically detects whether the master process exists. When the main process is abnormally exited, a callback function of the slave process triggers a PCIe resource reset function, the reset function accesses a BAR0 register file (for example, a register address) through the VF device information of the main process, and finds a reset register through address offset to quickly initiate the recovery of a PCIe link.

An embodiment of the present application provides another processing method for data transmission, as shown in fig. 3, including the following steps:

step 301, starting a main process (namely the second process);

step 302, the main process creates a config file (i.e. the configuration file) based on the DPDK frame, initializes VF device information and a large page memory space (i.e. the memory space);

Step 303, the master process starts a slave process (i.e. the first process described above);

step 304, starting from the process;

step 305, the slave process shares the VF device information and the large page memory space after the initialization of the master process;

step 306, registering a timer callback function from the process;

Step 307, creating a memory pool (mempool) from the process based on the large page memory space;

step 308, the main process obtains the address of the memory pool;

Step 309, the main process executes business logic;

Step 310, judging whether the main process normally exits based on the existence state of the config file, if yes, namely the config file exists, the main process normally exits, entering step 311, and if not, namely the config file does not exist, the main process abnormally exits, entering step 313;

Step 311, the main process releases PCIe resources;

Step 312, the master process shuts down the slave process;

step 313, triggering a callback function from the process to detect a config file;

Step 314, the slave process accesses the base address register file and mmap through the VF device information;

Step 315, the slave process determines the reset register address of the VF, writes the corresponding value to stop the DMA of the IP, and restores the PCIe link.

It should be noted that, both the step 314 and the step 315 require PCIe resource reset functions.

In the embodiment of the application, service software (a main process) is started, a config file is created under a shared directory based on a DPDK frame, VF equipment and a large page memory are initialized (a page table is created for each process to map a virtual address to a physical address), wherein the config file is additionally provided with a read-write lock which can only be written by the main process and is read by the slave process, and the method is used for recording the information of the service process initialization memory and storing the large page memory information into the shared directory. The master process pulls up a daemon (slave process), which also acquires BDF number information of the VF device based on the DPDK frame, and acquires large page memory information created by the master process through the shared directory, where the large page memory information includes a memory size, a large page type (1 GB or 2 MB), a physical memory descriptor (e.g., a segment Base address (Base) indicates a starting physical address of a segment, a segment Limit (Limit) indicates a length of memory occupied by a program (or data) segment, a segment attribute (Attr) describes an access right of the segment, and so on. The slave process registers a timer callback function for detecting whether the master process exists, then creates a memory pool according to the acquired large page memory information, and communicates with the master process to share the address of the memory pool. In the business process of the main process, the slave process judges whether the main process survives or not by detecting the config file at regular time, wherein the config file is recovered by the kernel after the main process exits. For example, config is read-write locked by the host process, the slave process will detect periodically, attempting to lock the Config file by calling froc functions, the failure of the lock to lock proves that the host process is alive, and the success of the lock to lock indicates that the host process is not alive. If the main process normally exits, resetting PCIe resources to close the slave process, otherwise triggering a callback function, calling a PCIe resource resetting function, accessing a BAR0 register file resource0 mapped by a kernel driver by the VF equipment information of the main process by the resetting function, determining the reset register address of the VF based on the virtual address of the file mmap and the address offset, writing a corresponding value to stop the DMA (direct memory access) action of the IP, and quickly recovering a PCIe link. Wherein the kernel driver has mapped the corresponding virtual function base address register (Virtual Function Base ADDRESS REGISTERS, VF BAR Registers) address space to the file in advance at the time of VF creation.

It should be noted that, in the kernel, each process has its own virtual address space, and this virtual address refers to the virtual address in the slave process, where the physical address has only one part, and the virtual address has multiple parts. The slave process of the user space cannot directly access the physical address, one process is set in Linux and needs to access the physical address through the virtual address, then the mmap is walked to tell the central processing unit (Central Processing Unit, CPU) to take the physical address through the mmap and then access the file of the register, wherein the mapping relation exists between the virtual addresses seen by the process.

The embodiment of the application provides a kernel driver for creating VF and initializing PCIe resources, as shown in FIG. 4, comprising the following steps:

step 401, starting a Linux system;

step 402, starting an acceleration card kernel driver and binding a corresponding PF;

Here, after the Linux system is started, the acceleration card kernel driver is started, and the corresponding PF is bound.

Step 403, creating VFs on each PF based on the SR-IOV function;

step 404, the kernel driver initializes each VFPCIe fabric information and maps the PCIe fabric information to a file;

Here, after the VF is created, the kernel driver is loaded, which initializes PCIe fabric information for each VF and maps the information into a system directory/sys/bus/pci/devices/BDF, which contains the mapped BAR0 register file resource0

Step 405, binding vfio-pci drivers corresponding to the VF devices;

Here, the VF device binding vfio-pci is driven to bypass the kernel.

The upper software transfers the data based on the VF DMA, step 406.

Here, the host process may operate the physical device to DMA transfer data.

Compared with the related art, the embodiment of the application has the following advantages:

1. After the upper layer software is abnormally exited, the PCIe link can be quickly restored, and the PCIe error and the condition that the PCIe link is hung up are avoided.

2. The master process and the slave process share the large page memory, the slave process keeps the large page memory information, the PCIe error and the PCIe link suspension are avoided under the condition that the data transmission performance is not affected, and the PCIe link can be quickly recovered.

3. Based on the BAR register file mapped by the kernel drive, the reset register is accessed when abnormality occurs through the virtual address and the offset address of the mmap, and reset action is carried out on the appointed VF, so that the influence of integral reset on the business functions of other VFs is avoided, and the BBU can quickly recover the business process.

4. By detecting the host process state, the PCIe resources of the VF are quickly reset based on the mapped register file when abnormal.

5. Based on the plurality of VFs created by the kernel-driven SR-IOV function, PCIe resources can be reset for the specific VFs which are abnormally exited by software, and normal interactive service data of other VFs are not influenced.

6. After the software is abnormal, the service process with the problem can be quickly and independently recovered, and the BBU does not need to process the service processes of other VFs on the PCIe accelerator card.

7. Compared with a hardware solution, the hardware is not required to be changed, the complexity of hardware design is reduced, and the cost is saved.

Based on the foregoing embodiments, an embodiment of the present application provides a processing apparatus for data transmission, as shown in fig. 5, where the processing apparatus 500 for data transmission includes:

the obtaining module 510 is configured to obtain, by using a first process, a memory space created by a second process, where the first process is a process created by the second process;

A first creating module 520, configured to create, by the first process, a memory pool based on the memory space when the second process starts the first process, where the memory pool is used to store data transmitted by the second process, and the first process and the second process communicate by sharing an address of the memory pool;

An execution module 530, configured to execute, by the second process, a service based on an address of the memory pool;

a first determining module 540, configured to determine, during a process of executing a service by the second process, whether the second process exits normally;

And the first closing module 550 is configured to close, by the first process, a data transmission channel on the corresponding virtual function device based on the virtual function device information corresponding to the second process, in the case that the second process exits abnormally.

In some embodiments, the first creation module 520 includes a mapping unit configured to map, by the first process, an address of the memory space to the first process, to obtain a mapped memory space, and a first determination unit configured to determine, by the second process, a part of the space in the mapped memory space as the memory pool.

In some embodiments, the first closing module 550 includes a second determining unit configured to access, by the first process, a register file based on the virtual function device information, to determine a virtual address corresponding to the virtual function device, the register file including a virtual address mapped by a physical address of a register of the virtual function device, a third determining unit configured to determine, by the first process, a physical address of a register corresponding to the virtual address based on the virtual address and a mapping relationship, wherein the mapping relationship is used to characterize a mapping relationship between the physical address of the register and a virtual address of a slave process, a fourth determining unit configured to determine, by the first process, a reset register corresponding to the virtual function device based on the physical address and an address offset of the register, and a closing unit configured to perform a reset operation on the reset register by the first process, and close a data transmission channel on the corresponding virtual function device.

In some embodiments, the first determining module 540 includes a creating unit configured to create a configuration file based on a data plane development suite framework through the second process, a detecting unit configured to detect a presence state of the configuration file through a callback function registered by the first process, and a fifth determining unit configured to determine whether the second process exits normally based on the presence state of the configuration file.

In some embodiments, the fifth determining unit comprises a first determining subunit, configured to determine that the second process exits abnormally when the presence status characterization of the configuration file does not exist, and a second determining subunit, configured to determine that the second process exits normally when the presence status characterization of the configuration file exists.

In some embodiments, the processing device 500 for data transmission further includes a first adding module configured to add a read-write lock to the configuration file through a first process, the read-write lock is configured to read data in the configuration file by the first process and write data in the configuration file by the second process, the configuration file is configured to determine whether the second process exits, a second adding module configured to add a write lock to the configuration file through a callback function registered by the first process, a second determining module configured to determine that the second process exists if the write lock addition fails, and a third determining module configured to determine that the second process exits if the write lock addition is successful.

In some embodiments, the processing device 500 for data transmission further includes a reset module configured to perform a reset operation on the accelerator card hardware through the second process when the second process exits normally, and a second shutdown module configured to shutdown the first process through the second process after the reset operation is completed on the accelerator card hardware.

The description of the apparatus embodiments above is similar to that of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, please refer to the description of the embodiments of the method of the present application.

It should be noted that, in the embodiment of the present application, if the method is implemented in the form of a software functional module, and sold or used as a separate product, the method may also be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be embodied in essence or a part contributing to the related art in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. The storage medium includes various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the application are not limited to any specific combination of hardware and software.

The embodiment of the application provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor realizes the method when executing the computer program.

Embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described method. The computer readable storage medium may be transitory or non-transitory.

Embodiments of the present application provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program which, when read and executed by a computer, performs some or all of the steps of the above-described method. The computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.

It should be noted that, in the embodiment of the present application, a hardware entity of an electronic device is provided, as shown in fig. 6, where the hardware entity of the electronic device 600 includes a processor 601, a communication interface 602, and a memory 603, where the processor 601 generally controls an overall operation of the electronic device 600. The communication interface 602 may enable the electronic device to communicate with other terminals or servers over a network. The memory 603 is configured to store instructions and applications executable by the processor 601, and may also cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or processed by various modules in the processor 601 and the electronic device 600, which may be implemented by a FLASH memory (FLASH) or a random access memory (Random Access Memory, RAM). Data transfer may occur between the processor 601, the communication interface 602 and the memory 603 via the bus 605.

It should be noted here that the description of the storage medium and the device embodiments above is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and the apparatus of the present application, please refer to the description of the method embodiments of the present application.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence number of each step/process described above does not mean that the execution sequence of each step/process should be determined by its functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be additional divisions of actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.

The units described as separate components may or may not be physically separate, and components displayed as units may or may not be physical units, may be located in one place or distributed on a plurality of network units, and may select some or all of the units according to actual needs to achieve the purpose of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of hardware plus a form of software functional unit.

It will be appreciated by those of ordinary skill in the art that implementing all or part of the steps of the above method embodiments may be implemented by hardware associated with program instructions, where the above program may be stored in a computer readable storage medium, where the program when executed performs the steps comprising the above method embodiments, where the above storage medium includes various media that may store program code, such as a removable storage device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Or the above-described integrated units of the application may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the related art in the form of a software product stored in a storage medium, including several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the methods described in the embodiments of the present application. The storage medium includes various media capable of storing program codes such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The foregoing is merely an embodiment of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application.

Claims

1. A method of processing data transmissions, the method comprising:

the method comprises the steps of obtaining a memory space created by a second process through a first process, wherein the first process is a process created by the second process;

Under the condition that the second process starts the first process, a memory pool is created by the first process based on the memory space, wherein the memory pool is used for storing data transmitted by the second process, and the first process and the second process communicate through sharing the address of the memory pool;

executing the service based on the address of the memory pool through the second process;

in the process of executing the service by the second process, determining whether the second process exits normally;

And under the condition that the second process abnormally exits, closing a data transmission channel on the corresponding virtual function device through the first process based on the virtual function device information corresponding to the second process.

2. The method of claim 1, the creating, by the first process, a memory pool based on the memory space, comprising:

mapping the address of the memory space to the first process through the first process to obtain a mapped memory space;

and determining part of the mapped memory space as the memory pool through the second process.

3. The method of claim 1, wherein the closing, by the first process, the data transmission channel on the corresponding virtual function device based on the virtual function device information corresponding to the second process, includes:

Accessing a register file based on the virtual function device information through the first process to determine a virtual address corresponding to the virtual function device, wherein the register file comprises a virtual address mapped by a physical address of a register of the virtual function device;

determining a physical address of a register corresponding to the virtual address through the first process based on the virtual address and a mapping relation, wherein the mapping relation is used for representing the mapping relation between the physical address of the register and the virtual address of a slave process;

Determining a reset register corresponding to the virtual function device by the first process based on the physical address and the address offset of the register;

And resetting the reset register through the first process, and closing a data transmission channel on the corresponding virtual function device.

4. The method of claim 1, the determining whether the second process is normally exited, comprising:

creating a configuration file based on a data plane development suite framework through the second process;

Detecting the existence state of the configuration file through a callback function registered by the first process;

And determining whether the second process exits normally or not based on the existence state of the configuration file.

5. The method of claim 4, the determining whether the second process exits normally based on the presence status of the configuration file, comprising:

determining that the second process is abnormally exited under the condition that the presence state representation of the configuration file does not exist;

And under the condition that the presence state of the configuration file represents presence, determining that the second process normally exits.

6. The method of claim 4 or 5, the method further comprising:

The method comprises the steps of adding a read-write lock to a configuration file through a first process, wherein the read-write lock is used for reading data in the configuration file by the first process and writing data in the configuration file by a second process, and the configuration file is used for judging whether the second process exits or not;

adding a write lock to the configuration file through a callback function registered by the first process;

Determining that the second process exists under the condition that the write lock addition fails;

And determining that the second process exits under the condition that the write lock is added successfully.

7. The method of any one of claims 1 to 6, further comprising:

under the condition that the second process normally exits, resetting the accelerator card hardware through the second process;

And after the accelerator card hardware completes the reset operation, closing the first process through the second process.

8. A processing apparatus for data transmission, the apparatus comprising:

9. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor implementing the steps of the method of any one of claims 1 to 7 when the program is executed.

10. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of claims 1 to 7.