CN114463163A - Heterogeneous multi-core image processing method and device - Google Patents
Heterogeneous multi-core image processing method and device Download PDFInfo
- Publication number
- CN114463163A CN114463163A CN202210053900.7A CN202210053900A CN114463163A CN 114463163 A CN114463163 A CN 114463163A CN 202210053900 A CN202210053900 A CN 202210053900A CN 114463163 A CN114463163 A CN 114463163A
- Authority
- CN
- China
- Prior art keywords
- operating system
- image
- data
- linux operating
- cache region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/161—Computing infrastructure, e.g. computer clusters, blade chassis or hardware partitioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/167—Interprocessor communication using a common memory, e.g. mailbox
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention provides a heterogeneous multi-core image processing method and a device, wherein the method comprises the following steps: A. the Linux operating system receives an acquisition instruction of a user, writes the acquisition instruction into the class shared memory, and triggers IPI interruption to a FreeRTOS operating system; B. the FreeRTOS operating system reads an acquisition instruction from the class shared memory and controls the USB3.0 industrial camera to acquire images; C. the Linux operating system puts the image into an image cache region according to frames; D. the image processor reads the image in the image buffer area, processes the image and puts the processing result into an FIFO stack; E. the image processor carries the processing result in the FIFO stack to the result cache region, and the Linux operating system sends the data in the result cache region to the user. The invention realizes the effective complementation of real-time property and operability, can finish the acquisition and processing of images at high speed, and enables the USB3.0 industrial camera to be directly accessed, thereby greatly reducing the research cost and the technical threshold of a high-speed image acquisition and processing system.
Description
Technical Field
The invention relates to a heterogeneous multi-core image processing method and device.
Background
Most of image acquisition interfaces in the current high-speed high-performance image processing system are realized by FPGA (field programmable gate array), the existing USB3.0 industrial camera in the market cannot be directly used, a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) camera needs to be built by oneself, and a data communication protocol is realized, so that the development cycle is long, and the labor cost is high.
The reasons why the existing USB3.0 industrial camera in the market can not be adopted mainly include the following points:
if the system adopts the existing USB3.0 industrial camera in the market, the ARM is generally adopted to acquire image data, the image data is transmitted to the PC end through a gigabit network or other interfaces, then the PC end carries out image processing (processing according to frames), and finally a required processing result is output. The complete process needs two communication links, and the PC image processing needs to be performed after receiving a complete image, so the processing speed is far lower than the frame rate of an industrial camera, and the requirement of high speed and high performance cannot be met.
Meanwhile, in order to realize communication with the industrial camera, the ARM end generally adopts a Linux system as a software operating system, and the Linux system is a non-real-time operating system, so that response to a hardware signal has a certain delay, and delay of a high real-time signal is easily caused. Meanwhile, the data interaction of a large data volume between the kernel mode and the user mode of the Linux system has a difficult problem.
Disclosure of Invention
The invention provides an image processing method and device for heterogeneous multi-core, which can realize effective complementation of real-time performance and operability, can finish acquisition and processing of images at high speed, enables a USB3.0 industrial camera to be directly accessed, and greatly reduces the research cost and technical threshold of a high-speed image acquisition and processing system without independently developing data acquisition and processing software of an FPGA (field programmable gate array) by a user.
The invention is realized by the following technical scheme:
a heterogeneous multi-core image processing method is based on a Zynq UltraScale + MPSoC chip and is used for controlling a USB3.0 industrial camera to acquire images and processing the acquired images, and comprises the following steps:
A. after receiving an acquisition instruction of a user, the Linux operating system writes the acquisition instruction into a class shared memory and triggers IPI to be interrupted to a FreeRTOS operating system, wherein the Linux operating system and the FreeRTOS operating system respectively run on four A53 hard cores and two R5 hard cores of a Zynq UltraScale + MPSoC chip;
B. the FreeRTOS operating system responds to the interruption in the step A, reads an acquisition instruction from the class shared memory and controls the USB3.0 industrial camera to acquire images according to the acquisition instruction;
C. outputting an image collected by the USB3.0 industrial camera to a Linux operating system, putting the image into an image cache region by frames according to a current writing address by the Linux operating system, recalculating a new current writing address, and taking an initial address of the image cache region as a new current writing address when the new current writing address exceeds a preset boundary;
D. the image processor reads the image in the image cache region according to a read enabling signal sent by a Linux operating system and a read address, processes the image according to frames, places the processing result into an FIFO stack, and adopts a concurrent execution process as the reading and processing process of the image processor, wherein the image processor runs in a PL unit of a Zynq UltraScale + MPSoC chip;
E. the image processor carries the processing result in the FIFO stack to a result cache region according to a write enable signal sent by the Linux operating system, the Linux operating system sends the data of the result cache region to a user, and the data transmission process is as follows: and taking the data with the set frame number and continuous frame number as a group of data, after the group of data is sent, if the communication result replied by the user is successful, continuing to send the next group of data, and otherwise, resending the group of data.
Further, the image buffer in step C includes a first-level buffer and a second-level buffer, both the first-level buffer and the second-level buffer include a first read-write control area and a first data area, the first data area of the first-level buffer stores the frame number, image information, and image data of an image, the first read-write control area of the second-level buffer stores the processing status, the first data area stores image data, the Linux operating system places the image into the first-level buffer according to the current write address, and copies the image data at the current read address of the first-level buffer to the current write address of the second-level buffer when the current write address data of the second-level buffer is processed, and the image processor reads the image data in the second-level buffer according to the read address.
Further, the shared memory class is composed of DDR4 memory areas which are not used by a Linux operating system and a FreeRTOS operating system.
Further, in the step a, after receiving the acquisition instruction of the user, the Linux operating system checks the acquisition instruction by using CRC16, if the check is correct, the frame type corresponding to the acquisition instruction is analyzed, and if the frame type is the same as the preset type, the acquisition instruction is written into the class shared memory.
Furthermore, the class shared memory comprises a second read-write control area and a second data area, the second read-write control area is used for preventing a FreeRTOS operating system and a Linux operating system from writing data at the same time, and the data area is used for data interaction between the FreeRTOS operating system and the Linux operating system.
Further, before the step a, the method further comprises the following steps: after the Linux operating system is started, creating Communication _ Thread, Camera _ Thread, RpmsgCtrl _ Thread sub-threads:
in the Communication _ Thread of the sub-Thread, a special Communication protocol stack based on the gigabit giant frame is established for communicating with the user;
establishing communication connection with the USB3.0 industrial Camera in the child Thread Camera _ Thread, and configuring the working mode of the USB3.0 industrial Camera;
in the sub-Thread RpmsgCtrl _ Thread, an inter-core communication mechanism based on class shared memory + IPI interrupt is created for data communication with a FreeRTOS operating system, and a collection instruction is transmitted to the class shared memory.
The invention is also realized by the following technical scheme:
the image processing device based on the heterogeneous multi-core image processing method comprises a main control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the main control core circuit comprises a Zynq UltraScale + MPSoC chip, a DDR4 memory chip, a QSPI FLASH memory chip and an EMMC memory chip which are respectively connected with the Zynq UltraScale + MPSoC chip, the communication transmission circuit and the acquisition control circuit are respectively connected with the Zynq UltraScale + MPSoC chip, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with an input end of a USB3.0 industrial camera, an output end of the USB3.0 industrial camera is connected with the communication transmission circuit, a Linux operating system, a FreeRTOS operating system and an image processor are operated on the Zynq UltraScale + MPSoC chip, an acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the acquisition instruction with the FreeRTOS operating system, and controls the USB3.0 image acquisition camera according to the acquisition instruction, the collected images are transmitted to a Linux operating system through a communication transmission circuit, the Linux operating system shares the images to an image processor, the image processor processes the images and shares processing results to the Linux operating system, and the Linux operating system sends the processing results to a user.
Further, the communication circuit comprises a USB ULPI transceiving unit connected with the output end of the USB3.0 industrial camera and a gigabit network PHY unit connected with a user.
Further, the acquisition control circuit comprises an encoder signal detection unit, an input IO circuit and an output IO circuit.
Furthermore, the model of the Zynq UltraScale + MPSoC chip is XCZU3EG-1SFVC784I of Zynq UltraScale + MPSoCs EG series, the model of the DDR4 memory chip is MT40A512M16GE, the model of the QSPI FLASH memory chip is MT25QU256ABA1EW9, and the model of the EMMC memory chip is MTFC8 GAKAJCN-4M.
The invention has the following beneficial effects:
1. the invention adopts a dual-operation system architecture of a FreeRTOS operation system and a Linux operation system, wherein the Linux operation system runs on four A53 hard cores of a Zynq UltraScale + MPSoC chip, the FreeRTOS operation system runs on two R5 hard cores of the Zynq UltraScale + MPSoC chip, the A53 hard core is a high-performance hard core, and the R5 hard core is a real-time hard core, so that the effective complementation of real-time property and operability is realized, and the problem of insufficient real-time property of single Linux system hardware in the prior art is solved; after receiving the acquisition instruction of the user, the Linux operating system writes the instruction into the class shared memory, triggering IPI interrupt to the FreeRTOS operating system, responding the interrupt by the FreeRTOS operating system, reading the acquisition instruction from the class shared memory, outputting the image acquired by the USB3.0 to the Linux operating system and putting the image into the image cache region by the Linux operating system, meanwhile, the image processor reads the image in the image buffer area according to the reading address and processes the image according to the frame, after the processing is finished, the processing result is transferred to the result buffer area and then sent to the user by the Linux operating system, through the class shared memory + IP interruption notice and the setting of the annular cache of the image cache region, the FreeRTOS operating system, the Linux operating system and the image processor can realize the real-time sharing of data, the reading and processing processes of the image processor are concurrent execution processes, so that the acquisition and processing of the image can be completed at high speed; the FreeRTOS operating system can directly control the USB3.0 industrial camera to acquire images according to the acquisition instruction, and the acquired images can be transmitted to the Linux operating system, so that the USB3.0 industrial camera can be directly accessed and used, a user does not need to independently develop data acquisition and processing software of the FPGA, and the research cost and the technical threshold of the high-speed image acquisition and processing system are greatly reduced.
Drawings
The present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a schematic block diagram of the apparatus of the present invention.
FIG. 2 is a flow chart of the method of the present invention.
Wherein, 11, Zynq UltraScale + MPSoC chip; 12. QSPI FLASH a memory chip; 13. an EMMC memory chip; 14. a DDR4 memory chip; 21. an encoder signal detection unit; 22. an input IO circuit; 23. an output IO circuit; 31. a USB ULPI transceiver unit; 32. a gigabit network PHY unit; 4. USB3.0 Industrial Camera.
Detailed Description
As shown in figure 1, the heterogeneous multi-core image processing device comprises a Zynq UltraScale + MPSoC chip, a master control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the master control core circuit comprises a Zynq UltraScale + MPSoC chip 11, a DDR4 memory chip 14, a QSPI FLASH memory chip 12 and an EMMC memory chip 13 which are respectively connected with the Zynq UltraScale + MPSoC chip 11 in a bidirectional way, the communication transmission circuit and the acquisition control circuit are respectively connected with the Zynq UltraScale + MPSoC chip 11, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with the input end of a USB3.0 industrial camera 4, the output end of the USB3.0 industrial camera 4 is connected with the communication transmission circuit, a Linux operating system, a FreeRTOS operating system and an image processor are operated on the Zynq UltraScale + MPSoC chip 11, an acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, and the FreeOS operating system shares the acquisition operating system with the MPOS operating system, the FreeRTOS operating system controls the USB3.0 industrial camera 4 to collect images according to the collection instruction, the collected images are transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the images to the image processor, the image processor processes the images and shares the processing results to the Linux operating system, and the Linux operating system sends the processing results to the user.
More specifically, the Zynq UltraScale + MPSoC chip 11 includes a PS unit and a PL unit, the PS unit includes 4 a53 hardcores (i.e., Cortex-a53 hardcores) and 2R 5 cores (i.e., Cortex-R5 hardcores), the PL unit is a programmable logic unit (FPGA), a Linux operating system runs on four a53 hardcores, a FreeRTOS operating system runs on R5 hardcores, and an image processor runs on the PL unit. The communication circuit comprises a USB ULPI transceiving unit 31 and a gigabit network PHY unit 32, wherein the USB ULPI transceiving unit 31 is connected with the output end of the USB3.0 industrial camera 4 in a bidirectional mode, the gigabit network PHY unit 32 is connected with a user in a bidirectional mode, and the USB ULPI transceiving unit 31 and the gigabit network PHY unit 32 are further connected with the Zynq UltraScale + MPSoC chip 11 in a bidirectional mode respectively. The acquisition control circuit comprises an encoder signal detection unit 21 and an input IO circuit 22, the output end of the encoder signal detection unit is connected with the Zynq UltraScale + MPSoC chip 11, the input end of the output IO circuit 23 is connected with the Zynq UltraScale + MPSoC chip 11, and the output end of the output IO circuit 23 is connected with the USB3.0 industrial camera 4. The encoder signal detection unit 21, the input IO circuit 22, and the output IO circuit 23 are all related art.
In this embodiment, the model of the Zynq UltraScale + MPSoC chip 11 is XCZU3EG-1SFVC784I of the Zynq UltraScale + MPSoCs EG series, the model of the DDR4 memory chip 14 is MT40a512M16GE, the model of the QSPI FLASH memory chip 12 is MT25QU256ABA1EW9, the model of the EMMC memory chip 13 is MTFC8GAKAJCN-4M, the model of the gigabit network PHY unit 32 is DP83867, and the model of the USB ULPI transceiver unit 31 is USB 3320C.
As shown in fig. 2, a heterogeneous multi-core image processing method includes the following steps:
A. initialization: the method specifically comprises the following steps:
a1, initializing the EMMC chip after being electrified, and initializing the PS end and the PL end of the Zynq UltraScale + MPSoC chip 11;
a2, booting the Linux operating system from the EMMC memory chip 13 to start, initializing the class shared memory, IPI interrupt and exclusive lock used for real-time data Communication after the Linux operating system is started, and creating four sub-threads of Communication _ Thread, Camera _ Thread, RpmsgCtrl _ Thread and Period _ Thread:
in the Communication _ Thread of the sub-Thread, a special Communication protocol stack based on a gigabit-capable giant frame is established for communicating with a user (namely a PC upper computer); the specific protocol stack establishing process is the prior art;
establishing communication connection with the USB3.0 industrial Camera 4 in the child Thread Camera _ Thread, and configuring the working mode of the USB3.0 industrial Camera 4;
in a sub-Thread RpmsgCtrl _ Thread, an inter-core communication mechanism based on class shared memory + IPI interruption is established for data communication with a FreeRTOS operating system, and an acquisition instruction is transmitted to the class shared memory;
creating a periodic management Thread in the child Thread Period _ Thread for managing parameter data, log information and the like;
a3, starting a FreeRTOS operating system by a Linux operating system, and after the FreeRTOS operating system is started, creating four subtasks of TriggerTask, RpmSeidetTask, RpmsSendTask and PeriodTask:
in the subtask TriggerTask, according to an acquisition instruction transmitted by a Linux operating system, a signal input to an IO circuit 22 or a signal of an encoder signal detection unit 21 is responded to control an acquisition behavior of the USB3.0 industrial camera 4;
establishing an inter-core communication mechanism based on class shared memory + IPI interruption in the subtask RpmSeeveTask, and receiving and processing appointed data from a Linux operating system;
receiving message queues from TriggerTask and PeriodoTask in a subtask RpmsgSendTask, and forwarding the message queues to a Linux operating system through an inter-core communication mechanism;
creating a periodic management task in a subtask periodic task for processing fault alarm;
B. the method comprises the steps that a user sets an instruction type of a USB3.0 industrial camera 4 acquisition action through a PC upper computer, the action instruction type comprises an IO signal or an acquisition instruction sent by the PC upper computer, if the set action instruction type is the IO signal, the start of the acquisition action of the USB3.0 industrial camera 4 is controlled by an external signal of an input IO circuit, the input IO circuit detects the responsibility of a FreeRTOS operating system in real time, if the set action instruction type is the acquisition instruction, after a Communication _ Thread of a Linux operating system receives the acquisition instruction, the CRC16 is adopted to check the acquisition instruction to determine whether the acquisition instruction generates an error in the transmission process, if the check is correct, a frame type corresponding to the acquisition instruction is analyzed, if the frame type is the same as a preset type (the setting of the type is carried out through the PC), the acquisition instruction is written into a class shared memory, and IPI interruption (inter-core interruption) is triggered to the FreeRTOS, otherwise, not processing;
C. the FreeRTOS operating system responds to the interruption in the step B and reads the acquisition instruction from the class shared memory, so that the real-time transmission of each phase parameter contained in the acquisition instruction between the Linux operating system and the FreeRTOS operating system is realized; and controlling the USB3.0 industrial camera 4 to collect images according to the collecting instruction;
the class shared memory is composed of DDR4 memory areas which are not used by a Linux operating system and a FreeRTOS operating system; the shared memory comprises a second read-write control area and a second data area, wherein the second read-write control area is used for preventing a FreeRTOS operating system and a Linux operating system from writing data at the same time, and the data area is used for data interaction between the FreeRTOS operating system and the Linux operating system;
D. outputting the image collected by the USB3.0 industrial camera 4 to a Linux operating system, putting the image into an image cache region according to the current writing address by the Linux operating system according to the frame, and recalculating a new current writing address, wherein the image cache region is an annular cache, namely when the new current writing address exceeds a preset boundary, taking the initial address of the image cache region as a new current writing address; the preset boundary is set according to the setting of the specific acquisition parameters of the USB3.0 industrial camera 4;
in this embodiment, the image buffer area includes a first-level buffer area and a second-level buffer area, both of which include a first read-write control area and a first data area, the first read-write control area of the first-level buffer area stores a current write address, a current read address, a mutex lock, and the like, the first data area of the first-level buffer area stores frame numbers, image information, and image data of images, the first read-write control area of the second-level buffer area stores a current write address, a current read address, and a processing state, the first data area of the second-level buffer area stores image data, the Linux operating system places images in the first-level buffer area by frames according to the current write address, and copies the image data at the current read address of the first-level buffer area to the current write address of the second-level buffer area when the current write address data of the second-level buffer area has been processed (i.e., read by the image processor), assigning a read address of the image processor, and opening a read enable signal; the image information refers to information such as image row size, column size and image size, and the image data refers to data actually contained in the image;
E. the image processor reads the image in the secondary cache region according to a read enable signal sent by a Linux operating system and a read address, the read data width is 8 bytes per beat, the image is processed according to frames, the processing result is put into an FIFO stack, the reading and processing processes of the image processor are concurrent execution processes, namely, the image is still read when being processed, and thus the timeliness of data processing can be ensured; the specific process of image processing is the prior art;
F. the image processor carries the processing result in the FIFO stack to a result cache area according to a write enable signal sent by a Linux operating system, sends a write completion interrupt signal to the Linux operating system after the carrying is completed, the Linux operating system sends the data in the result cache area to a Communication _ Thread after receiving the interrupt signal, the Communication _ Thread starts TCP jumbo frame transmission and transmits the data to a user, and the data transmission process is as follows: and after the group of data is sent, if the communication result replied by the user is successful, the next group of data is continuously sent, otherwise, the group of data is sent again, so that the transmission efficiency and reliability of the data can be ensured. The image buffer area and the result buffer area are also composed of DDR4 memory areas.
The above description is only a preferred embodiment of the present invention, and therefore should not be taken as limiting the scope of the invention, which is defined by the appended claims and their equivalents and modifications within the scope of the description.
Claims (10)
1. A heterogeneous multi-core image processing method is based on a Zynq UltraScale + MPSoC chip and is used for controlling a USB3.0 industrial camera to acquire images and processing the acquired images, and is characterized in that: the method comprises the following steps:
A. after receiving an acquisition instruction of a user, the Linux operating system writes the acquisition instruction into a class shared memory and triggers IPI to be interrupted to a FreeRTOS operating system, wherein the Linux operating system and the FreeRTOS operating system respectively run on four A53 hard cores and two R5 hard cores of a Zynq UltraScale + MPSoC chip;
B. the FreeRTOS operating system responds to the interruption in the step A, reads an acquisition instruction from the class shared memory and controls the USB3.0 industrial camera to acquire images according to the acquisition instruction;
C. outputting an image collected by the USB3.0 industrial camera to a Linux operating system, putting the image into an image cache region by frames according to a current writing address by the Linux operating system, recalculating a new current writing address, and taking an initial address of the image cache region as a new current writing address when the new current writing address exceeds a preset boundary;
D. the image processor reads the image in the image cache region according to a read enabling signal sent by a Linux operating system and a read address, processes the image according to frames, places the processing result into an FIFO stack, and the reading and processing processes of the image processor are concurrent execution processes, and the image processor runs in a PL unit of a ZynqUltraScale + MPSoC chip;
E. the image processor carries the processing result in the FIFO stack to a result cache region according to a write enable signal sent by the Linux operating system, the Linux operating system sends the data of the result cache region to a user, and the data transmission process is as follows: and taking the data with the set frame number and continuous frame number as a group of data, after the group of data is sent, if the communication result replied by the user is successful, continuing to send the next group of data, and otherwise, resending the group of data.
2. The heterogeneous multi-core image processing method according to claim 1, wherein: the image cache region in the step C comprises a first-level cache region and a second-level cache region, wherein the first-level cache region and the second-level cache region both comprise a first read-write control region and a first data region, the first data region of the first-level cache region stores the frame number, the image information and the image data of the image, the first read-write control region of the second-level cache region stores the processing state, the first data region stores the image data, the Linux operating system puts the image into the first-level cache region according to the current write address and copies the image data of the current read address of the first-level cache region to the current write address of the second-level cache region when the current write address data of the second-level cache region is processed, and the image processor reads the image data in the second-level cache region according to the read address.
3. The heterogeneous multi-core image processing method according to claim 1, wherein: the shared memory is composed of DDR4 memory areas which are not used by a Linux operating system and a FreeRTOS operating system.
4. The heterogeneous multi-core image processing method according to claim 1, 2 or 3, wherein: in the step a, after receiving the acquisition instruction of the user, the Linux operating system checks the acquisition instruction by using CRC16, if the check is correct, the frame type corresponding to the acquisition instruction is analyzed, and if the frame type is the same as the preset type, the acquisition instruction is written into the class shared memory.
5. The heterogeneous multi-core image processing method according to claim 3, wherein: the class shared memory comprises a second read-write control area and a second data area, wherein the second read-write control area is used for preventing a FreeRTOS operating system and a Linux operating system from writing data at the same time, and the data area is used for data interaction between the FreeRTOS operating system and the Linux operating system.
6. The heterogeneous multi-core image processing method according to claim 1, 2 or 3, wherein: before the step A, the method also comprises the following steps: after the Linux operating system is started, creating Communication _ Thread, Camera _ Thread, RpmsgCtrl _ Thread sub-threads:
in the Communication _ Thread of the sub-Thread, a special Communication protocol stack based on the gigabit giant frame is established for communicating with the user;
establishing communication connection with the USB3.0 industrial Camera in the child Thread Camera _ Thread, and configuring the working mode of the USB3.0 industrial Camera;
in the sub-Thread RpmsgCtrl _ Thread, an inter-core communication mechanism based on class shared memory + IPI interrupt is created for data communication with a FreeRTOS operating system, and a collection instruction is transmitted to the class shared memory.
7. The image processing apparatus based on the image processing method of any one of claims 1 to 6, wherein: the system comprises a master control core circuit, an acquisition control circuit and a communication transmission circuit, wherein the master control core circuit comprises a Zynq UltraScale + MPSoC chip, a DDR4 memory chip, a QSPI FLASH memory chip and an EMMC memory chip which are respectively connected with the Zynq UltraScale + MPSoC chip, the communication transmission circuit and the acquisition control circuit are respectively connected with the Zynq UltraScale + MPSoC chip, a user is connected with the communication transmission circuit, the acquisition control circuit is connected with the input end of a USB3.0 industrial camera, the output end of the USB3.0 industrial camera is connected with the communication transmission circuit, a Linux operating system, a FreeRTOS operating system and an image processor are operated on the Zynq UltraScale + MPSoC chip, an acquisition instruction input by the user is transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the acquisition instruction to the Linux RTOS operating system, the USB3.0 industrial camera is controlled by the FreeRTOS operating system according to acquire images, and the acquired images are transmitted to the Linux operating system through the communication transmission circuit, the Linux operating system shares the image to the image processor, the image processor processes the image and shares a processing result to the Linux operating system, and the Linux operating system sends the processing result to the user.
8. The image processing apparatus according to claim 7, characterized in that: the communication circuit comprises a USB ULPI transceiving unit connected with the output end of the USB3.0 industrial camera and a gigabit network PHY unit connected with a user.
9. The image processing apparatus according to claim 7, characterized in that: the acquisition control circuit comprises an encoder signal detection unit, an input IO circuit and an output IO circuit.
10. The image processing apparatus according to claim 7, 8 or 9, characterized in that: the model of the Zynq UltraScale + MPSoC chip is XCZU3EG-1SFVC784I of Zynq UltraScale + MPSoCs EG series, the model of the DDR4 memory chip is MT40A512M16GE, the model of the QSPI FLASH memory chip is MT25QU256ABA1EW9, and the model of the EMMC memory chip is MTFC8 GAKAJCN-4M.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210053900.7A CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210053900.7A CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114463163A true CN114463163A (en) | 2022-05-10 |
| CN114463163B CN114463163B (en) | 2024-08-13 |
Family
ID=81409948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210053900.7A Active CN114463163B (en) | 2022-01-18 | 2022-01-18 | Heterogeneous multi-core image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463163B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934631A (en) * | 2022-12-30 | 2023-04-07 | 武汉麓谷科技有限公司 | Intelligent storage platform based on MPSoC |
| CN116055653A (en) * | 2023-03-27 | 2023-05-02 | 南京芯驰半导体科技有限公司 | Image data processing method, device, electronic device and storage medium |
| WO2024108499A1 (en) * | 2022-11-24 | 2024-05-30 | 芯原微电子(上海)股份有限公司 | Image signal processing driving method, control module, processing module, and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107493419A (en) * | 2017-09-27 | 2017-12-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of integral miniaturized high-speed realtime graphic acquisition device |
| CN108566357A (en) * | 2017-12-21 | 2018-09-21 | 中国科学院西安光学精密机械研究所 | Image transmission and control system and method based on ZYNQ-7000 and FreeRTOS |
| US20190087359A1 (en) * | 2015-09-15 | 2019-03-21 | Gatekeeper Ltd. | System and method for securely connecting to a peripheral device |
| CN109558174A (en) * | 2018-11-28 | 2019-04-02 | 江苏艾萨克机器人股份有限公司 | RTOS-GPOS dual operating systems robot controller based on ZYNQ dual core processor |
| CN112527722A (en) * | 2020-12-09 | 2021-03-19 | 深圳市显控科技股份有限公司 | Heterogeneous multi-core embedded system and data interaction method thereof |
-
2022
- 2022-01-18 CN CN202210053900.7A patent/CN114463163B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190087359A1 (en) * | 2015-09-15 | 2019-03-21 | Gatekeeper Ltd. | System and method for securely connecting to a peripheral device |
| CN107493419A (en) * | 2017-09-27 | 2017-12-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of integral miniaturized high-speed realtime graphic acquisition device |
| CN108566357A (en) * | 2017-12-21 | 2018-09-21 | 中国科学院西安光学精密机械研究所 | Image transmission and control system and method based on ZYNQ-7000 and FreeRTOS |
| CN109558174A (en) * | 2018-11-28 | 2019-04-02 | 江苏艾萨克机器人股份有限公司 | RTOS-GPOS dual operating systems robot controller based on ZYNQ dual core processor |
| CN112527722A (en) * | 2020-12-09 | 2021-03-19 | 深圳市显控科技股份有限公司 | Heterogeneous multi-core embedded system and data interaction method thereof |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024108499A1 (en) * | 2022-11-24 | 2024-05-30 | 芯原微电子(上海)股份有限公司 | Image signal processing driving method, control module, processing module, and medium |
| CN115934631A (en) * | 2022-12-30 | 2023-04-07 | 武汉麓谷科技有限公司 | Intelligent storage platform based on MPSoC |
| CN115934631B (en) * | 2022-12-30 | 2023-10-27 | 武汉麓谷科技有限公司 | Intelligent storage platform based on MPSoC |
| CN116055653A (en) * | 2023-03-27 | 2023-05-02 | 南京芯驰半导体科技有限公司 | Image data processing method, device, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114463163B (en) | 2024-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114463163A (en) | Heterogeneous multi-core image processing method and device | |
| CN105474125B (en) | For tracking the technology for waking up locking and using | |
| CN108156181B (en) | Vulnerability detection method based on coroutine asynchronous IO and vulnerability scanning system thereof | |
| CN116541227B (en) | Fault diagnosis method and device, storage medium, electronic device and BMC chip | |
| CN109347884B (en) | Method and device for converting real-time Ethernet to field bus and storage medium | |
| CN113468097B (en) | Data exchange method based on system on chip | |
| US10152275B1 (en) | Reverse order submission for pointer rings | |
| CN112100090A (en) | Data access request processing method, device, medium and memory mapping controller | |
| CN114880259B (en) | Data processing method, device, system, electronic equipment and storage medium | |
| CN108984442A (en) | A kind of acceleration-controlled system based on Binarization methods, chip and robot | |
| WO2019173018A1 (en) | Non-intrusive on-chip debugger with remote protocol support | |
| US10088523B2 (en) | Debug adapter | |
| CN115599719A (en) | An FPGA-based FIFO interface multi-channel DMA controller | |
| CN108958903B (en) | Embedded multi-core central processor task scheduling method and device | |
| CN109992539B (en) | Double-host cooperative working device | |
| EP4439307A1 (en) | Log subsystem, log system, vehicle log system, and vehicle | |
| CN116483643A (en) | GPU debugging method, device, equipment and storage medium | |
| CN113010336A (en) | Application processor crash field debugging method and application processor | |
| CN116166582A (en) | Data reading and writing method, system, equipment and storage medium | |
| GB2423165A (en) | Host controller interface for packet-based timeshared bus | |
| CN109144844A (en) | Method for tracing, device, equipment and machine readable media | |
| US10803007B1 (en) | Reconfigurable instruction | |
| CN112084099A (en) | Method, device and equipment for obtaining alarm state value based on host and storage medium | |
| CN113721989A (en) | Multiprocessor parallel operation system and computer architecture | |
| CN118838859B (en) | Data transmission method, product, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |