CN117621145B - A flexible robotic arm system for fruit maturity detection based on FPGA - Google Patents

Abstract

The invention discloses a fruit maturity detection flexible mechanical arm system based on an FPGA, which comprises a flexible touch sensing mechanical arm module, a data preprocessing module and a fruit maturity detection classification module based on the FPGA; the flexible touch sensing mechanical arm module comprises a six-degree-of-freedom cooperative mechanical arm module, a flexible clamp holder module, a control center module and a flexible touch sensing unit module; the control center module comprises a control unit and a monitoring unit; the data preprocessing module comprises a principal component analysis module and a K-Means clustering algorithm module; the fruit maturity detection and classification module based on the FPGA mainly comprises an on-chip cache module, a DMA module, a ResNet accelerator module, a DDR external storage module, an AXI bus module and an ARM processor module; the ResNet accelerator module includes a convolution layer, a pooling layer, a batch normalization layer, a ResNet network model layer, and a Dropout layer. Compared with machine vision and manual classification, the fruit classification line has the advantages of high efficiency, high precision, low power consumption and the like, and is more suitable for being applied to the fruit classification line.

Description

A flexible robotic arm system for fruit maturity detection based on FPGA

Technical Field

The invention relates to the technical field of flexible tactile perception and fruit maturity detection, and in particular to a fruit maturity detection flexible mechanical arm system based on FPGA.

Background Art

At present, traditional fruit maturity detection technology mainly relies on experience or the use of tools such as sugar detection analyzers, acidity detection analyzers, hardness testers, etc. to detect and judge the appearance, soluble sugar content, acidity, hardness or soluble solids of fruits, and then determine the maturity level of fruits according to relevant standards. According to these standards, fruit maturity assessment is a laborious and time-consuming process, large-scale real-time detection is not possible, and fruits that pass destructive detection cannot be eaten or sold. Machine learning and various non-destructive testing methods have been used to predict and classify the freshness of fruits. Although these technologies can accurately assess the maturity of fruits, they often require high measurement environments, complex equipment, massive amounts of data, and a large amount of processing, making automatic detection and intelligent grading of fruit maturity challenging.

The basic component of the robotic arm system is the tactile sensor, which enables the robotic arm to interact with the environment accurately, quickly and safely. However, the application and development of tactile sensing are constrained by the rigidity, difficulty in bending and high robustness of existing sensors, which are usually made of silicon or other rigid semiconductor materials. Rigid sensors, especially those used in agricultural applications, have difficulty adapting to the complex curved surfaces of fruits and are prone to damaging soft and delicate fruits. Due to the large amount of computation required by neural network algorithms, general-purpose processors are no longer suitable for computationally intensive tasks. FPGAs are good at matrix operations and are in line with the parallel thinking of neural networks, especially algorithm models represented by convolutional neural networks.

Summary of the invention

In view of the above-mentioned technical deficiencies, the present invention provides a fruit maturity detection flexible robotic arm system based on FPGA, which greatly improves the detection speed of fruit maturity and effectively solves the problem of fruit loss.

In order to solve the above technical problems, the present invention adopts the following technical solutions: a flexible mechanical arm system for fruit maturity detection based on FPGA, the system comprises a flexible tactile sensing mechanical arm module, a data preprocessing module, and a fruit maturity detection and classification module based on FPGA;

The flexible tactile sensing mechanical arm module comprises a six-degree-of-freedom collaborative mechanical arm module, a flexible gripper module, a control center module and a flexible tactile sensing unit module; the six-degree-of-freedom collaborative mechanical arm module is used as the supporting part and the overall framework of the intelligent mechanical arm system, and is mainly used to complete the grading action of the fruit, and the flexible gripper module is used to clamp and release the fruit; the control center module is used to control the clamping movement of the flexible gripper, the grading movement of the mechanical arm and the clamping force of the fruit, and the flexible tactile sensing unit is used to sense the hardness of the fruit in real time when the flexible gripper clamps the fruit; the control center module comprises a control unit and a monitoring unit, and the control unit transmits information through the information transmission layer to command the movement of the mechanical arm and the clamping and releasing of the fruit by the gripper, and the monitoring unit continuously tracks the angular position and posture of the mechanical arm so as to intelligently control its operation and issue a fault alarm;

The data preprocessing module includes a principal component analysis module and a K-Means clustering algorithm module. The principal component analysis module is used to reduce the dimension of the original data. The K-Means clustering algorithm module is used to cluster the hardness data set after the dimension reduction by the principal component analysis module, increase the effective features of the tactile perception capture data, and improve the classification accuracy of multiple types of targets;

The FPGA-based fruit maturity detection and classification module mainly includes an on-chip cache module, a DMA module, a ResNet10 accelerator module, a DDR external storage module, an AXI bus module, and an ARM processor module; the ResNet10 accelerator module includes a convolution layer, a pooling layer, a batch normalization layer, a ResNet network model layer, and a Dropout layer, which are used to output target classification, obtain the maturity of fruits at different stages, and feed back the data to a control unit, which executes commands according to the feedback data to direct the movement of the robotic arm.

The six-degree-of-freedom collaborative robot arm is made of soft silicone. The flexible gripper is composed of four flexible fingers and is operated by a pneumatic valve, so that it adheres to the surface of the fruit while grasping it, achieving non-destructive grasping of the fruit; the perception and execution layer of the gripper is mainly controlled by a digital servo, and the power transmission is achieved through the engagement of the servo and the mechanical clamp, which has the advantages of fast response speed, strong controllability, high precision, and high torque. Each inner surface of the flexible finger is attached with a flexible tactile sensor, and the four flexible fingers form a three-dimensional arrangement of four sensors, which can evenly measure the hardness of four positions of the fruit.

The flexible tactile sensor uses a piezoresistive flexible film sensor, which has the advantages of close-range measurement, simple structure, wide measurement range, and low cost. The general characteristics of the flexible sensor can fully contact and sense the fruit and provide more accurate measurement data. The flexible tactile sensing unit measures the hardness of the fruit through a flexible tactile sensor. When grabbing the fruit, it senses the hardness of the fruit at different maturity stages and feeds back the pressure to the information transmission layer. According to the information of the information transmission layer, the FPGA-based fruit maturity detection and classification module classifies the fruits at different maturity stages.

The control center mainly implements the control and execution layers based on the information of the information transmission layer to realize the grabbing and releasing of the manipulator. The information transmission layer performs terminal control on the manipulator, can control the state of the manipulator in real time, and perform intelligent control, fault alarm, etc. The flexible sensing unit receives the pressure information of the fruit grabbed by the manipulator through the sensor in the execution layer, and obtains the pressure values of different maturity levels.

The mapping relationship between firmness and maturity can be used to predict and evaluate the maturity of fruits.

The linear relationship between the pressure value and the voltage can realize the corresponding relationship between the voltage and the measured pressure.

The clamp structure design adopts a mixture of bionic finger type and bite type, and the mixture of the two ensures the stability of clamping, and can more reliably measure the hardness value of a certain point of the fruit.

The flexible sensor is fixed to the black claw-shaped rubber of the manipulator by double-sided tape, which is convenient for replacing sensors of different sizes and categories and cleaning the manipulator.

The system is used for maturity grading in sorting lines, positioning packaging production lines or maturity distribution centers, rather than outdoor fruit picking. The system is tested and evaluated using fruits of different maturity levels, and the hardness data of fruits of different maturity levels are collected. The hardness data is used to train the ResNet10 module to correctly identify the maturity of the fruit;

The FPGA-based fruit maturity detection and classification module completes the sensing of the sensing execution layer by the flexible pressure sensor and transmits the sensing information of the sensing layer to the information transmission layer. The FPGA control element integrates multiple sensors and can simultaneously measure the hardness values of multiple fruit points.

The on-chip cache module provides cache space for intermediate variables generated during the operation of the ResNet10 model accelerator module, and enables multiple modules in the ResNet10 model accelerator module to work under different cache modules respectively, which can ensure that the data generated by each module during operation do not interfere with each other, and avoid the ResNet10 model accelerator module from frequently accessing the DDR external storage module, thereby improving the speed of data access and achieving further acceleration.

The DMA module receives instructions issued by the ARM processor module via the AXI bus.

The FPGA-based fruit maturity detection and classification module mainly includes an on-chip cache module, a DMA module, a ResNet10 model accelerator module, a DDR external storage module, an AXI bus module, and an ARM processor module; the ResNet10 accelerator module includes a convolution layer, a pooling layer, a batch normalization layer, a ResNet network model layer, and a Dropout layer, which are used to output target classification, obtain the maturity of fruits at different stages, and feed the data back to the control unit; the main function of the batch normalization layer is to standardize data, improve the generalization ability of the training network, and avoid the influence of singular data on the model; the The pooling layer is an important component of the classification module, which can reduce the scale of the convolutional neural network model, improve the model calculation speed, and improve the robustness of feature extraction. Its working method is to gradually reduce the size of the represented space; the pooling layer uses the maximum pooling layer to reduce the size of the tactile map; the ResNet network model is composed of superimposed ResNet blocks; compared with the traditional neural network, the ResNet network has an additional direct channel, which can skip the intermediate layer and directly reach the state before output; a dropout layer is added between two ResNet blocks to prevent the problem of good training effect but poor test effect, that is, the problem of overfitting;

The AXI bus module is a high-performance, high-bandwidth, low-latency on-chip bus, which is divided into three interface protocols: AXI4-Lite, AXI4, and AXI4-Stream. The ARM processor controls the DMA module, ResNet10 model accelerator module, and on-chip cache module through the AXI4-Lite bus, and the fruit hardness data is sent from the DMA module to the DDR external storage module through the AXI-Stream bus.

The ARM processor module is a control module for the operation of the overall method, and is internally controlled by a state machine.

The beneficial effects of the present invention are:

1. The low Young's modulus of the flexible sensor enables it to adapt to soft materials and complex environments, and has significant advantages in non-destructive bending fruit grasping. By integrating flexible sensing technology into the field of agricultural product quality control, the quality and safety of agricultural products can be improved. Especially in the field of fruit quality detection, the application of flexible sensors can significantly reduce the loss rate of fruits and ensure food quality and safety.

2. The most common classification technology is based on machine vision and manual maturity classification. Machine vision technology requires a high-light environment, and the classification effect is unstable due to changes in factors such as daylight and distance. The disadvantages of manual grading are more obvious, and the grading standards will not only change due to personal differences, working hours, and fatigue. The present invention is based on fruit hardness data, which is sensed by a flexible sensing unit during the robot grasping process. Therefore, daylight and distance have no effect on the operation of the system. In addition, the present invention also has the advantages of high efficiency, high precision, and high stability. Compared with machine vision and manual grading, it is more suitable for application in fruit grading lines.

3. The 32×32 tactile map data were processed by the K-means clustering method and input into the ResNet10 model, which improved the target classification effect. Secondly, FPGA is rich in resources, and various combinations of internal logic units, multipliers and RAM can efficiently implement a variety of complex algorithms. As a hardware device, FPGA is different from the von Neumann CPU architecture. It does not require instruction fetching and decoding, and can implement algorithms in parallel. GPU also belongs to the von Neumann architecture, but the power consumption of GPU is higher than that of FPGA. In addition, FPGA has the characteristics of parallel computing, low power consumption and reconfigurability, which has great advantages in deploying convolutional deep learning algorithms.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG. 1 is a flow chart of the system modules of the present invention.

FIG. 2 is a diagram of a pressure value measured by a sensor of the present invention and a voltage conversion module.

FIG3 is a flow chart of the K-Means clustering algorithm of the present invention.

FIG. 4 is a framework diagram based on FPGA of the present invention.

FIG5 is a diagram of the ResNet10 model of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Example

The present invention provides a fruit maturity detection flexible robotic arm system based on FPGA according to an embodiment of the present invention, which specifically relates to the technical field of flexible tactile perception and FPGA-accelerated fruit maturity detection algorithm, including a flexible tactile perception robotic arm module, a data preprocessing module, and an FPGA-based fruit maturity detection classification module, as shown in FIG1 .

The FPGA-based fruit maturity detection flexible robotic arm system mainly comprises the following steps:

S1: When grasping the fruit, the flexible tactile sensing unit senses the hardness of the fruit at different maturity stages, feeds back the pressure data to the information transmission layer, and generates hardness data;

S2: preprocess the hardness data;

S3: The pre-processed data passes through the fruit maturity detection and classification module of the FPGA and serves as the input data of the ResNet10 accelerator module. It is classified by the trained ResNet10 model to obtain the maturity of fruits at different stages, and the data is fed back to the control unit.

S4: The control unit directs the movement of the robotic arm and the gripping and releasing of the fruit by the gripper through the information transmitted by the information transmission layer. At the same time, the monitoring unit continuously tracks the angular position and posture of the robotic arm in order to intelligently control its operation and issue fault alarms.

The data preprocessing consists of two modules: principal component analysis module and K-Means clustering algorithm, and mainly includes the following steps:

S1: Assuming that a fruits of different maturity are grasped twice, each finger of the robot will obtain 2a hardness information, then the four flexible fingers will finally obtain a 2a×4 perception hardness data set, so the original data is a four-dimensional array, and the number of samples n is 2a. The dimension of the data set is reduced through principal component analysis and most of the information is retained. The specific form of the original data is as follows:

The main steps to reduce the dimensionality of raw data using principal component analysis are as follows:

S11: Average value Subtracting the mean from the original data yields X′ _i = Xi _{- Mi} and a data matrix X′ = {X′ ₁ , X′ ₂ , X′ ₃ , X′ ₄ } ^T with a mean of zero. The formula for calculating the covariance of the new data matrix (X′) is:

S12: Calculate the eigenvalues and eigenvectors of the covariance matrix and diagonalize the covariance matrix. The main elements of the diagonalized matrix are the eigenvalues of the covariance matrix, and the eigenvalues are sorted from large to small: |λ ₁ |≥|λ ₂ |≥|λ ₃ |≥|λ ₄ |

S13: Select m principal components and calculate the variance contribution rate:

S14: The M eigenvectors corresponding to the M eigenvalues form a new eigenvector matrix P, which transforms the data into a new space constructed by the K eigenvectors to complete data dimensionality reduction. Since the variance contribution rate of the first two principal components is very high, the eigenvectors (P) of the first two principal components are selected to construct a new data set, and the original data set is simplified to two dimensions, thereby obtaining a hardness data set after principal component analysis dimensionality reduction.

X ₁ = PX′ (6)

S2: Use the classic K-means clustering algorithm to cluster the hardness data set after the principal component analysis dimension reduction, and classify the targets after clustering to increase the effective features of the tactile perception capture data and improve the classification accuracy of multiple types of targets. The basic process of the K-means clustering algorithm is shown in Figure 3, which mainly includes the following steps:

S21: Randomly select K centroid points, and calculate the category to which each sample belongs according to the minimum distance principle, that is, the shortest distance between the sample and the K centroid points; after multiple iterations, determine whether the position of the centroid point has not changed or has only changed a little, that is, whether the distance between the centroid points of the previous generation and the next generation has converged. If it has not converged, the centroid iteration will continue to cycle and re-cluster. If it converges, the iteration ends and the clustering of the K-type target is achieved.

S22: Introduce the distortion function J, as shown in formula (5). The distortion function represents the sum of the squares of the distances between each sample and its corresponding centroid. When the distortion function reaches the minimum value, the clustering effect achieves the best result.

In the formula, represents the cluster centroid x ⁽ⁱ⁾ closest to the sampling point, c ⁽ⁱ⁾ represents The purpose of the K-means algorithm is to find the smallest c ^{(i) and} Make the distortion function J reach the minimum value.

The present invention adopts the ZYNQ 7020 FPGA chip produced by Xilinx as the core of the fruit maturity detection and classification module.

The FPGA-based fruit maturity detection and classification module mainly comprises the following steps:

S1: The preprocessed data CAM_DATA is sent to the DDR external storage module via the DMA module and the AXI bus module; the on-chip cache module provides cache space for the intermediate variables generated during the operation of the ResNet10 accelerator module.

S2: The DMA module receives instructions from the ARM processor module via the AXI bus. When DATA_WRITE=1, the DMA module sends the data to the DDR external memory module. When the ARM processor module sends the instruction DATA_READ=1, the DMA module sends the data stored in the DDR external memory module to the ResNet10 accelerator module for calculation.

S3: When the ResNet10 accelerator module receives the instruction TASK_START=1 sent by the ARM processor module, it performs convolution, pooling, and activation operations on the data sent to the DMA module. After the calculation is completed, the ResNet10 accelerator module sends a TASK_DONE=1 signal to the ARM processor module through the AXI bus module. After receiving the TASK_DONE=1 signal, the ARM processor module sends a DATA_TRAN=1 signal to the DMA module through the AXI bus module, so that the DMA module sends the calculation result of the ResNet10 accelerator module to the DDR external memory module;

S4: The ARM processor controls the DMA module, ResNet10 accelerator module, and on-chip cache module through the AXI4-Lite bus.

The ResNet10 module mainly includes the following steps:

S1: The preprocessed data is converted into a 32×32 tactile map as the input of the ResNet10 model.

S2: The input data passes through a convolutional layer, a batch normalization layer, a maximum pooling layer, two ResNet module layers, and a fully connected layer to output the target type. The input data is convolved with the filter kernel in the convolutional layer, and the convolution process is shown in formula (6).

In the formula represents the weight, represents the offset of the i-th filter in the l-th layer, x ^l (j) represents the j-th local area in the l-th layer, and * is used to calculate the dot product of the kernel and the local area. The input of the convolutional layer is the tactile map reflecting different target pressure data.

Then the convolutional data is passed through the batch normalization layer to standardize the data, improve the generalization ability of the training network and avoid the influence of singular data on the model. The batch normalized data uses the ReLU function to speed up the model training, and then gradually reduces the size of the represented space through the pooling layer. In the maximum pooling, the maximum element in each pooling area is selected and defined by formula (7).

where _pk,(i,j) is the output of the pooling operator associated with the kth feature map, _Rk,(p,q) is the element at position (p,q) in the pooling region, and Q(i,j) represents the pooling region around (i,j).

S3: The final fully connected layer outputs the target classification, thereby obtaining the maturity of fruits at different stages and feeding the data back to the control unit;

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (4)

1. A flexible mechanical arm system for detecting fruit maturity based on FPGA, characterized in that: the system comprises a flexible tactile sensing mechanical arm module, a data preprocessing module, and a fruit maturity detection and classification module based on FPGA; the flexible tactile sensing mechanical arm module comprises a six-degree-of-freedom collaborative mechanical arm module, a flexible clamp module, a control center module, and a flexible tactile sensing unit module; the six-degree-of-freedom collaborative mechanical arm module is used as a supporting part and an overall framework of an intelligent mechanical arm system to complete the grading action of fruits, and the flexible clamp module is used to clamp and release fruits; the control center module is used to control the clamping movement of the flexible clamp, the grading movement of the mechanical arm, and the clamping force of the fruit, and the flexible tactile sensing unit is used to perceive the hardness of the fruit in real time when the flexible clamp clamps the fruit; the control center module comprises a control unit and a monitoring unit, the control unit transmits information through an information transmission layer to command the movement of the mechanical arm and the clamping and release of the fruit by the clamp, and the monitoring unit continuously tracks the angular position and posture of the mechanical arm so as to intelligently control its operation and issue a fault alarm; The data preprocessing module includes a principal component analysis module and a K-Means clustering algorithm module. The principal component analysis module is used to reduce the dimension of the original data. The K-Means clustering algorithm module is used to cluster the hardness data set after the dimension reduction by the principal component analysis module, increase the effective features of the tactile perception capture data, and improve the classification accuracy of multiple types of targets. The module includes the following steps: S1: Assuming that a fruits of different maturity are grasped twice, each finger of the robot will obtain 2a hardness information, then the four flexible fingers will finally obtain a 2a×4 perception hardness data set. Therefore, the original data is a four-dimensional array, and the number of samples n is 2a. The principal component analysis is used to reduce the dimension of the data set and retain most of the information. The specific form of the original data is as follows: The steps to reduce the dimensionality of the original data using principal component analysis are as follows: S11: Average value Subtracting the mean from the original data yields Xi′=Xi-Mi and a data matrix X′={X ₁ ′,X ₂ ′,X ₃ ′,X ₄ ′} ^T with a mean of zero. The formula for calculating the covariance of the new data matrix (X′) is: S12: Calculate the eigenvalues and eigenvectors of the covariance matrix, and diagonalize the covariance matrix. The elements of the diagonalized matrix are the eigenvalues of the covariance matrix. The eigenvalues are sorted from large to small: |λ ₁ |≥|λ ₂ |≥|λ ₃ |≥|λ ₄ |; S13: Select m principal components and calculate the variance contribution rate: S14: The M eigenvectors corresponding to the M eigenvalues form a new eigenvector matrix P, which converts the data into a new space constructed by the K eigenvectors to complete data dimensionality reduction. Since the variance contribution rate of the first two principal components is very high, the eigenvectors P of the first two principal components are selected to construct a new data set, and the original data set is simplified to two dimensions, thereby obtaining a hardness data set after PCA dimensionality reduction. _X1 = PX′ S2: The hardness data set after the principal component analysis dimension reduction is clustered using the K-means clustering algorithm. After clustering, the targets are classified to increase the effective features of the tactile perception capture data and improve the classification accuracy of multiple types of targets. The process of the K-means clustering algorithm includes the following steps: S21: Randomly select K centroid points, and calculate the category to which each sample belongs according to the minimum distance principle, that is, the shortest distance between the sample and the K centroid points; after multiple iterations, determine whether the distance between the centroid points of the previous generation and the next generation converges. If it does not converge, the centroid iteration will continue to cycle and re-cluster. If it converges, the iteration ends and the clustering of the K-type target is achieved; S22: Introduce the distortion function J, which represents the sum of the squares of the distances between each sample and its corresponding centroid. When the distortion function reaches its minimum value, the clustering effect achieves the best result. In the formula, represents the cluster centroid x ⁽ⁱ⁾ closest to the sampling point, c ⁽ⁱ⁾ represents The distance between c (i) and x ⁽ⁱ⁾ . The purpose of the K-means algorithm is to find the smallest c ⁽ⁱ⁾ and Make the distortion function J reach the minimum value; The FPGA-based fruit maturity detection and classification module comprises an on-chip cache module, a DMA module, a ResNet10 accelerator module, a DDR external storage module, an AXI bus module, and an ARM processor module; the ResNet10 accelerator module comprises a convolution layer, a pooling layer, a batch normalization layer, a ResNet network model layer, and a Dropout layer, which are used to output target classification, obtain the maturity of fruits at different stages, and feed the data back to the control unit; the pooling layer is an important component of the classification module, which can reduce the scale of the convolutional neural network model, improve the model calculation speed, and improve the robustness of feature extraction; its working method is to gradually reduce the size of the represented space; the pooling layer adopts the maximum pooling layer to reduce the size of the tactile map; the ResNet network model layer is composed of superimposed ResNet blocks; compared with the traditional neural network, the ResNet network has an additional direct channel, which can skip the intermediate layer and directly reach the state before output; a Dropout layer is added between two ResNet blocks to prevent the problem of good training effect but poor test effect, that is, the problem of overfitting. The on-chip cache module provides cache space for the intermediate variables generated during the operation of the ResNet10 accelerator module, and enables multiple modules in the ResNet10 accelerator module to work under different cache modules respectively, which can ensure that the data generated by each module during operation do not interfere with each other, and avoid the frequent access of the ResNet10 accelerator module to the DDR external storage module, thereby improving the speed of data access and achieving further acceleration; the DMA module receives instructions issued by the ARM processor module via the AXI bus; the AXI bus module is a high-performance, high-bandwidth, low-latency on-chip bus, which is divided into three interface protocols: AXI4-Lite, AXI4, and AXI4-Stream. The ARM processor controls the DMA module, the ResNet10 model accelerator module, and the on-chip cache module through the AXI4-Lite bus, and the fruit hardness data is sent from the DMA module to the DDR external storage module through the AXI-Stream bus; the ARM processor module is the control module for the operation of the overall method, and a state machine is used to control it internally; The FPGA-based fruit maturity detection and classification module comprises the following steps: S1: The preprocessed data CAM_DATA is sent to the DDR external storage module via the DMA module and the AXI bus module; the on-chip cache module provides cache space for the intermediate variables generated during the operation of the ResNet10 accelerator module; S2: The DMA module receives instructions from the ARM processor module via the AXI bus. When DATA_WRITE=1, the DMA module sends the data to the DDR external memory module. When the ARM processor module sends the instruction DATA_READ=1, the DMA module sends the data stored in the DDR external memory module to the ResNet10 accelerator module for calculation. S3: When the ResNet10 accelerator module receives the instruction TASK_START=1 sent by the ARM processor module, it performs convolution, pooling, and activation operations on the data sent to the DMA module. After the calculation is completed, the ResNet10 accelerator module sends a TASK_DONE=1 signal to the ARM processor module through the AXI bus module. After receiving the TASK_DONE=1 signal, the ARM processor module sends a DATA_TRAN=1 signal to the DMA module through the AXI bus module, so that the DMA module sends the calculation result of the ResNet10 accelerator module to the DDR external memory module; S4: The ARM processor controls the DMA module, ResNet10 accelerator module, and on-chip cache module through the AXI4-Lite bus; The ResNet10 accelerator module includes the following steps: S1: Convert the preprocessed data into a 32×32 tactile map as the input of the ResNet10 accelerator module; S2: The input data passes through a convolutional layer, a batch normalization layer, a max pooling layer, two ResNet block layers, and a fully connected layer to output the target type; the input data is convolved with the filter kernel in the convolutional layer, and the convolution process is shown in formula (6); In the formula represents the weight, represents the offset of the i-th filter in the l-th layer, x ^l (j) represents the j-th local area in the l-th layer, and * is used to calculate the dot product of the kernel and the local area; the input of the convolutional layer is the tactile map reflecting the pressure data of different targets; Then, the convolutional data is standardized through a batch normalization layer to improve the generalization ability of the training network and avoid the influence of singular data on the model. The batch normalized data uses the ReLU function to speed up the model training, and then the pooling layer is used to gradually reduce the size of the represented space. In the maximum pooling, the maximum element in each pooling area is selected and defined by formula (7). Where _pk,(i,j) is the output of the pooling operator associated with the kth feature map, _Rk,(p,q) is the element at position (p,q) in the pooling region, and Q(i,j) represents the pooling region around (i,j); S3: The final fully connected layer outputs the target classification, thereby obtaining the maturity of fruits at different stages and feeding the data back to the control unit; The system is tested and evaluated using fruits of different maturity levels, and the hardness data of fruits of different maturity levels are collected. The hardness data is used to train the ResNet10 accelerator module to correctly identify the maturity of the fruit; The six-degree-of-freedom collaborative robot arm is made of soft silicone material; the flexible gripper is composed of four flexible fingers and is operated by a pneumatic valve, so that it adheres to the surface of the fruit while grasping, thereby achieving non-destructive grasping of the fruit; the perception and execution layers of the flexible gripper are controlled by a digital servo, and power transmission is achieved through the engagement of the servo and the mechanical clamp; each inner surface of the flexible finger is attached with a flexible tactile sensor, and the four flexible fingers form a three-dimensional arrangement of four sensors, which can evenly measure the hardness of four positions of the fruit; The control center implements the control and execution layers according to the information of the information transmission layer, and realizes the grasping and releasing of the manipulator; the information transmission layer performs terminal control on the manipulator, can control the state of the manipulator in real time, and perform intelligent control and fault alarm; the flexible tactile sensing unit receives the pressure information of the fruit grasped by the manipulator through the flexible tactile sensor in the execution layer, and obtains the pressure values of different maturity levels; The flexible tactile sensing unit measures the hardness of the fruit through a flexible tactile sensor. When grabbing the fruit, it senses the hardness of the fruit at different maturity stages and feeds back the pressure to the information transmission layer. The flexible tactile sensor uses a piezoresistive flexible film sensor, which has the advantages of close-range measurement, simple structure, wide measurement range and low cost. The FPGA-based fruit maturity detection and classification module completes the sensing of the execution layer by the flexible tactile sensor, and transmits the sensing information of the sensing layer to the information transmission layer; the FPGA control element integrates multiple flexible tactile sensors, which can measure the hardness values of multiple points of the fruit in real time. 2. According to the FPGA-based fruit maturity detection flexible robotic arm system of claim 1, it is characterized in that: the mapping relationship between the hardness and maturity can predict and evaluate the maturity of the fruit; the linear relationship between the pressure value and the voltage can realize the corresponding relationship between the voltage and the measured pressure. 3. According to claim 2, a FPGA-based fruit maturity detection flexible robotic arm system is characterized in that: the clamp structure design adopts a mixture of bionic finger type and bite type, the mixture of the two ensures the stability of clamping, and can more reliably measure the hardness value of a certain point of the fruit; the flexible tactile sensor is fixed on the black claw-shaped rubber of the manipulator by double-sided tape, which is convenient for replacing flexible tactile sensors of different sizes and categories and cleaning manipulators. 4. The FPGA-based fruit maturity detection flexible robotic arm system according to claim 3 is characterized in that the system is used in a sorting line, a positioning packaging production line or a maturity distribution center for maturity grading, rather than outdoor fruit picking.