CN118917976A

CN118917976A - First-aid effect analysis method and system based on dummy

Info

Publication number: CN118917976A
Application number: CN202411103100.7A
Authority: CN
Inventors: 曹月; 孔凡; 清续伟
Original assignee: Xiangle Interactive Tianjin Technology Co ltd
Current assignee: Xiangle Interactive Tianjin Technology Co ltd
Priority date: 2024-08-13
Filing date: 2024-08-13
Publication date: 2024-11-08
Anticipated expiration: 2044-08-13
Also published as: CN118917976B

Abstract

The present invention provides a method and system for analyzing the effectiveness of first aid based on a simulated person, and relates to the technical field of simulation analysis. The method and system include using a three-dimensional reconstruction algorithm to generate a triangular patch mesh model of soft tissue, tetrahedralizing the triangular patch mesh model of the soft tissue, and generating a discretized model suitable for first aid simulation; using a collision detection algorithm to detect the contact state between a user's hand and the first aid simulation discretized model in real time, modeling a virtual finger as a particle-spring system with stiffness and damping, and using a position-based dynamics method to calculate multi-point tactile feedback force in real time according to the contact state between the virtual finger and the soft tissue surface. When a collision event is detected, the collision force is applied to the first aid simulation discretized model as a boundary condition to drive the soft tissue to deform; extracting the operation features corresponding to the user's hand when a collision event occurs, combining with a preset operation evaluation rule library, and determining the first aid operation quality score through a preset quality evaluation model.

Description

First-aid effect analysis method and system based on dummy

Technical Field

The invention relates to a simulation analysis technology, in particular to a first-aid effect analysis method and system based on simulation people.

Background

The traditional first-aid training mainly depends on theoretical learning and simulated human training, and has the problems of high training cost, poor interactivity, difficult objective evaluation and the like.

At present, the virtual reality technology has been applied to the field of medical education, such as a surgery simulation training system, an anatomic visualization platform and the like. The system provides immersive learning experience by constructing a realistic virtual scene and an interactive mode, and improves learning efficiency and enthusiasm. However, existing virtual reality medical educational systems focus mainly on the flow training of specific medical operations, lacking assessment and feedback mechanisms for the learner's comprehensive abilities.

In the field of emergency training, a simulator is a common teaching tool, and provides a training environment close to reality for students by simulating physiological characteristics and pathological reactions of human bodies. However, the traditional simulators lack intelligent interaction and feedback capability, and objective evaluation and guidance cannot be given according to the performance of students. In addition, the use cost of the simulator is high, maintenance and updating are difficult, and popularization and application of the simulator in emergency training are limited.

Disclosure of Invention

The embodiment of the invention provides a first-aid effect analysis method and a first-aid effect analysis system based on a simulator, which can solve the problems in the prior art.

In a first aspect of an embodiment of the present invention,

A method of providing a human-based first aid success analysis, comprising:

Collecting tomographic data of key parts of a human body by using medical image equipment, extracting outline information of soft tissues from the tomographic data by using a medical image processing technology, generating a triangular patch grid model of the soft tissues by using a three-dimensional reconstruction algorithm, carrying out tetrahedral subdivision on the triangular patch grid model of the soft tissues, and generating a discretization model suitable for emergency simulation by adopting a self-adaptive grid encryption strategy and a grid smoothing algorithm;

Adopting a collision detection algorithm to detect the contact state between the hand of a user and the emergency simulation discretization model in real time, modeling a virtual finger as a mass point-spring system with rigidity and damping, adopting a position-based dynamic method to calculate a multipoint tactile feedback force in real time according to the contact state between the virtual finger and the surface of soft tissues, and applying the collision force to the emergency simulation discretization model as a boundary condition when a collision event is detected to drive the soft tissues to deform;

extracting operation characteristics corresponding to the hands of a user when a collision event occurs, wherein the operation characteristics comprise an operation position, an operation speed, an operation force and an operation track, and determining an emergency operation quality score through a preset quality evaluation model by combining a preset operation evaluation rule base; and generating personalized operation prompts and improvement strategies in a self-adaptive mode according to the emergency operation quality scores, wherein the preset quality assessment model is constructed based on a machine learning algorithm.

In an alternative embodiment of the present invention,

The method further comprises the steps of:

According to the mechanical characteristics of nonlinearity, anisotropy and incompressibility of human soft tissues, a high-order superelasticity constitutive model is adopted to set material parameters of the emergency simulation discretization model, a nonlinear control equation of soft tissue deformation is established, and a finite element method is utilized to carry out space discretization on the nonlinear control equation, so that deformation response of the emergency simulation discretization model under the action of external force is obtained.

In an alternative embodiment of the present invention,

Collecting tomographic data of key parts of a human body by using medical image equipment, extracting outline information of soft tissues from the tomographic data by using a medical image processing technology, generating a triangular patch grid model of the soft tissues by using a three-dimensional reconstruction algorithm, performing tetrahedral subdivision on the triangular patch grid model of the soft tissues, and generating a discretization model suitable for emergency simulation by adopting a self-adaptive grid encryption strategy and a grid smoothing algorithm, wherein the generation of the discretization model comprises the following steps:

Performing tomographic scanning on the target organ by using medical imaging equipment to obtain a series of two-dimensional tomographic images;

preprocessing and segmenting the two-dimensional tomographic image, improving the contrast and the signal-to-noise ratio of the image by adopting an image enhancement technology, and extracting the two-dimensional contour information of the region of interest of the soft tissue by combining a threshold segmentation algorithm;

performing triangular surface patch on the contours in the adjacent tomographic images by adopting Marching Cubes algorithm according to the two-dimensional contour information to generate a triangular surface patch grid model of the soft tissue;

optimizing and subdividing the triangular patch grid model, and eliminating irregular boundaries on the surface of the triangular patch grid model by adopting a Laplace smoothing algorithm; converting the optimized triangular patch grid model into a volume grid model, and generating a tetrahedral grid in the soft tissue according to the shape and topology of the volume grid model by adopting a tetrahedral subdivision algorithm to serve as a first-aid simulation discretization model.

In an alternative embodiment of the present invention,

Adopting a collision detection algorithm to detect the contact state between the hand of the user and the emergency simulation discretization model in real time, modeling the virtual finger as a mass-spring system with rigidity and damping, according to the contact state of the virtual finger and the soft tissue surface, calculating the multipoint tactile feedback force in real time by adopting a dynamic method based on the position comprises the following steps:

Acquiring three-dimensional position and posture information of a user hand in real time through a motion capture device, and obtaining motion data of a virtual finger; dividing virtual fingers into bounding box layers, constructing a bounding box tree, traversing the bounding box tree to exclude areas which are unlikely to collide, and reducing the calculation range of collision detection to local leaf nodes;

Pre-calculating and storing a distance field on the soft tissue surface to obtain a distance field voxel grid, mapping the vertexes of the virtual finger into the distance field voxel grid during collision detection, obtaining the directed distance from the vertexes to the distance field voxel grid through tri-linear interpolation, and judging that collision occurs if the distance is smaller than a given distance threshold;

the virtual finger is abstracted into a discretization model consisting of mass points and springs, and virtual fingers with different materials and shapes are simulated by adjusting the mass of the mass points, the rigidity and damping coefficient of the springs;

After collision is detected, a dynamic method based on position is adopted, and the position of the mass point of the virtual finger is iteratively optimized to meet collision constraint and spring constraint, so that the haptic feedback force at each contact point is calculated;

The magnitude and direction of the friction force at each contact point is calculated according to coulomb friction law, the sliding and viscous effects of the finger and the soft tissue surface are simulated, and the surface normal and curvature changes caused by soft tissue deformation are included in the calculation of the feedback force.

In an alternative embodiment of the present invention,

By iteratively optimizing the position of the mass points of the virtual finger to satisfy the collision constraint and the spring constraint using a position-based dynamics approach, calculating the haptic feedback force at each contact point includes:

Modeling a virtual finger as a discretized model consisting of mass points and springs, wherein the mass points represent joints and surface characteristic points of the finger, and the springs represent elastic connection between the mass points;

voxel segmentation is carried out on the soft tissue to obtain a uniform hexahedral grid, and each voxel comprises a material attribute and a stress-strain relation;

in the collision detection stage, judging whether the mass points of the virtual finger are in contact with the surface of the soft tissue or not, and calculating the local deformation of the soft tissue at the collision point;

according to the deformation quantity at the collision point and the motion state of the virtual finger particles, establishing a position-based kinetic equation set, and solving the displacement and the speed of the particles;

calculating the penetration depth of the virtual finger particles and the soft tissue surface according to the estimated positions of the virtual finger particles, and introducing the penetration depth as a position constraint into a dynamics equation set;

According to the initial length and the stiffness coefficient of the virtual finger spring, calculating the elastic displacement of the spring, and introducing the elastic displacement as internal force constraint into a kinetic equation set;

solving a kinetic equation set through iterative optimization, and updating the position and the speed of the virtual finger mass points while meeting collision constraint and spring constraint;

Calculating the interaction force applied to the soft tissue according to the displacement and the speed of the virtual finger particles, wherein the interaction force comprises positive pressure along the normal direction of the surface and friction along the tangential direction;

The calculated interaction force is reacted to the voxel node corresponding to the soft tissue surface, so that the deformation and stress redistribution of the soft tissue are caused;

and calculating and updating the interaction force between the virtual finger and the soft tissue in real time until the user stops operating or reaches the preset simulation time.

In an alternative embodiment of the present invention,

Extracting operation characteristics corresponding to the hands of a user when a collision event occurs, wherein the operation characteristics comprise an operation position, an operation speed, an operation force and an operation track, and determining the emergency operation quality score through a preset quality evaluation model by combining a preset operation evaluation rule base comprises:

in the virtual emergency training process, detecting collision events between the hands of a user and virtual soft tissues in real time, and recording the time stamp and duration of collision;

Extracting three-dimensional coordinates of virtual finger particles in a collision event, and mapping the three-dimensional coordinates to a local coordinate system of a virtual soft tissue to obtain operation position characteristics relative to the surface of the soft tissue;

calculating operation speed characteristics, including speed and direction, according to the position change and time difference of the virtual finger particles at the beginning and the end of the collision event;

calculating collision punishment force through the penetration depth and the contact speed of the virtual finger mass points and the soft tissues, and simultaneously calculating spring force according to the elastic displacement and the damping coefficient of the virtual finger mass points, and superposing the virtual finger mass points and the soft tissues to obtain operation force characteristics;

During the duration of a collision event, recording a three-dimensional coordinate sequence of virtual finger particles at a fixed sampling frequency, and performing smoothing and interpolation processing on the three-dimensional coordinate sequence to obtain a continuous operation track characteristic curve;

Inputting the extracted operation position features, operation speed features, operation force features and operation track features into a pre-trained quality evaluation model, wherein the model trains expert demonstration operation and different levels of user operation sample data by using a supervised learning algorithm, and establishes a mapping relation between the operation features and operation quality scores;

And the quality assessment model calculates the quality score of the current operation in real time according to the input information.

In a second aspect of an embodiment of the present invention,

A system for providing a human-based first aid success analysis, comprising:

The first unit is used for acquiring tomographic data of key parts of a human body by using medical image equipment, extracting outline information of soft tissues from the tomographic data by using a medical image processing technology, generating a triangular patch grid model of the soft tissues by using a three-dimensional reconstruction algorithm, performing tetrahedral subdivision on the triangular patch grid model of the soft tissues, and generating a discretization model suitable for emergency simulation by adopting a self-adaptive grid encryption strategy and a grid smoothing algorithm;

The second unit is used for detecting the contact state between the hand of the user and the emergency simulation discretization model in real time by adopting a collision detection algorithm, modeling the virtual finger as a mass point-spring system with rigidity and damping, calculating the multipoint touch feedback force in real time by adopting a position-based dynamics method according to the contact state between the virtual finger and the surface of the soft tissue, and applying the collision force to the emergency simulation discretization model as a boundary condition when a collision event is detected to drive the soft tissue to deform;

The third unit is used for extracting operation characteristics corresponding to the hands of the user when a collision event occurs, wherein the operation characteristics comprise an operation position, an operation speed, an operation force and an operation track, and a first-aid operation quality score is determined through a preset quality evaluation model by combining a preset operation evaluation rule base; and generating personalized operation prompts and improvement strategies in a self-adaptive mode according to the emergency operation quality scores, wherein the preset quality assessment model is constructed based on a machine learning algorithm.

In a third aspect of an embodiment of the present invention,

There is provided an electronic device including:

A processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the instructions stored in the memory to perform the method described previously.

In a fourth aspect of an embodiment of the present invention,

There is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method as described above.

The human body tomographic data are acquired through the medical image equipment, and the medical image processing and the three-dimensional reconstruction technology are used for generating a human body soft tissue discretization model with high precision and high fidelity, so that an anatomically accurate virtual environment is provided for first-aid simulation training, and the reality and the effectiveness of the training are improved. The self-adaptive grid encryption strategy and the grid smoothing algorithm are adopted to optimize the soft tissue model, so that a discretization model suitable for real-time simulation is generated, the calculation efficiency and the model quality are considered, and the fluency and the stability of simulation training are ensured.

The dynamic method based on the position is introduced, the multipoint touch feedback force between the virtual finger and the soft tissue is calculated in real time, the soft tissue is driven to deform by combining the collision detection algorithm, the realistic force touch interaction and tissue response are realized, and the immersion and substitution sense of the simulation training are enhanced. Extracting hand operation characteristics of a user, including position, speed, strength, track and the like, and combining a preset operation evaluation rule base and a machine learning-based quality evaluation model to evaluate the emergency operation quality of the user in real time, thereby providing objective and accurate feedback and guidance for the user, helping the user to find defects in time and improving the defects.

According to the emergency operation quality scores of the users, personalized operation prompts and improvement strategies are adaptively generated, targeted learning guidance is provided for the users with different levels, and the efficiency and pertinence of emergency skill training are improved.

Drawings

FIG. 1 is a flow chart of a method for simulated human-based first aid success analysis in accordance with an embodiment of the present invention;

FIG. 2 is a schematic diagram of a system for first aid success analysis based on a dummy in accordance with an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.

Fig. 1 is a flow chart of a method for first aid success analysis based on a dummy according to an embodiment of the invention, as shown in fig. 1, the method includes:

S101, acquiring tomographic data of a key part of a human body by using medical image equipment, extracting outline information of soft tissues from the tomographic data by using a medical image processing technology, generating a triangular patch grid model of the soft tissues by using a three-dimensional reconstruction algorithm, performing tetrahedral subdivision on the triangular patch grid model of the soft tissues, and generating a discretization model suitable for emergency simulation by adopting a self-adaptive grid encryption strategy and a grid smoothing algorithm;

S102, adopting a collision detection algorithm to detect the contact state between the hand of a user and the emergency simulation discretization model in real time, modeling a virtual finger as a mass point-spring system with rigidity and damping, adopting a position-based dynamic method to calculate a multipoint touch feedback force in real time according to the contact state between the virtual finger and the surface of soft tissue, and applying the collision force to the emergency simulation discretization model as a boundary condition when a collision event is detected to drive the soft tissue to deform;

S103, extracting operation characteristics corresponding to the hands of a user when a collision event occurs, wherein the operation characteristics comprise an operation position, an operation speed, an operation force and an operation track, and determining an emergency operation quality score through a preset quality evaluation model by combining a preset operation evaluation rule base; and generating personalized operation prompts and improvement strategies in a self-adaptive mode according to the emergency operation quality scores, wherein the preset quality assessment model is constructed based on a machine learning algorithm.

In an alternative embodiment of the present invention,

The method further comprises the steps of:

Illustratively, a high-order superelastic constitutive model is selected to describe the stress-strain relationship of the soft tissue for the nonlinear mechanical properties of human soft tissue. Common high-order superelastic constitutive models include the Ogden model, the Mooney-Rivlin model, the Yeoh model, and the like. Taking the Ogden model as an example, its strain energy function consists of the main stretch ratio, volume ratio and material parameters, which are used to characterize the non-linear and incompressible properties of soft tissue.

And determining the material parameters of the selected high-order superelastic constitutive model according to literature data or experimental test data. And fitting material parameters by using an optimization algorithm (such as a genetic algorithm, a particle swarm algorithm and the like) so as to ensure that the predictive result of the constitutive model has the highest fitness with experimental data.

Based on the high-order superelastic constitutive model, a nonlinear control equation of soft tissue deformation is established. And deducing a balance equation and a constitutive equation of soft tissue deformation by adopting a virtual work principle or a minimum potential energy principle to form a nonlinear partial differential equation set. Taking the virtual work principle as an example, the control equation of soft tissue deformation considers factors such as strain, stress, displacement, physical strength, face force and the like, and describes the mechanical balance relation in the soft tissue deformation process.

And performing spatial discretization on the nonlinear control equation by adopting a finite element method. The soft tissue region is divided into tetrahedral unit grids, and a proper interpolation function (such as linear interpolation, quadratic interpolation and the like) is selected to approximate the displacement field inside the unit. And converting the nonlinear control equation into a nonlinear algebraic equation set by using a Galerkin method, wherein the stiffness matrix and the displacement field are in nonlinear relation, and the external force load vector reflects the stress condition of the soft tissue.

And solving a nonlinear algebraic equation set to obtain a displacement field and stress strain distribution of soft tissue deformation. Common nonlinear solving methods include Newton-Raphson iteration, modified Newton-Raphson iteration, arc length, and the like. Considering the large deformation and large rotation characteristics of soft tissue deformation, the total Lagrangian method or the updated Lagrangian method is adopted to treat nonlinear geometrical effects.

And applying the displacement field obtained by solving to the node of the first-aid simulation discretization model, updating the coordinates of the surface of the soft tissue and the body grid, and realizing the deformation response of the soft tissue under the action of external force. Meanwhile, according to stress strain distribution, physical quantities such as strain energy, stress strain and the like in the soft tissue deformation process are calculated and used for subsequent links such as collision detection, tactile feedback, quality evaluation and the like.

In order to improve the calculation efficiency of soft tissue deformation response, numerical skills such as reduction integral, hourglass control, quality polycondensation and the like can be adopted, the number of unit integral points is reduced, a zero energy mode is restrained, and a quality matrix is simplified. In addition, the method can accelerate the nonlinear finite element solving process by utilizing methods such as GPU parallel computing, preprocessing technology and the like, and realizes real-time interactive simulation of soft tissue deformation.

By the technical scheme, the nonlinear deformation behavior of the human soft tissue under the action of external force can be accurately and efficiently simulated, realistic visual and force feedback is provided for first-aid simulation training, and the reality and effectiveness of the training are improved. Meanwhile, the physical quantity of soft tissue deformation response provides an important basis for operation quality evaluation, and is helpful for realizing intelligent skill evaluation and personalized guidance.

In an alternative embodiment of the present invention,

Illustratively, a series of two-dimensional tomographic images are obtained by tomographic scanning of a target organ (e.g., heart, liver, etc.) using a medical imaging device (e.g., CT, MRI, etc.). During scanning, appropriate scanning parameters (such as layer thickness, layer spacing, resolution, etc.) are selected to ensure image quality and data integrity.

The obtained two-dimensional tomographic image is preprocessed and segmented. Firstly, image enhancement technology (such as histogram equalization, median filtering and the like) is adopted to improve the contrast and the signal-to-noise ratio of the image, and the edge and detail information of the soft tissue are highlighted. Then, a threshold segmentation algorithm (such as Otsu thresholding method, region growing method and the like) is combined to extract the two-dimensional contour information of the region of interest of the soft tissue. And determining an optimal threshold or a seed point according to the gray value range and morphological characteristics of the soft tissue, and realizing automatic segmentation of the soft tissue and the background.

And performing triangular surface patch on the contours in the adjacent tomographic images by adopting Marching Cubes algorithm according to the extracted two-dimensional contour information, and generating a triangular surface patch grid model of the soft tissue. The Marching Cubes algorithm determines the topological relation of the isosurface according to the gray value of the vertex of the voxel by traversing the voxels in the adjacent tomographic images, generates a local triangular patch and finally constructs a complete three-dimensional surface grid model.

And optimizing and subdividing the generated triangular patch grid model. And eliminating irregular boundaries and jagged noise on the surface of the triangular patch grid model by adopting a Laplace smoothing algorithm. Laplace smoothing smoothes and homogenizes the mesh surface by iteratively moving the mesh vertices to the average positions of adjacent vertices. Meanwhile, according to the curvature and detail characteristics of the grid model, triangular patches are adaptively subdivided, the grid density is increased, and the accuracy and the authenticity of the model are improved.

And converting the optimized triangular patch grid model into a volume grid model. And generating a tetrahedral mesh in the soft tissue by adopting a tetrahedral subdivision algorithm according to the shape and the topological relation of the triangular patch mesh model. Common tetrahedral subdivision algorithms include Delaunay tetrahedralization, forward edge method, octree method, and the like. In the tetrahedral subdivision process, the grid density and the cell size are adaptively adjusted according to the grid quality criteria (such as the minimum dihedral angle, the volume ratio and the like), so that the generated tetrahedral grid is ensured to have good shape and numerical stability.

And carrying out quality evaluation and optimization on the generated tetrahedral mesh model. Quality indicators (e.g., condition number, tortuosity, etc.) of tetrahedral units are calculated, and low quality units are identified and deleted. And a grid smoothing algorithm (such as Laplacian smoothing, optimized smoothing and the like) is adopted to improve the quality of the tetrahedral grid and the regularity and the orthogonality of the units. If necessary, the local area is re-dissected or grid refined to meet the precision requirement of numerical simulation.

The optimized tetrahedral mesh model is used as an emergency simulation discretization model for subsequent physical modeling and numerical simulation. And defining the material properties, boundary conditions and load conditions of the soft tissues according to the node coordinates and the cell topological relation of the tetrahedral mesh, and constructing a finite element equation set. And solving the dynamic response of the soft tissue under the action of external force by adopting an explicit or implicit time integration method to obtain physical field quantities such as deformation, stress strain and the like.

By the technical scheme, the human soft tissue discretization model with high precision and high fidelity can be constructed from medical image data through a series of steps of image processing, three-dimensional reconstruction, grid generation and the like. The model not only can accurately describe the anatomical structure and morphological characteristics of the soft tissue, but also can reflect the mechanical properties and deformation behavior of the soft tissue, and provides reliable physical basis and data support for emergency simulation training.

In an alternative embodiment of the present invention,

Illustratively, motion data of a virtual finger is obtained by capturing three-dimensional position and posture information of a user's hand in real time by a motion capture device (e.g., a data glove, an optical tracking system, etc.). The virtual finger is mapped into the virtual environment so that it remains synchronized with the user's hand motion.

And carrying out bounding box hierarchical division on the virtual fingers, and constructing a bounding box tree. The virtual finger is recursively divided into multiple layers of bounding boxes according to the geometric shape and the spatial distribution of the virtual finger by adopting methods such as AABB (Axis-Aligned Bounding Box) or OBB (OrientedBounding Box). When collision detection is performed, the intersection test among bounding boxes is utilized to rapidly exclude the area which is unlikely to collide, the calculation range of collision detection is reduced to a local leaf node, and the detection efficiency is improved.

The distance field is pre-computed and stored for the soft tissue surface. And calculating the shortest distance from each vertex to other vertices of the soft tissue surface by adopting algorithms such as a fast travelling Method (FAST MARCHING Method) or distance transformation (Distance Transform) and the like, and generating a distance field voxel grid. During collision detection, mapping the vertexes of the virtual fingers into a distance field voxel grid, and obtaining the directed distance from the vertexes to the soft tissue surface through tri-linear interpolation. If the distance is less than the given distance threshold, the vertex is determined to collide with the soft tissue.

The virtual finger is abstracted into a discretized model consisting of mass points and springs. According to the geometric shape and topological structure of the virtual finger, the virtual finger is discretized into a plurality of mass points, and the mass points are connected through springs. By adjusting mass of mass points, rigidity and damping coefficient of springs, virtual fingers with different materials and shapes can be simulated. The particle motion is acted by spring force, damping force and external force, and the position and speed of the particle are updated by a numerical integration method (such as an explicit Euler method, an implicit Euler method and the like).

After collision of the virtual finger and the soft tissue is detected, a dynamic method based on the position is adopted to calculate the haptic feedback force. And iteratively optimizing the particle positions of the virtual fingers to enable the particle positions to meet collision constraint and spring constraint. The collision constraint ensures that the virtual finger does not penetrate the soft tissue surface, and the spring constraint maintains the shape and mechanical properties of the virtual finger. At each contact point, a normal feedback force and a tangential friction force are calculated based on the relative position and velocity of the virtual finger and the soft tissue. The normal feedback force is related to penetration depth and contact stiffness, and the tangential friction force is calculated according to coulomb friction law.

Surface normal and curvature changes due to soft tissue deformation are incorporated into the calculation of the feedback force. When the virtual finger is in contact with soft tissue, the soft tissue may deform, resulting in a change in surface normal and curvature at the point of contact. And simulating the deformation of the soft tissue by a finite element method or a mass point-spring method to obtain the surface normal direction and curvature information after deformation. The information is used for correcting the normal feedback force at the contact point, so that the normal feedback force is adapted to the deformed soft tissue surface, and the sense of reality of the tactile feedback is improved.

Based on the calculated haptic feedback force, a haptic feedback device (e.g., force feedback glove, haptic vibrator, etc.) is driven to provide a haptic stimulus to the user. The feedback force is mapped to the degree of freedom of the haptic feedback device, and the motor or the vibrating element is controlled to generate corresponding force or vibration effect to simulate the touch feeling when the virtual finger is contacted with soft tissues. Meanwhile, according to the contact state and the material property, audio feedback such as contact sound, friction sound and the like can be added, so that multi-sense immersion is enhanced.

By the technical scheme, real-time collision detection and tactile feedback between the virtual finger and the soft tissue can be realized. The collision detection is accelerated through the hierarchical bounding box and the distance field, so that the calculation efficiency is improved; simulating deformation and stress conditions of the virtual finger through a mass point-spring model and a position-based dynamic method; by taking into account the effect of soft tissue deformation on the contact force, the realism of the haptic feedback is improved. The comprehensive application of the technologies can provide vivid and natural touch interaction experience for users, and enhance the immersion and teaching effect of first-aid simulation training.

In an alternative embodiment of the present invention,

The virtual finger is illustratively modeled as a discretized model consisting of mass points and springs. The mass points represent the joints and surface features of the finger and are used to describe the shape and motion state of the finger. The mass points are connected through springs, and the rigidity and the damping coefficient of the springs determine the elasticity and the damping characteristics of the fingers. By adjusting the mass, position and spring parameters of the mass points, virtual fingers with different shapes and materials can be simulated.

The soft tissue is subjected to voxel segmentation and discretized into a uniform hexahedral mesh. Each voxel contains material properties (e.g., density, young's modulus, etc.) and stress-strain relationships (e.g., superelastic model, visco-elastic model, etc.). And establishing a physical model of the soft tissue by a finite element method or a mass point-spring method, and calculating deformation and stress distribution of the soft tissue under the action of external force.

In the collision detection stage, whether the particles of the virtual finger are in contact with the soft tissue surface or not is judged. And accelerating algorithms such as a hierarchical bounding box, a distance field and the like are adopted to quickly screen out particles and voxels which are likely to collide. For the contacted particles, the depth and direction of penetration of the particles through the soft tissue surface is calculated, and the local deformation of the soft tissue at the contact point is estimated from the penetration depth.

And establishing a dynamic equation system based on the position according to the deformation quantity at the collision point and the motion state of the virtual finger particles. The kinetic equation set comprises Newton's equation of motion of the mass point and Hooke's law of the spring, and describes the motion rule of the mass point under the action of external force and constraint force. And solving a kinetic equation set by a numerical integration method (such as an explicit Euler method, an implicit Euler method and the like) to obtain the displacement and the speed of the particles.

And calculating the penetration depth of the virtual finger particles with the soft tissue surface according to the estimated positions of the virtual finger particles. The penetration depth is introduced into the kinetic equation set as a position constraint to ensure that the virtual finger does not excessively penetrate soft tissue. The product of penetration depth and contact stiffness can be used to calculate the normal contact force as the external binding force for particle motion.

And calculating the elastic displacement of the spring according to the initial length and the stiffness coefficient of the virtual finger spring. And introducing the elastic displacement as an internal force constraint into a dynamic equation set, and maintaining the shape and mechanical properties of the virtual finger. The product of the spring force and the spring displacement can be used to calculate the internal restraining force between the mass points at the two ends of the spring.

And (3) solving a kinetic equation set through iterative optimization, and updating the position and the speed of the virtual finger mass points while meeting collision constraint and spring constraint. Common optimization algorithms include Sequence Quadratic Programming (SQP), lagrangian multiplier method, and the like. Through multiple iterations, the virtual finger reaches an equilibrium state under the action of external force and constraint force, and a stable and real interaction effect is obtained.

And calculating the interaction force applied to the soft tissue according to the displacement and the speed of the virtual finger particles. The interaction forces include positive pressure in the normal direction of the surface and friction in the tangential direction. Positive pressure is related to penetration depth and contact stiffness, and friction is calculated according to coulomb's law of friction, and is proportional to positive pressure and friction coefficient. And applying the calculated interaction force on the voxel node corresponding to the soft tissue surface to cause deformation and stress redistribution of the soft tissue.

The interaction force is applied as an external load to a physical model of the soft tissue, and the deformation response of the soft tissue is calculated by a finite element method or a particle-spring method. The deformation of the soft tissue may cause changes in surface normal and curvature, which in turn affect collision detection and interaction force calculation at the next time step. The two-way coupling interaction between the virtual finger and the soft tissue is realized by updating the shape and the mechanical state of the soft tissue in real time.

And calculating and updating the interaction force between the virtual finger and the soft tissue in real time until the user stops operating or reaches the preset simulation time. In the interaction process, parameters of collision detection, constraint solving and physical simulation are dynamically adjusted according to the motion of the virtual finger and the deformation of the soft tissue so as to balance the calculation efficiency and the sense of reality. Meanwhile, the calculated interactive force is fed back to force feedback equipment, such as a touch glove or a touch pen, so that a user obtains real touch experience.

Through the technical scheme, vivid and stable interactive simulation between the virtual finger and the soft tissue can be realized. The dynamic method based on the position realizes the bidirectional coupling between the finger and the soft tissue by introducing collision constraint and spring constraint and simultaneously considering the deformation and motion state of the virtual finger and the physical characteristics and deformation response of the soft tissue. And (3) solving a kinetic equation set through iterative optimization, and ensuring natural motion and real touch feeling of the finger under physical constraint. The technical scheme has important significance for improving simulation quality and user experience in application fields such as medical operation training, product design evaluation and the like.

In an alternative embodiment of the present invention,

Illustratively, during virtual emergency training, collision events between a user's hand (virtual finger) and virtual soft tissue are detected in real time by a collision detection algorithm. When a collision is detected to occur, the time stamp and duration of the collision are recorded for subsequent operational feature extraction and quality assessment.

Three-dimensional coordinates of the virtual finger particles in the collision event are extracted and converted from a global coordinate system to a local coordinate system of the virtual soft tissue. By calculating the position of the virtual finger particles relative to the soft tissue surface, the operation position characteristics are obtained, and the contact position and depth of the finger on the soft tissue are represented.

The operating speed characteristics are calculated based on the change in position and time difference of the virtual finger particles at the beginning and end of the collision event. The component of the velocity in each direction is obtained by dividing the variation of the particle position on three coordinate axes by the time difference, and then the magnitude and direction of the velocity are calculated. The operational speed characteristics reflect the speed and direction of the user's hand motion.

The collision punishment force is calculated through the penetration depth and the contact speed of the virtual finger particles and the soft tissues. The greater the penetration depth, the faster the contact speed, and the greater the penalty force for preventing excessive penetration of the soft tissue by the finger. Meanwhile, the spring force is calculated based on the elastic displacement (displacement relative to the initial position) of the virtual finger mass point and the damping coefficient. The larger the elastic displacement, the larger the spring force, for maintaining the shape and mechanical properties of the finger. And superposing the collision punishment force and the spring force to obtain an operation force characteristic, and reflecting the acting force exerted on the soft tissue by the user.

The three-dimensional coordinate sequence of virtual finger particles is recorded at a fixed sampling frequency (e.g., 30 times per second) for the duration of the collision event. Smoothing the acquired coordinate sequence to remove high-frequency noise and jitter, and generating a continuous track curve between sampling points through an interpolation algorithm (such as spline interpolation). The smoothed and interpolated coordinate sequence forms the operational trajectory features reflecting the path and continuity of the user's hand motion.

And inputting the extracted operation position features, operation speed features, operation dynamics features and operation track features into a pre-trained quality evaluation model. The model adopts a supervised learning algorithm, such as a Support Vector Machine (SVM), a random forest and the like, to train sample data of expert demonstration operation and different level user operation. And (3) by establishing a mapping relation between the operation characteristics and the operation quality scores given by the expert, learning the association rules between different operation characteristic combinations and the operation quality.

The quality assessment model calculates the quality score of the current operation in real time according to the input operation characteristics. The score may be a continuous value (e.g., 0-100 points) or may be a discrete scale (e.g., excellent, good, pass, fail). The scoring result reflects the similarity degree and quality level between the user operation and the standard operation, and can be used for timely feeding back and guiding the user to improve the operation.

In order to improve accuracy and robustness of quality assessment, multiple assessment indexes such as operation completion time, operation precision, operation stability and the like can be introduced, and scores of different indexes are weighted and combined to obtain a comprehensive quality score. Meanwhile, through cross verification, data enhancement and other technologies, the generalization capability and adaptability of the quality evaluation model are improved, so that the quality evaluation model can meet evaluation requirements of different users and different operation scenes.

In the virtual first aid training, the quality evaluation result is displayed in real time, and the user is guided to optimize operation by combining feedback modes such as vision and hearing. For example, visual elements such as color codes, progress bars and the like can be used for intuitively displaying the operation quality; the key steps and common errors of the operation are reminded to the user through voice prompt, warning sounds and the like. Multimodal feedback helps to enhance the operational awareness and learning effect of the user.

After training is finished, a detailed quality assessment report is generated, wherein the detailed quality assessment report comprises various index scores of operations, comparison analysis with standard operations, summary of advantages and disadvantages and the like. The report may help the user to fully examine his or her performance, identify where improvement is needed, and provide references and targets for the next training. Through continuous training and evaluation, the user can gradually improve the operation quality to reach the standard and skilled level.

According to the technical scheme, the user operation characteristics are extracted in real time, and the automatic evaluation and feedback of the virtual emergency operation are realized by combining the pre-trained quality evaluation model. The scheme integrates technologies such as collision detection, feature extraction, machine learning and the like, can comprehensively evaluate key elements such as the position, the speed, the force, the track and the like of operation, and provides objective and accurate quality evaluation for users. Meanwhile, through multi-mode feedback and detailed reports, users are helped to intuitively understand operation performance, training is pertinently improved, and learning efficiency and effect are improved. The technical scheme has important value for improving the intelligent level and teaching quality of virtual first-aid training.

Fig. 2 is a schematic structural diagram of a system based on simulated human first aid success analysis according to an embodiment of the present invention, as shown in fig. 2, the system includes:

In a third aspect of an embodiment of the present invention,

There is provided an electronic device including:

A processor;

a memory for storing processor-executable instructions;

In a fourth aspect of an embodiment of the present invention,

The present invention may be a method, apparatus, system, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for performing various aspects of the present invention.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims

1. A method for simulated human-based first aid success analysis, comprising:

2. The method according to claim 1, wherein the method further comprises:

3. The method of claim 1, wherein acquiring tomographic data of a critical part of a human body with a medical imaging device, extracting profile information of soft tissue from the tomographic data by a medical image processing technique, generating a triangular patch grid model of the soft tissue with a three-dimensional reconstruction algorithm, performing tetrahedral subdivision on the triangular patch grid model of the soft tissue, and generating a discretization model suitable for emergency simulation with an adaptive grid encryption strategy and a grid smoothing algorithm comprises:

4. The method of claim 1, wherein detecting in real time a contact state between a user's hand and the emergency simulation discretization model using a collision detection algorithm, modeling a virtual finger as a particle-spring system with stiffness and damping, and calculating in real time a multi-point haptic feedback force using a position-based dynamic method based on the contact state of the virtual finger with a soft tissue surface comprises:

5. The method of claim 4, wherein calculating the haptic feedback force at each contact point using a position-based dynamics approach by iteratively optimizing the position of the mass points of the virtual finger to meet the collision constraint and the spring constraint comprises:

6. The method of claim 1, wherein extracting the operation features corresponding to the user's hand in the event of a collision, the operation features including an operation position, an operation speed, an operation force, and an operation track, and determining the emergency operation quality score by a preset quality assessment model in combination with a preset operation assessment rule base comprises:

7. A system for simulated human-based first aid success analysis for implementing the method of any of the preceding claims 1-6, comprising:

8. An electronic device, comprising:

A processor;

a memory for storing processor-executable instructions;

Wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 6.

9. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 6.