This work presents a novel fusion mechanism for estimating the three-dimensional trajectory of a ... more This work presents a novel fusion mechanism for estimating the three-dimensional trajectory of a moving target using images collected by multiple imaging sensors. The proposed projective particle filter avoids the explicit target detection prior to fusion. In projective particle filter, particles that represent the posterior density (of target state in a high-dimensional space) are projected onto the lower-dimensional observation space. Measurements are generated directly in the observation space (image plane) and a marginal (sensor) likelihood is computed. The particles states and their weights are updated using the joint likelihood computed from all the sensors. The 3D state estimate of target (system track) is then generated from the states of the particles. This approach is similar to track-before-detect particle filters that are known to perform well in tracking dim and stealthy targets in image collections. Our approach extends the track-before-detect approach to 3D tracking using the projective particle filter. The performance of this measurement-level fusion method is compared with that of a track-level fusion algorithm using the projective particle filter. In the track-level fusion algorithm, the 2D sensor tracks are generated separately and transmitted to a fusion center, where they are treated as measurements to the state estimator. The 2D sensor tracks are then fused to reconstruct the system track. A realistic synthetic scenario with a boosting target was generated, and used to study the performance of the fusion mechanisms.
ABSTRACT In this work we study the problem of detecting and tracking challenging targets that exh... more ABSTRACT In this work we study the problem of detecting and tracking challenging targets that exhibit low signal-to-noise ratios (SNR). We have developed a particle filter-based track-before-detect (TBD) algorithm for tracking such dim targets. The approach incorporates the most recent state estimates to control the particle flow accounting for target dynamics. The flow control enables accumulation of signal information over time to compensate for target motion. The performance of this approach is evaluated using a sensitivity analysis based on varying target speed and SNR values. This analysis was conducted using high-fidelity sensor and target modeling in realistic scenarios. Our results show that the proposed TBD algorithm is capable of tracking targets in cluttered images with SNR values much less than one.
ABSTRACT Target detection and tracking with passive infrared (IR) sensors can be challenging due ... more ABSTRACT Target detection and tracking with passive infrared (IR) sensors can be challenging due to significant degradation and corruption of target signature by atmospheric transmission and clutter effects. This paper summarizes our efforts in phenomenology modeling of boosting targets with IR sensors, and developing algorithms for tracking targets in the presence of background clutter. On the phenomenology modeling side, the clutter images are generated using a high fidelity end-to-end simulation testbed. It models atmospheric transmission, structured clutter and solar reflections to create realistic background images. The dynamics and intensity of a boosting target are modeled and injected onto the background scene. Pixel level images are then generated with respect to the sensor characteristics. On the tracking analysis side, a particle filter for tracking targets in a sequence of clutter images is developed. The particle filter is augmented with a mechanism to control particle flow. Specifically, velocity feedback is used to constrain and control the particles. The performance of the developed "adaptive" particle filter is verified with tracking of a boosting target in the presence of clutter and occlusion.
This paper presents a metric for finding optimal sensor and target geometries that provide accura... more This paper presents a metric for finding optimal sensor and target geometries that provide accurate estimates of bias during target tracking with a single sensor taking measurements of bearing. Since the bias cannot be measured directly, it is shown how to manipulate the equations of a Kalman filter to produce a pseudo measurement of bias and its associated measurement error covariance. These measurement error covariances are used to form a Cramer-Rao lower bound (CRLB) on the bias estimation variance as a function of sensor and target geometries. It is shown that highly accurate estimates of bias can be produced using a single sensor, even if the kinematic state estimate of the target is poor.
ABSTRACT We study the problem of distributed velocity align-ment among a group of nonholonomic ag... more ABSTRACT We study the problem of distributed velocity align-ment among a group of nonholonomic agents in 2 and 3 dimensions. Inspired by social aggregation phenomena such as flocking and schooling in birds and fish, we develop vision based control laws for velocity alignment and motion coordination. The proposed control laws are distributed, in the sense that only infor-mation from neighboring agents are included. Furthermore, the control laws are coordinate-free and do not rely on measurement or communication of heading information among neighbors, but instead requires measurements of bearing, optical flow and time-to-collision, all of which can be measured using vision.
Modular Research Rover and Gesture Control System for EVA. Joel Richter, Kevin Sloan, Elizabeth B... more Modular Research Rover and Gesture Control System for EVA. Joel Richter, Kevin Sloan, Elizabeth Barnwell, Adrienne Benski, Amy Blank, Kevin Clark, Lisa Legget,Steve McGuire, Nima Moshtagh, Paul Smidansky 2002. As ...
2020 IEEE 23rd International Conference on Information Fusion (FUSION)
This paper presents a metric for finding optimal sensor and target geometries that provide accura... more This paper presents a metric for finding optimal sensor and target geometries that provide accurate estimates of bias during target tracking with a single sensor taking measurements of bearing. Since the bias cannot be measured directly, it is shown how to manipulate the equations of a Kalman filter to produce a pseudo measurement of bias and its associated measurement error covariance. These measurement error covariances are used to form a Cramer-Rao lower bound (CRLB) on the bias estimation variance as a function of sensor and target geometries. It is shown that highly accurate estimates of bias can be produced using a single sensor, even if the kinematic state estimate of the target is poor.
Ground telescopes enable low-cost tracking and characterization of meter-class space objects. Sin... more Ground telescopes enable low-cost tracking and characterization of meter-class space objects. Since a telescope may be tasked to observe multiple fields of the sky, the time between observations for each object may vary from several seconds to tens of minutes. Long propagation times with nonlinear dynamics are challenging for traditional filtering methods such as the Extended Kalman Filter (EKF). Sampling-based filters based on the Particle Filter (PF) are promising for this type of problem but typically require maintaining a large number of samples. In this work, we evaluate the Homotopy Particle Filter (HPF) which promises effective performance with orders of magnitude fewer particles. The performance of the HPF is evaluated against GEO satellite observations collected by a ground telescope at Lockheed Martin’s Space Object Tracking (SPOT) facility.
We propose a biologically inspired, distributed coordination scheme based on nearest-neighbor int... more We propose a biologically inspired, distributed coordination scheme based on nearest-neighbor interactions for a set of mobile agents equipped with vision sensors. It is assumed that each agent is only capable of measuring two quantities relative to its nearest neighbors: the time- to-collision and optical flow. We prove that the proposed distributed control law results in alignment of headings and flocking, even when the topology of the interconnection changes with time, if a a weak notion of connectivity in the proximity graph is maintained. Connections between the pro- posed scheme and distributed synchronization of nonlinearly coupled oscillators is discussed. Index Terms— Cooperative control, multiagent systems, vision-based control, Lyapunov stability.
In many applications where communication delays are present, measurements with earlier time stamp... more In many applications where communication delays are present, measurements with earlier time stamps can arrive out-of-sequence, i.e., after state estimates have been obtained for the current time instant. To incorporate such an Out-Of-Sequence Measurement (OOSM), many algorithms have been proposed in the literature to obtain or approximate the optimal estimate that would have been obtained if the OOSM had arrived in-sequence. When OOSM occurs repeatedly, approximate estimations as a result of incorporating one OOSM have to serve as the basis for incorporating yet another OOSM. The question of whether the "approximation of approximation" is well behaved, i.e., whether approximation errors accumulate in a recursive setting, has not been adequately addressed in the literature. This paper draws attention to the stability question of recursive OOSM processing filters, formulates the problem in a specific setting, and presents some simulation results that suggest that such filters are indeed well-behaved. Our hope is that more research will be conducted in the future to rigorously establish stability properties of these filters.
A catadioptric camera with a large field of view is desirable in many applications such as survei... more A catadioptric camera with a large field of view is desirable in many applications such as surveillance systems, video conferencing and robot navigation. In this thesis, catadioptric cameras with parabolic mirrors are studied in details. A NetVision 360 camera from Remote Reality Inc. is used as the imaging system. Variety of topics has been covered in this thesis such as generating panoramic images, calibration of omni-directional cameras, panoramic stereo and visual tracking in omni-directional videos. Because omni-directional images are distorted and cannot be easily perceived by human eyes, the panoramic view of the scene can be helpful. Two different methods called back projection and linear projection are implemented to produce panoramic images. However, in order to generate panoramic images we need to have some information about the intrinsic properties of the imaging system such as the center of the omni-image and the parameter h of the parabolic mirror. A circle-based method is used for calibration of the omni-directional cameras. Omni-directional images can be used for 3-D scene reconstruction. This is done
2015 18th International Conference on Information Fusion (Fusion), 2015
Homotopy particle filters (HPFs), recently developed by Daum and Huang [1], present an alternativ... more Homotopy particle filters (HPFs), recently developed by Daum and Huang [1], present an alternative nonlinear filtering approach to sampling-based particle filters. Homotopy filters perform information update using the flow of particles to regions with high measurement likelihood. The particle flows in HPFs are solutions to a Fokker-Plank equation governing the dynamics of the posterior density function. The resulting partial differential equation is highly under-determined and has many solutions. In this work we study the nonzero-diffusion flow, and show its advantage in multisensor fusion. The nonzero-diffusion flow was chosen because it has the form of an information filter (inverse-covariance filter), and this feature makes it suitable for integration of measurement contributions from different sensors. The effectiveness of the HPF with nonzero diffusion flow as a fusion mechanism was evaluated for tracking a moving target using multiple range and bearing sensors. It is shown tha...
This work presents a novel fusion mechanism for estimating the three-dimensional trajectory of a ... more This work presents a novel fusion mechanism for estimating the three-dimensional trajectory of a moving target using images collected by multiple imaging sensors. The proposed projective particle filter avoids the explicit target detection prior to fusion. In projective particle filter, particles that represent the posterior density (of target state in a high-dimensional space) are projected onto the lower-dimensional observation space. Measurements are generated directly in the observation space (image plane) and a marginal (sensor) likelihood is computed. The particles states and their weights are updated using the joint likelihood computed from all the sensors. The 3D state estimate of target (system track) is then generated from the states of the particles. This approach is similar to track-before-detect particle filters that are known to perform well in tracking dim and stealthy targets in image collections. Our approach extends the track-before-detect approach to 3D tracking using the projective particle filter. The performance of this measurement-level fusion method is compared with that of a track-level fusion algorithm using the projective particle filter. In the track-level fusion algorithm, the 2D sensor tracks are generated separately and transmitted to a fusion center, where they are treated as measurements to the state estimator. The 2D sensor tracks are then fused to reconstruct the system track. A realistic synthetic scenario with a boosting target was generated, and used to study the performance of the fusion mechanisms.
ABSTRACT In this work we study the problem of detecting and tracking challenging targets that exh... more ABSTRACT In this work we study the problem of detecting and tracking challenging targets that exhibit low signal-to-noise ratios (SNR). We have developed a particle filter-based track-before-detect (TBD) algorithm for tracking such dim targets. The approach incorporates the most recent state estimates to control the particle flow accounting for target dynamics. The flow control enables accumulation of signal information over time to compensate for target motion. The performance of this approach is evaluated using a sensitivity analysis based on varying target speed and SNR values. This analysis was conducted using high-fidelity sensor and target modeling in realistic scenarios. Our results show that the proposed TBD algorithm is capable of tracking targets in cluttered images with SNR values much less than one.
ABSTRACT Target detection and tracking with passive infrared (IR) sensors can be challenging due ... more ABSTRACT Target detection and tracking with passive infrared (IR) sensors can be challenging due to significant degradation and corruption of target signature by atmospheric transmission and clutter effects. This paper summarizes our efforts in phenomenology modeling of boosting targets with IR sensors, and developing algorithms for tracking targets in the presence of background clutter. On the phenomenology modeling side, the clutter images are generated using a high fidelity end-to-end simulation testbed. It models atmospheric transmission, structured clutter and solar reflections to create realistic background images. The dynamics and intensity of a boosting target are modeled and injected onto the background scene. Pixel level images are then generated with respect to the sensor characteristics. On the tracking analysis side, a particle filter for tracking targets in a sequence of clutter images is developed. The particle filter is augmented with a mechanism to control particle flow. Specifically, velocity feedback is used to constrain and control the particles. The performance of the developed "adaptive" particle filter is verified with tracking of a boosting target in the presence of clutter and occlusion.
This paper presents a metric for finding optimal sensor and target geometries that provide accura... more This paper presents a metric for finding optimal sensor and target geometries that provide accurate estimates of bias during target tracking with a single sensor taking measurements of bearing. Since the bias cannot be measured directly, it is shown how to manipulate the equations of a Kalman filter to produce a pseudo measurement of bias and its associated measurement error covariance. These measurement error covariances are used to form a Cramer-Rao lower bound (CRLB) on the bias estimation variance as a function of sensor and target geometries. It is shown that highly accurate estimates of bias can be produced using a single sensor, even if the kinematic state estimate of the target is poor.
ABSTRACT We study the problem of distributed velocity align-ment among a group of nonholonomic ag... more ABSTRACT We study the problem of distributed velocity align-ment among a group of nonholonomic agents in 2 and 3 dimensions. Inspired by social aggregation phenomena such as flocking and schooling in birds and fish, we develop vision based control laws for velocity alignment and motion coordination. The proposed control laws are distributed, in the sense that only infor-mation from neighboring agents are included. Furthermore, the control laws are coordinate-free and do not rely on measurement or communication of heading information among neighbors, but instead requires measurements of bearing, optical flow and time-to-collision, all of which can be measured using vision.
Modular Research Rover and Gesture Control System for EVA. Joel Richter, Kevin Sloan, Elizabeth B... more Modular Research Rover and Gesture Control System for EVA. Joel Richter, Kevin Sloan, Elizabeth Barnwell, Adrienne Benski, Amy Blank, Kevin Clark, Lisa Legget,Steve McGuire, Nima Moshtagh, Paul Smidansky 2002. As ...
2020 IEEE 23rd International Conference on Information Fusion (FUSION)
This paper presents a metric for finding optimal sensor and target geometries that provide accura... more This paper presents a metric for finding optimal sensor and target geometries that provide accurate estimates of bias during target tracking with a single sensor taking measurements of bearing. Since the bias cannot be measured directly, it is shown how to manipulate the equations of a Kalman filter to produce a pseudo measurement of bias and its associated measurement error covariance. These measurement error covariances are used to form a Cramer-Rao lower bound (CRLB) on the bias estimation variance as a function of sensor and target geometries. It is shown that highly accurate estimates of bias can be produced using a single sensor, even if the kinematic state estimate of the target is poor.
Ground telescopes enable low-cost tracking and characterization of meter-class space objects. Sin... more Ground telescopes enable low-cost tracking and characterization of meter-class space objects. Since a telescope may be tasked to observe multiple fields of the sky, the time between observations for each object may vary from several seconds to tens of minutes. Long propagation times with nonlinear dynamics are challenging for traditional filtering methods such as the Extended Kalman Filter (EKF). Sampling-based filters based on the Particle Filter (PF) are promising for this type of problem but typically require maintaining a large number of samples. In this work, we evaluate the Homotopy Particle Filter (HPF) which promises effective performance with orders of magnitude fewer particles. The performance of the HPF is evaluated against GEO satellite observations collected by a ground telescope at Lockheed Martin’s Space Object Tracking (SPOT) facility.
We propose a biologically inspired, distributed coordination scheme based on nearest-neighbor int... more We propose a biologically inspired, distributed coordination scheme based on nearest-neighbor interactions for a set of mobile agents equipped with vision sensors. It is assumed that each agent is only capable of measuring two quantities relative to its nearest neighbors: the time- to-collision and optical flow. We prove that the proposed distributed control law results in alignment of headings and flocking, even when the topology of the interconnection changes with time, if a a weak notion of connectivity in the proximity graph is maintained. Connections between the pro- posed scheme and distributed synchronization of nonlinearly coupled oscillators is discussed. Index Terms— Cooperative control, multiagent systems, vision-based control, Lyapunov stability.
In many applications where communication delays are present, measurements with earlier time stamp... more In many applications where communication delays are present, measurements with earlier time stamps can arrive out-of-sequence, i.e., after state estimates have been obtained for the current time instant. To incorporate such an Out-Of-Sequence Measurement (OOSM), many algorithms have been proposed in the literature to obtain or approximate the optimal estimate that would have been obtained if the OOSM had arrived in-sequence. When OOSM occurs repeatedly, approximate estimations as a result of incorporating one OOSM have to serve as the basis for incorporating yet another OOSM. The question of whether the "approximation of approximation" is well behaved, i.e., whether approximation errors accumulate in a recursive setting, has not been adequately addressed in the literature. This paper draws attention to the stability question of recursive OOSM processing filters, formulates the problem in a specific setting, and presents some simulation results that suggest that such filters are indeed well-behaved. Our hope is that more research will be conducted in the future to rigorously establish stability properties of these filters.
A catadioptric camera with a large field of view is desirable in many applications such as survei... more A catadioptric camera with a large field of view is desirable in many applications such as surveillance systems, video conferencing and robot navigation. In this thesis, catadioptric cameras with parabolic mirrors are studied in details. A NetVision 360 camera from Remote Reality Inc. is used as the imaging system. Variety of topics has been covered in this thesis such as generating panoramic images, calibration of omni-directional cameras, panoramic stereo and visual tracking in omni-directional videos. Because omni-directional images are distorted and cannot be easily perceived by human eyes, the panoramic view of the scene can be helpful. Two different methods called back projection and linear projection are implemented to produce panoramic images. However, in order to generate panoramic images we need to have some information about the intrinsic properties of the imaging system such as the center of the omni-image and the parameter h of the parabolic mirror. A circle-based method is used for calibration of the omni-directional cameras. Omni-directional images can be used for 3-D scene reconstruction. This is done
2015 18th International Conference on Information Fusion (Fusion), 2015
Homotopy particle filters (HPFs), recently developed by Daum and Huang [1], present an alternativ... more Homotopy particle filters (HPFs), recently developed by Daum and Huang [1], present an alternative nonlinear filtering approach to sampling-based particle filters. Homotopy filters perform information update using the flow of particles to regions with high measurement likelihood. The particle flows in HPFs are solutions to a Fokker-Plank equation governing the dynamics of the posterior density function. The resulting partial differential equation is highly under-determined and has many solutions. In this work we study the nonzero-diffusion flow, and show its advantage in multisensor fusion. The nonzero-diffusion flow was chosen because it has the form of an information filter (inverse-covariance filter), and this feature makes it suitable for integration of measurement contributions from different sensors. The effectiveness of the HPF with nonzero diffusion flow as a fusion mechanism was evaluated for tracking a moving target using multiple range and bearing sensors. It is shown tha...
Uploads
Papers