One of the primary goals of Artificial Intelligence research is to develop machines with human-li... more One of the primary goals of Artificial Intelligence research is to develop machines with human-like intelligence, perception and reasoning. In this direction teaching apprentice agents by observing demonstrations delivered by experts is a framework of imitation learning that can provide improved solutions and it is possible to significantly outperform the demonstrator. Inverse reinforcement learning (IRL) is a paradigm relying on Markov Decision Processes (MDPs) that has a twofold target: to learn optimum policies of autonomous agents for solving complex tasks from successful demonstrations, and also to discover the unknown reward function that could explain the expert behavior. In this article we are addressing the trajectory prediction problem in the aviation domain by using an IRL approach. The proposed learning scheme provides an imitation process where the algorithm tries to imitate demonstrated trajectories, exploiting raw trajectory data enriched with contextual features and learn an efficient reward model that is learned during imitation and has generalization capabilities to unknown cases. We show several experimental results using real trajectory data from the Spanish FIR that confirms the effectiveness of our approach in automatically predicting trajectories.
In this work we investigate the use of hierarchical multiagent reinforcement learning methods for... more In this work we investigate the use of hierarchical multiagent reinforcement learning methods for the computation of policies to resolve congestion problems in the air traffic management domain. To address cases where the demand of airspace use exceeds capacity, we consider agents representing flights, who need to decide on ground delays at the pre-tactical stage of operations, towards executing their trajectories while adhering to airspace capacity constraints. Hierarchical reinforcement learning manages to handle real-world problems with high complexity, by partitioning the task into hierarchies of states and/or actions. This provides an efficient way of exploring the state–action space and constructing an advantageous decision-making mechanism. We first establish a general framework of hierarchical multiagent reinforcement learning, and then, we further formulate four alternative schemes of abstractions, on states, actions, or both. To quantitatively assess the quality of solutions of the proposed approaches and show the potential of the hierarchical methods in resolving the demand–capacity balance problem, we provide experimental results on real-world evaluation cases, where we measure the average delay per flight and the number of flights with delays.
With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the air t... more With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the air traffic management (ATM) domain, this article studies the use of artificial intelligence and machine learning (AI/ML) methods to learn air traffic controllers’ (ATCOs) policy in resolving conflicts among aircraft assessed to violate separation minimum constraints during the en route phase of flights, in the tactical phase of operations. The objective is to model how conflicts are being resolved by ATCOs. Towards this goal, the article formulates the ATCO policy learning problem for conflict resolution, addresses the challenging issue of an inherent lack of information in real-world data, and presents AI/ML methods that learn models of ATCOs’ behavior. The methods are evaluated using real-world datasets. The results show that AI/ML methods can achieve good accuracy on predicting ATCOs’ actions given specific conflicts, revealing the preferences of ATCOs for resolution actions in specific c...
Interpretability, explainability, and transparency are key issues to introducing artificial intel... more Interpretability, explainability, and transparency are key issues to introducing artificial intelligence methods in many critical domains. This is important due to ethical concerns and trust issues strongly connected to reliability, robustness, auditability, and fairness, and has important consequences toward keeping the human in the loop in high levels of automation, especially in critical cases for decision making, where both (human and the machine) play important roles. Although the research community has given much attention to explainability of closed (or black) prediction boxes, there are tremendous needs for explainability of closed-box methods that support agents to act autonomously in the real world. Reinforcement learning methods, and especially their deep versions, are such closed-box methods. In this article, we aim to provide a review of state-of-the-art methods for explainable deep reinforcement learning methods, taking also into account the needs of human operators—th...
This book constitutes the thoroughly refereed post-conference proceedings of the First Internatio... more This book constitutes the thoroughly refereed post-conference proceedings of the First International Workshop on Organised Adaptation in Multi-Agent Systems, OAMAS 2008, held in Estoril, Portugal, in May 2008 as an associated event of AAMAS 2008. The 6 revised full papers presented together with 2 invited lectures were carefully selected during two rounds of reviewing and improvement for inclusion in the book. The papers address all current issues of organized adaptation, for purposes of self-healing, self-protection, self- ...
The following list of data sets is derived from the final data set of trajectory synopses availab... more The following list of data sets is derived from the final data set of trajectory synopses available at https://zenodo.org/record/2563256 . We have converted the original data into a) ESRI shapefiles and b) using RDF-Gen (https://zenodo.org/record/2556747) into RDF triples w.r.t. the datAcron ontology.
This data set is the RDF conversion w.r.t. the datAcron ontology, of the contextual maritime data... more This data set is the RDF conversion w.r.t. the datAcron ontology, of the contextual maritime data available at https://zenodo.org/record/1167595 . It has been generated by the RDF-Gen method on the data sets describing sea ports (World Port Index, Ports of Brittany, SeaDataNet fishing ports) and protected regions (fishing areas, fishing interdiction, Natura2000).
One of the primary goals of Artificial Intelligence research is to develop machines with human-li... more One of the primary goals of Artificial Intelligence research is to develop machines with human-like intelligence, perception and reasoning. In this direction teaching apprentice agents by observing demonstrations delivered by experts is a framework of imitation learning that can provide improved solutions and it is possible to significantly outperform the demonstrator. Inverse reinforcement learning (IRL) is a paradigm relying on Markov Decision Processes (MDPs) that has a twofold target: to learn optimum policies of autonomous agents for solving complex tasks from successful demonstrations, and also to discover the unknown reward function that could explain the expert behavior. In this article we are addressing the trajectory prediction problem in the aviation domain by using an IRL approach. The proposed learning scheme provides an imitation process where the algorithm tries to imitate demonstrated trajectories, exploiting raw trajectory data enriched with contextual features and learn an efficient reward model that is learned during imitation and has generalization capabilities to unknown cases. We show several experimental results using real trajectory data from the Spanish FIR that confirms the effectiveness of our approach in automatically predicting trajectories.
In this work we investigate the use of hierarchical multiagent reinforcement learning methods for... more In this work we investigate the use of hierarchical multiagent reinforcement learning methods for the computation of policies to resolve congestion problems in the air traffic management domain. To address cases where the demand of airspace use exceeds capacity, we consider agents representing flights, who need to decide on ground delays at the pre-tactical stage of operations, towards executing their trajectories while adhering to airspace capacity constraints. Hierarchical reinforcement learning manages to handle real-world problems with high complexity, by partitioning the task into hierarchies of states and/or actions. This provides an efficient way of exploring the state–action space and constructing an advantageous decision-making mechanism. We first establish a general framework of hierarchical multiagent reinforcement learning, and then, we further formulate four alternative schemes of abstractions, on states, actions, or both. To quantitatively assess the quality of solutions of the proposed approaches and show the potential of the hierarchical methods in resolving the demand–capacity balance problem, we provide experimental results on real-world evaluation cases, where we measure the average delay per flight and the number of flights with delays.
With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the air t... more With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the air traffic management (ATM) domain, this article studies the use of artificial intelligence and machine learning (AI/ML) methods to learn air traffic controllers’ (ATCOs) policy in resolving conflicts among aircraft assessed to violate separation minimum constraints during the en route phase of flights, in the tactical phase of operations. The objective is to model how conflicts are being resolved by ATCOs. Towards this goal, the article formulates the ATCO policy learning problem for conflict resolution, addresses the challenging issue of an inherent lack of information in real-world data, and presents AI/ML methods that learn models of ATCOs’ behavior. The methods are evaluated using real-world datasets. The results show that AI/ML methods can achieve good accuracy on predicting ATCOs’ actions given specific conflicts, revealing the preferences of ATCOs for resolution actions in specific c...
Interpretability, explainability, and transparency are key issues to introducing artificial intel... more Interpretability, explainability, and transparency are key issues to introducing artificial intelligence methods in many critical domains. This is important due to ethical concerns and trust issues strongly connected to reliability, robustness, auditability, and fairness, and has important consequences toward keeping the human in the loop in high levels of automation, especially in critical cases for decision making, where both (human and the machine) play important roles. Although the research community has given much attention to explainability of closed (or black) prediction boxes, there are tremendous needs for explainability of closed-box methods that support agents to act autonomously in the real world. Reinforcement learning methods, and especially their deep versions, are such closed-box methods. In this article, we aim to provide a review of state-of-the-art methods for explainable deep reinforcement learning methods, taking also into account the needs of human operators—th...
This book constitutes the thoroughly refereed post-conference proceedings of the First Internatio... more This book constitutes the thoroughly refereed post-conference proceedings of the First International Workshop on Organised Adaptation in Multi-Agent Systems, OAMAS 2008, held in Estoril, Portugal, in May 2008 as an associated event of AAMAS 2008. The 6 revised full papers presented together with 2 invited lectures were carefully selected during two rounds of reviewing and improvement for inclusion in the book. The papers address all current issues of organized adaptation, for purposes of self-healing, self-protection, self- ...
The following list of data sets is derived from the final data set of trajectory synopses availab... more The following list of data sets is derived from the final data set of trajectory synopses available at https://zenodo.org/record/2563256 . We have converted the original data into a) ESRI shapefiles and b) using RDF-Gen (https://zenodo.org/record/2556747) into RDF triples w.r.t. the datAcron ontology.
This data set is the RDF conversion w.r.t. the datAcron ontology, of the contextual maritime data... more This data set is the RDF conversion w.r.t. the datAcron ontology, of the contextual maritime data available at https://zenodo.org/record/1167595 . It has been generated by the RDF-Gen method on the data sets describing sea ports (World Port Index, Ports of Brittany, SeaDataNet fishing ports) and protected regions (fishing areas, fishing interdiction, Natura2000).
Uploads
Papers