The regret-minimization paradigm has emerged as a powerful technique for designing algorithms for... more The regret-minimization paradigm has emerged as a powerful technique for designing algorithms for online decision-making in adversarial environments. But so far, designing exact minmax-optimal algorithms for minimizing the worst-case regret has proven to be a difficult task in general, with only a few known results in specific settings. In this paper, we present a novel set-valued dynamic programming approach for designing such exact regret-optimal policies for playing repeated games with discounted losses. Our approach first draws the connection between regret minimization, and determining minimal achievable guarantees in repeated games with vector-valued losses. We then characterize the set of these minimal guarantees as the fixed point of a dynamic programming operator defined on the space of Pareto frontiers of convex and compact sets. This approach simultaneously results in the characterization of the optimal strategies that achieve these minimal guarantees, and hence of regret...
Abstract : The first part of the work yielded new results on simulated annealing and neural netwo... more Abstract : The first part of the work yielded new results on simulated annealing and neural networks while the second part focused on large deviations in high- speed communication networks. Neural Networks, Communication Networks, Simulated Annealing, High-Speed Networks.
To provide statistical guarantees of QoS, the Internet requires a measurement infrastructure for ... more To provide statistical guarantees of QoS, the Internet requires a measurement infrastructure for estimating available resources from actual tra c. In this paper, we outline an algorithm that collects a histogram of the occupancy of a single-server FCFS queue at packet arrival times, and infers the loss rate and delay distribution from such measurements. Direct estimation of such QoS parameters typically leads to estimators with a large variance. To reduce this variance, we t a bu er occupancy model, a sum of exponentials, to the histogram using a weighted least-squares algorithm. Furthermore, we compute batch means to minimize the bias due to the positive correlation between measurements. In this manner, we provide an e cient and robust approach to QoS estimation.
This paper is a brief tutorial on IEEE Time-Sensitive Networks. These networks are designed for a... more This paper is a brief tutorial on IEEE Time-Sensitive Networks. These networks are designed for applications where latency is critical, such as process control, automotive and aerospace, and augmented or virtual reality.
We explore a dynamic approach to the problems of call admission and resource allocation for commu... more We explore a dynamic approach to the problems of call admission and resource allocation for communication networks with connections that are differentiated by their quality of service requirements. In a dynamic approach, the amount of spare resources is estimated on-line based on feedbacks from the network's quality of service monitoring mechanism. The schemes we propose remove the dependence on accurate traffic models and thus simplify the tasks of supplying traffic statistics required of network users. In this paper we present two dynamic algorithms. The objective of these algorithms is to find the minimum bandwidth necessary to satisfy a cell loss probability constraint at an asynchronous transfer mode (ATM) switch. We show that in both schemes the bandwidth chosen by the algorithm approaches the optimal value almost surely. Furthermore, in the second scheme, which determines the point closest to the optimal bandwidth from a finite number of choices, the expected learning tim...
... By. Chen, Chih Liang and Mai, Tony K. Table of Contents. Page. ... sabotaging their own netwo... more ... By. Chen, Chih Liang and Mai, Tony K. Table of Contents. Page. ... sabotaging their own network is an example of insider attack. Denial of Service – An analogy of this problem is a bad guy preventing a good guy from. getting useful work done. A good example of this problem is the ...
We describe an approximate dynamic programming (ADP) approach to compute approximations of the op... more We describe an approximate dynamic programming (ADP) approach to compute approximations of the optimal strategies and of the minimal losses that can be guaranteed in discounted repeated games with vector-valued losses. Such games prominently arise in the analysis of regret in repeated decision-making in adversarial environments, also known as adversarial online learning. At the core of our approach is a characterization of the lower Pareto frontier of the set of expected losses that a player can guarantee in these games as the unique fixed point of a set-valued dynamic programming operator. When applied to the problem of regret minimization with discounted losses, our approach yields algorithms that achieve markedly improved performance bounds compared to off-the-shelf online learning algorithms like Hedge. These results thus suggest the significant potential of ADP-based approaches in adversarial online learning.
WiFi networks suffer from severe network utility degradation due to the usage of diverse modulati... more WiFi networks suffer from severe network utility degradation due to the usage of diverse modulation and coding schemes. The proportional-fair allocation, that has been shown to be a good remedy, can be enforced through the proper selection of contention window values. This has been achieved so far for centralized systems by an explicit solution of an optimization problem or, as proposed recently, by following a learning-based approach. In this paper, we present the first fully distributed solution in which each of the WiFi nodes independently tunes its contention window to achieve proportional fairness. Our solution is therefore applicable also for a set of collocated, unconnected WiFi networks. We compare the throughput and air-time allocation that this algorithm achieves to the values achieved by standard WiFi binary exponential back-off and values achieved by known centralized algorithms.
We describe an approximate dynamic programming (ADP) approach to compute approximately optimal st... more We describe an approximate dynamic programming (ADP) approach to compute approximately optimal strategies and approximations of the minimal losses that can be guaranteed in discounted repeated games with vector losses. At the core of our approach is a characterization of the lower Pareto frontier of the set of expected losses that a player can guarantee in these games as the unique fixed point of a set-valued dynamic programming (DP) operator. This fixed point can be approximated by an iterative application of this DP operator compounded by a polytopic set approximation, beginning with a single point. Each iteration can be computed by solving a set of linear programs corresponding to the vertices of the polytope. We derive rigorous bounds on the error of the resulting approximation and the performance of the corresponding approximately optimal strategies. We discuss an application to regret minimization in repeated decision-making in adversarial environments, where we show that this...
As vehicles get equipped with increasingly complex sensors and processors, the communication requ... more As vehicles get equipped with increasingly complex sensors and processors, the communication requirements become more demanding. Traditionally, vehicles have used specialized networking technologies designed to guarantee bounded latencies, such a the Controller Area Network (CAN) bus. Recently, some have used dedicated technologies to transport signals from cameras, lidars, radars, and ultrasonic sensors. In parallel, IEEE working groups are defining Ethernet standards for time-sensitive networks (TSN). This paper describes an Ethernet-based architecture with provable guaranteed performance and simple configuration that is suitable for supporting the communication requirements of many vehicles.
The regret-minimization paradigm has emerged as a powerful technique for designing algorithms for... more The regret-minimization paradigm has emerged as a powerful technique for designing algorithms for online decision-making in adversarial environments. But so far, designing exact minmax-optimal algorithms for minimizing the worst-case regret has proven to be a difficult task in general, with only a few known results in specific settings. In this paper, we present a novel set-valued dynamic programming approach for designing such exact regret-optimal policies for playing repeated games with discounted losses. Our approach first draws the connection between regret minimization, and determining minimal achievable guarantees in repeated games with vector-valued losses. We then characterize the set of these minimal guarantees as the fixed point of a dynamic programming operator defined on the space of Pareto frontiers of convex and compact sets. This approach simultaneously results in the characterization of the optimal strategies that achieve these minimal guarantees, and hence of regret...
Abstract : The first part of the work yielded new results on simulated annealing and neural netwo... more Abstract : The first part of the work yielded new results on simulated annealing and neural networks while the second part focused on large deviations in high- speed communication networks. Neural Networks, Communication Networks, Simulated Annealing, High-Speed Networks.
To provide statistical guarantees of QoS, the Internet requires a measurement infrastructure for ... more To provide statistical guarantees of QoS, the Internet requires a measurement infrastructure for estimating available resources from actual tra c. In this paper, we outline an algorithm that collects a histogram of the occupancy of a single-server FCFS queue at packet arrival times, and infers the loss rate and delay distribution from such measurements. Direct estimation of such QoS parameters typically leads to estimators with a large variance. To reduce this variance, we t a bu er occupancy model, a sum of exponentials, to the histogram using a weighted least-squares algorithm. Furthermore, we compute batch means to minimize the bias due to the positive correlation between measurements. In this manner, we provide an e cient and robust approach to QoS estimation.
This paper is a brief tutorial on IEEE Time-Sensitive Networks. These networks are designed for a... more This paper is a brief tutorial on IEEE Time-Sensitive Networks. These networks are designed for applications where latency is critical, such as process control, automotive and aerospace, and augmented or virtual reality.
We explore a dynamic approach to the problems of call admission and resource allocation for commu... more We explore a dynamic approach to the problems of call admission and resource allocation for communication networks with connections that are differentiated by their quality of service requirements. In a dynamic approach, the amount of spare resources is estimated on-line based on feedbacks from the network's quality of service monitoring mechanism. The schemes we propose remove the dependence on accurate traffic models and thus simplify the tasks of supplying traffic statistics required of network users. In this paper we present two dynamic algorithms. The objective of these algorithms is to find the minimum bandwidth necessary to satisfy a cell loss probability constraint at an asynchronous transfer mode (ATM) switch. We show that in both schemes the bandwidth chosen by the algorithm approaches the optimal value almost surely. Furthermore, in the second scheme, which determines the point closest to the optimal bandwidth from a finite number of choices, the expected learning tim...
... By. Chen, Chih Liang and Mai, Tony K. Table of Contents. Page. ... sabotaging their own netwo... more ... By. Chen, Chih Liang and Mai, Tony K. Table of Contents. Page. ... sabotaging their own network is an example of insider attack. Denial of Service – An analogy of this problem is a bad guy preventing a good guy from. getting useful work done. A good example of this problem is the ...
We describe an approximate dynamic programming (ADP) approach to compute approximations of the op... more We describe an approximate dynamic programming (ADP) approach to compute approximations of the optimal strategies and of the minimal losses that can be guaranteed in discounted repeated games with vector-valued losses. Such games prominently arise in the analysis of regret in repeated decision-making in adversarial environments, also known as adversarial online learning. At the core of our approach is a characterization of the lower Pareto frontier of the set of expected losses that a player can guarantee in these games as the unique fixed point of a set-valued dynamic programming operator. When applied to the problem of regret minimization with discounted losses, our approach yields algorithms that achieve markedly improved performance bounds compared to off-the-shelf online learning algorithms like Hedge. These results thus suggest the significant potential of ADP-based approaches in adversarial online learning.
WiFi networks suffer from severe network utility degradation due to the usage of diverse modulati... more WiFi networks suffer from severe network utility degradation due to the usage of diverse modulation and coding schemes. The proportional-fair allocation, that has been shown to be a good remedy, can be enforced through the proper selection of contention window values. This has been achieved so far for centralized systems by an explicit solution of an optimization problem or, as proposed recently, by following a learning-based approach. In this paper, we present the first fully distributed solution in which each of the WiFi nodes independently tunes its contention window to achieve proportional fairness. Our solution is therefore applicable also for a set of collocated, unconnected WiFi networks. We compare the throughput and air-time allocation that this algorithm achieves to the values achieved by standard WiFi binary exponential back-off and values achieved by known centralized algorithms.
We describe an approximate dynamic programming (ADP) approach to compute approximately optimal st... more We describe an approximate dynamic programming (ADP) approach to compute approximately optimal strategies and approximations of the minimal losses that can be guaranteed in discounted repeated games with vector losses. At the core of our approach is a characterization of the lower Pareto frontier of the set of expected losses that a player can guarantee in these games as the unique fixed point of a set-valued dynamic programming (DP) operator. This fixed point can be approximated by an iterative application of this DP operator compounded by a polytopic set approximation, beginning with a single point. Each iteration can be computed by solving a set of linear programs corresponding to the vertices of the polytope. We derive rigorous bounds on the error of the resulting approximation and the performance of the corresponding approximately optimal strategies. We discuss an application to regret minimization in repeated decision-making in adversarial environments, where we show that this...
As vehicles get equipped with increasingly complex sensors and processors, the communication requ... more As vehicles get equipped with increasingly complex sensors and processors, the communication requirements become more demanding. Traditionally, vehicles have used specialized networking technologies designed to guarantee bounded latencies, such a the Controller Area Network (CAN) bus. Recently, some have used dedicated technologies to transport signals from cameras, lidars, radars, and ultrasonic sensors. In parallel, IEEE working groups are defining Ethernet standards for time-sensitive networks (TSN). This paper describes an Ethernet-based architecture with provable guaranteed performance and simple configuration that is suitable for supporting the communication requirements of many vehicles.
Uploads
Papers by Jean Walrand