Statisctics

We present space-efficient algorithms for performing Pear-son's chi-square goodness-of-fit test in a streaming setting. Since the chi-square test is one of the most well known and commonly used tests in statistics, it is surprising that there has been no prior work on designing streaming algorithms for it. The test is not based on a specific distribution assumption and has one-sample and two-sample variants. Given a stream of data, the one-sample variant tests if the stream is drawn from a fixed distribution and the two-sample variant tests if two data streams are drawn from the same or similar distributions. One major advantage of using statistical tests over other quantities commonly measured by streaming algorithms is that these tests do not require parameter tuning and have results that can be easily interpreted by data analysts. The problem that we solve in this paper is how to compute the chi-square test on streams with minimal assumptions about the stream and no parameter configuration required. We give rigorous proofs showing that it is possible to compute the chi-square statistic with high fidelity and an almost quadratic reduction in memory in the continuous case, but the categorical case only admits heuristic solutions. We validate the performance and accuracy of our algorithms through extensive testing on both real and synthetic data sets.

Un vuelo de una compañía aérea tiene 200 plazas disponibles. La compañía ha estimado la probabilidad de que una persona que ha comprado un billete no se presente al embarque. El resultado es 0.03. Por este motivo, la compañía ha vendido 202 billetes para este vuelo. Suponiendo que hay independencia entre los eventos que describen si las personas que han comprado el billete se presentan o no al embarque, calcula la probabilidad de que una persona que ha comprado el billete se quede sin plaza. Motiva la respuesta. Problema 2 Un piloto de una compañía aérea está preocupado a causa de un reciente accidente aéreo, y se pregunta cuál es la probabilidad de tener al menos un accidente aéreo grave durante su actividad profesional. Trabaja en una compañía aérea para la cual la probabilidad de que ocurra un accidente grave durante un vuelo se ha estimado en 1/(4.7 · 10 6). El piloto realiza 400 vuelos por año y prevé trabajar 20 años más. Calcular la probabilidad de que el piloto tenga al menos un accidente grave en los 20 años que le quedan de actividad. Problema 3 Suppose we repeat the experiment of tossing a fair coin n times and observe the sequence of heads and tails. Let Y be the random variable that represents the number of times tails occurs in n repetitions of the experiment. The probability mass function for the random variable Y is p[Y = k ] = n k p n , k = 0, 1, ... , n, with p = 1 2. Represent both numerically and graphically with Matlab this probability mass function for n = 10. Add the Matlab code to the solution. Problema 4 Suppose we repeat the experiment of tossing a fair coin n times and observe the sequence of heads and tails. Let the waiting time of tails be the index of the repetition of the experiment at which the first tail occurs. Let Z be the random variable that represents the waiting time of tails in n repetitions of the experiment. The probability mass function for the random variable Z is p[Z = k ] = p k , k = 1, 2, ... with p = 1 2. Represent both numerically and graphically with Matlab this probability mass function for n = 10. Add the Matlab code to the solution.