Real Time Nework Anomaly Detection Using Relative Entropy Altyeb Altaher Sureswaran Ramadass, Ammar Almomani National Advnced IPv6 Center of Excellence National Advnced IPv6 Center of Excellence Universiti Sains Malaysia,Malaysia Universiti Sains Malaysia,Malaysia Abstract-As Penng,Malaysia Penng,Malaysia Altyeb@nav6.org Sures,ammrali@nav6.org the computer networks continue to increse in size, complexity and importance, the network security issue becomes more and more importnt. n this paper, we propose a real time anomaly detection system based on relative entropy. The proposed system captures the network traic packets and then uses relative entropy and adaptive ilter to dynamically determine the traic changes nd to examine anomaly. The paper is orgnized as follows. Related work in section II . Section III presents he network traic attributes and he network entropy. Section IV presents he proposed nomaly detection mehod. Section V evaluates he efectiveness of or proposed scheme. Section VI concludes he paper. whether the traic change is normal or contains II. Or experimental results show that the proposed system is RELATED WORK eicient for on-line anomaly detection, using traic trace collected in hih-speed links. The Kywords- Network securiy ; anomay detection; entropy theoy. failre of raditional signatre-based in detecting polymorphic nd nseen malware, orients he reserch in network secrity to directions. The ntropy of diferent packet attributes ndr normal nd anormal network conditions have been nalyzed in [6-8,10]. he abnmal network traic I. worms. Fher work on As he computer ntworks continue to increase in size, complexity nd importnce, he network is afected by diferent attacks, such as DoS, port scnning nd INTRODUCTION security issue becomes more nd more importnt. fmding out wheher he entropy values of diferent atributes are highly correlated is given in [6]. n or paper we also observe his phnomenon nd vriy he conclusion made in [6]. n [9], he auhors make use of he concept of maximum ntropy to build up a normal ntwork "Malware" is an abbreviation for 'malicious sotwre' nd is distribution baseline and hen use relative entropy to detect he used to refer to any sotware designed to cause dmage to a anomalies. However, he baseline distribution in [9] is based on single computer, server, or computer network[1].According to he Kasprsky labs in February 2011, 252,187,961 malicious combination TCP/IP protocol of ield diferent hey only attributes which attribute-values take hree are protocol mens vry ields he lrge. programs detected [2]. This worryngly high number is only Subsequently, likely to increase, especially as the malwre author's incentives consideration nd heir expriments showed that it would into for writing such sotwre is now manly a inncial one. generate a large According to its propagation mehods, malicious code is should be ordered and labeled according to their featres and usually classiied into he following categories he complex preprocess will decrease he ability for detecting [3][4][5]: viruses, worms, Trojan horses, backdoors nd spyware . Due to the siniicnt loss nd dmages induced by featre st. Moreover, he raw packt data nomalies in real-time. malicious executables, he malwre detection becomes one of he most III. critical issues in he ield of computer secrity. Crrently, most widely-used malwre dtection sotware uses signatre-based mehod to recognize threats . Signatres re sequnces of bytes in the machine code of he malwre. The nability of traditional signatre based malware detection approaches to catch polymorphic and new, previously nseen malwres has shited he focus of malwre detection reserch to ind more genralized and scalable featres hat cn identiy malicious behavior as a process instead of a single static signatre. n his paper, we propose a real time nomaly detection system NETWORK TRAFFIC ATTRIBUTES AND NETWORK ENTROPY n his section, we describe he important network traic attributes for nomalies detection. We also describe the computation on he entropy values of the network traic attributes. A. Network Traic Attributes Ater reserching n customer service lows in an nterprise, we summarize he work lows of its customer service system are as follows: based on relative entropy. The system uses the relative ntropy to nalyze he ntwork trafic and detect he anomalies. 978-1-4577-1169-5/11/$26.00 ©2011 IEEE 258 Traic attributes hat are especially important (because heir adaptive ilter to exmine he traic changes and determines rapid change dring typical attacks) and used dring process of wheher he traic is normal or contains anomaly. anomaly detection are [11]: • Sorce and destination IP adress, • Sorce and destination port, • Number of bytes and packets snt to he remote hosts, B. • Number of bytes packets received by the local host, • TCP lags, especially SYN, RST and FIN lags • ration of he connection Adaptive Detection Threshold Setup To detect anomalies, we need a method to clearly diferentiate network anomalies rom he normal behavior. Therefore we introduce an adaptive hreshold to diferentiate between steady "normal" network traic behavior, and non-steady network traic behavior. We irst compute ntropy values of degree distributions in each time interval, and hen n or approach we take into consideration he following: compute mean entropy in a particular time interval. We also source/destination IP nd port nmber, use variance to relect he deviation between normal and Number of bytes sent/received. These attributes were selected because a siniicant nmber of worm attacks cause changes in the values of hese attributes and herefore could be recognized as an anmalous state [12]. abnormal behavior. Let us assume, he measred ntropy Y be a random variable with mean E( y) = . nd var( y) = r2 . hen, he Threshold is deined as follow Threshold Network Entropy B. = l±3*" (2) The entropy of nrmal network traic behavior is less han Entropy is a measrement of he disrder of a system. If he equal he hreshold Beyond his normal region, he ntropy system tnds to be in disorder, its ntropy increases towards 1; r if he system tnds to be in order, hn its ntropy decrease represents raic events as anomalous and assigns a severity towards o. level depending upon its deviation rom he normal region. We can view crtain attributes of packets hat we captre in a period of time as a set. The entropy of he packet attribute­ value can be deined as: The processing ngine in he proposed online detection system = detect anomalies. (1) ;=1 The anomaly detection system captres network traic Where in H(P), he P(x) is as follows: ( XI ) = Anomaly Detection Methodoloy uses an eicient lightweight mehodology based on entropy to H(P) ip(xi)logP(xi) p C. every time window (30 sec) and store he source and destination IP adress of he all lows in he database, hn it Number· of . pkts· with· Xi as· certain· aribute • inds he nmber of lows sent to distinct destination IP Total. number· of· pks adress. Based on he number of total lows and he nmber of It should be notes hat he work in [6-8,10] takes his approach to check for the difernces between he normal and abnormal network action. lows sent to distinct destination he algorihm computes he entropy value for his time window. he algorihm calculates he detection hreshold as entropy average value plus or minus he hree standard deviations. If he enropy value of network traic low exceeds the hreshold hen he network traic low is considred as nomaly. IV. THE PROPOSED REAL TIME ANOMALY DETECTION V. SYSTEM n his section, we rst give an overview of he proposed nomaly detection System. Second, we describe or adaptive detection hreshold setup. Thn we presnt or mehodology for computing he ntropy and self-adjusting he hreshold to raise an alert whn anormal traic is detected. A. PERFORMANCE EVALUATION The ntwork topology for he experiment is shown in Fig.l. All he network traic captred using he network interace crd (NIC) on PC3 in Fig.3 and stored in database, he database design consists of one table and stores all information used and needed by our proposed system. The proposed real time anomaly etection system description The proposed system captres he network traic packets and hen uses relative ntropy and adaptive iltr to dynamically detrmine he traic changes. It hn applies 259 entropy. To validate the eiciency of our online anomaly detection technique, we used real time network traic rom the National Advanced IPv6 Center of Excellence - University Internet Science Malaysia (USM). Next we injected the Witty Worm AI ntk cwu ipwrd using ht n.t bpt.r on PC3and hin :ud h e lDAdmnl yhl d.n� dataset of CAIDA into the online network traic. Our experimental results show that the proposed system is eicient high sttd nt' MlUy d.tn sstem for on-line anomaly detection because it is based on the entropy which increases the sensitivity of the detection process to uncover well-known or nknown nomalies. Frthermore, the use of adaptive treshold results in lower false alarm rate. Or ongoing work rther analyzes the traic anomalous featres, and extends the methodology proposed in this paper pe3 High seed newo; anomaly detecion sysem to diagnose additional network-wide anomalies REFERENCES Figre 1 : The network topology for the experiment [I] P. Szor. The t of Computer Virus Research and Defense. Addison Wesley for Symantec Press, New Jersey, 2005. [2] Zakorzhevsky, 2011. Monthly Malware S tatistics. Available rom: http://www.securelist.colen/analysis/2047921821M0nthly_Malware_St atistics_June_2011 [Accessed 2 July 2011). [3] Adleman, L.: An abstract theory of computer viruses (invitedtalk). In: CRYPTO '88: Proceedings on Advances in Cryptology, pp. 354-374, 1990. [4] Filiol, E: Computer Viruses: rom Theory to Applications.Springer, Heidelberg ,2005. [5] McGraw, G., Morrisett, G.: Attacking malicious code: report to the infosec research council. IEEE Sotw. 17(5, pp. 33-41), 2002. [6] G. NychisOV. Sekar, D. G. Anderson, etc. "An Empirical Evaluation of Entropy-based Anomaly Detection" Proceedings of the 8th ACM SIGCOMM conference on Internet measurement" ACM Press, 2008,pp151-156. [7] D.Brauckho, B. Tellenbach, A Wagner, etc. "Impact of traic sampling on anomaly detection metrics." Proceedings of the 6th ACM SIGCOMM conference on Intenet measurement. ACM Press, 2006, ppI59-164. We used real time network traic rom National Advanced IPv6 Center of Excellence at the University Science Malaysia (USM). Throughout measrements remarkable of or the similarity experimentation, hih speed except for network a few the entropy traic peaks. show These exceptional entropy values represent the magnitude of traic featres' distributional variations dring the measrement period. We picked sample snapshots of time where peaks are observed, and investigated the network traic measrements. To validate the eiciency of our high speed network anomalies detection, we have injected anomaly network traic which is Witty Worm dataset of CAIDA [15] in the online network traic. -[..ry [8] ALakhina, M. Crovella, and C. Diot. "Mining anomalies using traic feature distributions". Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer. -U�ld -lW�th,t communications.ACM Press, 2005, pp217-218 [9] Tie h«on<hl Figre 2 : Examples of detected anomalies by our proposed high speed network anomaly detection system. Figre 2 shows the diferent changes before and dring the witty worm njection. We have injected the witty worm trafic at the 80th second. It can be seen that prior to the worm injection time, the entropy values of the network traic vary in a permitted scale, since sorce adresses and destination adresses do not follow repeated or speciic manner. However, ater the injection of the witty worm traic, it can be clearly seen that there is an obvious decrease n the entropy of the witty worm traic. This because the worm replicates itself and produces similar traic featres. Although the entropy values of the witty worm traic have low entropy values, we observed that most of the witty worm traic exceeds the upper threshold. VI. Yu Gu, AMcCalium and D.Towsley. "Detecting Anomalies in Network Traic Using Maximum Entropy Estimation" Tech. rep., Department of Computer Science, UMASS, Amherst, 2005. In https:llwww.usenix.org/events/imc05/tech/ ull_papers/gu/gu.pdf [10] A Wagner, and B. Plattner. "Entropy Based Worm and Anomaly Detection in Fast IP Networks". 14th IEEE International Workshops on Enabling Technologies: Inrastructure for Collaborative Enterprise, June, 2005.IEEE Press, ppl72-I77. [II] A. Beach, M. Modaf, Y.Chen, Network Traic Anomaly Detection andCharacterization. cs.northwestern.edu/-ajb200/anomaly%20detecti on%20paper%201.0.pdf [12] A. Lakhina, M. Crovella, C. Diot, Characterization of Network-Wide Anomalies in Traic Flows. Technical Report BUCS-2004-020, BostonUniversity,http://citeseer.ist.psu.eduI715839.htnl, 2004 [13] NLANR network traic packet header traces, http://pma.nlanr.netlTraces/Traces/long/credl200108101 available at [14] MAWI Working Group Traic Archive, available at http://tracer.csl.sony.cojp/mawi/samplepointB/2006/200601251400.hnl [15] The Witty Worm Http:// www.caida.org lanalysis 1 security 1 witty, 200 CONCLUSIONS n the paper we describe a system and presented a real time anomaly detection tecnique based on the notion of relative 260