Analyzing Tool for Radar Data
Transcription
Analyzing Tool for Radar Data
Analyzing Tool for Radar Data An approach to improve the analysis work PUSHPARAJ SILVA Master of Science Thesis Stockholm, Sweden 2006 Analyzing Tool for Radar Data PUSHPARAJ SILVA Master’s Thesis in Computer Science (20 credits) at the School of Computer Science and Engineering Royal Institute of Technology year 2006 Supervisor at CSC was Lars Kjelldahl Examiner was Lars Kjelldahl TRITA-CSC-E 2006:050 ISRN-KTH/CSC/E--06/050--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.csc.kth.se Analyzing tool for radar data An approach to improve the analysis work Abstract Analysis of radar data has a quite important role, not just in the work of extending the libraries, which are used in military aircrafts to discover and identify enemy radars and other similar threats. It has also an important role in the process of developing the advanced systems that discovers, identifies and distinguishes the great number of radars and similar threats in an environment. These systems have as a task to from a quantity of input data, containing information of pulses, correlate these to a number of unique emitters (i.e. reports of radars) to output data. A unique radar can, as its characterization changes over time, have a number of emitter reports. Each such emitter report is updated with the new information of the radar as it is discovered once again. With the purpose to improve the systems; the input and output are analyzed with the intention of gaining an understanding of how the system really performed their tasks. SaabTech already today utilizes a tool called ANNALYS. This tool is to a great benefit for the analysts as they easier can see relations between the output and input data, but also relations between the various pulses in the input data. Despite this unique tool it is quite time-consuming to accomplish certain analyses. One of the main problems behind this is that a function that can relate the output with input data does not exist, which forces the analysts to do this work manually. The difficulty of this correlation lies in principle in the lack of sufficient data, which directly can be used to relate the output with input data. We have therefore in this master’s project designed and implemented an automatically solution to this problem, which will make the analyzing much easier for the analysts. We have also in this master’s project discovered a relation between the data in the output, which have made it possible to relate different emitters originating from the same radar. This will make it easier to get an image of how a certain radar works over time. This kind of knowledge is crucial as it directly may contribute to a radically improvement of the identification of radars. Beside of have designed and implemented the above mentioned discovery we have also, with the purpose of simplifying the analysis work even more, designed and implemented some functionality as histograms, statistics etc. of the data. The above functionality has been implemented after a study of the current used analysis tool, and also after an understanding of how the data are analysed. This study has been done, knowing the needs that existed and that were a lack in the current application. Some studies have also been made of how the systems in the military aircrafts create the output from the input data. The resulting application of this master’s project has made it much easier for the analysts to analyze both the pulse data and the emitter data, not only separately, buy also jointly. Analysverktyg för radardata Ett försök att förbättra analysarbetet Sammanfattning Analys av radardata har en viktig roll, inte bara i arbetet med att utöka biblioteken, som används i militära flygplan med syfte att upptäcka samt identifiera fienders radar och andra liknande hot. Den har även en stor avgörande roll i processen med utvecklingen av de avancerade system som kan upptäcka, identifiera samt särskilja bland det stora antalet radar och liknande hot i en miljö. Dessa system har till uppgift att från en mängd indata innehållande information om pulser, korrelera dessa till ett antal unika emittrar (d.v.s. rapporter om radar) till utdata. En unik radar kan i och med att dess karakteristik förändras över tiden, ha ett antal emitter rapporter. Varje ny sådan emitter rapport uppdateras med den nya informationen om radarn i samband med att den upptäcks på nytt. I syfte att förbättra systemen så analyseras indata samt utdata, för att genom detta få en förståelse för hur systemen egentligen utförde deras uppgifter. SaabTech använder redan idag ett verktyg kallat ANNALYS. Detta är till stor hjälp för analytikerna, då de enklare kan se samband inte bara mellan utdata samt indata men även samband mellan de olika pulserna i indata. Trots detta unika verktyg är det väldigt tidskrävande att utföra vissa typer analyser. Ett av de stora problemen ligger i att det inte finns någon funktion som kan relatera utdata med indata, vilket leder till att analytikerna måste göra detta manuellt. Svårigheten i denna relatering ligger i princip i att tillräckligt med data saknas, som direkt kan relatera utdata med indata. Vi har därför i detta examensarbete designat och implementerat en automatisk lösning på detta problem, vilket kommer att förenkla arbetet stort för analytikerna. Vi har även i detta examensarbete upptäckt samband i utdata, vilket har gjort att vi har kunnat relatera olika emittrar från samma radar till varandra. Detta gör att det blir enklare att få en bild över hur en viss radar arbetar över tiden. En sådan kunskap är viktig i och med att det direkt kan bidra till att identifieringen av olika radar kan förbättras radikalt. Förutom att ha designat och implementerat ovanstående upptäckt har vi, för att förenkla analysarbetet ytterligare, även designat och implementerat funktionalitet som histogram, statistik etc. över data. Ovanstående funktionalitet har implementerats efter att en studie gjorts på det nuvarande analysverktyget, och även efter en förståelse av hur data analyseras samt vilka behov som fanns och som saknades i den nuvarande applikationen. Studier har också gjorts på hur systemen i de militära flygplanen skapar utdata av indata. Den resulterande applikationen i detta examensarbete har gjort det något enklare för analytikerna att analysera både pulsdata och emitterdata, inte endast var för sig, utan även gemensamt. Acknowledgements I want to express my appreciation to a number of people who helped in various ways in the work of this master’s project. First, I would like to thank SaabTech and especially Magnus Kamél and Jan Wennström for giving me this opportunity to do the master’s project there. I would also like to thank both my supervisors, Lars Kjelldahl at KTH and Peter Danielsson at SaabTech, for their effort of making this master’s project to be a success. I also want to thank Tomas Westerlund for contributing with information that made it easier to understand the current systems used at Saab Avitronics. Jan-Åke is also someone that I would like to thank. He contributed with Java classes that could plot pulses in a graph and also some guidance of how it worked. This made this master’s project somewhat easier, as I could concentrate on the main task of it. Inger Petterson contributed with tools and documents, which made it easier to work with the data and to get an understanding of it. Therefore, I would also like to thank her. Finally, I would especially like to thank my family; my mother Jacintha and my sister Dilukshini, for providing advice, encouragement, and support at various stages in this master’s thesis. My mother has through my whole life always been there for me; encouraging me, leading me in the right paths and given me strength in drawbacks. Without her I would never have come this far. Therefore, I would like to dedicate this thesis to her, as a symbol for my endless love and gratitude to her. Pushparaj Silva 2006-03-20, Stockholm Table of contents 1 Introduction................................................................................................................................ 1 1.1 Background ......................................................................................................................... 1 1.2 Problem ............................................................................................................................... 1 1.3 Goal..................................................................................................................................... 2 1.4 Purpose................................................................................................................................ 3 1.5 Method ................................................................................................................................ 3 1.6 Delimitation ........................................................................................................................ 4 1.7 Equipment ........................................................................................................................... 4 2 Electronic Warfare ..................................................................................................................... 5 2.1 What is Electronic Warfare................................................................................................. 5 2.2 Classification of Electronic Warfare ................................................................................... 6 2.3 Analysis............................................................................................................................... 7 3 Radar Basics............................................................................................................................... 8 3.1 Electromagnetic Waves....................................................................................................... 9 3.2 Modulation .......................................................................................................................... 9 3.2.1 Amplitude Modulation ............................................................................................... 10 3.2.2 Frequency Modulation ............................................................................................... 11 4 Electronic Support.................................................................................................................... 12 5 Pulse Processing....................................................................................................................... 15 5.1 Pulse Sorting ..................................................................................................................... 15 5.1.1 Prior Sorting............................................................................................................... 16 5.1.2 Subsequent Sorting..................................................................................................... 17 5.2 Pulse Descriptor ................................................................................................................ 19 5.3 Pulse Correlation............................................................................................................... 20 6 Analysis of Tools and Data ...................................................................................................... 21 6.1 Analysis of Existing Tools ................................................................................................ 21 6.2 Analysis of Data................................................................................................................ 23 6.2.1 vbd-data...................................................................................................................... 23 6.2.2 mvc-data..................................................................................................................... 23 6.3 Discovered Complications ................................................................................................ 23 6.3.1 Lack of Data............................................................................................................... 24 6.3.2 Fluctuations and Inaccuracies of Parameter Values................................................... 25 6.3.3 TOA Wrap-around ..................................................................................................... 25 7 Solution Proposals.................................................................................................................... 27 7.1 Emitter-to-Pulse Correlating ............................................................................................. 27 7.1.1 Solution One .............................................................................................................. 27 7.1.2 Solution Two.............................................................................................................. 27 7.1.3 Solution Three............................................................................................................ 28 7.1.4 Solution Four.............................................................................................................. 29 7.1.5 The Employed Solution.............................................................................................. 29 7.2 Statistical Solutions........................................................................................................... 30 7.2.1 Statistical Values........................................................................................................ 30 7.2.2 Filter ........................................................................................................................... 30 7.2.2 Histogram................................................................................................................... 30 8 Design and Implementation ..................................................................................................... 31 8.1 GUI.................................................................................................................................... 31 8.1.1 Graphs ........................................................................................................................ 32 8.1.2 Toolbar ....................................................................................................................... 34 8.1.3 Information Area........................................................................................................ 35 8.1.4 Access Keys ............................................................................................................... 35 8.2 UML.................................................................................................................................. 35 8.3 File and Data Handling ..................................................................................................... 37 8.3.1 Parsing of Files........................................................................................................... 37 8.3.2 Storage of Data in a Data Structure............................................................................ 39 8.3.3 Storage of Data in Database ....................................................................................... 42 8.4 Emitters-to-Pulse Correlating............................................................................................ 44 8.4.1 Filter ........................................................................................................................... 44 8.4.2 PRI Analysis .............................................................................................................. 45 8.5 Emitter Correlation ........................................................................................................... 48 9 Discussion and Conclusion ...................................................................................................... 50 9. 1 User Case ......................................................................................................................... 51 9.2 Future Work ...................................................................................................................... 54 References................................................................................................................................... 56 Appendix A ................................................................................................................................. 59 1 Introduction 1 Introduction 1.1 Background Saab Avitronics is a company that develops radar warnings systems to be used in military aircrafts. The radar warning systems have the task to warn the pilot, or the crew, of the threats that exists in the surroundings. The threats in this case are different kind of radar stations (enemies) that can detect this aircraft. The task of the radar warning systems is therefore to detect and measure signals from different kinds of radar stations and display the threats on a graphical unit. The signals come from electromagnetic sources and can either be pulses or continuous waves. The radar warning system can afterwards get some properties from the signal; these properties can be amplitude, frequency, direction etc. With this information a signal descriptor word (SDW) is created, which contain measured and derived pulse- and CW signal data. For pulses, this descriptor word is often called a pulse descriptor word (PDW). We have in this master’s project concentrated on pulses, and hence, the term pulse descriptor will be used in the remainder of this report. The received pulses can then be sorted in groups, called bursts, where each burst contains PDW: s from the same radar. A group of similar bursts represent the observed emitters, and each emitter gets a description. The radar warning system can afterwards, with the help of a library, identify these emitters using several algorithms. This process will be more elaborated in this master’s thesis. An essential part here is the process from the measurement to the identifying of the emitters. In the development work with this process, the measured data of the pulses and the emitters are used. These data are used to offline analysis. Since the size of the generated data is large, some tools have been made to make it easier to do the analysis work. These tools provide a variety of presentation capabilities to view specific raw data, for example: frequency, amplitude, pulse width, pulse repetition interval and other raw data. This will contribute to the review and analysis of the data. 1.2 Problem The figure below shows the process from receiving of the pulses to the graphical presentation on the radar warning display. At stage A, the signals are received and are an input to the video processor. The video processor1 measures different parameters of each pulse such as: amplitudes, frequencies, pulse widths, time of arrivals, etc. These measures are coded to digital form and are put together to a pulse descriptor word (PDW). The pulse descriptors are then sent to the pulse processor, which in turn arrange the pulses in order and create a so called active emitter file or “tracks” of the observed emitters. These tracks join a number of pulses to emitters, which are shown on the radar warning display. This figure and the process will be more elaborated in the master’s thesis. Data from A, B, C, D and E can be extracted from different flight tours. These data are then analysed, with the help of different analysing tools, by the personnel. This is done to see if everything in the process went as they should, but also to try to find out new correlations in the data. 1 Video-receiving unit. 1 1 Introduction Before this project started, the personnel had to switch between different tools if they wanted to compare the data between different stages. For example, if they wanted to compare the data from C with the data from B, two different tools had to be used. In the analysis work, the personnel try to see if there are any relations between the data, which before this project required some educated guesses. This work was time-consuming, and it could be difficult (and in some cases almost impossible) to see the relations. L Video Processor (VP) Signal A Amplitude, Frequency, Pulse width, Direction etc. are determined for each pulse. After this is done, a pulse descriptor word is cretaed for each pulse. y ar Puls Processor (PPY): B Pulse descriptors are sent. Pulses r ib Pulses are arranged in order. After this is done, an active emitter file is created (also called ”tracks”) consisting of the observed emitters. Radar Warner Computer (RWC): Aircraft Weaponsystems, Emitters, etc. are determined with the help of a library. C D E ”Tracks” (active emitter file) are sent. Pulses are arranged in order, and ”tracks” are created. Emitters Figure 1. The pulse handling process The main problem in this master’s project was, for these reasons, to improve the situation for the personnel that performs the analysis work. 1.3 Goal It is a known fact that the human has some limitations when it comes to directly see correlations between great quantities of data. Therefore, an immense problem is that it is difficult to see the relation between the data from the different stages: A, B, C, D and E (see Figure 1). Before this project, there existed an analysis program (called ANNALYS) that helped the analysts with their analysis work, but there were a lack of important functions in this program (like those mentioned in section 2). Saab Avitronics had therefore needs to further develop this tool with functions that showed new correlations between the data, and which thereby further would improve the situation for the analysts. According to the delimitations that were made (see chapter 1.6); we have in this master’s project only concentrated on studying the data from stage C and B, and tried to show the relations between these data. The second goal was to find out if some new program functions could be implemented that would make it easier to do the analysis work. An example was to look at the data from a statistical point of view. 2 1 Introduction The program that was the result of this master’s project will be referred as ANNALYS Ext2 in the remainder of this master’s thesis. 1.4 Purpose The purpose of this project was, as also could be read under section 1.3, to make it easier for the analyst with the analysis work. If new relations between the measured data can be found, it may contribute to the development of the radar warning systems. Therefore, it is of crucial importance to find more correlations between the data, so that the discovery, classification, identification and localization of the radar stations (the threats) will be more specific and correct. Another purpose was to analyse how the pulse processor – PPY (see Figure 1) related the different pulses to emitters. It could, for example, have occurred that the PPY split some pulses into two emitters even though it should only be one. By relating the data from stage C to stage B, it is easier to see which pulses that belong to each of the split emitters. Hence, it is easier to understand why PPY split the emitter to two emitters. Was it some pulse parameters that differed great between the two groups of pulses, which in turn built up the two emitters? Was it because of a big time difference between the pulse bursts? These and many similar questions may be answered with a function that can relate the data from the two stages. 1.5 Method The first step was to analyse and study the current analysis program (ANNALYS). This gave an understanding on how the current tools worked, what they did, what functions they had and what the functions did. The second step, which also went somewhat hand in hand with the previous step, was to look into and study the data. Here, we tried to find out if some relations between the emitters and the pulse data could be found. A question we had here was if the data from stage C (the emitters) directly could point out their origin pulse data at stage B (see Figure 1). At this stage, we also studied the kind of algorithms and mathematics that could be used to accomplish this. Many of the questions we had were answered at the above stage, which was to a help in the progressing of the next phase; the designing and implementation of ANNALYS Ext. The implementation was done in Java, with the help of the development tool Eclipse. We first implemented the new relating properties that were found in the previous stage. Afterwards, some new program functions were designed and implemented. After gaining the important understanding of the tools at the first stage, it was easier to look into what functions that seemed to be needed. At that stage we found out the existing functions that were not satisfied by the analysts, in the sense that those functions did not support the analysts as much as they could. We also discovered some new functions that could be implemented to satisfy the analysts and help them in their complicated work of analysing. To confirm that everything worked as it should, some testing was also done. During the whole development process tests were done using fake data. However, to make sure that the program worked as it should with real data; test data of flight tours had to also be used. This final test was made by the supervisor at SaabTech (Peter Danielsson), as these data were confidential. The last phase of this thesis project was to write a thesis report. Notes was taken during the whole thesis project, which was to a help when writing this thesis report. 2 Which stands for ANNALYS Extension. 3 1 Introduction 1.6 Delimitation Some delimitation was made in this master’s project, so that it would not become too extensive or take too much time. These delimitations were: • This master’s project emphasized on the analysing work and tried to find the relations in the measured data (algorithms that can relate the data etc.). • This master’s project only concentrated on pulses (pulse radar). • The computer program ANNALYS Ext, that was the result of this master’s project, had not as a requirement to work as a complete application. • The time limit in this master’s project limited the study to only to look into the relation between emitters and pulses (data between stage C and B, see Figure 1). • Manuals of the Pulse Processor (PPY) was analyzed, but due to the time limit, this analyze/study could not be extensive. 1.7 Equipment The equipment that was needed to perform this master’s project was first of all a PC to perform the programming. The Java development tool that was used for this is called Eclipse3. The analysis tool (called ANNALYS), which was used before this project was initiated, was also needed. These tools were installed only on some computers; hence, access to these computers was also needed. 3 www.eclipse.org 4 2 Electronic Warfare 2 Electronic Warfare This chapter will shortly describe what electronic warfare is, and why it is so important in today’s warfare. This will give a basic background on what the thesis report is about. Hence, it will be easier to comprehend the remainder of the report as it will be easier to understand to what, and to where, the result of the thesis will be used. Electronic warfare is a large concept that includes several areas. These warfare methods can be used to both look for enemies and to actively affect the attacker and also to protect the own systems and functions [18]. In warfare it is crucial to have knowledge in this area, which directly affects the result of a war. 2.1 What is Electronic Warfare Electronic Warfare is defined by Josefsson and Berggren [18] as: Electronic warfare is a summarized term of military measures to discover, exploit, affect, complicate or prevent the enemy’s use of electronic measures, which takes advantage of electromagnetic wave propagation, and also own measurements to reduce the effect of the enemy’s electronic warfare. The purpose with electronic warfare is defined as the following: [18] The purpose of electronic warfare is to reduce the effects of the enemy’s systems and to maintain the effects from the own corresponding system independent of the enemy’s electronic warfare. This can be understood in a more technical way as the activity where one, by using electromagnetic energy, affects the enemy with the intention to reduce the effects of there use of electromagnetic energy. [18] The figure below illustrates the relation between the utilizing of electronic measures between the enemy and us. Figure 2. The relation in the use of electronic measures [18] 5 2 Electronic Warfare As can been seen in the figure above, there are also possibilities to prevent the enemy to use their electronic measures. An example is to block an auto programmed jammer4 with broad band noise jamming5. [18] The goal of electronic warfare is: [18] • Get information of the enemy; for example, about his command and control warfare system and how it is used. This information can then be used as a foundation for the own tactical/operational plan, e.g. how to use counter measurements against these systems. • Make it harder for the enemy to use his practiced tactical behaviour. • Secure the own command and control warfare ability. • To increase the effect of the own weapon effort. On a warfare technical level, the electronic warfare is an important, if not a crucial, support for the survivor of our weapon platforms6. 2.2 Classification of Electronic Warfare Electronic warfare can be classified into three parts: [18] [24] • Electronic support (ES), where electromagnetic energy is used to search for, discover, identify and localize electromagnetic sources. • Electronic attack (EA), where electromagnetic energy is used to reduce or destroy the enemy’s system functions and warfare capability. • Electronic protection (EP), which are measures to reduce the effect of the enemy’s electronic warfare. This is also measures to avoid electromagnetic conflicts. The figure below shows a further classification of electronic warfare, but a more elaborated discussion of its elements will not be made in this thesis. Electronic Warfare (EW) Electronic Support (ES) Transmitters Electronic Protection (EP) Electronic Attack (EA) Emissions Radio Optronic Radar Navigation Electronic Support Measures Electronic Attack Measures Figure 3. Classification of electronic warfare [18] [24] The result of this master’s project will gain the electronic support, by making it easier to do the analyzing work, and thereby benefit electronic attack and electronic protection. The figure below shows the course of events in tactical electronic warfare. As said before the result of this master’s project, ANNALYS Ext, will be a benefit to the analysis part, by finding new properties of the measured data and new functions that does not exist in the currently used 4 The deliberate radiation, reradiation, or reflection of electromagnetic energy with the object of impairing the use of electronic devices, equipment, or systems by an enemy. [14] 5 More information about these techniques can be found in the references. 6 These can for example be; JAS Gripen, F-18 etc. 6 2 Electronic Warfare analysis program (ANNALYS). This will help the human operators so that they more easily can see correlations in the measured data. Hence, it will lead to a further development of the electronic warfare methods at SaabTech. Figure 4. The cycle of electronic warfare [18] 2.3 Analysis The main purposes of the analysis during warfare are to give a picture of the enemy’s network structure, i.e. which station is the main station respective substations in the different networks and where they are located. One also wants to find out where there are accumulations of stations and thereby getting a picture of where the staffs are located, and what military level the respective staffs has. Also other conclusions can be drawn from the analysis. [18] Important aspects in the analysis of the signals are: [18] • Traffic directions • Traffic intensity • Traffic behaviour • Signal type (fingerprints) • Frequency using: centre frequency, bandwidth, modulation etc. • Band spread: signal strength etc. • Call signals • Voice recognition • Textual contents • Human mistakes The information that the analysis phase contributes with will be used as a basic data for decision-making. The decisions to be taken can be new efforts of electronic attack, etc. 7 3 Radar Basics 3 Radar Basics This chapter will shortly describe what radar is and give a basic description of how it works. An introduction to concepts, parameters etc. that will be used in this master’s project will also be given. The word radar is an abbreviation for the expression “Radio Detection and Ranging”, which is a technology where electromagnetic waves are used to discover an object and to determine certain properties of it. The electromagnetic waves that are transmitted will reflect back from the object (an echo) and be received by a radar system. A number of properties can then be measured from the echoes. For example, by measuring the time difference in this course of action, the distance to the object can be determined. The velocity of the object can be determined by finding out the difference in frequency between the signal that is transmitted and the received one (the so called Doppler frequency). There is much other information that can be found out, which will be discussed in later chapters. The frequencies that are used are usually around 1-30 GHz (so called microwaves). [4] [24] There are two kinds of radars: pulse radar and CW (continuous wave) radar. The pulse radar can measure the angle and distance to the target. CW radars transmit a continuous wave and can in its original version measure angle and velocity on the target, but can not measure the distance. However, today there are variants of CW radars that can measure distances too. [24] The most used type of radar is the pulse radar. As the name reveals, microwaves are transmitted as pulses. The waves reflect against objects and are returned as an echo. To be sure to discover an object it is required that it is hit by several pulses. Hence, the radar continuously transmits pulses with a certain time interval, which is often called the pulse repetition interval (PRI) and is measured in seconds. A different term for this, which is sometimes used, is pulse period (T). Another parameter that is used is the pulse repetition frequency (PRF), which indicates the number of pulses transmitted every second (1/PRI). These parameters are illustrated in the figure below. [24] Pulse Width Listening time Pulse Repetition Interval Figure 5. Illustration of different pulse parameters [24] The pulse width is the time length of the pulse. Usually the pulse width, noted τ, is in the order of one thousandth of the pulse repetition interval. The value range of the pulse width reaches from a fraction of a microsecond to several milliseconds. The pulse repetition interval is defined as the time from the start of the pulse to the start of the next pulse; this can be seen in the figure above. The term listening time is the time in which the receiver can gather information from the reflected pulses (echoes). [24] An observation should be made that the radar can change the PRF during the transmission of the pulses. PRF switching is when the PRF is changed after that a certain number of pulses have been transmitted. The other technique of this is that the PRF is changed between each pulse. These two techniques will be more elaborated in chapter 4. [24] 8 3 Radar Basics 3.1 Electromagnetic Waves The field intensity on electromagnetic waves varies in sinus form. The distance between two points with the same phase in two consecutive waves, a period, is defined as the wave length (λ) and is measured in meters. The figure below illustrates this. Figure 6. A wave length [24] The number of waves per second is defined as the frequency (f), and has the unit Hertz (Hz). 1 Hz = 1 period/second. The relation between wave length, frequency and velocity is: v = λ⋅ f [m / s] The velocity of electromagnetic waves in vacuum is 300 000km/s, which is noted as c (the speed of light), so the above formula would be: [18] [24] [27] c = λ⋅ f [m / s] 3.2 Modulation The signals that are transmitted from radars are always modulated, if they are to contain any useful information. In practice, there are three main ways to modulate a signal: amplitude modulation (AM), frequency modulation (FM) and phase modulation (PM). The two lastmentioned methods are actually variants of each other, as a change in frequency also will change the phase. Therefore, the below subsections will only elaborate the two firstmentioned modulation concepts. [18] [24] A radar transmitter transmits a carrier wave with a stable frequency, which is shown in the figure below. Figure 7. Carrier wave. [24] In Figure 8, the information carrying sinus wave is shown. This wave will modulate the carrier wave in different ways: [24] 9 3 Radar Basics • The amplitude is changed to get an amplitude modulated signal (Figure 9) • The frequency is changed to get a frequency modulated signal (Figure 10) Figure 8. The modulating signal, i.e. the information. [24] Figure 9. Amplitude modulation. [24] Figure 10. Frequency modulation. [24] 3.2.1 Amplitude Modulation An ordinary radar station that transmits pulses is amplitude modulated. When using amplitude modulation the amplitude of the transmitter’s carrier wave is affected, so that it is varied at the same rate as the signal strength. [18] Figure 9 shows a signal, which is a combination of two frequencies. A carrier wave having the form: Ec ⋅ sin ωc t where ωc = 2πf c and a modulated wave, which is carrying the information, having the form: E m ⋅ sin ω m t where ω m = 2πf m The amplitude (A) for the modulated signal would then be: A = E c ⋅ sin ωc t + E m ⋅ sin ω m t ⋅ sin ωc t In all the above three equations the variable E is equal to the top value of the wave, ω is the angular velocity and t stands for time. The variable c stands for the carrier wave, and m stands for the message wave (the modulated signal). [24] 10 3 Radar Basics 3.2.2 Frequency Modulation The second alternative to modulate a signal with information is with frequency modulation. CW radars often use a FM modulated signal to be able to measure the distance to an object. Figure 10 shows an example of a frequency modulated signal, where the carrier wave sometimes is pushed together (the frequency increases) and pulled apart (the frequency decreases) when the carrier wave has high respective low amplitude. [24] Frequency modulated signals can be described with the formula: A = E sin ωc t + 2πΔf sin ωm t ωm where ωc = 2πf c ωm = 2πf m Here, the variable E is equal to the top value of the carrier wave and Δf is the maximum frequency deviation. The variable c stands for the carrier wave, and m stands for the message wave (the modulated signal). [24] As can be noted the amplitude (A) of the modulated signal (the information) determines the size of the carrier wave’s frequency deviation. It can also be noted that the modulated signal’s frequency (fm) determines the velocity of the carrier wave’s frequency deviation. [24] 11 4 Electronic Support 4 Electronic Support This chapter contains a short description of the information that is received by the radar warning system, which is the information that the Video Processor has extracted from the pulses. These data can then be exported from the radar warning system at stage B (see Figure 1) for analysis. An explanation of pulses and their characterizations will also be given in this chapter. Electronic support measure is the term for detecting and analysing radar signals. The information from this process will be the basis for the development of electronic warfare and protection. This information is used to build up a signal threat library (also shown in Figure 1), which later on will be used in radar warnings systems of military aircrafts etc. The great amount of signals that are received by the receiver of the radar warning system, from the surroundings, is arranged in order by separable parameters. These parameters can be extracted from each pulse, and are stored in a so called pulse descriptor word. The primary pulse parameters, which are extracted, require only a single pulse. These parameters are: • Time of Arrival (TOA) • Pulse Width (PW) • Frequency • Amplitude • Angle of arrival (AOA) – Also called direction of arrival (DOA) The most useful parameters for arranging the pulses in order are: frequency, angle of arrival and pulse width. Another parameter that is available, and which will be used in this master’s project, is the Time attribute. This attribute indicates the exact time for the pulse arrival. The difference between Time and TOA is that TOA is a counter with a wrap-around at its maximum value, which will lead to a problem discussed in section 6.3.3. With two or more pulses also the pulse repetition frequency (PRF) can be determined. [2] [24] The frequency is not always stable, as it can be varied in different ways. It may be increased or decreased from pulse to pulse. It may be changed in random or in some specific pattern. It can also be changed in some prescribed pattern during each pulse, which is called intrapulse modulation. [27] More details above the above stated parameters will be elaborated in section 5.2. In some systems and modes7, the PRF is a constant; in others it varies. A constant PRF is shown in Figure 11, where each bar is a pulse. It should be noted that a system with constant PRF also can be varied unintentional. This can occur because of different kind of disturbances from the surroundings. [2] Figure 11. Constant PRF. [8] The reason for the variation of the PRF is to prevent sampling and integer multiples of Doppler shift. If this occurs, the radar senses a Doppler shift of zero and is blind to targets moving at these velocities (called blind velocities). PRF variation used with the intention to 7 Each platform has different kind of modes of the weapon systems. Example of modes can be search mode, locked-on mode etc. 12 4 Electronic Support prevent blind velocities is called PRF staggering, and of which the pulse interval is changed for each certain number of pulses. These pulse intervals are often, depending on where it is used, recurrent after a certain number of pulses, where each sequence is called a burst. [2] [8] [24] Example: 1024 pulses are transmitted with the same interval at which the interval is decreased or increased. This is repeated 6 times, always with different intervals, until it is returned to the same origin interval and to start over again etc. The PRF may also be varied to prevent jammers from locking on to the radar. This technique is called PRF jittering, where the pulse intervals are random for each pulse. Jittering makes it harder to analyse the radar pulses. [8] This PRF varying technique is shown in Figure 12. The reader can find more information about these techniques in the referenced books. Figure 12. PRF Jittering. [8] Some systems use burst waveforms, where groups of pulses are transmitted. The time interval between each burst group is called burst repetition interval (BRI). Figure 13 shows how burst waveforms can look like. The rate within and between the groups may change. [8] Burst Repetition Interval, BRI Figure 13. Burst waveform. [8] There are different kind of BRI types, which gives different values on the BRI and different shapes on the waveforms. The reason for this is that the radar can scan in different ways. These types are listed below: [8] [30] • Sector: Scans from right to left in a definite sector. This can be observed if the electronic support measures system is positioned at one edge of the radar scanner sector, which will result in that the BRI value is less in the shorter edge and greater in the other. For example; 7s on one edge and 1s on the other, hence, the BRI changes from 7s to 1s and then back to 7s and again to 1s etc. The figure below illustrates this event: Figure 14. Sector scan [7] • Circular: Scans in a circular area of 360º. This can be observed by noticing that BRI always has the same value. For example it is always 8s. • Raster: Scans in a pattern shown in the figure below, and then repeats the same pattern over and over again. 13 4 Electronic Support Figure 15. Raster scan [7] It should be pointed out that also the BRI can, theoretically, be changed according to the techniques mentioned above about the PRF. [2] The amplitude is often varied between the pulses in a radar scan, if we are not dealing with a radar that has locked on to us. The reason for this is that when a radar scans through a sector, the amplitude is at its greatest when we are in the same direction as the direction of the radar scan. When the radar has another direction, pulse signals may still be received, which is pulses from the side lobes. The result of this is that the amplitude values of these received pulses is less than of those received when we were in the same direction as the radar scanning. More information about this can be read in the referenced books. Every type of radar has different characteristic parameters. The more parameters that can be intercepted and determined, the higher is the possibility to determine the type of radar. It is also important to determine which mode a radar currently is using. Radars in search mode are a less critical threat than radars that have locked on to the aircraft. Radars usually have a great number of different modes. Therefore, it is important that different combinations of pulse width, PRF etc. are stored in the threat library, and thus making it easier to determine which threat is the most critical one. [24] 14 5 Pulse Processing 5 Pulse Processing It is important to understand how the pulse processor does its work, and thus, making it easier to understand the relationship between the data at stage B and the data at stage C (see Figure 1). Therefore, this chapter will briefly describe how the pulse processing is done in Saab Avitronics; how the pulses are arranged in order into bursts, and then to tracks to the final creation of the so called active emitter file. Some problems that can occur in this sorting process will also be discussed, to finally end this chapter with an elaboration of the pulse descriptor and its parameters. The radar warning system will receive a great number of pulses from a large number of radars. It could be up to millions of pulses each second. All these pulses must somehow be processed by the radar equipment. When the pulses have been received by the radar equipment, different parameters can be measured from each pulse. These are the parameters that will be used as a foundation in this master’s project. The measured values are afterwards coded to digital form and put together to a pulse descriptor word (PDW). These descriptors are stored in order by time, which means that the descriptors will not be grouped by radar stations. [2] However, a grouping in this manner of the descriptors is important; therefore the next subsection will elaborate the process of doing this, which is called pulse sorting. This is the process of creating emitters of pulses. The descriptors will thereafter be discussed in chapter 5.2. To group the different descriptors, and thereby also the pulses, to their respective emitters, a number of association measures are used. These association measures, which are probability measures, are used as guidance in the association process of the descriptors to the emitters. [19] The direction of arrival is generally regarded as the best initial sorting parameter as it cannot be varied by the radar from pulse to pulse. The pulse width, on the other hand, is not a good sorting parameter. Even if the transmitter has a constant pulse width, it can be measured with large differences in value of different pulses. [2] [9] The time of arrival value for a pulse has none at all sorting value observed as an individual value. It is not until there are a collection of pulses and that pulse trains have been found, which constitutes of a regular or almost regular pattern, that the pulses can be related and thereby the pulses can get sorted. [2] The frequency is a fine sorting parameter, but even though the total frequency area is large, there is a tendency that certain radar types are congregated in a relatively small frequency area. Another disadvantage is that the frequency relatively easy can be varied by means of frequency hopping. [2] Pulse amplitude does not reveal anything about the radar that the pulses originated from. However, by creating amplitude differences between pulses it is possible to analyse relations among pulses. The analysing is based on the probability that consecutive pulses have about the same amplitude. Also here, variations of the amplitude can occur between pulses through affects of wave propagation by using frequency hopping. [2] 5.1 Pulse Sorting Pulse sorting is the measure used to arrange the pulse descriptors in a collected description, called emitters, which is labelled as an active emitter file (AEF). This sorting is done in two steps: prior sorting and subsequent sorting, where different sorting algorithms are used for the different steps. The figure below shows a simplified picture of the sorting process. 15 5 Pulse Processing Amplitude Intercepted Pulses Sorting algorithms Time Sorting algorithms Update of measured parameters Update of measured parameters Update of measured parameters Figure 16. A simplified figure of how pulse processing is done. [28] 5.1.1 Prior Sorting The first sorting algorithms applied, groups the pulses by: Angle of Arrival (AOA), Frequency (FQ) and Pulse Width (PW) and also with some support from the TOA information. This phase of the sorting process is called the prior sorting. The used algorithms divide the pulses into cells (also called pigeon holes), where each cell, ideally, only contains PDW: s with same origin and in the same time range that is represented by the cell. After this is done, a so called burst file, i.e. group of pulses, is created. The figure below illustrates this concept. [2] [28] Frequency A Cell Pu ls h idt W e AOA – Angle Of Arrival Figure 17. Sorting by FQ, PW and AOA. [2] When the first PDW arrives to the prior sorting it results in that a new cell is created and some range limits is given to its dimensions. The pulse descriptors arriving after this initial step are stored in this cell, only if their AOA, FQ and PW values are in range for the cell. If this is not the case, a new cell is created for this PDW. This is only done if the range limits of the new cell are not within the limits for other cells. If this is, however, the case, two or several cells are joined together and this newly created cell gets new range limits after this process. [2] 16 5 Pulse Processing Frequency The prior sorting can give arise to some problems, one of the problems occurs when the signals comes from a frequency hopping radar. This problem is illustrated in Figure 18. Pu ls h idt W e AOA – Angle Of Arrival Figure 18. A prior sorting of signals from a frequency hopping radar. [2] As one can see, the right cells in the figure above have not been merged. The reason for this is that enough pulses have not been received to be able to combine these cells. Therefore, it is crucial that there exist a superior analyse function with an ability to discover similar contents in different cells, and that afterwards can take a decision to merge these cells. [2] There exist several more problems in the prior sorting that leads to a scattering of the PDW: s in different cells and also that PDW: s end up in wrong cells. These problems will, however, not be discussed in this master’s thesis. 5.1.2 Subsequent Sorting When the cells have a minimum of five and a maximum of 100 PDW: s with the largest time difference of 50 ms, the cells are ready to be sent of to the subsequent sorting. Here, several sorting algorithms are used that sorts the bursts by: Angle of Arrival, Frequency, Pulse Width, Pulse Repetition Interval and Amplitude. This will result in the creation of the file called active emitter track file. This active emitter file will then be used by the radar warning computer to identify the discovered objects. [2] [28] As can be seen, some of the parameters used in the sorting process in the prior sorting are also used here. Although this can be perceived as strange or overdo, it has it simple explanation; as the pulses are passed through different algorithms, better statistical measures can be found. This will result in that more efficient and intelligent estimations can be done to get a correct sorting of the pulses. Amplitudes can give rise to some problems in the sorting, as it may be difficult to analyse if the amplitudes of the pulses in the same cell should be divided into two distinctively groups or not. The reason for this difficulty arises if each pulse is followed by another pulse with lower effect. This can either mean that we are dealing with two different emitters (radars) or only one emitter, of which the case then would be that the pulse with the lower effect is merely a reflected pulse. [2] The reader can find more information about this in the referenced books. 17 5 Pulse Processing The PRI sorting of the pulses in each cell can also be somewhat problematic. Figure 19 shows histograms of how the PRI: s can look like with no errors. The errors can occur when it have to deal with missing pulses. This can lead to an estimation of pulse repetition intervals that is longer than normal. This is illustrated in Figure 20. [2] Figure 19. PRI histograms when no errors have occurred. [2] Figure 20. A PRI histogram illustrating when pulses are missing in a cell. [2] It can also have occurred that some pulses have incorrectly been placed in wrong cells. Hence, some extra pulses exist in some cells. This is illustrated in the figure below. [2] Figure 21. A PRI histrogam illustraing when there are extra pulses in a cell. [2] A stagger analysis is done to find out the burst interval (BRI) in the case of PRI staggering8. In this analysis the TOA difference between every n: th pulse is calculated and compared. As a start n is set to two (every other pulse) and if not all the TOA differences, except a few, 8 This has been discussed in chapter 4. 18 5 Pulse Processing resides in a narrow interval, the value of n is increased with one unit until the TOA differences fulfils the above criteria. The sequence of pulse intervals within the burst can now easily be determined. If no value of n that fulfils the criteria above can be found, either the interval is to narrow or it is not a staggered signal but a jittered one. [2] To find out the PRI for a signal that is jittered, usually an approximated value is calculated as: PRI ≈ TOAn ⋅ TOA0 n As can be observed by the above formula, the PRI value only depends on the first and the last TOA value, and it is not at all taking intermediate TOA values into consideration. Therefore, a better estimation is to use the method of least squares, in which case a PRI value, PRI0, is obtained by: ∑ [TOA ⋅ (t n σ TOA = i =0 i + i ⋅ PRI 0 )] 2 0 n +1 = minimum This method considers all the measured values and is therefore to prefer. [2] [21] The figure below illustrates this. With the knowledge of PRI0 a standard deviation value can easily be calculated. Figure 22. Thirteen pulses from a jittered sequence of pulses.The line represents PRI0, which is the mean pulse interval obtained by using the method of least squares. [2] 5.2 Pulse Descriptor The pulse description words contain the parameters mentioned in chapter 4. Apart from these pulse parameters, the pulse descriptor can also contain a number of flags, each with a size of a few bits (often only one), indicating special circumstances, e.g. that a certain intrapulse 19 5 Pulse Processing modulation has been detected. Details of these flags will not be mentioned for confidential reasons. [1] [2] Below is an example of how a pulse descriptor can look. Only an example can be given as this information is confidential at SaabTech. These pulse descriptors have a size of 64 bits. The table indicates the number of bits needed for each parameter in the pulse descriptor and with information of their resolutions and the measurement areas. Parameter Resolution Measurement area Number of Bits TOA 0.1 μs 1.7 s 24 AOA 0.7 º 360 º 9 Frequency 1 MHZ 2-18 GHz 14 Pulse Width 50 ns 100 μs 11 Amplitude 1 dB 60 dB 6 Size of Pulse descriptor 64 Table 1. Parameters in pulse descriptors. Their resolution, measurement area and number of bits required to store them. The numeric values indicated are only examples from [2] as these are confidential. 5.3 Pulse Correlation To get a rapid comparison of which PDW: s that match to existing track files, the PDW data is hashed into an integer number that summarizes the key radar parameters. All the track files whose hash values are identical to the PDW’s hash value are considered as possible matches. If there are multiple matches, the Angle of Arrival is used to resolve the matching conflict. Hence, if the current system offers a high Angle of Arrival resolution, the ambiguities in matching new data with existing track files will be eliminated. After the hash has been matched with a track file, the track file is updated with the new data. [29] If an exact match is not found, the hash value is checked for a close match. A close match could be the result of noisy or corrupt data, or a parameter altering threat. Track files that do not contain similar key radio frequency characteristics are eliminated. Also here, the parameter Angle of Arrival is used to eliminate ambiguities in matching the PDW with a track file. [29] If no match can be found for the PDW then either a new radar from a new threat or a mode change from an already identified radar has been detected. A new track file is therefore created and the PDW is copied into it. [29] 20 6 Analysis of Tools and Data 6 Analysis of Tools and Data This chapter will discuss the current used analysis program, called ANNALYS; how this program and its tools work. There will also be a discussion of the data; what format they have, how they are stored etc. The findings, which can be read about in this chapter, will give guidance to the design and the implementation of ANNALYS Ext. This chapter will finally conclude with an elaboration of the problems that will or may arise, and which should be taken into consideration when solving the problems in this master’s project. The first that was analysed was the current used ANNALYS program. The findings here simplified the construction of ANNALYS Ext, both graphically and structurally. A study was made of how the current program was built up. What functionality and graphics of the current program that also could be implemented in ANNALYS Ext, in their existing form or somewhat improved. Also, some other discoveries were made during this phase; what functions and statistical information that were missing in the current program, when considering the needs of the analysts. The chapter will then continue with describing the data and the data format. This is sufficient in order to comprehend the structure of the input files in ANNALYS Ext, but also to get knowledge of the kind of data that can be found in these files. 6.1 Analysis of Existing Tools The analysis of the tools in ANNALYS showed that it today is possible to make some comparisons of different parameters of pulses in the data at stage B (see Figure 1), called vbd-data. This data is stored in a specific binary format and can be opened in ANNALYS. A more detailed discussion of this data will be given in chapter 6.2. There is a tool in ANNALYS that is used to analyse the pulse data, at which different parameters of the pulses can be plotted graphically. The existing analysis possibilities are that plotted radar scans9 respective pulses can be zoomed in, where the information from each pulse can be retrieved by clicking on it in the graph. These graphs can be shown below each other, with plots of the different parameters over time. A zoom in on one of the graphs also zooms in the other graphs as much, which simplifies the analysis work. This zoom in functionality is something that also could be to a great usefulness in ANNALYS Ext. However, the two plotted graphs of emitters respectively pulses will not have an affect on each other, i.e. a zoom in on one of the graphs will not affect the other one. The reason for this is that an intuitive way to design this does not exist, as two different data sets are displayed in the graphs. There is also a window that shows the information10 textually about the different pulses, where the information from each pulse is in one row and where they are ordered by time. This makes it easier to see if there are any correlations between pulses that are adjacent in time. This information can later on be used to filter the pulses. When a pulse in the graph is clicked on, the row with the information of the pulse is selected. A window of this kind could also be implemented in ANNALYS Ext to make it easier to read the information of each pulse. 9 A transverse or swept sector or volume of airspace with a recurring pattern, by means of a controlled directional beam from a radar antenna [14] 10 The information that is listed are; Frequency, Amplitude, Pulse Width etc. 21 6 Analysis of Tools and Data Another function that is available is that the user can filter the pulses by setting some limits on the pulse parameters11 so that only the filtered pulses are shown in the graphs. A filtering leads to a smaller selection of pulses are shown, which makes it easier to see correlations between pulses. Example: First a filtering is done by AOA, which leads to the finding that the frequency for this radar scan is always in a distinct interval. Hence, a filtering on these values in frequency is done. This results in that also other radar scans from the same radar but at different points in time can be seen. All these radar scans is called tracks. A filtering function is also something that could be implemented in ANNALYS Ext, where subsets of the pulses can be seen in the graph using this function. However, the number of parameters that can be filtered on is more limited than those available in ANNALYS. There are, however, no possibilities to see different statistical values on the data in ANNALYS. Examples of the kind of statistical values that could be of interest to see are; how often does a radar change the frequency in a radar scan, between which frequencies does it hop etc. Hence, this could be a possible implementation of a program function that would simplify the analysis work. Another tool in ANNALYS is used to analyse the emitter report data, called mvc-data, which is the data at stage C (see Figure 1). A description of this data will be given in chapter 6.2. Each emitter report consists of information from one radar scan. Hence, a radar emitter often, if not always, has several emitter reports. The emitter reports for an emitter are connected through a variable called PPY track number. Today, a function that graphically shows all the pulses that belongs to an emitter does not exist. Therefore, another implementation of a program function that could be in the scope of this master’s project is a function that when a emitter report is selected, automatically finds out all this emitter’s emitter reports and show all their pulses. This would simplify the analysis work, by making it easier to see how often radar scans from an emitter repeats and evolves, and also how the radar parameters are changed for each scan. In this scenario, as well, some statistical values can be helpful. Example of some statistical values in this case could be; how many pulses that, on average, constructs the chosen emitter report’s emitter and the mean value of a pulse selection’s Frequency, Amplitude and PW. The information from each emitter report can be analyzed in a so called Browser, where it is possible to browse forward and backward among the emitter reports. An alternative implementation of this functionality could also be done in ANNALYS Ext. Instead of a browser, the same design as showing the information of the pulses could be used to show information of the emitters; a window that shows the most important information in each emitter report row wise ordered by time. There also exists a function where all the emitter reports can be seen in a map over time, as a video sequence. The map shows what path the ESM system went, and where the emitters where discovered. This is also something that simplifies the analysing work, by making it easier to see which emitter reports actually could be the same emitter when knowing their direction and position. Also here, it is possible to zoom in and to filter by different emitter parameters12 and thereby to make a selection of what emitters to be shown in the map. Even though this is something that is desirable, it will not be implemented as a part of this master’s project. The reason for this is that a map function is too difficult and too advanced to be in the scope of it. 11 Example of pulse parameters that can be filtered by are; Frequency, Amplitude, Pulse Width, AOA etc. 12 Examples of parameters are; mean value and deviation of PRI and PW. Other parameters are; Amplitude minimum and maximum, AOA, Emitter record no etc. 22 6 Analysis of Tools and Data 6.2 Analysis of Data This subchapter will describe the format of the data input files that will be used in this master’s project. As this information can be sensitive, only an essential description will be given. 6.2.1 vbd-data The pulse data is stored in a data file called vbd. This file is in a binary data form, which makes it necessary to parse it in a certain way with bit operations, scaling etc. to correctly understand it in an application. To not have to do the same procedure again and as the program ANNALYS Ext will be used alternately with ANNALYS, the vbd file has to first be parsed in ANNALYS and then extracted to a txt file containing the pulse information in an easier readable manner. To extract the values to a txt file, a report format file (.rpt) have been created in this project, and which states the structure of the output and the data that should be in it. This report format file is an input to ANNALYS, which then will export the data according to this file. 6.2.2 mvc-data The data that can be extracted from the radar system is called spr data. This file contains a combination of the data from stage C and stage D (see Figure 1). However, the aim with this project is to work with data from stage C, which is the mvc data. Hence, a tool called sprtomvc developed by SaabTech will be used. This tool can extract the mvc part of the spr data file and create a mvc binary data file. Also here, it is unnecessary to parse the mvc binary data and to deal with bit operations, scaling etc. Therefore a tool called mvctool will be used, which also have been developed by SaabTech. This tool will parse the binary file and create an easier readable txt file consisting of the information needed. The mvc-txt file contains information of different parameters of each track. The available data at stage C (see Figure 1) for emitters are Amplitude and TOA (time-ofarrival) for the first, last and max-amplitude pulses. Also, mean and standard deviation for Frequency, PW and PRI13 (Pulse Repetition Interval) for the tracks as well as for all the bursts are available. A TOA vector and a BRI14 (Burst Repetition Interval) vector are also available data at stage C. These vectors contain only information for the last 64 pulses. Therefore, it will be difficult to relate the emitters to the origin pulses. For continuous waves (CW) the data that can be found at stage C are: Amplitude and TOA. As have been said, each track has information of the mean and standard deviation value for the parameters Frequency, PW, PRI, BRI etc. Hence, a statistical value that can be of interest is how much that differs between these values from different radar scans (i.e. different tracks/emitters) from the same radar. These values can also be to an importance when finding out which pulses that belong to a certain emitter. When knowing the mean value and the deviation value, these parameters can, yet, be used even though they can change or fluctuate between pulses. 6.3 Discovered Complications In this subchapter, discussions of what problems that can arise, and be to a difficulty when solving this master’s project are given. These problems have been discovered during the analysis phase of the tools and the data. 13 14 The time-interval between each pulse. The time-interval between each burst. 23 6 Analysis of Tools and Data Amplitude According to [14] a radar scan means “a transverse or swept sector or volume of airspace with a recurring pattern, by means of a controlled directional beam from a radar antenna”. An example of this is that a radar scans from right to left, when we are located at the centre. This will cause the signal, which is received by our electronic support measures (ESM) system, and thereby also the amplitude to be as strongest at the centre and decreasing at both sides. The figure below shows how a radar scan can look like, where the angle is the angle of direction to the ESM in the radar’s point of view. Angle Figure 23. Example of how a radar scan can look like, recevied by a ESM equipment. [5] If the radar scans are overlapping, a problem that can occur is that it is more difficult to see which of the pulses that belong to which radar scan. The large data set makes the probability of this to occur to be very high. This force an implementation of the program that can find a parameter value, among the pulses of the overlapping radar scans, that can be distinguishable between the different radar scans. AOA could in these situations be a good first parameter to utilize. 6.3.1 Lack of Data The TOA and BRI vector values that can be found at stage C (see Figure 1) and that are stored in the emitter reports will only consist of values above a certain amplitude limit15. The difficulties that thus can arise, when the data at stage C are to be related to data at stage B, is that it will be difficult to discover a whole radar scan. This means that also the pulses, whose TOA and BRI values not are stored and transmitted in the emitter report (pulses before the 64 last pulses, but also pulses below the amplitude limit) have to be related to an emitter. It should be noted that several radar scans, from different or the same radar (with several lobes), can overlap. Hence, making the situation of relating the data more complicated. It could also happen that some certain pulses not are received by, or can not be processed by, the video processor. Hence, this will lead to some gaps occurring in the radar scans. This problem will complicate the relating, by making it more difficult to find consecutive pulses in the radar scans. Besides these problems, the main difficulty is that there is not enough data in the emitter data file so that a correct relating to its pulses can be made. Therefore, a good solution as possible has to be found in this project that can give a guidance of which pulses that may belong to a selected emitter. 15 This value is classified and will therefore no be revealed, as it will not be used in this master’s project. 24 6 Analysis of Tools and Data 6.3.2 Fluctuations and Inaccuracies of Parameter Values Another problem that can occur when dealing with the relating of the emitter report data to its pulse data is that the values of the parameters can change from pulse to pulse. The most important parameters that probably will change in value are Frequency, Pulse Width and PRF (and hence also PRI). It should also be noted that the Pulse Width also can be varied in value even if it is set to a constant by the radar. The cause of this will not be discussed here. Amplitude An example of this kind of complication is that the radar can be frequency hopping, and thus, leading to fluctuations of the Amplitude of the received pulses in the signal. The figure below illustrates this: [5] Angle Figure 24. Example of how a radar scan can look like, recevied by a radar warning system, when it is a frequency hopping signal. [5] The reason for the arisen of this phenomenon will not be discussed in the scope of this master’s thesis. The interested one are referred to the references for information about this. Often, the Amplitude among the pulses is changed during a radar scan. This only happens if it is not a radar that has locked on. The amplitude of a radar that is in a locked-on mode is instead always a constant. More about this can be read about in chapter 4. The Amplitude can, however, not be manipulated by the radar, and hence, the conclusion that can be drawn by this is that consecutive pulses have about the same amplitude. By using probability theories, pulses belonging to a radar scan can be correlated. The main problem of the fluctuations is that it makes it more difficult to see correlations between the pulses. Hence, these parameters may, of the above reasons, not be used in the correlation process of the emitters and pulses. Will this mean that only the TOA and AOA, as they can not be manipulated by the radars, are the only parameters that can be used as they are the most accurate ones? Another problem is that the PRI value is calculated as the time difference between two received pulses. The problem occurs as all pulses may not be received or can not be measured. This leads to gaps occurring in the pulse data, and therefore, the PRI value is not consistent but can vary substantial. However, if a subset of pulses has the same PRI value, the first guess that can be made is that these pulses belong to the same emitter. Due to the problems mentioned above, with the fluctuations of the values, it is probably important to make use of the mean and standard deviation values for Frequency, PW, PRI and BRI. 6.3.3 TOA Wrap-around The TOA value, having the unit µs, is a counter that goes from 0 to a maximum value, which restarts over and over again through the pulse data file. Therefore it will be difficult to map emitters to pulses using this value. The problem in this is that the recording of the emitter 25 6 Analysis of Tools and Data data and the pulse data does not have to start at the same time - they do not have to be synchronized, which makes it to an impossibility to know which TOA value in the emitter data that maps to which TOA value in the pulse data. Therefore the implementation of ANNALYS EXT can have a counter that keeps the value of how many times the TOA value has been reset. The real TOA value of a pulse is therefore retrieved by the equation: TOAcurrent + counter ⋅ TOAMax Yet, one problem will still remain; how to synchronize the emitter data and the pulse data. Therefore, instead of using the above stated solution, a solution that uses a Time parameter will be implemented. The Time parameter, which can be found in both the emitter data and the pulse data, states the exact time of the pulse, respectively emitters from hour down to µs. Still, the TOA value has to be used to map the emitters correctly to its pulses. This has to be done, as the Time value for an emitter and all its pulses are (obviously) not the same. The solution here will be to check the TOA value in a Time interval of [Time – TOAMax, Time + TOAMax]. This approach will give us unique TOA values in that interval. 26 7 Solution Proposals 7 Solution Proposals This chapter will briefly discuss some solution proposals of how to relate the emitters to pulses. At the end of this chapter there will be a wider elaboration of the employed solution. There will also be a discussion of why the chosen solution was employed rather than the other proposals. It is quite important that there are several possible solutions to solve a problem. This does not only show that one has carefully analyzed a problem, but it is also a gain to be able to select from different solution proposals in the implementation phase. All of the given solution proposals have a numerous of advantages and disadvantages. However, at the end, the solution with the most number of advantages and with disadvantages that does not have such a great affect on the result has been chosen. 7.1 Emitter-to-Pulse Correlating The below subchapters will elaborate on different kind of solutions that can be used when trying to find out which pulses that belongs to which emitter. This is the main task of this master’s project, and therefore the concentration will be on this. 7.1.1 Solution One The first solution proposal that we came up with was that the program initially should divide the pulses in groups with respect to its AOA and TOA value. The main reason for wanting to do this is that these two values are the ones that are relatively reliable, and that cannot be manipulated between pulses by the radar. Hence, this solution will result in that the number of possibilities, of pulses that an emitter can originate from, is minimized. When the user selects an emitter in the application, the only thing to do is to check in one of the groups having the corresponding values of AOA and TOA as the selected emitter. An alternative could also be to group by other parameters as Frequency, PW and Amplitude. However, as these parameters often can be changed, it is better to not group by these. The big disadvantage with this solution is that no consideration is taken to other parameters, as it is impossible to take the other parameters to consideration (they vary too much). An interesting alternative could be to group the pulses by intervals, i.e. the pulses having the AOA value in the interval from n to m is put in group x etc. However, this solution is not sufficient as the grouping can be misleading, in the way that pulses of an emitter can exist in a junction of groups. This solution will, hence, result in that the pulses that belongs to the selected emitter is not the correct ones, as some pulses can disappear in this process. The conclusion of this is that a grouping of pulses should not be done in beforehand. Therefore, a better solution is, when an emitter has been selected to apply this solution in an area around the emitter’s time value. A subset of these pulses probably belongs to the selected emitter. Another drawback with this solution is that it is more difficult to implement a manual filter function, so that a user manually can filter on some data. 7.1.2 Solution Two The second solution proposal was to retrieve the parameters of the selected emitter. Afterwards, the application should filter the pulses on these parameter values. However, one should have in mind that the Amplitude, Frequency and PW are not indicated as exact values for each pulse in the emitter data. Instead, these values are indicated as a mean value with a standard deviation based on all pulses that constructed an emitter. 27 7 Solution Proposals It is important to notice that filtering on mean value ± standard deviation results in that the extreme pulses at the end of the radar scans are removed, i.e. the pulses located at both ends of the normal distribution. The interesting aspect is if Frequency, Amplitude and PW can on the whole be used as filtering parameters. The questioning part is how the results will be affected by the changes that could arise in these parameters, as the radar intentionally changes or manipulates them. Amplitude The above question arise another; can probabilities be used to determine the pulses? If one pulse at the time is determined and put into a set, this set can be used to determine the next pulse by using probabilities. When the number of pulses in the set is increasing, the number of possible pulses that can belong to a radar scan reduces. The figures below illustrate this concept on the parameters amplitude and frequency: These are the alternatives for the next pulse in this radar scan. This pulse is the closest to the already determined pulses in the radar scan. Hence, this pulse belongs to this radar scan. Here, only the amplitude has been taken to consideration. These pulses are already determined to belong to this radar scan. Time Frequency Figure 25. Determination by probabilities concidering the amplitude. These pulses are already determined to belong to this radar scan. As can be seen, minor variations can occur. This pulse is the closest to the already determined pulses in the radar scan. Hence, this pulse belongs to this radar scan. Here, only the frequency has been taken to consideration. These are the alternatives for the next pulse in this radar scan. Time Figure 26. Determination by probabilities concidering the frequency. 7.1.3 Solution Three This solution’s idea is that all the pulses are saved in a database, which will improve, simplify and make it more effective to filter on different parameters. When the user selects an emitter in the application, the emitter’s values of the parameters: Amplitude, Frequency, PW, AOA are used to filter on the pulses stored in the database. The filtering values that are used for the last three mentioned parameters are the mean values and the standard deviation values,16 which can be found in the emitter data file. This approach is used as these parameters can be changed from pulse to pulse in a radar scan belonging to an emitter. It should be pointed out that this will result in that the pulses having extreme values of these parameters will not be 16 mean value ± standard deviation are the boundary values, when filtering. 28 7 Solution Proposals included in the set of pulses belonging to the selected emitter. The reason for this is that only the standard deviations are available. 7.1.4 Solution Four This solution differs much from the previous ones. The main idea here is to find a function that can depict the pulse graph with respect to its amplitude. This function have to be found based only on the first, last and maximum pulse values, as these are the only pulses that have both a time and an amplitude value. Although it will not be that easy to find a function that uniquely can find all the pulses that belong to an emitter, it seems like this solution anyhow can give an estimated set of which pulses that may belong to an emitter. The disadvantage with this solution is that it probably will miss several pulses, as they are not positioned on the graph. The reason for this is that a radar can change the PRI value, which will make it difficult to find a graph that represents a correct picture of how the pulses are arrived. It could be understood from the amplitude-time graph in the application ANNALYS, that the pulses varies according to a quadratic form. Therefore, the available parameters in the emitter data are used to find a function having the form P(x) = c1 + c2x + c3x2. To find this function interpolation will be used. This process will be more elaborated in the next chapter. After finding this function, the difficulty is to find correct PRI values (or ΔTOA) from the emitter data. The first attempt will only be to try using the most common PRI value, which will be retrieved by analyzing the last 64 TOA values that can be found in the emitter data. The pulses that have been retrieved as a result of this solution must also be checked to be in the correct intervals of the values: Frequency, PW and AOA. 7.1.5 The Employed Solution Two solutions: solution three and four, have been implemented, as they differ much and can give different answers of pulses that belong to an emitter. The difference in the answers will be that solution four will give a less number of pulses as an answer than solution three. The reason for this is that solution four will try to find all the correct pulses that belong to the emitter with the help of a function. Hence, this will result in that a fewer pulses are found. It should be pointed out that all the pulses that belong to this emitter would not be found, as this will require a more extensive PRI analysis. Hence, resulting in that the input to the function (ΔTOA) will have to be more correct. The cause for this is that a radar can be PRF staggering, i.e. it is changing its PRF value and thereby the PRI values are also changed. This means that the ΔTOA values can be different from pulse to pulse. The solution stated in solution fours only considers the most frequent ΔTOA value, and use that value to be the one and only ΔTOA throughout the whole radar scan. Another cause for this is that the pulse signals can be phase coded17, i.e. data are sent between two pulses. These data pulses will, hence, have different PRI value than the end pulses. This will not be taken into consideration in the solution that will be employed. The main reason for not analyzing more in, and to further develop the PRI analysis, is that it is not possible to include that in the scope of this master’s project. The PRI analysis requires a great deal of effort from the application, and also a more understanding and analyzing of the pulses are required. 17 This concept will not be elaborated more in this master’s thesis. The interested one is referred to the literature given at the end of this thesis. 29 7 Solution Proposals Solution three, on the other hand, uses a filtering function. As boundary values18 are used to accomplish this, more possible pulses that belong to this emitter will be given as an outcome. As can be seen in the solution proposals, solution three has been built up by understanding that a solution stated before could not be done in a sufficient manner, and which therefore have been improved. The improvement has either been that the whole solution has been changed or that just a part of it has been changed to the better. 7.2 Statistical Solutions As solutions from the statistical point of view is in the scope of this thesis, the below chapters will elaborate more on the kind of solutions that can be in the area of this. These subchapters will, unlike the above chapter, only state the solution that will be implemented. 7.2.1 Statistical Values There are different kinds of statistical values that could be of interest here. The mean value is, however, the value that has the most important affect when analyzing the data. In the emitter data, this value is already available, indicating the mean value of the pulses in the emitter track. Our mean value will therefore be based on the pulses that have been found when the user selects an emitter, i.e. the pulses in the pulse selection. This value can not only be compared with the value in the emitter data, but can also give an answer of how much it differs from the boundary values in the pulse selection. Hence, giving clues of how certain this pulse selection is. The boundary values are therefore also something that will be stated in the application for the current pulse selection. 7.2.2 Filter A manually filter function will also be implemented, which can be utilized by the user to see the values that have been used in the filtering of the pulse parameters, in order to retrieve the current pulse selection. These values can also be changed by the user, if another filtering of the pulse parameters is wanted. This functionality can be to an advantage if the user, after a pulse selection have been retrieved by selecting an emitter, wants to do another filtering with the intention to see the pulses around the filtered area. 7.2.2 Histogram Another statistical functionality that will be implemented is histograms. The histograms will be shown of the most important values in a pulse selection: Amplitude, Frequency and PW. This statistical functionality is outstanding in the sense that it is simplifying for the analysts, by making it easier to see the selection of pulses in a more summarized view. This makes it easier to see if the pulse selection, which has been obtained, actually contains two or several radar scans rather than one. The information gained from this analysis can then be used to filter a pulse selection on narrower upper and lower limits. This idea of functionality was greatly appreciated by SaabTech and by the analysts, as it simplifies their work even more. Histograms are an excellent way for the human mind to interpret a summarization of a large statistical data. 18 A minimum value and a maximum value. 30 8 Design and Implementation 8 Design and Implementation This chapter will describe the design and the implementation of the application, which is the result of this master’s project. First the GUI of the application will be discussed, as it is important that the GUI is simple and plain so that it is easy to understand and use. The chapter will then continue with describing an UML representation of the application design. This will give the reader a better overview of how the application is intended to be implemented and work. Afterwards, some functions of the application will first be described with an explanation of the chosen design to conclude with a description of the implementation. We start this chapter with a brief elaboration of how the application should work in general. The user is able to open two files; an emitter file and a pulse file. These files are parsed and the application extracts and saves the important information in an understandable format. After the parsing has been done, the data should be represented as dots in two separate graphs; an emitter graph and a pulse graph. The users should be able to work on both of these graphs. There should be some basic functionality like: zooming in, zooming out and the dots in the graph should be clickable. When a dot is clicked on, the information of the pulse or emitter that it represents should be displayed. When an emitter is selected, this should lead to its pulses being seen as a selection in the pulse graph. The functions that will be implemented in this application are, as they were found in this master’s project to be of a great importance for the analysts, following: • A filter function, which makes it possible for the user to manually set filter values on Time, TOA, Frequency, Amplitude, PW and AOA. • A statistics function that show some statistics of the pulse selection. • A histogram function that shows histograms of the parameters: Amplitude, Frequency and PW for a pulse selection. • An emitter correlation function that correlates the emitters. This means that the emitters that originate from the same radar are found. • A function to show the pulses of the correlated emitters (this can only be done if an emitter correlation already has been done). SaabTech did not have any formal requirements of the functionality and performance of the application. However, the supervisor at SaabTech had requests that the aim of the application should be on time optimization rather than memory optimization. The reason for this is that it should be fast to work, when doing the analyzing work, as this work is very extensive. Therefore, the concentration in this project will be to make the application as fast as possible when the analysts work with the data, letting it be acceptable to be somewhat slow when parsing the files, and also to consume somewhat more memory. It was also important to know what kind of computer systems that will be used to run this application on. The systems that are planned to be used with this application are modern with good performance and have around 2 GB in memory. This allows the program to consume somewhat more memory making it possible to make it faster when working with it. As the data consist of a couple of thousand of emitters and a couple of hundredth of thousandths of pulses, it is important that it is fast to work with the data and that the dots can be drawn and updated quickly in the graphs. 8.1 GUI The GUI of the application had to be very intuitive and very user-friendly. It should be easy for any user without any knowledge of how the application works, to understand what 31 8 Design and Implementation functions it has and how they could be used. Although this was not the main task of this master’s thesis, we understood that it had an important role. It would simplify to a great deal for the analysts when working with this application, if the GUI is easy to understand and use. Before elaborating more about the GUI, the figure below shows the resulting GUI: Toolbar Emitter information area Pulse information area Emitter graph Pulse graph Figure 27. The GUI of the application with a description of some of its fields. Let us now have a discussion of some of the important parts of this GUI. 8.1.1 Graphs The graphs; emitter graph and pulse graph shown in the figure above, are able to show a timeamplitude graph, a time-frequency graph or a time-PW graph. To simplify the code, all the knowledge for this will be placed in one single class called GraphSettings. In this way, only this class knows what values the ticks, dots etc. should have given an emitter or a pulse. To simplify the work with this application and to just have to concentrate on the important parts of this master’s thesis, namely to relate the emitters to its pulses, a graphic package from an earlier project at SaabTech was used. This graphic package has methods and functionality to draw a graph and dots in a Canvas (java.awt.Canvas). This was therefore used to draw pulses and emitters as dots in the graph. This package did contain functionality to zoom in and out. However, something that was missing was a square that shows the area that is selected by the user to zoom in on. As this seemed as something that made it easier to the analyst when working with this tool, this square was created. Hence, some changes have been made in the classes in the package from SaabTech. Another problem with this package was the use of the class Canvas. This class belongs to the awt package, which makes it to a heavy-weight component19, while the rest of the GUI will be Swing components. It is a know fact by the Java developers to rather not combine awt components with Swing components, as this can lead to some conflicts [11]. Therefore, the Canvas was changed to a JPanel instead. A more important functionality that was missing in the package was a support to click on the drawn dots in the graph and retrieve the object; an emitter or a pulse, which it represents. This is something that is important to have in ANNALYS EXT, as the user should be able to click on an emitter to retrieve its pulses, or just to click on an emitter or pulse to retrieve its 19 An elaboration of this can be found in [11]. 32 8 Design and Implementation information. Therefore, support for this has been implemented as a part of this master’s project. To make the graph clickable; the clicked position20 in the graph where the user has clicked will be retrieved. However, it is not sufficient to only know this pixel position as all the dots in the Canvas is drawn as filled circles with radii of two pixels. The center positions of these circles are the pixel position that the value (amplitude, frequency or PW) of the emitter or pulse has in the graph. The approach to allow the user to click at any position of the drawn dot, and thereby selecting an emitter or a pulse, is to use an offset value around the center position. The offset value is represented by the radii of a dot. Therefore, it will be possible to click on a dot in the area; center position ± offset, in both the x-axis and y-axis. The above stated problem also makes it more difficult to uniquely find the pulse or emitter object in a data structure that has been selected. There are several approaches to solve this. The easiest way is to iterate through all the objects (pulses or emitters) and then check that the position that have been clicked on in the graph is in the area; center position of the dot ± offset, which means that this is the selected pulse or emitter. However, this approach will do the searching on O(n), which is not time optimized when dealing with this quantity of objects (mainly considering the number of pulses). A more detailed elaboration of a better approach that has been taken to solve this problem will be given in section 8.3.2. The implementation of displaying these dots in the graphs arose some difficulties. The first dilemma had to do with the great quantity of pulses, which had to be drawn in the graph. The maximum number of pulses could be up to 900 000. However, the test data that was used, when implementing this application, consisted of approximately 250 000 pulses. Although, this makes each repaint of the graph to take some time21. Hence, this was something that had to be optimized. The first attempt that was tried was to utilize the class BufferedImage, which is buffering the image in the memory. It is as a rule better to draw on a buffered image than directly on the screen (by using the Graphics object sent to the paint() method). [6] This approach led to an improvement by roughly one second. Although, this seemed, still, not sufficient enough in this application. However, many tries to find information about graphical improvements led to nowhere. We have therefore in this master’s project not been able to find an effective solution to handle this quantity of drawing of pulses in the graph. As Java has to be considered to be slow, even though there are developments going on in this area, we have to conclude that there is not much we can do about this problem. Although we could not find a solution to make the repainting more efficient when it comes to that quantity of objects in the graph, we implemented a solution so that the repainting only occurs when it have to. Also here, we got a great benefit of using a buffered image. When the graph has to be redrawn, the drawings always occur in the background on the buffered image. While a user zooms in on the graph, a zoom in square has to be drawn. When this happens, the saved buffered image is first drawn on the graph area22 and then to end with drawing the square. In this way the graph does not have to be redrawn from scratch23 each time the zoom in square has to be drawn24. The figure below illustrates this process. 20 The x and y coordinates. Around 7-8 seconds. 22 This means that the drawing is done on the Graphics object that is sent to the paint() method. 23 i.e. it is not necessary to redraw all the dots. 24 This occurs whenever the size of the zoom in square changes. 21 33 8 Design and Implementation Step1: Draw the saved image on the graph area Image saved in memory The graph area Step 2: Draw the zoom in square on the graph area Figure 28. The process of drawing the zoom in square on the graph area. We also experienced some delays when the graph had to be redrawn each time the window was hidden, i.e. when the window gets dirty. This was also solved utilizing the buffered image, which have been saved in the memory. The process of this was similar to the above; only the saved image was drawn on the graph area. Hence, the dots in the dirty area did not have to be redrawn from scratch. The only time the pulse graph is redrawn from scratch is whenever an emitter is selected and a pulse selection has to be displayed in the pulse graph. Furthermore, the only time the emitter graph is redrawn from scratch is whenever an emitter correlation has been done, and the correlated emitters have to be displayed in the emitter graph. In all other cases, the back buffer, i.e. the saved buffered image, is drawn. This technique is called double buffering, which is widely utilized by developers dealing with graphics. [6] As it is really fast to draw an image rather than redrawing the whole graph from scratch, the users of this application can save some time when working with it. 8.1.2 Toolbar It was assumed to be of a great importance to have a toolbar with the most common functions in the application. This will make it easier for the analyst to quickly select the tool to work with. The tools have been placed in the toolbar in an intuitive way. This can be seen in Figure 27 The first four tools in the toolbar have to do with working with the graphs: select mode, zoom in, zoom out and redrawing both graphs. The latter tools have to do with common used functions in the application. These functions will be more elaborated later in this chapter. To help the user even more, each time the user holds the mouse over one of the tools, information of how it is used are displayed as a tool tip and also at the bottom of the application. 34 8 Design and Implementation 8.1.3 Information Area There are two information areas in the application; one for the emitter information and one for the pulse information. These can be seen in Figure 27. Whenever an emitter or pulse is selected the information of the selected object should be displayed in its respective text area. The information will be displayed row wise with a parameter and its value. Some information that can be found in the emitter data file will not be included, as this information seems unnecessary for the analyst. 8.1.4 Access Keys To make it even more easily and faster to work with the application, some access keys were implemented for the most common functions. They are displayed in the menus, next to the menu item, which makes it easy to find out what kind of access keys that exists. The access keys that probably will be used most frequently are the ones for quickly changing the graphs between time-amplitude, time-frequency and time-PW. 8.2 UML Below a simplified UML design can be seen, which will give a better understanding of how the different parts of the application works and are related. It should be pointed out that all the implemented classes are not shown, with the reason to not make it too unreadable. 35 8 Design and Implementation Figure 29. UML design of the application. Much of the content in the class CommonGraph comes from an earlier project at SaabTech. However, as has been said earlier, some changes and improvements have been made to it as an approach to get the extended functionality wanted in this application. The Main class has the responsibility of the whole application. This class initiates and displays the application GUI (Window). The parsing of the emitter and pulse files is also in the responsibility of this class, which runs these parsing in two different threads. More about this can be found in section 8.3.1. As can be seen in the UML the design pattern of singletons has been utilized in this project. The reason for this is that some of the classes in this application are used in common by all other classes that utilize them in the application. This means that only one instance of these classes exists, and which are created when they are called with the method getObject() for the first time. The two storages; EmitterStorage and PulseStorage, have also been designed as singletons as their only task is to store their respective objects; emitters and pulses. Furthermore, these storages are accessed by disparate objects throughout the application system and therefore require a global point of access. 36 8 Design and Implementation The two selections; EmitterSelection and PulseSelection, have also been designed as singletons as they also stores pulse respectively emitter objects. Each time a filter of pulses or a PRI analysis is done, the current subset of pulses are stored in the PulseSelection. The similar applies to the EmitterSelection, which stores the emitter objects when an emitter correlation has occurred. It should be pointed out that new objects are not created when they are stored in these selections. Hence, the memory usage does not increase by this.25 Two settings classes (not shown in the above UML design); GraphSettings and Settings are used, in order to have common settings classes for the application in whole. They both are designed according to the singleton design principle, as they had to be accessed from several parts of the application. The GraphSettings is utilized to hold the settings for the shown graphs. With this approach, all the knowledge of the graphs is managed in common by one single class. This is needed as the axis values of the dots that are to be added in the graph and the values of the ticks, depends on the type of graph that are to be shown. Hence, this class determines this information given the type of graph and an emitter or a pulse, depending on the circumstances. The Settings class is utilized for similar reasons, which provides a single place to set and retrieve all of the global settings that affect the application as a whole. Other classes where the design pattern of singletons has been utilized can be seen in the UML design above. 8.3 File and Data Handling The techniques utilized to parse the files and storing the data have an important role of the efficiency and the performance of this application. Therefore, it is important to discuss the techniques that have been implemented, as they seemed to be the best solution in this project. This section will first discuss the parsing process of the files. Afterwards, we will continue the discussion with how the data are stored in the application, in order to get the best performance from the application. The pulse data are also stored in a database to make the filtering functions in the application to work most smoothly and efficient. Hence, we will conclude this section with a discussion of this. 8.3.1 Parsing of Files The parsing of the both files occurs directly after the user has selected the two files; an emitter file and a pulse file. These two files both have the format txt; hence, they only contain raw text. As the parsing is somewhat time-consuming as well as to avoid making the GUI unusable during that time; these are run in two separate threads. To accomplish this, we have in this project utilized a class named SwingWorker26. Although the parsing occurs in different threads, which does not affect the GUI, it is indeed important that the parsing is done as quickly as possible. The pulse file, which the application has to parse, has normally a size of above 30 MB27. Therefore, one has to count with some time consumption. The emitter file is normally somewhat smaller, and is usually above 7 MB. However, it should be pointed out that the files that was used in test purpose, when developing this application, was somewhat smaller; the pulse file was on 27MB and the emitter file on 7 MB. Yet, these files are really large, bearing in mind that they are only text files. 25 The mentioned functionalities will be discussed later in this chapter. More information about this class can be found in [23]. 27 This is a great deal, considering that it is only a txt file. 26 37 8 Design and Implementation Lord and Reddy assets that many Java programs that utilize I/O are in deed in need of performance tuning, as inefficient I/O is a common problem in Java. They go on maintaining that the I/O performance issues usually overshadow all other performance issues, making them the first area to concentrate on when tuning performance. Therefore, according to them, I/O efficiency should be a high priority for developers looking to optimize performance. However, this can be challenging in Java. [22] Therefore, we have in this project understood that if we want to improve the parsing of the files, we have to concentrate on improving the way these large files are read into the application. Different input streams and file readers in Java were used, without giving the desired result in the terms of performance and time-optimization. We also tried with native buffers28 in Java and also used that to map the file into memory. The last attempt seemed like a bad idea, as the files are so large. As a consequence of this, we used a custom file reader named BufferedFileReader29, which according to [22] is the fastest one out there when it comes to reading large files. So what is the difference between the BufferedReader and with the custom BufferedFileReader? The big difference has to do with the method readLine(). The method readLine() in BufferedReader creates an instance of StringBuffer to hold the characters in the line it reads. It then converts the StringBuffer to String, resulting in two more copies per character. The BufferedFileReader class utilizes a modified readLine() method that avoids the extra, double-copy in most cases. [22] One observation that was made during the parsing of the files was that when the files where located on a network storage this process could take up to 30s. This did not change even if we used different computers with different performances30. The conclusion that can be drawn from this is that the problem must be in that the files had to be accessed over a network. To parse the files when they are stored locally gives a faster access to the files, which made the time of the parsing to drop down to below 5s. This means that the users of this application have to take this into consideration when using it; if they want the application and the parsing of the files to be faster, they have to have the both files stored locally - although, this is not considered as a limitation. Another thing that was tested was to use different size of the buffer, and in this way try to see which size is the best. Below, the results can be seen: Buffer size (byte) Time(ms) 256 4290 512 4216 1 024 4185 2 048 4155 4 096 4155 8 192 4125 10 240 4202 Table 2. The numbers in the table are based on runnings on a computer with 1GB memory and a processor of 3.2GHz. The times that have been specified are median times of a number of excecutions of the application. 28 These classes can be found in the java.nio package in Java. This class can be downloaded from [17]. 30 Two different computers were used; one with 1GB memory and a processor of 3.2GHz and the other one with only 256MB memory and a processor of only 700MHz. 29 38 8 Design and Implementation As can be seen in the table above, it was most effective to use a buffer size of 8 192 bytes. However, one has to keep in mind that there are also other things that matters when measuring the execution time of an application. An example of this can be that other resources need execution times etc. Thus, these times should not be taken as such reliable. Yet, they can give some guidance of which buffer size that is best to use. The table above shows that the median times are decreased each time the buffer size is increased until we reach a buffer size of 8 192 bytes. The conclusion that, hence, has been taken from this is that a buffer size of 8192 bytes is the best to use in our application. Each piece of information, i.e. the different parameters, is stored row wise in both files. Hence, when parsing the files, several calls to the method readLine() in the BufferedFileReader is done. After each call to readLine(), the wanted information from that line is extracted using String expression comparison. When all the information of an emitter or pulse has been extracted, an Emitter respective Pulse object is created storing this information. Thus, there will be as many Emitter and Pulse instances as there are emitters and pulses stored in the files. In the remainder of this master’s thesis we will use the word item to refer to Emitter or Pulse objects. Let us continue with an explanation of how we have decided to store these items. 8.3.2 Storage of Data in a Data Structure It is quite important that the items, e.g. Pulses and Emitters, are stored in a way that makes the operations in the application to work fast and smoothly. The choice of technique that has been used to store the items in the application has mainly been focused on to simplify and optimize the searching of a certain pulse or emitter. The searching is done at any time a dot in the graph has been clicked on, and the application has to find out which item that was selected. As the searching of a certain item is done on a great number of items, this is obviously something that has to be improved. We have also been focusing on to optimize the iteration of the stored items, which are done in different kind of analyzing in this application. More about the analyzing processes will be elaborated in section 8.4. We have in this project analyzed different methods that could be utilized to store the data in an efficient way. Before having a brief discussion of these different methods, it should be pointed out that the maximum Time range (hence, also the x-axis range in the graph) can be up to 900 000 000 µs. The first method we analyzed was to build a matrix of the whole drawing area in the graph; a mapping of the x-axis and the y-axis. This will give us only O(1) to find a certain item. However, we understood early that this was not such a good idea, as the values that had to be mapped are so large. Even if it is as optimized as it can be when it comes to finding a certain item it would require a great deal of heap memory. Therefore, we also analyzed different utilizations of the more familiar method named lazy hashing31. The solution that was accepted in this application is utilized by having a Hashtable that is as large as the time difference of the graph (i.e. TimeMax - TimeMin). Thus, the key in this Hashtable will be the value of the x-axis, which is the time. All the items will be placed in a LinkedList, and which are stored as the element in the Hashtable. When storing the pulse objects in linked lists, the information of at what position the pulse object is stored (i.e. the index of the LinkedList) is saved in the item. This will simplify the access of a specific pulse object. An elaboration of this will be given in section 8.3.3. 31 A term used by Viggo Kann, professor at KTH. 39 8 Design and Implementation The solution of using lazy hashing gives one a good compromise between time complexity and memory complexity. It will be really fast to find a certain item and requires not such much memory. We also considered using an array or an ArrayList32, instead of a hash table, where the size is as large as the maximum x-axis value. As we have to map the x-axis, which can be up to 900 000 000 (µs), it seemed better to use a hash table. Even if this will not always give us O(1) when finding an element. The reason for choosing a hash table is obvious; an array and an ArrayList have a memory complexity that is not proportional with the number of inserted elements. A hash table, however, has a memory complexity that is proportional with the number of inserted elements. Like arrays, hash tables provide constant-time O(1) lookup on average, regardless of the number of items in the table. However, one has to keep in mind that the rare worst-case for lookups can be as bad as O(n). Therefore, compared to other associative data structures, hash tables are most useful when a large number of data records are to be stored. [15] As each element in the hash table is a linked list it will take O(x) to construct the data structure (where x is the time difference), but it will be fast to retrieve a certain item in it. As the operations performed on the Emitter and Pulse objects differs (this will be explained later in this chapter), they had to be stored in a somewhat different way. As we were in need of accessing the data in a sorted manner when it came to Emitter objects, we had to have a data structure that had an order of the objects it stored. However, the hash table has per definition no order of the items; therefore we had to use a data structure named TreeMap. The TreeMap works almost the same as the hash table, but orders the elements (or rather the keys) it stores using the algorithm Red-Black tree. The drawback of using this is that when we store new objects into the data structure we get a time complexity of O(log n). The reason for this is that the ordering has to be maintained each time a new object is stored. To get a certain object from a TreeMap requires, as well, a time complexity of O(log n). This can be compared to a hash table, which only requires O(1). Linked lists are stored as elements in both cases – when utilizing a hash table as a tree map. We could also have used arrays instead of linked lists. However, as the inserts of the new items (Emitter and Pulse objects) are done dynamically33 as they are “found” in the parsing process, we get some benefits using linked lists. Some of them are self-explanatory. The table below indicates the time-complexity of the both data structures: Operation Array LinkedList add O(n) O(1) get O(1) O(n) delete O(n) O(1) Table 3. The time complexity of different operations on the data structures; array and linked list. The drawback we get by using linked lists is that they do not allow random access but only sequential access to elements. Therefore, as can be seen in the table, we get a time complexity of O(n) looking up an element by its index. Although, this makes it somewhat unsuitable having a linked list we had to use it as the insertions had to be done dynamically. However, it should be pointed out that each linked list will not contain that much items (Pulse or Emitter objects). Hence, the selection of using a linked list, array or array list will 32 33 The construction of an array and an ArrayList is similar as an ArrayList internally utilizes an array. i.e. we do not know how many Pulses/Emitters that should be inserted into each linked list. 40 8 Design and Implementation not affect the application that much. However, when comparing linked lists with array lists; the linked lists are a benefit when it comes to growth of the data structures. Let us now have a discussion of how the searching is done when the user have clicked on the graph with the intention to select an emitter or a pulse. When this happen, we have to do a searching in a Time interval of clicked position in the grapht ± offset and then to check if an item can be found in that area. This will be accomplished by the following algorithm: Algorithm findItem(pos, items) Input: A Point pos containing the x and y position that was clicked on A HashTable items containing all the Pulse/Emitter objects Output: The item (Pulse or Emitter) that was clicked on, otherwise null offset_x ← the radii of a dot in x-axis in the same scale as the graph clicked_x ← the clicked x position in the same scale as the graph clicked_y ← the clicked y position in the same scale as the graph for i ← (clicked_x – offset_x) to (clicked_x + offset_x) do LinkedList ll ← LinkedList with index i in items, if it exists Iterator iter ← The iterator of ll while iter has more elements do current ← next element in iter current_x ← x value of current in the same scale as the graph current_y ← y value of current in the same scale as the graph if ( (clicked_x <= (current_x + offset_x)) and (clicked_x >= current_x – offset_x) and (clicked_y <= current_y + offset_y) and (clicked_y >= current_x + offset_x) ) then return current return null Pseudo code 1. The pseudo code for the algortihm of finding out the Pulse/Emitter object that was clicked on in the graph. As can be comprehended from the above pseudo code; as soon as we have found an item (Pulse or Emitter object) that are located within the interval of a dot, it will be returned. The pseudo code also reveals that the maximum number of linked lists that are iterated is 2 × offset_x. Hence, the time complexity of the for loop is O(1). The time complexity of the while loop will then, if the linked lists contains m elements, be O(m). Hence, the resulting time complexity of the whole algorithm will with that be O(m). Just to weigh against, if we had used a sorted array instead of a linked list we could have found a certain item with the complexity O(log m) using binary search, with the drawback that the creation of the data structure would take some more time. However, we would not experience a big difference as the items stored (the value of m) is not that large. The selected data structures have been implemented in two singletons named PulseStorage and EmitterStorage, where the PulseStorage stores all the Pulse objects and the EmitterStorage stores all the Emitter objects. 41 8 Design and Implementation 8.3.3 Storage of Data in Database It was very important that the application was fast to work with in the analysis process. Hence, it was acceptable by SaabTech that the parsing at the beginning took some time, making the application’s analysis functions more efficient and faster. Therefore, we chose to use a database to store all the pulses’ information (their parameters). Also, the index value of each pulse, indicating where they are stored in the linked list, is saved. Utilizing a database gives the application the filtering functions it needs. As the application had to make a great deal of filtering of the data we understood that a database, having indexes over the parameters (columns) that where queried, would be the most efficient to use. The reason for this is that this, hopefully, is implemented in a database system in a most efficient way. This approached design seemed better than implementing the wanted database functions in the application ANNALYS EXT, as the indexing techniques in a database seemed more efficient. In addition to storing the pulses according to section 8.3.2, the information was at the same time also saved in the database. At first we tried the familiar database MS Access. However, the time it took to create a pulse table and to store all the pulse information was not acceptable. We understood that the problem lay in that all the information had to be written to the hard drive, which made it all a bit slower. Therefore, we realized that an embedded in-memory database was needed. The database that was chosen is named HSQLDB34, which is the fastest open source database system out there. [16] This database system has the possibility to create databases in memory or also cashing them to the hard drive. The latter is to prefer if it is to be used on systems with little memory and big tables. [26] However, although we have to store pulse information for up to 900 000 pulses, we are in need of getting as high performance as possible from the database operations in the application, as these operations will be executed quite often. Hence, we will store our database in memory. As the computers that this application will execute on will have around 2GB in memory, this alternative seemed to be the most optimal for this case. Having HSQLDB as an embedded database to ANNALYS EXT did not make the application to become dramatically larger, as the HSQLDB jar file only has a size of 600 kb. However, the usage of a database made the requirements of the application to require at least 512MB in memory.35 Utilization of a profiler36 showed that approximately 64MB was required to store 254 614 rows (pulses). The total memory consumption for the whole application, including the database, resulted in 120MB according to the profiler. It was important that the storing of the pulse information in the database is done as smoothly and quickly as possible. Therefore, a so called PreparedStatement was used, as the only difference when storing the pulse information in the database is the values of its parameters. A prepared statement is the most efficient to use for single queries with multiple executions for varying parameters, as it is a precompiled SQL statement. In other words, the database only has to compile that query during the first execution. The query is then stored in the database’s query cache. When a new pulse is inserted, the only change that has to be made on the prepared statement is the values of the pulse parameters. This can be done as the prepared statements accept bind variables. [20] Here, as well, the information of what index the Pulse object has in the linked list, in which it is stored, is a great benefit.37 When a wanted pulse, after e.g. a filtering of parameters, is 34 The website of this open source product is: www.hsqldb.com. A test with 256MB in memory made the application a bit slow, probably because of that the virtual memory was used, which resides on the hard drive. 36 The product used for this is named JProfiler, http://www.ej-technologies.com/products/jprofiler/overview.html 37 This has been explained in section 8.3.2. 35 42 8 Design and Implementation found in the database, the information of its Time and linked list index can be retrieved from the database. This information can then be used to get a direct access to the Pulse object in the PulseStorage. An attempt was also made in this project to store the Pulse objects itself in the database. However, this made the application much slower. The profiler we used showed that HSQLDB made new copies of the objects, and hence, not storing the existing objects. Therefore, this design was not implemented. Another drawback with this design was that HSQLDB did not support queries of the parameters in stored objects.38 Let us now continue with a discussion of how the inserts of the pulse information in the database were optimized. This was accomplished by using the methods addBatch() and executeBatch() that can be found in the class PreparedStatement. [20] Although, it is important to point out that it is not to an advantage to have too many insert clauses stored in memory using the method addBatch(). Therefore, depending on the size of each insert clause and the size of the memory in the computer that is used, a weighing up has to be done of how many insert clauses that should be saved in memory before they are executed and thereafter cleansed from the memory. The benefit of using this approach is that the number of calls that has to be done to the JDBC39 driver reduces, hence, saving time. [20] However, it was uncertain if this would affect our application, as it is utilizing an in-memory database. Therefore, some analyses were done to see how the execution times varies with different number of insert clauses saved in memory and then executed according to the above mentioned procedure. In the test scenario there were 254 614 rows that had to be stored in the database, where each row has two long values together with six double values. As a long value and a double value are both 64 bits, this will result in 8 × 64 = 512 bits = 64 bytes for each row, giving a total of 254 614 × 64 ≈ 15.5MB. A table of the executions times can be seen below: no of insert clauses stored in memory before execution Time(ms) 0 7702 10 6992 100 6960 1 000 6860 10 000 6596 50 000 6814 100 000 6879 254 614 (all) 7066 Table 4. The numbers in the table are based on runnings on a computer with 1GB memory and a processor of 3.2GHz. The times that have been specified are median times of a number of excecutions of the application. As the table above indicates; it is most effective to store 10 000 insert clauses in the memory before executing them. However, one has to keep in mind that there are also other things that matters when measuring the execution time of an application. An example of this can be that 38 There are databases that support this, but they are much slower than HSQLDB. Hence, none of these were selected. 39 Java Database Connectivity. 43 8 Design and Implementation other resources need execution times etc. Thus, these times should not be taken as such reliable. Yet, they can give some guidance of how many insert clauses to save in memory before executing them. The table above shows that the median times are decreased each time the number is increased until we reach a number of 10 000. The conclusion that, hence, has been taken from this is that it is most optimal to save 10 000 insert clause in memory before executing them. To retrieve information from the database, using filtering queries, and storing them as a pulse selection took around 20 seconds as its best. This was totally unacceptable, as the usage of the application was required to work smoothly and quickly. The first, and obvious, attempt to solve this was to create indexes of the parameters that were queried. Hence, the parameters: Time, TOA, Amplitude, Frequency and PW were indexed. This resulted in that information could be retrieved and stored in a selection on its best at around 0.2 seconds, a time that was more acceptable. Although, the worst scenario was that it could take up to 10 seconds. However, using a prepared statement instead of a statement improved this to always be done in below 0.2 seconds, often this time was below 0.05s. Although the indexing made the queries to be executed faster, it had its drawbacks. The creation of the indexes made the initiation40 to almost take the double of its time (12 seconds). This was, however, acceptable by SaabTech. 8.4 Emitters-to-Pulse Correlating It is now time to elaborate on how the emitter-to-pulse correlating was implemented. As was discussed in chapter 7.1, two solutions were implemented as they differed and both seemed to be good solutions. Although, they quite often give different answers; rather than using one of the implemented solutions, they can be used together and, thus, give a stronger analysis tool. As default the method that will be used in the application when the user selects an emitter in the graph will be the filtering solution, of which implementation will be discussed in the next section. However, the user has the possibility to select the PRI-analysis solution in the settings menu. The implementation of this solution will be discussed in section 8.4.2. Both these solutions will, however, not give the correct answer of exactly all the pulses that belong to the selected emitter. The reason for this is that, as was said in section 6.3.1, the lack of data in the emitter data file, which makes it impossible to get a unique relation between the emitters to its pulses. Although, the result will give the analyst a guidance of which pulses that may belong to the selected emitter. Hence, making it easier for the analyst to find the correct pulses, or analysing the whole pulse selection. The reason for this is that the number of pulses in the pulse selection compared to the total number of pulses has been narrowed by a big factor. Before continuing, it should be pointed out that the emitters that are created in the emitter data are based on the pulses in radar scans having their amplitude over a certain limit41. Therefore, there will not be any information of these excluded pulses in the emitter data. In the remainder of this master’s thesis this limit will be denoted as limitampl. It should also be noted that the parameters stated in this chapter are the parameters from the emitter data, if nothing else is said. 8.4.1 Filter The obvious parameters that had to be used in the filtering of this solution are: Frequency, Amplitude, PW and AOA. According to what have been discussed in previous chapters; the 40 41 Parsing of the files and storing the pulse information in the database. This limit is confidential and can, hence, not be revealed here. 44 8 Design and Implementation parameters: TOA and Time will also be used. As was said in section 7.1.3 the filtering is done on the interval [mean value – standard deviation, mean value + standard deviation]. To also be able to get the pulses that reside outside the limitampl, we have increased the boundary values with a percentage factor. In other words; pulses that have values of the parameters that are more uncertain are also included - a more variation of the values is acceptable. The user has also the possibility to choose if the analysis should be done more or less accurate. If the user selects to have a more accurate analysis this percentage factor is set to a smaller value. Hence, a fewer pulses are included as those belonging to an emitter as only the most certain pulses are included. This has been implemented as that the interval [min, max] becomes smaller by a percentage factor, which means that a narrower scope is allowed for these parameters. This approach will somewhat also solve the problem of loosing the extreme pulses42 in this filtering, but it will not be certain that these pulses are included. To get the pulses in the correct time scope; a filtering had to also be done on the Time and TOA attributes. Therefore, as was said in section 6.3.3, a filtering of the Time is done using the interval [Time – TOAMax, Time + TOAMax]. In addition to this the TOA value will be filtered on the interval [TOAfirst pulse – n × ΔTOA, TOAlast pulse + n × ΔTOA], where n is a fixed numeric number. This contributes by giving a result that includes pulses that resides outside the limitampl. The filtering is done by querying the database and retrieving the filtered pulses. The information Time and linked list index in the retrieved result set are then used to get the real objects in the PulseStorage. The approached design have made this possible, as this information is enough to uniquely point out a single Pulse, and hence, get a direct access to the real object.43 The following prepared statement is used to get the filtered pulses: SELECT Pulse_Index, Time FROM PULSE_TABLE WHERE TIME BETWEEN ? AND ? AND TOA BETWEEN ? AND ? AND Amplitude BETWEEN ? AND ? AND Frequency BETWEEN ? AND ? AND PW BETWEEN ? AND ? AND AOA BETWEEN ? AND ? SQL code 1. The prepared statement used to filter on the pulse data in the database. As default all the values will be set to the maxium values they have in the pulse data. The Pulse_Index is the linked list index. 8.4.2 PRI Analysis In was noticed that the amplitude of pulses, in a radar scan, often varied quadratic over time. It was also noticed that a locked on radar scan for obvious cause varied its amplitude as a constant. Hence, these observations were utilized to implement another solution of correlating the emitters to pulses. This solution has been elaborated in chapter 7.1.4; let us therefore have a discussion of how this solution was implemented. 42 These are the pulses that does not have a value inside the interval [mean value - standard deviation, mean value + standard deviation]of at least one of the parameters that are filtered on. 43 the Time attribute points out the position in the Hashtable, and the index attribute points out the position in the LinkedList where the Pulse is stored 45 8 Design and Implementation As have been said before; there are three known points44 in the emitter data: [TOABeg, AmplitudeBeg], [TOAMax, AmplitudeMax], [TOAEnd, AmplitudeEnd]. An interpolation can therefore be done to fit these three points to a quadratic polynomial; P(x) = c1 + c2x + c3x2, where x is the TOA values. However, if the radar is in locked on mode, these three points will instead be fit to a constant with the same amplitude over time. The following steps are done to retrieve this polynomial: [10] Algorithm quadraticInterpoaltion( xx, yy ) Input: Vector xx with the x-coordinates; [TOABeg, TOAMax, TOAEnd]. Vector yy with the y-coordinates; [AmplBeg, AmplMax, AmplEnd]. Output: The quadratic polynomial; P(x) = c1+c2x+c3x2 A ← 1 TOA Beg TOA Beg 2 1 TOA Max 1 TOA End TOA Max 2 TOA End 2 c ← A-1 × yy Function p ← c1+c2x+c3x2 return p Pseudo code 2. The pseudo code for the algortihm of fitting three know points to a quadratical polynomial. Before retrieving the coefficients c from the equation c = A-1 × yy, the matrix A-1 (matrix inversion) has to be found. There are many ways of finding the inversion of a matrix, but the easiest way to implement this in an application is by using the equation: A −1 = 1 adj( A ) det( A ) As can be seen in the formula, det(A) can not be equal to 0, as a matrix is invertible if and only if det(A) ≠ 0.45 Cramer’s Rule46 was then used to retrieve the coefficients c1, c2 and c3, in the following manner: det( A1 ) det( A ) det( A2 ) c2 = det( A ) det( A3 ) c3 = det( A ) c1 = Where Ai is the matrix obtained by replacing the entries in the i: th column of A by the entries in the matrix: ⎡ Ampl Beg ⎤ yy = ⎢⎢ Ampl Max ⎥⎥ ⎢⎣ Ampl End ⎥⎦ An easy method to calculate the determinant of a square matrix is to utilize expansion by minors. A minor Mij of the n × n matrix A is the determinant of the n-1 × n-1 matrix formed by 44 TOA and Amplitude for the same pulse. Proof of this theorem can be found in [3] page 109. 46 More information about and proof of this rule can be found in [3] page 111. 45 46 8 Design and Implementation removing the i: th row and the j: th column from the matrix A. The determinant is then calculated from the sum of minors multiplied by cofactors taken over a single row or column; det( A ) = aij ⋅ C ij , where aij is the element in row i and column j and where Cij, the ∑ cofactor of aij, is; Cij = (-1)i+j·Mij. [3] [25] Any row or column can be chosen across which to take the sum of. If, for an example, the first row is chosen for the sum, then the determinant in terms of minors would be: [3] [25] det( A ) = a11 ⋅ M 11 − a12 ⋅ M 12 + a13 ⋅ M 13 − ... + a1n ⋅ M 1n Now that the theory behind this solution is clear, let us now have a discussion of how the calculation of the determinant was implemented. The calculation is done by recursively applying the above formula to the determinant, then to each minor of the determinant, then to each minor of the minor of the preceding step until a simple calculation can be performed. The recursive algorithm for this algorithm is as following: Algorithm det( matrix, n ) Input: A matrix having the size n × n, n>0 Output: The determinant of matrix if n = 2 then d ← a11·a22 - a12·a21 else d ← 0 for i ← 1 to n do M1i ← A with row 1 and column i deleted d ← d + (-1)(i-1)·matrix1i·det( M1i, n - 1 ) return d Pseudo code 3. The pseudo code for the algortihm of finding the determinant of a n × n matrix. [25] The determinants in Cramer’s rule were calculated utilizing the above algorithm, of which results could be used in simple calculations to retrieve the wanted coefficients: c1, c2 and c3. Hence, the quadratic polynomial P(x) was found. The remaining of the PRI analysis is to find the pulses on this curve, which will be discussed next. For each TOAi = TOAi-1 + ΔTOA, the amplitude Ampli is calculated as Ampli = P(TOAi), for a certain number of i: s. Now that the point [TOAi, Ampli] on the curve have been found, the closest pulse to this point has to be found, which is illustrated in the following figure: [TOAi, Ampli] Ampli - Y Closest Pulse [X, Y] TOAi - X Distance = ( TOAi − X )2 + ( Ampli − Y )2 Figure 30. A figure illustrating how the closest pulse to the point [TOAi, Ampli] on the quadratic curve is found. 47 8 Design and Implementation However, to avoid a square root in the implementation, the distance has been calculated in square. Only if the distance is less than or equal to the area of a dot in the graph, it is included as a pulse belonging to the selected emitter. This had to be done to avoid that pulses found in a large distance from the point on the curve - still being the closest pulse, are included. Besides this, an included pulse also has to fulfil the interval values of Amplitude, Frequency, PW and AOA according to the values in the selected emitter. The above method is applied on the following TOA values, where the incrementing are made using the value ΔTOA: • n number of TOA values after TOAEnd • n number of TOA values before TOABeg • The TOA values between TOABeg and TOAMax • The TOA values between TOAMax and to the first TOA value in the TOA vector47 • All the TOA values in the TOA vector. In this way some pulses below the limitampl value, and that belongs to the selected emitter, will be included. However, depending on the value of n, all the pulses below the limitampl value, and that belongs to the selected emitter, can not be found. The reason for this is that the emitter data does not contain the required data, to know the exact value of n that should be selected, so that all of the selected emitter’s pulses can be found. 8.5 Emitter Correlation An emitter correlation function seemed to be of a great importance as it would help the analyst by correlating the emitters that originates from the same radar. As a reminder to the reader; the emitters (emitter tracks) in the emitter data is updated information from different radars. This means that one radar, which has been discovered, can have many emitter records of its information. If the characteristics of this radar changes, a new emitter record is created in the emitter data. This emitter record has the same record number, but with some differences in the information stored. The record numbers can be reused if a radar has been disappeared etc. This can be done, as new and updated emitter tracks will not be created for this radar. A parameter named state indicates the current state of an emitter in the emitter data. If a radar has been newly discovered and the record number of it is used for the first time, the state will be set to one. If, however, the record number have been reused, and it is a newly discovered radar the state will have the value two. If state has the value four, it means that this emitter have been updated with information from an already discovered radar (having the same record number as this radar’s earlier emitters in the emitter data file). Finally, a state having the value eight means that this emitter is passive (it is in a locked-on mode), in other words; information about this radar will no longer be updated in new emitters, as this is not necessary. It should be pointed out that when the warning system measures emitters they are not saved in the emitter data file until they are recorded by the system. Hence, emitters having a state of one or two can have been measured but not saved into the emitter data file. Therefore, when a recording once have been started, the first emitters in the emitter data can start with a value four as state (instead of one or two). This has been taken into consideration in the implementation of this function. The two parameters; record number and state can, thus, be utilized to find out the emitters that are related in the way that they contain updated information from the same radar. If the selected emitter has a state that is equal to four, i.e. this emitter has been updated, a searching 47 As a reminder, this TOA vector contains the TOA information for the last 64 pulses in a radar scan and having an amplitude value above limitampl. 48 8 Design and Implementation is done backwards (in time) for emitters having the same record number. This is done until an emitter with a state value of one or two is found. All the emitters found in between are added to an emitter selection, as all these emitters belongs together. The same thing is done in a forward search, again, until an emitter with a state value of one or two is found. Once again, all the emitters in between are added to an emitter selection. If, however, the selected emitter has a state value of one or two, only a search forward for emitters is needed. A selected emitter having a state value of eight makes it only necessary to do a search backwards. These operations are done in the same way as mentioned above. As all the emitters are stored in a TreeMap in the EmitterStorage48 a different approach had to be made to retrieve the same functionality as a backward search. This had to be done as a previous() method or similar does not exist for the class TreeMap. Since the emitters in the TreeMap are stored in order by their Time attribute, this functionality could be implemented in the following manner: • Do a search from the first element in the TreeMap until the selected emitter is reached. The aim of this search is to find the last emitter, let us call it emitterfound, occurred in this ordered iteration having the value one or two of the parameter state. • Afterwards, all the emitters from emitterfound to the selected emitter are added into the emitter selection, if and only if they have the same record number as the selected emitter. The forward search is more obvious, where a search is done from the selected emitter and forward until an emitter having a value of state of one or two is reached. All the emitters between is then added to the emitter selection, if and only if they have the same record number as the selected emitter. In these searching techniques, the methods tailMap() and headMap() in the class TreeMap was to a great benefit. The advantage of these methods is also that their operations are done in O(1), as the objects they return are backed up by the real TreeMap. Hence, new data structures are not constructed for these operations. There were also some speculations of storing the emitter data in a database table. However, as this solution would have led to that another table must be created, which would have taken even more memory space in the HSQLDB database, the solution of searching through the TreeMap seemed better. Another reason for selecting this alternative is that the number of emitters normally is below 1000, which is not that large. Hence, searching in the data structures does not take such long time. 48 This has been discussed in section 8.3.2. 49 9 Discussion and Conclusion 9 Discussion and Conclusion This last chapter concludes the work and start with a general discussion of the project’s results. There will also be a section demonstrating a user case of the application and its analysis process. The final section will be a discussion of some of the future work, in this area and to the application, which can be made. The main goal of this project was to make the analysis of the pulse and emitter data for the analysts much easier. This goal was divided into two sub goals; the first to relate the emitters to their pulses that they originated from, the second to find and implement some statistical functionality. Two solutions have been presented and implemented to solve the first sub goal (relating the emitters to their pulses). Both these solutions will, however, not give an exact solution to this problem. The main reason for this is the lack of data in the emitter data file, which makes it impossible to get a one-to-one relation to an emitter’s pulses. However, a good estimation of this could be given by giving the user a guidance of which pulses that may belong to an emitter. This will probably help the analyst to a great extent, as it will be much easier to analyse a much smaller subset of the great number of pulses, which exist in the pulse data. The first solution used an approach of filtering the pulses utilizing a database. All the filtered values can be seen in a filter dialog, which has been implemented. This can, hence, be used to manually filter to get at narrower or wider area of pulses that are shown in a pulse selection. The second solution, an implementation of a PRI analysis, gives the analyst a much smaller pulse selection than the first solution. The reason for this is that only the pulses that fit into the quadratic function, which have been found, are included. Although, it should be pointed out that all the pulses that belong to a selected emitter will no be found, as this will require a more extensive PRI analysis49. The analyst will, although, gain from having these both solutions in the application. The motivation of this is that the results, from these two solutions, rather then to be used separately can be bring together to improve the analysis efficiency. The conclusion of this project is that it was shown that it is really difficult to find a unique identification of which pulses that belong to a certain emitter. The main difficulty for this lied in that there were a lack of data in the emitter data file; making it impossible to obtain a one-to-one relationship between the emitters and their pulses. It would therefore have been desirable to have more data available. The optimal would have been to have data for all the pulses in a radar scan, in other words a one-to-one relationship. However, this would mean that the emitter data file has to be enormously much larger than it is today, and that the storage table in the pulse processor have to be increased significantly. Hence, this may be an impossibility. A different potential possibility would have been to store the data for AmplBeg and AmplEnd and the corresponding Frequency and PW values for the start and end pulses of a radar scan, and not only for the pulses above the limitampl value. This would have made it possible to certainly retrieve all the pulses below the limitampl value. It would also have been desirable to have tighter information about the pulses in a radar scan, i.e. not just for the beginning, maximum and end pulses. Another potential possibility would have been to; instead of a TOA vector for the last 64 pulses have a more scattered TOA vector with information about some pulses at the beginning, some pulse at the maximum and some pulses at the end. This would have made it possible to make a better fitted PRI analysis. 49 This has been discussed in chapter 7.1. 50 9 Discussion and Conclusion The statistical functionality that has been implemented has been very appreciated by SaabTech, as these functions will help the analysts to perform their work in a smoother and easier way. The functionalities to implement have in this project been found by studying the application environment and analysis methods, performed by the analysts, in their current application. The correlation tool, which has been implemented, was also very appreciated by SaabTech, as the analysts today have to work several hours to obtain the same results. The reason for this is that the analyst has to do this kind of analysis manually; an emitter is selected to retrieve its data to continue with a filtering of the pulses in accordance to the emitter information. This goes on with finding the next emitter with the same record number and a search for its pulses etc. The process continues until the analyst has found a wanted number of related emitters and their respective pulses. As can be understood this process is quite time-consuming. Hence, this implemented tool can be used to quickly and simply see the pulses of all related emitters. The histogram and statistical tools can then be used to analyse the correlated emitters and their pulses to get an understanding of how the radar works. It should be noted that the application that has been developed in this project has as a requirement to be run on a computer with good performance and with an internal memory of at least 512MB. The data files that are to be analyzed have to be stored locally to obtain the best performance, instead of on a network disk. Hence, it will, as have been discussed in this master’s thesis, be faster to access these files and thereby the processing of the files will be faster. 9. 1 User Case Let us now describe a normal user case of the application. In this user case, the first thing that happens is that a user selects an emitter in the emitter graph in an intention to obtain all its pulses viewable in the pulse graph as a pulse selection. The user can now, by selecting pulses in the pulse selection seen in the pulse graph, retrieve information about these. When an emitter or a pulse is selected, their information is displayed in the information area at the left in the application. The figure below shows this scenario, where the emitters are shown in the upper graph and the pulses in the lower graph: 51 9 Discussion and Conclusion Figure 31. In this scenario, an emitter have been selected and its pulses are seen in the pulse graph. A pulse has also been selected by the user and both the emitter information and pulse information are displayed in the information area to the left. Some of the displayed GUI have been erased of confidential reasons. The user has now possibilities to utilize the different tools in the application. As an example, the histogram tool can be used to easier see the distribution of different values of the parameters: Amplitude, Frequency and PW. This can be used to see if the selection of pulses, which have been given, may depict two different radar scans from two different radars. The user can in these cases either perform the same selection of the emitter after that have selected to perform a more accurate analysis in the Settings menu, or use the filter tool to filter on narrower intervals. The figure below shows the histogram and filter tools: 52 9 Discussion and Conclusion Figure 32. This scenario is a continuation of the prevois one. Here, the histogram and filter tools can be seen. Some of the displayed GUI have been erased of confidential reasons. The user can now use the tool to find the correlated emitters. Afterwards, the pulses of these correlated emitters can be retrieved by using the corresponding tool. At this time, the user can see how the frequency etc. varies between the different emitters (the correlated emitters) by using the histogram tool. The statistics function can also be used, to see the mean value of all the pulses belonging to all the correlated emitters. These tools may give the user an easier understanding of how the analyzed radars work. Figure 33. This scenario shows the correlated emitters and their pulses. Some of the displayed GUI have been erased of confidential reasons. 53 9 Discussion and Conclusion It should be mentioned that all the application’s functionality has not been described in this user case. 9.2 Future Work In all studies and development of systems it is of a great importance that there exists some knowledge of the kind of future work that can be made. This, not only to get an understanding of what improvements that can be made to it, but also to get knowledge of the next stages in the study or development process. Hence, this section will describe some of the future work of this project and its resulting application. The first improvement that can be made in the application is to implement a more extensive PRI analysis that can make a better depiction of a radar scan of a selected emitter. The difficulty of this problem is, as have been said earlier, that a radar intentionally can change its PRI in different intervals. This means that the pulses are sent in different intervals in one and the same radar scan. A study should, hence, be done of how these PRI values are changed. Given the data in the emitter data file, can a true depiction of this be obtained? Or it is needed to have more data? We believe that, if this tool can be improved, presumably a one-to-one relationship between the emitters and their pulses can be found. The motivation for this is that the pulses only have to be correctly fitted to the function that has been retrieved. Hence, this would, according to us, result in that all the pulses in a radar scan can be found. Another improvement that could be made is presumably to utilize the so called burst vector, which can be found in the emitter data. This vector contains information of different tracks (or actually, so called bursts) that have been merged to create one single track. This single track will, hence, contain information as mean values etc. based on all these tracks together (i.e. the mean value based on all the pulses in all these merged tracks). The figure below illustrates this: One Track Tracks (or acutally bursts) containing a number of pulses Figure 34. The process of where a certain number of tracks have been merged to a single track. The reason of this cause is that instead of having all these tracks as different emitter records, the radar warning system understands by its algorithms that these tracks should be merged as their values are somewhat similar and that they originates from the same radar. Although, the single track that has been created will, however, contain the following information of the merged tracks: mean value and standard deviation value for Amplitude, Frequency, PW etc. As probably can be understood; the mean values and the standard deviation values will be less accurate in a merged track. Hence, a filtering of this information will lead to a wider interval being used. This will result in that more pulses are included in a pulse selection. Therefore utilizing the burst vector, where several separate filtering are done based on the information in the burst vector, may have been a better approach. 54 9 Discussion and Conclusion A different kind of future work is that the emitter correlation tool can be used as a tool to automatically create libraries50 with radar information. This can be done, if this tool can be improved in an approach to find the correct relations between correlated emitters and their pulses in a pulse selection (which can be done if the PRI analysis can be improved). Afterwards, characteristics in this pulse selection have to be obtained in order to get knowledge of how the radar works. However, before getting to this phase; it is required that the application have been improved to manage to uniquely find exactly all the pulses that belongs to the correlated emitters. 50 These libraries are used in the military aircrafts to discover and identify different threats. 55 References References [1] Advanced ESM capabilities for shipboard applications. http://72.14.207.104/search?q=cache:snQKXsc0sK8J:www.widebandsystems.com/publicati ons/Data%2520Sheets/Early_Warning_for_Maritime_Forces.pdf+%22pulse+descriptor%22 &hl=sv&lr=lang_en|lang_sv An article explaining some of the parameters in pulse descriptor words. [2] ANDERSSON B. ET AL. 1998. Signalspaningsteknik del 2 This book gives a mathematical and a more advanced understanding of the theories behind radar signals. [3] ANTON H. AND RORRES C . 2000. Elementary Linear Algebra. John Wiley & Sons, Inc. ISBN 0-471-17052-6 Chapter 2.4 describes cofactor expansion and Cramer’s Rule. [4] BARTION D. 1988. Modern Radar System Analysis. Artech House, Inc. ISBN 0-89006-170-X This book covers basic theoretical aspects of radar. [5] BERGKVIST B. 2005. Marinradarerfarenheter. PowerPoint Presentation, SaabTech. Good figures over examples of plotted radar scans. [6] ECKSTEIN R. 2005. Learning Java 2D, Part 2. Sun Developer Network (SDN). http://java.sun.com/developer/technicalArticles/GUI/java2d/java2dpart2.html Contains information about Java 2D; BufferedImage, double buffering etc. [7] ED P. 1996. EW Tutorial. http://ourworld.compuserve.com/homepages/edperry/ewtutor2.htm Contains an explanation of different methods used to scan with a radar. [8] EDDE B. 1993. Radar – Principles, Technology, Applications. Prentice Hall, Inc. ISBN 0-13-752346-7 Chapter 2 in this book gives an explanation of the parameters that are transmitted in radar signals. [9] Electronic Warfare and Radar Systems Engineering Handbook. Naval Air Warfare Center Weapons Division. An article that gives an explanation of some of the parameters that can be measured of the pulses. [10] ERIKSSON G. 2002. Numeriska algoritmer med Matlab. Universitetsservice US AB. Chapter 3 contains information about interpolation. [11] FOWLER A. Mixing heavy and light components. Sun Developer Network (SDN). http://java.sun.com/products/jfc/tsc/articles/mixing/ Gives an explanation of why it is not recommended to mix heavy (awt) and light (Swing) components in Java. [12] FRÉDÉRIC P. 2004. An introduction to Java fast true color 2D animation and games graphic.YOV408 Technologies. http://www.yov408.com/articles/javagraph.pdf Contains information about graphical strategies in Java 56 References [13] GALATI G. 1993. Advanced radar techniques and systems. Peter Peregrinus Ltd. ISBN 0-86341-172-X This book gives an advanced understanding of the radar. It also treats the mathematics behind radar and radar operations. [14] Glossary. Naval Air Warfare Center Weapons Division. https://ewhdbks.mugu.navy.mil/Glossary.htm A radar glossary. [15] Hash table. WIKIPEDIA – THE FREE ENCYLOPEDIA. http://en.wikipedia.org/wiki/Hashtable Information about the hash table and about its complexity etc. [16] JAMIE. Performance Benchmarking of Embedded Databases. http://jamie.ideasasylum.com/notebook/index.php?id=4 A Benchmarking of different embedded database, among which are HSQLDB that was used in this project. [17] JALICS P. 2003. BufferedFileReader. Cleveland State University. http://grail.cba.csuohio.edu/~jalics/notes/IttAJava/bufferedfilereader/bufferedfilereader.java To download the file BufferedFileReader.java. [18] JOSEFSON L. AND BERGGREN G. 1997. Telekrig. Enator Försvarsmedia AB. This book gives a basic introduction to electronic warfare and especially radar. [19] JÖNSSON L. AND ROLDAN-PRADO R. 1991. Analys av radarvarnardata med tonvikt på associering av sensorrapporter. National Defence Research Establishment Department of Information Technology. ISSN 0347-3708 This compendium gives an understanding of how radar warning data are analyzed. [20] KALIDINDI R. AND KALIDINDI R. 2005. Best practices to improve performance in JDBC. PreciseJava . http://www.precisejava.com/javaperf/j2ee/JDBC.htm Contains information about how to improve the performance of the connection between the JDBC driver and the database, [21] Least Square Method. efunda – engineering fundamentals. http://www.efunda.com/math/leastsquares/leastsquares.cfm A description of the method of least squares. [22] LORD D. AND REDDY A. Java I/O Performance Tuning. Sun Microsystems, Inc. http://www.kegel.com/java/wp-javaio.html An article that describes different methods that can be used to improve the I/O in Java. [23] MULLER H. AND WALRATH K. Using a Swing Worker Thread. Sun Developer Network (SDN). http://java.sun.com/products/jfc/tsc/articles/threads/threads2.html Gives information of how the SwingWorker class works. [24] SANDQVIST A AND GERDLE P. 2004. Lärobok i telekrigföring för luftvärnet. Mediablocket AB. This book gives a basic introduction to electronic warfare and especially radar. [25] SCHMITT S. 2004. Evaluating 3 × 3 Determinants by Recursion. http://home.att.net/~srschmitt/script_determinant3.html Discuss how the determinant can be calculated, and gives the algorithm for doing this recursively. 57 References [26] SIMPSON B. AND TOUSSI F. 2005.Hsqldb User Guide. The HSQLDB Development Group. http://www.hsqldb.org/doc/guide/ A guide of the database system HSQLDB, which is used in this project. [27] STIMSON G. 1998. Introduction to Airborne Radar. SciTech Publishing Inc. ISBN 1-891121-01-4 This book gives a more advanced introduction to radar, the characteristics of pulses etc. [28] WESTERLUND T. 2005. Pulse Processing – PPY. PowerPoint Presentation, SaabTech. Gives an understanding of how the Pulse Processing works at SaabTech. [29] WILBER G. AND PLAISTED S.1989. Intelligent Real-Time Electronic Warfare. IEEE. Gives an explanation of how PDW processing is done in Boeing Military Airplanes. [30] WILEY R. 1993. Electronic Intelligence: The Analysis of Radar Signals. Artech House, Inc. ISBN 0-89006-592-6 Gives an understanding of how the radar signals or observed and analyzed. 58 Appendix A Appendix A This appendix contains references that have been used in this master’s project. However, they have not been referenced in this thesis. [1] ANDERSSON B. ET AL. 1998. Signalspaningsteknik del 1 This book gives an introduction to part 2 of the series of books. [2] BOYD J. ET AL. Electronic Countermeasures. Peninsula Publishing. ISBN 0-932146-00-7 Chapter 5, 7 and 11 gives an understanding of signals and the analysis of signals and data. [3] COOK C. AND BERNFELD M. 1993. Radar Signals: An Introduction to Theory and Application. Academic Press. ISBN 0-89006-733-3 This book gives a theoretical knowledge of properties (both mathematical and theory) of radar. [4] PEEBLES P. 1998. Radar Principles. John Wiley & Sons, Inc. ISBN 0-471-25205-0 This book gives a basic and advanced understanding of the radar. It also treats the mathematics behind radar and radar operations. [5] RAEMER H. 1997. Radar System Principles. CRC Press, Inc. ISBN 0-8493-9481-3 This book emphasizes the basic theoretical concept required for analysis of radar systems. [6] The VolatileImage API User Guide. Sun Microsystems. http://java.sun.com/j2se/1.4/pdf/VolatileImage.pdf Contains information about the VolatileImage. 59 TRITA-CSC-E 2006: 050 ISRN-KTH/CSC/E--06/050--SE ISSN-1653-5715 www.kth.se