University of California, San Diego
Transcription
University of California, San Diego
Technical Paper University of California, San Diego Faculty Advisor: Dr. John Kosmatka Graduate Advisor: Chad Foerster Project Manager: Joshua Egbert Computer Engineering Lead: Shane Grant Safety Pilot: Thomas Hong Team: Matt Wu, Andrew Swingle, Dmitriy Obukhov, Tony Myers, Peter Xu, Tim Wheeler, Daniel Bedenko and Lewis Anderson May 2010 2009-2010 project manager:Joshua Egbert ([email protected]) 2010-2011 project manager: Andrew Swingle ([email protected]) Abstract This paper provides an overview of the design of UCSD’s UAS, designed to meet the requirements of the 2010 AUVSI student UAS competition. The team’s custom built “Falco” airframe is controlled by a Procerus Kestrel autopilot. While in the air, the plane flies through a designated set of way points and then searches a field for targets using an HD Sony FBC-H11 block video camera. The camera is mounted on a pan tilt gimbal system that points the camera at commanded GPS locations. The video is streamed live to the ground digitally over a 2.4GHz link, where it is processed and displayed to a user. Image processing takes place in two steps. First, regions of interest in a frame are located, and then target parameters are extracted from these regions. Telemetry data from the autopilot is synchronized with the video to provide accurate target positions. Safety is a priority as the aircraft can be controlled autonomously via a ground station, using a 900MHz pass through, or through a 72MHz R/C controller hard cutover. This paper describes the design and development of the UAS in three components. First, the systems design philosophy and its rationale is presented, then individual component design is described, and the paper concludes with a description of the testing that has been performed to show that the mission can be completed successfully. Contents 1 System Overview 1.1 Team . . . . . . . . . . . . 1.2 Mission Requirements and 1.3 Brief System Overview . . 1.4 Competition Preview . . . . . . . Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Flight Systems Design and Overview 2.1 Airframe . . . . . . . . . . . . . . . . . . 2.1.1 Introduction . . . . . . . . . . . 2.1.2 Design Methodology . . . . . . . 2.1.3 Competitive Assessment . . . . . 2.1.4 Design . . . . . . . . . . . . . . . 2.2 Autopilot . . . . . . . . . . . . . . . . . 2.2.1 Procerus Kestrel Autopilot . . . 2.2.2 Virtual Cockpit Ground Station 2.2.3 Software in Loop Simulation . . 2.2.4 Autopilot Safety Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . 4 . 4 . 4 . 5 . 5 . 7 . 7 . 8 . 9 . 10 3 Payloads Design and Overview 3.1 Overview . . . . . . . . . . . 3.2 Cameras . . . . . . . . . . . . 3.3 On Board Processing . . . . . 3.4 Gimbal . . . . . . . . . . . . 3.5 Payload Communications . . 3.5.1 Wireless Links . . . . 3.5.2 Serial Data Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 10 11 12 13 14 14 14 4 Ground Systems Design and Overview 4.1 Graphical User Interface Software . . . . . . . . . . . . 4.2 Image processing . . . . . . . . . . . . . . . . . . . . . 4.2.1 Image Rectification . . . . . . . . . . . . . . . . 4.2.2 Autonomous Target Location and Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 15 16 16 17 . . . . . . 18 18 18 18 19 19 20 . . . . . . . . . . . . . . 5 Testing, Performance and Safety 5.1 Individual Systems Tests . . . . . 5.1.1 Autopilot Testing . . . . . 5.1.2 Imagery Testing . . . . . 5.1.3 Airframe Testing . . . . . 5.1.4 Communications Tests . . 5.2 Full Systems Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 2 2 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Acknowledgments 21 6.1 Sponsors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 6.2 Faculty and Staff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1 SYSTEM OVERVIEW 1 1.1 System Overview Team The University of California San Diego’s AUVSI student UAS competition team is comprised of an interdisciplinary group of students. The team is led by Joshua Egbert, an aerospace engineer who has been an active member of the team for four years. Shane Grant leads the computer science portion of the team which includes Lewis Anderson, Tony Myers, Doug Libby, and Irving Ruan. The rest of the team consists of two aerospace engineers, Andrew Swingle and Tim Wheeler, two electrical engineers, Peter Xu and Dmitriy Obukhov, and a mechanical engineer, Matt Wu. The team’s faculty advisors are Dr. John Kosmatka, a professor in the Mechanical and Aerospace as well as the Structural Engineering departments, and Dr. Ryan Kastner of the Computer Science and Engineering department. Chad Foerster, a graduate student in structural engineering, serves an advisory role in the areas of autopilot integration and project management. Additionally, Thomas Hong acts as the team’s safety pilot and adviser on the airframe. 1.2 Mission Requirements and Goals AUVSI’s 2010 student UAS competition, sponsored by Seafarer chapter, simulates a real world mission as described in the rules for the 2010 competition. In the simulated environment, the UAV must autonomously navigate through a set of waypoints, staying close to the pre-determined path to avoid detection by a simulated enemy. There are two targets along the path, one located directly under and a second located off to the side of the flight path, but with a known location. After proceeding through the set of waypoints, the UAV enters a search area at which point it must navigate and explore the area to locate an identify an unknown number of targets. Finally, to simulate real world re-tasking, the search area will change during the mission and an additional target that is not a pre-determined shape must be located. The UCSD team’s goal for 2010 was not only to design a UAS with the capability to complete all of the mission objectives while maximizing autonomy, reliability, and safety, but to also design a UAS that replicates the functionality of a real world UAV. To accomplish both of these goals the team made drastic design changes including an overhaul of the imaging system despite the success of the imaging system in 2009. All the imagery payloads and the ground systems run independently of flights so that safe flight through the autopilot or safety pilot link are not compromised if any vision component fails. 1.3 Brief System Overview UCSD’s UAS is made up of three subsystems: flight systems, payloads, and ground systems. The flight systems include the airframe custom built in 2009 to meet the mission requirements, and the autopilot which is responsible for the autonomous operation of the airframe. The autopilot is managed via a ground station laptop which allows the operator to control the airplane, upload flight plans, and view aircraft diagnostics. The payloads are comprised of a microcontroller to handle input and output from the various devices and wireless communications, a camera gimbal system, a power supply, and transmitters. The camera gimbal system uses a single Sony FBC-H11 block camera that can shoot video in up to 1080i quality with automatic focus, white balance and serial controlled zoom. The gimbal for the camera was designed and built by the team to accomplish the mission. It is capable of ±90◦ tilt and continuous 360◦ pan with a resolution of .225◦ . Each component of 2 UC San Diego AUVSI 1 SYSTEM OVERVIEW 1.4 Competition Preview the payloads, including the autopilot, is housed in a separate custom built container on the airplane with standard interconnects for power and communications for ease of maintenance. The ground systems include the software and hardware used to process video, identify targets, and control the camera gimbal system on the airplane. The ground station includes a video capture card that collects the video and feeds it into a graphics processing unit (GPU) to automatically locate and identify targets within the image. The ground station computer also receives a serial feed from the microcontroller that contains all the telemetry data necessary to geo-reference the pixels in each video frame so that the GPS coordinates of the targets can be located. 1.4 Competition Preview Our UAS is designed both for performance and ease of use. Easy-access panels on the top of the airframe and a compartmentalized payload configuration allow for quick troubleshooting if either the payloads or the autopilot are not functioning properly. The RC safety pilot acts as pilot-in-command and has the final say on aircraft airworthiness. A powerful two monitor computer for imagery and a laptop for the autopilot have separate communications channels to the aircraft which are checked before each flight. The autopilot’s ground station is used to program the takeoff procedure and the initial navigation route (which can be dynamically changed in flight if necessary). The image station computer positions the camera so that the lens is safe for takeoff and begins processing video. The project manager will go through a final safety checklist with the team to verify the go-no go criteria. Upon successful completion of this, the UAS will be readied for its mission by arming its flight motor. The autopilot ground station gives the command to take off . After an autonomous take off , the aircraft transitions into a loiter while the team performs diagnostics, ensuring control and communications are satisfactory for mission completion. The image station then instructs the camera gimbal to operate in stabilization mode, where the gimbal’s position compensates for the airplane’s attitude. Simultaneously, the autopilot is commanded to begin waypoint navigation. At this point, the mission is fully underway. As the UAV navigates through its waypoints, the video camera wirelessly streams video in real time and the on board microprocessor synchronizes telemetry data with the video. This video passes through an image processing algorithm running on the image station computer, the results of which are sent through a character and shape recognition algorithm that automatically returns the target parameters. The targeting operator points the camera using a joystick, which instead of directly controlling the camera gives GPS target locations, and on board calculations performed on the microprocessor in the air point the camera at the desired GPS coordinate. When a target is located using the search, the camera will be fixed on the target’s GPS point then zoomed in to provide a better quality image of the target. The UAS is fully capable of changing the search pattern, or exploring a new location as required by the mission since this ability is critical in a real-world environment. Once all the targets have been found, the operator commands the UAS to land on a specified runway after which the motor is disarmed thus completing the mission. 3 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW 2 Flight Systems Design and Overview 2.1 2.1.1 Airframe Introduction Figure 1: The UCSD Falco Airframe, shown in payload access and flight configurations The UCSD Falco is an electric tractor propeller aircraft with a mid-placed wing, conventional tail, and tricycle landing gear. The Falco has a steerable nose gear and full aerodynamic controls with the addition of inboard flaps on the wings. With a wingspan of 10.25 ft and a length of 6.2 ft, the aircraft has a no-payload takeoff weight of 17 lbs, a maximum takeoff weight of 42 lbs, and a nominal mission takeoff weight, NMTW, of roughly 28 lbs. At NMTW, the aircraft has a clean configuration stall speed of 20 knots, a cruising speed of 42 knots for maximum efficiency, and a dash speed of 80 knots with a flight duration of over 30 minutes. The airframe meets and/or exceeds all requirements placed by the AUVSI Seafarers Chapter and the Academy of Model Aeronautics. Its design in 2009 was the direct result of interaction with the current and past members of the UCSD AUVSI team as well as market projections for a low endurance research UAS airframe. Considering the airplane meets the mission requirements and is comparable to real world UAV airframes the team chose to focus on the development of other mission critical systems for the 2010 competition. Nevertheless, a synopsis of the airframe design process and abilities will be presented to provide a comprehensive of the picture of all facets of our system design. 2.1.2 Design Methodology The team demanded a portable electric airframe capable of carrying easily accessible, heavy and voluminous payloads. Portability would aid in transportation as well as reduce shipping costs a realistic concern for a San Diego based team. Electric propulsion reduced the vibrations caused by a reciprocating engine and the associated exhaust residue which was a hassle for transportation, maintenance/repair, and cameras. Past experience with various airframes showed that payloads accessibility and volume were top-level concerns. High load capacity would allow room for payload and airframe weight growth in future years. Key performance parameters for the competition mission were payloads capacity, cruise velocity, flight duration, load-factor, and good handling characteristics (table 1). 4 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW Parameter Payloads Weight Payloads Volume Cruise Velocity Flight Duration Load Factor Safety Factor Specification >10 >1000 <45 >25 ±5 >1.2 2.1 Airframe Units Pounds Cubic Inches Knots Minutes G’s at MTOW Table 1: Team derived key performance parameters 2.1.3 Competitive Assessment The Falco must be competitive during the AUVSI competition, but its design must also be justified with a competitive assessment of the current UAV airframe market. Competition included the ACR Manta, Raytheon Cobra, Arcturus T-15E, Optimum Solutions Condor 300, and the UCSD modified Sig Rascal 110 (figure 2). These aircraft had capabilities similar to requirements set forth by the team, but were inadequate for one reason or another. The commercial UAV airframes, excluding the Sig Rascal, were far too costly with only the T-15E being electric powered. All five airframes required large shipping crates and a station wagon or SUV for short-haul transportation. The T-15E would need landing gear for AUVSI operations. The Sig Rascal 110 was previously used and modified by the UCSD team to electric propulsion and tricycle gear, but it still lacked the payload volume and easy payloads accessibility. In order to be competitive, the airframe needs to be cheaper to produce, easier to ship and transport, offer a reasonable payload volume and weight, and maximize structural and flight efficiency. Specifications for the competitors along with the Falco are listed in table 2. Figure 2: Competitors from top right CW: UCSD Sig Rascal 110, Raytheon Cobra, ACR Manta, Arcturus T-15, Optimum Solutions Condor (center) 2.1.4 Design The Falco airframe is based roughly on the Sig Rascal 110, so many of its parameters have a grounded basis in a proven design. As an example, the handling qualities of the Sig Rascal are well-liked by the team’s safety pilot so the same tail volume coefficients are used on the Falco. A conventional layout and similar sizing to the Sig Rascal also simplified the transition of the 5 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW Parameter Wing Span Length Propulsion MTOW Payload weight Payload volume Cruise velocity Units Feet Feet Type Pounds Pounds cu-in knots UCSD Falco 10.25 6.5 Electric 42 25 1600 33-45 Manta 8.8 6.2 Gas 61 15 776 39-90 2.1 Cobra 10.2 9.3 Gas >100 45 2600 50-60 Condor 10.6 6.2 2xElectric 40 13 <1000 49 T-15E 10.8 6 Electric 45 10 800 50 Airframe Sig110 9.2 6.3 Electric 25 18 <1000 35-45 Table 2: Key flight parameters of competitors listed for comparison Kestrel autopilot from the Sig Rascal to the Falco. The mid-placed wing is a compromise between static lateral stability and payloads access. With this wing placement, the center of gravity is kept below the wing while the entire upper surface of the fuselage can be opened up for very easy access to payloads. The wing is a triple-tapered design which mimics the efficiency of an elliptical planform while maintaining the manufacturability and stall characteristics of a rectangular or single/double tapered wing. A rectangular main spar with a single rectangular carry-through joiner will increase structural efficiency when compared to round spars and joiners used on some competitive airframes. Thorough aerodynamic performance analysis was performed on XFLR5, an airfoil and aircraft analysis tool for approximating properties at low Reynolds numbers (figure 3) before the configuration was finalized [1]. The fuselage and tail boom configuration allow the aircraft could fit into a mid-size Figure 3: XFLR5 allowed the aerodynamic design to be refined with relative ease sedan when broken down so that more members of the team can transport the aircraft, simplifying flight test logistics. Both the vertical and horizontal stabilizers detach from the boom to further decrease shipping and stowing volume. The Falco’s tricycle landing gear layout provides good ground stability for the autonomous takeoff and landing of the aircraft. This layout also protects the camera gimbal and propeller from impact during landing. The nose and main gear can be removed by using one allen-key each in order to reduce shipping volume. The electric propulsion system is the same system used on the Sig Rascal in past years. The system chosen is a Neutronics Corporation 1515-2y motor with a 6.7:1 gearbox controlled by a Castle Creations Phoenix HV-85 speed controller. The aircraft uses a standard APC propeller, or a stiffer Mezjlik hollow-molded carbon fiber propeller, which offers an increase in propulsion efficiency. A 6 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW 2.2 Autopilot total of 12 5500mAh lithium-polymer battery cells in series provide the aircraft with a nominal 44.4 volts and a maximum installed current draw of 65 amps. The propulsion system has been shown in flight testing to allow for missions of over 30 minutes. A summary of all aircraft parameters is provided in table 3. Parameter Wing Span Length Height Wing Area Aspect Ratio Usable Payload Volume Maximum Payload Weight Structural Weight Empty Takeoff Weight Nominal Mission Takeoff Weight Maximum Takeoff Weight Load factor at MTOW Flight Duration at NMTW Stall, Cruise, Max Velocity at NMTW Propulsion Power Units Feet Feet Feet Square Feet Cubic Inches Pounds Pounds Pounds Pounds Pounds Gs Minutes Knots Specifications 10.25 6.50 2.35 10.24 10.20 1600 25 9.2 17.3 25-28 42 ±5 >30 20, 33-45, 80 Neu 1515-2y, 6.7:1 Gearbox, Mezjlik 20x11w Prop 2x Neu 5500 6-cell Lithium Polymer Packs Table 3: Falco Specifications 2.2 2.2.1 Autopilot Procerus Kestrel Autopilot One of the primary mission objectives for the competition is autonomy, so an autopilot is needed to control the airframe for the mission. The requirements for an autopilot are: • Robust autonomous control: Autonomous waypoint navigation Autonomous takeoff and landing capabilities Reprogrammable in mission • Ease of integration with a custom built airframe • Cost effectiveness • Ability to interface with a serial device Using these criteria, the team selected Procerus Technology’s Kestrel autopilot. It weighs under 17 grams and it has all of the capabilities necessary to complete the mission. Several teams have used this autopilot in past years and the UCSD team used it during the 2008 and 2009 competitions. The 7 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW 2.2 Autopilot team has demonstrated in testing and at the 2009 competition the ability to takeoff, navigate, and land autonomously. A major design change from previous years was to use the autopilot as the sole sensor suite on the airplane for attitude and GPS for all systems, thus consolidating measurements for flight controls and imaging, which had been separated in previous iterations of the project. GPS, IMU, Pressure Magnetometer Laser Altimeter 900 MHz link Kestrel 2.4 GHz R/C Commands Telemetry RX Mux Virtual Cockpit Servos Figure 4: Command, communications, and control structure for the UCSD UAS The autopilot hardware uses a suite of sensors to provide the necessary data for controlling the aircraft, including a magnetometer, three gyroscopes, three accelerometers, dynamic and static pressure ports and a GPS receiver. The Kestrel uses an onboard 8-bit 29MHz processor to handle all the sensor data and run the code necessary for controlling the aircraft via four servo ports on the Kestrel unit and eight additional servo ports on a servo expansion board. The Kestrel uses a Maxstream 9XTend 900MHz 1W wireless modem to communicate with the ground station; this connects to the bottom face of the autopilot’s PCB board through the modem header. The autopilot is housed in an aluminum enclosure with a fan so that it remains cool and protected. Figure 5: The Kestrel autopilot. 2.2.2 Virtual Cockpit Ground Station The ground station for the Kestrel autopilot consists of a Commbox and Virtual Cockpit software. The software is loaded on a laptop connected to the Commbox via RS232; which in turn communicates with the autopilot via a Maxstream wireless modem. The Commbox has its own internal battery, allowing the ground station to be used in the field without external power. A standard R/C transmitter interfaces with the Commbox through a trainer cable, allowing manual control of the aircraft through the Kestrel. 8 UC San Diego AUVSI 2 FLIGHT SYSTEMS DESIGN AND OVERVIEW 2.2 Autopilot The Virtual Cockpit (VC) allows the user to configure the Kestrel and safely operate the unit during autonomous and semi-autonomous flight . The team used the VC to setup the Kestrel for the specific servo arrangement on the aircraft and calibrate the sensors on the Kestrel for its particular mounting arrangement within the airframe. The VC also features a PID gain window, used to tune the various PID control loops for acceptable response under steady flight. Waypoint navigation is activated through the VC. The user uploads a list of waypoints or dynamically creates a list by clicking on a geo-referenced image file. In addition, the user can manually enter additional waypoint parameters if needed. The VC and Kestrel have the ability to modify the list of waypoints in flight, allowing for dynamic retasking. A heads up display with relevant aircraft parameters and modes is present in the main window of the VC at all times for the benefit of ground station personnel (figure 6). Telemetry data for each flight is automatically stored by the software for post analysis and replying the flight in VC. More detailed data can be extracted and logged from the various sensors on the autopilot through a data logging window in VC. This option allows for precise analysis of the sensor system outputs for troubleshooting. Figure 6: Image of the autopilot ground station showing a Google Earth image of one of the fields where extensive testing was done for Kestrel autopilot. The orange outline is a flight path that was used during testing. 2.2.3 Software in Loop Simulation After the 2009 competition there were two issues that needed to be addressed with the autopilot. There were problems with higher mode transitions between takeoff, and the second is that there were long period altitude oscillation observed during flight. These issues could have been fixed with flight testings, but due to university concerns about performing autonomous flights in the San Diego area, fully autonomous flight testing have been limited in preparation for the 2010 competition compared to previous years. In order to deal with this challenge a software in loop simulation was developed to interface the Virtual Cockpit software with the RealFlight 4.5 simulator. Using the mass properties and 9 UC San Diego AUVSI 3 PAYLOADS DESIGN AND OVERVIEW aerodynamic characteristics of the UAV a model was developed for the simulation environment in RealFlight 4.5. From there the PID gains were tuned in the virtual environment to eliminate the two outstanding issues the team observed in the previous configuration. The simulation environment also allowed the team to practice using the autopilot in order to be fully prepared to perform an autonomous flight. When the team finally had an opportunity to fly the airplane autonomously the system showed improved performance as compared to the 2009 competition configuration. 2.2.4 Autopilot Safety Considerations Autopilot safety is realized in two ways in the UAS. The first is the built in autopilot failsafe that govern its behavior if communication with the ground station is lost. The second is an RX-Mux (receiver multiplexer), which allows the RC pilot to take control of the airframe at any point during the flight. The autopilot failsafes that are used are in correspondence with the rules set up by AUVSI for the competition. After 30 seconds of no communications the autopilot will return to home and loiter, and after three minutes without communications the autopilot will put the airframe into a spiraled dive. The RxMux (figure 9) allows for one set of servo outputs to be controlled by two different sources. The autopilot and the RC pilot are input signals to the mux and the RC pilot specifies which input is used. Therefore, if anything goes wrong at any point in the mission, or the autopilot is not behaving as expected, the RC pilot can take control. This capability is especially valuable when first integrating the autopilot with the airframe. Figure 7: RxMux board without any servos attached 3 3.1 Payloads Design and Overview Overview The payloads for the UAV are a Sony FBC-H11 block camera, a gimbal assembly that houses and points the camera, a Digi Rabbit microprocessor that handles all the inputs and outputs for the payloads, a power supply unit, and a digital video transmitter. The goal of the payloads system is to stabilize and control the camera gimbal assembly such that the camera points at a commanded GPS position on the ground and can zoom in to collect high resolution target data. The payloads system is designed to be modular with each individual component having its own enclosure to allow for easy of trouble shooting and organization of the payloads. The payloads system also interacts with the autopilot over a serial link to receive telemetry data as seen in figure 8, and then transmits all necessary data to the ground system for processing. 10 UC San Diego AUVSI 3 PAYLOADS DESIGN AND OVERVIEW 3.2 Cameras Figure 8: Block diagram of the three systems, with arrows indicating flow of information. 3.2 Cameras One of the primary objectives of the competition is to locate targets and identify the following parameters: background color, shape, orientation, alphanumeric, and alphanumeric color. In our design we considered a single still still digital camera, multiple USB cameras, an analog video camera, and a high definition block camera. The functional requirements for identifying targets are: • Resolve the targets at altitudes up to 500 ft • Lightweight and small enough to physically mount in the airframe • Take color image/video • Transmit video/images to the ground or on board computer in real time • 120 degree field of view to locate any off-path targets Considering the team had committed to designing a pointing gimbal, the field of view restrictions could be met with physically moving the camera instead of the solely with camera’s field of view. Therefore, video quality and features such as remote zoom capability, auto-focus, and auto-white balance were the most important aspects in selecting a camera, which lead to the Sony FBC-H11 block camera being used in the system as seen in table 4. The Sony camera can stream video in varying quality between standard definition NTSC up to 1080i HD. The limit on video quality is therefore in the transmitter used to stream the video from the ground, which is discussed later. The camera also has auto-focus and auto-white balance features which allow for good video to be taken in any lighting conditions that would be expected at competition. Another important feature of the camera is that it has up to 10X optical zoom which can easily be controlled by the microprocessor over the serial link, and this allows for sufficient resolution for target identification from any altitude allowed during the competition 11 UC San Diego AUVSI 3 PAYLOADS DESIGN AND OVERVIEW Image Quality Features Weight Size Lens Distortion Cost Power Consumption Ease of Interfacing Total Weight 4 3 2 2 4 1 1 1 FBC-H11 4 4 3 3 2 1 3 3 55 3.3 Canon S3-IS 3 3 1 2 3 4 2 1 38 On Board Processing TSC-TC33USB 2 2 4 4 1 4 4 2 36 Analog 1 1 2 1 4 1 4 34 Table 4: Camera trade study between Sony FBC-H11 block camera, Canon S3-IS digital still camera, Sentek TSC-TC33 USB video cameras, and a Blackwidow AV analog video camera. 3.3 On Board Processing For the payloads system to perform the mission the camera, autopilot, and gimbal need to function together, and therefore an on board microprocessor is needed in order to handle serial communications between the three payloads devices and the ground. The requirements for this processor were: • 4 serial ports: 3 for payloads devices, 1 for wireless link to ground • Lightweight and low power consumption • Processing power to do gimbal calculations • Parallel processing capability • Inexpensive Figure 9: Rabbit BL1800 processor. In 2009, the team interfaced with the payloads system computer with a 1.8GHz Intel Pentium processor, which requires a full operation system to run. This has more processing power than is necessary to meet the functional requirements of the on board processor, ands its power consumption 12 UC San Diego AUVSI 3 PAYLOADS DESIGN AND OVERVIEW 3.4 Gimbal was very high. Therefore, this year the team chose to use a Rabbit BL1800 processor in the system because it meets the requirements with the minimum amount of processing power. The Rabbit BL1800 processor stands out from other microcontrollers for its ability to handle serial communication. There are four serial ports available on the Rabbit, and each serial port has a dedicated pin connection, which exactly fits the systems requirements. Also, unlike many other microcontrollers, the rabbit is programmable using a relatively high level language, Dynamic C, which is similar to the standard C programming language. Dynamic C allowed the team to develop a lightweight operating system which managed the several concurrent processes the microcontroller was tasked with. Finally, there are no moving parts involved with the Rabbit processor and its main board, which makes the system very compact and allowed it to fit inside an relative small enclosure. 3.4 Gimbal The camera gimbal is an essential part of the payloads system because it allows a user to use the camera to locate and identify the ground targets in support of the overall mission. The camera was selected before the gimbal design was finalized, and therefore it drove a few of the requirements for the gimbal design. The rest of the requirements were driven by standard flight conditions of the airframe. These requirements for the design were: • 360◦ continuous rotation in azimuth direction • ±90◦ rotation for elevation • 1.5◦ resolution in both degrees of freedom • 30◦ /s elevation change • RS-232 controllable Figure 10: Right: CAD model of the gimbal assembly. Left: Assembled gimbal mounted underneath the airplane. With these minimum requirements in mind a two degree of freedom gimbal using stepper motors to drive its motion was designed and fabricated as seen in figure 10. The finished gimbal met or 13 UC San Diego AUVSI 3 PAYLOADS DESIGN AND OVERVIEW 3.5 Payload Communications exceeded all of the requirements, with the resolution on the gimbal pointing being .225◦ in both degrees of freedom. The gimbal has five operational modes all controlled by the rabbit microprocessor which takes inputs from the autopilot and receives control signals from the ground station: manual mode, stabilized mode, lock on target mode, look down mode, and takeoff/landing mode. In manual mode the gimbal is controlled manually through joystick operated from the ground station. The joystick provides an intuitive and cost effective way to manually control the motion of the camera. The manual mode does not automatically compensate for the motion of the plane, and therefore its primary purpose is initial testing and troubleshooting. While in stabilized mode the gimbal maintains position with respect to the heading of the airframe, and allows user input to change the offset. In stabilized mode, the gimbal compensates for pitch and roll of the airplane,and this mode is use during mission when searching for targets. Once a target has been located using stabilize mode, target lock mode is activated, and the gimbal maintains lock on a GPS position in order for the camera to zoom in and get a better image of the target. In the look down mode the gimbal returns to a predetermined downward looking position. This command quickly centers the camera to a known position so that the user can easily have a good reference for the camera is pointing. Finally, in takeoff/landing mode the gimbal points the camera in a direction that would protect the camera and the lens from any damage that might occur due to debris on the runway. In this mode the camera is pointed straight up into the chassis. 3.5 3.5.1 Payload Communications Wireless Links There are a total of four wireless links from the ground to the airplane during a mission. The autopilot communicates with the Virtual Cockpit ground station over a 900MHz link. The rabbit microprocessor also uses a 900 MHz link, on a different channel, and both of these 900MHz links are wireless serial connections. The safety pilot uses a standard 72MHz RC controller data link so that he can take over control of the servos if there is anything wrong with the mission. The last data link to the ground is the video link, which is a 2.4GHz Coded Orthogonal Frequency Division Multiplexing (COFDM) video link, which is a digital video signal. The transmitter used to maintain this video link was loaned to UCSD by Broadcast Microwave Services, and it allows video to be transmitted digitally instead of using an analog signal. This transmitter eliminates noise in the video which was observed in the team’s video during previous years, and therefor it provides much better video quality on the ground. The transmitter, however can only transmit NTSC video which is 720X486 resolution and therefore this is the bottleneck in the video quality. 3.5.2 Serial Data Links There are four serial links between the Rabbit microprocessor and the payloads systems, and these operate at the following baud rates and logic levels: • Camera : RS-232, 38400 • Autopilot: TTL , 115200 • Gimbal: RS-232, 9600 14 UC San Diego AUVSI 4 GROUND SYSTEMS DESIGN AND OVERVIEW • Wireless modem to ground station: CMOS, 115200 The microprocessor runs an operating system programmed by the team which cycles through the various commands it sends and receives at a rate of approximately 30Hz so that control of the payloads by the ground station is fluid. 4 4.1 Ground Systems Design and Overview Graphical User Interface Software The ground station software is entirely custom written for this competition. It is encapsulated into one program, referred to as the “Image Station.” The main purpose of the Image Station is to provide a graphical user interface (GUI), figure 11, displaying essential flight information in an intuitive and organized fashion. It is the processing core of the entire imaging pipeline because it runs on specialized hardware that can handle complex computation in real time. Figure 11: The two monitor graphical user interface developed for displaying and analyzing target data. The left columns display telemetry data and warnings, in the middle are video options and the real time video stream, and the right is a map displaying the current flight. The image station displays a constantly updating feed of live video from the airplane. It is designed in such a manner that this video can be overlayed with various information such as GPS, or the video can be toggled to display currently active image filters such as saliency. Video is acquired through a capture card which is agnostic to input format - this means that we can connect virtually any type of video device to our system and have it perform in a predictable fashion. The video data is constantly being processed by the imaging pipleine, which first calculates a saliency map to extract small regions of the image to be run through the more intensive object recognition algorithms. This data is saved into a database and classified into one of three categories: “candidate regions,” which contains interesting regions as determined by saliency,“possible targets,” which contains autonomously identified targets, and finally “verified targets,” which are “possible targets” that have been manually confirmed by the human operator. In addition to a heads up display style of displaying airplane telemetry data, the data is also displayed in a tabular fashion that updates frequently. This information is logged to disk so that it 15 UC San Diego AUVSI 4 GROUND SYSTEMS DESIGN AND OVERVIEW 4.2 Image processing is available for later analysis. All input data to the image station comes from a wireless serial link to the microcontroller on the airplane. The image station requires no other connectivity to operate; it simply needs a video source and the data link. An operator can manipulate the gimbal by using either a game pad or the keyboard and mouse. For example, moving the game pad joystick causes an instantaneous response to the gimbal position which can be observed by watching the video display or looking at the telemetry data that is constantly updated. Additional functionality include changing the camera zoom level and entering various modes of operation for the gimbal. The image station also possesses the capability to record video and quickly export images to disk. Video is automatically compressed and split into manageable sized files. Finally, the image station renders a map that tracks the current position of the airplane, current viewing area of the camera, and targets found. The user can interact with this map to send specific GPS coordinates to the gimbal to lock to, or add and remove targets manually. 4.2 4.2.1 Image processing Image Rectification An essential task for imagery captured from an aerial platform is to exactly determine the physical location of any point in an image - a process known as georeferencing. In order to find the latitude and longitude of a potential target, it was necessary to perform a series of coordinate transformations to take the camera position vector to a local WGS 84 Cartesian coordinate system. Under this WGS 84 coordinate system, actual physical locations can be derived for any point in the image. To accomplish this, the following rotation matrix was constructed: R = Rcamera,gimbal Rgimbal,platf orm Rplatf orm,unrotated Runrotated,W GS84local Where each Rij takes one frame of reference to another coordinate system. The individual Rij were constructed either via Euler angles or simple change of basis matrices that converted from North-East-Down to North-West-Up coordinate systems. With this position vector in the local WGS 84 coordinate system, we can then find the location of the airplane based upon its GPS data in this same coordinate system. With this last piece of information, we are able to project any point in the image onto the earth and derive its physical location. Given our low altitude, we make the simplifying assumption that the earth is a flat p lane, which causes negligible error and reduces computation complexity significantly. Using this information, we can use the four corner points of the image to create a homography a transformation mapping points on the image to points on the earth plane. Doing this removes the effects of projective distortion and gives our image the appearance of being viewed from directly overhead from a perpendicular viewing location. The importance of this is that before under projective perspective, shapes such as squares may have appeared as any arbitrary quadrilateral. After rectifying the image, right angles are restored to how they should look as well as parallel lines, meaning that circles look like circles and squares look like squares. This is essential for performing automated target recognition. 16 UC San Diego AUVSI 4 GROUND SYSTEMS DESIGN AND OVERVIEW 4.2.2 4.2 Image processing Autonomous Target Location and Identification Given the fact that even when operating at standard definition the onboard camera captures 320mb of data per second, we needed an approach to filter this data quickly and reliably. The solution the team chose to use was saliency, a biologically inspired approach for finding regions of interest in an image. By imitating the methods that a human eye uses to effortlessly find interesting regions in the field of view, the saliency algorithm identifies regions of each frame that stand out based upon various types of contrast including color and edge intensity. Since the targets we are searching for are well defined shapes (strong edges) that consist of solid colors they will inevitably cause a large contrast with their background (mostly natural objects such as grass and foliage, which are all closely colored and have no well defined edges in general). The end result of this is that for each frame of video we autonomously filter out an overwhelming proportion of it as “non interesting” and do not process it to look for an actual target. This allows us to keep our analysis real time. The output of the saliency algorithm is a heat map of sorts which displays more “interesting” areas as bright regions, while regions we deem “unintersting” are darker as seen in figure 12. Figure 12: Example of the Saliency algorithm running on a set of practice targets. The targets appear as bright white regions in this noisy example, and these regions would be fed through the autonomous identification algorithms. The saliency algorithm itself is implemented to run on a graphics processing unit (GPU), which allows for massively parallel computation allowing us to achieve real-time analysis for both standard and high definition images. An additional advantage using the GPU is that we can perform other CPU bound processes without the overhead of saliency processing to slow down computation. It is important to note that saliency does not make any subjective discrimination as to what type of objects it finds “interesting” or not. The algorithm has no knowledge of what a target is, what constitutes background noise, or what we would like it to find interesting. It simply uses biologically motivated conditions for finding a region “interesting” that match what the human visual system would do given a similar situation. This means that some regions determined as interesting may in fact be irrelevant to our purposes, though the chances of a target region being deemed unintersting are, in our testing, very low. Additionally, we can exploit knowledge about targets to further reduce the output of the saliency 17 UC San Diego AUVSI 5 TESTING, PERFORMANCE AND SAFETY Figure 13: Two samples of the autonomous target and shape recognition algorithm which show the shape and alphanumeric being extracted from the full color image of the target. algorithm - domain specific knowledge such as expected physical dimensions. From an operator perspective, saliency makes searching for targets a much more feasible in a time constrained environment - it acts like a second set of eyes scanning over images and finding things to look at. By highlighting potential targets in real time, the human operator can perform other tasks while watching the video stream without fear of missing a target. Once these interesting regions are found via saliency, they are processed using a trained algorithm that performs more complex analysis to autonomously determine target parameters such as location, shape, letter, and color information. 5 Testing, Performance and Safety 5.1 5.1.1 Individual Systems Tests Autopilot Testing The UCSD team began the 2009-2010 year with a functioning autopilot that could perform all of the mission objectives. Because the university was concerned about legal and liability issues regarding autonomous flying, the primary method of testing the autopilot for most of the year was software in loop simulations. However, the team did perform test flights near the end of the year to ensure that everything was performing as expected. 5.1.2 Imagery Testing To test the image systems in a realistic setting the team constructed over 20 competition style targets, and testing for the imagery systems was done in segments as functionality was developed. Initially, the camera was used to take pictures of targets from the roof of buildings on campus to collect sample target data. Then the camera was statically mounted in the airframe to judge image quality from the air. However, until the gimbal was installed in the airplane it was difficult to locate 18 UC San Diego AUVSI 5 TESTING, PERFORMANCE AND SAFETY 5.1 Individual Systems Tests targets on the ground. Once the gimbal was completed and installed in the airplane additional testing was performed, which showed that targets could be identified from a reasonable altitude even without zoom implemented. Finally, remote zoom on the camera was implemented, which allowed all targets to be identified. 5.1.3 Airframe Testing The first Falco airframe was constructed 2.5 lbs below its estimated weight of 19.5 lbs and met its static proof load of 6 Gs. Assembly time out of the shipping crate was below 5 minutes requiring nothing more than 3 allen-keys. Taxi testing proved that the aircraft absolutely did not tip over under maximum lateral forces thus protecting the wing-tips and propeller. The first flights were conducted without payloads at its empty flying weight in order to learn its handling characteristics and expand its flight envelope. With the center of gravity at its forward limit, the aircraft was not able to stall while still being able to rotate well under takeoff speed and flare upon landing. The control surfaces provided excellent control and authority, and the aircraft did not require any trim due to the team’s extensive aero-analysis. Drag was significantly reduced when compared to the Sig Rascal; this was noticed by the 30% decrease in battery usage when compared to flights of the same duration and takeoff weight. The empty aircraft (more susceptible to winds and gusts) was flown in crosswinds at 17 knots gusting up to 22. The only fix needed for the airframe since its initial design on the drawing board was that the motor’s anti-vibration bulkhead needed to be strengthened. A dead-stick landing off the runway due to this problem showed that the structures were durable enough while still maintaining a very competitive empty weight fraction. Again, after flight testing, the airframe was able to be broken down in less than 5 minutes and transported in a mid-sized sedan with an open seat for a passenger. Storage and shipping volume can be seen in figure 14. Figure 14: Shipping configuration; fuselage, tail and boom are between the wings 5.1.4 Communications Tests With four different wireless signals being transmitted from the air to the ground, testing to ensure that there was no interference was essential. Initially, the team’s safety pilot wanted to use a 2.4GHz RC controller to control the airplane, however this was in the same frequency spectrum as the video link. Therefore, ground range testing was performed which showed that the safety 19 UC San Diego AUVSI 5 TESTING, PERFORMANCE AND SAFETY 5.2 Full Systems Tests pilot intermittently would lose control of the airplane due to the video transmitter saturating the frequency. This testing drove the decision to place the safety pilot in the 72Mhz frequency spectrum to avoid any potential conflict between signals on the plane and the safety line. Tests were also done to ensure the two different 900Mhz links on the airplane did not cause any harmful interference. There are multiple channels to us in the 900Mhz band, so this turned out not to be a very large issue. Another set of communication tests that was performed were range tests for each of the individual signals. The safety pilot’s 72MHz signal is a standard RC link and worked well as soon as it was implemented. Range test were performed in the air for the 2.4GHz video link, and it was found that the link was solid up to the desired range, but it was sensitive to being blocked by carbon in the airframe. Therefore, the antenna was moved out to the tail where line of sight was always maintained. Finally, range tests were performed across the UCSD campus with the rabbit’s 900MHz link, which showed that even across the ground the connection could be maintained for the distance required by the competition. 5.2 Full Systems Tests Full system tests were performed to show that all of the individual components could function together in the airframe. Also, it was necessary to show that the communications could function simultaneously. All of the integrated components can be seen in figure 15. The full systems test were performed locally in San Diego at an RC field with the autopilot on board processing GPS and attitude, but not controlling the airplane. Targets were placed in the field for each test and as gimbal functionality was increased better images were collected. Video and stills from all flight tests were captured using the ground station computer to be analyzed later to determine areas where the image systems could be improved. Progressing towards a final competition ready state the team intends to perform fully autonomous practice missions These final tests will focus on completing simulated setup and mission within the time constraints placed on the competition. Full target messages will be spelled out, and the members of the team operating the ground systems will be able to locate them. Figure 15: The image on the left shows the payloads systems after being fully integrated with the airframe. The far left is the video transmitter, the three aluminum boxes are the rabbit microprocessor, power supply, and autopilot enclosures. At the right of the image are the gimbal and its drivers.The image on the right is 2009’s configuration, which is much more disorganized than the enclosure design used this year. 20 UC San Diego AUVSI REFERENCES 6 Acknowledgments 6.1 Sponsors 6.2 Faculty and Staff We would like to thank the following members of the UC San Diego community for all of the guidance that they have given our team: • Professor John Kosmatka, Faculty Advisor, Structural and Mechanical/Aerospace Engineering • Dr. Mike Wiskerchen and Tehseen Lazzouni, California Space Grant Consortium • Chad Foerster, Graduate Student, UCSD SE Department • Professor Ryan Kastner, Computer Science and Engineering • Thomas Hong, Graduate Student, UCSD SE Department • Ben Ochoa of 2d3 for help with geo-referencing • Scripps Institution of Oceanography Machine Shop References [1] XFLR5. http://xflr5.sourceforge.net/xflr5.htm. 21 UC San Diego AUVSI
Similar documents
University of California, San Diego
Peppys flight characteristics are derived from modifying a known airframe for predictable results.
More information