Communication software for a blimp

Transcription

Autonomous Systems Lab
Prof. Roland Siegwart
Bachelor-Thesis
Communication software
for a blimp
Spring Term 2012
Supervised by:
Laurent Kneip
Konrad Rudin
Author:
Matthias Burri
Declaration of Originality
I hereby declare that the written work I have submitted entitled
Communication software for a blimp
is original work which I alone have authored and which is written in my own words.
Author(s)
Matthias
Burri
Supervising lecturer
Roland
Siegwart
With the signature I declare that I have been informed regarding normal academic
citation rules and that I have read and understood the information on ’Citation
etiquette’ (http://www.ethz.ch/students/exams/plagiarism_s_en.pdf). The
citation conventions usual to the discipline in question here have been respected.
The above written work may be tested electronically for plagiarism.
Place and date
Signature
ii
Preamble
Thanks to Laurent Kneip and Konrad Rudin for all the given support for this thesis
and the whole Project Skye.
Thanks to Lorenz Meier and the Pixhawk crew for the support while implementing the flight management unit of Skye and the debugging of the IMU operating
system. Without this valuable help, Skye would not be flying yet.
Thanks to Markus Achtelik for the ROS camera-driver, the Wi-Fi equipment generously placed at our disposal and numerous hours of support.
Thanks to Andreas Schaffner, Anton Ledergerber, Christina Grob, Claudio Ruch,
Daniel Meier, Fiona Sartori, Johannes Weichart, Luca Müri, Lukas Gasser, Lukas
Mosimann, Matthias Krebs, Miro Käch, Nicolas Vuilliomeret, Randy Michaud and
Simon Laube. It was amazing to work with you, thanks a lot!
iii
Contents
Abstract
vii
1 Introduction
1.1
Motivation
1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Global Structure
1
3
2.1
Overview of Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . .
3
2.2
Wireless Communication Channels . . . . . . . . . . . . . . . . . . .
4
2.3
Wired Communication Channels . . . . . . . . . . . . . . . . . . . .
4
2.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
3 Low Level Communication
7
3.1
Piloting and Telemetry Concept . . . . . . . . . . . . . . . . . . . . .
7
3.2
Realisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
3.2.1
Software Low Level Communication . . . . . . . . . . . . . .
9
3.2.2
Xbee Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
3.2.3
RC Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.3
4 Image Handling Software
15
4.1
Image Handling Task . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
4.2
Realisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
4.2.1
The Cameras on Skye
. . . . . . . . . . . . . . . . . . . . .
17
4.2.2
Image Handler . . . . . . . . . . . . . . . . . . . . . . . . . .
18
4.2.3
Dynamic Bandwidth Adaptation . . . . . . . . . . . . . . . .
20
4.2.4
UDP Mavlink ROS Bridge
. . . . . . . . . . . . . . . . . . .
22
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
4.3
5 Results
5.1
Wireless Communication . . . . . . . . . . . . . . . . . . . . . . . . .
23
23
5.1.1
Stress Test Xbee 868 MHz . . . . . . . . . . . . . . . . . . . .
23
5.1.2
Range Test Xbee 2.4 GHz . . . . . . . . . . . . . . . . . . . .
24
5.1.3
Wi-Fi Test . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
Image Handler Write Performance . . . . . . . . . . . . . . . . . . .
25
5.2.1
Comparison in Terms of Image Format . . . . . . . . . . . . .
25
5.2.2
Comparison Rosbag versus Image Handler Code . . . . . . .
26
5.3
Dynamic Bandwidth Adaptation . . . . . . . . . . . . . . . . . . . .
26
5.4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
5.2
6 Conclusion and Outlook
6.1
6.2
Conclusion
29
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
6.1.1
Image Handling . . . . . . . . . . . . . . . . . . . . . . . . . .
29
6.1.2
. . . . . . . . . . . . . . . . . . .
29
6.1.3
Dynamic Bandwidth Adaptation . . . . . . . . . . . . . . . .
29
6.1.4
Low Level Communication
. . . . . . . . . . . . . . . . . . .
30
6.1.5
Whole System Performance . . . . . . . . . . . . . . . . . . .
30
Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
A Manuals
31
A.1 Instruction RC for SKYE . . . . . . . . . . . . . . . . . . . . . . . .
B Calculations
32
33
B.1 Needed Bandwidth for Low Level Communication . . . . . . . . . .
C Code
34
35
C.1 Camera Class Reference . . . . . . . . . . . . . . . . . . . . . . . . .
35
C.1.1 Detailed Description . . . . . . . . . . . . . . . . . . . . . . .
39
C.1.2 Constructor & Destructor Documentation . . . . . . . . . . .
39
C.1.3 Member Function Documentation
. . . . . . . . . . . . . . .
39
C.1.4 Member Data Documentation . . . . . . . . . . . . . . . . . .
44
C.2 Modus Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . .
48
50
50
. . . . . . . . . . . . . . .
51
52
C.3 Mavlink_UDP_link Class Reference . . . . . . . . . . . . . . . . . .
53
55
55
. . . . . . . . . . . . . . .
56
58
C.4 Converter Class Reference . . . . . . . . . . . . . . . . . . . . . . . .
60
64
64
. . . . . . . . . . . . . . .
65
70
D Measurements
75
D.1 Xbee 868MHz Stress Test . . . . . . . . . . . . . . . . . . . . . . . .
76
D.2 Xbee 2.4GHz Rangetest . . . . . . . . . . . . . . . . . . . . . . . . .
77
D.3 Available Wi-Fi Bandwidth in Function of Distance . . . . . . . . . .
78
D.4 Achieved Image Saving for 3 Formats . . . . . . . . . . . . . . . . . .
81
D.5 Image Handler versus Rosbag . . . . . . . . . . . . . . . . . . . . . .
82
D.6 Bandwidth Adaptation Demonstration Plots . . . . . . . . . . . . . .
83
E Datasheets
85
E.1 Xbee Pro 868 MHz . . . . . . . . . . . . . . . . . . . . . . . . . . . .
86
E.2 Xbee Laird Techologies 2.4 GHz
. . . . . . . . . . . . . . . . . . . .
88
E.3 Prosilica . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
89
E.4 MV Blue Fox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91
Bibliography
93
vii
Abstract
This project deal with the implementation of reliable wireless communication and
data-logging channels between a flying blimp and a ground-station. Different channels exist: the low-level board can communicate telemetry and control commands
via a radio modem as well as RC commands over a standard 2.4 GHz RC link. On
the other hand, the high-level board employs a driver for the camera along with onboard down-sampling and compression. Time-varying bandwidth limitations form
a special challenge when it comes to the choice of suitable communication protocols.
This thesis was part of the Focus project Skye that developed and constructed
a small, unmanned, and highly agile spherical blimp.
Acronyms and Abbreviations
ETH
D-MAVT
D-ITET
ASL
ZHdK
DRZ
Eidgenössische Technische Hochschule
Swiss Federal Institute of Technology
Departement für Maschinenbau und Verfahrenstechnik
Department of Mechanical and Process Engineering
Departement für Informationstechnologie und Elektrotechnik
Department of Information Technology and Electrical Engineering
Autonomous Systems Lab
Zürcher Hochschule der Künste
Zürich University of the Arts
Disney Research Zürich
Technical Acronyms
IMU
FMU
CCD
HLB
LLB
GS
UAV
MAV
Mavlink
UDP
TCP
IP
UART
PPM
DoF
GUI
HMI
ROS
3D
GPS
CAD
CPU
VGA
Inertial Measurement Unit
Flight Management Unit
Charge Coupled Device
High Level Board
Low Level Board
Ground-Station
Unmanned Aerial Vehicle
Micro Aerial Vehicle
Micro Aerial Vehicle Communication Protocol
User Datagram Protocol
Transmission Control Protocol
Internet Protocol
Universal Asynchronous Receiver Transmitter
Pulse Position Modulation
Degree of Freedom
Graphical User Interface
Human Machine Interface
Robot Operating System
Three Dimensional
Global Positioning System
Computer Aided Design
Central Processing Unit
Video Graphics Array
1
Chapter 1
Introduction
This chapter gives an introduction to the UAV communication problematic and
shortly presents the focus project Skye that integrates modules developed in this
work.
1.1
Motivation
The focus project Skye had the goal to develop an agile blimp that could record
high quality images from air. The difference from existing quadrocopter solutions
should be the possibility to stay in air for minimum four times longer and be save
enough to fly over crowd. The team had the vision of a “flying eye”.
This lead to a spherical design of the hull and a tetrahedral motor alignment. Why
do we need communication with the blimp? The blimp is semi-autonomous, it is
intended to provide intuitive control from a ground-station as if the user would
simply move a “flying eye”.
Figure 1.1 shows the system. For more details about every component and the
concept of Skye please refer to the final report [4].
Figure 1.1: Overview of the system Skye
Chapter 1. Introduction
2
Focus project Skye is based at the Autonomous Systems Lab (ASL) lead by Prof.
Siegwart. The initial input to the project was jointly given to the students by
Disney Research Zurich (DRZ), a research lab of “The Walt Disney Company”, and
ASL in form of a product development idea. This bachelor-thesis takes place in the
context of the entire project and deals with the communication software and the
on-board image handling.
Even if the software is intended for use on a blimp, the work done could also be
used on similar unmanned aerial vehicles and/or with over camera configurations.
Figure 1.2: Problem Description
For communication different channels had to be implemented for telemetry, control commands and image streaming. For image handling the challenges were the
adaptation to the bandwidth that changes over time with the position of the UAV.
3
Chapter 2
Global Structure
This chapter gives an overview of the communication structure of Skye. Even if all
the code was implemented for this project, it could easily be ported to similar flying
platforms. The chapter goes on with describing the entire software implementation,
and closes with a short summary.
2.1
Overview of Data Flow
The piloting signals given by the user could reach the blimp via three ways. (See
2.1) The first is through a laptop with the QGroundControl program as interface
over Wi-Fi. This is the normal way for all user signals regarding camera settings,
and in the opposite direction for streamed images. The second way also is through
the laptop and QGroundControl, however over an Xbee-Link. This is the normal
way for all navigation commands and the telemetry signals. The last way is a
standard RC. This way is planned as backup if Xbee communication fails.
Figure 2.1: Overview Communication Structure
Chapter 2. Global Structure
4
A single board computer - the so called High Level Board - manages all the image
processing. It receives images from the cameras, saves them on the hard disk and
compresses them for live streaming down to ground-station. In the case of Skye,
there are three cameras on board but the same structure would work with one single
camera.
The flight management runs on a micro controller integrated in the Inertial Measurement Unit (IMU) board. It functions as a bridge to the different sensors and
actuators types. The flight management unit processes all the commands needed
for navigation and telemetry. It fulfils three tasks: It receives the navigation commands, feeds them to the right controller, and collects all interesting parameters of
our system.
2.2
Wireless Communication Channels
As seen in the beginning of this chapter, Skye uses three wireless communication
channels. The Low Level Board receives and sends data over an Xbee-chip. This
communication is limited to a few kilobits per second but is quite stable, which
is needed for controlling a flying object. In case the Xbee communication fails, a
standard RC-link is connected to the flight management unit, so that the blimp
could be controlled by normal RC-commands. The existence of this third channel
also has legal reason.1 . In Switzerland you are not allowed to fly fully autonomously.
This means that you must always have the possibility to take control over your
remote aircraft and are not allowed to fly out of the line of sight.
The High Level Board sends and receives data over common Wi-Fi. This is somewhat risky, because Wi-Fi is quite limited in distance, but the advantage is a large
bandwidth for short distances, so that images in low quality can be streamed at
high frequency.
Communication Data transmitted
Limitation
Advantage
Wi-Fi
Pictures
Short Range
Large Bandwidth
Xbee
Navigation Commands
and Telemetry
Limited
width
Good Reliability
RC-Link
Navigation Commands
Unidirectional
Communication
Band-
High Reliability
Table 2.1: Overview of Communication Channels
2.3
Wired Communication Channels
Beside the wireless connection from ground-station to the blimp, there are also
wired connection on the blimp. The high level board is connected to the cameras
via USB and Gigabit Ethernet. A serial UART allows communication between high
level and low level board. Finally the IMU is connected to radio chip, GPS module
and the motor platforms over UART.
The wired connections are presented here for completeness. This thesis only does the
communication from the ground-station until the flight management unit (FMU)
1 Verordnung
des UVEK über Luftfahrzeuge besonderer Kategorien, 1. April 2011
5
2.4. Summary
and communication of the high level board with the cameras. The communication
of the FMU with the motor-platforms was an own semester project [1].
Wired
nel
Chan-
Data transmitted
Connection
Gigabit Ethernet
High Resolution Images
HLB to Prosilica Camera
USB
Low Resolution Images
HLB to Bluefox Camera
Sata
Pictures
HLB to SSD
UART
FMU
FMU
Commands and Telemetry
FMU
FMU
to
to
to
to
Xbee
Motor Platforms
HLB
GPS
Table 2.2: Overview of Wired Communication Channels
2.4
Summary
This chapter gave an overview over the communication and image-handling structure of an image capturing remote aerial vehicle like a blimp. The data ways of
images and steering commands as well as telemetry were shortly described and the
arguments for each channel were presented.
In the next chapter the high level part of the software will be engrossed.
Chapter 2. Global Structure
6
7
Chapter 3
This chapter contain the work done on the low-level part of the system. This is
the part of communication that handles the transmission of control commands and
telemetry. For the transmission of images please read chapter 4.
First comes a detailed explanation of concept and structure with an overview on
used software and protocols, then a section about the realisation of the Xbee Link,
after that a few words about the RC link, and finally a short summary.
3.1
Piloting and Telemetry Concept
The concept of Skye envisages that on the blimp a micro controller would do the
flight management.1 The advantage of micro controllers are that they are embedded
and weight only few grams. Furthermore, separating flight management from image
handling on distinct processors keeps the system more modular. Fly with a different
payload than a camera system would be possible without major difficulties.
The second important part of the steering concept was pretty intuitive the groundstation. That the solution would have to contain a laptop was pretty intuitive,
because using only a remote control makes it nearly impossible to receive feedback
from the blimp.
A touch-screen and a 3D mouse were selected as input devices in order to ensure a
user-friendly and intuitive control of the 6 DoF system Skye2 .
The decision of implementing a standard remote control was taken for the sake of
security. The RC allows to bypass the laptop signal if this fails or to interrupt
autonomous flights and manually take over the UAV if it behaves in an abnormal
way. In the RC mode Skye takes over control of two degrees of freedom and presents
itself as a non-holonomic vehicle to the user. This finally allows to control the blimp
in a air-plane-like fashion.
The RC transmitter sends analog PPM signal and an own receiver feeds them
directly to the microprocessor on the IMU board. This is the most sure communi1 The flight management consists in receiving commands, process them and then sends the right
signal to the right motors. In the other direction sensor values are to be read out, processed and
sent to ground station.
2 Skye is designed to fly without any connection between translation and rotation. Therefore
the moving direction and the camera direction can be arbitrary chosen unlike a fixed camera on
an over UAV.
Chapter 3. Low Level Communication
8
cation way since it will always be possible to control the system as long as the IMU
and the receiver have power.3
The possibility to bypass the laptop was realised by directly checking inside the
controller app whether regular control is allowed. If the switch on the remote is put
in the right position, this mode is prohibited and all control input messages sent
by the ground-station are ignored. The flight management unit then falls back to
a basic, well-tested and reliable manual control mode.
Figure 3.1 shows the detailed structure of the low level part of Skye.4
On the top we see the structure of one of the four motor-platforms. They are
all identical and fixed in a tetrahedral arrangement to the spherical hull of Skye.
Below the motor-platform we see the IMU board with the different apps running in
the autopilot and an SD card for logging. At the foot of the figure we can see on
the left side the laptop and on the right side the remote.
3 More reliability would only be possible if a redundant communication system would be added
and to really be redundant this would also need a power source with redundant circuits. Such
a solution makes sense for space mission, but for the developing of an unmanned air vehicle this
would be an overkill.
4 Even if this document only threats the communication from ground-station to the UAV rest
of system is shown to allow better understanding.
9
3.2. Realisation
Figure 3.1: Detailed Low Level Software Structure
3.2
Realisation
The following section handles the software chosen for the realisation of the low level
communication concept.
3.2.1
Software Low Level Communication
At the informatics department of ETH Zürich, project Pixhawk tackles quite
similar tasks as Skye. For different UAVs a complete and exhaustive software
communication framework were developed. This set of tools include a GUI program
called QGroundControl, a communication protocol called Mavlink and different
autopilots running on the platform for flight managements. On the hardware side
the pxIMU board was developed that integrated all the needed sensors for the
inertial measurement unit (IMU) and a microprocessor.
10
In other words, the aimed solution pre-exists for quadrocopters and coaxial mini
helicopters. To reinvent the wheel was not the goal and as the team developing the
human machine interface had chosen QGroundControl as a graphical user interface,
and the team developing the electronic the pxIMU, two first reasons for using the
Mavlink protocol were given. The next good reason was that Mavlink was specially
designed for micro and unmanned aerial vehicles in comparison to most commonly
used internet protocols.
In comparison to the User Datagram Protocol (UDP), the generated overhead is
substantially smaller since the header uses only eight bytes instead of the usual
sixty-four in UDP packages. Compared to an other very common protocol, the
transmission control protocol (TCP), the Mavlink protocol had the advantage of
not retransmitting damaged messages. This is important in order to ensure that
always the freshest control commands arrive at the system, and no buffering delays
are going along with increasing rate of broken packages. This is important because
any delay could possibly conduct to loss of control or even a crash of the UAV.
3.2.2
Xbee Link
Requirements
A wireless module with the following properties had to be chosen for the physical
transmission of any information:
• Safe and reliable
• Transmitting a least 1.5 kB/s and ideally 3 kB/s.
Both requirements are related to the data transmitted over this channel. First the
operator has to constantly be able to take over control of the unmanned vehicle5 the
unmanned vehicle, and and this requires a reliable connection.
The second requirement follows from an estimation of the messages size that has
to be transmitted (see table 3.1)), followed by an estimation of the continuously
available bandwidth (see Appendix B.1). The result shows that a minimum of
500 B/s from ground-station to blimp and 985 B/s in the over direction was required.
The bandwidth requirement of sporadic messages mainly needed at take-off, for
changing flight mode, and at the end of a mission are negligible in this evaluation.
First solution: Xbee Pro 868 MHz
The first envisaged solution that theoretically fulfills the requirements was a XBeePRO 868 long-range embedded RF module. According to the data sheet (see Appendix E.1) this radio module is able to transmit 24 kb/s. This is for bidirectional
communication and would give 1.5 kB/s in each direction. The line-of-sight range
was indicated to be up to 40 km, so even with not optimal antennas a range of some
kilometres would be assured. As the UAV has to be operated in line-of-sight, the
probability to lose connection should be negligible.
5 “Wer
ein Modellluftfahrzeug mit einem Gewicht bis 30 kg betreibt, muss stets direkten Augenkontakt zum Luftfahrzeug halten.” Verordnung des UVEK über Luftfahrzeuge besonderer
Kategorien[5], Art.17
“User of a model air craft with a weight less of 30 kg, have always direct sight contact to the air
craft.”
11
Direction
Ground-station to UAV
UAV to ground-station
3.2. Realisation
Contained Information
Frequency of use
Control Input
Set Mode
Set Parameter
Waypoints
Always
Specific moment
Specific moment
Before an autonomous flight
Position
Attitude
Battery Status
Heartbeat
Get Mode
Get Parameter
Get Waypoints
Always
Always
Always
Always
Specific moment
Specific moment
Specific moment
Table 3.1: Used Messages
So the bandwidth requirement seams fulfilled. When the ordered modules arrived,
we started off with a connection test, and everything seemed to be in best order
first. On the final platform, however, the communication fails randomly after some
seconds to half an hour of operation time, which seriously complicated the entire
testing phase.
One of the problems that could soon be identified was the transmission power. If
this is greater than the minimum 6 the modules need at least a distance of 10 m in
between in order to operate well.
The next problem with XBee-PRO 868 modules was harder to identify: the operating system of the FMU board had a configuration lack so that writing on more
than one serial devices lead to a crash. Finding this out took a couple of weeks
since the problem was erroneously searched in the MavLink app, the configuration
of the Xbee module and the design of the motherboard7 .
After knowing the first two problems and still having connection breakdowns a
stress test was made. (see section 5.1.1 and appendix D.1) It turned out that the
modules stopped working after maximally 170 kB of received data and only restart
to work after a complete shut-down has been performed. The exact reason for this
problem has not been determined yet, because it was more convenient to try out
another module. Possible reasons could be a desired shutdown caused by overheat
protection, a hard-to-see damage on the hardware layer during the assembly or
simply a defect chip.
Final solution: Xbee Laird Technologies 2.4 GHz
After the problems with the XBee-PRO 868 team-decision to search and buy new
chips was taken.
The alternative found was a Laird Technologies 2.4 GHz Wireless Module that has
the same Xbee footprint as the XBee-PRO 868. This means that integration on
Skye would cause minimal problems.
6 The minimum transmit power was 1 mW and for longer range the modules were configured to
25 mW
7 This board was also developed for Fokus Project Skye. For more information please see
the relative semester [1].
12
Following the data sheet (see Appendix E.2) this radio module is able to transmit
500 kb/s and has a range of 240 m indoor and up to 2.4 km outdoor if the modules are
within in line-of-sight. The baud rate of the serial devices of the IMU was configured
to 115200 b/s so that the modules only could transmit about 14.375 kb/s. For more
details about the available bandwidth and the achieved ranges with Skye please
see section 5.1.2 and appendix D.2.
The major drawback of this solution was the limited range but this range is still
big enough to trust in these chips, that worked perfectly from on the first day.
3.2.3
RC Link
The RC channel was realised with a 7 channel Futaba remote control.
From the seven channels available three channels are used to control the orientation
of Skye and one channel to give thrust in camera direction. The fifth channel is
used to enable and disable the control over Xbee and the sixth to switch between
the two different control modes8 . The last channel is a trim channel that is used
to set the altitude the blimp should hold. This would allow the user to focus on
steering the blimp in the right direction since he had not to care about the altitude.
Table 3.2 gives an Overview of the RC setup. For further information how to fly
Skye with RC see the manual in the Appendix A.1.
Channel
Functionality Observation
1
ROLL
2
Nick/Pitch
3
Throttle
4
Yaw
5
Link
ON/OFF
If ON the ground station can do nothing over Xbee
6
Controller
1/2
Switch between controllers
7
Elevation
control
Trim button
Table 3.2: Overview of RC channels
3.3
Summary
This chapter handles the communication concept between ground-station and UAV
and the RC link. For normal operation the user will control the UAV over an
8 While testing the first control mode was a “direct control” and the second an “assisted control”
mode. The assisted control mode is stabilising t rotation of Skye. Since the blimp is spherical
and the center of buoyancy and gravity are at the same point, the system is only Lyapunov stable.
Together with the fact that a sphere has nearly zero resistance against rotation, the system is
not easily controllable over remote without the angular velocity controller if it is already rotating.
For further information about the control modes please refer to the bachelor-thesis about the
Human Machine Inteface [2]. More information about the different implemented controllers see
the bachelor-thesis related [3].
13
3.3. Summary
Xbee link. For emergency, a standart remote control allows to take over manual
control and safely come back to earth. Then the chosen software for realisation of
the concept were presented with the Mavlink protocol for transmission of data and
QGroundControl as GUI. After that the realisation of the radio connection and the
occurred problems with XBee-PRO 868 were discussed. Last but not least, the
concept and realisation of the RC link was presented.
In the next chapter the image handling on the high level part of the system will be
handled also as the image transmission and adaptation to the bandwidth.
14
15
Chapter 4
Image Handling Software
This chapter contains the work done on the high level part of the system. First
comes the concept of the image handling, in the second part you will find details
about realisation with the used software, the image handling features and communication to ground-station. At the end you will find a short summary.
The final project concept of Focus project Skye1 , planned the blimp with two
main functions: fly agile movements and record high quality images. On the level of
communication all needed to fly is basically integrated in the embedded communication (see chapter 3). Let’s now show the whole image handling done on the high
level part of the system presenting the payload of Skye.
4.1
Image Handling Task
The basic principle of any digital camera involves three steps: Images are recorded
by a CDD chip that converts light into a discrete signal. The second step consists of
converting this data into the desired format. The last one consists of transmitting
the image to memory medium or displaying it on the screen, in order to save or
show it to the user, respectively.
For a filming UAV the challenges of these tasks are that the images are not taken at
the place at which the operator is, that the amount of data that can be transmitted
over wireless is in general lower than those over cable and that the connection is
less sure against suddenly interruption.
The concept chosen for Skye was to separate the image recording, processing and
saving on different hardware components, instead of having them all integrated in
a single component (see figure 4.1).
For image recording Skye carries different cameras that are connected to a computer. From this processor the images are then sent to a hard disk on-board and a
downsized and compressed version of the image is sent to the ground-station.
1 see
the final report for details [4], pp. 45-50.
Chapter 4. Image Handling Software
16
Figure 4.1: Concept Image Handling
This solution was chosen because it is an ambivalent. The same computer could be
used for different cameras or even for something completely different i.e. running a
controller.
The external hard disk has the advantage of being able to be removed very fast.
This is important for in field use and the project goal to not have to pause more
than five minutes before restarting with exchanged battery and new storage place2 .
Saving images on-board on a hard disk is also a good protection of data losses due
to bad connection.
For the cameras DRZ demands a camera resolution of about 5 mega pixels and a
global shutter to prevent disturbances caused by any rolling shutter for computer
vision. It occurs that a single camera could not fulfil all requirements (see section
4.2.1). Therefore a second, low resolution camera was installed. To expand the field
of possible application to generation of 3D reconstruction a second low resolution
camera was added to the concept.
Live view from an operating UAV was a big goal since the begin of the project,
and the developers of the human machine interface3 needed it for the navigating
mode with the touch-screen. Unfortunately Wi-Fi bandwidth often drops if sender
or receiver are moving, causing delays in the image transmission. To minimize this
effects the decision to implement a dynamic bandwidth adaptation in the image
handler node was taken.
4.2
Realisation
For the realisation of the image handling, project supervisors highly recommended
the use of the Robot Operation System(ROS)4 , since ROS had prior been used
for similar projects with good results. It has the benefit of coming with preimplemented nodes for some camera types.
After choosing ROS, an Ubuntu version was preferred as operation system for the
laptop and High Level Board. The fact that Disney wanted a touch-screen tablet
PC for piloting had caused some trouble in the beginning. Tablet PCs have not yet
been on the market for very long, and thus it was difficult to find a good enough
driver to really use the touch-screen technology. In the end, the touch-screen is
working well on Ubuntu 11.10, but because of a missing driver for 11.10 the high
level board is working with Ubuntu 10.04 LTS.
2 For
Project goals, please refer to the final project report [4], pp. 45-50
the related bachelor-thesis for further information [2].
4 This software is a meta operating system, meaning it runs within another operation system.
In ROS, it is possible to define single nodes and have them intercommunicate. Each node is a
program by itself, which can send and receive messages. As example one camera node publishes
images, and a second node receives the message that contains all the data.
ROS is supported on Ubuntu, while versions for Windows and Mac are not supported.
3 See
17
4.2. Realisation
In this section you will first find a part informing about the camera system installed
on Skye, and then sections handling in details the realisation of the image handler
node, the dynamic bandwidth adaptation and the UDP Mavlink ROS bridge.
In figure 4.2 you see how the concept was realised.
Figure 4.2: Overview Image Handling
4.2.1
The Cameras on Skye
The following section is a summary of the camera system on Skye. For more
information, please refer to the final report [4] or the data-sheets of the cameras
(See E.3 and E.4).
The blimp finally carried three cameras. Let’s have a look why: DRZ demands a
camera resolution of around 5 mega pixel and a global shutter5 to prevent disturbances caused by any rolling shutter for computer vision. This lead to the choice
of the Prosilica board level camera. The recording modes defined in table 4.1 set
additional requirements to the system.
5 This means that the value of any pixel is acquired at the same moment. Most of the cameras
read out the sensor line by line this leads to deformed images.
High quality images
Resolution
5 MP
Global shutter
must
18
Prosilica
GB 2450 C
Matrix Vision
BlueFOX
MLC200wC
yes
yes
no
no
yes
yes
yes
no
54 g
115 g
54◦ H x 41.9◦ V
14 g
3g
138◦ H
recording modes
frame rate
min 24 fps @ 0.3 MP
min 2 fps @ 5 MP
general information
Weight camera
Weight lens
Angle of view
Table 4.1: Camera system
One can see that all the must-have demands could be fulfilled by Prosilica. But
due to technical reasons, it would not be possible to take high quality as well as low
quality images for the live stream at the same time. Therefore, Skye had to carry
a second camera with low resolution to enable live stream. Installing two identical
low resolution cameras provided additional information for 3D reconstruction in
post fly processing.
4.2.2
Image Handler
This section presents more details on the development of the image handler node,
which is the main software piece to be created for the High Level Board since the
camera driver nodes are already existing.
First step for realisation of a code was to define the parameter of the node and
the structure to store it. Then came the implementation of all functions needed
for sending and saving. Finally the code was extended with a dynamic bandwidth
adaptation (see 4.2.3).
Choice of the available parameters
For image recording it was necessary to give the possibility to choice the format
and corresponding compression. These are important points because format has
a great influence on the amount of storage place needed and compression would
highly influence the time needed to save an image.
If algorithms should reprocess images later for 3D reconstruction, it is important
that the images there not compressed or only compressed lossless. A format that
supports lossless compression is for example the PNG-format.
Another use of the node could be to grasp images published with a high frequency
and save them in order to later generate a Video. In this case you need a good
compression that allows you to record longer sequences. An optimal format for this
case is the JPEG-format. JPEG compressed not only much more data than PNG,
the compression is also quite faster.
A third scenario is that the image data should be conserved raw. The data stream
generated by the camera contains more information than the saved image. For
19
4.2. Realisation
example the raw image is often a few bits larger than the final image, due to the
used colour interpolation algorithms. The raw image format is in general different
for any camera model, but one format that also saves raw images is the TIFF
format.
Further to format and corresponding compression, a path6 to the folder where the
image should be saved and a image name was needed. For uniqueness of any image
a time stamp will be added automatically.
The last parameter needed for image saving was save percent7 , that allows to save
only a fraction of the incoming images.
To reduce the image size below the limit that could be transmitted with the Mavlink
protocol and for convenience, the possibility of resizing and rotating images was
added to the image handler node.
The last part that had to be configurable was the image sending. With send percent
the percentage of sent images could be set like the percentage for saving and a topic
name for publishing can be specified.
The described parameters in table 4.2 are that is configurable in the image handler
node. The node is capable to store multiple modus so that for example a first
modus could be used for saving images at high frequency with high resolution and
a second to save a frame every few seconds with low resolution to have a preview.
To enable and disable efficiently the image saving or sending function that are not
always needed the boolean save image and send image there also implemented.
Functionality Summary of the options
Save
Resize
Rotate
Send
Supported Formats: JPEG,PNG and TIFF
Resize the image to be less or equal the maximal high and width
Possible range: 0 - 360◦ but optimized for 90◦ ,180◦ and 270◦
Possibility to publish the image to different nodes
Table 4.2: Summary Functionalities Image Handler
Configuration Communication
The image handler node could be reconfigured in three ways. The first implemented
was with a ROS message containing the fields of the configuration struct. This solution would function but if the message was updated, all over nodes communicating
with the image handler had also to be adapted.
An improved solution was the use of the ROS parameter server : The parameter
server stores the parameters of all running nodes in the same way, and allows loading the parameter from a configuration file while starting the node or could be
reconfigured with a GUI package(see figure 4.3).
6 Finally this was realised with two parameters on the ROS server: rootpath and path. This
separation allows for example having the same folder structure and path on the file system and an
external hard disk. To save the images on the external hard disk instead of the local one only the
rootpath had to be changed.
7 Only fraction of the type 1 are possible values. Any other value will be rounded to the nearest
n
fraction.
20
Figure 4.3: Dynamic Reconfigure GUI of Image-Handler
So the configurations like destination folders and image formats are loaded from
configuration files then launched, and the reconfigure tool allows the user to adapt
quite easy to actual wishes while the system is running.
4.2.3
Dynamic Bandwidth Adaptation
Let’s now discover the dynamic bandwidth adaptation . Even if the code is implemented inside the same class that the image handling, the concept merits to have
its one section.
The image transmission from the UAV to the ground is done over Wi-Fi. The
argument for this technology were
• Wi-Fi is widely-used support and cheap hardware
• Wi-Fi transmits enough data for live stream in VGA resolution
• Wi-Fi frequencies are available all over the world8 .
8 This is i.e. not the case for the Xbee: The frequency band of 868 MHz is only available in
European Union and for flying in the United States another module working 900 MHz is needed.
21
4.2. Realisation
The first big issue was to find a way to get information about the actual bandwidth.
The chosen strategy had three points:
1. Remember the size of the sent image
2. Measure how many bytes were not transmitted
3. Calculate the actual bandwidth and adapt the image stream
The first point was easy since the node anywhere is treating the image data. The
second point was a bit more complicated, since the desired information were on
a different layer than the ROS nodes. First try was to measure the data in the
send buffer of the socket sending the image. Measurements were done in the UDP
Mavlink ROS node but comes out that send buffer is not dynamic. The next step
was then to try out several network monitors, because this one got the dynamic
information. Goal was to find a monitoring tool that could be run from command
line to launch it with system function in C++. Further it should be possible to
execute it without super-user rights. The need of root rights was the main problem
for most of the networking tools but with ss9 a program that fulfils the requirements
was found.
To calculate the actual bandwidth need a low pass filtering for the received measurements, because measured values very often make big jumps. So the measured
values are average with the last n there n is the window size of the filter. The dynamic bandwidth adaptation is implemented with a state machine. First the actual
bandwidth is estimated twice and the percentage of sent image adapted to it in the
first mode of the modus vector10 . After that the node tries the next f ind_limit
images to increase the stream if the send queue gets smaller and reduce it in the
other case. Finally the node assumes that correct bandwidth was found and the
next keep_state images it does not adapt the stream.
Figure 4.4: Dynamic Bandwidth Adaption
For the details of implementation please see the Appendix C. In section 5.3 you will
find the results of working software.
9 Stands
for Socket Statistics
is assumed that only one image mode of the stream modes in the image handler is sending
over Wireless.
10 It
4.2.4
22
UDP Mavlink to ROS Bridge
On the high level board all the whole image handling is done with ROS code. The
system of ROS with the each program in a node is programmed in a way that it does
not matter if the nodes are running on the same machine or not. The condition to
launch a system other multiple machines is only that they have a secure shell (SSH)
connection. For developing and testing of Skye it is a good way to have the ground
station as well as the high level board working on ROS. The main disadvantage of
ROS is that it is only supported for Ubuntu unlike QGroundControl used as human
machine interface.
The way to get rid of this dependency in future is to have a ROS node on the Linux
machine that communicates with the ground-station with the UDP protocol. The
reason is that UDP is one of the two most important protocol used in Internet and
therefore all operating system understand it.
The node is structured is in three threads11 running in parallel. (See table 4.3) The
treads allows that images are streamed to ground-station while Mavlink messages
are received. The node is able to read values from the ROS parameter server and
send them to the parameter and calibration widget of QGroundControl. It is also
possible to write parameter values from the ground-station in the node. At the
moment the received values were not written systematically written on the ROS
parameter server and but it is enough to configure the image stream.
Thread
Task(s)
Heart-Beat
Mavlink Receive
Bandwidth Measure
Show that system is alive and transmission of parameters
Read the incoming bytes and decode Mavlink messages
Measure UDP socket send queue length
Table 4.3: Overview Threads UDP Mavlink ROS Bridge
4.3
Summary
This chapter gave you an idea of available features for the image handling part
of the UAV. First was shown the concept, then the focus was on the realisation
with a side step to the on-board camera system. The configuration options of the
image-handler node there described with possible scenarios of use. Last was shown
the connection node between the ROS world and the human machine interface.
Let’s now have a look at the performance achieved with the implemented system in
the next chapter.
11 Threads
are basically program that runs in parallel but share for example some data.
23
Chapter 5
Results
This chapter will give you an overview of the achieved results with the developed
software for this thesis. After the realisation of the code the interesting was the
performance that could be achieved on a real system. If not mentioned otherwise
all the tests were done one the single board computer1 flying on Skye. For range
tests a laptop2 as ground-station was further used.
5.1
Wireless Communication
For wireless communication three test there done. The first was to find out the
problem with the 868 MHz modules. The second was made to have an estimation
of the distance the user could have to the UAV before connection is dropping. The
last test was the same as the third but for Wi-Fi instead of Xbee.
5.1.1
Stress Test Xbee 868 MHz
The stress test was made indoor and the modules were at 15 m in a line-of-sight.
The configuration was the standard one until the transmitter power was higher
with 25 mW. The used software was hterm and both modules were sending with
the same frequencies a message of a give size. (See Appendix D) While testing the
modules blocked in the best case after 170’000 B(see figure 5.1). Once the modules
blocked the modules only send again after powering them off completely. After
these measurements the decision was taken in the team that new Xbee chips have
to be bought.
1 For
TM D510 Processor.
R
interested readers it is a LP-170 PICO-ITX Board with an Intel Atom
TM
R
ThinkPad T400 with an Intel
2 Duo processor
2 Lenovo
Chapter 5. Results
24
Figure 5.1: Xbee Stress Test
5.1.2
Range Test Xbee 2.4 GHz
The goal of this test was to get an idea of the range that Skye will be under control
if the steer it with a2.4 GHz Xbee, since the 868 MHz Xbee had much higher range
according to datasheets. The test was done indoor at ETH Hönggerberg in big
hall 100 m long. The outdoor test was also made at ETH Hönggerberg along the
Wolfgang-Pauli-Strasse. The street is 400 m long but there are building and threes
so that only 160 m were available for testing.
As you can see in figure 5.2, the bandwidth drops a bit fast in the building as
outdoor. Outdoor you see quite well then the module arrives near buildings, the
connection drops. If you show in the appendix D.2 you will see in the measured
data that at almost 400 m a few bytes passed again, but connection get lost again.
This has to do with the reflection of the radio waves, and it seems that it had a
wall that strengthened the signal.
Figure 5.2: Xbee Range Test Indoor and Outdoor
5.1.3
Wi-Fi Test
This test was done on the roof top of the CLA building at ETH Centrum. The
router and the high level board were both posed on the ground.
The test gave a very pessimistic result: the Wi-Fi connectivity gets lost after only
45 m to 50 m (see figure 5.3 ). Indeed it comes out that the Wi-Fi is not so bad. A
25
5.2. Image Handler Write Performance
week later testing dynamic bandwidth adaptation and having the router on a meter
over ground and carrying the high level board on the arms, connectivity was still
available after 102 m. So the not optimal placement of the antennas has a lot of
influence on your testing results.
So the test brings out that in line-of-sight 100 m are still possible even if there is no
live stream but only some images per second. If you have not perfectly a line-ofsight as on the ground, this range drops to 40 m and if you want to have live view
around a corner only a few meters are possible.
Figure 5.3: Wi-Fi Bandwidth Limit in Function of Distance
5.2
Image Handler Write Performance
Two tests were made for the save speed of the image handler. The first was to
compare the limit of images that could be saved any second for the different formats
the system supports. (See table 5.1) The second test was to compare the image
handler to an existing product: ROSBAG. This package is part of the ROS system
and designed to save the messages that are sent between the ROS nodes. The
recording function does not any conversion of the data and save the entire message
in its own format so that the images had to be extract later on. The idea of these
two tests was to allow future user to choice the right configuration.
5.2.1
Comparison in Terms of Image Format
In table 5.1 you can see the result of the first test and we can say that the limit
of the system is around 23 frames per second VGA images. The first proposed
explication is the limited CPU on the system. Since this is a challenge if all three
cameras are running at the same time on Skye, it is not the only limiting factor.
Another is that the imwrite function3 used in the code to write the image to disk
is working in a blocking mode. That means that every time an image is saved, time
is lost waiting on confirmation that image is written correctly to disk.
3 This
function is part of the OpenCV library
Chapter 5. Results
Saved Images per Second
Saved Images per Second
26
Resolution
PNG
JPEG
TIFF
0.3 MP
5 MP
21.58
1.63
18.69
1.05-1.24
22.98
2.42
Table 5.1: Achieved Image Saving Speed for 3 Formats
The test results lead to not subscribe the image of all cameras in a single node but
have an image handler node for any camera running. A possible future issue to
improve the speed would be to treat incoming image in several parallel threads so
that blocking of one thread does not block processing of a new message.
5.2.2
Comparison Rosbag versus Image Handler Code
The result of the second test is shown in figure 5.4. This test was made on the laptop
with an image publishing node that published a series of images of a precedent flight.
The test goals were to find the maximal save rate and to become an idea of how
many images were lost if the system is running at its limits of CPU and read-write
speeds.
The result was that the image handler is able to save more images per second than
rosbag but those rosbag losses less images. Save fewer images but lose no image
sounds contradictory, in fact it is not. rosbag save as much data as possible every
second and the data that comes more is buffered. The Image Handler on the over
hand has only a callback queue of ten images. This means that only ten images
received are buffered to be saved. If the node receives more images that it is able
to save, the last received images are discarded.
Figure 5.4: Comparison Image Handler and Rosbag
So rosbag is better if you will be sure to get all images, but be aware that if a buffer
is going greater and greater this can be quite dangerous and even lead to a system
crash. The Image Handler has the advantage of saving with a higher rate. Further
the images have already a format that could be open on any computer while the
images in the rosbag-file has first to be extract and uses about 2.5 times the space
on disk.
5.3
The aim of dynamic bandwidth adaptation was to note overload the network by
sending to much data. In figure 5.5 you see the resulting behaviour of the image
27
5.4. Summary
handling system. The principle is working and the adaptation takes place, even if
there are many jumps in the curve. This jump are nearly impossible to avoid do to
the discrete nature of the send per cent parameter in the image handler. For example
if the node sends every image instead of ever second, this is an augmentation of 100
Figure 5.5: Working Bandwidth Adaptation
The ways to avoid such jumps could be to change the frame rate of the camera
instead of changing the fraction of saved images. This will be a bit more linear
but had the disadvantage image are also recorded not regularly. A much simpler
way would be a switch steered by the send queue. If the send queue is empty the
switch allows image publishing in the image handler node otherwise images are not
transmitted. This enforces that the send queue is empty from time to time and
would help avoiding delays in image transmission.
5.4
Summary
In this chapter the results of the different tests done for this thesis were presented.
The first test presented was the Xbee 868 MHz stress-test that lead to order the Xbee
2.4 GHz modules (see chap 3). The next two tests show the available bandwidth
in function of distance for the Xbee 2.4 GHz and the Wi-Fi. After the wireless
test come to performance test of the Image-Handler. They look at the influence
of format type on the image write performance and compare the image-handler to
rosbag. Last but not least, the behaviour of the dynamic bandwidth adaptation
was presented.
Let’s now get to the conclusion.
Chapter 5. Results
28
29
Chapter 6
Conclusion and Outlook
This chapter should give you an overview of that was achieved and learned, and
give an Outlook of that could further be integrated to the developped software.
6.1
6.1.1
Conclusion
Image Handling
The image handler is working and at least a fourth million images were already
recorded during test and demo flights. The required image saving rates could be
achieved and the quality of the entire framework has been gradually improved during
the intensive testing phases. Only, the fact that the save rate of the image handler
is just under 25 Hz is a little disappointing. In general, the image handler represents
a useful and modular tool for live streaming, and the real-time resize and rotate
functions were quite handy features.
6.1.2
A similar node was existing for serial interfaces. With this example code and good
documentation [6] basic communication from ROS to QGroundControl and image
transmission was implemented quite quick. It is a good basic for future features.
Especially the handling of the parameter should be extended from some parameters
to a generic solution. This solution will then allow to dynamically reconfigure the
parameter server on a system running ROS over Mavlink from a ground-station
that does not know ROS.
6.1.3
The idea of the bandwidth adaptation turned out to be good, but the realisation
quite tricky. Available bandwidth is difficult to predict especially in and around
buildings due to reflections. This way, a displacement of a couple of meters can
cause sudden jumps in the available bandwidth. The implemented solution more or
less follows the available bandwidth but, in the worst case, even a tree or a rotation
of the blimp can already lead to the next jump in the stream rate. The easiest
solution would have been to simply implement a non-buffering transmission for the
Chapter 6. Conclusion and Outlook
30
images. If the send queue is not empty, the image is simply dropped instead of
adding it to the transmit buffer.
6.1.4
The integration of the Xbee was a lot more difficult than expected. The main problem here was not that the module would have been to complicated to operate, but
the complex context of an entire integrated system complicating the error search.
The main lesson of this experience: be careful with out-of-the-box solutions. They
mostly work, but a time buffer needs to be previewed for all unexpected eventualities.
The transmission of the navigation commands is now working stably and reliably
up to 180 m. Longer ranges with the module on the blimp are certainly possible,
as long as it stays within line-of-sight and does not go too close to the ground. For
security it is advised to make a range test before starting any outdoor flight at least
prior to the first take-off.
6.1.5
Whole System Performance
Working live view allows to navigate Skye like a eye in the Air. The user sees
what is in front of the camera, gives inputs over touch-screen, the Blimp flies in the
desired direction and the images are recorded on-board. With the 2.4 GHz Xbee
system control loss due to communication problems has never occurred up to now,
and the indoor flight over a big crowd at in the main building of ETH Zürich was
a great success and proof of reliability.
The hole blimp is still a prototype and it is still a long way to become a real market
product. But after the year of Focus project , the project continues and , and
will for sure lead to other blimp prototypes.
6.2
Outlook
The communication software for the Blimp is set up and all the functions are working. As of now, issue will be to make the use of the camera system simpler to use
for people that are not familiar with software development. On the image handling
side it would be interesting to do more than only sending and saving. Going further
and for example doing motion detection or collision avoidance are only two options
of what could implemented next on the powerful high-level processor.
31
Appendix A
Manuals
32
APPENDIX A. MANUALS
RC Control
THROTTLE (3)
Y
A
W
(4)
NICK / PITCH (2)
Y
A
W
R
O
L
L
THROTTLE
R
O
LL
(1)
NICK / PITCH
Switches
Switch
Channel
To User
Away from User
Note
B
7
Enable Controller
Disable
Controller
Controller stabilises the angular velocities.
D
intern
exp(-20%)
No exp()
5
RC Link ON
RC Link OFF
A
E
G
Link OFF: QgroundControl has the control
over Skye
H
Trimm Channel (6)
Not used yet. Set the vertical velocity. Turn left to let Skye sink and right to let Skye go up.
Note:
To prevent turn the power switch on and off in the proper sequence
1. Pull throttle stick to idle position
2. Turn on the transmitter power
3. Confirm the proper model memory has been selected (normaly nothing to do)
4. Test all controlls
5. Make a range check
6. After flying bring throttle stick to idle positition and engage the kill switch to disarm the motors
7. Turn off the transmitter Power
33
Appendix B. Calculations
34
Appendix B
Calculations
B.1
Needed Bandwidth for Low Level Communication
Command
Fields
Size
Frequency
Bandwith
Control Input
Header
3D vector direction
3D vector orientation
Checksum
25 B
20 Hz
500 B/s
Position
Header
Timestamp
Latitude
Longitude
Altitude
3D vector speed
Compass heading
Checksum
28 B
5 Hz
140 B/s
Attitude
Header
Timestamp
3D vector orientation
Orientation quaterion
Checksum
32 B
25 Hz
800 B/s
12 B
3 Hz
36 B/s
9B
1 Hz
9 B/s
Header
Accu ID
4 Cell Voltages
Battery Status
Current Battery
Battery remaining
Checksum
Heartbeat
Header
System Identity
System Status
Operating Mode
Checksum
Table B.1: Estimation Needed Low Level Bandwidth
35
Appendix C
Code
The hole code implemented for image handling on Skye is contained in three ROS
packages. The first is called skye_msgs and only contains the ROS messages that
were needed on Skye. The second is called Video and contains the image handling
nodes that does the work on the UAV and an image player that can republishes
recorded images of a real flight.
All the image handling code is available on “www.github.com/skye-git”.
C.1
Camera Class Reference
#include <camera.h>
Public Member Functions
• void add_modus (std::vector< Modus > &modi_to_add)
Add new modus to the modi that are already in use.
• void band_apt_refresh_data (const skye_msgs::ConnectionStatePtr
&con_state)
Write the incoming new data about the connection in the state.
• void callback (const sensor_msgs::ImageConstPtr &image)
Callback for images published without Camera ( p. 35) information.
• Camera (ros::NodeHandle &nh, CamID cam_id, Verbosity_t
verbos=VERBOSITY_EVENTS, bool long_timestamp=false)
Camera ( p. 35) Constructor.
• void configure_modus (Video::CameraConfig &config, uint32_t level)
Update the modus and changes it if changes are wanted by the dynamic reconfigure
gui.
• void optimize_to_bandwidth ()
Choose the optimal modus adapted to the actual Bandwidth.
Appendix C. Code
36
• void update_camera_info (const sensor_msgs::CameraInfoConstPtr
&info)
• void update_modus_ros (const skye_msgs::ImagehandlerModusConstPtr
&msg)
Update the modus and changes it if changes are wanted in the form of a ROS
message.
Public Attributes
• image_transport::ImageTransport it
The node handle for the ROS image transport.
Private Member Functions
• void band_apt_find_limit ()
This functions checks the length of send queue. If this one is filling up, the
numbers of sent images is reduced else we will try to send one more image.
• void band_apt_guess_bandwidth ()
Knowing how many messages were sent and how many are in the send queue, a
guess of the available send percentag is made.
• void band_apt_init ()
Prepare the variable need for bandwidth estimation and set state to WAIT_MEASUREMENTS.
• void band_apt_send_data ()
• void change_send_rate (Modus &modus, int send_every_XXX_image)
Change the amount of sent images of the modus.
• int pos_pub_of_topic (std::string topic)
Gives position of the topic in the publisher vector and return -1 if the publisher
isn’t declared.
• bool resize_image (cv::Mat &image, Modus &modus, double &fx, double
&fy)
resize image to be smaller or equal the max high and width
• bool rotate_image (cv::Mat &src, cv::Mat &dest, Modus &modus)
Rotate the image. The function is very efficient for rotating 90,180 and 270
degrees and primarily implemented to switch images that there upside down do
the camera fixation on the UAV.
• bool save_image (cv_bridge::CvImageConstPtr
msgs::CameraInfoConstPtr info, Modus modi)
Save an image with the settings specified in the modus.
cvImageP,
sensor_-
37
C.1. Camera Class Reference
• bool send_image
modi)
(cv_bridge::CvImageConstPtr
cvImageP,
Modus
Send an image with the wished parameters.
• void show_all_pub_topic ()
Show all the topic the publisher vector contains.
• uint64_t time_stamp ()
Time stamp used for saving images to avoid overwriting files with the same name.
Private Attributes
• BandwidthAdaptationsState_t bandwidth_state
The actual state of the bandwidth adaptation.
adaptation for the possible states.
See the method bandwidth_-
• CamID_t camera_id
The private node handle of the camera.
• long int counter
Counts handled images.
• int counter_find_limit
Counter of the how many steps are done.
• int counter_same_state
Counter of how many cycles with assumed same bandwidth were done.
• int find_limit
Constant that defines how many step is done to approach the nearest the real
bandwidth.
• boost::format g_format
Needed for the assembling of the image name to save with the time stamp and
the path.
• std::deque< unsigned long int > guessed_available_bandwidth
Sample container memorising the available bandwidth for the last steps.
• int keep_state
Constant that defines how many images the bandwidth has to be assumed equal
after an estimation cycles.
• std::deque< ros::Time > last_stamps
Sample container to memorise the moment when the bytes were sent.
• uint64_t last_time_stamp
Contains time info of the last saved image and is needed to avoid saving to image
with the same timestamp.
Appendix C. Code
38
• double max_step
Indicates the maximum factor that the new send percentage could be greater than
the old. If set to two, the bandwidth adaptation is not allowed to send all images
if before only 25% were sent, because the factor would be four and by this higher
than the maximum step.
• double mean_available_bandwidth
The average of bytes the could be sent per seconds. If no limit is present the
number simply indicates the used bandwidth.
• double mean_queue_len
The average of bytes retained in the send queue of the udp_mavlink_ros_node
calculated with the information of the connection state messages.
• std::deque< unsigned long int > message_size
Sample container for memorise how many bytes were sent.
• std::vector< Modus > modi
Container with all active modes. This makes possible to handle the same image
in different ways. I.e it could be saved in two different formats or republished
with with different topics to different nodes.
• double old_mean_queue_len
The average of bytes retained in the send queue of the step before. This information is need to now if the send queue is growing up or decreasing.
• ros::NodeHandle prn
The ROS subscriber that will subscribe Modus ( p. 48) messages. Is here to allow
over node to reconfigure the node with ROS messages.
• std::vector< image_transport::Publisher > pub
The vector of publisher contains a single publisher for any active Modus ( p. 48)
and makes possible to publish the handled images under different topics. For
example the normal /cam_namespace/single_shotand cam_namespace/image_stream.
• ros::Publisher pub_bandwidth
The ROS publisher that will publishes the used bandwidth and the percentage of
images that are sent.
• bool pub_initialized
Boolean value to check if the publisher for the Bandwidth messages is already
initialised. If the camera instance never received a feedback from the udp_mavlink_ros_node, the image are probably not streamed to an over machine,
and bandwidth estimation and adaptation will not be started.
• dynamic_reconfigure::Server<
server_
Video::CameraConfig
>
reconfigure_-
The reconfigure server makes possible do dynamicly reconfigure the image handler
node containing an instance of the camera class.
• std::deque< unsigned long int > send_queue_lengths
39
Sample container for the data need to calculate mean_queue_len.
• ros::Subscriber sub_Command
The ROS subscriber that will subscribe Modus ( p. 48) messages. Is here to allow
over nodes to reconfigure the node with ROS messages.
• ros::Subscriber sub_connect
The ROS subscriber that will subscribe ConnectionState messages. Is here to
receive feedback from the udp_mavlink_ros about how many image could be sent.
• Verbosity_t verbosity
The verbosity level of the image handling functions.
• int window_size
This number indicates how many old states are remembered before being discarded.
• bool yyyymmdd_enabled
Switches for the time stamp after the image name. If true a long time stamp of
the form YYYYMMDDHHMMSS∗∗ is used. If false only a time stamp of the
format HHMMSS∗∗ is appended to the filename.
C.1.1
Detailed Description
Definition at line 60 of file camera.h.
C.1.2
Constructor & Destructor Documentation
C.1.2.1
Camera::Camera (ros::NodeHandle & nh, CamID cam_id,
Verbosity_t verbos = VERBOSITY_EVENTS, bool long_timestamp
= false)
Camera (p. 35) Constructor.
Parameters
nh The nodehandle in which the Camera (p. 35) is used
cam_id The ID of the Camera (p. 35)
verbos The niveau of verbosity (Silent, Events, Debug)
long_timestamp If true, the saved images will have the form: "frame
name"+"date"+"time of day" else only "frame name"+"time of day"
Definition at line 720 of file camera.cpp.
C.1.3
Member Function Documentation
C.1.3.1
void Camera::add_modus (std::vector< Modus > &
modi_to_add)
Add new modus to the modi that are already in use.
Appendix C. Code
40
Parameters
modi_to_add Vector with all the new modi that should be added
C.1.3.2
void Camera::band_apt_find_limit () [private]
This functions checks the length of send queue. If this one is filling up, the numbers
of sent images is reduced else we will try to send one more image.
Send queue grew up => reduce send percentage
Send queue get shorter => send more images
Publish the new data
C.1.3.3
void Camera::band_apt_guess_bandwidth () [private]
Knowing how many messages were sent and how many are in the send queue, a
guess of the available send percentag is made.
Published the estimated bandwidth
Change Image stream to actual bandwith
C.1.3.4
void Camera::band_apt_init () [private]
Prepare the variable need for bandwidth estimation and set state to WAIT_MEASUREMENTS.
Fill the queue with zeros.
C.1.3.5
void Camera::band_apt_refresh_data (const
skye_msgs::ConnectionStatePtr & con_state)
Write the incoming new data about the connection in the state.
Parameters
con_state Message containing the queue_size of the send and receive buffer,
the amount of sent bytes and over connection information.
Init publisher to publish measured bandwidth by receiving the first ConnectionState
message
Write new data in memory
Reject the oldest elements
Calculate basic statistics for the over functions
Activate bandwidth adaptation if allowed
41
C.1.3.6
void Camera::band_apt_send_data () [private]
C.1.3.7
void Camera::callback (const sensor_msgs::ImageConstPtr &
image_msg)
Callback for images published without Camera (p. 35) information.
Parameters
image_msg The new received image received from the camera associate to
this Object
dummy Camera (p. 35) info Pointer to allow to use exactly the same callback loop
convert image msg to OPEN CV image
Do for any saved Modus (p. 48):
1. Save image if wanted
2, Send image if wanted and if needed resize before and/or rotate
optimize our modus to the bandwidth that is available
C.1.3.8
void Camera::change_send_rate (Modus & modus, int
send_every_XXX_image) [private]
Change the amount of sent images of the modus.
Parameters
modus The modus how to change
send_every_XXX_image the denominator of the the fraction of image to
send. If send_every_XXX_image is equal 4, 0.25% of the images are
sent.
Update the modus
Update the server with change to inform over nodes.
C.1.3.9
void Camera::configure_modus (Video::CameraConfig &
config, uint32_t level)
Update the modus and changes it if changes are wanted by the dynamic reconfigure
gui.
Parameters
config The parameters sent by the reconfigure server
Appendix C. Code
42
level
Initialise the configurations if not.
Take a single shoot of all modus if wanted
Add a new modus if user apply changes to the Image Handler node
Update reconfigure server with actual values
C.1.3.10
void Camera::optimize_to_bandwidth ()
Choose the optimal modus adapted to the actual Bandwidth.
BAND_ADAPT_DISABLED: The user doesn’t want to adapt dynamically to the
bandwidth limits.
BAND_ADAPT_INITIALISATION: The state machine is started with BAND_ADAPT_INITIALISATION and the needed queues for statistics initialised.
BAND_ADAPT_WAIT_MEASUREMENTS: The node wait for information
about the connection before beginning bandwidth adaptation.
BAND_ADAPT_GUESS_LIMIT: Calculate how many of the sent bytes could be
transmitted. The limit is guessed and the modus adapted to this limit.
BAND_ADAPT_GUESS_LIMIT2: Does the same as the state before. The Limit
is guessed twice to faster come to the near the right estimation.
BAND_ADAPT_FIND_LIMIT: Tries to find the perfect condition with changing
only a little around the guessed limit. The problematic is if 1/2 images are sent. So
incrementing by one the denominator means double the send bytes. To limit this
effect stream every image is only allowed if the send queue is completely empty.
BAND_ADAPT_KEEP_LIMIT: Once the limit of bandwidth is known approximately the bandwidth is guessed constant for the number of steps defined in keep_state. After that the cycle begins again with BAND_ADAPT_GUESS_LIMIT.
C.1.3.11
int Camera::pos_pub_of_topic (std::string topic) [private]
Gives position of the topic in the publisher vector and return -1 if the publisher
isn’t declared.
Parameters
topic The name of the searched topic
Check all existing publisher if one publishes the wanted topic
C.1.3.12
bool Camera::resize_image (cv::Mat & image, Modus &
modus, double & fx, double & fy) [private]
resize image to be smaller or equal the max high and width
43
Parameters
image The image that should be resized
modus The modus containing the limits to apply to the image
fx The factor of resizing in x direction
fy The factor of resizing in y direction
C.1.3.13
bool Camera::rotate_image (cv::Mat & src, cv::Mat & dest,
Modus & modus) [private]
Rotate the image. The function is very efficient for rotating 90,180 and 270 degrees
and primarily implemented to switch images that there upside down do the camera
fixation on the UAV.
Parameters
src The image to rotate
dest The rotated image
modus The modus containing the settings for rotation
C.1.3.14
bool Camera::save_image (cv_bridge::CvImageConstPtr
cvImage, sensor_msgs::CameraInfoConstPtr info, Modus
camera_modus) [private]
Save an image with the settings specified in the modus.
Parameters
cvImage Image that has to be saved
info Information about the camera settings related to the image
camera_modus Modus (p. 48) containing the parameters how to save images.
Assembly filename
Set parameters depending on format
Write image to disk
C.1.3.15
bool Camera::send_image (cv_bridge::CvImageConstPtr
cvImage, Modus modi) [private]
Send an image with the wished parameters.
Parameters
cvImage image to send
Appendix C. Code
44
modi The modus in which the images have to be sent.
Check if publisher is already existing
Define publisher if it is not already existing
Publish the image
C.1.3.16
void Camera::show_all_pub_topic () [private]
Show all the topic the publisher vector contains.
C.1.3.17
uint64_t Camera::time_stamp () [private]
Time stamp used for saving images to avoid overwriting files with the same name.
C.1.3.18
void Camera::update_camera_info (const
sensor_msgs::CameraInfoConstPtr & info)
C.1.3.19
void Camera::update_modus_ros (const
skye_msgs::ImagehandlerModusConstPtr & msg)
Update the modus and changes it if changes are wanted in the form of a ROS
message.
Parameters
msg The Modus (p. 48) message containing the new parameters.
Check if the Command is for this camera
Save the new modi
Clear old Modus (p. 48) if wanted
C.1.4
Member Data Documentation
C.1.4.1
BandwidthAdaptationsState_t Camera::bandwidth_state
[private]
The actual state of the bandwidth adaptation.
adaptation for the possible states.
See the method bandwidth_-
45
C.1.4.2
CamID_t Camera::camera_id [private]
The private node handle of the camera.
C.1.4.3
long int Camera::counter [private]
Counts handled images.
C.1.4.4
int Camera::counter_find_limit [private]
Counter of the how many steps are done.
C.1.4.5
int Camera::counter_same_state [private]
Counter of how many cycles with assumed same bandwidth were done.
C.1.4.6
int Camera::find_limit [private]
Constant that defines how many step is done to approach the nearest the real
bandwidth.
C.1.4.7
boost::format Camera::g_format [private]
Needed for the assembling of the image name to save with the time stamp and the
path.
C.1.4.8
std::deque<unsigned long int> Camera::guessed_available_bandwidth [private]
Sample container memorising the available bandwidth for the last steps.
C.1.4.9
image_transport::ImageTransport Camera::it
Appendix C. Code
C.1.4.10
46
int Camera::keep_state [private]
Constant that defines how many images the bandwidth has to be assumed equal
after an estimation cycles.
C.1.4.11
std::deque<ros::Time> Camera::last_stamps [private]
Sample container to memorise the moment when the bytes were sent.
C.1.4.12
uint64_t Camera::last_time_stamp [private]
Contains time info of the last saved image and is needed to avoid saving to image
with the same timestamp.
C.1.4.13
double Camera::max_step [private]
Indicates the maximum factor that the new send percentage could be greater than
the old. If set to two, the bandwidth adaptation is not allowed to send all images
if before only 25% were sent, because the factor would be four and by this higher
than the maximum step.
C.1.4.14
double Camera::mean_available_bandwidth [private]
The average of bytes the could be sent per seconds. If no limit is present the number
simply indicates the used bandwidth.
C.1.4.15
double Camera::mean_queue_len [private]
The average of bytes retained in the send queue of the udp_mavlink_ros_node
calculated with the information of the connection state messages.
C.1.4.16
std::deque<unsigned long int> Camera::message_size
[private]
Sample container for memorise how many bytes were sent.
47
C.1.4.17
std::vector<Modus> Camera::modi [private]
Container with all active modes. This makes possible to handle the same image in
different ways. I.e it could be saved in two different formats or republished with
with different topics to different nodes.
C.1.4.18
double Camera::old_mean_queue_len [private]
The average of bytes retained in the send queue of the step before. This information
is need to now if the send queue is growing up or decreasing.
C.1.4.19
ros::NodeHandle Camera::prn [private]
The ROS subscriber that will subscribe Modus (p. 48) messages. Is here to allow
over node to reconfigure the node with ROS messages.
C.1.4.20
std::vector<image_transport::Publisher> Camera::pub
[private]
The vector of publisher contains a single publisher for any active Modus (p. 48) and
makes possible to publish the handled images under different topics. For example
the normal /cam_namespace/single_shotand cam_namespace/image_stream.
C.1.4.21
ros::Publisher Camera::pub_bandwidth [private]
The ROS publisher that will publishes the used bandwidth and the percentage of
images that are sent.
C.1.4.22
bool Camera::pub_initialized [private]
Boolean value to check if the publisher for the Bandwidth messages is already initialised. If the camera instance never received a feedback from the udp_mavlink_ros_node, the image are probably not streamed to an over machine, and bandwidth
estimation and adaptation will not be started.
C.1.4.23
dynamic_reconfigure::Server<Video::CameraConfig>
Camera::reconfigure_server_ [private]
The reconfigure server makes possible do dynamicly reconfigure the image handler
node containing an instance of the camera class.
Appendix C. Code
C.1.4.24
48
std::deque<unsigned long int> Camera::send_queue_lengths
[private]
Sample container for the data need to calculate mean_queue_len.
C.1.4.25
ros::Subscriber Camera::sub_Command [private]
The ROS subscriber that will subscribe Modus (p. 48) messages. Is here to allow
over nodes to reconfigure the node with ROS messages.
C.1.4.26
ros::Subscriber Camera::sub_connect [private]
The ROS subscriber that will subscribe ConnectionState messages. Is here to receive
feedback from the udp_mavlink_ros about how many image could be sent.
C.1.4.27
Verbosity_t Camera::verbosity [private]
The verbosity level of the image handling functions.
C.1.4.28
int Camera::window_size [private]
This number indicates how many old states are remembered before being discarded.
C.1.4.29
bool Camera::yyyymmdd_enabled [private]
Switches for the time stamp after the image name. If true a long time stamp of the
form YYYYMMDDHHMMSS∗∗ is used. If false only a time stamp of the format
HHMMSS∗∗ is appended to the filename.
The documentation for this class was generated from the following files:
• /home/matthias/ros_workspace/skye/Video/include/Video/camera.h
• /home/matthias/ros_workspace/skye/Video/src/camera.cpp
C.2
Modus Struct Reference
#include <modus.h>
49
C.2. Modus Struct Reference
• void info (std::string name_of_namespace)
The Function prints out the information contained in the Modus ( p. 48).
• bool known_format (std::string fromat)
Returns true if the format is known (JPEG, PNG of TIFF to this day) false else.
• Modus (const Video::CameraConfig &config)
Constructor to fill a new Modus ( p. 48) with configuration of a CamearaConfig
message from the dynamic reconfigure server.
• Modus (const skye_msgs::ImagehandlerModusConstPtr &msg)
Constructor for incoming ROS Messages.
• Modus (bool takeshot=false, bool save_image=false, int save_of_one=25, std::string format="png", int quality=0, std::string frame_name="frame", std::string path="Pictures", bool send_image=true, int
send_of_one=1, int max_high=5000, int max_width=5000, bool
keep_proportion=true, int rotation=0, std::string topic_pulish="/imh_image")
Standart Constructor.
• void show (std::string name_of_namespace)
Gives out all the values of the modus. This function was needed only for developpement, info() ( p. 51) gives you better readable answers.
Public Attributes
• std::string format
Saving format for the images. By now PNG, JPEG and TIFF are supported.
• std::string frame_name
Name of the saved frames. A time stamp will be added to make so that any image
has an unique name.
• bool keep_proportions
If true the side ratio will be kept the same through resizing.
• int max_high
For the maximal high of the image that will not be resized.
• int max_width
For the maximal width of the image that will not be resized.
• std::string path
Path where the Pictures has to be saved.
• int quality
Appendix C. Code
50
Indicates the quality the saved image should have. If the format is JPEG the
range is form 0 to 100. For PNG the compression level goes from 0 (No compression) to 9 (Maximal Compression).
• int rotations
Degrees the images will be rotated. The rotation function is optimised for 90,180
and 270 degrees.
• bool save_image
If true the received images the fraction of images defined in save_of_one are
saved.
• unsigned int save_of_one
If save_of_one = 4, every 4th image will be saved.
• bool send_image
If true the received images will be resized and rotated like configured and the
published under /namespace_cam/topic_to_publish.
• unsigned int send_of_one
If send_of_one = 4, every 4th image will be sent.
• bool takeshot
Set to true for saving one image and automatically reset to zero in the save
function of Camera ( p. 35).
• std::string topic_publish
The image is published under /namespace_cam/topic_to_publish if send_image
is true.
C.2.1
Definition at line 13 of file modus.h.
C.2.2
C.2.2.1
Modus::Modus (bool takeshot = false, bool save_image
= false, int save_of_one = 25, std::string format =
"png", int quality = 0, std::string frame_name = "frame",
std::string path = "Pictures", bool send_image = true, int
send_of_one = 1, int max_high = 5000, int max_width
= 5000, bool keep_proportion = true, int rotation = 0,
std::string topic_pulish = "/imh_image")
Standart Constructor.
Definition at line 30 of file modus.cpp.
51
C.2. Modus Struct Reference
C.2.2.2
Modus::Modus (const skye_msgs::ImagehandlerModusConstPtr &
msg)
Constructor for incoming ROS Messages.
C.2.2.3
Modus::Modus (const Video::CameraConfig & config)
Constructor to fill a new Modus (p. 48) with configuration of a CamearaConfig
message from the dynamic reconfigure server.
Parameters
config The message from the server with the updated values.
C.2.3
C.2.3.1
void Modus::info (std::string name_of_namespace)
The Function prints out the information contained in the Modus (p. 48).
Parameters
name_of_namespace By giving the namespace of the camera the printed
configurations could be better separated.
C.2.3.2
bool Modus::known_format (std::string format)
Returns true if the format is known (JPEG, PNG of TIFF to this day) false else.
Parameters
format The format you want to now if it is available.
C.2.3.3
void Modus::show (std::string name_of_namespace)
Gives out all the values of the modus. This function was needed only for developpement, info() (p. 51) gives you better readable answers.
Parameters
name_of_namespace By giving the namespace of the camera the printed
configurations could be better separated.
Appendix C. Code
C.2.4
C.2.4.1
std::string Modus::format
52
Saving format for the images. By now PNG, JPEG and TIFF are supported.
C.2.4.2
std::string Modus::frame_name
Name of the saved frames. A time stamp will be added to make so that any image
has an unique name.
C.2.4.3
bool Modus::keep_proportions
If true the side ratio will be kept the same through resizing.
C.2.4.4
int Modus::max_high
For the maximal high of the image that will not be resized.
C.2.4.5
int Modus::max_width
For the maximal width of the image that will not be resized.
C.2.4.6
std::string Modus::path
Path where the Pictures has to be saved.
C.2.4.7
int Modus::quality
Indicates the quality the saved image should have. If the format is JPEG the range
is form 0 to 100. For PNG the compression level goes from 0 (No compression) to
9 (Maximal Compression).
C.2.4.8
int Modus::rotations
Degrees the images will be rotated. The rotation function is optimised for 90,180
and 270 degrees.
53
C.3. Mavlink_UDP_link Class Reference
C.2.4.9
bool Modus::save_image
If true the received images the fraction of images defined in save_of_one are saved.
C.2.4.10
unsigned int Modus::save_of_one
If save_of_one = 4, every 4th image will be saved.
C.2.4.11
bool Modus::send_image
If true the received images will be resized and rotated like configured and the published under /namespace_cam/topic_to_publish.
C.2.4.12
unsigned int Modus::send_of_one
If send_of_one = 4, every 4th image will be sent.
C.2.4.13
bool Modus::takeshot
Set to true for saving one image and automatically reset to zero in the save function
of Camera (p. 35).
C.2.4.14
std::string Modus::topic_publish
The image is published under /namespace_cam/topic_to_publish if send_image
is true.
The documentation for this struct was generated from the following files:
• /home/matthias/ros_workspace/skye/Video/include/Video/modus.h
• /home/matthias/ros_workspace/skye/Video/src/modus.cpp
C.3
Mavlink_UDP_link Class Reference
#include <mavlink_UDP_link.h>
• std::string get_connection_state ()
• int get_recv_queue ()
Appendix C. Code
54
• int get_send_queue ()
• int get_size_receive_buffer ()
• int get_size_send_buffer ()
• bool getInitialised ()
GetInitialisation Get if the Mavlink_UDP_link ( p. 53) was already initialised.
• void init (int dest_port=14550, std::string dest_host="127.0.0.1")
Init the UDP Communication.
• Mavlink_UDP_link (int dest_port=14550, std::string
host="127.0.0.1",
int
local_port=12345,
std::string
host="127.0.0.1")
dest_local_-
Default constructor the UDP Communication.
• void receive_loop ()
• void receive_mavlink_msg (mavlink_channel_t
message_t &msg, bool &stop_condition)
chan,
mavlink_-
Receive bytes and decode one MAVLink messages then returns.
• int receive_udp_bytes (mavlink_channel_t chan)
Receive one char (uint8_t).
• void send_udp_bytes (mavlink_channel_t chan, uint8_t ∗ch, uint16_t
length)
Send one char (uint8_t) over a comm channel.
• void show_state ()
Show connection stat.
• void update_state ()
Measure the length of the receive and send queue queue as well as the connection
state and write it in the corresponding fields.
• ∼Mavlink_UDP_link ()
Destructor Destructor.
• int connect_retry (int sockfd, const struct sockaddr ∗addr, socklen_t
alen)
Try to connect until connection or timeout.
• int initserver (int type, const struct sockaddr ∗addr, socklen_t alen, int qlen)
55
Private Attributes
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
sockaddr_in addr_dest
sockaddr_in addr_local
sockaddr_in addr_sender
struct addrinfo ∗ ailist
struct addrinfo ∗ aip
std::fstream bandwidth_file
std::string command_band_measure
std::string connection_state
std::string dest_host
int dest_port
socklen_t fromlen
struct addrinfo hint
bool initilized
std::string local_host
int local_port
int lost_bytes
char receive_buffer [BUFFER_SIZE]
int receive_socket
int received_bytes
int recv_queue
int send_queue
int sent_bytes
int size_receive_buffer
int size_send_buffer
int udp_socket
C.3.1
Definition at line 43 of file mavlink_UDP_link.h.
C.3.2
C.3.2.1
Mavlink_UDP_link::Mavlink_UDP_link (int dest_port =
14550, std::string dest_host = "127.0.0.1", int local_port =
12345, std::string local_host = "127.0.0.1")
Default constructor the UDP Communication.
Definition at line 303 of file mavlink_UDP_link.cpp.
C.3.2.2
Mavlink_UDP_link::∼Mavlink_UDP_link ()
Destructor Destructor.
Appendix C. Code
C.3.3
C.3.3.1
int Mavlink_UDP_link::connect_retry (int sockfd, const
struct sockaddr ∗ addr, socklen_t alen) [private]
Try to connect until connection or timeout.
Parameters
sockfd Socket
addr Socket address
alen Length of the socket address
C.3.3.2
std::string Mavlink_UDP_link::get_connection_state ()
Returns
The connection state.
C.3.3.3
int Mavlink_UDP_link::get_recv_queue ()
Returns
Size of the receive queue
C.3.3.4
int Mavlink_UDP_link::get_send_queue ()
Returns
Size of the send queue
C.3.3.5
int Mavlink_UDP_link::get_size_receive_buffer ()
Returns
Size of the receive buffer
C.3.3.6
int Mavlink_UDP_link::get_size_send_buffer ()
Returns
Size of the send buffer.
56
57
C.3.3.7
bool Mavlink_UDP_link::getInitialised ()
GetInitialisation Get if the Mavlink_UDP_link (p. 53) was already initialised.
C.3.3.8
void Mavlink_UDP_link::init (int dest_port = 14550,
std::string dest_host = "127.0.0.1")
Init the UDP Communication.
Parameters
dest_host Hostname or IP
dest_port Port number
C.3.3.9
int Mavlink_UDP_link::initserver (int type, const struct
sockaddr ∗ addr, socklen_t alen, int qlen) [private]
C.3.3.10
void Mavlink_UDP_link::receive_loop ()
C.3.3.11
void Mavlink_UDP_link::receive_mavlink_msg
(mavlink_channel_t chan, mavlink_message_t & msg,
bool & stay_in_loop)
Receive bytes and decode one MAVLink messages then returns.
Parameters
chan Mavlink channel
msg Message to fill in the received message
stay_in_loop Boolean to break the loop
C.3.3.12
int Mavlink_UDP_link::receive_udp_bytes
(mavlink_channel_t chan)
Receive one char (uint8_t).
Parameters
chan MAVLInk channel to use, usually MAVLINK_COMM_3 = udp_socket
C.3.3.13
void Mavlink_UDP_link::send_udp_bytes
(mavlink_channel_t chan, uint8_t ∗ ch, uint16_t length)
Send one char (uint8_t) over a comm channel.
Appendix C. Code
58
Parameters
chan MAVLink channel to use, usually MAVLINK_COMM_3 = udp_socket
ch Pointer to send buffer array
length The length of the send buffer array
C.3.3.14
void Mavlink_UDP_link::show_state ()
Show connection stat.
C.3.3.15
void Mavlink_UDP_link::update_state ()
Measure the length of the receive and send queue queue as well as the connection
state and write it in the corresponding fields.
C.3.4
C.3.4.1
sockaddr_in Mavlink_UDP_link::addr_dest [private]
C.3.4.2
sockaddr_in Mavlink_UDP_link::addr_local [private]
C.3.4.3
sockaddr_in Mavlink_UDP_link::addr_sender [private]
C.3.4.4
struct addrinfo∗ Mavlink_UDP_link::ailist [private]
C.3.4.5
struct addrinfo ∗ Mavlink_UDP_link::aip [private]
C.3.4.6
std::fstream Mavlink_UDP_link::bandwidth_file [private]
59
C.3.4.7
std::string Mavlink_UDP_link::command_band_measure
[private]
C.3.4.8
std::string Mavlink_UDP_link::connection_state [private]
C.3.4.9
std::string Mavlink_UDP_link::dest_host [private]
C.3.4.10
int Mavlink_UDP_link::dest_port [private]
C.3.4.11
socklen_t Mavlink_UDP_link::fromlen [private]
C.3.4.12
struct addrinfo Mavlink_UDP_link::hint [private]
C.3.4.13
bool Mavlink_UDP_link::initilized [private]
C.3.4.14
std::string Mavlink_UDP_link::local_host [private]
C.3.4.15
int Mavlink_UDP_link::local_port [private]
C.3.4.16
int Mavlink_UDP_link::lost_bytes [private]
C.3.4.17
char Mavlink_UDP_link::receive_buffer[BUFFER_SIZE]
[private]
Appendix C. Code
C.3.4.18
60
int Mavlink_UDP_link::receive_socket [private]
C.3.4.19
int Mavlink_UDP_link::received_bytes [private]
C.3.4.20
int Mavlink_UDP_link::recv_queue [private]
C.3.4.21
int Mavlink_UDP_link::send_queue [private]
C.3.4.22
int Mavlink_UDP_link::sent_bytes [private]
C.3.4.23
int Mavlink_UDP_link::size_receive_buffer [private]
C.3.4.24
int Mavlink_UDP_link::size_send_buffer [private]
C.3.4.25
int Mavlink_UDP_link::udp_socket [private]
• /home/matthias/mav/mavlink-ros-pkg/udp_mavlink_ros/include/udp_mavlink_ros/mavlink_UDP_link.h
• /home/matthias/mav/mavlink-ros-pkg/udp_mavlink_ros/src/mavlink_UDP_link.cpp
C.4
Converter Class Reference
#include <converter_mavlink_ros.h>
• void callback (const sensor_msgs::CompressedImageConstPtr &image)
Callback done for any compressed ROS image received.
61
C.4. Converter Class Reference
• void callback (const sensor_msgs::ImageConstPtr &image)
Callback done for any ROS image received.
• Converter (ros::NodeHandle &nh, Mavlink_UDP_link &link, std::string
subscribe_topic_name, std::string topic, int quality=80)
Initialise all the members of the converter instance concerning ROS and the
Mavlink Parameter server. The connection will be initialised in the init method.
• void handle_mavlink_message (mavlink_message_t &received_msg)
Decode the received mavlink message.
• void mavlink_missionlib_send_gcs_string (const char ∗string)
Packs a string in a mavlink message and send it.
• void mavlink_missionlib_send_message (mavlink_message_t ∗msg)
Send any mavlink message.
• float mavlink_pm_get_param (std::string key)
Get a parameter specified by its name.
• float mavlink_pm_get_param (int pos)
Get a paramter specified by position.
• void mavlink_pm_message_handler
∗msg)
(const
mavlink_message_t
Handle the Mavlink messages relative to the parameter server.
• void mavlink_pm_queued_send (void)
Send low-priority messages at a maximum rate of xx Hertz.
• void mavlink_pm_reset_params ()
reset all parameters to default
• void mavlink_pm_set_param (int pos, float value)
Set a parameter specified by its position.
• void mavlink_pm_set_param (std::string key, float value)
Set a parameter specified by its name.
• ∼Converter ()
Public Attributes
• image_transport::ImageTransport it
Appendix C. Code
62
• bool from_mavlink_to_ros (mavlink_data_transmission_handshake_t handshake, std::vector< mavlink_encapsulated_data_t > encapsulated_data)
Receive Mavlink images and convert them to ROS images.
• bool
from_ros_to_mavlink
msgs::CompressedImageConstPtr &image)
(const
sensor_-
Send compressed ROS images over Mavlink If the images are always compressed
for the ROS transmission this function would avoid to uncompress the image
with image transport just to compress it again to sent it over Mavlink.
• bool from_ros_to_mavlink (cv::Mat &cvImage)
Convert an Image in mavlink images and send it over UDP to QGroundControl.
• void measure_udp_connection ()
Measure the send queue length of the UDP port The measured send queue length is
used by the Camera Class of the Video package to adapt dynamically the amount
of images that were transmitted.
Static Private Member Functions
• static void ∗ heartbeat_loop (void ∗ptr)
Loop running in own thread and emit an heart-beat-message needed for Mavlink
communication Further the node also transmits a parameter from the server in
every loop.
• static void ∗ receive_mavlink (void ∗ptr)
running loop to receive MAVLINK Message
Private Attributes
• pthread_attr_t bandwidth_measure_attr
Thread attributes for the thread measuring the connection state.
• pthread_t bandwidth_measure_thread
Thread identifier for the thread measuring the connection state.
• mavlink_channel_t chan
Mavlink channel used for communication. For the project SKYE MAVLINK_COMM_3 was the UPD channel and MAVLINK_COMM_0 the serial.
• int counter
Counter of the threated images by the Converter ( p. 60).
• int error_counter
Error counter of the Converter ( p. 60).
63
• pthread_attr_t heartbeat_attr
Thread attributes for the thread send heart-beat an parameters.
• pthread_t heartbeat_thread
Thread identifier for the thread send heart-beat an parameters.
• int image_format
The image format following the Mavlink enum in the common messages. (see
Mavlink header library).
• int image_size
Length of the image after compression in Bytes.
• image_transport::Subscriber image_sub
• pthread_attr_t mavlink_receive_attr
Thread attributes for the thread receiving Mavlink messages.
• pthread_t mavlink_receive_thread
Thread identifier for the thread receiving Mavlink messages.
• std::vector< int > params_jpg
Parameters for JPEG compression.
• std::vector< int > params_png
Parameters for PNG compression.
• mavlink_pm_storage pm
Storages the parameters of the Converter ( p. 60) and the rosparam server.
• ros::NodeHandle prn
The ROS subscriber for communicate outside the node where the instance of this
class is initialised.
• image_transport::Publisher pub
The publisher for the images received over Mavlink.
• ros::Publisher pub_connection_state
The ROS publisher for the ConnectionState-messages.
• int quality
Indicates the quality the saved image should have. If the format is JPEG the
range is form 0 to 100. For PNG the compression level goes from 0 (No compression) to 9 (Maximal Compression).
• std::string subscribe_topic_name
Name of the image topic the instance of the Class is subscribed to.
• std::string topic_name
Appendix C. Code
64
Name of the ROS image topic this Class publishes with the received Mavlink
images.
• Mavlink_UDP_link & udp_link
UDP Link with all send and write functions.
C.4.1
Definition at line 31 of file converter_mavlink_ros.h.
C.4.2
C.4.2.1
Converter::Converter (ros::NodeHandle & nh, Mavlink_UDP_link & link, std::string subscribe_topic_name,
std::string topic, int quality = 80)
Initialise all the members of the converter instance concerning ROS and the Mavlink
Parameter server. The connection will be initialised in the init method.
Parameters
nh The nodehandle in which the Converter (p. 60) is used
link The UDP Link instance over whose the communication is done.
subscribe_topic_name The name of the image topic to subscribe
topic The name of the published images
quality The quality of the images to transmit.
Check if udp_link is initialised and do it if not
Set the Mavlink identidy of the System and his state
Send one Heartbeat to enable the ground station to communicate with this node
Set image settings
Define image compression parameters
Start thread that receive mavlink messages
Start heartbeat thread
Init parameter server
Todo
Get parameters from rosparam server
Definition at line 952 of file converter_mavlink_ros.cpp.
C.4.2.2
Converter::∼Converter ()
65
C.4.3
C.4.3.1
void Converter::callback (const sensor_msgs::CompressedImageConstPtr & image)
Callback done for any compressed ROS image received.
Parameters
image The received compressed ROS image
Warning
Never used on real system and therefore no guarantee that the method is working.
C.4.3.2
void Converter::callback (const sensor_msgs::ImageConstPtr
& image)
Callback done for any ROS image received.
Parameters
image The received uncompressed ROS image
Share the image with cv_bridge
Convert the image from ROS to Mavlink
Measure UDP connection and publish the result in a ROS message
C.4.3.3
bool Converter::from_mavlink_to_ros (mavlink_data_transmission_handshake_t handshake, std::vector<
mavlink_encapsulated_data_t > encapsulated_data)
[private]
Receive Mavlink images and convert them to ROS images.
Todo
Implement the decompression of the received images
Warning
Never used on real system and therefore no guarantee that the method is working.
Check in how many messages the image has be sent
assembly image data
Decompress
Fill in ROS image and publish it
Appendix C. Code
C.4.3.4
66
bool Converter::from_ros_to_mavlink (const
sensor_msgs::CompressedImageConstPtr & image) [private]
Send compressed ROS images over Mavlink If the images are always compressed
for the ROS transmission this function would avoid to uncompress the image with
image transport just to compress it again to sent it over Mavlink.
Todo
Implement read out of the image high and width out of the image header.
C.4.3.5
bool Converter::from_ros_to_mavlink (cv::Mat & cvImage)
[private]
Convert an Image in mavlink images and send it over UDP to QGroundControl.
Parameters
cvImage The image that should be send over MAVLINK
The image conversion from ROS to Mavlink undergoes the following steps:
1. Do JEPG compression
2. Check in how many messages the image has to be sent
3. Send a handshake to QGroundControl to tell image properties.
4. Send the image packed in 253 bytes messages.
Alternatively the same thing can be done with PNG compression instead of JPEG.
But while this format is less compressible, only smaller images could be sent. The
reason is that the image could maximum be split in 255 messages.
C.4.3.6
void Converter::handle_mavlink_message
(mavlink_message_t & received_msg)
Decode the received mavlink message.
Parameters
received_msg mavlink_message_t The encoded mavlink message that
should be decoded.
Note
this function could easily expand to any over mavlink message.
67
C.4.3.7
void ∗ Converter::heartbeat_loop (void ∗ ptr) [static,
private]
Loop running in own thread and emit an heart-beat-message needed for Mavlink
communication Further the node also transmits a parameter from the server in
every loop.
Parameters
ptr Pointer casted to Converter (p. 60) Class type to allow access of non
static member functions of this Class from inside of a thread.
C.4.3.8
void Converter::mavlink_missionlib_send_gcs_string (const
char ∗ string)
Packs a string in a mavlink message and send it.
Note
This Function is copied from Mavlink/Missionlib. For source code see
www.github.com/mavlink/mavlink.
For more information, please visit:
http://qgroundcontrol.org/mavlink/
(c) 2009-2012 Lorenz Meier <[email protected]>
C.4.3.9
void Converter::mavlink_missionlib_send_message
(mavlink_message_t ∗ msg)
Send any mavlink message.
Note
C.4.3.10
float Converter::mavlink_pm_get_param (std::string key)
Get a parameter specified by its name.
Appendix C. Code
68
Parameters
key String containing the name of the requested parameter.
Returns
he value of the parameter or -1 if an invalid position is specified
C.4.3.11
float Converter::mavlink_pm_get_param (int pos)
Get a paramter specified by position.
Parameters
pos Postion of the requested parameter in the array.
Returns
The value of the parameter or -1 if an invalid position is specified
C.4.3.12
void Converter::mavlink_pm_message_handler (const
mavlink_message_t ∗ msg)
Handle the Mavlink messages relative to the parameter server.
Note
C.4.3.13
void Converter::mavlink_pm_queued_send (void)
Send low-priority messages at a maximum rate of xx Hertz.
This function sends messages at a lower rate to not exceed the wireless bandwidth.
It sends one message each time it is called until the buffer is empty. Call this
function with xx Hertz to increase/decrease the bandwidth.
Note
69
C.4.3.14
void Converter::mavlink_pm_reset_params ()
reset all parameters to default
Warning
DO NOT USE THIS IN FLIGHT!
Note
C.4.3.15
void Converter::mavlink_pm_set_param (int pos, float
value)
Set a parameter specified by its position.
Parameters
pos Position of the requested parameter in the array.
value New value of the parameter
C.4.3.16
void Converter::mavlink_pm_set_param (std::string key,
float value)
Set a parameter specified by its name.
Parameters
key String containing the name of the requested parameter.
value New value of the parameter
Appendix C. Code
C.4.3.17
70
void Converter::measure_udp_connection () [private]
Measure the send queue length of the UDP port The measured send queue length is
used by the Camera Class of the Video package to adapt dynamically the amount
of images that were transmitted.
C.4.3.18
void ∗ Converter::receive_mavlink (void ∗ ptr) [static,
private]
running loop to receive MAVLINK Message
Parameters
ptr Pointer casted to Converter (p. 60) Class type to allow access of non
static member functions of this Class from inside of a thread.
C.4.4
C.4.4.1
pthread_attr_t Converter::bandwidth_measure_attr
[private]
Thread attributes for the thread measuring the connection state.
C.4.4.2
pthread_t Converter::bandwidth_measure_thread [private]
Thread identifier for the thread measuring the connection state.
C.4.4.3
mavlink_channel_t Converter::chan [private]
Mavlink channel used for communication. For the project SKYE MAVLINK_COMM_3 was the UPD channel and MAVLINK_COMM_0 the serial.
C.4.4.4
int Converter::counter [private]
Counter of the threated images by the Converter (p. 60).
C.4.4.5
int Converter::error_counter [private]
Error counter of the Converter (p. 60).
71
C.4.4.6
pthread_attr_t Converter::heartbeat_attr [private]
Thread attributes for the thread send heart-beat an parameters.
C.4.4.7
pthread_t Converter::heartbeat_thread [private]
Thread identifier for the thread send heart-beat an parameters.
C.4.4.8
int Converter::image_format [private]
The image format following the Mavlink enum in the common messages. (see
Mavlink header library).
C.4.4.9
int Converter::image_size [private]
Length of the image after compression in Bytes.
C.4.4.10
image_transport::Subscriber Converter::image_sub
[private]
C.4.4.11
image_transport::ImageTransport Converter::it
C.4.4.12
pthread_attr_t Converter::mavlink_receive_attr [private]
Thread attributes for the thread receiving Mavlink messages.
C.4.4.13
pthread_t Converter::mavlink_receive_thread [private]
Thread identifier for the thread receiving Mavlink messages.
C.4.4.14
std::vector<int> Converter::params_jpg [private]
Parameters for JPEG compression.
Appendix C. Code
C.4.4.15
72
std::vector<int> Converter::params_png [private]
Parameters for PNG compression.
C.4.4.16
mavlink_pm_storage Converter::pm [private]
Storages the parameters of the Converter (p. 60) and the rosparam server.
C.4.4.17
ros::NodeHandle Converter::prn [private]
The ROS subscriber for communicate outside the node where the instance of this
class is initialised.
C.4.4.18
image_transport::Publisher Converter::pub [private]
The publisher for the images received over Mavlink.
C.4.4.19
ros::Publisher Converter::pub_connection_state [private]
The ROS publisher for the ConnectionState-messages.
C.4.4.20
int Converter::quality [private]
Indicates the quality the saved image should have. If the format is JPEG the range
is form 0 to 100. For PNG the compression level goes from 0 (No compression) to
9 (Maximal Compression).
C.4.4.21
std::string Converter::subscribe_topic_name [private]
Name of the image topic the instance of the Class is subscribed to.
C.4.4.22
std::string Converter::topic_name [private]
Name of the ROS image topic this Class publishes with the received Mavlink images.
73
C.4.4.23
Mavlink_UDP_link& Converter::udp_link [private]
UDP Link with all send and write functions.
• /home/matthias/mav/mavlink-ros-pkg/udp_mavlink_ros/include/udp_mavlink_ros/converter_mavlink_ros.h
• /home/matthias/mav/mavlink-ros-pkg/udp_mavlink_ros/src/converter_mavlink_ros.cpp
Appendix C. Code
74
75
Appendix D
Measurements
Ground
station
Airship
RF
Modul
0.00
10.00
20.00
30.00
40.00
50.00
3'600
3'600
3'600
3'600
3'600
3'600
100
1'000
500
500
250
100
0.0
3'600
250
900'000
1
2
360'000
900'000
3 1'800'000
1 360'000
3 3'600'000
2 1'800'000
2
3 3'600'000
2 1'800'000
3 1'800'000
Sent
[Bytes]
141432
169138
163738
147014
139166
151416
165874
113536
107764
167737
2'000.0
3'000.0
4'000.0
Sent Bytes per second
1'000.0
Xbee Stress Test
A to B
B to A
1'250.0
1'250.0
1'666.7
1'000.0
3'333.3
2'500.0
1'250.0
3'333.3
2'500.0
1'666.7
Page 1
392.9
234.9
151.6
408.4
128.9
210.3
230.4
105.1
149.7
155.3
6
12
18
6
18
12
12
18
12
18
Received Send [Bytes Received Duratio
[Bytes]
/ s]
[Bytes / s] n [ min]
Xbee 868 MHz Stress Test
3'600
3'600
3'600
Repetit Delay
ions [x0.1 s]
1'000
500
500
Size of
Messge
[Bytes]
Xbee 868 MHz Stress Test
Received Bytes [%]
Observation
Note
The test was made indoor with a line-of-sight of
15 Meters
3.15
5.99
9.32 Stopped to receive after 1600 messages
Stopped to receive after 2000 messages and
18.43
receiving again later
40.84 Stopped after maximum 1.5 houur
3.87
8.41 Stopped to receive after ~2000 messages
9.10
18.79
31.43 Stopped after maximum 1.5 houur
Transmitted
[%]
76
APPENDIX D. MEASUREMENTS
77
D.2
D.2. Xbee 2.4GHz Rangetest
Xbee 2.4GHz Rangetest
tables with measurements . . .
4.2
4.2
4.2
4.2
4.2
8.0
8.0
8.0
8.0
8.0
8.0
8.0
8.0
15.0
15.0
15.0
15.0
15.0
15.0
15.0
15.0
15.0
15.0
41.5
41.5
41.5
41.5
41.5
41.5
41.5
41.5
45 - 50
23
20
15
10
5
23
20
15
10
8
6
5
4
23
20
15
10
5
5
4
3
2
1
23
15
10
2
1
1
0
0
Image
Distance
Publication
[m]
[hz]
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
480
Image
Height
[Pixel]
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
752
Image
Width
[Pixel]
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
24'365
21'658
16'243
10'829
5'414
24'365
21'658
16'243
10'829
8'663
6'497
5'414
4'332
24'365
21'658
16'243
10'829
5'414
5'414
4'332
3'249
2'166
1'083
24'365
16'243
10'829
2'166
1'083
541
406
271
Theoretic
Color
Bandwith
Channels
[KB /s]
7.54
7.27
6.57
6.68
5.04
4.61
5.14
5.21
5.24
5.26
5.89
5.05
4.04
1.95
1.99
1.52
1.25
1.19
2.21
1.94
1.92
2.03
1.03
0.39
0.43
0.39
0.40
0.42
0.45
0.39
0.27
Page 1
8'162
7'875
7'119
7'238
5'463
4'996
5'568
5'643
5'670
5'695
6'381
5'463
4'375
2'114
2'157
1'647
1'350
1'291
2'391
2'103
2'084
2'194
1'116
426
461
424
437
456
492
418
290
7'786
7'514
6'793
6'905
5'212
4'766
5'313
5'384
5'393
5'434
6'089
5'211
4'175
2'018
2'057
1'572
1'288
1'231
2'273
2'007
1'988
2'094
1'064
406
439
404
416
434
470
400
276
Observed Observed
Observed
Bandwith Bandwidth
Rate [hz]
[KB/s]
2 [KB/s]
Available Bandwidth in Function of Distance
WiFi Range Test
We are sending below the bandwidth limit
Limit of Connection
Network seems allways to switch between 130 and 620 KB/s
Observation
78
Transmitted Byte Stream [KB/s]
0
5'000
10'000
15'000
20'000
25'000
0
5'000
10'000
20'000
Page 2
Emited Byte Stream [KB/s]
15'000
Bandwidth Limits for Different Distances
WiFi Range Test
25'000
Published Images
15 Meter
8 Meter
4.2 Meter
41.5 Meter
D.3. AVAILABLE WI-FI BANDWIDTH IN FUNCTION OF DISTANCE
79
41.5
0
1000
2000
3000
4000
5000
6000
7000
8000
15
4.2
8
0
Distance [m]
Bandwidth [KB/s]
5
10
7598.789304
5630.825797
1925.812815
444.8124594
Mean Bandwidth
[KB/s]
Bandwidth :
Bandwidth 2:
15
25
Page 3
Distance [m]
20
30
35
40
45
Measure 2
Measure 1
Calculated with the image size times observed rate.
Calculated by data size recorded on disk.
Bandwidth Available in Function of Distance
7249.326347
5369.950455
1836.436232
424.1360993
Mean Bandwidth 2
[KB/s]
WiFi Range Test
80
14370900
14381500
14353400
14084300
14161100
14104500
14143800
14211000
14233900
14254800
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
19.06.2012
14373602
14385305
14360507
14093200
14170000
14113701
14151101
14215800
14242800
14263901
Stop
27.02
38.05
31.07
49
49
52.01
33.01
58
49
51.01
505
821
714
102
59
85
80
61
79
123
raw8
raw8
raw8
raw8
raw8
raw8
raw8
raw8
raw8
raw8
jpeg
png
tiff
jpeg
jpeg
png
tiff
jpeg
png
tiff
-
-
-
100
0
85
100
0
85
0
204'673
264'178
270'210
941'078
3'344'915
3'131'647
3'961'750
3'294'754
3'145'063
3'903'089
Number
Camera Save Compress
of
Size[Bytes]
Format Format
ion
images
752
752
752
2448
2448
2448
2448
2448
2448
2448
480
480
480
2050
2050
2050
2050
2050
2050
2050
360960
360960
360960
5018400
5018400
5018400
5018400
5018400
5018400
5018400
18.69
21.58
22.98
2.08
1.20
1.63
2.42
1.05
1.61
2.41
Save
Width High
Size [Pixel] Rate
[Pixel] [Pixel]
[Hz]
The prosilica camera was streaming at maximal rate and the imagehandler did not processed the images to send them.
Condition 3
The prosilica camera was streaming at maximal rate and the imagehandler processed the images to send them.
Condition 2
The bluefox camera was publishing at maximal rate. This was about 46 hz and the imagehandler processed the images to send them.
Condition 1
Start
Date
Duration
[s]
Achieved Image Saving with Image Handler for 3 Formats
Condition 1
Condition 1
Condition 1
Condition 2
Condition 2
Condition 2
Condition 2
Condition 3
Condition 3
Condition 3
Condition:
D.4. ACHIEVED IMAGE SAVING FOR 3 FORMATS
81
363
169.03
123
114.58
210.04
215.49
9.81
19.46
19.54
13.92
17.07
16.64
3561
3289
2403
1595
3586
3586
0.00
20.00
40.00
60.00
80.00
0
40
Images Sent per Second
20
Rosbag
Image Handler
Image Losses
60
Observation: @30 Hz and @50 Hz MEMORY was used to 100%
100.00
Rosbag
10 18404400 18464710
30 19060300 19085200
50 19145200 19170500
20
30
50
3586
3586
3586
1595
3586
3586
Page 1
Save Rate [Hz]
Image
Handler
0.00
5.00
10.00
15.00
20.00
25.00
0
-
0
0
0
Compression
393'148
395'257
412'651
1'003'135
1'003'904
1'003'904
Images
Size
[Bytes]
20
752
752
752
752
752
752
Image
Width
[Pixel]
40
480
480
480
480
480
480
Image
High
[Pixel]
60
Image Handler
Rosbag
Image Save Rate
png
png
png
sensor_msgs
sensor_msgs
sensor_msgs
Format
Images Sent per Second
99.30
91.72
67.01
100.00
100.00
100.00
Images
Duration Saverate Saved
Sent Received
Sent per Start Time Stop Time
[s]
[Hz]
Images Images Percent
Second
Rosbag versus Image Handler
Saved Images [%]
Image Handler versus Rosbag
82
0
0.2
0.4
0.6
0.8
1
Distance
[m]
Send percentage
0
0
1.34E+09
1.34E+09
20
Distance [m]
60
80
1.34E+09
1.34E+09
Time Stamp
1.34E+09
Postion in Funciton of Time
40
1.34E+09
1.34E+09
100
120
1.34E+09
Decreasing Distance
Increasing Distance
Send Percentage in Function of Distance with Working Bandwidth
Adaptation
Demonstration Dynamic Bandwidth Adaptation: Plots
D.6. BANDWIDTH ADAPTATION DEMONSTRATION PLOTS
83
Note
Bandwidth [Bytes/s]
0
20
60
Distance [m]
40
If you want the data that was used to plot, please take contact.
0
1000000
2000000
3000000
4000000
5000000
6000000
80
100
Decreasing Distance
Increasing Distance
Estimated Bandwidth in Function of Distance with Working
Bandwidth Adapation
84
85
Appendix E
Datasheets
86
APPENDIX E. DATASHEETS
XBee-PRO® 868
Long-Range Embedded RF Modules for OEMs
XBee-PRO 868 modules deliver wireless connectivity
with exceptional RF range performance for European
applications.
Overview
XBee-PRO 868 embedded RF modules provide extended-range
wireless connectivity to end-point devices. These modules use a
proprietary 868 MHz point-to-multipoint protocol for European applications. Capable of 500 mW of EIRP transmit power and -112 dBm receive sensitivity, the XBee-PRO 868 is the most
powerful XBee product ever offered. Supporting RF line-of-sight distances up to 40 km, these modules are ideal for challenging environments where RF penetration and transmission distance are
critical to the application.
Application Highlight
As a part of Digi’s XBee® family of RF products, these modules
are easy to use, share a common hardware footprint, and are fully
interoperable with other XBee products utilizing XBee-PRO 868 technology. They are available in a variety of different protocols
to suit different applications, enabling users to substitute one XBee module for another with minimal development time and risk.
Digi’s unsurpassed offering of Drop-in Networking products provides
users with seamless communication between devices. XBee adapters
provide wireless connectivity to electronic devices in wired networks.
ConnectPort™ X gateways enable users to access and configure remote devices in a network.
Features/Benefits
• 868 MHz short-range device G3 band for Europe
• No configuration needed for out-of-the-box RF communications
• Common XBee footprint for a variety of RF modules
• Simple-to-use peer-to-peer or point-to-multipoint topology
Related Products
• Software selectable transmit power
• Outdoor RF line-of-sight range up to 40 km
• Multiple antenna options
Gateways
Development Kits
Adapters
www.digi.com
E.1. XBEE PRO 868 MHZ
87
Platform
XBee-PRO® 868
Performance
RF Data Rate
24 Kbps (limited to 10% duty cycle)
Indoor/Urban Range
Up to 1800 ft (550 m)
Up to 25 miles (40 km) with dipole antenna
(Italy only) up to 10 miles (16 km) with dipole antenna (13.7 dBm)
Outdoor/RF Line-of-Sight Range
Transmit Power
1 mW (0 dBm) to 315 mW (+25 dBm)
Receiver Sensitivity (1% PER)
-112 dBm or 500 mW EIRP
Features
Serial Data Interface
3.3V CMOS Serial UART
Configuration Method
API and AT commands
Frequency Band
868 MHz ISM
Interference Immunity
Multiple transmissions, acknowledgements
Serial Data Rate
1.2 Kbps to 230.4 Kbps (non-standard rates available)
ADC Inputs
6 (10-bit)
Digital I/O
13
Antenna Options
Wired Whip, U.FL connector, RPSMA connector
Networking & Security
Encryption
128-bit AES
Reliable Packet Delivery
Retries/Acknowledgments
Addressing Options
Network ID, 64-bit address
Channels
Single channel
Power Requirements
Supply Voltage
3.0 – 3.6VDC
Transmit Current
500 mA typical at 3.3V (800 mA max)
Receive Current
65 mA typical
Power-Down Current
55 uA typical @3.3V
Regulatory Approvals
FCC (USA)
No
IC (Canada)
No
ETSI (Europe)
Yes (Italy 25 mW max)
C-TICK (Australia)
No
Telec (Japan)
No
Point-to-Multipoint / Star
Peer-to-peer
(top view)
(side views)
PIN 20
PIN 1
1.297”
(32.94mm)
PIN 10
shield-to-PCB
0.080” ±0.020
(2.03mm ±0.51)
0.020”
(0.51mm)
0.031”
0.110”
(0.79mm) (2.79mm)
0.050”
(1.27mm)
PIN 11
0.866”
(22.00mm)
0.960”
0.160”
(4.06mm)
0.079”
(2.00mm)
Visit www.digi.com for part numbers.
DIGI SERVICE AND SUPPORT - You can purchase with confidence knowing that Digi is here to support you
with expert technical support and a strong five-year warranty. www.digi.com/support
Digi International
877-912-3444
952-912-3444
[email protected]
Digi International
France
Digi International
KK
Digi International
(HK) Limited
+33-1-55-61-98-98
www.digi.fr
+81-3-5428-0261
www.digi-intl.co.jp
+852-2833-1008
www.digi.cn
© 2008-2011 Digi International Inc.
All rights reserved. Digi, Digi International, the Digi logo, the Making Wireless M2M Easy logo, ConnectPort, XBee and XBee-PRO are
trademarks or registered trademarks of Digi International Inc. in the United States and other countries worldwide. All other trademarks are
the property of their respective owners. All information provided is subject to change without notice.
91001490
B2/311
BUY ONLINE • www.digi.com
88
2.4 GHz Wireless Module
LT2510
Innovative Technology
for a Connected World
THE FASTEST WAY TO WIRELESS
Laird Technologies’ fifth generation 2.4GHz FHSS module sets the standard for industrial
RF communication. Based on proprietary FlexRF™ technology, this globally-accepted
module will exceed most OEM application and performance requirements.
Embedded with Laird Technologies’ robust server-client protocol, the LT2510 permits
an unlimited number of clients to synchronize to a single server for low latency
communications. The server and all clients in a network can communicate with any
radio in range via either addressed or broadcast packets. The configuration and test
software allows OEMs to design and test networks to suit their applications.
Enhanced API commands provide packet routing control and network intelligence.
With its field-proven FHSS air interface protocol, the LT2510 rejects RF noise, excels
in multipath scenarios, allows for co-located systems, and provides an extremely
reliable communication link. It also provides a more robust, but simpler, link than
ZigBee for RF applications that do not require a mesh topology.
With a throughput of up to 280 Kb/s, LT2510 delivers speedy data rates.
In addition, variable output power options (up to +21 dBm) enable communication
over distances that aren’t achievable with competing technologies. At the same
time, a range of ultra-low power modes plus low Tx/Rx power consumption make
the LT2510 ideal for power-restrictive or battery-operated applications. The mini
SMT package is well-suited for space-constrained designs and is available in
pick-and-place packaging for volume manufacturing. A pluggable version with
two single row headers is also available for ease of integration.
global solutions: local support
USA: +1.800.492.2320
Europe: +44.1628.858.940
Asia: +852.2268.6567
[email protected]
www.lairdtech.com/wireless
TM
FEATURES
MARKETS
• Very robust in the presence
of interference
• High throughput
• Ultra-low power consumption
• Long range capability
• Miniature SMT form factor
• Global acceptance
• Integrated battery monitor,
temperature sensor, GPIOs
and ADC
• Simple integration
• Commercial buildings
• Field surveillance
• Utility management
• Recreation
• Fleet telemetry
E.3. PROSILICA
89
Description
5 Megapixel single-board CCD camera with GigE Vision
The 5 Megapixel GB2450 is a very high-resolution CCD single-board camera with Gigabit
Ethernet output (GigE Vision®). The GB2450 incorporates the high-quality Sony ICX625 CCD
image sensor that provides superior image quality, excellent sensitivity, and low noise.
●
●
●
●
Sony ICX625 Progressive Scan CCD
15 fps at full resolution at 2448x2050
Available with optional vertical connector orientation
Models:
GB2450, 2448x2050, 15 fps, CCD, Mono
GB2450C, 2448x2050, 15 fps, CCD, Color
GB2450-V, 2448x2050, 15 fps, CCD, Mono, Vertical
GB2450C-V, 2448x2050, 15 fps, CCD, Color, Vertical
Explanation of model suffixes (-P, -V, -PV, portrait, vertical)
❍
❍
❍
❍
❍
http://www.alliedvisiontec.com/us/products/cameras/gigabit-ethernet/prosilica-gb/gb2450.html
1
90
Specifications
Prosilica GB
Interface
2450
IEEE 802.3 1000baseT
Resolution
2448 x 2050
Sensor
Sony ICX625
Type
CCD Progressive
Sensor Size
Type 2/3
Cell size
3.45 µm
Lens mount
C/CS
Max frame rate at
full resolution
15 fps
A/D
14 bit
On-board FIFO
16 MB
Output
Bit depth
Mono modes
8/12 bit
Mono8, Mono12Packed, Mono16
Color modes YUV
YUV411, YUV422, YUV444
Color modes RGB
RGB24, BGR24, RGBA24, BGRA24
Raw modes
Bayer8, Bayer12Packed, Bayer16
General purpose inputs/outputs (GPIOs)
TTL I/Os
1 input, 1 output
Opto-coupled I/Os
1 input, 1 output
RS-232
1
Power/Mass/Dimensions/Regulations
Power requirements
(DC)
5V - 25V
Power consumption
(12 V)
3.8 V
Mass
54 g
Body Dimensions
(L x W x H in mm)
Regulations
51 x 89 (board-size W x L)
RoHS, Class A
Download Prosilica GB2450 technical drawing (click here)
http://www.alliedvisiontec.com/us/products/cameras/gigabit-ethernet/prosilica-gb/gb2450.html
2
MVProspekt_BlueFOX-MLC_RZ02.qxd 26.05.2010 11:21 Uhr Seite 1
mvBlueFOX-MLC
E.4. MV BLUE FOX
91
Tiny single-board USB 2.0 camera series
One board –
multiple
impressions
www.matrix-vision.de
■ compact industrial camera series with USB 2.0
■ high quality gray scale and color CMOS sensors
■ 8 Mbyte onboard memory
■ 10 bit ADC
■ hardware LUT 10 bits to 8 bits
■ 55 dB dynamic range / ≤ 110 dB high dynamic range (HDR)
mvBlueFOX-MLC
The mvBlueFOX-MLC module is a fully
featured compact single-board camera
perfectly targeted for space and cost
sensitive OEM applications.
more and up-to-date infos see ▼
www.matrix-vision.com/mvBlueFOX-MLC
A superior image quality in combination with a
very high frame rate make the camera ideally
suited for embedded applications.
◗ USB 2.0 interface (up to 480 Mbit/s)
◗ CMOS with full frame shutter
◗ pipelined global shutter
◗ very high frame frequency of 90 fps
◗ 8 Mbyte onboard memory for secure image transmission
◗ hardware LUT 10 bits to 8 bits
◗ Micro-PLC for real-time sequencing (HRTC)
◗ digital I/O:
1/1 opto-isolated or 2/2 TTL compliant digital
◗ connectors:
Mini-B USB or USB header
USB
Header
digital I/O
Software
◗ Linux® and Windows® drivers, 32bit/64bit (mvIMPACT Acquire)
◗ DirectShow® driver
◗ comes with mvIMPACT Base
◗ mvIMPACT and other third-party software supported
Recognize
A n a ly z e
The driver in combination with the FPGA
represent a perfect team which reduces the PC
load to a minimum.
Hardware
Available sensors
◗ asynchronous external trigger
◗ S-mount or optional C(S)-mount
◗ ADC resolution: 10 bits
◗ AOI with horizontal and vertical step width of 2 pixels
◗ automatic gain control (AGC)
◗ automatic exposure control (AEC)
◗ high dynamic range mode (110 dB)
◗ bus powered < 2.5 W
◗ size without lens (w x h x l): 35 x 33 x 25 mm
◗ weight without lens: approx. 10 g
◗ permissible ambient temperature 5..55 °C
Application areas
◗ machine vision
◗ surveillance
◗ security
◗ life science
◗ medical application
Decide
MVProspekt_BlueFOX-MLC_RZ02.qxd 26.05.2010 11:21 Uhr Seite 2
mvBlueFOX-MLC
92
Available sensors
Model name (CMOS)
Model variant
mvBlueFOX -MLC200w
G
C
Gray
Color
752 x 480
90 Hz
full frame
1/3 "
6x6
6 µs - 4 s1)
yes
USB 1.1 / USB 2.0
Aptina
MT9V034
1)
depending on pixel clock
◗ resolution of active area
◗ maximum frame rate
◗ transfer type
◗ sensor category
◗ pixel size in [µm] (width x height)
◗ integrations time
◗ overlap capabilities
◗ USB type
◗ sensor manufacturer
◗ sensor name
Other sensors to be added on demand
Spectral characteristics of sensor (from sensor data sheet w/o filter)
Dimensional drawing
front view
side view
back view
without S-mount
Assembly variants
◗ Mini-B USB connector and digital I/O on pin header
1/1 opto-isolated or 2/2 digital I/O
◗ USB via header without Mini-B USB connector
◗ female board connector instead of pin header
◗ 3 different S-mount depths
◗ C(S)-mount compatibility using mvBlueCOUGAR-X flange
Accessories
◗ USB 2.0 cable, USB2-A to USB2-Mini B, length up to 8m
◗ lenses on demand
Different connection
variants possible.
Legal notice: The contents of this brochure are intended to provide information only and to show
possible examples. We reserve the right to change technical data and construction at any time
without prior notice. The technical specifications of customer systems and of our current products
have to be clarified when ordering. Date 05/2010
Recognize
A n a ly z e
Decide
MATRIX VISION GmbH · Talstrasse 16 · DE-71570 Oppenweiler · Phone: +49-71 91- 94 32- 0 · Fax: +49-71 91- 94 32- 288 · [email protected]
93
Bibliography
Bibliography
[1] Laube Simon, Gasser Lukas. Design of the electronic systems for an unmanned
blimp (draft). Student project work ETH Zürich, Spring Semester 2012.
[2] Ledergerber Anton, Krebs Matthias. Human-machine interfaces for operating a
blimp (draft). Bachelor’s thesis ETH Zürich, Spring Semester 2012.
[3] Meier Daniel, Mueri Luca. Agile blimp controller design (draft). Bachelor’s
thesis ETH Zürich, Spring Semester 2012.
[4] Team Skye. Final report project skye. Student project final report ETH Zürich,
Spring Semester 2012.
[5] UVEK. Verordnung des UVEK über Luftfahrzeuge besonderer Kategorien. Das
Eidgenössische Departement für Umwelt Verkehr Energie und Kommunikation,
2011.
[6] W. Richard Stevens, Stephen A. Rago. Advanced Programming in the UNIX
Environment. 2nd edition, 2005.

Communication software for a blimp

Transcription

Similar documents

COBRACAM PRO

SceneTracker From DVTEL

6 Channel HD and 8 Analog Audio Recording

June

ele ctronics - rv care online catalog

ximea mq022

Photographer: David Fincher / Model: Rooney Mara / Shot on RED

August 2015 - Gene`s Camera Store

Trix Rabbit Story board

TopConvert - Komax Group