Key Tests For Evaluating Performance of GP

Transcription

Spirent Introduction
Welcome Canada
Who is Spirent?
 HQ: Crawley, UK
 Positioning HQ: Paignton, UK
 Founded 1936
 Three Divisions:
 Networks & Applications, Wireless & Service Experience,
and Service Assurance
 2015 Sales
 $477.1M
 Testforce (Canada) /Sales Support
 Sales, Sales Engineers and Tech Support* located in North America
 Testforce has been exclusive distributor for Spirent in Canada since 2002 (14 years)
Spirent Communications
PROPRIETARY AND CONFIDENTIAL
2
Who is Spirent (Positioning/Navigation)?
Proven and Trusted
 30 years experience in positioning technology
 Spirent is the GPS/GNSS vendor choice of key GNSS
Organizations. Used by
 GNSS system operators (GPS, Galileo, GLONASS, etc.)
 Chip set and GPS/GNSS receiver manufacturers
 GNSS subject matter, academic and defense/military experts
 Models created from ground up and tested over 30 years.
 Proven models mean trusted results. No false positives or incorrect fails reported
due to simulator performance
 Signal integrity and accuracy consistent throughout product line
3
Who is Spirent (Positioning/Navigation)??
Proven and Trusted
 Uncompromised quality and performance at all levels
 Spirent is known for quality and high performance. We also offer solutions for
new users to GNSS, fundamental testing needs and production test.
 Quality never compromised – Quality test equipment important at all phase of
test and levels of
 Expertise and Information Resource
 White papers and eBooks available on a variety of topics from how to select a
simulator to running fundamental GNSS tests to application specific testing
needs.
 New white papers/eBooks released regularly
 www.spirent.com
4
Certifications and Validations
 Spirent simulators have been fully certified by GNSS system operators for nearly 20 years
 GPS-JPO security approval for SA/A-S and EVTP testing
 GPS Wing security approval for modernised user equipment testing (MUE)
 Galileo capability validated by the Galileo Design Authority (ESA)
 Spirent has been approved by the Russian Federation for GLONASS Simulators
 Additionally Simulation by Spirent simulators has been validated by numerous user segment
authorities including:
•
RTCM11010.2 testing (Maritime GNSS equipment)
•
IEC 61108-3 (Personal Locator Beacons)
•
3GPP (Smartphones)
5
Navigation/Position GNSS Product Line
GNSS Test Equipment
Robust PNT
Automation
Professional Services
6
Navigation/Positioning Solutions
Positioning Test Equipment
• GNSS Simulators from A to Z
• RF Record and Playback for field testing
Robust PNT
• Test for GNSS receiver resiliency
• Test equipment and services
Automation
• Software to automate ALL test equipment and provide reports
• Off the shelf and custom solutions
Professional Services
• GNSS simulator and test expertise
• Test Scenarios to test plan development to custom services
7
Navigation/Position GNSS Test Equipment
GSS6300
GSS6300M
GSS6700
GSS6425
Single Channel
5 constellations
4-8 channels
Embedded
Controller
12 channels
Multi software
RF Record &
Playback
Production
Vendor Selection/Integration/Validation
Field Test
GSS9000
Multi-Frequency
16 channels /
160 Channels
R&D
8
Introducing the Spirent Interference Detector
The Spirent
Interference Detector is
a fixed listening device
that monitors the RF
environment in the
GPS bands, over a
range
of approx. 500m, for
potential sources of
interference to the GPS
signal.
9
Which Test Methodology is best fit?
When to use a simulator vs RF record and playback?
Method / Attribute
Live-Sky
Simulation
RPS
Repeatable



Controllable


Partial
Reference truth



Realistic

Representative

 Proving performance requires a Simulator. It is the only way to get known accurate test
data
 User Experience is also necessary. Recorded real signals best options because is
repeatable
10
Commercial Applications & Use-Case Examples
• Hovering/vibration
• Landing drone on moving vehicle
• Multi-frequency or dual-antenna
• Can you confirm your location is
accurate?
• Has your design decreased
GNSS receiver performance?
IoT /
Wearables
Drones
Timing
Automotive
• How does your device handle leap
seconds?
• How do segment errors or GNSS
outages impact performance?
• Does your vehicle combine GNSS,
CANbus or sensors for positioning?
• Are you drive testing or using
repeatable test solution?
11
Defense/Space Applications & Use Case Examples
• Jet fighter landing on aircraft
carrier (moving object)
• Testing 6 DOF with flight
simulator and HIL
• Test extreme inflight maneuvers in
lab before flying
• Add terrain obscurations to test
performance
Vehicles /
aircraft /
autonomous
Highdynamics
CRPA / antijamming
Space
• Are you able to accurately predict
performance at launch, orbit and reentry?
• Testing GNSS and IMU
simultaneously
• Test phase shifting, spectral
filtering or adaptive beamforming.
• Radiated/anechoic chamber or
conducted
12
Key Tests For Evaluating
Performance of
GPS/GNSS Modules
Canada - 2016
13
Agenda
 GNSS constellation updates
 Why is GPS/GNSS testing needed?
 Identify key basic static tests
 Testing tools
 Q&A
 Summary
14
GNSS Constellation Update
40 % Receivers are Galileo Ready
[1]
Source: GPS World, May 2016
15
GPS Modernization
Source: US PNT Co-ordination Office, March 2014
16
GPS Constellation Status @ January 2015 (still the same)
 31 Satellites in orbit
 30 Operational
•
3 - GPS IIA L1 P(Y)+C/A L2 P(Y)
-
•
12 - GPS IIR
-
•
Launched 98-04
7 - GPS IIR-M Added - L2C, M-code L1 & L2
-
•
Launched 90-96
Launched 05-09
8 - GPS IIF Added - L5
-
Launched 10- (4 launched in 2014)
 1 in Maintenance
•
GPS IIR-M
17
GLONASS
 Full system in 2010, after
Decree No. 587 August 2001
 Today 24 satellites in orbit, 24 operational
 Recent setbacks & problems
 July 2013  3 satellites launch failure
 5 December 2010  3 satellites launch failure
 April 2014  Incorrect ephemerides upload
 New-generation Russian GLONASS-K satellite began regular broadcasts
on Feb. 15
 Additional monitoring facilities planned Nicaragua, Antarctica
Source: http://glonass-iac.ru/en/GLONASS/ & GPS world
18
Galileo Update
Galileo Space Segment will include a constellation of a total of
30 Medium Earth Orbit (MEO) satellites, including 6 spares, in a
.
so-called
 Experimental phase (two satellites)
 launched respectively December 2005 and April 2008
 In-Orbit Validation (IOV) phase (four satellites)
 First pair was launched Oct 2011 & next pair in Oct 2012
 Full Operational Capability (FOC) phase (four IOV
satellites plus 26 FOC satellites)
 an intermediate initial operational capability (IOC) milestone
with 18 satellites in operation
 Four pairs of FOC satellites launched by Soyuz from French
Guiana, between Aug 2014 and Dec 2015
Source: http://www.esa.int/Our_Activities/Navigation/The_future_-_Galileo/What_is_Galileo
19
BeiDou Update
BeiDou will include a constellation of a total of 5 GEO, 3 IGSO
& 27 Medium Earth Orbit (MEO) satellites, by end of 2020
 A total of 16 satellites were launched during in phase 1 as part of regional
coverage by end of 2011
 Global coverage started in 2015 with the first launch of a new-generation of
satellites
 The fifth of the new series, the middle-Earth-orbiting
(MEO) satellite will join its four predecessors
 China launched the 22nd BeiDou satellite into orbit on Tuesday.
Source:http://en.beidou.gov.cn/
20
Objectives
 Analyze and compare the relative tracking performance of various GNSS
receivers such as tracking sensitivity, number of tracking channels, etc.
 Understand how to evaluate the time a receiver takes to calculate the first
position fix and measure the accuracy of the computed position to a true location
 Review metrics to consider during the performance evaluation
 Test Setup Automation
21
Why Test: Fair Comparison
Data sheets contain all the parameters that help developers select the right
manufacturer – but do they give the complete picture?
Satellite
Signals
Systems
Drop
Shock
Simultaneous
Tracked
Channels
Acquisition
Sensitivity
Altitude
Rating
Reacquisition
Type of
measurements
Vibration
Correction
Sources
Tracking
Sensitivity
SBAS
Accuracy
Time to
First Fix
Maximum
Speed
Channels
Update
Rate
PPS out
Maximum
Altitude
22
Classification of tests (What to test)
There is not an standard reference for Standalone GPS testing,
but the most common test are:
 Time to First Fix
 Acquisition sensitivity
 Tracking sensitivity
 Time to reacquisition
 Static Position accuracy
There is not a pass/fail criteria for each test
Critical - Test before and after. Must test prior to
integration to compare performance post integration
23
Applications: Which Performance Factors are Most Important?
Galileo
Mobile
Devices
GLONASS
Military
Applications
GPS
Commercial
Air Travel
Automotive
COMPASS
Space
Rail
Survey
24
Simulation
 Control
RF Constellation
Simulator System
 Repeatability
Test environment
scenario generation
 Flexibility
 Efficiency
 Completeness
Comparison of
receiver calculated
PVT with known
simulation data
?
RF signals
commensurate
with ICD
NMEA
or other
data
25
Metrics used to measure performance
TTF
Yield
Position
Accuracy
Signal
Strength
26
GNSS test categories
Field Test
Measurement in real-world
 Issue reports
 Review results
Integration
Measure inside your product
Chipset
Measurements performance
before integration
NMEA
data
27
Common GNSS tests
Test 5: Time to reacquisition
Field Test
Measurement in real-world
Test 4: Static Position accuracy
Integration
Measure inside your product
Test 3: Tracking sensitivity
Test 2: Acquisition sensitivity
Chipset
Measurements performance
before integration
Test 1: TTFF
28
Test 1: Time to First Fix (TTFF)
What is it?
Time to First Fix (TTFF) describes the time and process required for a GPS device
to acquire enough usable satellite signals and data to obtain a PVT solution.
Cold Start
Warm Start
Hot Start
• Time unknown
• Time known
• Time known
• Almanac
unknown
• Almanac
known
• Almanac
known
• Ephemeris
unknown
• Ephemeris
unknown
• Ephemeris
known
• Position
unknown
• Position known
(rough)
• Position known
(rough)
29
How to test
Simulator
Receiver
What to look for
• Static location
• Duration: 24 hours
• 24 Hours Static
v5-03
• ColdStart: Reset
receiver’s NVRAM
• WarmStart: Acquire
full alamanc & clear
ephemeris
• HotStart: Simple
receiver restart
• Extract NMEA data
and compare against
truth
• Due to stochastic
nature several TTFFs
should be obtained
30
Expected results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
UTC of Position
Latitude
N or S
Longitude
E or W
GPS quality indicator (0 invalid; 1 GPS fix; 2 Diff.GPS fix)
Number of satellites in use [not those in view]
Horizontal dilution of position
Antenna altitude above/below mean sea level (geoid)
Meters (Antenna height unit)
Geoidal separation (Diff. between WGS-84 earth ellipsoid and mean geoid
Meters (Units of geoidal separation)
Age in seconds since last update from diff. reference station
Diff. reference station ID#
Checksum
TIME: 184946.000 is HHMMSS.SS
 After performing a cold start to the receiver
 Monitor the first change in the GPS quality
indicator from 0 to 1 or 2
 If there is a change in the previous
indicator. Check for the presence of latitude
and longitude in the message
 The TTFF is the time of performing a cold
or warm start to the first reported Fix (LLA)
31
Field Test
Measurement under
real-world
Integration
Measure in your product
Chipset
Measurements from factory
Test 1: TTFF
32
Understand Acquisition & Tracking Stages
Acquisition is coarse & tracking is fine
Source: http://what-when-how.com/a-software-defined-gps-and-galileo-receiver/gnss-receiver-operation-overview-gps-and-galileo-receiver/
33
What is it?
The Acquisition Sensitivity of a GNSS receiver is defined as the minimum signal
level that is required to obtain a PVT solution.
ACQUISITION SENSITIVITY
RF
power
 A Cold Start is performed in each
power increment
-130 dBm
 Usually the increments is
performed by steps of 0.5dB or
1dB
-145 dBm
 Time between tests must be taken
in account by the tester
-160 dBm
Frequency domain
34
How to test
Simulator
Receiver
What to look for
• Static location
• Every 30 secs signal
strength increases by
0.5 dB
• User Action File:
Acqu_sens.act
• ColdStart: Reset
receiver’s NVRAM
• WarmStart: Acquire
full alamanc & clear
ephemeris
• HotStart: is the same
as reacquisition test
• Observe CN0s
reported
• AGNSS receivers
can acquire below
-150 dBm
35
Expected results
1
Total number of messages of this type in this cycle
 Perform a cold start as many times you
2
Message number
3
Total number of SVs in view
 Decrease the power
4
SV PRN number
5
Elevation in degrees, 90 maximum
6
Azimuth, degrees from true north, 000 to 359
7
SNR, 00-99 dB (null when not tracking)
8 - 11 Information about second SV, same as field 4-7
12 - 15 Information about third SV, same as field 4-7
 Decrease the power from the nominal
power level dB by dB
 Check for SATID and SNR of each satellite
reported
 Once the SNR of the satellites reported is
unstable you reach its acquisition sensitivity
16 - 19 Information about fourth SV, same as field 4-7
36
Field Test
Measurement under
real-world
Test 4: Position accuracy
Integration
Chipset
Test 1: TTFF
37
What is it?
The minimum satellite power level where the receiver can keep tracking the code
and carrier phase and maintain a position fix.
TRACKING SENSITIVITY
RF
power
 Single Cold Start performed in the
beginning of the test
-130 dBm
 Time between tests must be taken
in account by the tester
-145 dBm
 A statistical analysis is required.
Several test need to be performed
by the tester
-160 dBm
Frequency domain
38
How to test
Simulator
Receiver
What to look for
• Static location
• Every 30 secs signal
strength decreases
by 5/0.5 dB
• User Action File:
track_sens.act
• Receiver is in
continuous tracking
mode – no changes
needed during the
course of this test
• Observe CN0s
reported
• Receivers continue to
track satellites below
acquisition sensitivity
39
Expected results
1
Total number of messages of this type in this cycle
 Perform a cold start as many times you
2
Message number
3
Total number of SVs in view
 Decrease the power
4
SV PRN number
5
Elevation in degrees, 90 maximum
6
Azimuth, degrees from true north, 000 to 359
7
SNR, 00-99 dB (null when not tracking)
8 - 11 Information about second SV, same as field 4-7
12 - 15 Information about third SV, same as field 4-7
 Decrease the power from the nominal
power level dB by dB
 Check for SATID and SNR of each satellite
reported
 Once the SNR of the satellites reported is
unstable you reach its acquisition sensitivity
16 - 19 Information about fourth SV, same as field 4-7
40
Field Test
Measurement under
real-world
Integration
Chipset
Test 1: TTFF
41
What is it?
The purpose of this Key Performance Indicator is to test how accurately the receiver
can determine its true position.
 Cold Start
 Warm Start
 Hot Start
Picture reference: http://blog.oplopanax.ca/2012/11/calculating-gps-accuracy/
42
How to test
Simulator
Receiver
What to look for
• Any static scenario
can be used
• Receiver is in lock
mode – no changes
needed during the
course of this test
• Several measures to
position accuracy:
CEP, SEP, 3DRMS,
67% error & 95%
error
43
Expected results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
UTC of Position
Latitude
N or S
Longitude
E or W
Checksum
LATITUDE
DDMMM.MMMMM => D+MMM.MMMMM/60
LONGITUDE
DDDMM.MMMMM =>DDD+MM.MMMM/60
 Considerate if it is N/S or W/E sign
dependency
 A known position (LLA)
 Calculate the distance between the known
position to the reported position. Typically it is
known as horizontal/vertical error
44
Accuracy Measures
Dimensions
Accuracy
Measure
Probability
(%)
Typical
Usage
1
rms
68
Vertical
2
CEP
50
Horizontal
2
rms
63-68
Horizontal
2
R95
95
Horizontal
3
2drms
95-98
Horizontal
3
rms
61-68
3-D
3
SEP
50
3-D
Ref: ArtigoAcuraciaGPSsemAutor.pdf
http://gpsworld.com/gps-accuracy-lies-damn-lies-and-statistics/
45
Field Test
Measurement under
real-world
Test 4: Static Position Accuracy
Integration
Test 3: Tracking Sensitivity
Test 2: Acquisition Sensitivity
Chipset
Test 1: TTFF
46
What is it?
Re-acquisition time is the time necessary for a receiver to regain a PVT solution
after total loss of all received signals
Outage simulated
47
How to test
Simulator
Receiver
What to look for
• Any scenario
• On/off commands to
control duration of
signal loss
• User Action File: reacqu_time.act
• Receiver is in
continuous tracking
mode – no changes
needed during the
course of this test
• Observe time to fix
and CN0s
• Satellite geometry is
a factor for this test
48
Expected results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
UTC of Position
Latitude
N or S
Longitude
E or W
Checksum
 Monitor the change in the GPS quality
indicator from 0 to 1
 If there is a change in the previous indicator.
Check for the presence of latitude and
longitude in the message
 The TTR is the time that the receiver takes to
reacquire and lock positon when and outage
happened
 Time to Reacquisition is faster than TTFF
49
Testing tools
Spirent simulators and RPS
R&D
Integration
Verification
Production
GSS9000
GSS6700
GSS6425
GSS6300 (M)
Processes
Application examples
50
PT TestBench
Spirent’s PT TestBench is an automation and reporting software tool, which
provides you with an integrated test solution, enabling characterisation and
vulnerability assessment of GNSS receivers using pre-defined Test Cases.
51
PT TestBench：
User Interface
iTest Activities
streamline testing
efforts around the
most common
automation tasks
Create & edit test cases
in an easy to edit format
that doesn’t
requirement scripting
编辑
Quickly add automated
test steps through
capture and analysis.
Easily execute your
test and view test
reports or share result
with others
Use chart or table to show
the key data extracted
from the test, store the
data into external database
for later usage
Add analysis to create robust
test cases. Responses can
be abstracted to allow test
case portability and ease of
maintenance
52
Sample Reports
53
Time to start thinking about Vulnerabilities?
Data from Broadcom
Corporation
54
Q&A
55
Summary
What did we learn today?
 Understand your application and GNSS related use cases
 There are no defined standards for GPS/GNSS testing,
but there are common tests effective for defining performance
 Understand the results
 For more information visit:
 www.spirent.com
 Follow us on LinkedIn
:
Spirent Communications / Spirent Positioning and Navigation
 Please provide feedback and complete brief survey
 Contact us for more info:
 [email protected]
 [email protected]
56

Key Tests For Evaluating Performance of GP

Transcription

Similar documents

spirent studio performance - Info-Point

PDF 1.14Mb - Spirent Corporate

Convergence Challenges for the SME

Slides

Manufacturing Energy Management Solution (MEMS)

Proprietary and Confidential — AREA 17

Cybex Trazer

GEOTRAX HARA-300

2 aeronautical industries to materials a4m 8 2 13 salvato

Business Group Admin

Spirent mX2 40GbE and 10GbE Dual-Speed Test Modules

Helping a complex world connect