Mobile App Monitoring Quick Start Tutorial

Transcription

Mobile App Monitoring
Quick Start Tutorial
Mobile App Monitoring 8.0
July 2014
Copyright Notice
Copyright © 1995-2014 Keynote Systems, Inc. All rights reserved
THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY
EXPRESS REPRESENTATIONS OF WARRANTIES. IN ADDITION, KEYNOTE DISCLAIMS ALL
IMPLIED REPRESENTATIONS AND WARRANTIES, INCLUDING ANY WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT OF
THIRD PARTY INTELLECTURAL PROPERTY RIGHTS.
All text and figures included in this publication are the exclusive property of Keynote and may not be
copied, reproduced, or used in any way without the express permission in writing of Keynote.
Information in this document is subject to change without notice and does not represent a commitment
on the part of Keynote. Keynote may have patents or pending patent applications covering subject matter
in this document. The furnishing of this document does not give you any license to these patents except
as expressly provided in any written license agreement from Keynote.
The trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other
countries include Keynote®, DataPulse®, CustomerScope®, Keynote Customer Experience Rankings®,
Perspective®, Keynote Red Alert®, Keynote WebEffective®, The Internet Performance Authority®,
MyKeynote®, SIGOS®, SITE®, keynote® The Mobile & Internet Performance Authority™, Keynote
FlexUse®, Keynote DeviceAnywhere®, DeviceAnywhere®, Keynote DemoAnywhere®, Keynote
MonitorAnywhere®. All related trademarks, trade names, logos, characters, design and trade dress are
trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries
and may not be used without written permission. All other trademarks are the property of their
respective owners.
Please forward any comments or suggestions regarding this document to Keynote Support.
Keynote Systems, Inc.
777 Mariners Island Blvd.
San Mateo, CA 94404
ii
Keynote Confidential
Contents
About This Document ...................................................................................................... 5
Document Outline ........................................................................................................................................ 5
Typographical Conventions........................................................................................................................ 5
Contacting Support ...................................................................................................................................... 6
Additional Documentation ......................................................................................................................... 6
1
Set Up for Scripting ............................................................................................... 7
2
Begin Scripting—Launch Application................................................................ 9
3
Action—Searching for an Item in eBay ............................................................ 12
4
5
3.1
Click Search Field ......................................................................................................................... 12
3.2
Verifying Search Field with Branches ....................................................................................... 12
3.3
Populating Script Branches ......................................................................................................... 15
3.4
Script Steps after Verification ..................................................................................................... 17
3.5
Action—eBay Home Button ....................................................................................................... 19
Timers .................................................................................................................... 20
4.1
Verifying the eBay Home Page .................................................................................................. 20
4.2
Timing Application Launch........................................................................................................ 22
4.3
Timing Search Results ................................................................................................................. 25
Test Case and Timer Declaration ...................................................................... 26
5.1
Create a Test Case ........................................................................................................................ 26
5.2
Add Actions .................................................................................................................................. 26
5.3
Timer Declaration ........................................................................................................................ 27
6
Provisioning a Test Case..................................................................................... 30
7
Viewing Results in MyKeynote ......................................................................... 35
iii
About This Document
This document is a tutorial that takes you through creating, provisioning, and viewing the results of a
simple monitor script in Keynote’s Mobile App Monitoring for creating and executing automated monitor
scripts on smart devices.
Mobile App Monitoring is an enterprise-class monitoring service for mobile devices. With Mobile App
Monitoring, production support teams can easily write/record monitor scripts and schedule them at any
frequency on real, live devices. You can define custom thresholds for acceptable performance.
You use our client software Studio to control devices remotely over the Internet/network to create scripts.
Device interaction in Studio is based on Keynote’s core Direct-to-Device technology, which enables
remote (network) access to real, smart devices on live carrier networks.
Script provisioning occurs in the Keynote Service Center and results can be viewed in MyKeynote.
Document Outline
In this tutorial, you will construct and provision a simple monitor script to launch the eBay application
and enter a search string. In Mobile App Monitoring, monitor scripts are test cases composed of smaller
units called actions, screen verifications, and timers, which mark key portions of your monitor script so
you can apply and track performance thresholds.
When writing automated scripts on a real device, you must take steps to ensure that you always start
from a known place such as the device home screen or the application home screen. The action script for
resetting the device to the home screen is provided for you to use in your test case.
In this document:

Set Up for Scripting talks about launching Studio and creating a project to contain your scripts.

Begin Scripting—Launch Application walks you through the creation of a simple script to launch an
application using script recording while interacting with a device.

Action—Searching for an Item in eBay walks you through creating another script to search for an
item in eBay. Verifying device screens is also discussed.

Timers shows you how to set up timers around key portions of your script.

Test Case and Timer Declaration walks you through creating a test case composed of actions and
auto-generating the list of timers used in the test case.

Provisioning a Test Case provides instructions on scheduling a monitor script in the KSC.

Viewing Results in MyKeynote shows you how to access results in MyKeynote.
Typographical Conventions
The table below describes the typographical conventions used this document.
Style
Element
Examples
Blue
Links and email addresses
http://www.keynote.com
The Document Outline section describes the
structure of this manual.
5
Style
Element
Examples
Bold
User interface elements such as menu items
Click My Devices in Studio.
Monospace
Commands, code output, filenames,
directories
Right-click the project’s test cases
directory.
Monospace bold
User input
In a command window, type adb devices.
Italic
Document titles and emphasis
Refer to the Keynote Guide to Test and Monitor
Scripting in Studio for detailed scripting
instructions.
Contacting Support
If you have any comments or suggestions regarding this document, contact Keynote Support at
http://support.keynote.com or 1-888-KEY-SYST (539-7978).
Additional Documentation
You can find additional information at http://www.keynote.com/support/mobile-app-monitoringdocumentation.html.
You can also access documentation from the Help menu in Studio.
6
1
Set Up for Scripting
To begin scripting in Mobile App Monitoring:
1
Launch Studio and log in. Ensure that you are pointing to your Mobile App Monitoring environment
(click Show Advanced—your trainer can help you Change the active profile if you need to).
2
When you’ve logged in to Studio, select Monitoring > Scripting tab. The projects in your
environment are displayed. Projects are containers for scripts and associated devices and timers.
7
3
Select and double-click to open the project with starter scripts—your trainer will provide the name of
the project. (Your trainer will let you know if there is more than one project with starter scripts.)
4
Double-click the actions folder and double-click the script GoHome to view it. You will use this
mini-script, which resets the device to the home screen, in the monitor script that you will construct
in this tutorial. Monitor scripts are test cases composed of actions like this, states, and timers.
NOTE When working with scripts, you can maximize the screen space by minimizing the sidebar
(View > Minimize Sidebar). You can also adjust the size of any pane.
8
2
Begin Scripting—Launch Application
In this exercise you will record a simple script to launch the eBay application.
1
In your project, right-click the actions folder and select New Action.
2
Enter a name for the script, e.g., LaunchEbay, and click Create.
You will see a tab for the new action in the workspace. A list of project devices is displayed.
3
Select the iPhone device and click Implement. When creating a UI-based script (as opposed to an
object-based script), an action requires device-specific implementations to account for differences in
interfaces. Devices have different screen sizes, resolutions, operating systems, and OS versions.
Applications can be different for different platforms.
The workspace now displays a blank script canvas with a command toolbar to the left and a pane for
the device to the right.
9
4
Click Acquire Device to lock it so you can interact with it to define your monitor script.
5
If necessary, return the device to the home screen. You can interact with devices in Studio by clicking
on the device screen and buttons. You can enter data by clicking on the device or by using your
computer keyboard—refer to the Mobile App Monitoring Device Interaction Guide for all the device
control functionality available in Studio.
6
You are now ready to record. Click Record above the script canvas and then click the eBay
application icon on the device. Click Stop Recording when done.
10
The recorder inserts a Find And Touch command. Command properties in the center pane show the
area of the device screen that you just clicked. The device displays the eBay application (right).
App launched
Icon touched to
launch app
7
To check your script, return the device to the home screen and then click Run above the script canvas.
The script stops automatically when done and you can see a quick result bar above the script canvas.
Result bar
8
You are done recording this script. Check in the script to version control.
11
3
Action—Searching for an Item in eBay
In this next action implementation, you will enter a search string in eBay and view search results before
returning the application to its main screen. This requires the addition of logic to account for the
possibility that the search field already has a search string in it, which you need to clear.
3.1
Click Search Field
1
As for the previous script above, create and name a new action, e.g., SearchItem. Then select your
device and click Implement.
2
Click Record above the script canvas and click search icon
Recording.
3.2
1
in the application search field. Stop
Verifying Search Field with Branches
Next, you need to implement a reference point (checkpoint) with two possibilities: a.) there is no
search string in the field and b.) there is a search string that you need to clear before proceeding.
Drag the Wait Event command
from the toolbar onto the script canvas. The command
properties automatically display the current device screen where you have set focus in the eBay
search field.
12
2
In the command properties (center pane) use your mouse to select an area that uniquely verifies that
the search field is empty. For example, select “Search eBay.” Save the command.
3
Open the advanced options (Show Options Editor).
4
Set the Pixel tolerance slider at 100. This means that a 100% of the pixels in the selected image must
be matched at runtime. Save advanced settings.
5
In the Wait Event command, add a branch for the possibility that there is pre-existing text in the
search field:
a
Click Add Logic. The command now shows a default branch (Timeout) for script action when the
command times out, i.e., is not able to complete screen verification.
13
Timeout branch in
script canvas
Timeout branch in
command properties
b
Next, on the device, use your computer to enter any search string (to set up the condition that the
search field contains a preexisting search string).
c
In the command, click Add Event. This creates a branch in which you can verify that the search
field has a pre-existing string. The device screen showing the search string is automatically
captured in the added branch.
Added branch in script
canvas
d
Using your mouse, select an area that uniquely verifies that the search field is populated. For
example, select the “clear” icon
6
14
Added branch in command
properties
. Save the command.
For this branch too, be sure to open advanced options (Show Options Editor) and set Pixel tolerance
to 100.
7
In the Wait Event command, you can double-click to change the name of an event (branch). Enter a
name in the dialog box that appears and click OK. Rename your branches to identify what they
represent, e.g., “IsSearchString” and “IsClear.”
The new names are updated in the script canvas and in the command.
You can also rename the command itself to easily identify what it verifies, e.g., “SearchField.” Click
the rename command icon
3.3
and enter the name you wish. Save the command.
Populating Script Branches
Now you can define script action based on whether a prior search string has been found:
1
Drag the Send Keys command
a pre-existing search string.
2
Click Record in the Send Keys command and then click the clear button on the device.
from the toolbar to the “IsSearchString” branch—the branch for
15
3
Click Stop in the command. You have now recorded a command to clear the search string.
4
onto the canvas
Insert a brief wait time after clearing the search field—drag the Wait command
below Send Keys. This allows the Mobile App Monitoring to “catch up” with the device before you
enter a search string.
5
Notice that there is a Continue command
in the branch of Wait Event for when search field is
clear. Continue is a placeholder command that allows you to connect back to the main flow of the
script; leave this command as is.
6
Select the Error tab of the Fail command. You can trigger error reporting here. Leave the default error
as is.
16
NOTE You can create and store your own error types in project properties for use in scripts—select
Edit Error Types. Errors created in the MAM User Errors category are available to all users in your
account.
This completes setting up a screen verification of the search field.
3.4
Script Steps after Verification
This next set of steps inserts a search string after completing verification of the search field.
1
Drag a Send Keys command into the Tie placeholder that appears when you Add Logic to the Wait
Event command. The Tie placeholder enables you to “tie” branches together and define script steps
after the verification.
2
In the Send Keys command, click in the Text to send field. Using your computer keyboard, enter the
search string “laptop.” The search string will be entered into the device when the script is executed.
3
Rename Send Keys to identify what it does, e.g., “SearchString,” and Save the command.
17
4
Drag in the Wait command to insert a wait time of a few seconds. Again, this allows Mobile App
Monitoring to “catch up” with the device after entering a search string.
5
You can now Record clicking the search button
6
Stop Recording and drag a Success command
to enter a search string.
18
in the eBay application.
to close out the script. This completes the action
3.5
Action—eBay Home Button
Record a final action to reset the eBay application to its home page, e.g., eBayHome, by clicking the home
icon
at the bottom of the application screen.
19
4
Timers
You are now ready to build timers around verifications of key portions of your eBay interactions.
4.1
Verifying the eBay Home Page
In order to monitor the time taken to launch the application, you must verify the application home page
and insert timers around the verification.
1
Ensure that the device is returned to the eBay home page.
2
Open the script to launch the eBay app, e.g., LaunchEBay.
3
from the toolbar onto the script canvas. You will create an
Drag the Wait Event command
image-based verification of the eBay home screen.
4
In the command pane use your mouse to select the blue home button that uniquely verifies the eBay
home screen. Save the command.
20
5
Open the advanced options (Show Options Editor) and ensure that the Pixel tolerance slider is set to
100. Save advanced settings.
6
In the Wait Event command, click Add Logic. The command now shows the Timeout branch for
script action when the command times out.
7
As before, you’ll need to insert an error in the Fail command’s Error tab. You can create an error in
the MAM User Errors Category, e.g., “App Start Error.” Save the command.
21
8
In the Wait Event command, you can double-click to change the name of an event (branch). Enter a
name in the dialog box that appears and click OK. Rename your branch to identify what it represents,
e.g., “IsHome.”
The new names are updated in the script canvas and in the command.
You can also rename the command itself to easily identify what it verifies, e.g., “CheckEbayHome.”
Click the rename command icon
4.2
and enter the name you wish. Save the command.
Timing Application Launch
Once you have created and inserted a command to capture and verify application launch, you are ready
to time the transaction.
1
Create a timer—right-click the timers folder in your project and select New Timer. A timer enables
you to measure the performance of a section of the script.
2
Enter a name to identify what you intend to measure, e.g., LaunchTime, and click Create.
22
You will see a tab for the new timer in the workspace.
3
Check in your timer to save it. It is automatically scanned for problems.
4
To apply this timer to your script, you will need to insert Timer commands
to indicate start and
end points of the section you wish to measure. In this script, to measure the time taken to launch the
eBay application on the home screen, you will need to start measurement after launching the
application and end measurement after verifying its home screen, i.e., before and after the Wait Event
command inserted in section 4.1 above.
a
Drag in a Timer command
above the Wait Event.
23
5
24
b
Select the Timer you just created from the drop-down list.
c
Select the Start radio button to indicate transaction start point.
d
Rename the timer command to identify its location and what it times, e.g., “LaunchStart.”
e
Save the command.
To indicate transaction end point, insert a Timer command in the Tie branch of Wait Event.
a
In the second Timer command, select the “Launch Time” Timer you just created.
b
Select the Stop radio button.
c
Rename the command to identify its position and what it does, e.g., “LaunchStop.“
d
Save the command.
4.3
1
Timing Search Results
In the SearchItem script, create a verification of the search results screen.
Hint: Use text that uniquely identifies the search results screen, e.g., “Refine.”
2
Populate the branches of Wait Event as described in section 4.1 above.
3
In the Wait Event command, select an Error type from the MAM User Errors Category.
4
Create a timer, e.g., SearchResultsTime.
5
Insert the start timer above the search results verification Wait Event. Insert end timers as shown in
section 4.2 above.
NOTE You can insert a stop timer in the Tie placeholder below Wait Event or within every Wait
Event branch excluding Timeout and branches where you fail the script.
25
5
Test Case and Timer Declaration
You are now ready to construct the test case that will be deployed as your monitor script. The test case is
constructed around action calls and timers built around verifications of key portion of your eBay script.
5.1
Create a Test Case
1
In your project, right-click the test cases folder and select New Test Case.
2
Enter a name for the script, e.g., EbayLaunchAndSearch, and click Create.
You will see a tab for the new test case in the workspace. A list of project devices is displayed to the
right.
3
Select the iPhone device and click Save Selected to interact with it as you define your test case.
5.2
Add Actions
The test case you will construct consists of actions to reset the device to the home screen, launch the eBay
application, enter a search string, and return the application to its home screen. You will create and insert
timers to monitor two key portions of this scenario: the time taken to launch the application and the time
taken to display search results.
26
1
Expand your project folders so you can see all scripts. Drag the action to reset the device into the test
case—your trainer will provide the name of the action. An Execute Action command
automatically inserted into the test case script.
is
2
Drag the next action, LaunchEbay, onto the script canvas.
6
Drag in the actions to search for an item (SearchItem) and return eBay to its home screen
(eBayHomeButton).
5.3
Timer Declaration
When using timers in your measurement, you need to auto-generate a list of timers for use by MAM.
1
Right-click your test in the project directory > Generate Script Timers.
27
2
As the timer sequence can vary depending on the test case device, select the correct device.
A test case parameter, DapScriptPageList, containing the timer list is automatically created. To
check this, you can open the parameter in test case properties (right-click test case > Properties).
28
NOTE If you rename a timer, you must: a.) Call the renamed timer from the script that uses it, and b.)
regenerate the parameter to capture the name change in its value.
3
Before running the test case in Studio, check the Timer Consistency box at the top of the script
canvas – this checks that each start timer has a corresponding stop timer and that timer names have
been declared correctly in the DapScriptPageList test case parameter.
7
After completing scripting, Run the test case in Studio to troubleshoot any errors and refine timers if
necessary.
8
Publish your project (right-click the project directory > Publish Project) so that your test case is
available for provisioning in KSC.
29
6
Provisioning a Test Case
In order to schedule a test case to run at regular intervals and collect monitoring data, you must provision
it in the Keynote Service Center (KSC). You can only provision measurements on private agents (devices)
in KSC.
1
Log in to KSC. All the measurements provisioned using your agreement ID are listed.
2
Select Add to bring up the Enter Agreement ID screen. Enter your ID and click OK.
30
3
Select the Mobile App Monitoring product and click Continue.
4
Fill out measurement details as follows:
a
Enter an Alias (80 characters or less) to identify your measurement. For example, use a
combination of agent MCD (number), test case, and device type, e.g.,
10012eBaySearchingGS4 or 10008CheckBalanceiPhone5.
b
Select an Agent on which to run the measurement.
c
Select a Measurement Interval. This determines the maximum number of measurements per
hour. The maximum number of runs per hour is 12, at intervals of 5 minutes.
Be sure to check the run time of your test case in Studio before choosing a measurement interval.
Be sure to choose an interval that is longer than your total test case execution time + a minute’s
buffer for acquiring/releasing the device. If your script runs for 8 minutes, you must choose an
interval of 15 minutes or more. If your total execution time is 4 minutes, you can schedule
measurements at 5-minute intervals.
d
Specify a Max Run Time in seconds. This is a maximum run time after which execution is cut off
with a maximum time exceeded error. Note that reducing the maximum run time does not result
in a greater number of monitor runs. Use this value instead as the maximum acceptable monitor
run time. Script runs that exceed this time are stopped.
e
Set an Expiration date—your monitor will continue at the specified interval until this date.
31
f
Click Choose from Script Library to select a script. Each script is listed with the project
containing it, publication timestamp, and the name of the user who edited and published it. You
can Filter the list by any string.
There can be several entries for a script if it has been edited and published by several users. In the
image below, the script “Beta” has been published at different times by three different users and
therefore has three entries.
g
32
Select a radio button, scroll to the bottom of the list and click Select.
h
5
Your selections are listed on the measurement configuration page. Click Continue.
Select your script from the first available drop-down list next to a relative time slot.
33
6
Other slots at your chosen time interval (every 15 minutes in this case) are automatically populated
with your script. Click Submit.
A confirmation screen appears. Click Continue at the bottom to exit to the measurements list.
34
7
Viewing Results in MyKeynote
To view results in MyKeynote:
1
Log in to MyKeynote with administrative credentials.
2
Navigate to Graphs > Analyze:
3
Select the measurement to be viewed:
a
Enter a search term in Find by and click Search to find measurements containing the search term.
b
Select one or more measurements (as applicable) in the results window.
35
c
Select Scatter Plot in the section Click a graph type to generate.
d
Select a time range; two options are available:
e
36

Relative time range: Select the time period from the present time for which measurement
data is to be displayed.

Absolute time range: Select the absolute time range for which measurement data is to be
displayed.
Click Generate Graph.
Performance Graph tab displays the measurement data points in elapsed time format (zooming
in to 200% will provide greater details):
NOTE Hovering over any of the data points will display basic parameters for the measurement:
f
Select a data point and drill down to view details for that measurement:
i
In the Performance Graph view, click on the data point.
ii
In the Table view, click on the Page Detail for the particular data point.
Measurement details are shown as follows:

General section containing basic details related to the measurement (zooming in to 200% will
provide greater details).
37

Screenshots section containing the image(s) for each successful timer/page:
NOTES MyKeynote displays full-scale images.
Each image is titled to display the following: Timer Sequence Number, Timer Name, Timer duration.
38

Mobile App Monitoring Quick Start Tutorial

Transcription

Similar documents

view flyer - AppleUsers.org

MobiGuide: Homecare per Pazienti con Fibrillazione Atriale

a PDF here.

Tweaks for better Performance

Maxx Money Kids` Club

Vinícius R. P. de Carvalho

STS APP GUIDE – IOS DEVICE

VALENCIA 2 Y100

Credit Cards - General Electric Credit Union

The Common Application

AURA-SOMA® POMANDERS - colourfulsoulconnections.com.au