1503 Debugging Simulation Models — Advanced

Transcription

370
1503
Debugging Simulation Models — Advanced
CONFIDENTIAL — RESTRICTED ACCESS:
This information may not be disclosed, copied, or transmitted in any format without the prior written consent of OPNET Technologies, Inc.
© 2010 OPNET Technologies, Inc.
Debugging Simulation Models (Advanced) Lab
Overview
Welcome to the Debugging Simulation Models — Advanced lab. The purpose of this lab is to show
you how to use the C/C++ source code debugger to debug OPNET simulations. A source code
debugger can be helpful in diagnosing problems with OPNET simulations. After following these
exercises, you should be able to use a C/C++ source code debugger in debugging any OPNET
simulation.
Labs
•
Using a C/C++ debugger to catch a simulation abort
•
Using a C/C++ debugger and ODB – Advanced
•
Using execution trace to validate changes
•
Using advanced memory tracking features
Note
•
To run the labs involving the C/C++ debugger, you must install the Debugging Tools for
Windows suite found on Microsoft’s web site:
http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx. The version used for
these labs is 6.12.2.633.
•
To run Lab 2 you need to install the Windiff utility on the machine where OPNET Modeler
is installed.
•
To successfully run these Labs, you need to update your preference file by adding the full
path to the cdb.exe file and to the windiff.exe file. For example, on a 32-bit machine you
could add the following entries:
des.windows_32bit_debugger_executable_path: "C:\\Program Files\\Debugging Tools for
Windows\\cdb.exe"
mdiff_diff_prog
: "C:\\Program Files\\Microsoft Platform SDK\\Bin\\WinDiff.Exe"
CONFIDENTIAL – RESTRICTED ACCESS: This information may not be disclosed, copied, or transmitted in any format without the prior written consent of OPNET Technologies, Inc.
1
Lab 1: Using a C/C++ Debugger to Catch a Simulation Abort
Overview
This lab will show you how to use a C/C++ debugger to diagnose a simulation abort.
Run Simulation up to the Abort
1. Run OPNET Modeler® and choose File > Open.
2. Select 1503.prj.
3. Switch to scenario lab1 (choose Scenarios > Switch To Scenario > lab1).
4. Run the simulation (Ctrl+Shift+R or choose DES > Run Discrete Event Simulation).
5. Observe program abort.
View Call Stack and Code
1. Choose Help > Error Log > Open.
2. Scroll up to the function call stack for the program abort.
a. Note the process model name and state.
b. Also note the block line number.
2
3. Return to the OPNET Modeler window.
4. Open process model gna_clsvr_mgr_1503_lab1.
5. Open the function block
and find line 3707.
6. Examine the program block for possible causes of the problem.
7. Invalid memory access often means an invalid pointer. It is likely that sess_ptr pointer is not
valid.
8. Close the function block by clicking on the X in the upper-right corner of the window.
Preparing a Simulation for Debugging
1. Set the compilation flags to include debugging information.
a.
Go to process model gna_clsvr_mgr_1503_lab1.
b.
Choose Compile > Compile Code (Advanced).
c.
Note that correct debug compilation flags are set for the development kernel.
d.
Click Cancel.
3
2. Recompile
the process model.
Re-Run Simulation with Debugger
1. Open the Configure/Run DES
dialog box.
2. If you see the following dialog, click on the Detailed… button, to bring up the advanced
version of the Configure/Run DES dialog
4
3. Select the Common tree view item, and make sure the Simulation Kernel menu is set to
development or Based on ‘kernel_type’ preference if the preference is set to development.
Or
5
4. Click on the Run button in the lower-right.
5. When the Simulation Execution dialog comes up, select Simulation > Attach Windows
Debugger (CDB).
6
6. Click Continue.
7. When the abort occurs, the debugger will automatically bring you to the error.
Find and Fix the Problem
1. Observe the following:
•
Source code of the current function is loaded.
•
An arrow points to the source code at the line of the abort
•
You can see the value of sess_ptr in the Local and State Variables tab at the bottom of
the window (you may need to scroll down to see the variable).
7
We conclude that the cause of the abort is the use of the variable sess_ptr, which has an invalid
value (NIL). The value is taken from the state variable sess_list, so further investigation will focus
on how the variable got the bad value. A recoverable error about invalid list index, which was issued
immediately before the crash, hints that there may be something wrong with the list iteration logic.
Indeed, the for loop where crash occurred traverses the list backwards, starting at the sess_list_size
position, but list position indexes start at 0. Thus correct backward list iteration should start from
session_list_size-1 and continue until i is non-negative.
2. Select Simulation > Detach Windows Debugger (CDB).
3. Close Simulation Execution dialog.
4. Return to the process model: select the gna_clsvr_mgr_1503_lab1 window.
5. Open the function block.
6. Correct the for loop bounds at the line 3706:
for (i = sess_list_size-1; i >= 0; i--)
7. Close and save your changes.
8. Recompile the process model.
9. Return to the Project Editor (the 1503-lab1 window).
10. Run the simulation. Hit Continue if you’re at the ODB> prompt.
11. Observe that the simulation runs to completion.
End of Lab 1
8
Lab 2: Using a C/C++ Debugger and ODB — Advanced
Overview
This lab shows you how to use a C/C++ debugger in concert with the OPNET debugger (ODB). The
techniques demonstrated in this lab are useful for debugging most simulation model problems.
In this lab, we will observe a number of recoverable errors involving the use of an invalid statistic
handle. We will start by analyzing the first error message and extracting some useful information
from the error message. Next, we will run the OPNET debugger and attach the CDB debugger to the
simulation process. Using the information gained from the error message, we will then run the
simulation up to the event of the error; we will see (by stepping through the CDB debugger) where
the bug occurs.
Run Simulation and Examine Problem
1. Return to the 1503 window in the Project Editor.
2. Select scenario lab2.
3. Run the simulation.
4. Observe a number of recoverable errors.
5. Open the error log (choose Help > Error Log > Open).
6. Scroll up and view the function call stack for the first error encountered:
•
Note the process model name and state.
•
Also note the block line number.
•
Note that the error occurred at event 4995.
Observation: All errors are reported by one particular module "top.Campus.This
Workstation Does Some Web Browsing And Maybe Something Else Too.mac".
9
At this point, we will take a look at the code where the error message occurred: the call to
op_stat_write() in the ethernet_mac_phys_pk_accept function called from
ethernet_mac_v2_1503_lab2 process model.
7. Return to the OPNET Modeler window.
8. Open process model ethernet_mac_v2_1503_lab2.
9. Open the function block and scroll to line 592, which is the block line number reported in the
error log.
10. Observe 4 op_stat_write () calls in that block:
Conclusion: At least one of statistic handle does not seem to be initialized correctly for a given
process instance. Let's find a piece of code where those statistic handles are initialized.
11. Search for global_ete_handle, one of questionable statistic handles in the same function
block.
12. Search reveals that all relevant statistic handles seem to be properly initialized in function
ethernet_mac_stat_init2 ().
10
Hypothesis 1: ethernet_mac_stat_init2 () is not called. To check this hypothesis lets use the
source level debugger to see if that function is actually called.
13. Return to 1503 project window.
14. To configure the simulation to run ODB, open the Configure/Run DES dialog box.
15. Select Execution > OPNET Debugger in the tree view.
16. Check the Use OPNET Simulation Debugger (ODB)… checkbox.
17. Run the simulation (click on the Run button).
18. An ODB Console will appear.
19. Select Simulation > Attach Windows Debugger (CDB)
20. To set a source breakpoint in function ethernet_mac_stat_init2at CDB:
a. Type "bp ethernet_mac_stat_init2" in the CDB: prompt area.
11
b. Switch to the Source Breakpoints tab at the bottom of the debugger window. Note that
your new breakpoint has been recorded.
21. Click the Continue button.
22. Simulation stops when it hits the breakpoint set in ethernet_mac_stat_init2.
Conclusion: Hypothesis 1 failed.
Hypothesis 2: ethernet_mac_stat_init2 () is not called for a specific instance of the process
model that is running inside of "top.Campus.This Workstation Does Some Web Browsing
And Maybe Something Else Too.mac" module.
To verify this hypothesis lets set a Conditional breakpoint that is trapped in the specific instance
of the module.
23. First let’s deactivate the breakpoint that was set in ethernet_mac_stat_init2 (). Right-click
on the breakpoint's diamond in the source view and choose "Delete Breakpoint".
24. From the object tree observe that the object id of the module in question is 14481.
25. Observe that one of mac process state variables contains the parent module object id.
12
26. Let’s set a breakpoint at line 770 of the ethernet_mac_v2_1503_lab2.pr.c source file that is
triggered when my_objid state variable value is 14481. CDB syntax for that is:
WARNING: Be very careful when choosing the correct quote marks for the above command.
13
•
The ` quote before the ethernet_ and after the :770 is a back quote (or grave accent)
(found in the upper-left part of your keyboard).
•
The “ quote before the j @@c++ and after the ‘gc’ is a double-quote (found near the
enter key).
•
The ‘ quote around the gc and before the ; are single-quotes (same key as the doublequote).
•
The entire command spelled out character-by-character:
o bp<space>
o <backquote>
o ethernet_mac_v2.pr.c:770
o <backquote>
o <space><double-quote>
o j @@c++(op_sv_ptr->my_objid == 14481)
o <single-quote><single-quote><semi-colon>
o <single-quote>
o gc
o <single-quote>
o <double-quote>
•
The command itself is also in the file
c:\op_models\lab2_conditional_breakpoint_command.txt. You can copy and paste
the command from the file into the debugger.
27. Click the Continue button and observe that execution stopped at the breakpoint that we just
set. You can verify that my_objid state variable value in the Local and State Variables
panel is indeed 14481.
14
Conclusion: Hypothesis 2 failed too.
Hypothesis 3. Statistic handles are initialized correctly, but something later incorrectly modifies the
memory that contains the statistic references.
To verify this hypothesis we will use a Watchpoint: a type of breakpoint that is triggered when the
value stored at particular address in memory changes.
28. Remember that global_ete_handle was one of statistic handles suspected to become
corrupted. Expand global_ete_handle state variable line in the Local and State Variables
panel and observe that all fields are set to zero.
29. Let’s set a watch point that is triggered when global_ete_handle.validation value changes.
a. First we need to determine the address of that field in memory. CDB's "dt -b
op_sv_ptr" will print the value of the op_sv_ptr pointer and offsets from that
address to all structure members.
15
The numbers circled in red in the image below may be different from what you see when
running your simulation.
....
b. This tells us that the address of global_ete_handle.validation field in memory is
op_sv_ptr value (0x0284b658 in the example above) plus global_ete_handle field
offset (which is +0x080) plus the validation field offset (which is +0x0c).
c. With this knowledge we can set a watchpoint for the 4 bytes of the validation field
with this CDB command:
ba w4 0x0284b658 +0x80+0x0c
NOTE: Remember to use your numbers!
30. Click on Continue button. Execution will stop at the point where the statistic is registered in
the current function.
31. Click Continue button again.
32. Observe that the watch point is trapped in seemingly unrelated code in the INIT stat enter
executives
16
33. Open the ethernet_mac_v2_1503_lab2 process model.
34. Open the state variables block
It appears that state variable my_node_name is big enough to carry up to 25 characters. The
length of node name in question (This Workstation Does Some Web Browsing And Maybe
17
Something Else Too) exceeds this limitation and thus name copy in the INIT state corrupts the
memory of global_ete_handle field.
Hypothesis 3 turned out to be true. Let’s fix the problem.
Solution 1: Modify the "my_node_name" to carry up to 255 characters.
Solution 2: Use your implementation. Hint: APIs that check array bounds such as strncpy () or
op_ima_obj_attr_get_str () combined with dynamically allocated string may be more
appropriate.
35. Implement your change, compile ethernet_mac_v2_1503_lab2.
36. Open 1503 project and if simulation debugging session is still running, detach ODB and quit
the simulation.
37. Re-run the simulation (may use Ctrl-Shift-R shortcut) and observe that no errors are
reported.
Postscript
Note the use of conditional breakpoints and watchpoints. Conditional breakpoints were used to stop
at a very convenient point in the source code. This wasn’t the only way to get to that point (you
could use the ODB command prostop), but it is by far the most straightforward and direct way.
Watchpoints are an excellent way to test a common hypothesis: a value becomes invalid or corrupted
between a known valid state and a known invalid state. Much debugging time is often spent trying to
find corruption that is far removed from a reported error state. Using watchpoints saves much of this
time.
End of Lab 2
18
Lab 3: Using Execution Trace to Validate Changes
Overview
This lab introduces the "execution trace": An OPNET simulation kernel feature that allows us to
efficiently find the point where two simulations diverge. We found this functionality to be
particularly useful for validation of model changes and optimizations.
Here is the setup: A creative model developer implemented certain model improvements before
leaving for a long vacation on a remote island. Your job is to see if the changes affect the model
correctness and to resolve any issues if found.
Compare Simulation Results
1. Open 1503 project and switch to lab3_1 scenario. This scenario contains the original
(reference) model. "New and improved" model is implemented in the scenario lab3_2.
2. Run simulations for both scenarios
a. Choose "Scenarios->Manage Scenarios". Project's scenario management dialog
appears.
b. Left-click in the Results column for the lab3_1 scenario row and choose the
<collect> option (or the <recollect> option if results for the scenario already exist).
c. Repeat the procedure for lab3_2 scenario and click OK.
19
d. "DES Execution Manager" window will appear. You can check the progress of
simulations by clicking on the "View Details" button for a particular run.
3. Compare simulation results:
a. Click on the View Results button in the project toolbar.
b. Choose to view results for the Current Project.
c. Select results for both lab3_1 and lab3_2.
20
d. Pick Global Statistics->Ftp->Download Response Time (sec) statistic and set
Presentation style to Overlaid Statistics.
Conclusion: Results from the simulation of the modified model do not match the reference.
Find the Cause of a Difference
4. Collect the execution trace of the reference scenario:
a. Switch to lab3_1 scenario.
b. Open Configure/Run DES dialog and navigate to Execution->Advanced>Application panel.
c. Type -etrace_dump c:\ref_etrace in the Additional command line arguments
field.
21
d. Run simulation to capture the reference execution trace in c:\ref_trace file.
5. Compare execution trace of the modified scenario:
b. Open Configure/Run DES dialog and navigate to Execution->Advanced>Application panel.
c. Type -etrace_diff c:\ref_etrace in the Additional command line arguments field.
NOTE: This time the option is "-etrace_diff", not "-etrace_dump" as we are now
comparing execution pattern against the original, not collecting it.
d. Run simulation and once it quickly completes click Simulation Console button.
e. Scroll down to the bottom and observe report of detected simulation execution
discrepancy:
Observation: Simulation execution pattern was identical until event #12399. Next step is to
capture the detailed trace of activity during this event to see where execution diverges.
6. Capture full trace of event #12399 execution:
22
b. Hit Ctrl-R to open simulation configuration dialog and turn on ODB in Execution>OPNET Debugger panel.
c. Click Run.
d. At ODB prompt type:
•
"evst #12399" and hit Enter. This will set the breakpoint at the event of
interest
•
"c" and hit Enter. This will run the simulation until the above set breakpoint
•
"f" and hit Enter. This will turn on the full trace
•
"n" and hit Enter. This will stop at the next simulation event after producing
the full trace of execution of the event # 12399
e. Choose Simulation->Save Console... menu command and save the console content
in C:\trace1 file.
23
f. Switch to scenario lab3_2 and repeat steps b through e this time saving the console
trace into C:\trace2.
7. Compare captured traces:
a. Choose Run... from the Windows Start menu.
b. Type windiff at the prompt and click OK.
c. Choose File->Compare Files...
d. Select C:\trace1 and click Open. Another file chooser will reappear.
e. Select C:\trace2 and click Open.
f. Click on Expand button.
g. Scroll through the differences ignoring differences in used model names and pointers
until you see a difference in computed service time.
24
Observation: Function gna_clsvr_mgr_service_time_compute() produced different result.
8. Open gna_clsvr_mgr_1503_lab3 process model. This is the process model used by scenario
lab3_2.
9. Open Function Block and search for gna_clsvr_mgr_service_time_compute function
definition.
10. Observe odd-looking asymmetry in service_time computations between lines 2147 and
2155.
25
11. Indeed, you can confirm by looking at the gna_clsvr_mgr_1503_orig process model,
processing_speed is specified in bytes per second, where response_size is specified in bits.
12. To fix the discrepancy, modify line 2147 to divide the response_size by the
processing_speed multiplied by 8.
13. Save and Compile fixed gna_clsvr_mgr_1503_lab3 process model.
14. Rerun lab3_1 and lab3_2 scenarios without ODB and observe that simulation now produce
identical results and execution trace reports no differences.
Postscript
Using Execution Trace coupled with ODB tracing we were able to easily track down a problem in
the modified model code without much knowledge about the nature of the model code modification
or much knowledge about the model code in general.
End of Lab 3
26
Lab 4: Using Advanced Memory Tracking Features
Overview
This lab introduces advanced memory tracking features that can be used to track down memory
leaks.
Run Simulation and Observe Memory Usage
1. Go back to project 1503 and switch to scenario lab4.
2. Open the Configure/Run DES dialog box.
3. Select Execution > OPNET Debugger in the tree view and uncheck the Use OPNET
Simulation Debugger (ODB) checkbox, if currently checked.
4. Run the simulation.
5. Click the Memory Usage tab in the Simulation Sequence dialog box. Notice the rapidly
increasing memory usage.
6. Pause the simulation after the Elapsed time shows 15s.
27
7. Click the Memory Stats tab.
8. Click on the Get Latest Data button.
9. De-select the OPNET Kernel checkbox in the lower-right corner.
10. Click on the Use (KB) column header.
11. Note that packet-related items are at the top of the list.
28
12. Resume the simulation, wait a few more seconds, then click on the Pause and Get Latest
Data button again. Check the Show differences checkbox. The table should now look
something like this:
13. Note that the Memory Stats table helpfully shows you that ams_atm_cell packet related
memory use is on the rise.
Conclusion: The system generates packets, but they are not being destroyed when they should be.
This creates a packet build-up.
Question: Although it seems obvious that ATM cells are the packets that are building up, can we
discover which module—even which function—is responsible for the packet build-up?
Collect More Information about Packet Allocation
1. Stop the simulation by clicking the Stop button. Click Close to close the dialog.
2. Open the Configure/Run DES dialog box.
29
3. Select Runtime Displays > Memory Usage in the tree view.
4. Select the Generate memory source tracing information (Only available with the
sequential development kernel) checkbox.
5. Select the Only for specific memory categories radio button.
6. Click in the top row of the text entry field and type the word Packet (then hit Enter).
7. Click Run.
View Source of Leaked Packets
1. Let the simulation run for about 20-30 seconds, and then hit Pause.
2. Click on the Memory Sources tab in the Simulation Sequence dialog box.
3. Click on the Update button.
30
4. Notice that one function—ams_aal_disp_v3()—seems responsible for most of the packet
build-up.
Conclusion: Packets are being leaked at the ATM layer. Note that most of the packets outstanding
were allocated in the enter execs of the conn irpt state. In other words, it appears that ATM cells
are being created, but perhaps they are not being destroyed when receiving packets that are to be
passed to the AAL layer.
Let’s check our hypothesis by looking at the FTP server, and finding out if packets are being
destroyed before being sent to the AAL layer.
31
Find and Fix the Problem
1. Right click on the top ams_aal_disp_v3 entry and select Show Details.
2. In a little while a table showing all memory blocks that were allocated from the specified
function will appear.
3. This information is sufficient to trace any one of the leaked packet using ODB event and
packet tracing. We will leave this as an optional exercise, but here is the brief outline of the
procedure:
a. Set the breakpoint at leaked packet creation event.
b. Capture full trace of the event and gather IDs of created packets.
c. Trace created packets (pktrace ODB command) to see which module / process
accessed packets last. That process is likely responsible for the leak.
4. Investigation revealed that bulk of packets seemed to be last processed by the "to AAL"
state's enter executives of the ams_atam_layer_v3_1503_lab4 process model.
5. Double-click on one of Voice Stn nodes in the project editor.
6. Double-click on the ATM_layer module.
7. Double-click on the enter execs of the to AAL state. In this state, we expect ATM cells to be
destroyed after AAL packets are decapsulated. The most likely place we can expect a call to
op_pk_destroy() is right before or after an op_pk_send_…() call.
8. Find (Ctrl+F) instances of op_pk_send().
32
9. Note the problem (failure to destroy the ATM cell). Somebody forgot to remove debugging
statements...
10. Remove #if DEBUGGING_SOMETHING and #endif statements.
11. Re-compile the process model.
33
12. Go to the Project Editor and re-run simulation (Ctrl+Shift+R). Note the decrease in memory
usage.
Postscript
Using the new simulation memory tracking features can be very useful in tracking down memory
leaks. However, keep in mind that the more knowledge you have about an OPNET model, the better
chance you have in fixing it. In the end, techniques take a back seat to model knowledge.
End of Lab 4
Summary
Congratulations! You now should be comfortable debugging simulation models in OPNET using a
C/C++ source code debugger. For anyone who develops process models, this is a valuable skill to
have. It will enable you to save time debugging any type of problem that comes your way. The
techniques shown in these labs are sufficient for the vast majority of debugging problems and will
serve you well for any process modeling project you will see in the future.
34

1503 Debugging Simulation Models — Advanced

Transcription

Similar documents

Power devices - Crosslight Software

When Colette Foisy-Doll arrived in Qatar last year

ASE2000 Version 2 - Applied Systems Engineering Inc.

sendai mediatheque - toyo ito

To a brochure about our facility, resources

Simulation Experience

2016_Supplement_Spon..

Development of a Statewide EMS Stroke Triage Protocol

Introduction to OPNET Modeler

HOW TO: Design Mobile Ad Hoc Networks and Protocols January 23, 2007

Lecture Manual