cPanel Training `10 - howtofixservers.com

cPanel Technical Training
Kevin Asklund
Technical Trainer / Advanced Support
[email protected]
Additional Training assistance from:
Dave Lanning - Level 3 Tech Analyst - Migration Specialist
Wednesday, December 9, 2009
It’s easier to support what you know so one of my goals today is to
train you in advanced practices of supporting cPanel and WHM.
Integration - xml-api
Strace / Debugging
Bash / Shell w/ Grep, Awk & SED
CPAN / Perl
Tomcat
Ruby
DNS
MySQL Optimization
cPanel Training ’10
Wednesday, December 9, 2009
cPanel Training
Strace / Debugging.
cPanel Training ’10
Wednesday, December 9, 2009
What is Strace?
Strace - trace system calls and signals
Strace Is not a debugger
Install strace using: /scripts/ensurerpm strace
cPanel Training ’10
Wednesday, December 9, 2009
What are system calls and signals?
System calls: (see man syscalls)
Signals: (see man errno.h)
read
write
open
close
stat
fork
connect
getuid
getgid
setuid
setgid
execve
chmod
chown
SIGINT (ex. ctrl-c)!
SIGKILL (kill -9)!
ENOENT
EPERM
ENOPERM
EACCESS if -1 (permission denied)
SIGSEGV (Segmentation fault)
cPanel Training ’10
Wednesday, December 9, 2009
Example script to run for errors
#!/usr/bin/perl
#this script reads in /etc/passwd and writes out a copy where it adds a
bunch of useless stuff to it
open my $passwd_fh , "/etc/passwd";
open my $bizarro_fh , ">bizarro.txt";
foreach my $line ( readline $passwd_fh ) {
$line =~ s/^(\w+)/(BIZARRO $1)/;
5 + 7;
print $bizarro_fh $line;
#THIS COMMENT WILL STAND OUT IN THE STRACE WHEN IT
READS IN THE PERL SCRIPT
}
cPanel Training ’10
Wednesday, December 9, 2009
Now lets test it!
strace -f -v -s 4096 /home/stracer/public_html/test.pl
-f = follow forks
-v = verbose
-s = string size - gives us up to 4096 characters
cPanel Training ’10
Wednesday, December 9, 2009
Now you will see the following:
Execution of the script:
execve("./test.pl", ["./test.pl"], ["SHELL=/bin/bash",
Reading the script into memory:
read(3, "#!/usr/bin/perl\n#
Opening our files:
open("/etc/passwd", O_RDONLY|O_LARGEFILE) = 3
<bunch of stuff>
open("bizarro.txt", O_WRONLY|O_CREAT|O_TRUNC|
O_LARGEFILE, 0666) = 4
<bunch of stuff>
cPanel Training ’10
Wednesday, December 9, 2009
(CONT’D)
Reading /etc/passwd:
read(3,"root:x:0:0:root:/root:/bin/bash\ndaemon:
Memory Allocation:
brk(0x1b333000)
= 0x1b333000
brk(0x1b333800)
= 0x1b333800
brk(0x1b334000)
= 0x1b334000
Writing our bizarro.txt file:
write(4,"(BIZARRO root):x:0:0:root:/root:/bin/bash\n
Strace will not show:
Any program logic
Computation
Strace is not a debugger
cPanel Training ’10
Wednesday, December 9, 2009
Procedure for stracing cpanel
Capture as little as necessary:
1. Set up your problematic action
2. Start your strace command
3. Execute action to reproduce error
4. Ctrl-c the strace
strace -v -f -s 4096 -o /root/strace.cpsrvd -p `cat /var/run/cpsrvd.pid`
-o = log output to file specified
-v = verbose
-f = follow forks
-s = String size
-p = Process ID to strace
cPanel Training ’10
Wednesday, December 9, 2009
Once you get the strace output, just view the file.
1. Start with the error message!
Search thru the log file for the error messages displayed
32116 write(1, “There was a problem creating the”, 32)=32
Then start scrolling up for the last action performed before the error was written to stdout
32116 open("/home/scooby/etc/snoopy.com/passwd", O_WRONLY|O_CREAT|O_APPEND|
O_LARGEFILE, 0666) = -1 EACCES (Permission denied)
You can also specify what system calls to strace:
“strace -e trace=file” will trace only file accesses, or
“strace -e trace=network” will trace only network activity.
cPanel Training ’10
Wednesday, December 9, 2009
Errors have “-1” after them, so you use egrep to find just these Errors:
Egrep '= -1' <strace output filename>
strace -e
egrep '= -1' trace.file <- to show affected calls
write( = sometimes useful to search for 'write(' command
stat = gets info about the file and writes it to buffer
fstat = gets info about the open file associated with the file descriptor and writes it to buffer.
read = read file
open = open file
mmap = adds to memory
munmap = removes (unmaps region) from memory
[EACCES] Search permission denied for a component of the path
[ENOENT] File or path does not exist (No such file or directory)
[SIGSEGV] = Segmentation Fault
cPanel Training ’10
Wednesday, December 9, 2009
FreeBSD
Strace—ported to BSD
Ktrace—most effective in our experience
Truss—very similar to strace in syntax
strace can be installed thru ports and can sometimes be helpful but ktrace is preferred on
FreeBSD systems.
cd /usr/ports/devel/strace && make install clean
ktrace -i -f /path/to/file.log -p PID#
-C = stops tracing all processes
-c = stops tracing indicated process
-i = similar to follow forks
-f = file to log to - passes trace to all future children of the process
-p = process ID
cPanel Training ’10
Wednesday, December 9, 2009
FreeBSD CONT’D
Truss -o /root/truss.log -f -s 4096 -p PID#
Truss -o /root/truss.log -f -s 4096 command-to-trace
-f = follow forks
-o = log output to file specified
-s = string size
-p = process ID
cPanel Training ’10
Wednesday, December 9, 2009
Tracing Randomly Dying Processes:
strace -vFf -s 4096 -e trace=signal $(ps -C exim h | awk '{ print “-p “ $1 }')
strace -vFf -s 4096 -o /root/strace.cpsrvd -p `cat /var/run/cpsrvd.pid`
strace -vFf -s 4096 -o /root/strace.httpd $(ps -C httpd h | awk ' { print “-p “ $1 }')
strace -f -v -s 4096 -p PID#
strace -f -v -s 4096 command
Introduction to bash debugging:
bash -x
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html
cPanel Training ’10
Wednesday, December 9, 2009
This command sits in a loop waiting for a process from that user to spawn.
$ x=1; while [ $x = 1 ]; do process=`pgrep -u username`; if [ $process ]; then x=0; fi; done;
strace -vvtf -s 256 -p $process
cPanel Training ’10
Wednesday, December 9, 2009
Questions?
cPanel Training ’10
Wednesday, December 9, 2009