log2timeline

log2timeline
- helping you to create super timelines since 2009 -
Kristinn Guðjónsson
The 2011 Digital Forensics and Incident Response Summit
Austin, TX, 2011
SANS 2011 Digital Forensics and Incident Response Summit
Who am I?
•
•
•
•
•
•
•
M.Sc. in computer and communication network engineering
Worked in forensics and information security since 2005
SANS certifications: GCIA, GCIH, GCFA gold
SANS mentor
Author of log2timeline
Blog author at the SANS forensics blog
Author of the blog: blog.kiddaland.net
SANS 2011 Digital Forensics and Incident Response Summit
Super Timeline?
• List of timestamps with associated data
▫ Extracted from multiple sources
 Filesystem
 Registry (Windows)
 Log files, metadata, …
• Why?
▫ We are trying to tell a story.
▫ Temporal proximity.
▫ Data correlation.
SANS 2011 Digital Forensics and Incident Response Summit
Example Super Timeline
Date
Description
Fri Jan 16 2009 23:15:20
[SetupAPI Log] (Entry written) DriverContext: Reported hardware ID(s) from device parent bus. …
[USBSTOR/DISK&VEN_M-SYS&PROD_DELL_MEMORY_KEY&REV_4.50/086086412140E1C2&0]…
[USBSTOR/DISK&VEN_M-SYS&PROD_DELL_MEMORY_KEY&REV_4.50/086086412140E1C2&0]. Warning:
[STORAGE/RemovableMedia/7&1ad0a3a9&0&RM]…
Fri Jan 16 2009 23:18:10
[Shortcut LNK] (Modified/Access/Created) E:/Blue Harvest Business Plan v1.doc <-./Documents and
Settings/Donald Blake/Recent/Blue Harvest Business Plan v1.lnk- which is stored on a local vol type Removable- SN 0xf434f590 - …
Fri Jan 16 2009 23:18:15
[Shortcut LNK] (Modified/Access/Created) E:/CONFIDENTIAL_SPREADSHEETS.zip <-./Documents and
Settings/Donald Blake/Recent/CONFIDENTIAL_SPREADSHEETS.lnk- …
Fri Jan 16 2009 23:18:19
[Shortcut LNK] (Modified/Access/Created) E:/TIVO Research - CONFIDENTIAL.doc <-./Documents and
Settings/Donald Blake/Recent/TIVO Research - CONFIDENTIAL.lnk…
Fri Jan 16 2009 23:18:19
[Shortcut LNK] (Modified/Access/Created) E:/ <-./Documents and Settings/Donald Blake/Recent/DBlake
Personal (E).lnk…
Fri Jan 16 2009 23:18:26
[Internet Explorer] (index.dat creation time/Last Access) User: Donald Blake URL:file:///E:/Blue Harvest
Business Plan v1.doc (file: ./Documents and Settings/Donald Blake/Local
Settings/History/History.IE5/MSHist012009011220090119/index.dat)
Fri Jan 16 2009 23:18:26
Fri Jan 16 2009 23:18:26
[Internet Explorer] (Last Access) User: Donald Blake URL:file:///E:/Blue Harvest Business Plan v1.doc (file:
./Documents and Settings/Donald Blake/Local Settings/History/History.IE5/index.dat)
/Documents and Settings/Donald Blake/Recent/Blue Harvest Business Plan v1.lnk
SANS 2011 Digital Forensics and Incident Response Summit
Example Super Timeline
SANS 2011 Digital Forensics and Incident Response Summit
Brief History
SANS 2011 Digital Forensics and Incident Response Summit
Brief History
SANS 2011 Digital Forensics and Incident Response Summit
Brief History
SANS 2011 Digital Forensics and Incident Response Summit
Brief History
…and then came version 0.60
aka the killer dwarf release
SANS 2011 Digital Forensics and Incident Response Summit
Version 0.60 - today
• Engine rewritten
▫ Front-end separated
▫ Logic in engine
• More of an object-oriented approach
▫ Input modules inherit parent module
▫ Makes it easier to add modules
• Pre-processing libraries introduced.
• New modules and other enhancements.
SANS 2011 Digital Forensics and Incident Response Summit
Version 0.60
• 43 input modules
• 11 output modules
• 2 pre-processing modules
apache2_
access
apache2_
error
chrome
encase_
dirlisting
evt/evtx
jp_ntfs_ch
ange
exif
ff_
bookmark
firefox2
firefox3
ftk_
dirlisting
generic_
linux
iehistory
iis
isatxt
mactime
mcafee
mft
mssql_
errlog
ntuser
opera
oxml
pcap
pdf
prefetch
recycler
restore
safari
sam
security
setupapi
skype_sql
software
sol
squid
syslog
system
tln
volatility
win_link
wmiprov
xpfirewall
SANS 2011 Digital Forensics and Incident Response Summit
Changes in Structure
• Prior versions
▫
▫
▫
▫
Logic in front-end
Code replicated in different front-ends
Input modules opened files
Each file opened twice
• New structure
▫ Engine separated, logic there
▫ Front-end parses parameters
▫ Engine opens files
SANS 2011 Digital Forensics and Incident Response Summit
How to Create a Front-end?
#!/usr/bin/perl
use Log2Timeline;
# import the library that contains the log2timeline engine
my $l = Log2Timeline->new(
„file‟ => '/mnt/analyze',
# point to the file/directory to parse
‟
„recursive' => 1,
# we want to recursively go through stuf
#'hostname' => '',
# to include a hostname (done in preprocessing)
'input' => 'winxp',
# which input modules to use (this is a Win XP machine)
'output' => 'csv',
# what is the output module to be used
#'offset' => 0,
# the time offset (if the time is wrong) 2996
#'exclusions' => '',
# an exclusion list of one exists
#'text' => '',
# text to prepend to path of files (like c:)
#'append' => 0,
# we are appending to an output file, instead of writing a new one
'time_zone' => 'CST6CDT',
# the time zone of the image
'preprocess' => 1,
# turn on pre-processing modules
) or die( 'unable to start log2timeline');
$l->start;
sub print_line($)
{
my $line = shift;
print $line;
}
SANS 2011 Digital Forensics and Incident Response Summit
Pre-Processing
• Gather information prior to running
▫ Not associated with timestamps
▫ Share information with input modules
• Two simple modules added
▫ Time zone settings and hostname
▫ Default browser, both system and user
SANS 2011 Digital Forensics and Incident Response Summit
Pre-Processing
log2timeline -f winxp -z EST5EDT -m C: -r -p . > /cases/bodyfile
Start processing file/dir [.] ...
Starting to parse using input modules(s): [winxp]
[PreProcessing] The default browser of user smith according to registry is:
(FIREFOX.EXE)
[PreProcessing] Unable to determine the default browser for user default user
[PreProcessing] Unable to determine the default browser for user networkservice
[PreProcessing] Unable to determine the default browser for user localservice
[PreProcessing] Hostname is set to SIMTTO-LAPTOP
[PreProcessing] The timezone according to registry is: (USMST) US Mountain Standard
Time
[PreProcessing] The timezone settings are NOT overwritten so the settings might have to be
adjusted.
[PreProcessing] The default system browser is: : IEXPLORE.EXE ("C:\Program
Files\Internet Explorer\IEXPLORE.EXE" -nohome)
Loading output file: csv
SANS 2011 Digital Forensics and Incident Response Summit
Pre-Processing
date
time
sourcetype user desc
Internet
5/13/11 3:39:57 Explorer
Internet
5/13/11 3:39:57 Explorer
Firefox 3
10/22/09 15:25:52 history
notes
Not the default
URL:file:///C:/Documents%20and%20Settings/smith/My% browser
smith 20Documents/THIS_IS_THE_DOCUMENT.txt
(FIREFOX.EXE)
Not the default
browser
smith URL::Host: My Computer
(FIREFOX.EXE)
Bookmark URL Karadzic plans to boycott trial
(http://news.bbc.co.uk/go/rss/-/2/hi/europe/8319869.stm) Default browser for
smith [8319869.stm] count 0
user
SANS 2011 Digital Forensics and Incident Response Summit
Registry Parsing
• Old userassist changed to ntuser
• Behavior changed
▫ All keys inside a hive parsed
• Includes code from RegRipper
▫ And regtime
• Added modules to parse
▫
▫
▫
▫
SYSTEM
SOFTWARE
SAM
SECURITY
SANS 2011 Digital Forensics and Incident Response Summit
Filesystem Parser - $MFT
• Ported analyzeMFT into log2timeline
▫ Thanks to David Kovar for allowing me to do that
• $STDINFO and $FILENAME timestamps
included
• Simple timestamp manipulation detection
▫ Prone to false positives/negatives
SANS 2011 Digital Forensics and Incident Response Summit
Is There More New Stuff?
• Very simple first version of a Skype parser
▫ Only works on the SQLite database
▫ Grabs basic chat information
• Module to parse the output from jp
▫ Parses the NTFS change log
• Default output is now CSV
• Bug fixes and minor improvements
date
2/12/10
1/18/10
time
sourcetype type
Skype
14:39:47 History
user
desc
Kristinn
Gudjonsson
MSG written to Rob Lee (<user>): this is the chat message…
Chat Sent (<username>) (edited)
Skype
22:35:35 History
Kristinn
Gudjonsson
Chat Sent (<username>) MSG written to Rob Lee (<user>): and I‟m talking some more….
SANS 2011 Digital Forensics and Incident Response Summit
… ohh and one more thing
• Version 0.60 now works on Windows
▫ Instructions on how to install in docs/INSTALL
▫ Thanks to Chris Pogue for creating the install documentation
…but how do we extract those sexy
super timelines?
SANS 2011 Digital Forensics and Incident Response Summit
Extraction Process
• Pretty tedious task
▫ Bunch of commands need to be issued
▫ Possible to write a script to make life easier
• Things can be simplified
▫ Remember the new structure of the front-end?
▫ And the new modules that are available?
SANS 2011 Digital Forensics and Incident Response Summit
The old method
timescanner –z ZONE –d MNTPOINT –w BODYFILE
fls –r –m C: IMAGE >> BODYFILE
regtime.pl –m HKLM-SYSTEM –r
MNTPOINT/WINDOWS/System32/config/system >> BODYFILE
regtime.pl –m HKLM-SAM –r
MNTPOINT/WINDOWS/System32/config/SAM>> BODYFILE
regtime.pl –m HKLM-SECURITY–r
MNTPOINT/WINDOWS/System32/config/SECURITY >> BODYFILE
regtime.pl –m HKLM-SOFTWARE–r
MNTPOINT/WINDOWS/System32/config/software >> BODYFILE
mactime –d –b BODYYFILE –z ZONE DATE_RANGE > CSVFILE
SANS 2011 Digital Forensics and Incident Response Summit
The new (although manual)
• ntfs-3g does not show the $MFT file
▫ Need to extract the $MFT
icat myimage.dd 0 > myimage.mft
log2timeline –f mft –z EST5EDT –m C: -w
/cases/bodyfile.txt
log2timeline –f winxp –z EST5EDT –m C: -r –p
/mnt/windows_mount –w /cases/bodyfile.txt
l2t_process –b /cases/bodyfile.txt 01-15-2010..01-25-2010
> /cases/timeline.txt
SANS 2011 Digital Forensics and Incident Response Summit
The new (automated SIFT)
• Simple frontend created: log2timeline-sift
▫ Included in the extra folder
• Can be installed easily
apt-get install log2timeline-sift-perl
• Options:
▫
▫
▫
▫
▫
-i IMAGE_FILE
-c CONF (default /etc/log2timeline/sift.conf)
-z ZONE
-w (is a Windows 7)
-p NR
SANS 2011 Digital Forensics and Incident Response Summit
log2timeline-sift
• To extract the super timeline using the script
▫ Creates a folder called /cases/timeline
• Partition image (not a whole disk image)
log2timeline-sift –z EST5EDT –p 0 xp_dblake.dd
• Disk image:
log2timeline-sift –z EST5EDT disk_image.dd
SANS 2011 Digital Forensics and Incident Response Summit
log2timeline-sift
• Sample run
log2timeline-sift.pl -z EST5EDT -i /images/xp_dblake.dd -p 0
Image file (/images/xp_dblake.dd) has not been mounted. Do you want me to mount it for
you? [y|n]: y
This is a partition image, let's attempt mounting it directly.
Image file mounted successfully as /mnt/windows_mount
Loading output file: csv
[PreProcessing] Unable to determine the default browser for user donald blake
[PreProcessing] Unable to determine the default browser for user default user
[PreProcessing] Unable to determine the default browser for user networkservice
[PreProcessing] Unable to determine the default browser for user localservice
[PreProcessing] Hostname is set to ASGARD
[PreProcessing] The timezone according to registry is: (EST) Eastern Standard Time
[PreProcessing] The timezone settings are NOT overwritten so the settings might have to be
adjusted.
[PreProcessing] The default system browser is: : IEXPLORE.EXE ("C:\Program Files\Internet
Explorer\iexplore.exe" -nohome)
Loading output file: csv
and then what?
SANS 2011 Digital Forensics and Incident Response Summit
Life After Collection
• Normal super timeline contains LOT of data
▫ Finally we have something to spend time on
• Necessary to reduce the dataset
• How?
▫ Read at the speed of light
▫ Use mactime output and the script mactime
▫ Load everything into Excel and pray
▫ Use databases or Splunk
▫ The good ol‟ grep method
grep “^05\/1[2-9]\/2011” timeline.txt
SANS 2011 Digital Forensics and Incident Response Summit
Is There a Life After Collection?
• Isn‟t it possible to create a tool to assist?
▫ Well yes there is…
• l2t_process added to meet this demand
▫ Included with log2timeline
▫ Works in a similar fashion as mactime
▫ Parses the CSV and TAB format of log2timeline
SANS 2011 Digital Forensics and Incident Response Summit
l2t_process
• Usage
l2t_process –b BODYFILE [-w white] [-k dirty] [DATE_RANGE]
• What does it do you ask?
▫
▫
▫
▫
▫
▫
Sort entries based on time
Filter based on date range
Removes duplicate entries
Compare entries to a keyword or whitelist file
Warn if it detects “suspicious” MFT entries
Create scatter plots
SANS 2011 Digital Forensics and Incident Response Summit
l2t_process - keyword
$cat keyfile
this_is_the
$l2t_process –b timeline.txt -k keyfile > time_key.txt
Building keyword list...DONE (1 keywords loaded)
Total number of events that fit into the filter (got printed) = 16
Total number of duplicate entries removed = 3
Total number of events skipped due to keyword filtering = 1281973
Total number of processed entries = 1281989
Run time of the tool: 36 sec
cat time_key.txt
date,time,timezone,MACB,source,sourcetype,type,user,host,short,desc,version,filename
,inode,notes,format,extra
04/20/2011,08:06:32,EST5EDT,...B,FILE,NTFS $MFT,$SI [...B] time,-,-,c:/Documents
and Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,{SUSP ENTRY
- timestomp? - second prec. $SI [MACB] FN rec AFTER SI rec} c:/Documents and
Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,2,c:/Documents
and Settings/smith/My Documents/THIS_IS_THE_DOCUMENT.txt,18113,,Log2t::input::mft,…
SANS 2011 Digital Forensics and Incident Response Summit
Timestamp Manipulation
• Done through the Windows API
▫ ZwSetInformationFile
▫ NtSetInformationFile
▫ Allows setting the whole 64 bits
▫ Many tools only use second precision
▫ Timestomp from Metasploit one of those:
/* it doesnt matter what the millisecond value is because the ntfs resolution for file timestamps is only
up to 1s */
systemtime->wMilliseconds = 0;
• The API only changes the $STDINFO timestamp
▫ The $FILENAME is untouched
SANS 2011 Digital Forensics and Incident Response Summit
How Do We Then Detect Those Manipulations?
• Two methods
▫ Detect timestamps that have ms equal to zero
▫ Detect timestamps where $FN occurs later than
$SI
• Problems with this approach
▫ Not all files with zero ms. time are “bad”
▫ $FN timestamps are updated when files are
copied or moved
• Pretty easy to fool
▫ Use methods that set the ms. to a random value
SANS 2011 Digital Forensics and Incident Response Summit
Other methods
• Sequential MFT entry number allocation
• Malware often hides inside Windows\System32
▫ Patches update several files
▫ Malware introduces few changes
▫ “Hide in plain sight”
• What l2t_process does to detect manipulations
▫ $MFT module includes notes if entries are suspicious
▫ The –i (include) option includes suspicious entries
outside the date range
▫ Maps the relationship between MFT entry nr. and
creation time
Scatter Plots
[2139] /WINDOWS/system32/evil.exe [{SUSP ENTRY - second prec. $SI [M...] FN rec AFTER SI rec} ]
SANS 2011 Digital Forensics and Incident Response Summit
Summary
• log2timline has been evolving since 2009
▫ And keeps doing that
▫ Developed on my own time
 Donations and feedback run tool development
• Version 0.60 allows complete super timeline creation
▫ And runs on most platforms
▫ Easy to integrate into other scripts
▫ l2t_process assists with data reduction