CRAXweb: Automatic web application testing and attack generation
2 likes1,110 views
This paper proposes to test web applications and generate the feasible exploits directly and automatically, including cross-site scripting and SQL injection attacks. Our target is to generate the attack string and reproduce the results, emulating the manual attack behavior. In contrast with other traditional detection and prevention methods, we can certainly determine the presence of vulnerabilities and prove the feasibility of attacks. This automatic generation process is mainly based on a dynamic software testing method-symbolic execution by S2E. We have applied this automatic process to several known vulnerabilities on large-scale open source web applications, and generated the attack strings successfully. Our method is web platform independent, covering PHP, JSP, Rails, and Django.
![Constraint Solving
? What¡¯s the XSS exploit of the given sample
code?
Sample code
1 <?php
2
$input = $_GET['id'];
3
for($i=0; $i<strlen($input); $i++)
4
echo chr(ord($input[$i])+1);
5
?>
16](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-16-320.jpg)
![Constraint Solving
? What¡¯s the XSS exploit of the given sample
code?
¨C Symbolic request & response
HTTP Request
Unknown input (XSS attack)
GET /index.php?id=[ input ] HTTP/1.1
Host: example.com
HTTP Response
Known output (an alert script)
HTTP/1.1 200 OK
Context-type: text/html
Sample code
<html>
some text [ output ]
</html>
1 <?php
2
$input = $_GET['id'];
3
for($i=0; $i<strlen($input); $i++)
4
echo chr(ord($input[$i])+1);
5
?>
17](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-17-320.jpg)
![Constraint Solving
? What¡¯s the XSS exploit of the given sample
code?
¨C Symbolic request & response
¨C Add JavaScript code as target character
? output = <script>alert(document.cookie)</script>
HTTP Response
;rbqhos=¡
HTTP Request
GET /index.php?id=[ input ] HTTP/1.1
Host: example.com
Sample code
<script>¡
1 <?php
2
$input = $_GET['id'];
3
for($i=0; $i<strlen($input); $i++)
<html>
some text [ output ]
4
echo chr(ord($input[$i])+1);
</html>
5
?>
HTTP/1.1 200 OK
Context-type: text/html
18](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-18-320.jpg)
![Constraint Solving
? What¡¯s the XSS exploit of given sample code?
¨C Symbolic request & response
¨C Add JavaScript code as target character
? output = <script>alert(document.cookie)</script>
¨C Solve the constraint
? input = ;rbqhos=`kds¡¯cnbtldms-bnnjhd(;,rbqhos=
HTTP Response
;rbqhos=¡
HTTP Request
GET /index.php?id=[ input ] HTTP/1.1
Host: example.com
Sample code
<script>¡
1 <?php
2
$input = $_GET['id'];
3
for($i=0; $i<strlen($input); $i++)
<html>
some text [ output ]
4
echo chr(ord($input[$i])+1);
</html>
5
?>
HTTP/1.1 200 OK
Context-type: text/html
19](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-19-320.jpg)
![Path Constraints
Input
Path constraint
Target output
Solved output
input[0]
chr(input[0]+1)
<
;
input[1]
chr(input[1]+1)
s
r
input[2]
chr(input[2]+1)
c
b
input[3]
chr(input[3]+1)
r
q
input[4]
chr(input[4]+1)
i
h
input[5]
chr(input[5]+1)
p
o
input[6]
chr(input[6]+1)
t
s
input[7]
chr(input[7]+1)
>
=
input[8]
chr(input[8]+1)
a
`
input[9]
chr(input[9]+1)
l
k
¡
¡
¡
¡
20](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-20-320.jpg)
![Exploit Generation of Single URL
? This method can check security risk of a single
URL
HTTP Response
HTTP/1.1 200 OK
Context-type: text/html
<script>alert(document.cookie)</script>
<html>
some text [ output ]
</html>
mysql_query
admin or 1=1--
SELECT * FROM user
WHERE user=[symbolic]
21](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-21-320.jpg)
![Single Path Concolic Execution
? In order to reduce the overhead on symbolic
execution
HTTP Request
HTTP Request
GET index.php?abc=[
Host: 123.123.123.123
] HTTP/1.1
Symbolic execution:
Explore all possible paths
GET index.php?abc=[AAAAA] HTTP/1.1
Host: 123.123.123.123
Single path concolic execution:
Only explore the path of the given
input
23](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-23-320.jpg)
![Exploit Generator
SELECT * FROM user WHERE user=[symbolic] ¡¡
symbolic
Sample code
1 <?php
2
$input = base64_decode($_GET[¡®user']);
3
mysql_query(¡°SELECT * FROM user
4 WHERE user=¡±. $input);
5 ?>
... x.php?user=YWRtaW4gb3Ig...
41](https://image.slidesharecdn.com/craxwebsere2013-140211003600-phpapp01/85/CRAXweb-Automatic-web-application-testing-and-attack-generation-41-320.jpg)
![Padding oracle [opkoko2011]](https://cdn.slidesharecdn.com/ss_thumbnails/paddingoraclepublic-130806071305-phpapp01-thumbnail.jpg?width=560&fit=bounds)
![[Èô¿ÊÓ‹®‹] Challenges and Solutions of Window Remote Shellcode](https://cdn.slidesharecdn.com/ss_thumbnails/challengesandsolutionsofwindowremoteshellcode-171118141605-thumbnail.jpg?width=560&fit=bounds)
More Related Content
Viewers also liked (6)
Similar to CRAXweb: Automatic web application testing and attack generation (20)
Recently uploaded (20)
CRAXweb: Automatic web application testing and attack generation
- 1. CRAXweb: Web Testing and Attacks through ¡°QEMU¡± in S2E Shih-Kun Huang National Chiao Tung University Hsinchu, Taiwan skhuang@cs.nctu.edu.tw
- 2. Motivation ? Symbolic Execution is effective to crash applications ¨C Catchconv, Bitfuzz, Taintscope, and Ardilla (PHP) ¨C Should be effective for Web Testing ? Symbolic Execution can also automate exploit generation process ¨C AEG, MAYHEM, CRAX ¨C Should be feasible to automate Web Attack (exploit) generation
- 3. How Effective of Automatic Exploit Generation for non-web applications ? Mplayer (1.5MLOC) (CVE-2008-0630) ¨C MPlayer 1.0rc2 and SVN before r25823 ¨C 3.6 seconds ? Microsoft Office Word (CVE-2012-0158) ¨C Microsoft Office < 2010 ¨C 216 seconds ? Nginx (CVE-2013-2028) ¨C nginx 1.3.9/1.4.0 stack buffer overflow ¨C 8 seconds
- 4. Problems of Symbolic Web Testing and Attacks ? Hard to Implement Symbolic Execution Platform for Web ¨C MIT¡¯s Ardilla not in public and only for PHP ¨C Various number of Web platforms: PHP, JSP, Python, Perl, Ruby, ASP ? Variety of Attack Methods ¨C Non-web attacks: stack, heap, format, integer, uninitialized uses, race,¡ ¨C OWASP top attacks: injection, XSS, CSRF,¡
- 5. Web Platform Independent Testing ? (PHP,JSP,ASP,NodeJS,Python,Ruby,¡) symbolic execution engine ? ¨C QEMU¨Cbased symbolic execution engine -> S2E ? Issues ¨C Performance should be the primary consideration ¨C Will symbolic semantics be preserved ? Across between Web semantics and llvm semantics.
- 6. Attack Independent Exploit Generation ? Taint Analysis ¨C Input tainted operations ? Symbolic Continuations (what to do next ?) ¨C Symbolic program counter (Symbolic EIP) ? Where the EIP points to ¨C Symbolic SQL query ? Where the SQL commands run ¨C Symbolic HTML response ? Where the Javascript executes ¨C Symbolic command argument ? Where the shell commands run
- 7. The power of Symbolic Computation ? Symbolic Execution ¨C Generating Testing input, following all feasible branches ? Concolic Execution ¨C Generating Testing input, following a concrete input path and the associated branches ? Exploit Generation ¨C Generating Exploit input, following a concrete Crash/Anomaly input path and branch to the associated ¡°shell code¡± ¨C Path Constraint generated by the crash input ¨C Constraints of Symbolic ¡°continuations¡± branching to the shell code
- 8. Symbolic Execution ? Explore every possible path of a program ¨C Record path information in path constraint Path constraint 1 Symbolic input A program Path constraint 2 Path constraint 3 2014/2/11 Liu Huan „¢šg A Generic Web Testing and Attack Generation Framework 8
- 9. Concolic Execution ? Begin with a random input ? Use false path constraint to generate another input case Output1 Input 1 Path constraint 1 Input 2 A program Output2 Path constraint 2 Output3 Input 3 Path constraint 3 ¡¡ 2014/2/11 Liu Huan „¢šg A Generic Web Testing and Attack Generation Framework ¡¡ 9
- 10. Exploit Generation ? Record the path constraint of the given crash input Crash Input: x A program Output: y Path constraint 2014/2/11 Liu Huan „¢šg A Generic Web Testing and Attack Generation Framework 10
- 11. Constraint Solving Unknown input: x A program Output: y Path constraint ? Given program output y, constraint solving is the way to generate input x Output: y + Solve constraint Value of input x Path constraint 11
- 12. Constraint Solving ? If f(x) = 100, what¡¯s the value of x? Known output =100 Unknown input: x Sample code 1 int f(x){ 2 int y=x+10; 3 if (y >0) 4 return y; 5 else 6 return y; 7 } 12
- 13. Constraint Solving ? If f(x) = 100, what¡¯s the value of x? ¨C Use symbolic execution to get path constraint Sample code PC of path 1 Path constraint PC of path 2 X+10 > 0 X+10 <= 0 1 int f(x){ 2 int y=x+10; 3 if (y >0) 4 return y; 5 else 6 return y; 7 } 13
- 14. Constraint Solving ? If f(x) = 100, what¡¯s the value of x? ¨C Use symbolic execution to get path constraint ¨C ¡ß f(x) = y = X+10 = 100 Known output =100 ¡à Add path constraint X + 10 = 100 Sample code PC of path 1 PC of path 2 Path constraint X+10 > 0 X+10 <= 0 Add constraint from known information X+10 = 100 X + 10 = 100 1 int f(x){ 2 int y=x+10; 3 if (y >0) 4 return y; 5 else 6 return y; 7 } 14
- 15. Constraint Solving ? If f(x) = 100, what¡¯s the value of x? ¨C Use symbolic execution to get path constraint ¨C ¡ß f(x) = y = X+10 = 100 Known output =100 ¡à Add path constraint X + 10 = 100 ¨C Solve the constraint ? x = 90 Sample code input: x=90 PC of path 1 PC of path 2 Path constraint X+10 > 0 X+10 <= 0 Add constraint from known information X+10 = 100 X + 10 = 100 Constraint solving X = 90 No solution 1 int f(x){ 2 int y=x+10; 3 if (y >0) 4 return y; 5 else 6 return y; 7 } 15
- 16. Constraint Solving ? What¡¯s the XSS exploit of the given sample code? Sample code 1 <?php 2 $input = $_GET['id']; 3 for($i=0; $i<strlen($input); $i++) 4 echo chr(ord($input[$i])+1); 5 ?> 16
- 17. Constraint Solving ? What¡¯s the XSS exploit of the given sample code? ¨C Symbolic request & response HTTP Request Unknown input (XSS attack) GET /index.php?id=[ input ] HTTP/1.1 Host: example.com HTTP Response Known output (an alert script) HTTP/1.1 200 OK Context-type: text/html Sample code <html> some text [ output ] </html> 1 <?php 2 $input = $_GET['id']; 3 for($i=0; $i<strlen($input); $i++) 4 echo chr(ord($input[$i])+1); 5 ?> 17
- 18. Constraint Solving ? What¡¯s the XSS exploit of the given sample code? ¨C Symbolic request & response ¨C Add JavaScript code as target character ? output = <script>alert(document.cookie)</script> HTTP Response ;rbqhos=¡ HTTP Request GET /index.php?id=[ input ] HTTP/1.1 Host: example.com Sample code <script>¡ 1 <?php 2 $input = $_GET['id']; 3 for($i=0; $i<strlen($input); $i++) <html> some text [ output ] 4 echo chr(ord($input[$i])+1); </html> 5 ?> HTTP/1.1 200 OK Context-type: text/html 18
- 19. Constraint Solving ? What¡¯s the XSS exploit of given sample code? ¨C Symbolic request & response ¨C Add JavaScript code as target character ? output = <script>alert(document.cookie)</script> ¨C Solve the constraint ? input = ;rbqhos=`kds¡¯cnbtldms-bnnjhd(;,rbqhos= HTTP Response ;rbqhos=¡ HTTP Request GET /index.php?id=[ input ] HTTP/1.1 Host: example.com Sample code <script>¡ 1 <?php 2 $input = $_GET['id']; 3 for($i=0; $i<strlen($input); $i++) <html> some text [ output ] 4 echo chr(ord($input[$i])+1); </html> 5 ?> HTTP/1.1 200 OK Context-type: text/html 19
- 20. Path Constraints Input Path constraint Target output Solved output input[0] chr(input[0]+1) < ; input[1] chr(input[1]+1) s r input[2] chr(input[2]+1) c b input[3] chr(input[3]+1) r q input[4] chr(input[4]+1) i h input[5] chr(input[5]+1) p o input[6] chr(input[6]+1) t s input[7] chr(input[7]+1) > = input[8] chr(input[8]+1) a ` input[9] chr(input[9]+1) l k ¡ ¡ ¡ ¡ 20
- 21. Exploit Generation of Single URL ? This method can check security risk of a single URL HTTP Response HTTP/1.1 200 OK Context-type: text/html <script>alert(document.cookie)</script> <html> some text [ output ] </html> mysql_query admin or 1=1-- SELECT * FROM user WHERE user=[symbolic] 21
- 22. Exploit Generation ? Generate exploit of a web application 22
- 23. Single Path Concolic Execution ? In order to reduce the overhead on symbolic execution HTTP Request HTTP Request GET index.php?abc=[ Host: 123.123.123.123 ] HTTP/1.1 Symbolic execution: Explore all possible paths GET index.php?abc=[AAAAA] HTTP/1.1 Host: 123.123.123.123 Single path concolic execution: Only explore the path of the given input 23
- 24. Restriction 24
- 25. Outline ? Introduction ? Background ? Method ¨C Exploit Generation ¨C System Architecture ? Related Work ? Evaluation ? Conclusion and Future Work 25
- 26. System Architecture ? ? ? ? Symbolic Environment on S2E CRAXWeb Architecture CRAX Framework Detail of CRAXWeb ¨C Web Crawler ¨C Symbolic Request Sender ¨C Symbolic Data Sensor ¨C Exploit Generator 26
- 27. S2E (Selective Symbolic Execution) Symbolic data sender Exploit generator 27
- 28. S2E (Selective Symbolic Execution) Symbolic data sender For XSS attack Symbolic data sensor Exploit generator 28
- 29. S2E (Selective Symbolic Execution) Symbolic data sensor Symbolic data sender For SQL injection attack Exploit generator 29
- 30. CRAXWeb Architecture Test unit S2E QEMU (server) Web application Symbolic data sensor s2e_myop Sym. socket Web crawler Symbolic Sym. request Socket Web Server sender Expolit generator Report Sym. Socket Symbolic data sensor s2e_myop STP Solver (client) 30
- 32. Web Crawler Test unit S2E QEMU (server) Web application Symbolic data sensor s2e_myop Sym. socket Web Web crawler crawler Symbolic Sym. request Socket Web Server sender Expolit generator Report Sym. Socket Symbolic data sensor s2e_myop STP Solver (client) 32
- 33. Web Crawler (Burp Suite) Web application GET index.php?abc=xxxxx HTTP/1.1 Host: example.com Web crawler Database POST index.php HTTP/1.1 Host: example.com Content-length: 40 a=xxxx&b=xxx 33
- 34. Symbolic Request Sender Test unit S2E QEMU (server) Web application Symbolic data sensor s2e_myop Sym. socket Web crawler Symbolic Symbolic request request sender sender Expolit generator Sym. Web Server Socket Report Sym. Socket Symbolic data sensor s2e_myop STP Solver (client) 34
- 35. Symbolic Data Sender Web crawler Database 1. Experiment request Control node Symbolic data sender 2. Experiment response Web application 2014/2/11 35
- 36. Symbolic Data Sensor Test unit S2E QEMU (server) Web application Symbolic Symbolic data data sensor sensor s2e_myop Sym. socket Web crawler Symbolic Sym. request Socket Web Server sender Expolit generator Report Sym. Socket Symbolic Symbolic data data sensor sensor s2e_myop STP Solver (client) 36
- 37. Symbolic Data Sensor Sensitive data Symbolic data sensor Exploit generator If it is a symbolic data, The sensor can call exploit generator Web security issues XSS SQL injection ¡ 2014/2/11 Sensor location HTTP Response mysql_query() ¡ 37
- 38. Other Web Security issues Sensor location PHP Python Remote file Inclusion Directory traversal Command injection Code Injection File upload include(), include_once() ¡ fopen(), file() ¡ include(), require()¡ open()¡ system(), file()¡ system(), exec()¡ eval()¡ move_uploaded_file(), rename(), ¡ eval()¡ open()¡ 38
- 39. Exploit Generator Test unit S2E QEMU (server) Web application Symbolic data sensor s2e_myop Sym. socket Web crawler Symbolic Sym. request Socket Web Server sender Exploit Expolit generator generator Report Sym. Socket Symbolic data sensor s2e_myop STP Solver (client) 39
- 41. Exploit Generator SELECT * FROM user WHERE user=[symbolic] ¡¡ symbolic Sample code 1 <?php 2 $input = base64_decode($_GET[¡®user']); 3 mysql_query(¡°SELECT * FROM user 4 WHERE user=¡±. $input); 5 ?> ... x.php?user=YWRtaW4gb3Ig... 41
- 42. Outline ? ? ? ? ? ? Introduction Background Method Related Work Evaluation Conclusion and Future Work 42
- 45. Experiment Monitor CRAX Web Guest QEMU 45
- 49. Evaluation for Web platform independence Test case ~= echo(¡°A¡±x50) OT >= 12hr PHP JSP Rails Django ASP Framework - - 3.2 0.96.1 - OS Linux Linux Linux Linux Windows Server Apache-2.2.19 Tomcat-7.0.2 Webrick Built-in IIS-5.1 Kernel PHP-5.3.6 JDK-7u2 Ruby-1.9.3 Python-2.6.6 ASP-3.0 Bind Port 80 8080 3000 8000 80 Symbolic response time 18.50s 6.72min 7.45min 32.72s OT Without constraints 16.42s 3.25min 5.62min 24.02s OT 49
- 50. Evaluation for XSS OT >= 15min Test Case Line Of Code # of crawled request # of XSS # of XSS (vulnerable) by MIT Time per exploit Time for all crawled request Schoolmate-1.5.4 8,125 452 19 14 0.30min 107.78min + 30OT Webchess-1.0.0rc2 6,504 410 5(4) 13 0.80min 94.38min + 313OT Faqforge-1.3.2 1,710 28 4 4 0.20min 5.74 min EVE 904 12 2 2 0.42min 4.94min Test Case Line Of Code Platform # of crawled request # of XSS (vulnerabl e) Time per exploit Time for all crawled request SimpGB-1.49.02 41,296 PHP 1,299 33(57) 0.91min 7.67hr + 334OT DedeCms-5.6 84,544 PHP 1,111 11(13) 0.48min 8.32hr + 9OT Django-admin-0.96.1 3,558 Python 5 1 5.29min 5.29min + 4OT Discuz!-6.0 67,088 PHP 613 0(1) 0.85min 8.37hr + 12OT Joomla-1.6 253,711 PHP 215 0(7) 2.17min 1.26hr + 117OT 50
- 51. Evaluation for SQL injection Test Case Schoolmate Webchess Faqforge 1.54 1.0.0rc2 EVE 1.3.2 Testlink phpreci- 1.8.4 piebook 2.24 Line of code 8125 6504 1710 904 144913 52631 CVE - - - - 2009- 2009- 4238 4883 # of crawled request 269 65 7 9 218 65 # of SQLi (vulnerable) 12 6 3 3 9 6 # of SQLi by MIT 6 12 1 2 - - Time per exploit 0.55 min 0.39 min 0.27 min 0.24 3.24min 4.89min 2.12 706.4min 315.2min min (30 TO) (32 TO) 934 18047 6322 min Time for all crawled 148.58 min 25.15 min 1.88min requests # of all solved constraints 952 15254 1104 TO: Timeout 51
- 52. Outline ? ? ? ? ? ? Introduction Background Method Related Work Evaluation Conclusion and Future Work 52
- 53. Automatic Web Attack Generator ? Based on symbolic execution ¨C White box ¨C Only support specific language ? Based on reply value of server ¨C Black box ¨C Hard to handle encrypted data 53
- 54. Related Work Approach year Attacks/ Detectd Generation Algorithm W/B Box WB Plateform SAFELI 2008 SQLI Attack Statically inspect bytecode of application Apollo 2008 WB PHP WB PHP 2010 Malformed HTML Use Concolic execution to find bugs in PHP Detect web applications XSS, SQLI Attack It combines concrete and symbolic execution to covers paths XSS, SQLI Attack Attack gramma and symbolic execution Adrilla 2009 Kudzu WB JavaScript PIUIVT 2010 XSS, SQLI Attack Perturbation based Algorithm WB Java MySQLInject or NKSI Scan 2011 SQLIJ Attack BB PHP 2012 SQLIJ Attack BB JSP, ASP CRAX Web 2012 XSS, SQLI Attack Blind SQL Injection based on True/False, Order by Modulize SQL Injection patten to generate attack string Single path symbolic execution WB XSS: All, SQLI: PHP JAVA 54
- 55. Related Work Approach Year Attacks / Detectd SAFELI Apollo 2008 2008 Adrilla 2009 SQLI Attack Malformed HTML Detect XSS, SQLI Attack Kudzu PIUIVT MySQLInjector NKSI Scan 2010 2010 2011 2012 CRAX Web 2012 W / B Plateform Box W JAVA W PHP W PHP XSS, SQLI Attack XSS, SQLI Attack SQLI Attack SQLI Attack W W B B JavaScript JAVA PHP JSP, ASP XSS, SQLI Attack W XSS: All, SQLI: PHP 55
- 56. Conclusion ? A framework to generate exploit of web application ¨C Support XSS and SQL injection Web application CRAX Web Vulnerability Report ? A successful trial of Symbolic Execution for Web by S2E 56
- 57. Future Work ? Implement this structure on other kind of exploit generation Other Web Security issues Remote file Inclusion / Local File Inclusion Directory traversal Command injection Code Injection File upload 2014/2/11 Target Functions include(), include_once(), require(), requireonce()¡ fopen(), file(), unlink¡ system(), file()¡ eval()¡ move_uploaded_file(), rename(), ¡ Liu Huan „¢šg A Generic Web Testing and Attack Generation Framework 57
- 58. Open Doors to More Work ? Symbolic Executions by S2E for ¨C PHP, Python ¨C JSP, Ruby ¨C ASP, Perl ¨C Node JS