Oyedata for OData Security Assessments

The Open Data Protocol (OData) is an open web protocol for querying and updating data. OData enables the creation of HTTP based RESTful  data services that can be used to publish and edit resources with simple HTTP messages.  OData is intended to be used to expose and access information from a variety of sources including relational databases, file systems, content management systems, and traditional web sites. It allows a consumer to query a data source over HTTP protocol and get results back in formats like Atom, JSON or plain XML. OData can be termed as JDBC/ODBC for the internet.

The protocol is relatively new and is being adopted by many major software manufacturers such as Microsoft, IBM, and SAP but hasn’t been publically explored in terms of security. As more applications, websites, and frameworks support OData, a larger attack surface becomes available to attackers.

Oyedata is a new tool to perform black-box OData security testing and help secure OData deployments. I wrote Oyedata from a penetration testing perspective and its the major features are summarized below:

1.     Intuitive GUI based tool written in C#.

2.     Ability to create attack templates from local and remote Service Documents and Service Metadata Documents.

3.     Support for XML and JSON data formats.

4.     Ability to export attack templates in JSON and XML formats that can be fed to custom Fuzzing code.

5.     Ability to engage the OData services for manual testing.

6.     Data generator for EDMSimpleType test data generation.

7.     Ability to generate “Read URIs” for Entities, Entity Properties and Entity Property Values.

8.     Ability to generate attack templates for Creation of new Entries, updating existing Entries, Service Operation invocation, Entry deletion etc…

9.     Ability to identify Keys, Nullable and Non-Nullable Properties and indicate the same in the attack templates.

10.  Web proxy, HTTP and HTTPS support and Error logging.

Image shows Oyedata retrieving an OData Service Metadata document 

Image shows various Create, Read, Delete and Update operations of the “Categories” Feed along with the supported Service Operation nodes

The tool is now available for download from McAfee website from this URL. Please send in your suggestions and feedback.

CAPTCHA Re-Riding Attack

This attack was voted at #8 in Top Ten Web Hacking Techniques of 2012

CAPTCHA Re-Riding Attack bypasses the CAPTCHA protection built into the web applications. The attack exploits the fact that the code that verifies CAPTCHA solutions sent by the user during form submissions does not clear the CAPTCHA solution from the HTTP Session. 

Impact: A large number of successful submissions on CAPTCHA protected pages by riding on a single CAPTCHA solution. 

A typical scenario to demonstrate the vulnerability is explained below. 

1.       A user visits register page of the website.

2.       The website creates an HTTP session, assigns it a SESSIONID and returns the register page to the user along with the SESSIONID cookie. The register page also contains one image tag which directs the browser to retrieve a CAPTCHA and display it on screen.

3.       Upon parsing the image tag, the browser sends out request for the CAPTCHA.

4.       The server side code creates a new CAPTCHA with random text and CAPTCHA solution is stored in the HTTP session.

5.       CAPTCHA image is then sent to the client and is then displayed by the browser.

6.       Browser sends CAPTCHA solution along with form fields for verification.

7.       Server side code retrieves CAPTCHA solution from the HTTP Session and verifies it against the solution provided by the client.

8.       If verification is successful, client is sent to next logical step in the registration process. If not, client is redirected to the register page (step 1 above).

Figure 1: Image shows an example Register page that supports CAPTCHA

Analysis of the CAPTCHA generation and verification process reveals the following:

1. The captcha.php is the only page responsible for updating the HTTP session with correct CAPCHA solution. The first ingredient.
2. CAPTCHA solution inside the HTTP session is not explicitly cleared during the verification process. Yes, you guess it right. This is the second and the most important ingredient for CAPTCHA Re-Riding Attacks.
3. When registration fails (for any reason), the web applications continue to use the same HTTP session and SESSIONID. We will not look into this further.
4. When registration succeeds, the user is redirected to next step and the CAPTCHA generation page (/captcha.php) is not likely to be called for current SESSION again.  This allows the CAPTCHA solution to stay in the HTTP store for as long as SESSION is valid. Following are the likely scenarios to be seen when CAPTCHA verification is successful.
1. The web application generates a new SESSIONID for the same HTTP session for known security reasons. This implementation is most likely to be seen. Combine this behavior with first and second ingredients above and you have a successful CAPTCHA Re-Riding attack.
2. The web application continues to use the same SESSIONID for the same HTTP session.  Here we have more things to worry than just the CAPTCHA. For now, combine this behavior with first and second ingredients above and you have a successful CAPTCHA Re-Riding attack again.
3. The web application generates a completely new HTTP session with new or same SESSIONID. For CAPTCHA Re-riding Attack, this scenario is not exploitable.

For scenarios 4.a and 4.b, the HTTP Session continues to hold the CAPTCHA solution as it is not explicitly cleared by the CAPTCHA verification code. Since /captcha.php is not going to be called again (and we will not let the call happen anyway), the same CAPTCHA solution continues to exist in HTTP session. Let us now see how 4.a & 4.b scenarios above can be exploited to make multiple successful submissions using a CAPTCHA solution.

Exploiting Scenario 4.b:

1.       Load the register page of the target website in a web browser.

2.       Solve the CAPTCHA manually, and submit the form.

3.       Record this form submission using a web proxy. This request contains a valid SESSIONID, valid form fields and a valid CAPTCHA solution.

4.       Create a custom script or use any tool like Burp intruder that can repeatedly send this request to server. With each request change the unique values (like User ID) to create multiple new accounts with a single CAPTCHA solution.

Exploiting Scenario 4.a:

1.       Load the register page of the target website in a web browser.

2.       Solve the CAPTCHA manually, and submit the form.

3.       To make things easy, trap this request in a web proxy and do not allow it to reach the web server. This request contains a valid SESSIONID, valid form fields and a valid CAPTCHA solution.

4.       Create a custom script or use any tool like Burp intruder that can repeatedly send this request to server.

5.       Submit one request.

6.       Upon successful submission, the web application will reset the current SESSIONID and send new SESSIONID back in response headers.

7.       Change the value of SESSIONID in recorded request (step 3) to the value copied from response in Step 6 above.

8.       Go to step 5.

9.     We will be able to make multiple successful submissions with single CAPTCHA solution.

Using one time tokens along with CAPTCHAs on the register pages may still be exploitable with a few additional lines of attack code. The best defense is to reset CAPTCHA solution inside the HTTP session during the CAPTCHA verification stage. It is also important to note that when a website relies on third party CAPTCHA  provider it does not maintain any session information at its end and CAPTCHA is performed by the CAPTCHA provider and these websites are not vulnerable to CAPTCHA Re-Riding Attack.

Sqlitespy for Sqlite Database analysis

Sqlite is the ubiquitous database for mobile applications on iPad, iPhone and Android. It is also used by certain internet browsers, web application frameworks and software products for their local storage needs. While doing penetration tests, we often see sensitive information like usernames, passwords, account numbers, SSN etc… insecurely stored in these databases. Thus, every penetration test requires comprehensive analysis of the local databases being used.

While analyzing databases, a penetration tester repeatedly does the following:

1. Opens the database in sqlite reader (sqlite3 or other readers)
2. Views various tables and columns to understand database layout and schema.
3. Analyzes the storage for sensitive information.

As the number and size of database increases, the analysis time increases exponentially. To escape the recurring pain, I wrote a ruby script to automate this process. The script achieves the following:

1. Analyzes multiple databases in a single run.
2. Queries and displays database schema.
3. Provides an option to run search on Table and Column Names for quick analysis.
4. Performs case-insensitive regular expression search (default). This can be controlled with command line options to one’s requirements.
5. Displays Database, Tables and Row Number reference for every successful match.
6. Dumps database rows on a successful match.
7. Looks for search strings in the following:
• Table Name
• Column Names
• Actual Data

Sqlitespy dependencies are listed below:

1. Ruby
2. Sequel Gem
3. Sqlite3

Figure 1: Image shows sqlitespy help

Figure 2: Image shows sqlite sample run with multiple search strings and row information dump for a successful match

Figure 3: Image shows sqlitespy sample run with minimal information

Figure 4: Image shows sqlite database schema dump

Sqlitespy Code Follows:

 #Author: Gursev Singh Kalra  
 require 'rubygems'  
 require 'optparse'  
 require 'ostruct'  
 require 'sequel'  
   
   
 class CmdLineOptions  
   
  def self.parse(args)  
   options = OpenStruct.new  
   options.dbs = []  
   options.sstrings = []  
   options.show_schema = false  
   options.case_sensitive = false  
   options.exact = false  
   options.verbose = false  
   options.rowdump = false  
   options.metadata = false  
   
   opts = OptionParser.new do |opts|  
   opts.banner = "Usage: sqlitespy.rb [options]\n\nSpecific Options:"  
   
   opts.on("-d", "--database DATABASE_PATH",  
    "Sqlite database to analyze.") do |db|  
    options.dbs << db  
   end  
   
   opts.on("-s", "--show-schema", "Show database schema") do |show|  
    options.show_schema = show;  
   end  
   
   opts.on("--find x,y,z", Array, "Strings to search") do |list|  
    options.sstrings = list  
   end  
   
   opts.on("-c", "--case-sensitive", "Perform case sensitive search. Default is case insensitive.") do |case_sensitive|  
    options.case_sensitive = case_sensitive;  
   end  
   
   opts.on("-e", "--exact--match", "Perform exact match for the search strings") do |v|  
    options.exact = v;  
   end  
   
   opts.on("-r", "--row-dump", "Dump Database Row when a match is found") do |v|  
    options.rowdump = v;  
   end  
   
   opts.on("-m", "--metadata", "Look for search strings only in DB metadata (table and column names)") do |v|  
    options.metadata = v;  
   end  
   
   opts.on("-v", "--verbose", "Verbose output") do |v|  
    options.verbose = v;  
   end  
   
   opts.on_tail("-h", "--help", "Show this message") do  
    puts opts  
    exit  
   end  
   
   end  
   
   opts.parse!(args)  
    options  
   end# parse()  
   
 end# class CmdLineOptions  
   
 options = nil  
   
 begin  
  options = CmdLineOptions.parse(ARGV)  
 rescue (OptionParser::InvalidOption)  
  $stderr.puts "[-] Invalid option "  
  options = CmdLineOptions.parse(ARGV+["-h"])  
 end  
   
 if(options.dbs.length == 0)  
  $stderr.puts "[-] No Database available. Exiting !!"  
  exit  
 end  
   
 dbs = []  
 options.dbs.uniq!  
   
 dbs = options.dbs.collect do |db|  
  begin  
   throw Errno::ENOENT unless(File.file?(db))  
   Sequel.sqlite(db).tables  
   db  
  rescue  
   $stderr.puts "[-] \"#{db}\" is not a sqlite database"  
   nil  
  end  
 end  
   
 options.dbs = dbs.compact  
   
 if(options.dbs.length == 0)  
  $stderr.puts "[-] No Database available. Exiting."  
  exit  
 end  
   
 options.sstrings.uniq!  
   
 if(options.show_schema)  
   
  puts  
  puts "+"*80  
  puts "Database Schemas"  
  puts "+"*80  
   
  options.dbs.each do |db|  
   puts  
   puts "[DATABASE] #{db}"  
   Sequel.sqlite(db) do |dbhandle|  
    dbhandle.tables.each do |table|  
     puts "\t[TABLE] #{table}"  
     puts "\t\t[COLUMNS] #{dbhandle[table.to_sym].columns.join(', ')}"  
    end  
   end  
  end  
   
  puts "-"*80  
   
 end  
   
 regex_strings = []  
 regex_strings = options.sstrings.collect do |search|  
   
  regexstr = ""  
  regex = nil  
  if(options.exact)  
   regexstr = "^#{search}$"  
  else  
   regexstr = "#{search}"  
  end  
   
  if(options.case_sensitive)  
   regex = Regexp.new("#{regexstr}")  
  else  
   regex = Regexp.new("#{regexstr}", Regexp::IGNORECASE)  
  end  
  regex  
   
 end  
   
 options.sstrings = regex_strings  
   
 options.dbs.each do |database|  
   
  if(options.verbose)  
   puts  
   puts "+"*80  
   puts "Analyzing Database '#{database}'"  
   puts "+"*80  
  end  
   
  Sequel.sqlite(database) do |databasehandle|  
   databasehandle.tables.each do |table|  
    if(options.verbose)  
     puts  
     puts "-"*80  
     puts "Analyzing Table '#{table}'"  
     puts "-"*80  
    end  
   
    options.sstrings.each do |regex|  
     if(regex.match(table.to_s))  
      puts "[+] Table Name Match Found -> Database '#{database}' -> TABLE '#{table}'"  
     end  
    end  
   
    #Column Name Search  
    databasehandle[table.to_sym].columns.each do |column_name|  
     options.sstrings.each do |regex|  
      if(regex.match(column_name.to_s))  
       puts "[+] Column Name Match Found -> Database '#{database}' -> TABLE '#{table}' -> COLUMN '#{column_name}'"  
      end  
     end  
    end  
   
    #Data Search  
    if(options.sstrings.length > 0 && !options.metadata)  
     row = 0  
     databasehandle[table].each do |rowHash|  
      row = row + 1  
      rowHash.each do |key, value|  
       options.sstrings.each do |regex|  
        if(regex.match(value.to_s))  
         puts "[+] Data Match Found -> Database '#{database}' -> TABLE '#{table}', COLUMN '#{key}' -> ROW '#{row}'"  
         puts "\t[*] Row Dump\t=>\t#{rowHash.values.join('|')}" if(options.rowdump)  
        end  
       end  
      end  
     end  
    end  
   end  
  end  
 end  

JSON CSRF with Parameter Padding

JavaScript Object Notation (JSON) format is one of the prominent data exchange formats of the contemporary web applications. When a web application implements JSON, Cross Site Request Forgery (CSRF) payload delivery gets bit tricky because of query string and JSON format mismatch. With couple of tricks however, we can successfully execute CSRF attacks with JSON payloads.

Let’s assume that the browser sends the following JSON to the web server.

{"a":1,"b":{"c":3}}

Scenario 1: One of the mechanisms to execute JSON CSRF is to use the entire JSON payload as parameter name in a self submitting form. For example, loading the HTML code below and clicking the submit button sends malicious JSON to the web server:

1. <html>
2. <form action=http://192.168.1.41:3000 method=post enctype="text/plain" >
3. <input name='{"a":1,"b":{"c":3}}' type='hidden'>
4. <input type=submit>
5. </form>
6. </html>

At line# 2, the enctype form attribute is set to text/plain so that the JSON gets delivered as is. The enctype attribute may not be required, but is good to have. At line# 3, entire JSON payload is provided as a parameter name. When the form gets posted, the payload is delivered and CSRF executes.

Image below shows JSON payload delivery with the technique described above.

This technique may fail in some cases when the server side JSON parsers reject the incoming JSON because of the trailing ‘=’ character.

Scenario 2: JSON Parameter Padding to the rescue

In scenario 1, the trailing ‘=’ character may ruin the party when server side JSON parsers enforce strict parsing rules. To overcome this, an additional parameter can be padded towards the end of JSON payload to send a well formed JSON. Similar to GET & POST parameter processing, JSON parsers will successfully parse the JSON, pick the required parameters and ignore the extraneous ones. This allows a successful CSRF attack against vulnerable web applications. 

Below, the HTML code in scenario 1 is modified to add an extraneous parameter to the JSON payload:

1. <html>
2. <form action=http://192.168.1.41:3000 method=post enctype="text/plain" >
3. <input name='{"a":1,"b":{"c":3}, "ignore_me":"' value='test"}'type='hidden'>
4. <input type=submit>
5. </form>
6. </html>

At line# 3, the component in red is the original JSON and the blue component helps add the extraneous parameter to the JSON payload. The screenshot below shows the JSON payload delivered when the above HTML is executed. The ignore_me parameter absorbs the trailing '=' character and has a value "=test". 

The end result, successful server side JSON Parsing and CSRF goodness :)

Image shows a well formed JSON sent using parameter padding

It is important to note that the discussed attack vectors may not work if the server validates the “Content-Type” request header to represent a JSON payload.

Decoding BigIP Cookie

BigIP cookie contains internal network IP and port information in encoded format. When decoded, these cookies can help create an internal network map with potential web server IPs and their ports.

F5 has described the encoding algorithm here. It works like this:

1. If the IP address is a.b.c.d, it is encoded as d*256^3 + c*256^2 + b*256 +a
2. To encode the port is to take the two bytes that store the port and reverse them. Thus, port 80 becomes 80 * 256 + 0 = 20480. Port 1433 (instead of 5 * 256 + 153) becomes 153 * 256 + 5 = 39173.
3. These values are combined into cookie as <Encoded IP Address>.<Encoded Port Address>.<Componenet we are not concerned about>

These decoding mechanisms are packed into the following ruby script:

#!/usr/bin/ruby
#Cookie: BIGipcookie => 404007104.20480.0000
#Cookie: BIGipcookie => 404007104.39173.0000

if (ARGV.length == 0)
  $stderr.puts "No input provided. Run as \n\tbigip.rb BigIP Cookie Value"
  exit
end

ips = ARGV[0].split(".") 
encoded_val = ips[0].to_i
port_val = ips[1].to_i
ip = []
port = []

4.times do
  ip << encoded_val%256
  encoded_val /= 256
end

2.times do 
  port << port_val%256
  port_val /= 256
end

puts "IP Address : #{ip.join(".")}"
puts "Port       : #{port[0]*256 + port[1]}"

A Sample bigip.rb run

Evading Content Security Policy With CRLF Injection

Content Security Policy (CSP) was developed with the aim of reducing content injection attacks like Cross Site Scripting. CSP allows the developers to specify the permitted content sources for their web applications and relies on HTTP response headers to enforce content restrictions.

When CSP is implemented by the web application and supported by the web browser, content injection attacks can be performed by:

1. Exploiting flaws in browser CSP implementation
2. Manipulating HTTP response headers.

CRLF injection is one possible technique by which an attacker can control HTTP response headers. If client provided parameters are returned in response headers without any validation, CRLF injection can be used to bypass CSP restrictions.

For demonstrations, two web pages were setup with the following content at two different origins
Webpage 1: http://localhost:3000/csp
Content:
http://localhost:3333/xss.js

Webpage 2: http://localhost:3333/xss.js
Content:
alert('XSS’)


CRLF Injection and CSP:
If a HTTP response contains same HTTP header multiple times, different browsers interpret the headers  differently. Certain browsers interpret the first occurrence of the HTTP header, others choose the last one. Hence, positioning of CSP directive (X-Content-Security-Policy) in application response can play an interesting role. In the discussion below, we assume that the web application implements CSP and is vulnerable to CRLF injection:

Case 1: Attack vector is returned before the CSP header in the HTTP response headers:
Case 1a: If the browser picks the first occurrence of the CSP header, CRLF injection can then be used to insert a CSP header with following attack vector:

lang=en_US%0d%0aX-Content-Security-Policy: allow *

In this case, the web browser will interpret the first CSP header and will happily retrieve content from any malicious URL.

Image shows malicious CSP directive inserted before the legitimate header 

Case 1b: If the browser picks the last occurrence of the CSP header, following CRLF injection attack vector can be used to insert custom CSP header.

lang=en_US%0d%0aX-Content-Security-Policy: allow *%0d%0a%0d%0a

Two trailing occurrences of CRLF will push the CSP directive into the content and will not be interpreted as a CSP directive. This again allows attacker to bypass CSP protection and execute and source arbitrary content.

Image shows CSP directive pushed out to response body and rendered ineffective 

Case 2: Attack vector is returned after the CSP header in the HTTP response headers
Case 2a: If the browser picks the first occurrence of the CSP header, the CSP directive cannot be overridden for the current resource. For an attack to function one has to look into the possibility of exploiting HTTP Response Splitting.

Case 2b: If the browser picks the last occurrence of the CSP header, CRLF injection can be used to insert a malicious header similar to case 1a.

lang=en_US%0d%0aX-Content-Security-Policy: allow *

This will cause the browser to interpret the CSP directive as allow * to retrieve content from arbitrary URLs.

It was observed that when more than one X-Content-Security-Policy headers were received by Firefox (7.0.1), it securely defaulted to same origin policy for all content.

The POC below pushes the headers out to the response body by two CRLF sequences to achieve script execution.

Image shows script execution prevented from a different origin (http://localhost:3333)

Image shows successful script execution when the page was vulnerable CRLF injection

CAPTCHA Hax With TesserCap

This blog post was voted as 8th best in Top 10 Web Hacking Techniques of 2011 poll.

With the goal of creating a tool that can help security professionals and developers to test their CAPTCHA schemes, I conducted a research on over 200 high traffic websites and several CAPTCHA service providers listed on Quantcast’s Top 1 Million Ranking Websites.

During the same time frame, students at the Stanford University also conducted a similar research (PDF). Both research works concluded the obvious:

An alarming number of CAPTCHAs schemes are vulnerable to automated attacks.

I looked around, tested and zeroed in on Tesseract-OCR as my OCR engine. To remove color complexities, spatial irregularities, and other types of random noise from CAPTCHAs, I decided to write my own image preprocessing engine. After a few months of research, coding and testing in my spare time, TesserCap was born and is ready for release now.

TesserCap is a GUI based, point and shoot CAPTCHA analysis tool with the following features:

1. A generic image preprocessing engine that can be configured as per the CAPTCHA type being analyzed.
2. Tesseract-OCR as its OCR engine to retrieve text from preprocessed CAPTCHAs.
3. Web proxy support
4. Support for custom HTTP headers to retrieve CAPTCHAs from websites that require cookies or special HTTP headers in requests
5. CAPTCHA statistical analysis support
6. Character set selection for the OCR Engine

An example TesserCap image preprocessing and run on Wikipedia (Wikimedia’s Fancy CAPTCHA) is shown below:

Downloads

TesserCap and it's user manual can be downloaded from one of the following locations:

• http://www.opensecurityresearch.com/files/tessercap.zip -- No password protection on this zip file
• http://www.mcafee.com/us/downloads/free-tools/tessercap.aspx -- Use password as "foundstone" without quotes to extract this zip file.

Results

The two tables below summarize the CAPTCHA analysis performed using TesserCap for few popular websites and some CAPTCHA service providers. All these tests were performed using TesserCap’s image preprocessing module and Tesseract-OCR’s default training data.

Website	Accuracy*	Quantcast Rank
wikipedia	20-30%	7
ebay	20-30%	11
reddit.com	20-30%	68
CNBC	50+%	121
foodnetwork.com	80-90%	160
dailymail.co.uk	30+%	245
megaupload.com	80+%	1000
pastebin.com	70-80%	32,534
cavenue.com	80+%	149,645

CAPTCHA Provider	Accuracy*
captchas.net	40-50%
opencaptcha.com	20-30%
snaphost.com	60+%
captchacreator.com	10-20%
www.phpcaptcha.org	10-20%
webspamprotect.com	40+%
ReCaptcha	0%

*This accuracy maybe further increased by training the Tesseract-OCR engine for the CAPTCHAs under test.

Wikipedia

OpenCaptcha Preprocessing

OpenCaptcha Sample Run

Reddit

eBay