Recently we were assigned with a task to perform a penetration test on web application, where there was a limitation on how much time the session would be valid, enforced through authorization token. In order to bypass this restriction and allow burp suite to complete scanning and intruder tasks without failing, we had to create our own Burp extender class. Specifically, the method created, had to inspect every request for the tag ‘Authorization: Bearer ‘ in its headers and after that delete it and replace it with a new one in order for the current session to be valid. After the login request, the generated token would be included in the JSON keyword called ‘bearerToken’ inside the response body. The concept was to extract the bearer token from the response and replace it with the new existing token on the new request. The following request performs a login to the Web Application which enforces the creation of a new token in the response.

POST /some_path/rest/public/security/login HTTP/1.1
Host: ##############
Connection: close
Content-Length: 52
Accept: application/json, text/plain, */*
Origin: ################
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.89 Safari/537.36
Content-Type: application/json;charset=UTF-8
Referer: ################
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8

{"userId":"pentestcommon","password":"#######4321$"}

The above request issues the following response where the bearer Token can be found in the JSON keyword ‘bearerToken’

HTTP/1.1 200 OK
Date: Thu, 01 Sep 2016 10:34:33 GMT
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block;
X-Content-Type-Options: nosniff
Expires: 0
Cache-Control: private, no-cache, no-store, must-revalidate
Pragma: no-cache
Access-Control-Allow-Origin: ################
Access-Control-Allow-Credentials: true
Content-Type: application/json
Vary: Accept-Encoding
Set-Cookie: rememberMe=deleteMe; Path=/some_path; Max-Age=0; Expires=Wed, 31-Aug-2016 10:34:33 GMT;HttpOnly;Secure
Content-Length: 5346
Connection: close

{"bearerToken":"SJHJSHDDHDLJDWIDJWDOMKSNMKSHSKLSNFLSKJKSJSDWOWOSKSJKKSKJ","status":{"id":"OK","description":"OK"},"email":"[email protected]","fullname":"Pen Test Admin","version":"1.14.10","mId":"374859230","bId":null,"termId":null,

[....]

The above request – response is used from a macro which is explained later in this document, to generate a new Token in every scanner request to the application. Apparently, to do this for every request is time consuming. The best way to achieve this solution without having overhead to the scanning task is to check whether the response headers have ‘HTTP/1.1 401 Unauthorized’ header. In such case, a customized script must change the invalid Token.

Invalid Request

GET /some_path/rest/product/all HTTP/1.1
Host: some_website.com
Connection: close
Accept: application/json, text/plain, */*
Authorization: Bearer LFSJBVNSMVOJMOPECMOEPER39I3FLKNFOEIFJEOD0EFOEKFE49I3OKE
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
Referer: https://some_website.com/
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8

The above request is invalid and that can be realized from the response that has HTTP/1.1 401 Unauthorized Header as shown below

Unauthorized Response

HTTP/1.1 401 Unauthorized
Date: Mon, 29 Aug 2016 07:51:53 GMT
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block;
X-Content-Type-Options: nosniff
WWW-Authenticate: Bearer realm="########"
Vary: Accept-Encoding
Set-Cookie: rememberMe=deleteMe; Path=/gmp-web; Max-Age=0; Expires=Sun, 28-Aug-2016 07:51:53 GMT;HttpOnly;Secure
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=UTF-8

The above request includes a Token that has expired due to the time limitation of the application, while handling the token validity. The above Token has to be replaced with the new one which is generated from the macro. See the following request with the new Bearer Token.

Valid Request

GET /some_path/rest/product/all HTTP/1.1
Host: some_website.com
Connection: close
Accept: application/json, text/plain, */*
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
Referer: https://some_website.com/
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8
Authorization: Bearer SJHJSHDDHDLJDWIDJWDOMKSNMKSHSKLSNFLSKJKSJSDWOWOSKSJKKSKJ

For the above replacement to happen, a macro has been created on Burp Suite. The macro ran before the python script in order to generate the new Token by initiating a new login to the application. After the preceding procedure was done, the upcoming requests and responses had received the new authorized token. This procedure validates the requests and the scanner executes the tests properly. In order to accomplish this task, we had to use ISessionActionHandling API which introduces the method performAction that handles session requests and responses when triggered from Burp Suite macros.

The following code snippet demonstrates how the performAction function handles and analyzes requests and responses. The argument macroItems in particular, handles the responses when the requests are triggered from a macro object on Burp Suite. The following python script is used to manipulate the array which carries the response body, while changing it into a string format as well as getting the contents offset location in the response. From there, the Bearer token can be parsed and extracted. The script below checks if the header ‘Authorization: Bearer ‘ already exists in the request and if it does. It replaces it with the new one. Afterwards the new header will be overwritten on the current request to validate the request on scanner or any other related Burp Suite tool.

performAction

def performAction(self, currentRequest, macroItems):

        request_info = self.helpers.analyzeRequest(currentRequest)
        #Extract the Bearer token from the macro response
        macro_response_info = self.helpers.analyzeResponse(macroItems[0].getResponse())
        
        macro_msg = macroItems[0].getResponse()
        resp_body = macro_msg[macro_response_info.getBodyOffset():]
        macro_body_string = self.helpers.bytesToString(resp_body)
        bearer_token = json.loads(macro_body_string)
        bearer = bearer_token["bearerToken"]

		headers = request_info.getHeaders()
        req_body = currentRequest.getRequest()[request_info.getBodyOffset():]
    
        resp_headers = macro_response_info.getHeaders()  
        headers = request_info.getHeaders()
        auth_to_delete = ''
        for head in headers:
            if 'Authorization: Bearer ' in head:
                auth_to_delete = head        
        headers.remove(auth_to_delete)
        headers.add('Authorization: Bearer ' + bearer)        
        self.stdout.println('Header Checked at time :  {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))        
        self.stdout.println("-----------------------------------------------------------------"        )
        self.stdout.println("Adding new header - Authorization bearer: " + bearer)                
        self.stdout.println("-----------------------------------------------------------------")                
        self.stdout.println("Geting authorized..done\n\n")                
        # Build request with bypass headers        
        message = self.helpers.buildHttpMessage(headers, req_body)        
        # Update Request with New Header        
        currentRequest.setRequest(message)
        return 

Before the above code snippet executes, the burp API must be called, in order to initialize the communication with the python script during execution. The following code snippet provides the commands used in order for python interpreter to initialize the communication with burp API.

API Callbacks

import json
import datetime
from java.io import PrintWriter
from burp import IBurpExtender, IBurpExtenderCallbacks, ISessionHandlingAction
 
class BurpExtender(IBurpExtender, ISessionHandlingAction):
 
    NAME = "Bearer Authorization Token"
     
    def registerExtenderCallbacks(self, callbacks):
        self.callbacks = callbacks
        self.helpers = callbacks.getHelpers()
        callbacks.setExtensionName(self.NAME) 
        self.callbacks.registerSessionHandlingAction(self)    
        self.stdout = PrintWriter(callbacks.getStdout(), True)
        #self.stderr = PrintWriter(callbacks.getStdout(), True)
        self.stdout.println("Bearer Authorization Token \n")
        self.stdout.println('starting at time : {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))
        self.stdout.println("-----------------------------------------------------------------\n\n")
        return

The script below has been ported to python from an already implemented script written in ruby from Rory McCune (raesene). The ruby script created in order to manipulate JWT Tokens. Respectively, the following script was written in python to manipulate Bearer tokens:

import json
import datetime
from java.io import PrintWriter
from burp import IBurpExtender, IBurpExtenderCallbacks, ISessionHandlingAction
 
class BurpExtender(IBurpExtender, ISessionHandlingAction):
 
    NAME = "Bearer Authorization Token"
     
    def registerExtenderCallbacks(self, callbacks):
        self.callbacks = callbacks
        self.helpers = callbacks.getHelpers()
        callbacks.setExtensionName(self.NAME) 
        self.callbacks.registerSessionHandlingAction(self)    
        self.stdout = PrintWriter(callbacks.getStdout(), True)
        #self.stderr = PrintWriter(callbacks.getStdout(), True)
        self.stdout.println("Bearer Authorization Token \n")
        self.stdout.println('starting at time : {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))
        self.stdout.println("-----------------------------------------------------------------\n\n")
        return
    
    def getActionName(self):
        return self.NAME
     
    def performAction(self, currentRequest, macroItems):
        request_info = self.helpers.analyzeRequest(currentRequest)
        #Extract the Bearer token from the macro response
        macro_response_info = self.helpers.analyzeResponse(macroItems[0].getResponse())
        
        macro_msg = macroItems[0].getResponse()
        resp_body = macro_msg[macro_response_info.getBodyOffset():]
        macro_body_string = self.helpers.bytesToString(resp_body)
        bearer_token = json.loads(macro_body_string)
        bearer = bearer_token["bearerToken"]
        
        headers = request_info.getHeaders()
        req_body = currentRequest.getRequest()[request_info.getBodyOffset():]
    
        resp_headers = macro_response_info.getHeaders()  
        headers = request_info.getHeaders()
        auth_to_delete = ''
        for head in headers:
            if 'Authorization: Bearer ' in head:
                auth_to_delete = head        
        try:
            headers.remove(auth_to_delete)
        except:
            pass
        headers.add('Authorization: Bearer ' + bearer)        
        self.stdout.println('Header Checked at time :  {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))        
        self.stdout.println("-----------------------------------------------------------------"        )
        self.stdout.println("Adding new header - Authorization bearer: " + bearer)                
        self.stdout.println("-----------------------------------------------------------------")                
        self.stdout.println("Geting authorized..done\n\n")                
        # Build request with bypass headers        
        message = self.helpers.buildHttpMessage(headers, req_body)        
        # Update Request with New Header        
        currentRequest.setRequest(message)
        return

Bearer Authorization Token manipulation extender

For the purpose of this work we have implemented the following extender using eclipse IDE and Java. Specifically, in order to have a new tab in Burp Suite, we have used java swing. Furthermore, the following java extender will create a Tab named bearer which will have two separate columns showing the requests and responses every time the Token changes. You can find the code and the jar file here TwelveSec Github

package burp;
import java.awt.Component;
import java.io.PrintWriter;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

import javax.swing.JScrollPane;
import javax.swing.JSplitPane;
import javax.swing.JTabbedPane;
import javax.swing.JTable;
import javax.swing.SwingUtilities;
import javax.swing.table.AbstractTableModel;
import javax.swing.table.TableModel;

import org.json.JSONException;
import org.json.JSONObject;

public class BurpExtender extends AbstractTableModel implements ISessionHandlingAction ,ITab, IMessageEditorController
{
    private static final long serialVersionUID = 1L;
    private IBurpExtenderCallbacks callbacks;
    private IExtensionHelpers helpers;
    private PrintWriter stdout;
    private String date;
    ArrayList headers = new ArrayList();
    private final List log = new ArrayList();
    private JSplitPane splitPane;
    private IMessageEditor requestViewer;
    private IMessageEditor responseViewer;
	private String NAME = "Bearer Token";    

    private IHttpRequestResponse currentlyDisplayedItem;
    
    public void registerExtenderCallbacks(final IBurpExtenderCallbacks callbacks)
    {
        this.callbacks = callbacks;
        // set our extension name
        callbacks.setExtensionName("Authorization Token");        
        // obtain an extension helpers object
        helpers = callbacks.getHelpers();
      
        // obtain our output and error streams
        stdout = new PrintWriter(callbacks.getStdout(), true);

        SimpleDateFormat sdf = new SimpleDateFormat("dd-M-yyyy hh:mm:ss");
        date = sdf.format(new Date());
        // obtain our output and error streams
        PrintWriter stdout = new PrintWriter(callbacks.getStdout(), true);
              
        // write a message to our output stream
        stdout.println("Authorization Token\n");
        stdout.println("starting at time : " + date);
        stdout.println("-----------------------------------------------------------------\n\n");
     
        SwingUtilities.invokeLater(new Runnable() 
        {
            @Override
            public void run()
            {
                // main split pane
                splitPane = new JSplitPane(JSplitPane.VERTICAL_SPLIT);
                        
                // table of log entries
                Table logTable = new Table(BurpExtender.this);
                JScrollPane scrollPane = new JScrollPane(logTable);
                splitPane.setLeftComponent(scrollPane);

                // tabs with request/response viewers
                JTabbedPane tabs = new JTabbedPane();
                requestViewer = callbacks.createMessageEditor(BurpExtender.this, false);
                responseViewer = callbacks.createMessageEditor(BurpExtender.this, false);
                tabs.addTab("Request", requestViewer.getComponent());
                tabs.addTab("Response", responseViewer.getComponent());
                
                splitPane.setRightComponent(tabs);

                // customize our UI components
                callbacks.customizeUiComponent(splitPane);
                callbacks.customizeUiComponent(logTable);
                callbacks.customizeUiComponent(scrollPane);
                callbacks.customizeUiComponent(tabs);
                
                // add the custom tab to Burp's UI
                callbacks.addSuiteTab(BurpExtender.this);
                
                //register the handler that does that alters the HTTP request
                //this has to be enabled via the Burp Session handling options
                callbacks.registerSessionHandlingAction(BurpExtender.this);
                
            }
        });
    }
    
    @Override
    public void performAction(IHttpRequestResponse currentRequest, IHttpRequestResponse[] macroItems) {
        
        IRequestInfo rqInfo = helpers.analyzeRequest(currentRequest);
        IResponseInfo macro_repsonse_items = helpers.analyzeResponse(macroItems[0].getResponse());
        byte[] macro_msg = macroItems[0].getResponse();
        String Response = helpers.bytesToString(macro_msg);
        String Resp_Body = Response.substring(macro_repsonse_items.getBodyOffset());
       JSONObject json;
    try {
        json = new JSONObject(Resp_Body);
        boolean bearer_token = json.has("bearerToken");
        String Bearer = null; 
        if (bearer_token == true)
            Bearer = json.getString("bearerToken");
            
         // retrieve all headers
        headers = (ArrayList) rqInfo.getHeaders();
        
        // get the request
        String request = new String(currentRequest.getRequest());
        
        // get the request body 
        String messageBody = request.substring(rqInfo.getBodyOffset());
        
        // go through the header and look for the one that we want to replace
        for (int i = 0; i < headers.size(); i++){
           if(((String) headers.get(i)).startsWith("Authorization: Bearer "))
                 headers.remove(i);
                            
        }          
        
        headers.add("Authorization: Bearer " + Bearer);   
        // create the new http message with the modified header
        byte[] message = helpers.buildHttpMessage(headers, messageBody.getBytes());
        
        // replace the current request and forward it
        synchronized(log)
        {
            int row =log.size();
            log.add(new LogEntry("adding new header  - Authorization bearer: " + Bearer,"Header Checked at time : " + date, callbacks.saveBuffersToTempFiles(currentRequest)));
            log.add(new LogEntry("","-----------------------------------------------------------------", callbacks.saveBuffersToTempFiles(currentRequest)));
            log.add(new LogEntry("Geting authorized..done\n\n","", callbacks.saveBuffersToTempFiles(currentRequest)));
            log.add(new LogEntry("-----------------------------------------------------------------\n\n","", callbacks.saveBuffersToTempFiles(currentRequest)));
            fireTableRowsInserted(row,row);
        }
        
        currentRequest.setRequest(message);  
      
    } catch (JSONException e) {
     
        e.printStackTrace();
    }     
    }

    @Override
    public String getActionName() {
     
        return NAME;
    }
    
     @Override
        public String getColumnName(int columnIndex)
        {
            switch (columnIndex)
            {
                case 0:
                    return "Checked Time";
                case 1:
                    return "Outline";
                default:
                    return "";
            }
        }

    @Override
    public int getColumnCount() {
        
        return 2;
    }

    @Override
    public int getRowCount() {
        
        return log.size();
    }

    @Override
    public Object getValueAt(int rowIndex, int columnIndex)
    {
        LogEntry logEntry = (LogEntry) log.get(rowIndex);

        switch (columnIndex)
        {
            case 0:
                 return logEntry.Checked_Time;
            case 1:
                return logEntry.Outline;
            default:
                return "";
        }
    }

    @Override
    public IHttpService getHttpService() {
        // TODO Auto-generated method stub
        return null;
    }

    @Override
    public byte[] getRequest() {
        return currentlyDisplayedItem.getRequest();
    }

    @Override
    public byte[] getResponse() {
        return currentlyDisplayedItem.getResponse();
    }

    @Override
    public String getTabCaption() {
        return "Bearer";
    }

    @Override
    public Component getUiComponent() {
        return splitPane;
    }
    
    @Override
    public Class getColumnClass(int columnIndex)
    {
        return String.class;
    }

    private class Table extends JTable
    {

        private static final long serialVersionUID = 1L;

        public Table(TableModel tableModel)
        {
            super(tableModel);
        }
   
        
        @Override
        public void changeSelection(int row, int col, boolean toggle, boolean extend)
        {
            // show the log entry for the selected row
            LogEntry logEntry = (LogEntry) log.get(row);
            requestViewer.setMessage(logEntry.requestResponse.getRequest(), true);
            responseViewer.setMessage(logEntry.requestResponse.getResponse(), false);
            currentlyDisplayedItem = logEntry.requestResponse;
            super.changeSelection(row, col, toggle, extend);
        } 
    
    }
        private static class LogEntry
        {
            final IHttpRequestResponsePersisted requestResponse;
            final String Outline;
            final String Checked_Time;

            LogEntry(String Outline,String Checked_Time, IHttpRequestResponsePersisted requestResponse)
            {
               this.requestResponse = requestResponse;
               this.Outline = Outline;
               this.Checked_Time = Checked_Time;
            }
        }
    }

Burp Suite

In order for the above script to work, the following steps must be performed. First Jython standalone .jar has to be installed into Burp Suite in order for the python interpreter to be able to communicate with java API. Following this, the above python script has to be installed into extenders tab as shown at the image below.

After the two preceding steps, two macro objects are created as shown below. The picture shows two macros created, each one representing a different login request with the relevant credentials from pentestadmin and pentestcommon users.

The macro editor shown below has the login request ready to run when the macro will be called.

The only thing the macro has to do is to run the above request and login into the application. Once logged in, the new Token will be generated. To be more specific, In this implementation we want the macro to run only when the session is invalid. To know when the session is invalid a new session rule has been generated as shown at the image below.

The above session handling rule ‘Bearer Authorization Token Validation’ accepts requests from the scanner and if the response headers of those requests have the Header ‘HTTP/1.1 401 Unauthorized’, then the macro will be triggered to initiate a new login. Afterwards, the python script will run to extract and replace the new Token to the current request to make it valid. The image below shows how this is implemented with a session handling rule.

In case the above scenario doesn’t work, one solution is to run the macro in every request coming from burp scanner, but this will put extra overhead into the scanning process. Running the macro in every request will be time consuming, but also will be the most accurate solution because there won’t be any possibilities of the scanner failing. This solution can be shown below using the following session handling rule:

Summarize

To summarize, the above methodology provides a solution on testing Enterprise applications that involve security Authorization tokens into every HTTP requests.Furthermore, this solution provides a better approach to solve the problem of Burp suite automated scanning failures when Authorization tokens exist. In general, the penetration tester has to be more efficient and productive during their security assessments. Additionally, approaches like this provide excellent assistance to the penetration tester while organizing their time and effort better, optimizing the quality of work. In the contrary, the only drawback to this kind of approach is that there are extra working hours to be spend in order to implement the solution.

As a conclusion to the above , penetration testing solution development practices must be identified from penetration testers in early stages of the security assessment in order to provide automation in penetration testing tasks.Therefore, Burp Suite provides a flexible and easy to use API which gives the penetration tester the ability to create tools that will produce the maximum performance on their day to day work. Consequently, any penetration testing tool that provides the ability to extend its functionality becomes beneficial from the security test perspective while increases the possibilities of accuracy as well as ameliorate the quality of penetration tester’s work. Also Tools and methodologies need to be more generic in order to be used in every relevant cases with little or no changes to accomplish better results.