HMAC REST API Security

Now we’re going to talk a bit about security. I wouldn’t say this is really my area of expertise so take what follows with a bucket of salt, but I’ve recently been looking a bit deeper into REST API security; authenticating users, verifying requests… things of that nature. There are a number of different methods or protocols a developer can use to secure a REST API and all of them have strengths and weaknesses. One of the challenges that you face when handling REST API security is the fact that it is a principle of REST architecture to remain stateless where the server does not maintain any record of whether or not a user is authenticated/authorized (i.e. logged in via sessions). So in order to determine who is sending the request (and whether they are authorized to access a particular resource) on the server side, all of the information needed to handle this has to contained within the request coming from the client.

In other sections, some common methods of handling REST API authorization and authentication were discussed. Basic HTTP Authentication was discussed here and OAuth using Google as a provider was discussed here. In what follows, we’re going to look at another implementation that can also be a pretty good solution to securing REST APIs: HMAC. What is HMAC? No, it’s not a new burger that McDonalds is rolling out. HMAC stands for “hash-based message authentication code.” Like the name suggests, this means we’re going to be sending a hash (a jumble of letters and numbers) back and forth between the client and server and the system is going to be able to figure out (hopefully) if the request is coming from someone we trust or if it is coming from one of less than noble character.

There are some great articles that have been written on HMAC. Notably…

As is discussed in the articles above, the general idea behind HMAC is this… A client and a server know a secret key. This secret key is never sent over the wire. It is only used in combination with other pieces of data that *are* sent over the wire. That way, when we use our secret key and any other data transferred — say, a public key identifying the user (in the form of a header or a cookie), the message body, the current timestamp or anything else we want to use — and we run that data through an encryption algorithm such as SHA-1… we can create the same hash on both the client and the server! That’s how we know the request is valid. Only a client with the secret key would be able to reproduce the same hash it ends up sending to the server.

If all of this seems confusing at the moment, hopefully it will be a bit clearer when we look at some code a bit later. What we’re going to look at in what follows is an example of HMAC in action. So let’s get building! First, what we’re going to want to do is create our API, implement HMAC for authentication/authorization, and create a few endpoints to hit.

REST API: Server Side

I am fan of using the Slim Framework for building RESTful APIs — at least for demos and examples because it is very lightweight. Maybe I wouldn’t make it a first choice for a large-scale API, but it will work well for us here. For our API demo, we’ll only need to create a couple of endpoints for login and then also a test route that will only be able to be accessed by someone who is “logged in” and they’ll be authenticated using HMAC…

// LOGIN route
$app->post('/login', function () use ($app) {
    
    $request = (array) json_decode($app->request()->getBody());
    $username = $request['username'];
    $password = $request['password'];
    
    if($username === 'demo' && $password === 'demo') {
        $app->setCookie('PUBLIC-KEY', 'demo', '1 hour');
        $userArray = array(
            'username' => $username
        );
        echo json_encode($userArray);
    }
    else {
        $app->response()->status(401);
    } 
});

// TEST route
$app->get('/test', function () use ($app) {
    
    $data = array(
        'message' => 'Hooray! We have reached this route by passing authentication'
    ); 	
    echo json_encode($data);
 	
});

So as we can see the login accepts the username and password “demo” and “demo”. In a real application, you’d validate these credentials against what is stored in a database. After a successful login, a cookie gets set for the logged in user to identify him/her. This cookie will be used as part of our HMAC hash generation.

Now, a hacker on the same network using a tool like Firesheep (or something similar) might be able to sniff network traffic and steal this cookie. But, because the hacker won’t have the secret API key (because that does not get sent over the wire), that won’t even matter because the hacker will not be able to recreate the same hash without that key. You’ll see this in action later.

What Slim uses to authenticate requests is middleware. Basically this is code that runs before the request actually reaches the intended route (e.g. GET /products, POST /comments). So it is here where we’ll want to process the request data and determine whether or not we let the requester go through to the route. So this is where we’ll want to create our hash. The client will send the hash that they’ve created on their end in a header (you could also probably used a cookie if you preferred). What the server will do is take other data sent in the same request (a public key and the timestamp) and then those in combination with the secret key to create a hash of its own. If the hash matches the has sent from the client, then the request can be trusted.

As mentioned before, we’re going to want to timestamp our requests. We’ll want to to a check to make sure that the timestamp is a recent one and we’re also going to want to use the current time as one of the components in creating our hash. Why go to all this trouble? Because we want to do as much as we can to prevent our system from being vulnerable to replay attacks. Basically, if we can determine that the data sent to our API have been sent recently and is not from a substantial amount of time prior, then there is a greater degree of trustworthiness in the data being sent.

One of the challenges when dealing with figuring out “time” on the Internet are the different timezones between servers (where websites are hosted) and clients (where users are accessing the websites from). As a result, you can’t just compare unprocessed time values because the client’s time on his or her system might be completely different from the server’s time depending on where each is located in the world. Fortunately there is an equalizer in all of this: UNIX time. This is also sometimes referred to as microtime. Whatever name is used, it can be essentially described as the number of seconds that have elapsed since the UNIX epoch (0:00:00 January 1, 1970 GMT). As a result, this is normalized across all timezones and will return the same value regardless of the timezone in which the server or client is currently located. Because of this, it is probably the best candidate for reliably determining whether or not our request came recently.

PHP has a microtime function that we can use to get the time elapsed since the UNIX epoch. On the client-side, JavaScript does not have a baked in microtime function so we’ll have to write our own.

So let’s look at our Slim Framework middleware class in PHP to see how this all fits together…

class HMACAuth extends \Slim\Middleware
{   
    /**
     * @var array
     * 
     * Route string set to tell middleware to ignore authentication
     */    
    protected $allowedRoutes;
    
    /**
     * Constructor
     *
     * @param string  $realm The HTTP Authentication realm
     */
    public function __construct()
    {
        $this->allowedRoutes = array(
            'POST/user',
            'POST/login',
            'POST/logout'
        );  
    }
    
    /**
     * Deny Access
     *
    */    
    public function deny_access() {
        $this->app->response()->status(401);   
    }
    
    /**
     * Check Allowed Routes
     *
    */    
    public function check_allowed_routes($routeCheck) {
        
        foreach ($this->allowedRoutes as $routeString) {
            if($routeCheck == $routeString)
                return true;
        }
    
        //if we've gotten this far, route not found
        return false;
    }    
 
 
    /**
     * Check Timestamp
     *  
     * Uses the header value timestamp to check against the current timestamp
     * If the request was made within a reasonable amount of time (10 sec), 
     */
    public function check_timestamp() {
        $req = $this->app->request();   
        $clientMicrotime = $req->headers('X-MICROTIME');
        $serverMicrotime = microtime(true);
        $timeDiff = $serverMicrotime - $clientMicrotime;
    
        if($timeDiff <= 10) 
            return true;
        else
            return false;
    }    
    
    
    /**
     * Authenticate     
     *
     * This is the authenticate method where we check the hash from the client against 
     * a hash that we will recreate here on the sevrer. If the 2 match, it's a pass.
     */
    public function authenticate($token) {

        $cookies = $this->app->request()->cookies();
    
        if(isset($cookies['PUBLIC-KEY']))
            $message = $cookies['PUBLIC-KEY'];
        else
            return false;
    
        $timestamp = $this->app->request()->headers('X-MICROTIME');    
        if($token === $this->create_hash($message, $timestamp))
            return true;
        else
            return false;
    }    

    
    /**
     * Create Hash
     *
     * This method is where we'll recreate the hash coming from the client using the secret key to authenticate the 
     * request
     */
    public function create_hash($message, $timestamp) {
        $apiSecretKey = 'ABC123';
        return hash_hmac('sha1', $message.$timestamp, $apiSecretKey);
    }        
    
    /**
     * Call
     *
     * This method will check the HTTP request headers for previous authentication. If
     * the request has already authenticated, the next middleware is called. Otherwise,
     * a 401 Authentication Required response is returned to the client.
     */
    public function call()
    {
        $req = $this->app->request();
    
        if($this->check_allowed_routes($req->headers('REQUEST_METHOD').$req->getResourceUri()))    
            $this->next->call();
        else {

            if(!$this->check_timestamp()) {
                $this->deny_access();
            }
            else {
        
                // get our hash
                $hash = $req->headers('X-HASH');
                if ($this->authenticate($hash)) {    
                    $this->next->call();
                } else {
                    $this->deny_access();
                }

            }
        }
    }
}

All right, so let’s talk about what’s going on in this class. As is described in the Slim Framework middleware documentation, when the middleware class runs the entry point is the call() method. The first thing that this method checks is whether or not this route is in a list of “allowed routes.” We don’t want the middleware to run authentication checks on every single route. For example, we need to have have a way for an unauthenticated user to POST his or her credentials to the login route. If the middleware ran authentication before the credentials reached the route, the middleware would say “Bzzt! You’re not authenticated” and deny the user. Well… of course they’re not authenticated because they have not logged in yet. So we need to have a way to allow *some* things through. Any routes that we want to declare as “open” routes not needing authentication, we can add to the array in the class constructor in the format shown.

But if we have a route that can only be accessed by an authenticated user it is here where we need to do our validation and we need to check some things. First, we check to see if the timestamp sent is within a reasonable amount of time. You can change the amount of time to check against to whatever you like in the code. If the timestamp is deemed ok, we now need to confirm whether a hash we create on the server side matches one coming from the client side in the form of a header using the timestamp and whatever else we decide to use. In this case it’s a public id that gets set as a cookie when a user logs in with valid credentials. We take that the timestamp and our API secret key to create a hash. If the hashes match then obviously the client knew what the secret key was and we can, for the most part, trust the request. The API secret key (‘ABC123’) is hardcoded in this class for demonstration purposes, but in a real application you’d probably want to set this as a constant in some config file or, even better, have a unique private key per user that you fetch based off of the public id after a user successfully logs in and then use this value in every subsequent request. And you’d also definitely want to use something a little bit more complex than ‘ABC123’.

So if everything passes and is deemed a-ok, the request is let through to the route!

So now that we have the server side in place… next, we must figure out what we’re going to send in our request(s) on the client side.

Client Side

For the client side we’ll just send some data to server endpoints using jQuery, and we’ll be making heavy use of the jQuery.ajax() function. What we’re going to do is set some headers that we’ll pass along with our request. We’re also going to use CryptoJS which is a JavaScript library that gives us implementations of standard and secure cryptographic algorithms. It’s awesome that we don’t have to write any of that ourselves. We’re going to be using the SHA-1 HMAC hash function in CryptoJS but really you could use whatever one you like. Just make sure you use the same one on the server side as well. Otherwise, nothing will work.

First we’re going to want to “log in.” So we’ll set up a login form and log in. We’ll set our /login endpoint to only accept the credentials ‘demo’ and ‘demo’ for the username and password.

<h1>HMAC Authentication</h1>

<div class="loggedIn">
    <span class="name"></span>
    <a href="#" class="sendRequest">Send a request to /test</a>
</div>

<div class="loggedOut">
<a href="#" class="sendRequest">Try to send a request to /test (without being logged in)</a>

<h3>Sign in Below</h3>
<form id="loginForm">

    <p><label>Username</label><input type="text" id="username" /></p>
    <p><label>Password</label><input type="text" id="password" /></p>
    <p><a href="#" id="login">Login</a></p>
	
</form>
<div>

So to set up our app, we’ll namespace everything inside of “app” as is common in JavaScript applications. Because our application is just a small demo, it doesn’t matter too much, but we’ll do it anyway just to follow the decent practice of not hammering the global object.

var app = (function($){
    
    var baseURL = 'http://localhost/development/demos/hmac-authentication/api';
    var apiSecretKey = 'ABC123';
    
    var init = function(){
        $('.loggedIn').hide();
        $('#login').on('click', function(e){
            e.preventDefault();
            login();
        });
        
        $('#sendRequest').on('click', function(e){
            e.preventDefault();
            testRequest();
        });
    };
    
    var login = function() {
        
        var u = $('#username').val();
        var p = $('#password').val();

        $.ajax({
            type: "POST",
            url: baseURL + "/login",
            contentType: "application/json; charset=utf-8",
            dataType: "json",
            data: JSON.stringify({username: u, password: p}),                    
            success: function (data) {
                $('.loggedOut').hide();
                $('.loggedIn').show();
                $('.loggedIn .name').text("Hello, " + readCookie('PUBLIC-KEY') + " ");
            },
            error: function (errorMessage) {
                alert('Error logging in');
            }
        });
    };
    
    var testRequest = function() {
        var data = { test: 'test'}
        var timestamp = getMicrotime(true).toString();
        $.ajax({
            type: "GET",
            url: baseURL + "/test",
            contentType: "application/json; charset=utf-8",
            dataType: "json",
            beforeSend: function (request) {
                request.setRequestHeader('X-MICROTIME', timestamp);
                request.setRequestHeader('X-HASH', getHMAC(readCookie('PUBLIC-KEY'), timestamp));
            },    
            data: JSON.stringify(data),
            success: function (data) {
                alert(data.message);
            },
            error: function (errorMessage) {
               if(errorMessage.status == 401)
                   alert('Access denied');
            }
        });
    };

    var readCookie =  function (name) {
        var nameEQ = name + "=";
        var ca = document.cookie.split(';');
        for(var i=0;i < ca.length;i++) {
            var c = ca[i];
            while (c.charAt(0)==' ') c = c.substring(1,c.length);
            if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length);
        }
        return null;
    };  
    
    var getHMAC = function(key, timestamp) {
        var hash = CryptoJS.HmacSHA1(key+timestamp, apiSecretKey);
        return hash.toString();
    };

    var getMicrotime = function (get_as_float) {

      var now = new Date().getTime() / 1000;
      var s = parseInt(now, 10);

      return (get_as_float) ? now : (Math.round((now - s) * 1000) / 1000) + ' ' + s;
    };    
    
    return {
        init:init
    };

})(jQuery);

So here in our JavaScript we have functions to hit the login route and the test route. Our getHMAC() function will create the hash for us using the same API secret key that we had in our REST API middleware. The readCookie() function is just used to read the value set for us by the API upon login and will be used in creating the hash. Also, we mentioned previously that we needed to write our own “microtime” function in JavaScript because there is no native one in JavaScript like there is in PHP. The getMicrotime function accomplishes this.

Then at the end of it all we can call app.init() to initialize because that is the property exposed globally that we set in our return statement…

app.init();

So those are some general basics of using HMAC authentication for a REST API. Our demo was pretty simplistic but the important components are there and could easily be expanded upon to build a more sophisticated system using a database and a client-side framework such as Backbone.js, Ember.js or AngularJS. You can download the files below and run them in in your localhost development server. Note that the Slim Framework requires at least PHP 5.3 to run.

Download Files

Hopefully you have found this helpful. Until next time!

, , , , , 9bit Studios E-Books

Like this post? How about a share?

Stay Updated with the 9bit Studios Newsletter

12 Responses to HMAC REST API Security

  1. Ryan says:

    Good article , how can you protect the secret key stored in JS ? . I want to expose a public api but concerned about client side JS exposing secret key.

    • Ian says:

      Hi there Ryan– Yeah I’d say that is definitely one of the challenges of using HMAC. At least it is for me. I’m not really an HMAC expert so I’d be interested to hear what other people have come up with using this method. A general application-wide secret key could easily be seen by anyone who signs up for an account, so you’d probably want a secret key specific to a user. In some of the articles I’ve read they’ve suggested a per-user secret key that you email to a user when they sign up and I guess they could input it in and you could store it in local storage (which won’t get sent with each request and is pretty well supported on all modern device browsers). But that’s not the best UX IMO because if they ever cleared all of their history they’d have to input it in again. From what I’ve read AWS uses HMAC and even they send the secret key over the wire once when a user signs in. So I guess that’s an option but seems potentially vulnerable to MITM attacks at least for that short window of time. So it seems like whatever path is chosen, there is some sort of trade off somewhere.

  2. Kermit says:

    Excellent article Ian. I’ve been thinking about securing REST APIs and I found your article very interesting. I’d like to point out something…

    You say that someone listening on the wire (man in the middle) would be able to see the cookie, but, as the apiSecretKey is never transferred, and the hash is different time to time (because you calculate the hash using a timestamp), the man in the middle will not be able to figure out the value of the apiSecretKey. That’s correct, ok.

    But, there is no need to figure out the value of the apiSecretKey! If the man in the middle is seen the cookie, the hash value (because it is set in a HTTP header) and the timestamp value (because it is set in another HTTP header), ha can recreate the same HTTP query to the server and he will get the response back to him. The man in the middle only need to replicate the HTTP query that he has sniffed and resend it (in less than 10 ms) to obtain the response from the server. Recalculating the hash on the server and comparing it to the one sent by the client do not authenticate something, because you are sending the result of the hashing as well as the parameters for calculating that hash.

    Am I right? What am I missing? Why setting a cookie and hashing ensures you that the query is valid?

    • Ian says:

      That’s a really good point Kermit and honestly it’s something that I’d wonder if other readers might be able to offer up a good solution for that scenario. Maybe we could use the data we send in our hashing to protect against a MiTM doing a POST, PUT, or DELETE that is different from the one the genuine user is doing (I think), but yeah, as you said, someone could still GET a response back and it’s problematic if we’re sending sensitive data there. I think in that case you might want to look at other possibilities. It depends on your API and what you’re trying to accomplish. It’s not really my area of expertise and it’s something I’m trying to learn more about but I’m just not 100% sure how a REST API can stay secure without resorting to other methods… oAuth, SSL, (or just cheat and use sessions). I’m open to suggestions and hopefully this article starts a good conversation around this topic 🙂

      • Alex_PK says:

        HMAC is not made to protect against MITM attacks.
        If you are a MITM you don’t need to recreate the request and resend it within 10sec: you just listen to the response and use the data invisibly.

        You need SSL to protect against MITM.

        HMAC is a protocol made to “protect the server”. If you don’t have the hashing key you can’t forge new (different) requests. This limits the attack possibility a lot, as you have a limited timeframe AND it’s impossible to try and get different data.

        Disclaimer: I am NOT an expert either. This is what I understood implementing it on an API recently.

  3. chris says:

    Hi,Ian, it is a nice article, I know HMAC is used for API authentication very well, but I am a bit confused about how to use it for user authentication. My understanding is when the user successfully logins into the web server with his credential, the server will return the PUBLIC-KEY and store in the cookie. But then I have got two puzzles
    1.apart from the PUBLIC-KEY sent over the wire, anything more need to be sent over, like the calculated HMAC hash ? http://blog.earaya.com/blog/2012/08/14/hmac-user-authentication/ tells to also send the hash.

    2. apiSecretKey is important to be shared between client and server. in your case, I think the apiSecretKey is not sent over after user login, it is already in the client side. if the apiSecretKey is assumed to be known by the client before the login, this is more like the case of API authentication rather than user authentication.

    so can you tell me the difference between user authentication and API authentication with HMAC

    Thanks, best rgds

    • Ian says:

      Hi Chris. Sorry about the delay in response. I have been super busy through the holidays and other obligations. But I feel like now I have a little bit of breathing room 🙂 To answer your question, I don’t know if I know the distinction for sure. I’m not really the expert on this but from what I’ve read up on, it seems like a lot of different solutions are offered up for answering the question “Can this user make this request?” using this method. Whether that’s user authentication or API authentication I’m not sure. Like the post you pointed me to, the articles linked in this post also talk about what some organizations do to solve the problem depending on the API and your environment. I’m sure it’s different in a lot of cases and there’s probably no one exact way to do it that will be a fit for all circumstances.

  4. Alex Litvinenko says:

    Hello,

    I propose to replace

    if($timeDiff <= 10)

    with

    if(abs($timeDiff) app->request();   
      $clientMicrotime = $req->headers('X-MICROTIME');
      $serverMicrotime = microtime(true);
      $timeDiff = $serverMicrotime - $clientMicrotime;
         
      if($timeDiff <= 10) 
        return true;
      else
        return false;
    }
  5. s.shivasurya says:

    Wonderful article! i was worried how to authenticate users for my smartphone apps ! searching for a long time for HMAC php/JS article! found this today!
    Keep continuing write articles on REST api security! and other things too.

    – s.shivasurya,INDIA

  6. guysung says:

    Thanks for the great article in advance.

    In the code of client-side above, var apiSecretKey = ‘ABC123’ has been exposed.

    Is it ok, It looks very vulnerable to me because anybody can get the apiSecretKey by browsing the source code in a browser along with the public key as a cookie.

    FYI, I am not an security expert and trying to learn about it.

    Please let me know if I am missing something.

    Thanks much for sharing this again.

    • Ian says:

      Hey there guysung — No, you’re not missing anything. It’s stored in a variable for the demo but in a real application you’d want to get this value to a user somehow and store it somewhere (maybe local storage… which is not sent with each request). And it would be unique to each user, not publicly available in the client-side source. When the application starts, you could try to read this value and if it’s not there prompt the user to enter their key. Maybe you e-mail it to them when they sign up for your app. From what I’ve read AWS uses HMAC and they actually *do* send the secret key over the wire one time when a user signs in. So that apparently is another option too although it seems like that could possibly be vulnerable as well. But as I said, I’m not the expert. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *