HMAC verification issues

emn
Shopify Partner
9 0 7

I'm trying to validate the requests coming from Shopify but I'm unable to do so.

Here's the code I'm using to get the hmac hash from the request (from here😞

A protocol=http:// or https:// needs to be added to the message

$params = $request;
    
unset($params['signature']);
unset($params['hmac']);

$collected = array_map(function($key, $value) {
    return $key . "=" . $value;
}, array_keys($params), $params);

asort($collected);
$collected = implode('&', $collected);

$shared_secret = 'MySecretHere';
return hash_hmac('sha256', $collected, $shared_secret);

I'm even following the example provided in https://docs.shopify.com/api/authentication/oauth#verification

$message = 'shop=some-shop.myshopify.com&timestamp=1337178173';
$secret = "hush";
var_dump(hash_hmac('sha256', $message, $secret), "2cb1a277650a659f1b11e92a4a64275b128e037f2c3390e3c8fd2d8721dac9e2"); die();

the hash resulting from that code is: c2812f39f84c32c2edaded339a1388abc9829babf351b684ab797f04cd94d4c7

I have run the same example using ruby (as you suggest on the docs page), creating a file called 'hmacruby' with:

require 'openssl'

digest = OpenSSL::Digest.new('sha256')
secret = "hush"
message = "shop=some-shop.myshopify.com&timestamp=1337178173"

digest = OpenSSL::HMAC.hexdigest(digest, secret, message)

p digest

using the command line went to the same folder where that file is stored in and run:

$ ruby hmacruby

the result was: c2812f39f84c32c2edaded339a1388abc9829babf351b684ab797f04cd94d4c7

I also have tested the algorithm here and I'm having the same result. What is going on? 

SOLUTION:

As Zelf is pointing out here:

 

post app install, Shopify will no longer send the "code" parameter that the docs refer to. In fact now they send a "protocol" parameter, which is not in the docs. However, the docs are still correct, you need to grab all parameters and remove signature and hmac, then whatever is left over, simply sort lexicographically, then hash_hmac with secret token like in the code example above.


After install, adding "protocol=https://&"; to the beginning of the message works. so the entire message NEEDS TO HAVE every param received from Shopify + the protocol (which should be 'https://'), which should be added first thing.

"protocol=https://&shop=yourshop.myshopify.com&timestamp=1439924556"; including 'protocol='

Replies 54 (54)

emn
Shopify Partner
9 0 7

Hi Shayne!

full request:

array(3) {
["shop"]=> string(31) "my-devstore.myshopify.com"
["hmac"]=> string(64) "7e5bed48e9a7773204cbad9052993e7ddf837593b878e3a4208f597651b0a9fb"
["timestamp"]=> string(10) "1439924556"
}

what I'm trying to validate: "shop=praxisis-devstore.myshopify.com&timestamp=1439924556"

shopify's hmac: 7e5bed48e9a7773204cbad9052993e7ddf837593b878e3a4208f597651b0a9fb

my hmac: ebf1aecf5e396905c0b1af69f058390491d99f9af2508633661e483e2e03015a

Thanks!

emn
Shopify Partner
9 0 7

XXXXXXX is the app name.

emn
Shopify Partner
9 0 7

I'm using the one you need to click on the input field to see it (it says 'show secret' otherwise) Starts with "xxxx".

Running:

require 'openssl'

digest = OpenSSL::Digest.new('sha256')
secret = "xxxxTHERESTOFTHEKEY"
message = "shop=my-devstore.myshopify.com&timestamp=1439924556"

digest = OpenSSL::HMAC.hexdigest(digest, secret, message)

p digest

I'm getting the same hash I get with php: ebf1aecf5e396905c0b1af69f058390491d99f9af2508633661e483e2e03015a

emn
Shopify Partner
9 0 7

Adding "protocol=https://&"; to the begging of the message works. so the entire message NEEDS TO BE

"protocol=https://&shop=praxisis-devstore.myshopify.com&timestamp=1439924556"; including 'protocol='

Somebody needs to update this page, I've wasted half of my day dealing with this issue.

Thank you Shayne.

Mark_Borromeo
Shopify Partner
15 0 6

Nothing works for me atm. I get a return map like:

 

            [url] => shopify/callback
            [code] => 195fdf64272c1cb25e6008dcacf17584
            [hmac] => 346308335956bd1ed4dfb9ce514502cf61d122e76e754faa44263147b6d05678
            [shop] => bright-therapy-shopify-2016.myshopify.com
            [signature] => 95ca7fc2a3cdf32b288d75c929af2bae
            [state] => 1234560
            [timestamp] => 1448916681

I've removed hmac & signature, sorted it out, concatenated etc. but the return hmac is still different.

Dan_Merns
Shopify Partner
2 0 0

Worked for me without adding 

protocol=https://&;

Useful thread in any case.  The example provided in the API docs makes it seem as though you only have to pass the "shop" and "timestamp" values.  You also have to include the "code" and "state" query string values.  I'm guessing that the example is outdated.

Mark_Borromeo
Shopify Partner
15 0 6

Hi Dan, are you getting the same return map as me? and can you share your url combine? thank you.

sasi_varna_kuma
Shopify Partner
1 0 1

Hi Mark, Try without sorting the params.

The API explains neatly what should be done. We should not interpret the logic just from the example provided. People are mis-interpretting with the example provided that "only shop and timestamp are needed to compute hmac". That is not the case. Go back and read the API specification again and forget the example.

Below code works for me
 

function compute_hmac($request_vars ){

	if (isset($request_vars['signature'])) {
		unset($request_vars['signature']);	
	}
	
	if (isset($request_vars['hmac'])) {
		unset($request_vars['hmac']);	
	}
	$compute_array = array();

	foreach ($request_vars as  $k => $val) {
		$k = str_replace('%', '%25', $k);
		$k = str_replace('&', '%26', $k);	
		$k = str_replace('=', '%3D', $k);
		$val = str_replace('%', '%25', $val);
		$val = str_replace('&', '%26', $val);	
		$compute_array[$k] = $val;
	}

	$message = http_build_query($compute_array);
	$key = Config::get('app.shopify_client_secret');
	
	$digest = hash_hmac ( 'sha256' , $message , $key , false ) ; 

	return $digest;
}

 

Alex_Lacayo
Shopify Partner
4 0 0

Just struggled with this but got it figured out.

			$queryString = http_build_query(array('code' => $da['code'], 'shop' => $da['shop'], 'state' => $da['state'], 'timestamp' => $da['timestamp']));
			$match = $da['hmac'];
			$calculated = hash_hmac('sha256', $queryString, $this->_API['API_SECRET']);

 

Alex_Lacayo
Shopify Partner
4 0 0
			$queryString = http_build_query(array('code' => $da['code'], 'shop' => $da['shop'], 'state' => $da['state'], 'timestamp' => $da['timestamp']));
			$match = $da['hmac'];
			$calculated = hash_hmac('sha256', $queryString, $this->_API['API_SECRET']);

 

Mark_Borromeo
Shopify Partner
15 0 6

Hi sasi varna kumar,

I don't know if you've read the documentation properly. To quote, it actually says "The list of key-value pairs is sorted lexicographically, and concatenated together with".

Still, not working for me. So I've preferred using MD5. That works. I don't care if it stops on June. Since nobody cares.

 

Regards,

Mark

cookseytalbott
Visitor
3 0 0

This worked well and pointed out the failure of the documentation on the process.

The docs on HMAC validation really should be adjusted to the current reality because they are incorrect.
 

Jamie_D_
Shopify Staff (Retired)
533 1 92

Hi guys,

Please be aware that MD5 signature validation is now deprecated as of June 1st as per this announcement.

The example provided in the OAuth documentation is not meant to be representative of all requests sent by Shopify, and the parameters are subject to change. The important steps to follow for HMAC signature validation are the following:

  1. Retrieve _all_ of the request parameters sent from Shopify (not just the parameters shown in the example)
  2. Convert the parameters to a hash (or similar data structure)
  3. Remove the `hmac` parameter from the hash.
  4. Sort the keys lexographically (each key-value pair is joined by an '=' character) and concatenate the key-value pairs with an ampersand('&').
  5. Calculate the SHA256 hash of this digest using the application's secret key as the encryption key.

Hope this helps. If you continue to experience any issues with HMAC validation, please reply to this thread.

To learn more visit the Shopify Help Center or the Community Blog.

Mark_Borromeo
Shopify Partner
15 0 6

hi Jamie,

thank you for responding back. I mean, have you made a very simple test in PHP? Did you tried your process on validating through HMAC? If not man, I hope we can at least follow through.

Gaurav_Jain1
Shopify Partner
13 0 0

Hi Jamie,I have used below code in php,

$request_vars = array('code' => $code, 'shop' => $shop, 'timestamp' => $timestamp);

$compute_array = array();

foreach ($request_vars as  $k => $val) {
	$k 	 = str_replace('%', '%25', $k);
	$k 	 = str_replace('&', '%26', $k);	
	$k 	 = str_replace('=', '%3D', $k);
	$val = str_replace('%', '%25', $val);
	$val = str_replace('&', '%26', $val);	
	$compute_array[$k] = $val;
}

$message = http_build_query($compute_array);
$key 	 = 'MY SECRET';
		
$digest = hash_hmac('sha256' , $message , $key ); 

But still it is giving wrong hmac. Any idea why ?

jcrowe
New Member
8 0 0

Here is the PHP class I am wrote for hmac calculation that has been working for us. Note there is a different signing method when calculating the hmac for a webhook, simply use the 
signRequestData method when handling the authentication flow. When passing data in to the webhook signer method you should use

file_get_contents('php://input'))

When passing data into the signRequestData method you sould pass in all of the request params, use the super global $_REQUEST or your frameworks `Request::all()` method

<?php

class ShopifyRequestSigner
{

    /**
     * @var string
     */
    private $secretKey;

    /**
     * ShopifyRequestSigner constructor.
     *
     * @param $secretKey
     */
    public function __construct($secretKey)
    {
        $this->secretKey = $secretKey;
    }


    /**
     * @return mixed
     */
    public function signWebhookRequestData($data)
    {
        return base64_encode(hash_hmac('sha256', $data, $this->secretKey, true));
    }


    /**
     * @param array $data
     *
     * @return string
     */
    public function signRequestData(array $data)
    {
        $data = $this->stripSignatureAndHmac($data);

        $dataString = $this->getString($data);

        return hash_hmac('sha256', $dataString, $this->secretKey);
    }

    /**
     * @param array $data
     *
     * @return array
     */
    private function stripSignatureAndHmac(array $data)
    {
        unset($data['signature'], $data['hmac']);

        return $data;
    }

    /**
     * @param array $data
     *
     * @return string
     */
    private function getString(array $data)
    {
        $ret = '';

        $encodedData = $this->encodeData($data);

        $sortedData = $this->sortData($encodedData);

        foreach ($sortedData as $key => $value) {

            $value = is_array($value) ? $this->getString($value) : $value;

            $ret .= $key . '=' . $value . '&';
        }

        return rtrim($ret, '&');
    }


    /**
     * @param array $data
     *
     * @return array
     */
    private function encodeData(array $data)
    {
        $encoded = [];

        foreach ($data as $key => $value) {

            $encoded[$this->encodeKey($key)] = $this->encodeValue($value);
        }

        return $encoded;
    }


    /**
     * @param $string
     *
     * @return string
     */
    private function encodeValue($string)
    {
        return $this->encodeString($string);
    }


    /**
     * @param $string
     *
     * @return mixed
     */
    private function encodeKey($string)
    {
        $encoded = $this->encodeString($string);

        return str_replace('=', '%3D', $encoded);
    }


    /**
     * @param $string
     *
     * @return string
     */
    private function encodeString($string)
    {
        return str_replace(['&', '%'], ['%26', '%25'], $string);
    }


    /**
     * @param array $encodedData
     *
     * @return array
     */
    private function sortData(array $encodedData)
    {
        ksort($encodedData);

        return $encodedData;
    }
}

 

Jamie_D_
Shopify Staff (Retired)
533 1 92

Hey all,

If you're still having difficulties, the examples posted by sasi and jcrowe should help.

Please make sure that you're computing the digest using all of the parameters sent by Shopify. You should not be hardcoding any parameters as they will not be the same in all cases.

Cheers,

Jamie
Developer Support

To learn more visit the Shopify Help Center or the Community Blog.

Gaurav_Jain1
Shopify Partner
13 0 0

Hello Jamie,

Hmac matching doesn't work when I send state variable as below

'state='+encodeURIComponent('security_token=***&&url=http://redirect.mydomain.com/shopify.php')

And I do this becuase my application is a SAAS based, where domains are dynamic and I can not set all domains in App redirect uri. And there is only state variable which comes back as it is. So I added token and an url in that vairable. But in this case hmac calulation doesn't work.

Any idea why ?

Ryan86
Visitor
1 0 0

Make sure you don't have two API secret keys in partner central, and if you do, that you're validating it against the correct one.

Benjamin_Hill
Shopify Partner
2 0 0
function calc_encrypt (bodytext) {
    var HMAC_KEY = "XXXXXXXXX";
    var HMAC_ALGORITHM = 'SHA256';
    var cipher = crypto.createHmac(HMAC_ALGORITHM,HMAC_KEY);
    var newtoken = cipher.update(bodytext).digest('base64');
    //console.log("cipher: " + cipher + " and token: " + newtoken);
    return newtoken;
}

I am using the same javascript code pasted above for both of the following validations.  When I use Shopify's send sample webhook notification, the code successfully validates the token (you can see that in the attached graphic).  When I instead place an order in the store and it sends the webhook, the hash does not match.  I will send a second post with an image of that log.  Both myself and another developer have been looking at this.  Please advise.

Benjamin_Hill
Shopify Partner
2 0 0

here is the log of the failed validation.  thanks!

Dylan_Stamat
Shopify Partner
1 0 0

I ran into this issue as well following:  https://help.shopify.com/api/guides/authentication/oauth

Once I added the "state" name value pair to the message, everything worked fine.  I needed to make sure the state name value pair was in the correct order in the string as well, for example (in ruby):

message = "code=#{params[:code]}&shop=#{params[:shop]}&state=#{params[:state]}&timestamp=#{params[:timestamp]}"

I know we're supposed to write a method to concat "all" name value pairs into a string and remove the hmac, but manually constructing the string is a lot easier and apparently what most people are doing here.  If this auth method isn't going to change, I'd suggest updating the docs to reflect just using a hard coded string.

Jamie_D_
Shopify Staff (Retired)
533 1 92

Hey Dylan,

I would actually specifically avoid doing just that, as stated in the documention:

This query string is merely an example, and the request parameters provided by Shopify could be subject to change at sometime in the future. Please ensure that your verification strategy is not dependent on the exact parameters in the example above.

To learn more visit the Shopify Help Center or the Community Blog.

Jamie_D_
Shopify Staff (Retired)
533 1 92

@Benjamin Hill,

Sending a "sample notification" from the Shopify admin will cause the webhook to be signed with the shop's secret key which is displayed in the Settings / Notifications section of your admin. You'll need to use this same key in order to verify these webhooks.

To learn more visit the Shopify Help Center or the Community Blog.

Zelf
Shopify Partner
40 0 16

In my forum post I was having the same issues. I listed the fix for my particular situation.

Ravi_Misra
Shopify Partner
2 0 0

Worked for me, thanks Zelf!!

Lolo1
Excursionist
44 0 11

Hello Shopifiers!

For your coding pleasure we humbly submit the Ultimate Shopify Oauth2 HMAC Validation Function (in PHP, of course), written with an appropriate level of paranoia and in obsessive compliance with the Shopify instructions. It also seems to work in the real world.

Questions, comments, changes, improvements, wlecome...

protected function oauth_request_is_valid($request) {

    /*
     * This function validates a shopify oauth2 callback request.
     *
     * If the request is valid, the function returns 'true'. If the request is
     * not valid, the function returns 'false'.
     */

    /*
     * Background: A shopify oauth2 callback is an http or https GET request
     * with a query string that normally looks something like the following:
     *
     * ?code=7517e09c5400f294d27ef9ed67673c67
     * &hmac=ddaa7ebf8af8b32e0a4282a3aa6d6e1ad455fd8fa2f4c84f57d177fde1480338
     * &shop=my-beloved-store.myshopify.com
     * &state=us6SNN35TjHWtYIsYyBcP2DxnA9PDFIR
     * &timestamp=1473738946
     */

    /*
     * "To verify that a request is valid, first parse the query string into
     * a map of keys to values."
     */

    /*
     * The following call returns all http request query parameters as an
     * associative array (a "map of keys to values"). It uses the laravel framework,
     * but it's basically equivalent to the following in raw PHP:
     * $arrayQueryParams = $_REQUEST;
     */
    $arrayQueryParams = $request->all();

    /*
     * Before we get into the nitty gritty hmac signature verification, let's make sure
     * the request at least includes the expected query parameters.
     */
    $all_params_set =
        isset($arrayQueryParams['code']) &&
        isset($arrayQueryParams['hmac']) &&
        isset($arrayQueryParams['shop']) &&
        isset($arrayQueryParams['state']) &&
        isset($arrayQueryParams['timestamp']);

    // if not, return false
    if ( ! $all_params_set) {
        return false;
    }

    /*
     * Check the 'timestamp' to make sure this isn't an old request that was recorded
     * and is being played back (extremely unlikely, but theoretically possible). We'll
     * give the request a short time-to-live between the time it was created and the time
     * that we received it, really just enough time to allow for request latency (very
     * small, low single-digit seconds max) and clock skew between the shopify system
     * and this system (should also be small assuming this system is properly set up and
     * managed).
     *
     * Note that the 'timestamp' value is a unix timestamp (number of seconds since the
     * beginning of the Unix Epoch). Unix time is referenced to Coordinated Universal Time
     * (UTC), so we don't have to worry about time zone differences.
     */
    $ttl_sec = 60;
    if (time() - $arrayQueryParams['timestamp'] > $ttl_sec) {
        return false;
    }

    // save the value of the 'hmac' parameter
    $hmac = $arrayQueryParams['hmac'];

    // "The hmac entry is removed from the map, leaving the remaining parameters:"
    unset($arrayQueryParams['hmac']);

    /*
     * "The characters & and % are replaced with %26 and %25 respectively in keys
     * and values. Additionally the = character is replaced with %3D in keys."
     */

    /*
     * Translate the query array values - yes, doing it in one php statement is showing
     * off a bit... Note that the '%' has to be the first element in "search" array argument
     * to the string_replace() function (['%', '&']), because the "replace" argument
     * (['%25', '%26']) contains '%' characters...
     */
    $arrayQueryParams = str_replace(['%', '&'], ['%25', '%26'], $arrayQueryParams);

    // translate the query array keys
    $arrayQueryParamsClean = array();
    foreach ($arrayQueryParams as $key => $val) {
        $clean_key = str_replace(['%', '&', '='], ['%25', '%26', '%3D'], $key);
        $arrayQueryParamsClean[$clean_key] = $val;
    }

    /*
     * "Each key is concatenated with its value, separated by an = character, to create
     * a list of strings. The list of key-value pairs is sorted lexicographically, and
     * concatenated together with & to create a single string:"
     */
    $arrayKeyEqualsVal = array();
    foreach ($arrayQueryParamsClean as $key => $val) {
        $arrayKeyEqualsVal[] = $key . '=' . $val;
    }

    // sort the array, by values, in ascending lexicographical order
    sort($arrayKeyEqualsVal, SORT_STRING);

    // create the message string
    $message = implode('&', $arrayKeyEqualsVal);

    /*
     * Get the "secret" (the shopify "client secret" asscoiated with this shopify app).
     *
     * Note that the config() function below is specific to the laravel php framework -
     * it just gets the client secret out of a stored configuration file...
     */
    $secret = config('services.shopify.clientSecret');

    // create the hmac digest string
    $digest = hash_hmac('sha256', $message, $secret);

    /*
     * if the calculated hmac digest string matches the value of the received 'hmac'
     * query parameter, return true, else return false
     */
    return $digest === $hmac;

}

 

 

Lolo1
Excursionist
44 0 11

I submitted a follow-up post but then I decided it was dumb, but I don't know how to delete the post so I'm putting some junk in here.

Vineet_Saini
Shopify Partner
1 0 1

My question is regarding Shopify Fulfillment service and HMAC validation. 

For my store, I registered a fulfillment service for Fetching Stock only (Inventory_management = true) by calling Restful API /admin/fulfillment_services.json. Registration was successful and i successfully received a request from Shopify when i configured a product to use the fulfillment service(Request received from Shopify provided underneath).

For HMAC validation, i have built the string as following

1. &shop=node1.myshopify.com&sku=10001111&timestamp=1484330132

2. shop=node1.myshopify.com&sku=10001111&timestamp=1484330132

But in both cases, my digest is not matching with X-Shopify-Hmac-Sha256  value.  Can someone specify the params that i need to consider for constructing the message used for digest calculation?

 

Here is the message:

Request params:

 

Array

(

    [shop] => node1.myshopify.com

    [sku] => 10001111

    [timestamp] => 1484330132

)

 

 

Headers:

Array

(

    [X-Shopify-Shop-Domain] => node1.myshopify.com

    [X-Shopify-Hmac-Sha256] => EPXbh1nGRMnA/DMs0vNIuwcycJ9p6Zd7qSTWwDlvIFo=

    [Accept-Encoding] => gzip;q=1.0,deflate;q=0.6,identity;q=0.3

    [Accept] => */*

    [User-Agent] => Ruby

    [X-Newrelic-Id] => VQQUUFNS

    [X-Newrelic-Transaction] => PxQFWFNQXgNSAgdaVVdWAFNTFB8EBw8RVU4aUAENB1QDB18HCQNVAwIBVUNKQQoBBAFUUAZVFTs=

    [Connection] => close

    [Host] => webdev.XXXXXXXX.com

)

James_Mulroy
Shopify Partner
4 0 0

Hello,

I've been looking for infomation about the hmac verification, and now a bit stuck. Is there any infomation about the data before it is hashed. I current am unable to get API calls with multiple IDS to verifiy. I can only think i am structing the data the wrong way..

For example :
This will verifiy.

id=9975017348&protocol=https://&shop=adicer-alpaca.myshopify.com&timestamp=1488319505

But
This will not.

ids=9975017348,9974987012&protocol=https://&shop=adicer-alpaca.myshopify.com&timestamp=1488319620

because of the "ids" and the multiple values, I can not find any documentation on the subject. This call happens when you use the .

ANy help would be very helpful

Matthew_Todd
Visitor
1 0 1

Hi,

I was also having an issue with that exact case, finding that the documentation does not mention how to verify URLs with that type of data in the query string.

I found a forum post here that explained how to resolve this and found that this approach worked for me.

Hope this helps.

James_Mulroy
Shopify Partner
4 0 0

Hey Matthew,

Can't thank you enough, i was about to give up on this.

Will give it a try and let you know, I will also update jcrowes code and upload it if it works.

Ta

James_Mulroy
Shopify Partner
4 0 0

Hey Matthew,

Whilst this is a step in the right direction, the forum post you link to isn't a compleate solution.

How exsactly did this "work for you"?

Ta

James_Mulroy
Shopify Partner
4 0 0

Hello,

After a bit of guess work and fiinally realising i was missing some white space somewhere I got an array to verify. Building further on jcrowes' work I have done the following.

<?php

class ShopifyRequestSigner
{

    /**
     * @var string
     */
    private $secretKey;

    /**
     * ShopifyRequestSigner constructor.
     *
     * @param $secretKey
     */
    public function __construct($secretKey)
    {
        $this->secretKey = $secretKey;
    }


    /**
     * @return mixed
     */
    public function signWebhookRequestData($data)
		
    {
		//echo "<XMP>DATA STRING:".$data."</XMP>";
        return base64_encode(hash_hmac('sha256', $data, $this->secretKey, true));
    }


    /**
     * @param array $data
     *
     * @return string
     */
    public function signRequestData(array $data)
    {
        $data = $this->stripSignatureAndHmac($data);

        $dataString = $this->getString($data);
		
		
		//echo "<XMP>DATA STRING:".$dataString."</XMP>";

        return hash_hmac('sha256', $dataString, $this->secretKey);
    }

    /**
     * @param array $data
     *
     * @return array
     */
    private function stripSignatureAndHmac(array $data)
    {
        unset($data['signature'], $data['hmac']);

        return $data;
    }

    /**
     * @param array $data
     *
     * @return string
     */
    private function getString(array $data)
    {
        $ret = '';

        $encodedData = $this->encodeData($data);

        $sortedData = $this->sortData($encodedData);
		
		

        foreach ($sortedData as $key => $value) {
			
			if (is_array($value)){
				sort($value);
				$value2 .= '["'; //Start json array
				$value2 .= implode('", "',$value);// Fill jason array, NOTE WHITE SPACE AFTER Comma seperator
				$value2 .= '"]'; //End json array
				
				$ret .= $key."="; //Name the jason array 
				$ret .= $value2; // stick the string into ret
				$ret .= "&"; // plonk a & a the end, assuming other params will follow, will be removed later if at the end of the string
				
			}else{
				
				$value = is_array($value) ? $this->getString($value) : $value;
				
				$ret .= $key . '=' . $value . '&';
			}
			
			

        }
		
		//echo "<Xmp>".$ret."</XMP>";

        return rtrim($ret, '&');
    }


    /**
     * @param array $data
     *
     * @return array
     */
    private function encodeData(array $data)
    {
        $encoded = [];

        foreach ($data as $key => $value) {

            $encoded[$this->encodeKey($key)] = $this->encodeValue($value);
        }

        return $encoded;
    }


    /**
     * @param $string
     *
     * @return string
     */
    private function encodeValue($string)
    {
        return $this->encodeString($string);
    }


    /**
     * @param $string
     *
     * @return mixed
     */
    private function encodeKey($string)
    {
        $encoded = $this->encodeString($string);

        return str_replace('=', '%3D', $encoded);
    }


    /**
     * @param $string
     *
     * @return string
     */
    private function encodeString($string)
    {
        return str_replace(['&', '%','[',']'], ['%26', '%25','%5B','%5D','%22'], $string);
    }


    /**
     * @param array $encodedData
     *
     * @return array
     */
    private function sortData(array $encodedData)
    {
        ksort($encodedData);

        return $encodedData;
    }
}

My additon could do with a tidyup but should help others.

Ta

Daniel_K_
Tourist
6 0 1

Hi

So it looks like that most of the people are using PHP here. I am on NodeJS and keep running into issues when validing signature for webhooks. Here are two raw payloads I've got.

orders/create: http://pastebin.com/vg6wd87T
orders/cancelled: http://pastebin.com/2g7X9EDR

It's the same order that got created and cancelled soon after. Now here is full NodeJS environment where you can run the code. Note that there is secret key included as this is coming from a test app/shop so I don't mind sharing that.

https://runkit.com/fredyc/58bec1834bcfaa00136ee413

You can see both payload calculated there with same algorithm and secret key along with expected signatures I've copied from a raw requests. In first case it matches, however in second one it does not.

So what am I missing here? Same algorithm, same secret key and yet different result than Shopify is telling me. There is either something small missing from my algorithm or there is a bug on Shopify side.

Can someone perhaps verify this with PHP? I don't know where I would test that really.

harish_bisht
Shopify Partner
5 0 0

Hi

I have followed the app page instruction, and still i am getting the false in app page hmac validation, what i am doing wrong in this code?

 

import hashlib, base64, hmac

def _hmac_is_valid(body, secret, hmac_to_verify):
    hash            = hmac.new(body, secret, hashlib.sha256)
    hmac_calculated = base64.b64encode(hash.digest())
    print hmac_calculated
    return hmac_calculated == hmac_to_verify

secret = '<my app secret>'
hmaccs = 'b578f0c82f01ac872d9a36155187e7a177a02c3c98d253f773b75a55ea51f489'
data='protocol=https%3A%2F%2F&shop=harish-30.myshopify.com&timestamp=1489213912'

print _hmac_is_valid(data,secret,hmaccs)

 

Aleks1
New Member
10 0 0

Hi Harish,

You might want to try hash.hexdigest() if you are doing 'normal' hmac validation.

For vaidating webhook hmacs I believe you would use .digest() as you are in your example.

The docs touch on this, but it isn't quite as obvious as it maybe could be:

"Lastly, this string processed through an HMAC-SHA256 using the Shared Secret as the key. The message is authentic if the generated hexdigest is equal to the value of the hmac parameter."

"The HMAC verification procedure for OAuth is different from the procedure for verifying webhooks. To learn more about HMAC verification for webhooks, see the documentation for Using Webhooks."

Hope that helps,

Aleks.

harish_bisht
Shopify Partner
5 0 0

Hi Aleks

I have used the .digest() but still, i am getting false in hmac verification

Thanks

Aleks1
New Member
10 0 0

Harish,

It looks like the code you're using is from/based on Gavin Ballard's webhook decorator example?

If so (and assuming you're definately not trying to validate a webhook), try the following changes/example (based on the same example as the Shopify docs- replace with your secret/data etc. obviously) and let me know how you get on:

import hashlib, base64, hmac

def _hmac_is_valid(body, secret, hmac_to_verify):
    hash            = hmac.new(secret, body, hashlib.sha256)
    hmac_calculated = hash.hexdigest()
    print(hmac_calculated)
    print(hmac_to_verify)
    return hmac_calculated == hmac_to_verify

secret = 'hush'
hmaccs = '4712bf92ffc2917d15a2f5a273e39f0116667419aa4b6ac0b3baaf26fa3c4d20'
data='code=0907a61c0c8d55e99db179b68161bc00&shop=some-shop.myshopify.com&timestamp=1337178173'

print(_hmac_is_valid(data,secret,hmaccs))

For reference, I've done the following changes to the original code:

1. Changed the order of the parameters supplier to hmac.new() (see github comments ref the Gavin Ballard code https://gist.github.com/gavinballard/8513270).

2. Removed the base64.b64encode

3. Changed to hash.hexdigest()

Cheers,

Aleks.

harish_bisht
Shopify Partner
5 0 0

Hi Aleks

Thanks, now its working and got my error,

I am using protocol=https%3A%2F%2F instead of protocol=https://

Thanks Again

Aleks1
New Member
10 0 0

No problem Harish, glad its working for you now.

Cheers,

Aleks.

Cravid
Shopify Partner
2 0 0

I've had some issues with validating the HMAC of proxy requests with custom array or object arguments, e.g., the example from the docs:

query_string = "extra=1&extra=2&shop=shop-name.myshopify.com&path_prefix=%2Fapps%2Fawesome_reviews&timestamp=1317327555&signature=a9718877bea71c2484f91608a7eaea1532bdf71f5c56825065fa4ccabe549ef3"

Shopify parses the query string differently to HTTP standards. Parsing this query string would usually result into 

array:7 [
  "extra" => "2"
  "shop" => "name.myshopify.com"
  "path_prefix" => "/apps/awesome_reviews"
  "timestamp" => "1317327555"
  "signature" => "a9718877bea71c2484f91608a7eaea1532bdf71f5c56825065fa4ccabe549ef3"
]

A query string like extra=1&extra=2 is actually not an array, however Shopify handles it as one.

The problem with this behaviour is, that when you acutally pass array or object data from, e.g., an JS script in an AJAX request, Shopify again parses the query string differently resulting in something like this:

query_string = "foo%5B%5D=1&foo%5B%5D=2&shop=name.myshopify.com&path_prefix=%2Fapps%2Fproxy&timestamp=1490355845&signature=xxxxx"

array:4 [
  "foo[]" => array:2 [
    0 => "1"
    1 => "2"
  ]
  "path_prefix" => "/apps/proxy"
  "shop" => "name.myshopify.com"
  "timestamp" => "1490355845"
]

// php example
hash_hmac('sha256', "foo[]=1,2path_prefix=/apps/proxyshop=name.myshopify.comtimestamp=1490355845", $secret, false)

Same if you use objects with associative keys or hashes in your JS or whatever code. If you just parse the query string as HTTP intends it to do (and all frameworks to for you), it won't produces the same HMAC or signature as Shopify calculates and passes it.

The following PHP snippet also works with arrays and objects passed within a Shopify signed request:

function isValidRequest($secret)
    {
        if (isset($_REQUEST['hmac'])) {
            $hmac = $_REQUEST['hmac'];
            $separator = '&';
        } elseif (isset($_REQUEST['signature'])) {
            $hmac = $_REQUEST['signature'];
            $separator = '';
        } else {
            return true;
        }

        /**
         * Parse HTTP query string.
         */
        $params = [];
        foreach (explode('&', urldecode($_SERVER['QUERY_STRING'])) as $part) {
            list($key, $value) = explode('=', $part);
            if (isset($params[$key])) {
                if (!is_array($params[$key])) {
                    $tmp = $params[$key];
                    $params[$key] = [$tmp];
                }
                $params[$key][] = $value;
            } else {
                $params[$key] = $value;
            }
        }
        unset($params['signature']);
        unset($params['hmac']);

        /**
         * Re-build query string without signature or HMAC argument.
         */
        $query = [];
        foreach ($params as $key => $value)
        {
            if (is_array($value)) {
                $query[$key] = $key . '=' . implode(',', $value);
            } else {
                $query[$key] = $key . '=' . $value;
            }
        }
        ksort($query);

        $queryString = implode($separator, $query);

        return $hmac === hash_hmac('sha256', $queryString, $secret, false);
    }

Notice the required urldecode before parsing the query string. Also in OAUTH requests, e.g., app installation, you use & as separator, in proxy requests you use nothing (just as above '').

@Shopify:

It really annoys that you name and calculate your HMAC differently. In API requests its HMAC, in app proxy requests its signature. In webhooks its base64 encoded and I can simply pass the data received as JSON parsed as an array to the hash_hmac function. For all other requests I have to perform your custom parsing way instead of simply passing the query data as every framework or also $_GET would prepare them. In OAUTH you use & as separator in the query string, in proxy requests you use none at all (directly binding the arguments).

Roshan
Visitor
3 0 0
  • We are having trouble in verification of hmac for web hooks . Its working for some webhook request but not for all. I do not understand what wrong ?

code snippet we are using to verify webhook request 

var isValidRequestForWebhook = function(hmac , req){
    var message = JSON.stringify(req.body);
    message = message.split('/').join('\\/');
    message = message.split('&').join('\\u0026');
    
    var calculatedHmac = crypto.createHmac('SHA256', app_secret).update(message.toString('utf8')).digest('base64');

    return (hmac == calculatedHmac);
}

 

Daniel_K_
Tourist
6 0 1

@Roshan They key is not to touch the `req.body` in any way. Just pass it directly to `update` method of crypto. That works 100% of times.

Roshan
Visitor
3 0 0

@Daniel Crypto update method only accept string /buffer so I have to convert 'req.body' to string . 

But after this its not working for even a single request

updated code  snippet - 

var isValidRequestForWebhook = function(hmac , req){
    var message = req.body;
    var calculatedHmac = crypto.createHmac('SHA256', app_secret).update(message.toString('utf8')).digest('base64');

    return (hmac == calculatedHmac);
}

Daniel_K_
Tourist
6 0 1

Why are you calling `toString('utf8')` there? I said just pass it there directly 🙂 I am using this in a production app and works just fine...

    function calculateHmac(payload, digestEncoding) {
        const hmac = crypto.createHmac('SHA256', shopifySecret)
        hmac.update(payload)
        return hmac.digest(digestEncoding)
    }

 

Michael51
Visitor
1 0 0

Hi all,

I don't seem to have any issues with the HMAC validation myself over GET requests, however POST requests did cause considerable confusion when validating the provided signature.

The reason for this is because we were using a framework wrapper to return all request input, which included POST data when the HTTP method used was such.

To help others avoid the same obvious issue, I felt it worth documenting here that you should only be using the GET (query params) to validate the signature.

Tien_Do
Excursionist
40 0 6

Since documentation is still incorrect, I'd like to add a missing point here:

The HMAC Validation section talks about 3 parameters needed to validate a HMAC, they're code, shop, timestamp. But actually after user clicks Install button, your app is called with one more parameter - state, and state is needed to generate a correct hash (to validate against original HMAC).

But the sample code from Node.JS tutorial still works well, because it doesn't care what left in the map after stringified query string, it only removes hmac, and non-exist signature parameters with delete statements.

I accidentantlly failed to validate HMAC and found document is incorrect because I didn't copy and paste the tutorial code but write it myself, and I didn't use delete but build the map with {} operator.

Keep your customers informed with the best deals, sales events, announcements... https://apps.shopify.com/sticky-promo-bar

Pawan_Mundra1
New Member
6 0 0

Hi All,

I tride all code you guys posted on this page But I did not get success and i also tried the shopify example of hmac code verfication but i can not get success.

I implement hmac code verfication in both php and ruby code provided by shopify. Ruby and PHP generate same hmac code but this hmac code does not mach with shopify hmac code.

So Plz help me about this and send me the solution for this.

Thanks and Regards
Pawan Mundra
+919993391413
pawan964@gmail.com

Pawan Mundra +919993391413 pawan964@gmail.com Tech Lead