MWS Post Request with Google Scripts - javascript

I am trying to make a post request through google scripts to amazon to collect information.
We are trying to get our orders to MWS and transfer them to sheets automatically.
I got to the last step which is signing the request.
A few things I wasnt sure about:
They say we use the secret key for hashing,I only see a client secret
and an access key id, which do I use?
Do I add the URL as part of whats getting signed? On the MWS Scratch pad they add this as shown:
POST
mws.amazonservices.com
/Orders/2013-09-01
Does it have to be on separate lines does it need post and the rest of the stuff. Its a little unclear.?
I read online that the sha256 byte code gets base64 encoded, not the string literal, is that true?
I tried to hash the string that amazon gave to me with an online tool and compare it to the hash they provided and the base64 encode, thing matched. I tried decoding as well, nothing matched
Can someone please send me an example that works, so I can understand what happens and how it works?
Thank you!
Below is what I have so far:
function POSTRequest() {
var url = 'https:mws.amazonservices.com/Orders/2013-09-01?';
var today = new Date();
var todayTime = ISODateString(today);
var yesterday = new Date();
yesterday.setDate(today.getDate() - 1);
yesterday.setHours(0,0,0,0);
var yesterdayTime = ISODateString(yesterday);
var dayBeforeYesterday = new Date();
dayBeforeYesterday.setDate(today.getDate() - 2);
dayBeforeYesterday.setHours(0,0,0,0);
var dayBeforeYesterdayTime = ISODateString(dayBeforeYesterday);
var unsignedURL =
'POST\r\nhttps:mws.amazonservices.com\r\n/Orders/2013-09-01\r\n'+
'AWSAccessKeyId=xxxxxxxxxxx' +
'&Action=ListOrders'+
'&CreatedAfter=' + dayBeforeYesterdayTime +
'&CreatedBefore' + yesterdayTime +
'&FulfillmentChannel.Channel.1=AFN' +
'&MWSAuthToken=xxxxxxxxxxxx'+
'&MarketplaceId.Id.1=ATVPDKIKX0DER' +
'&SellerId=xxxxxxxxxxx'+
'&SignatureMethod=HmacSHA256'+
'&SignatureVersion=2'+
'&Timestamp='+ ISODateString(new Date) +
'&Version=2013-09-0';
var formData = {
'AWSAccessKeyId' : 'xxxxxxxxx',
'Action' : "ListOrders",
'CreatedAfter' : dayBeforeYesterdayTime,
'CreatedBefore' : yesterdayTime,
'FulfillmentChannel.Channel.1' : 'AFN',
'MWSAuthToken' : 'xxxxxxxxxxxx',
'MarketplaceId.Id.1' : 'ATVPDKIKX0DER',
'SellerId' : 'xxxxxxxxxx',
'SignatureMethod' : 'HmacSHA256',
'SignatureVersion' : '2',
'Timestamp' : ISODateString(new Date),
'Version' : '2013-09-01',
'Signature' : calculatedSignature(unsignedURL)
};
var options = {
"method" : "post",
"muteHttpExceptions" : true,
"payload" : formData
};
var result = UrlFetchApp.fetch(url, options);
writeDataToXML(result);
Logger.log(result);
if (result.getResponseCode() == 200) {
writeDataToXML(result);
}
}
function calculatedSignature(url) {
var urlToSign = url;
var secret = "xxxxxxxxxxxxxxxxxxx";
var accesskeyid = 'xxxxxxxxxxxxxxx';
var byteSignature = Utilities.computeHmacSha256Signature(urlToSign, secret);
// convert byte array to hex string
var signature = byteSignature.reduce(function(str,chr){
chr = (chr < 0 ? chr + 256 : chr).toString(16);
return str + (chr.length==1?'0':'') + chr;
},'');
Logger.log("URL to sign: " + urlToSign);
Logger.log("");
Logger.log("byte " + byteSignature);
Logger.log("");
Logger.log("reg " + signature);
var byte64 = Utilities.base64Encode(byteSignature)
Logger.log("base64 byte " + Utilities.base64Encode(byteSignature));
Logger.log("");
Logger.log("base64 reg " + Utilities.base64Encode(signature));
return byte64;
}

Step 1, creating the string to be signed
The string_to_sign is the combination of the following:
The string POST followed by a NEWLINE character
The name of the host, mws.amazonservices.com, followed by a NEWLINE
The API URL, often just /, or somthing like /Orders/2013-09-01, followed by a NEWLINE
An alphabetical list of all parameters except Signature in URL encoding, like a=1&b=2, not followed by anything
The minimum parameters seem to be the following:
AWSAccessKeyId is a 20-character code provided by Amazon
Action is the name of your API call, like GetReport
SellerId is your 14-character seller ID
SignatureMethod is HmacSHA256
SignatureVersion is 2
Timestamp is a date like 20181231T23:59:59Z for one second before new year UTC
Version is the API version like 2013-09-01
additional parameters may be needed for your call, depending on the value of Action
Please note:
The NEWLINE character is just "\n", not "\r\n"
The name of the host should not include "https://" or "http://"
The Signature parameter is necessary for the actual call later (see step 3), but is not part of the string_to_sign.
Your string_to_sign should now look somewhat like this:
POST
mws.amazonservices.com
/Orders/2013-09-01
AWSAccessKeyId=12345678901234567890&Action=ListOrders&CreatedAfter .... &Version=2013-09-01
Step 2, signing that string
Calculate a SHA256 hash of above string using the 40-character Secret Key
Encode this hash using Base64
In Pseudocode: signature = Base64encode( SHA256( string_to_sign, secret_key ))
Step 3, send the call
Send a HTTPS POST request, using the full alphabetical list of parameters, now including above signature as Signature somewhere in the middle, because you need to keep ascending alphabetical order.
https://mws.amazonservices.com/Orders/2013-09-01?AWSAccessKeyId....Version=2013-09-01
Step 4, processing the result
You should be getting two things back: a response header and a XML document. Be sure to evaluate the HTTP status in the header as well as all contents of the XML document. Some error messages are hidden deeply in XML while HTTP returns "200 OK".
Step 5, Advanced Stuff
If you use calls that require you to send a document, like SendFeed, you need to do the following additional steps:
Calculate the MD5 hash of your document
Encode this hash using Base64
In Pseudocode: contentmd5= Base64encode( MD5( document ))
Add Content-Type: text/xml (or whatever fits your document) as HTTP header
Add Content-MD5: plus the base64 encoded hash as HTTP header

This code was a great help for building a similar application. I corrected some small issues to bring it up to work:
before signing the ISODate need to be worked out to replace the ":" chars
var todayTime = ISODateString(today);
var todayISO = todayTime;
var todayTime_ = todayTime.replace(":", "%3A");
todayTime = todayTime_.replace(":","%3A");
The same for the calculated signature (quick & dirty solution, replace only 3 appearences and need to updated for more chars)
Logger.log(unsignedURL);
var tmpsignature = calculatedSignature(unsignedURL);
var orsignature = tmpsignature;
// encode special chars
tmpsignature = encodeURIComponent(orsignature);
}
I added a header and dropped the form, put all parameters in the url
var header = {
"x-amazon-user-agent": "GoogleSheets/1.0 (Language=Javascript)",
"Content-Type": "application/x-www-form-urlencoded"
// "Content-Type": "text/xml"
};
var options = {
"method" : "post",
"muteHttpExceptions" : true,
// "payload" : formData,
"header":header
};
And changed the call to url encoded (Form was not working, do not know why)
var url = 'https:mws-eu.amazonservices.com/Orders/2013-09-01?'+
'AWSAccessKeyId=<your stuff>'+
'&Action=GetOrder'+
'&SellerId=<your stuff>+
'&MWSAuthToken=<your token>'+
'&SignatureVersion=2'+
'&Timestamp='+todayTime'+ // remember to replace the ":" thru hex
'&Version=2013-09-01'+
'&Signature='+ tmpsignature+
'&SignatureMethod=HmacSHA256'+
'&AmazonOrderId.Id.1='+<your order;
parsed the response:
var result = UrlFetchApp.fetch(url, options);
//writeDataToXML(result);
Logger.log(result);
var xml = result.getContentText();
var document = XmlService.parse(xml);
This worked! :-)
I checked the signature also with
https://mws-eu.amazonservices.com/scratchpad/index.html

Related

How to get raw output from SHA1 using JS as PHP does?

I'm trying to dynamically generate a security header at Postman pre-request script. To do so, I need to transform the following code snippet from PHP to JS.
$password = "SECRETPASSWORD";
$nonce = random_bytes(32);
date_default_timezone_set("UTC");
$created = date(DATE_ATOM);
$encodedNonce = base64_encode($nonce);
$passwordHash = base64_encode(sha1($nonce . $created . sha1($password, true), true));
(Note the true flag at php's sha1() function, forcing raw output).
I've coded this code snippet so far:
var uuid = require('uuid');
var CryptoJS = require('crypto-js');
var moment = require('moment');
// Generate messageId
var messageId = uuid.v4();
pm.environment.set('messageId', messageId);
// Generate nonce
var nonce = uuid.v4();
var encodedNonce = CryptoJS.enc.Base64.stringify(
CryptoJS.enc.Utf8.parse(nonce)
);
pm.environment.set('nonce', encodedNonce);
// Generate created
var created = moment().utc().format();
pm.environment.set('created', created);
// Generate password hash
var password = 'SECRETPASSWORD';
var rawSha1Password = Buffer.from(CryptoJS.SHA1(password).toString(CryptoJS.enc.Base64), "base64").toString("utf8");
var passwordHash = CryptoJS.SHA1(nonce + created + rawSha1Password).toString(CryptoJS.enc.Base64);
pm.environment.set('passwordHash', passwordHash);
My JS script is almost working, the only problem seems to be the sha1 generation. Taking the following example values:
password: SECRETPASSWORD
nonce: 55d61876-f882-42f0-b390-dc662a7e7279
created: 2021-01-21T18:19:32Z
The output from PHP is:
encodedNonce: NTVkNjE4NzYtZjg4Mi00MmYwLWIzOTAtZGM2NjJhN2U3Mjc5
passwordHash: olI18mUowhmeCwjb1FJNHtTHYDA=
But, the output from JS is:
encodedNonce: NTVkNjE4NzYtZjg4Mi00MmYwLWIzOTAtZGM2NjJhN2U3Mjc5
passwordHash: tk/uYkL/3Uq0oIkYO0nlBGnV/0E=
As you can see, the encodedNonce is built correctly; however the passwordHash value is different. As I'm using Postman, I have a limited JS libraries available.
Taking this into account, how can I get the same result as the PHP one?
In the line
var rawSha1Password = Buffer.from(CryptoJS.SHA1(password).toString(CryptoJS.enc.Base64), "base64").toString("utf8");
the password hash is read into a buffer and then UTF-8 decoded. The UTF-8 decoding generally corrupts the data, see here. A possible solution is to concatenate the WordArrays instead of the strings:
function getPasswordHash(test){
// Generate nonce
var nonceWA = !test ? CryptoJS.lib.WordArray.random(32) : CryptoJS.enc.Utf8.parse('55d61876-f882-42f0-b390-dc662a7e7279');
console.log('nonce (Base64): ' + nonceWA.toString(CryptoJS.enc.Base64));
// Generate created
var created = !test ? moment().utc().format('YYYY-MM-DDTHH:mm:ss[Z]') : '2021-01-21T18:19:32Z';
var createdWA = CryptoJS.enc.Utf8.parse(created);
console.log('created: ' + created);
// Hash password
var pwd = 'SECRETPASSWORD';
var pwdHashWA = CryptoJS.SHA1(pwd);
// Hash nonce + created + pwd
var passwordHash = CryptoJS.SHA1(nonceWA.concat(createdWA).concat(pwdHashWA)).toString(CryptoJS.enc.Base64);
console.log('passwordHash: ' + passwordHash);
}
getPasswordHash(true); // with testdata
getPasswordHash(false); // without testdata
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.0.0/crypto-js.min.js"></script>
When the code is executed, the first call of getPasswordHash() uses the test data for nonce and date (test=true), the second call applies a random nonce and the current date (test=false) . The call with the test data returns the same result as the PHP code:
nonce (Base64): NTVkNjE4NzYtZjg4Mi00MmYwLWIzOTAtZGM2NjJhN2U3Mjc5
created: 2021-01-21T18:19:32Z
passwordHash: olI18mUowhmeCwjb1FJNHtTHYDA=
CryptoJS implements CryptoJS.lib.WordArray.random() as CSPRNG, which is the counterpart of the PHP method random_bytes(). The use of the uuid library is therefore actually not necessary. I have chosen CryptoJS.lib.WordArray.random() in my example because it is closest to the PHP code.

url encoding php data into json object

Please read - this isn't just simply doing json_encode from php to javascript.
Our billing system uses Authorize.net. Using the Authorize.net API, in PHP we create a token request. The data in the request passes customer info, balance, billing address, etc - that data is sent directly with PHP. In response we get a token to be processed in PHP - and then that token is embedded into the HTML form input.
After the credit card form is submitted, we get a javascript json response back to the form to be processed by other JS functions. All works fine until we have a customer with the & in their company name (IE: Bar & Grill)
The & triggers an error only in the json response back to the form - specifically the error we get is: unexpected end of json input which causes the rest of the scripts to error out.
So, the issue is, does the customer data in the PHP token request need to be urlencoded - or is there a special way to handle the json response? From what I can tell, Authorize simply returns the exact customer data in the json response - so if we url encode it on the front end (before the token request is sent), then does that mean we also need to url decode the json response.
Its kind of a chicken and an egg which came first problem.
Authorize.net Token Request (in PHP):
$customerData = new AnetAPI\CustomerDataType();
$customerData->setType("business");
$customerData->setId($ss['authCustID']);
$customerData->setEmail($ii['cEmail']);
// Set the Bill To info for new payment type
$billTo = new AnetAPI\CustomerAddressType();
$billTo->setFirstName($ii['cFirstName']);
$billTo->setLastName($ii['cLastName']);
$billTo->setCompany($ii['cName']); // #1 <----- IE: "Bar & Grill"
$billTo->setAddress($ii['cAddress']. " " .$ii['cAddress1']);
$billTo->setCity($ii['cCity']);
$billTo->setState($ii['cState']);
$billTo->setZip($ii['cZip']);
$billTo->setCountry("USA");
// set shipping profile
$shipTo = clone $billTo ;
// extend billTop profile
$billTo->setFaxNumber('8005551212') ;
$billTo->setPhoneNumber($ii['cPhone']);
//create a transaction
$transactionRequestType = new AnetAPI\TransactionRequestType();
$transactionRequestType->setTransactionType("authCaptureTransaction");
$transactionRequestType->setAmount($ss['balance']);
$transactionRequestType->setCustomer($customerData) ;
$transactionRequestType->setOrder($order) ;
$transactionRequestType->setBillTo($billTo) ;
$transactionRequestType->setShipTo($shipTo) ;
// Build transaction request
$request = new AnetAPI\GetHostedPaymentPageRequest();
$request->setMerchantAuthentication($merchantAuthentication);
$request->setRefId($refId);
$request->setTransactionRequest($transactionRequestType);
//execute request
$controller = new AnetController\GetHostedPaymentPageController($request);
$response = $controller->executeWithApiResponse(\net\authorize\api\constants\ANetEnvironment::PRODUCTION); // SANDBOX or PRODUCTION
$gToken=[] ;
if (($response != null) && ($response->getMessages()->getResultCode() == "Ok")) {
//echo $response->getToken()."\n";
$gToken["Error"] = 0 ;
$gToken["Token"] = $response->getToken() ;
} else {
//echo "ERROR : Failed to get hosted payment page token\n";
$errorMessages = $response->getMessages()->getMessage();
//echo "RESPONSE : " . $errorMessages[0]->getCode() . " " .$errorMessages[0]->getText() . "\n";
$gToken["Error"] = 1 ;
$gToken["errMsg"] = $errorMessages[0]->getCode() . ": " .$errorMessages[0]->getText() ;
}
Form/JSON response handler:
AuthorizeNetPopup.onReceiveCommunication = function (querystr) {
var params = parseQueryString(querystr);
//console.log(params) ;
switch (params["action"]) {
case "successfulSave":
AuthorizeNetPopup.closePopup();
break;
case "cancel":
AuthorizeNetPopup.closePopup();
break;
case "transactResponse":
// 'response' is a string value
// encode it as an object, to be passed to global function
// that will decode it again for PHP
console.log(params["response"]) ;
var response = JSON.parse(params["response"]); // #2 <--- ERROR: unexpected end of json input
//var response = params["response"];
httpReceipt(response) ;
AuthorizeNetPopup.closePopup();
break;
case "resizeWindow":
var w = parseInt(params["width"]);
var h = parseInt(params["height"]);
var ifrm = document.getElementById("iframeAuthorizeNet");
ifrm.style.width = w.toString() + "px";
ifrm.style.height = h.toString() + "px";
centerPopup();
break;
}
};
Do I url encode data at #1 (in php section) or do I url encode the json response before processing it (at #2) - or both? Very confused on how to handle this. Do we need to encode the data of eachparameter being added to the token request - or can we just encode the entire request before its submitted? Regardless of which end of the communication gets encoding/decoding applied, what are the proper encoding/decoding calls?
UPDATE
To illustrate flow:
paymentPage.html ->PHP generates token, embeds token into page form, also has iframe for paymentPageFrame.html
paymentPageFrame.html -> iFrame communicator page, relays msgs between paymentPage.html and authorize.net
paymentPage.html -> javascript onReceiveCommunication to process messages coming from paymentPageFrame.html
Turns out that authorize.net is returning a URL string to paymentPageFrame.html - the string is not urlencoded. The string is then getting passed back to the parent onReceiveCommunication at which point its being parsed with a custom parser:
function parseQueryString(str) {
var vars = [];
var arr = str.split('&');
var pair;
for (var i = 0; i < arr.length; i++) {
pair = arr[i].split('=');
vars.push(pair[0]);
vars[pair[0]] = unescape(pair[1]);
}
return vars;
}
This custom parser is then causing a data value that has the & in it (IE: "company" : "Bar & Grill") to split the name in half ultimately leading to the invalid/unexpected end of json input.
The string being passsed back to the iFrame, which in turn gets passed to the parent is:
action=transactResponse&response={"accountType":"Visa","accountNumber":"XXXX0623","transId":"62830720474","responseCode":"1","authorization":"185778","shipTo":{"firstName":"Bob","lastName":"Jones","company":"Backyard Bar & Grill","address":"123 Main St ","city":"Tampa","state":"FL","zip":"33606","country":"USA"},"orderDescription":"2020-12-01 bill for All Regions/All Sites","customerId":"1002-0","totalAmount":"1.50","orderInvoiceNumber":"11386-0","dateTime":"2/2/2021 4:57:53 PM","refId":"ref1612285039"}
Now that the string is in the parent page, I am trying to figure the best way to encode it then parse it so it won't break on parsing when a data value has & in it (IE: Backyard Bar & Grill)
So far I am trying, without sucess:
var urlstr = encodeURIComponent(response) ; // encode it first
var newUrl = new URLSearchParams(urlstr) ; // then proper parse it
But when I try to access the parameters, it returns null:
consoole.log(newUrl.get('action')) ;
> null
Your problem is that you split on the & characters which is also inside the JSON response. For a quick solution I suggest something like:
var str = 'action=transactResponse&response={"accountType":"Visa","accountNumber":"XXXX0623","transId":"62830720474","responseCode":"1","authorization":"185778","shipTo":{"firstName":"Bob","lastName":"Jones","company":"Backyard Bar & Grill","address":"123 Main St ","city":"Tampa","state":"FL","zip":"33606","country":"USA"},"orderDescription":"2020-12-01 bill for All Regions/All Sites","customerId":"1002-0","totalAmount":"1.50","orderInvoiceNumber":"11386-0","dateTime":"2/2/2021 4:57:53 PM","refId":"ref1612285039"}';
var response = JSON.parse(str.replace('action=transactResponse&response=', ''));
console.log(response);
console.log(response.shipTo.company);
JS fiddle: https://jsfiddle.net/vx9zr1b7/
This just replaces away the first parameter and the name of the second, so only the value of the second (and thereby the value of "response") will remain. Note that this only works under the assumption that the response is always in this very same format, and does not have any additional parameters in the response.
I suggest you put it inside your case "transactResponse": block.
This will parse you the JSON:
{
accountNumber: "XXXX0623",
accountType: "Visa",
authorization: "185778",
customerId: "1002-0",
dateTime: "2/2/2021 4:57:53 PM",
orderDescription: "2020-12-01 bill for All Regions/All Sites",
orderInvoiceNumber: "11386-0",
refId: "ref1612285039",
responseCode: "1",
shipTo: {
address: "123 Main St ",
city: "Tampa",
company: "Backyard Bar & Grill",
country: "USA",
firstName: "Bob",
lastName: "Jones",
state: "FL",
zip: "33606"
},
totalAmount: "1.50",
transId: "62830720474"
}
Note that the & became an & in the process of parsing the JSON. Therefore, if you are doing anything else with it than just displaying it in the browser, I suggest you html_entity_decode() the value on the PHP side.
After much tinkering around, i finally wrote my own parser that:
A: if exists, capture the response JSON first
B: then parse the remainder of the URL params:
C: return all params and response object as a single object
var str = 'action=transactResponse&response={"accountType":"Visa","accountNumber":"XXXX0623","transId":"62830720474","responseCode":"1","authorization":"185778","shipTo":{"firstName":"Preston","lastName":"Rolinger","company":"Backyard & Grill","address":"2808 W Foster Ave S ","city":"Tampa","state":"FL","zip":"33611","country":"USA"},"orderDescription":"2020-12-01 bill for All Regions/All Sites","customerId":"1002-0","totalAmount":"1.50","orderInvoiceNumber":"11386-0","dateTime":"2/2/2021 4:57:53 PM","refId":"ref1612285039"}&moreKey="other value"' ;
function parseResponse(str) {
var resInfo = [], res = {} ;
if (str.match("&response=")) {
var params = str.split("&response=") ; // leading params & response obj
if (params[1].match("}&")) {
resInfo = params[1].split("}&") ; // splits off anything at end of obj
res.response = JSON.parse(resInfo[0] + "}") ; // add } to complete obj string again
} else {
res.response = JSON.parse(params[1]) ; // else it already is the json string
}
if (resInfo[1]) {
params = params[0]+ "&" +resInfo[1] ; // join url params back together
} else {
params = params[0] ;
}
} else {
params = str ; // else the str has no "response" in it, process just params
}
params = new URLSearchParams(encodeURI(params)) ; //encode then parse
params.forEach(function(value, key) {
res[key] = value ;
}) ;
return res ;
}
var response = parseResponse(str) ;
console.log(response) ;

Binance API Signature with Google Scripts

I am stuck on how to correctlly include the signitue into my get command based off of the Binance API within Google Scripts. What it states is
SIGNED endpoints require an additional parameter, signature, to be sent in the query string or request body.
Endpoints use HMAC SHA256 signatures. The HMAC SHA256 signature is a keyed HMAC SHA256 operation. Use your secretKey as the key and totalParams as the value for the HMAC operation.
The signature is not case sensitive.
totalParams is defined as the query string concatenated with the request body.
What I have is:
function BinanceTrades() {
var curTime = Number(new Date().getTime()).toFixed(0)
var sKey = Utilities.computeHmacSha256Signature('symbol=LTCBTC&timestamp=' + curTime, '**mySeceretKey**');
Logger.log(sKey)
var headers = {'X-MBX-APIKEY': '**myKey**'}
var data = UrlFetchApp.fetch("https://api.binance.com/api/v3/allOrders?signature=" + sKey + "&symbol=LTCBTC&timestamp=" + curTime, {'headers' : headers})
Logger.log(data)
}
and the error I get is:
{"code":-1100,"msg":"Illegal characters found in parameter 'signature'; legal range is '^[A-Fa-f0-9]{64}$'."}
I am unsure of how to compute the HMAC SHA256 correctly and incorporate the totalParams.
My previous post was this.
How about these modifications?
Modification points :
From the manual you provided
In your case, the string which is used for the signature is "symbol=LTCBTC&timestamp=" + curTime.
The signature is the string of the unsigned hexadecimal.
But at Google Apps Script, the data which was encrypted by Utilities.computeHmacSha256Signature() is the bytes array of the signed hexadecimal.
The modified script which reflected above points is as follows.
Modified script :
function BinanceTrades() {
var key = '**myKey**';
var secret = '**mySeceretKey**';
var curTime = Number(new Date().getTime()).toFixed(0);
var string = "symbol=LTCBTC&timestamp=" + curTime;
var sKey = Utilities.computeHmacSha256Signature(string, secret);
sKey = sKey.map(function(e) {
var v = (e < 0 ? e + 256 : e).toString(16);
return v.length == 1 ? "0" + v : v;
}).join("");
var params = {
'method': 'get',
'headers': {'X-MBX-APIKEY': key},
'muteHttpExceptions': true
};
var url = "https://api.binance.com/api/v3/allOrders?" + string + "&signature=" + sKey;
var data = UrlFetchApp.fetch(url, params);
Logger.log(data.getContentText())
}
Note :
About the encryption of signature, it has already confirmed that this script works fine from the manual you provided.
I have no account for binance.com. So I couldn't run this script. I'm sorry.
When you run this script, if the error occurs, can you show me the error messages?

Javascript URL Query String Logic

I have an events listing page which can be filtered by type and also by date using query string variables.
I am trying to achieve the following logic using javascript/jQuery.
I have a calendar which fires a function when updated. When fired I need to implement the following logic:
If the current URL contains ?filter= then add &dateStart= to the end of the URL.
If the current URL contains ?filter= AND &dateStart= then keep the current filter value but replace the date query string with a new one.
If the current URL contains ONLY ?dateStart= then replace it with the new one.
I have tried various methods to achieve this but I keep hitting the problem of appending information to the end of the URL rather than replacing parts of it.
Any help would be appreciated.
Thanks.
You can try something like this:
NOTE: not tested.
var newDateValue;
var myPath = window.location.pathname
//check if path contains the different variables
var containsFilter = myPath.indexOf("?filter=") != -1 ? true : false;
var containsAppendedDateStart = myPath.indexOf("&dateStart=" != -1 ? true : false;
var containsDateStart = myPath.indexOf("?dateStart=" != -1 ? true : false;
if(containsFilter && !containsAppendedDateStart){
// If the current URL contains ?filter= then add &dateStart= to the end of the URL.
window.location.replace(window.location.href + "&dateStart=");
}else if(containsFilter && containsAppendedDateStart){
//If the current URL contains ?filter= AND &dateStart= then keep the current filter value but replace the date query string with a new one.
newDateValue = 10; // add your new value here
var splittedPathArray = myPath.split("&dateStart=");
var newUrl = window.location.protocol + "//" + window.location.host + "/" + splittedPathArray[0] + "&dateStart=" + addNewValue;
window.location.replace(newUrl);
}else if(containsDateStart){
// If the current URL contains ONLY ?dateStart= then replace it with the new one.
newDateValue = 15;// add your new value here
var splittedPathArray = myPath.split("?dateStart=");
var newUrl = window.location.protocol + "//" + window.location.host + "/" + splittedPathArray[0] + "?dateStart=" + addNewValue;
}
You can achieve this more easy with native Web API or vanilla javascript than with jQuery. As far as jQuery don't provide any specific function to work with query strings.
The new URLSearchParams object provide a few methods to work more easily with URL query strings. In your case for example you'll need to do something like this:
function updateQueryString(queryString, dateStart) {
var queryString = new URLSearchParams(queryString);
queryString.has('dateStart')
? queryString.set('dateStart', dateStart)
: queryString.append('dateStart', dateStart);
return queryString.toString();
}
for this solution you'll need a polyfill
Sadly this is not yet implemented by the majority of web browsers and you'll need to "polyfill" the URLSearchParams object for this solution to work properly. You'll have to add this line to the <head> section in your html:
<script src="https://cdn.rawgit.com/inexorabletash/polyfill/v0.1.14/polyfill.min.js"></script>
You can find more information about the URLSearchParams in the Mozilla Developers Network Documentation, the WHATWG specification for the URL Standard or the specification by the W3C
solution without polyfill
​
If you don't like to use edge features you still can do it without any extra polyfill. It would look like this:
function updateQueryString(queryString, dateStart) {
var qsObject = {};
queryString
.substring(1) // ignore '?'
.split('&').forEach(function (param) {
param = param.split('=');
qsObject[param[0]] = param[1];
});
qsObject['dateStart'] = dateStart;
return '&' + Object.keys(qsObject)
.map(function (key) {
return key + '=' + qsObject[key];
})
.join('?');
}
Call whatever version of the updateQueryString function you rather like this:
updateQueryString(windonw.location.search, dateStart)

Trying to Validate URL Using JavaScript

I want to validate a URL and display message. Below is my code:
$("#pageUrl").keydown(function(){
$(".status").show();
var url = $("#pageUrl").val();
if(isValidURL(url)){
$.ajax({
type: "POST",
url: "demo.php",
data: "pageUrl="+ url,
success: function(msg){
if(msg == 1 ){
$(".status").html('<img src="images/success.gif"/><span><strong>SiteID:</strong>12345678901234456</span>');
}else{
$(".status").html('<img src="images/failure.gif"/>');
}
}
});
}else{
$(".status").html('<img src="images/failure.gif"/>');
}
});
function isValidURL(url){
var RegExp = /(ftp|http|https):\/\/(\w+:{0,1}\w*#)?(\S+)(:[0-9]+)?(\/|\/([\w#!:.?+=&%#!\-\/]))?/;
if(RegExp.test(url)){
return true;
}else{
return false;
}
}
My problem is now it will show an error message even when entering a proper URL until it matches regular expression, and it return true even if the URL is something like "http://wwww".
I appreciate your suggestions.
Someone mentioned the Jquery Validation plugin, seems overkill if you just want to validate the url, here is the line of regex from the plugin:
return this.optional(element) || /^(https?|ftp):\/\/(((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:)*#)?(((\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]))|((([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.)+(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.?)(:\d*)?)(\/((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)+(\/(([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)*)*)?)?(\?((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)|[\uE000-\uF8FF]|\/|\?)*)?(\#((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)|\/|\?)*)?$/i.test(value);
Here is where they got it from: http://projects.scottsplayground.com/iri/
Pointed out by #nhahtdh This has been updated to:
// Copyright (c) 2010-2013 Diego Perini, MIT licensed
// https://gist.github.com/dperini/729294
// see also https://mathiasbynens.be/demo/url-regex
// modified to allow protocol-relative URLs
return this.optional( element ) || /^(?:(?:(?:https?|ftp):)?\/\/)(?:\S+(?::\S*)?#)?(?:(?!(?:10|127)(?:\.\d{1,3}){3})(?!(?:169\.254|192\.168)(?:\.\d{1,3}){2})(?!172\.(?:1[6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(?:(?:[a-z\u00a1-\uffff0-9]-*)*[a-z\u00a1-\uffff0-9]+)(?:\.(?:[a-z\u00a1-\uffff0-9]-*)*[a-z\u00a1-\uffff0-9]+)*(?:\.(?:[a-z\u00a1-\uffff]{2,})).?)(?::\d{2,5})?(?:[/?#]\S*)?$/i.test( value );
source: https://github.com/jzaefferer/jquery-validation/blob/c1db10a34c0847c28a5bd30e3ee1117e137ca834/src/core.js#L1349
It's not practical to parse URLs using regex. A full implementation of the RFC1738 rules would result in an enormously long regex (assuming it's even possible). Certainly your current expression fails many valid URLs, and passes invalid ones.
Instead:
a. use a proper URL parser that actually follows the real rules. (I don't know of one for JavaScript; it would probably be overkill. You could do it on the server side though). Or,
b. just trim away any leading or trailing spaces, then check it has one of your preferred schemes on the front (typically ‘http://’ or ‘https://’), and leave it at that. Or,
c. attempt to use the URL and see what lies at the end, for example by sending it am HTTP HEAD request from the server-side. If you get a 404 or connection error, it's probably wrong.
it return true even if url is something like "http://wwww".
Well, that is indeed a perfectly valid URL.
If you want to check whether a hostname such as ‘wwww’ actually exists, you have no choice but to look it up in the DNS. Again, this would be server-side code.
function validateURL(textval) {
var urlregex = /^(https?|ftp):\/\/([a-zA-Z0-9.-]+(:[a-zA-Z0-9.&%$-]+)*#)*((25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])){3}|([a-zA-Z0-9-]+\.)*[a-zA-Z0-9-]+\.(com|edu|gov|int|mil|net|org|biz|arpa|info|name|pro|aero|coop|museum|[a-zA-Z]{2}))(:[0-9]+)*(\/($|[a-zA-Z0-9.,?'\\+&%$#=~_-]+))*$/;
return urlregex.test(textval);
}
This can return true for URLs like:
http://stackoverflow.com/questions/1303872/url-validation-using-javascript
or:
http://regexlib.com/DisplayPatterns.aspx?cattabindex=1&categoryId=2
I written also a URL validation function base on rfc1738 and rfc3986 to check http and https urls. I try to hold this modular, so it can be better maintained and adapted to own requirements.
The RegExp in one line is show at end of this post.
The RegExp accept HTTP and HTTPS URLs with some international domain or IPv4 number. IPv6 is not supported yet.
window.isValidURL = (function() {// wrapped in self calling function to prevent global pollution
//URL pattern based on rfc1738 and rfc3986
var rg_pctEncoded = "%[0-9a-fA-F]{2}";
var rg_protocol = "(http|https):\\/\\/";
var rg_userinfo = "([a-zA-Z0-9$\\-_.+!*'(),;:&=]|" + rg_pctEncoded + ")+" + "#";
var rg_decOctet = "(25[0-5]|2[0-4][0-9]|[0-1][0-9][0-9]|[1-9][0-9]|[0-9])"; // 0-255
var rg_ipv4address = "(" + rg_decOctet + "(\\." + rg_decOctet + "){3}" + ")";
var rg_hostname = "([a-zA-Z0-9\\-\\u00C0-\\u017F]+\\.)+([a-zA-Z]{2,})";
var rg_port = "[0-9]+";
var rg_hostport = "(" + rg_ipv4address + "|localhost|" + rg_hostname + ")(:" + rg_port + ")?";
// chars sets
// safe = "$" | "-" | "_" | "." | "+"
// extra = "!" | "*" | "'" | "(" | ")" | ","
// hsegment = *[ alpha | digit | safe | extra | ";" | ":" | "#" | "&" | "=" | escape ]
var rg_pchar = "a-zA-Z0-9$\\-_.+!*'(),;:#&=";
var rg_segment = "([" + rg_pchar + "]|" + rg_pctEncoded + ")*";
var rg_path = rg_segment + "(\\/" + rg_segment + ")*";
var rg_query = "\\?" + "([" + rg_pchar + "/?]|" + rg_pctEncoded + ")*";
var rg_fragment = "\\#" + "([" + rg_pchar + "/?]|" + rg_pctEncoded + ")*";
var rgHttpUrl = new RegExp(
"^"
+ rg_protocol
+ "(" + rg_userinfo + ")?"
+ rg_hostport
+ "(\\/"
+ "(" + rg_path + ")?"
+ "(" + rg_query + ")?"
+ "(" + rg_fragment + ")?"
+ ")?"
+ "$"
);
// export public function
return function (url) {
if (rgHttpUrl.test(url)) {
return true;
} else {
return false;
}
};
})();
RegExp in one line:
var rg = /^(http|https):\/\/(([a-zA-Z0-9$\-_.+!*'(),;:&=]|%[0-9a-fA-F]{2})+#)?(((25[0-5]|2[0-4][0-9]|[0-1][0-9][0-9]|[1-9][0-9]|[0-9])(\.(25[0-5]|2[0-4][0-9]|[0-1][0-9][0-9]|[1-9][0-9]|[0-9])){3})|localhost|([a-zA-Z0-9\-\u00C0-\u017F]+\.)+([a-zA-Z]{2,}))(:[0-9]+)?(\/(([a-zA-Z0-9$\-_.+!*'(),;:#&=]|%[0-9a-fA-F]{2})*(\/([a-zA-Z0-9$\-_.+!*'(),;:#&=]|%[0-9a-fA-F]{2})*)*)?(\?([a-zA-Z0-9$\-_.+!*'(),;:#&=\/?]|%[0-9a-fA-F]{2})*)?(\#([a-zA-Z0-9$\-_.+!*'(),;:#&=\/?]|%[0-9a-fA-F]{2})*)?)?$/;
In a similar situation I got away with this:
someUtils.validateURL = function(url) {
var parser = document.createElement('a');
try {
parser.href = url;
return !!parser.hostname;
} catch (e) {
return false;
}
};
i.e. why invent the wheel if browsers can do it for you? But, of course, this will only work in the browser.
there are various parts of parsed URL exactly how browser would interpret it:
parser.protocol; // => "http:"
parser.hostname; // => "example.com"
parser.port; // => "8080"
parser.pathname; // => "/path/"
parser.search; // => "?search=test"
parser.hash; // => "#hash"
parser.host; // => "example.com:3000"
Using these you can improve your validating function depending on the requirements. The only drawback is that it will accept relative URLs and use current page server's host and port. But you can use it for your advantage, by re-assembling the URL from parts and always passing it in full to your AJAX service.
What validateURL won't accept is invalid URL, e.g. http:\:8883 will return false, but :1234 is valid and is interpreted as http://pagehost.example.com/:1234 i.e. as a relative path.
UPDATE
This approach is no longer working with Chrome and other WebKit browsers. Even when URL is invalid, hostname is filled with some value, e.g. taken from base. It still helps to parse parts of URL, but will not allow to validate one.
Possible better no-own-parser approach is to use var parsedURL = new URL(url) and catch exceptions. See e.g. URL API. Supported by all major browsers and NodeJS, although still marked experimental.
best regex I found from http://angularjs.org/
var urlregex = /^(ftp|http|https):\/\/(\w+:{0,1}\w*#)?(\S+)(:[0-9]+)?(\/|\/([\w#!:.?+=&%#!\-\/]))?$/;
This is what worked for me:
function validateURL(value) {
return /^(https?|ftp):\/\/(((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:)*#)?(((\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]))|((([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.)+(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.?)(:\d*)?)(\/((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)+(\/(([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)*)*)?)?(\?((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)|[\uE000-\uF8FF]|\/|\?)*)?(\#((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|#)|\/|\?)*)?$/i.test(value);
}
from there is is just a matter of calling the function to get a true or false back:
validateURL(urltovalidate);
I know it's quite an old question but since it does not have any accepted answer, I suggest you to use the URI.js framework: https://github.com/medialize/URI.js
You can use it to check for malformed URI using a try/catch block:
function isValidURL(url)
{
try {
(new URI(url));
return true;
}
catch (e) {
// Malformed URI
return false;
}
}
Of course it will consider something like "%#" as a well formed relative URI... So I suggest you read the URI.js API to perform more checks, for example if you want to make sure that the user entered a well formed absolute URL you may do like this:
function isValidURL(url)
{
try {
var uri = new URI(url);
// URI has a scheme and a host
return (!!uri.scheme() && !!uri.host());
}
catch (e) {
// Malformed URI
return false;
}
}
Import in an npm package like
https://www.npmjs.com/package/valid-url
and use it to validate your url.
You can use the URL API that is recently standard. Browser support is sketchy at best, see the link. new URL(str) is guaranteed to throw TypeError for invalid URLs.
As stated above, http://wwww is a valid URL.
The URL API can be used to validate the structure of a URL string.
An error is thrown when trying to serialise an invalid URL string into a URL object. This could be abstracted into a helper function (Typescript snippet below):
function isValidURL(URL: string) : boolean {
try {
new URL(string);
return true;
} catch (err) { return false; }
}
isValidURL('https://www.google.com'); // returns true
isValidURL('localhost:3000'); // returns true
isValidURL('not-a-valid-url'); // returns false
isValidURL('google.com'); // returns false (see footnote)
If you strictly want HTTP / web links to be valid, we can simply add a condition to the return statement:
...
const url = new URL(string);
return url.protocol === 'https:' || url.protocol === 'http:';
...
Granted, this approach comes with a few caveats:
No support for the URL API in Internet Explorer (could be fixed with a polyfill)
Without additional checks, URLs without either a protocol or port are seen as invalid (e.g. google.com is invalid but google.com:3000 is OK). This may be an unintended behaviour for some usecases.
If you're looking for a more reliable regex, check out RegexLib. Here's the page you'd probably be interested in:
http://regexlib.com/Search.aspx?k=url
As for the error messages showing while the person is still typing, change the event from keydown to blur and then it will only check once the person moves to the next element.
var RegExp = (/^HTTP|HTTP|http(s)?:\/\/(www\.)?[A-Za-z0-9]+([\-\.]{1}[A-Za-z0-9]+)*\.[A-Za-z]{2,40}(:[0-9]{1,40})?(\/.*)?$/);
My solution:
function isValidUrl(t)
{
return t.match(/^(http|https|ftp):\/\/(([A-Z0-9][A-Z0-9_-]*)(\.[A-Z0-9][A-Z0-9_-]*)+)(:(\d+))?\/?/i)
}
Demo : http://jsbin.com/uzimeb/1/edit
function checkURL(value) {
var urlregex = new RegExp("^(http|https|ftp)\://([a-zA-Z0-9\.\-]+(\:[a-zA-Z0-9\.&%\$\-]+)*#)*((25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9])\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9]|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9]|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[0-9])|([a-zA-Z0-9\-]+\.)*[a-zA-Z0-9\-]+\.(com|edu|gov|int|mil|net|org|biz|arpa|info|name|pro|aero|coop|museum|[a-zA-Z]{2}))(\:[0-9]+)*(/($|[a-zA-Z0-9\.\,\?\'\\\+&%\$#\=~_\-]+))*$");
if (urlregex.test(value)) {
return (true);
}
return (false);
}
I have found a great resource for comparing different solutions:
https://mathiasbynens.be/demo/url-regex
According to that page, only solution from diegoperini passes all tests. Here is that regex:
_^(?:(?:https?|ftp)://)(?:\S+(?::\S*)?#)?(?:(?!10(?:\.\d{1,3}){3})(?!127(?:\.\d{1,3}){3})(?!169\.254(?:\.\d{1,3}){2})(?!192\.168(?:\.\d{1,3}){2})(?!172\.(?:1[6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(?:(?:[a-z\x{00a1}-\x{ffff}0-9]+-?)*[a-z\x{00a1}-\x{ffff}0-9]+)(?:\.(?:[a-z\x{00a1}-\x{ffff}0-9]+-?)*[a-z\x{00a1}-\x{ffff}0-9]+)*(?:\.(?:[a-z\x{00a1}-\x{ffff}]{2,})))(?::\d{2,5})?(?:/[^\s]*)?$_iuS
I checked a lot of url validators in google and no one works for me. For example I'd like to see valid on links like 'aa.com'. I like silly check for dot sign in string.
function isValidUri(str) {
var dotIndex = str.indexOf('.');
return (dotIndex > 0 && dotIndex < str.length - 2);
}
It should not stay on beginning and end of string (for now we don't have top level domain names with one character).
Here's a regular expression which might fit the bill (it's very long):
/^(?:\u0066\u0069\u006C\u0065\u003A\u002F{2}(?:\u002F{2}(?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*\u0040)?(?:\u005B(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){6}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){5}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){4}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A)?[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){3}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,3}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,4}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,5}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,6}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2})\u005D|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?\u002E)+[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?))(?:\u003A(?:\u0030-\u0035\u0030-\u0039{0,4}|\u0036\u0030-\u0034\u0030-\u0039{3}|\u0036\u0035\u0030-\u0034\u0030-\u0039{2}|\u0036\u0035\u0035\u0030-\u0032\u0030-\u0039|\u0036\u0035\u0035\u0033\u0030-\u0035))?(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*|\u002F(?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])+(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*)?|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])+(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*)|[\u0041-\u005A\u0061-\u007A][\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002B\u002D\u002E]*\u003A(?:\u002F{2}(?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*\u0040)?(?:\u005B(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){6}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){5}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){4}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A)?[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){3}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,3}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,4}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035]))|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,5}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}|(?:(?:[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4}\u003A){0,6}[\u0030-\u0039\u0041-\u0046\u0061-\u0066]{1,4})?\u003A{2})\u005D|(?:(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])\u002E){3}(?:[\u0030-\u0039]|[\u0031-\u0039][\u0030-\u0039]|\u0031[\u0030-\u0039]{2}|\u0032[\u0030-\u0034][\u0030-\u0039]|\u0032\u0035[\u0030-\u0035])|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?\u002E)+[\u0041-\u005A\u0061-\u007A\u0030-\u0039](?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D]+)?[\u0041-\u005A\u0061-\u007A\u0030-\u0039])?))(?:\u003A(?:\u0030-\u0035\u0030-\u0039{0,4}|\u0036\u0030-\u0034\u0030-\u0039{3}|\u0036\u0035\u0030-\u0034\u0030-\u0039{2}|\u0036\u0035\u0035\u0030-\u0032\u0030-\u0039|\u0036\u0035\u0035\u0033\u0030-\u0035))?(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*|\u002F(?:(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])+(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*)?|(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])+(?:\u002F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)*)(?:\u003F(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040\u002F\u003F]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)?(?:\u0023(?:[\u0041-\u005A\u0061-\u007A\u0030-\u0039\u002D\u002E\u005F\u007E\u0021\u0024\u0026\u0027\u0028\u0029\u002A\u002B\u002C\u003B\u003D\u003A\u0040\u002F\u003F]|\u0025[\u0030-\u0039\u0041-\u0046\u0061-\u0066][\u0030-\u0039\u0041-\u0046\u0061-\u0066])*)?)$/
There are some caveats to its usage, namely it does not validate URIs which contain additional information after the user name (e.g. "username:password"). Also, only IPv6 addresses can be contained within the IP literal syntax and the "IPvFuture" syntax is currently ignored and will not validate against this regular expression. Port numbers are also constrained to be between 0 and 65,535. Also, only the file scheme can use triple slashes (e.g. "file:///etc/sysconfig") and can ignore both the query and fragment parts of a URI. Finally, it is geared towards regular URIs and not IRIs, hence the extensive focus on the ASCII character set.
This regular expression could be expanded upon, but it's already complex and long enough as it is. I also cannot guarantee it's going to be "100% accurate" or "bug free", but it should correctly validate URIs for all schemes.
You will need to do additional verification for any scheme-specific requirements or do URI normalization as this regular expression will validate a very broad range of URIs.
Try edit your isValidURL function as follows:
function isValidURL(url) {
var encodedURL = encodeURIComponent(url);
var isValid = false;
$.ajax({
url: "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22" + encodedURL + "%22&format=json",
type: "get",
async: false,
dataType: "json",
success: function(data) {
isValid = data.query.results != null;
},
error: function(){
isValid = false;
}
});
return isValid;
}
This should do the trick.

Categories

Resources