I would like to create an authTicket for use with the Sinch Web SDK, as described in the docs (Authentication by your backend).
For the server-side code required to do this, Sinch provides at least two examples:
https://github.com/sinch/sinch-js-ticketgen
https://github.com/sinch/php-auth-ticket
The first step is JSON encoding. However, if I run this in JavaScript or PHP respectively, I get different results:
JavaScript JSON.stringify(userTicket)
{
"applicationKey":"XXXXXXXXXXXXXXXX",
"identity":{"type":"username","endpoint":"johndoe"},
"created":"2017-04-12T12:34:56.789Z",
"expiresIn":86400
}
PHP json_encode($userTicket)
{
"identity":{"type":"username","endpoint":"johndoe"},
"expiresIn":86400,
"applicationKey":"XXXXXXXXXXXXXXXX",
"created":"2017-04-12T12:34:56.789Z"
}`
(Please disregard the whitespace, this has to do with StackOverflow formatting. I'm asking about order of keys.)
Later, this output is supposed to go into a hash function. Since both JSON strings have the keys in different order, both inputs won't possibly result in the same hashed output.
What is the correct algorithm to compute the authTicket, especially when it comes to the JSON encoding part?
Related
I'm developing an API with expressJS. This API is a semi-weblog service, and clients can create, update and delete their posts and contents. I have a sec urity concern about implementing its post and patch routes.
If the user injects some JS code and sends it to API to store in Mongodb, could these codes affect our API? How can I prevent users from posting and patching requests with any code inside them?
I have found "xss-clean" middleware to sanitize the user input body, is it enough for this purpose?
Because it is very important to me to ensure that I am using the correct middleware to protect this API, I am asking this question.
If the user injects some JS code and sends it to API to store in Mongodb, could these codes affect our API?
Generally speaking: It won't.
The code come into express as a message body. It gets parsed by your middleware into a data structure where it will appear as a string. You then put that string in an object of structured data that you pass through the Mongodb client API which sends it to the database with any escaping that is needed.
I have found "xss-clean" middleware to sanitize the user input body, is it enough for this purpose?
XSS is an attack in which data injected into an HTML document contains special characters which are treated as special characters in HTML.
e.g.
<h1>{{ your_name }}</h1>
Where your_name is data that contains <script>...</script>.
This is generally dealt with by applying proper escaping to the data (at a very basic level that means replacing < with <).
XSS won't affect your API directly.
If your data is going to be taken out of the Mongodb store and injected into an HTML document, then XSS is a consideration.
xss-clean is a wrapper around xss-filters.
xss-filters looks (I've only glanced at it) like a good module and is designed to be used as an output filter (i.e. run just before you insert data into an HTML document).
xss-clean works as an input filter, which isn't a good approach. It makes your data HTML safe at the expense of making it not useful for any purpose other than HTML. You might want to use the data in an email, or generate a report in Excel format.
Is there any library / NPM module / code sample to verify and decode a JWT in React Native?
I know I can do it manually. But I was hoping something like node-jsonwebtoken existed. (It does not work in React Native.)
https://github.com/auth0/jwt-decode only decodes but does not verify.
There's the RSA-Sign JavaScript Library that does what you want, though it uses some rather large and slow crypto libraries written in pure JS.
If you want to work a little to get a simpler and faster result, use the Wep Crypto API to get the browser to verify the signature for you. The exact code will depend on the key algorithm and format, but essentially you decode the base64url-encoded signature at the bottom of the JWT, then you call crypto.subtle.importKey with ["verify"] as the last argument (passing the key in JWK format or whatever else you can get), and then you use the resulting key to call crypto.subtle.verify. You'll need to turn the (decoded) token's signature into an ArrayBuffer, which can be accomplished with:
buffer = new Uint8Array([...signature].map(c => c.charCodeAt(0)))
Avoid fetching the key over the same network and just before verifying the signature (if the key is referenced by the 'jku' or given in the 'jwk' header fields of the token), since verifying its integrity is rather tricky inside the browser. Better if you can hardcode it in your source code after manual checking.
I'm looking for a way to transfer the raw file data of any file-type with any possible content (By that I mean files and file-content are all user generated) both ways using xhr/ajax calls in a Backbone front-end against a Django back-end.
EDIT: Maybe the question is still unclear...
If you open a file in an IDE (such as Sublime), you can view and edit the actual code that comprises that file. I'm trying to put THAT raw content into a JSON so I can send to the browser, it can be modified, and then sent back.
I posted this question because I was under the impression that because the contents of these files can effectively be in ANY coding language that just stringify-ing the contents and sending it seems like a brittle solution that would be easy to break or exploit. Content could contain any number of ', ", { and } chars that would seem to break JSON formatting, and escaping those characters would leave artifacts within the code that would effectively break them (wouldn't it?).
If that assumption is wrong, THAT would also be an acceptable answer (so long as you could point out whatever it is I'm overlooking).
The project I'm working on is a browser-based IDE that will receive a complete file-structure from the server. Users can add/remove files, edit the content of those files, then save their changes back to the server. The sending/receiving all has to be handled via ajax/xhr calls.
Within Backbone, each "file" is instantiated as a model and stored in a Collection. The contents of the file would be stored as an attribute on the model.
Ideally, file content would still reliably throw all the appropriate events when changes are made.
Fetching contents should not be broken out into a separate call from the rest of the file model. I'd like to just use a single save/fetch call for sending/receiving files including the raw content.
Solutions that require Underscore/jQuery are fine, and I am able to bring in additional libraries if there is something available that specializes in managing that raw file data.
Interesting question. The code required to implement this would be quite involved, sorry that I'm not providing examples, but you seem like a decent programmer and should be able to implement what's mentioned below.
Regarding the sending of raw data through JSON, all you would need to do to make it JSON-safe and not break your code is to escape the special characters by stringyfying using Python's json.dumps & JavaScript's JSON.stringyfy. [1]
If you are concerned about some form of basic tamper-proofing, then light encoding of your data will fit the purpose, in addition to having the client and server pass a per-session token back and forth with JSON transfers to ensure that the JSON isn't forged from a malicious address.
If you want to check the end-to-end integrity of the data, then generate an md5 checksum and send it inside your JSON and then generate another md5 on arrival and compare with the one inside your JSON.
Base64 encoding: The size of your data would grow by 33% as it encodes four characters to represent three bytes of data.
Base85: Encodes four bytes as five characters and will grow your data by 25%, but uses much more processing overhead than Base64 in Python. That's a 8% improvement in data size, but at the expense of processing overhead. Also it's not string safe as double & single quotation marks, angle brackets, and ampersands cannot be used unescaped inside JSON, as it uses all 95 printable ASCII characters. Needs to be stringyfied before JSON transport. [2]
yEnc has as little as 2-3% overhead (depending on the frequency of identical bytes in the data), but is ruled out by impractical flaws (see [3]).
ZeroMQ Base-85, aka Z85. It's a string-safe variant of Base85, with a data overhead of 25%, which is better than Base64. No stringyfying necessary for sticking it into JSON. I highly recommended this encoding algorithm. [4] [5] [6]
If you're sending only small files (say a few KB), then the overhead of binary-to-text conversion will be acceptable. With files as large as a few Mbs, it might not be acceptable to have them grow by 25-33%. In this case you can try to compress them before sending. [7]
You can also send data to the server using multipart/form-data, but I can't see how this will work bi-directionally.
UPDATE
In conclusion, here's my solution's algorithm:
Sending data
Generate a session token and store it for the associated user upon
login (server), or retrieve from the session cookie (client)
Generate MD5 hash for the data for integrity checking during transport.
Encode the raw data with Z85 to add some basic tamper-proofing and JSON-friendliness.
Place the above inside a JSON and send POST when requested.
Reception
Grab JSON from POST
Retrieve session token from storage for the associated user (server), or retrieve from the session cookie (client).
Generate MD5 hash for the received data and test against MD5 in received JSON, reject or accept conditionally.
Z85-decode the data in received JSON to get raw data and store in file or DB (server) or process/display in GUI/IDE (client) as required.
References
[1] How to escape special characters in building a JSON string?
[2] Binary Data in JSON String. Something better than Base64
[3] https://en.wikipedia.org/wiki/YEnc
[4] http://rfc.zeromq.org/spec:32
[5] Z85 implementation in C/C++ https://github.com/artemkin/z85
[6] Z85 Python implementation of https://gist.github.com/minrk/6357188
[7] JavaScript zip library http://stuk.github.io/jszip/
[8] JavaScript Gzip SO JavaScript implementation of Gzip
AFAI am concerned a simple Base64 conversion will do it. Stringify, convert to base64, then pass it to the server and decode it there. Then you won't have the raw file transfer and you will still maintain your code simple.
I know this solution could seem a bit too simple, but think about it: many cryptographics algorithms can be broken given the right hardware. One of the most secure means would be through a digital certificate and then encrypt data with the private key and then send it over to the server. But, to reach this level of security every user of your application would have to have a digital certificate, which I think would be an excessive demand to your users.
So ask yourself, if implementing a really safe solution adds a lot of hassle to your users, why do you need a safe transfer at all? Based on that I reaffirm what I said before. A simple Base64 conversion will do. You can also use some other algotithms like SHA256 ou something to make it a litter bit safer.
If the only concern here is that the raw content of your code files (the "data" your model is storing), will cause some type of issue when stored in JSON, this is easily availed by escaping your data.
Stringifying your raw code file contents can cause issues as anything resembling JavaScript or JSON will be parsed into an actual JSON object. Your code file data can and should be stored simply as an esacaped string. Your fear here is that said string may contain characters that could break being stored in JavaScript inside a string, this is alleviated by escaping the entire string, and thus double, triple, quadruple, etc. escaping anything already escaped in the code file.
In essence it is important to remember here that raw code in a file is nothing but a glorified string when stored in a database, unless you are adding in-line metadata dynamically. It's just text, and doing standard escaping will make it safe to store in whatever format as a string (inside "" or '') in JSON.
I recommend reading this SO answer, as I also referenced it to verify what I already thought was correct:
How To Escape a JSON string containing newline characters using JavaScript
I'm creating an application with Node.js and Mongo DB, rendering the views with Swig.
I have a database of business names, addresses and geo location data that is being plotted onto a Google map with pins.
I'd like to stop users from easily copying the raw JSON data using view source, Firebug, Chrome Dev tools etc.
I'm not after bank grade security, just want to make it hard enough for most users to give up.
I have two routes of delivering the JSON package to the browser:
1) Using Swig, passing the JSON package directly to the view. Problem is that a simple view source will show the JSON.
2) Requesting the data with an AJAX call. In this scenario the data is easily accessible with Chrome Dev tools.
What are my options?
Base-64 encode the string.
Then you can just base64-decode it in JavaScript.
That should make it sufficiently unreadable, no real security though - of course.
Plus it's fast.
You need to take care with UTF-8 characters (e.g. German äöüÄÖÜ, or French èéàâôû)
e.g. like this in JavaScript:
var str = "äöüÄÖÜçéèñ";
var b64 = window.btoa(unescape(encodeURIComponent(str)))
console.log(b64);
var str2 = decodeURIComponent(escape(window.atob(b64)));
console.log(str2);
example:
var imgsrc = 'data:image/svg+xml;base64,' + btoa(unescape(encodeURIComponent(markup)));
var img = new Image(1, 1); // width, height values are optional params
img.src = imgsrc;
More secure variant:
Return encrypted base64 encoded JSON, plus the decryption algorithm, base64 encode them server-side, bit-shift it a few bits, return via ajax, then de-bitshift the string on the webpage, pass it to eval, which will give you the decrypt function, then decrypt the encrypted base64 string, then base-64 decode that string.
But that takes only a few seconds more on the chrome debug console to decrypt, i did decrypt such a thing once, I think on codecanyon to get to a "Tabs" script for free; (don't bother for the tabs, they're bloatware, better invest the time to do it yourself) ;)
I think you find that nowadays here http://www.slidetabs.com/, but I don't know if the "encryption" method is still in there.
Additionally, you can also escape the string in JavaScript, that then looks like this:
var _0xe91d=["\x28\x35\x28\x24\x29\x7B\x24\x2E\x32\x77\x2E
...
x5F\x63\x6F\x6E\x74\x5F\x64\x75\x72\x7C\x76\x5F\x74\x61\x62\x73\x5F\x61\x6C\x69\x67\x6E\x7C\x76\x5F\x74\x61\x62\x73\x5F\x64\x75\x72\x7C\x76\x5F\x73\x63\x72\x6F\x6C\x6C\x7C\x63\x6F\x6E\x74\x5F\x61\x6E\x69\x6D\x7C\x63\x6F\x6E\x74\x5F\x66\x78\x7C\x74\x61\x62\x5F\x66\x78\x7C\x72\x65\x70\x6C\x61\x63\x65\x7C\x62\x61\x6C\x69\x67\x6E\x7C\x61\x6C\x69\x67\x6E\x5F\x7C\x75\x6E\x6D\x6F\x75\x73\x65\x77\x68\x65\x65\x6C\x7C\x73\x77\x69\x74\x63\x68\x7C\x64\x65\x66\x61\x75\x6C\x74\x7C\x6A\x51\x75\x65\x72\x79","","\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65","\x72\x65\x70\x6C\x61\x63\x65","\x5C\x77\x2B","\x5C\x62","\x67"]
;eval(function (_0x173cx1,_0x173cx2,_0x173cx3,_0x173cx4,_0x173cx5,_0x173cx6){_0x173cx5=function (_0x173cx3){return (_0x173cx3<_0x173cx2?_0xe91d[4]:_0x173cx5(parseInt(_0x173cx3/_0x173cx2)))+((_0x173cx3=_0x173cx3%_0x173cx2)>35?String[_0xe91d[5]](_0x173cx3+29):_0x173cx3.toString(36));} ;if(!_0xe91d[4][_0xe91d[6]](/^/,String)){while(_0x173cx3--){_0x173cx6[_0x173cx5(_0x173cx3)]=_0x173cx4[_0x173cx3]||_0x173cx5(_0x173cx3);} ;_0x173cx4=[function (_0x173cx5){return _0x173cx6[_0x173cx5];} ];_0x173cx5=function (){return _0xe91d[7];} ;_0x173cx3=1;} ;while(_0x173cx3--){if(_0x173cx4[_0x173cx3]){_0x173cx1=_0x173cx1[_0xe91d[6]]( new RegExp(_0xe91d[8]+_0x173cx5(_0x173cx3)+_0xe91d[8],_0xe91d[9]),_0x173cx4[_0x173cx3]);} ;} ;return _0x173cx1;} (_0xe91d[0],62,284,_0xe91d[3][_0xe91d[2]](_0xe91d[1]),0,{}));
You can then bring the string back like:
"\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65".toString()
But for a moderate coder (like me), to figure out the system and decrypt the data of all this combined will take only appx. 15-30 minutes, (experimential find, from the codecanyon-try).
It's questionable if such a thing is worth the expense of your time, because it takes somebody like me less time to reverse-engineer your "encryption" than it takes you to "code" it.
Note that if you put a string like "\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65" into your appllication, you may trigger false alarms on certain virus scanners (McAffee, TrendMicro, Norton, etc., the usual suspects).
You can also partition the JSON string into an array of JSON-string chunks, makes it harder to decrypt it (maybe rotating the sequence in the array according to a certain system might help as well).
You can also break the string into an array of char:
var x = ['a', 'b', 'c'];
You can then bring it back like
console.log(x.join(""));
You can also reverse the string, and put that into an array (amCharts does that).
Then you bring it back with
x.reverse().join("");
The last one might be tricky for utf-8, as you need to correctly reverse strings like "Les misérables" (see also this and this)
Since the data will go on your client's computer, there is no other way to fully protect that data than... not sending it.
So, you could render some views on the server side and send them to the client but it may not be doable in your case.
Other way, would be to send data, but to make it difficult for an unauthorized user to access to it.
If your application is using an user database, you could generate a fixed key per user and encrypt sensible data before sending it to the client, and then the client would decrypt it with the same key calculated on the client side.
In addition, you can fine tune which data you want to send or not send to each user.
If you want to protect the data betweeen the moment the client's receive it and the moment it goes in your map, I'm afraid it is not possible as the map component you're using is probably waiting for standard JSON data.
Anyway, it makes no sense to protect your data as it will be displayed on your map.
Everything that is passed to client is not safe, you can try obfuscating data, but in the end the place where you put in the map will be accessible by just adding a line of console.log()
Another option, I'm just speculating as I'm not really sure how google maps work, but you might firstly send only the geolocation to the map, this way you will have pins on the map, the only after clicking on the ping you could fetch other data from api (name, address). Google maps should support something like onclick.
Annoy a potential scraper/hacker with all the tricks everyone talks about on this thread and others. But as it's been said many times, once the data is sent to the client, it's basically unprotected.
Perhaps your thinking should involve these things too:
-How to identify when someone is scraping (e.g. monitoring IPs, thresholds, user activity, etc) and do something about it or at least identify the culprit.
-Put copyrights and other identification on any thing you can, to help other users see and understand that it's your data, not the scrapers'. Look at what artists have been doing already, for a long time.
-Lay hidden traps in your data to help identify it as unique; that only you know about and the scraper wouldn't bother to look for or too lazy to check. If the scraper uses your data publicly too, then maybe this can be used in a legal case, or at least you could publicly shame the offender.
I'm writing a JavaScript function that needs to uphold three properties:
be very small and lightweight - no external libraries
encode a string in such a way as to be able to be passed as a GET parameter
this string must be decoded again at its destination
Effectively, it authenticates the user by sending his username and password to a PHP page which then verifies it. This is done via GET because I haven't yet found a way of doing a background cross-domain POST request. The trouble is that if the user has a character such as '#' or similar in his password, it doesn't get sent properly.
Currently to avoid this, I encode() the password string before sending it, which allows it to be received without problems. However, I read that PHP's urldecode() is not a perfect analog for this, as there are corner cases which are treated differently (i.e. ' ', '+', etc). Sadly I cannot find this document anymore, so I cannot quote it, but the gist was that one of them converts spaces into '+' signs, which the other treats as an actual plus sign, or something like that...
As such, I'm looking for a Javascript function that can take a string and make it URL-safe, and which has a perfect reversal function in PHP so that the original string can be recovered.
The arguably awful code I currently use to achieve this:
login.onsubmit = function(){
loginFailMsg.style.display = 'none';
var inputs = login.getElementsByTagName('input');
var formdata =
'username='+inputs[0].value+'&password='+encode(inputs[1].value);
submit.src = formtarget+'/auth/bklt?'+formdata;
userinfo = undefined;
setTimeout(getUserinfo,300);
return false;
};
encodeURIComponent, PHP will decode it automatically when populating $_POST or $_GET
'&password='+encode(inputs[1].value)
Where's encode function coming from? Seems to me the quick answer to your question is using encodeURIComponent() instead, available since JavaScript 1.5. See also Comparing escape(), encodeURI(), and encodeURIComponent(); it does not encode everything either, but does encode all the server expects it to.
(As for cross-domain AJAX POST calls, I'd really have a look at "JSON with Padding". See JSONP with jQuery that I mentioned in the comments earlier. This will also prevent issues with the timeout you've randomly chosen, and jQuery will also help you, a lot, to get rid of inputs[0].value and the like. And, as you apparently already have a MD5 hash on the server, I'd really hash the password client side as well --see Karl's answer-- and compare those hashes instead. Respect your user's password and your own time, drop that no external libraries requirement!)
I don't think there's such a thing as a reversible hash function. There are plenty of javascript md5 libraries available, however.