I am getting a url from server with lot of data as query string
(E.g. http://www.test.com/?n=1,2,3,4,5,6,7,8.....100000) and I want to open it using window.open().
But the moment I pass the URL to window.open the url which gets gets truncated. After searching for sometime I could figure out that the maximum limit for URL is 2,083 characters(IE) so it passes PART of query string and truncates the rest..
How can I overcome this?
Please let me know if I need to provide more details.
I think the only solution is to use POST instead of GET. Just use a form, instead of window.open. Please see this older answer:
https://stackoverflow.com/a/17089124/907420
You could try URL shorteners, like goo.gl or bit.ly:
https://goo.gl/
Speaking of programming, you could try to shorten your URL-s yourself, for the given example:
Exact URL for your example, shorter by ~2000 characters.
Where x..y is translated on server side as range(x, y) -> 1,2,3,4,5...100000 for x and y being 1 and 100000
If you want to stick with GET (and I would recommend that you use POST) you can try to compress the parameters. Instead of giving a huge number of parameters you create a javascript object holding the parameters, jsonfy it and you end up with a string that can be compressed and uncompressed again at the other end. Afterwards you can deserialize the JSON string and you have your parameters. Depending on the number of parameters it might still not be sufficient for a GET request.
But at the end a POST request is the best solution I think.
I'm creating an application with Node.js and Mongo DB, rendering the views with Swig.
I have a database of business names, addresses and geo location data that is being plotted onto a Google map with pins.
I'd like to stop users from easily copying the raw JSON data using view source, Firebug, Chrome Dev tools etc.
I'm not after bank grade security, just want to make it hard enough for most users to give up.
I have two routes of delivering the JSON package to the browser:
1) Using Swig, passing the JSON package directly to the view. Problem is that a simple view source will show the JSON.
2) Requesting the data with an AJAX call. In this scenario the data is easily accessible with Chrome Dev tools.
What are my options?
Base-64 encode the string.
Then you can just base64-decode it in JavaScript.
That should make it sufficiently unreadable, no real security though - of course.
Plus it's fast.
You need to take care with UTF-8 characters (e.g. German äöüÄÖÜ, or French èéàâôû)
e.g. like this in JavaScript:
var str = "äöüÄÖÜçéèñ";
var b64 = window.btoa(unescape(encodeURIComponent(str)))
console.log(b64);
var str2 = decodeURIComponent(escape(window.atob(b64)));
console.log(str2);
example:
var imgsrc = 'data:image/svg+xml;base64,' + btoa(unescape(encodeURIComponent(markup)));
var img = new Image(1, 1); // width, height values are optional params
img.src = imgsrc;
More secure variant:
Return encrypted base64 encoded JSON, plus the decryption algorithm, base64 encode them server-side, bit-shift it a few bits, return via ajax, then de-bitshift the string on the webpage, pass it to eval, which will give you the decrypt function, then decrypt the encrypted base64 string, then base-64 decode that string.
But that takes only a few seconds more on the chrome debug console to decrypt, i did decrypt such a thing once, I think on codecanyon to get to a "Tabs" script for free; (don't bother for the tabs, they're bloatware, better invest the time to do it yourself) ;)
I think you find that nowadays here http://www.slidetabs.com/, but I don't know if the "encryption" method is still in there.
Additionally, you can also escape the string in JavaScript, that then looks like this:
var _0xe91d=["\x28\x35\x28\x24\x29\x7B\x24\x2E\x32\x77\x2E
...
x5F\x63\x6F\x6E\x74\x5F\x64\x75\x72\x7C\x76\x5F\x74\x61\x62\x73\x5F\x61\x6C\x69\x67\x6E\x7C\x76\x5F\x74\x61\x62\x73\x5F\x64\x75\x72\x7C\x76\x5F\x73\x63\x72\x6F\x6C\x6C\x7C\x63\x6F\x6E\x74\x5F\x61\x6E\x69\x6D\x7C\x63\x6F\x6E\x74\x5F\x66\x78\x7C\x74\x61\x62\x5F\x66\x78\x7C\x72\x65\x70\x6C\x61\x63\x65\x7C\x62\x61\x6C\x69\x67\x6E\x7C\x61\x6C\x69\x67\x6E\x5F\x7C\x75\x6E\x6D\x6F\x75\x73\x65\x77\x68\x65\x65\x6C\x7C\x73\x77\x69\x74\x63\x68\x7C\x64\x65\x66\x61\x75\x6C\x74\x7C\x6A\x51\x75\x65\x72\x79","","\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65","\x72\x65\x70\x6C\x61\x63\x65","\x5C\x77\x2B","\x5C\x62","\x67"]
;eval(function (_0x173cx1,_0x173cx2,_0x173cx3,_0x173cx4,_0x173cx5,_0x173cx6){_0x173cx5=function (_0x173cx3){return (_0x173cx3<_0x173cx2?_0xe91d[4]:_0x173cx5(parseInt(_0x173cx3/_0x173cx2)))+((_0x173cx3=_0x173cx3%_0x173cx2)>35?String[_0xe91d[5]](_0x173cx3+29):_0x173cx3.toString(36));} ;if(!_0xe91d[4][_0xe91d[6]](/^/,String)){while(_0x173cx3--){_0x173cx6[_0x173cx5(_0x173cx3)]=_0x173cx4[_0x173cx3]||_0x173cx5(_0x173cx3);} ;_0x173cx4=[function (_0x173cx5){return _0x173cx6[_0x173cx5];} ];_0x173cx5=function (){return _0xe91d[7];} ;_0x173cx3=1;} ;while(_0x173cx3--){if(_0x173cx4[_0x173cx3]){_0x173cx1=_0x173cx1[_0xe91d[6]]( new RegExp(_0xe91d[8]+_0x173cx5(_0x173cx3)+_0xe91d[8],_0xe91d[9]),_0x173cx4[_0x173cx3]);} ;} ;return _0x173cx1;} (_0xe91d[0],62,284,_0xe91d[3][_0xe91d[2]](_0xe91d[1]),0,{}));
You can then bring the string back like:
"\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65".toString()
But for a moderate coder (like me), to figure out the system and decrypt the data of all this combined will take only appx. 15-30 minutes, (experimential find, from the codecanyon-try).
It's questionable if such a thing is worth the expense of your time, because it takes somebody like me less time to reverse-engineer your "encryption" than it takes you to "code" it.
Note that if you put a string like "\x66\x72\x6F\x6D\x43\x68\x61\x72\x43\x6F\x64\x65" into your appllication, you may trigger false alarms on certain virus scanners (McAffee, TrendMicro, Norton, etc., the usual suspects).
You can also partition the JSON string into an array of JSON-string chunks, makes it harder to decrypt it (maybe rotating the sequence in the array according to a certain system might help as well).
You can also break the string into an array of char:
var x = ['a', 'b', 'c'];
You can then bring it back like
console.log(x.join(""));
You can also reverse the string, and put that into an array (amCharts does that).
Then you bring it back with
x.reverse().join("");
The last one might be tricky for utf-8, as you need to correctly reverse strings like "Les misérables" (see also this and this)
Since the data will go on your client's computer, there is no other way to fully protect that data than... not sending it.
So, you could render some views on the server side and send them to the client but it may not be doable in your case.
Other way, would be to send data, but to make it difficult for an unauthorized user to access to it.
If your application is using an user database, you could generate a fixed key per user and encrypt sensible data before sending it to the client, and then the client would decrypt it with the same key calculated on the client side.
In addition, you can fine tune which data you want to send or not send to each user.
If you want to protect the data betweeen the moment the client's receive it and the moment it goes in your map, I'm afraid it is not possible as the map component you're using is probably waiting for standard JSON data.
Anyway, it makes no sense to protect your data as it will be displayed on your map.
Everything that is passed to client is not safe, you can try obfuscating data, but in the end the place where you put in the map will be accessible by just adding a line of console.log()
Another option, I'm just speculating as I'm not really sure how google maps work, but you might firstly send only the geolocation to the map, this way you will have pins on the map, the only after clicking on the ping you could fetch other data from api (name, address). Google maps should support something like onclick.
Annoy a potential scraper/hacker with all the tricks everyone talks about on this thread and others. But as it's been said many times, once the data is sent to the client, it's basically unprotected.
Perhaps your thinking should involve these things too:
-How to identify when someone is scraping (e.g. monitoring IPs, thresholds, user activity, etc) and do something about it or at least identify the culprit.
-Put copyrights and other identification on any thing you can, to help other users see and understand that it's your data, not the scrapers'. Look at what artists have been doing already, for a long time.
-Lay hidden traps in your data to help identify it as unique; that only you know about and the scraper wouldn't bother to look for or too lazy to check. If the scraper uses your data publicly too, then maybe this can be used in a legal case, or at least you could publicly shame the offender.
Hoping this is a quick fix.
I'm sending a query string from an AngularJS application to a web API coded in C#. The string contains a "message" value which may or may not contain url-encoded HTML tags.
Here's a basic example:
msg = "<a>"
querystring = "/SERVERPATH/?id=1&msg=%3Ca%3E"
Sending the string above to my API results in a 500 Error and the "msg" value never actually reaches the server. On the other hand, adding a space before and after the "a" causes everything to work great.
msg = "< a >"
querystring = "/SERVERPATH/?id=1&msg=%3C%20a%20%3E"
Is there a special type of validation occurring that I don't know about, and is there a way to configure these rules myself?
This is probably web.config related, but I could be totally wrong about that. Any input would be greatly appreciated.
The error code 500 is for internal server error. It means your code do reach your web service. By your example (where on putting a space before and after 'a' makes everything work) I am pretty sure that request validation is failing at your web service. To understand request validation please visit this link.
A word of caution: Try not to disable request validation on your web service. Doing that poses a huge security threat.
You may consider a work around, where instead of using URLEncode you can use HTMLEncode for encoding the query string values (values only).
I wasnt quite sure how to ask this question, sorry.. but basically it's this..
I just started out w/ Ajax .. and with sending XMLhttpRequest in the background, i'm having problems w/ some html special characters in the form data , specially the & sign terminating the variable prematurely
eg. value "You & I" results to "You "
Now to explain this question from my perspective.. it's simply this.. if i submit my regular form using GET method, the same thing happens.. since the variables are URL-Encoded.. but if i set the form to POST method then everything is PRESERVED as i needed it..
Now I believe it has something to do w/ this (?)
hr.open("POST", link, true);
hr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
particularly on the form-urlencoded part.. is there an alternative?
i'd hate to have to run a cleanup routine on each variable that i pass thru Ajax :(
the & sign terminating the variable prematurely
& is supposed to terminate values in x-www-form-urlencoded data. You need to encode your data when building URLs.
if i submit my regular form using GET method, the same thing happens.. since the variables are URL-Encoded
No. If you submit a regular form using GET, then the inputed data will be URL Encoded and ampersand characters that appear as data will be represented by %26 and not as raw & characters so the same thing won't happen.
is there an alternative [to application/x-www-form-urlencoded]?
You can use any data format you like. Some are easier to generate than others (JSON is particularly easy to generate from a browser). You then have the challenge of decoding them on the server (form handling libraries don't tend to do JSON, so you'd have to get the raw POST data and decode it yourself).
Since your actual problem is solved by encoding your data, this is unnecessary.
I'm getting a string from the current window's fragment identifier (location.hash). I want to use that string as the argument to location.replace(str).
Under normal circumstances, the string will come from code I control, so I'm not worried about validating that the string is a URL. If the string isn't a URL, the call to replace will just fail. That's fine.
What I am concerned about is making sure the string is NOT a javascript: URL or anything else that would allow someone to run arbitrary Javascript on my domain. Currently, I'm just checking that that str.indexOf('http') == 0.
Is that enough or should I sanitize this string some more?
The sanitization you propose is not enough.
An attacker could redirect to a data:uri url that contains base64 encoded html/javascript. This would allow the attacker to execute arbitrary javascript code. For example, this code snippet will alert 'xss' (in firefox, safari and opera)
var data = 'data:text/html;base64,PHNjcmlwdD5hbGVydCgiWFNTIik8L3NjcmlwdD4=';
location.replace(data);
Besides, it may be possible to redirect to a anonymous FTP url, or use some other obscure protocol.
Instead of blacklisting protocols/keywords, use a whitelist approach instead. Maintain a list of good urls in your javascript code. Then, read the fragment identifier and see if it is in this known list of urls. If it is not, stop the process.
In security, whitelists are always preferable to blacklists.