Context: I have been building an application that creates and uses magnet links. I've been trying to find an efficient way to transfer a javascript object in the Querystring so that On the other side I can deserialize it into an object keeping the same types. with efficient i mean using as little characters as possible/transferring as much data as possible. I've found my application has a max of +-1500 characters in url.
At first I used original Querystring npm packages but these can change types on deserialize and also very inefficient on deeper objects.
eg:
var input = { age: 12, name:'piet'};
var qs = querystring.encode(input); // ?age=12&name=piet
var output querystring.decode(qs); // {age: '12', name: 'piet'
Then I've tried using json stringifying with and without base64 for Querystrings. But this left me most of the time with much bigger strings for simple objects.
var input = { age: 12, name:'piet'};
var qs = encodeURIComponent(JSON.stringify(input)); // "%7B%22age%22%3A12%2C%22name%22%3A%22piet%22%7D"
But this leaves me with rediculous long querystring because half of the characters get encoded and become 3x as long which almost doubles the length.
base64 encoding in this case is a much better solution:
var input = { age: 12, name:'piet'};
var qs = btoa(JSON.stringify(input)); // eyJhZ2UiOjEyLCJuYW1lIjoicGlldCJ9
I've been trying to Google for an efficient algorithm it but haven't really found a good solution. I've been looking into msgPack binary serialisation by then I would also have to base64 which probably ends with a longer string.
Is there a known more efficient algorithm for object to Querystring serialisation with static types? or would i have to create my own?
I've been thinking on on a simple query string algorithm that works as follows:
Query string order is imporotant, for next point:
Keys starts with . shows depth: ?obj&.property="test" = { obj : {property: "test" }}
First character in string defines its type: b=boolean, s=string,n=number, (etc if needed)
This would lead to much more efficient query string i think. but am i not building something that has already been made before?
Surely sticking the stringified object into a single URI value should work?
var test = { string: 'string', integer: 8, intarray: [1,2,3] };
encodeURIComponent(JSON.stringify(test))
// "%7B%22string%22%3A%22string%22%2C%22integer%22%3A8%2C%22intarray%22%3A%5B1%2C2%2C3%5D%7D"
JSON.parse(decodeURIComponent("%7B%22string%22%3A%22string%22%2C%22integer%22%3A8%2C%22intarray%22%3A%5B1%2C2%2C3%5D%7D")
// {string: "string", integer: 8, intarray: [1, 2, 3]}
The object parsed at the end has the same types as the object inputted at the start.
Just stick your object into a single key:
var url = 'http://example.com/query?key=' + encodeURIComponent(JSON.stringify(object));
Right? And on the server you just parse that single value into an object.
Related
I am trying to save local storage info but what happens is I am getting \ added to all the items that are strings I mean as some answers stated it is caused by double stringfy ? issue is my blob action that saves the local storage refuse to handle the it unless is it is stringifed, long story short could this double stringfy needed for the function cause issues later, I have no problem when parsing back but the look of the saved elements looks really weird ?
//const ls = JSON.stringify(localStorage);
// "id":"\"159e17e9-19b7-453d-8e10-a411c7424586\"
// "groups":"{\"bee7bdc4-d888-46e6-93d7-ed0c\" : :[{\"id\":\"0e6e6426-4d79-4eea-9180-06111dd2a0e3\"}]
const config = {a: 'fdfgdg', b: 2}
console.log(JSON.stringify(config))
When saving to localStorage use JSON.stringify (since Storage accepts only String values):
const myData = {some: "data", other: "stuff"};
localStorage.appData = JSON.stringify(myData);
When reading from localStorage, use JSON.parse:
const myData = JSON.parse(localStorage.appData || "{}");
console.log(myData); // {some: "data", other: "stuff"}
Don't be afraid of escaped double-quotes with backslash! Otherwise it would end up in an invalid String format
const test = "this is \"something\""; // properly escaped quotes
Get used to see in console (on in LocalStorage) the escaped strings used for Keys os String properties of a JSON stringified Object literal:
"{\"prop\": \"some value\"}" // Valid stringified JSON
perfectly fine.
Learn more about the JSON format.
JSON, in order to be valid, its keys need to be enclosed in double quotes. JSON.stringify will take care of adding quotes to a JS Object Literal keys for you.
When you are getting a stringfied object and you want him to become an object again, there is another command to do so
JSON.parse(obj)
var obj = {
id: '159e17e9-19b7-453d-8e10-a411c7424586',
groups: '000e17e9-19b7-453d-8e10-a411c7424500'
}
var stringfiedObject = JSON.stringify(obj)
console.log(stringfiedObject)
var objectFromStringfiedObject = JSON.parse(stringfiedObject)
console.log(objectFromStringfiedObject)
There's no "extra backslash", it's just your means of output show that extra backslash, because it is the way it renders string values (you probably use browser console for that, didn't you?).
Be sure, the actual stringified value does not contain any characters that are not supposed to be there.
As others have pointed out, things aren't going as planned because you are stringifying localStorage itself, as well as improperly stringifying and parsing your data.
It would be much easier just to save the object directly to localStorage and then retrieve it as needed, without having to fool around with stringifying and parsing. While these methods are necessary, all they accomplish is to convert your data into the format useable by localStorage anyway, but the data itself stays converted, that is, a float 122.55 is converted into a string "122.55" and it stays that way.
Wouldn't it be so much easier if you could directly store the number 122.55 and retrieve it that way also?
Have a look at localDataStorage, where you can transparently set/get any of the following "types": Array, Boolean, Date, Float, Integer, Null, Object or String. It seamlessly converts your data, stores it, and allows you to retrieve it just the way it went it. Store an object and get it back. Store a date and get it back. Store a number or a boolean and get those back.
Examples:
localDataStorage.set( 'key1', false );
localDataStorage.get( 'key1' ); --> false (boolean)
localDataStorage.set( 'key2', 122.55 );
localDataStorage.get( 'key2' ); --> 122.55 (float)
var obj = {
id: '159e17e9-19b7-453d-8e10-a411c7424586',
groups: '000e17e9-19b7-453d-8e10-a411c7424500'
}
localDataStorage.set( 'key3', obj );
localDataStorage.get( 'key3' ); --> {id: '159e17e9-19b7-453d-8e10-a411c7424586', groups: '000e17e9-19b7-453d-8e10-a411c7424500'} (object)
localDataStorage.set( 'key4', "this is \"something\"" );
localDataStorage.get( 'key4' ); --> 'this is "something"' (string)
All of the conversion work is done in the background for you: just set/get and go.
[DISCLAIMER] I am the author of the utility [/DISCLAIMER]
Looking for clean way to convert a javascript object containing arrays as values to a search param compatible query string. Serializing an element from each array before moving to the next index.
Using libraries such as querystring or qs, converts the object just fine, but handles each array independently. Passing the resulting string to the server (which I cannot change) causes an error in handling of the items as each previous value is overwritten by the next. Using any kind of array notation in the query string is not supported. The only option I have not tried is a custom sort function, but seems like it would be worse than writing a custom function to parse the object. Any revision to the object that would generate the expected result is welcome as well.
var qs = require("qs")
var jsobj = {
origString:['abc','123'],
newString:['abcd','1234'],
action:'compare'
}
qs.stringify(jsobj,{encode:false})
qs.stringify(jsobj,{encode:false,indices:false})
qs.stringify(jsobj,{encode:false,indices:false,arrayFormat:'repeat'})
Result returned is
"origString=abc&origString=123&newString=abcd&newString=1234&action=compare"
Result desired would be
"origString=abc&newString=abcd&origString=123&newString=1234&action=compare"
I tried reorder your json:
> var jsobj = [{origString: 'abc', newString: 'abcd' }, {origString: '123',
newString: '1234' }, {action:'compare'}]
> qs.stringify(jsobj,{encode:false})
'0[origString]=abc&0[newString]=abcd&1[origString]=123&1[newString]=1234&2[action]=compare'
But I don't know if this is a good alternative for your problem.
Chalk this up to misunderstanding of the application. After spending some more time with the API I realized my mistake, and as posted above by others, order does no matter. Not sure why my first several attempts failed but the question is 'answered'
I have some code that returns something like this:
body: '[
{
name: "name",
lastname: "lastname"
},
{
name: "name",
lastname: "lastname"
},
{
name: "name",
lastname: "lastname"
}
]'
Of course extracting the body is a simple object.body however, to get rid of the '' that wraps the array is just destroying me. I tried doing object.body.slice(1,-1) to get rid of them, it didnt work. I'm clearly not using the object properly.
How can I "extract" the array from within the body into a usable array?
It sounds like you want to evaluate the content of the string. Assuming whatever code building this string can't actually build JS objects to begin with, you can use eval or Function to generate the objects you need.
var data = eval(string);
Note that you must be sure that the source of the string is safe and reliable, otherwise you could be evaluating malicious code. There may also be performance consequences to using eval.
Using Function is a tiny bit safer, only because the code can not access your local variables. It should also avoid the performance costs of eval.
var data = Function("return (" + string + ");")();
Same warning about malicious code though.
If the string data was valid JSON, you could use JSON.parse, but it isn't, so you can't. To avoid security issues, either have the source provide valid JSON data, or write your own minimal parser to parse the data.
With a valid JSON string, you could just parse the string for an object with JSON.parse, if you have .
array = JSON.parse(string);
You can use split function and then take the first element, let me say
var a = '[{name: "name",lastname: "lastname"},{name: "name",lastname: "lastname"},{name: "name",lastname: "lastname"}]';
var s = a.split("'");
s[0] will return you the desired result
I am trying to create a JSON object in the particular format
{ "id": 12234 , "name": "Alex" , "gender": "Male"}
// My code starts here
var number = userid; // alert(userid) is 12345
var name= nameuser; // alert(nameuser) is Alex
var g= gender; // alert(gender) is male
var userObj = { "id": number , "name": name , "gender":g}
I tried `JSON.stringify(userObj); ,
that returns the object type as
{"id":"12345" , "name":"Alex" , "gender":"Male"}
but this is not what I want as I want the number to be 12345 and not "12345".
also tried stringifying the fields inside the object like
{ "id": number , "name":JSON.stringify(name) ,gender: JSON.stringify(g)}
but when I do alert(userObj) my object type is Object object and this is not a format the server recognises.
I am sure there is a workaround for this but I am unable to figure one out
JSON works with strings exclusively. It's invalid to have anything other than a string, array, or other JSON object in JSON. That said, a "number" is not allowed; it needs to be a string. Whatever you are working with with the JSON later needs to be able to change the string back it to a number, if necessary. JavaScript usually does a good job of coercing these.
On the server side you can just do something like
obj = json_decode(source)
obj.id = (int)obj.id
I think that your solution will come down to what you want to DO with the JSON, rather than how you want it to be formatted.
If you're using it in other JavaScript code, then the JSON.parse method is going to take care of most of your issue on the other side (with automatic type-casting dealing with the rest).
If you're using it on the server-side, again, PHP or similar will decode the object appropriately.
And if it's a more-strict language on your server, all you need to do is remember which parameters need to be cast to boolean or to int/float.
The below code from your question is valid JSON.
{ "id": 12234 , "name": "Alex" , "gender": "Male"}
I am guessing that the problem you have is that your userid variable is a string, not a number. If so, try
var number = parseInt(userid, 10);
(The 10 parameter indicates that you are using base 10 numbers as opposed to something like binary or hex).
store the object type as one of the property and use it to convert the way you want.
if I have a large javascript string array that has over 10,000 elements,
how do I quickly search through it?
Right now I have a javascript string array that stores the description of a job,
and I"m allowing the user to dynamic filter the returned list as they type into an input box.
So say I have an string array like so:
var descArr = {"flipping burgers", "pumping gas", "delivering mail"};
and the user wants to search for: "p"
How would I be able to search a string array that has 10000+ descriptions in it quickly?
Obviously I can't sort the description array since they're descriptions, so binary search is out. And since the user can search by "p" or "pi" or any combination of letters, this partial search means that I can't use associative arrays (i.e. searchDescArray["pumping gas"] )
to speed up the search.
Any ideas anyone?
As regular expression engines in actual browsers are going nuts in terms of speed, how about doing it that way? Instead of an array pass a gigantic string and separate the words with an identifer.
Example:
String "flipping burgers""pumping gas""delivering mail"
Regex: "([^"]*ping[^"]*)"
With the switch /g for global you get all the matches. Make sure the user does not search for your string separator.
You can even add an id into the string with something like:
String "11 flipping burgers""12 pumping gas""13 delivering mail"
Regex: "(\d+) ([^"]*ping[^"]*)"
Example: http://jsfiddle.net/RnabN/4/ (30000 strings, limit results to 100)
There's no way to speed up an initial array lookup without making some changes. You can speed up consequtive lookups by caching results and mapping them to patterns dynamically.
1.) Adjust your data format. This makes initial lookups somewhat speedier. Basically, you precache.
var data = {
a : ['Ant farm', 'Ant massage parlor'],
b : ['Bat farm', 'Bat massage parlor']
// etc
}
2.) Setup cache mechanics.
var searchFor = function(str, list, caseSensitive, reduce){
str = str.replace(/(?:^\s*|\s*$)/g, ''); // trim whitespace
var found = [];
var reg = new RegExp('^\\s?'+str, 'g' + caseSensitive ? '':'i');
var i = list.length;
while(i--){
if(reg.test(list[i])) found.push(list[i]);
reduce && list.splice(i, 1);
}
}
var lookUp = function(str, caseSensitive){
str = str.replace(/(?:^\s*|\s*$)/g, ''); // trim whitespace
if(data[str]) return cache[str];
var firstChar = caseSensitive ? str[0] : str[0].toLowerCase();
var list = data[firstChar];
if(!list) return (data[str] = []);
// we cache on data since it's already a caching object.
return (data[str] = searchFor(str, list, caseSensitive));
}
3.) Use the following script to create a precache object. I suggest you run this once and use JSON.stringify to create a static cache object. (or do this on the backend)
// we need lookUp function from above, this might take a while
var preCache = function(arr){
var chars = "abcdefghijklmnopqrstuvwxyz".split('');
var cache = {};
var i = chars.length;
while(i--){
// reduce is true, so we're destroying the original list here.
cache[chars[i]] = searchFor(chars[i], arr, false, true);
}
return cache;
}
Probably a bit more code then you expected, but optimalisation and performance doesn't come for free.
This may not be an answer for you, as I'm making some assumptions about your setup, but if you have server side code and a database, you'd be far better off making an AJAX call back to get the cut down list of results, and using a database to do the filtering (as they're very good at this sort of thing).
As well as the database benefit, you'd also benefit from not outputting this much data (10000 variables) to a web based front end - if you only return those you require, then you'll save a fair bit of bandwidth.
I can't reproduce the problem, I created a naive implementation, and most browsers do the search across 10000 15 char strings in a single digit number of milliseconds. I can't test in IE6, but I wouldn't believe it to more than 100 times slower than the fastest browsers, which would still be virtually instant.
Try it yourself: http://ebusiness.hopto.org/test/stacktest8.htm (Note that the creation time is not relevant to the issue, that is just there to get some data to work on.)
One thing you could do wrong is trying to render all results, that would be quite a huge job when the user has only entered a single letter, or a common letter combination.
I suggest trying a ready made JS function, for example the autocomplete from jQuery. It's fast and it has many options to configure.
Check out the jQuery autocomplete demo
Using a Set for large datasets (1M+) is around 3500 times faster than Array .includes()
You must use a Set if you want speed.
I just wrote a node script that needs to look up a string in a 1.3M array.
Using Array's .includes for 10K lookups:
39.27 seconds
Using Set .has for 10K lookups:
0.01084 seconds
Use a Set.