I'm trying to write a javascipt that will take a large amount values from a text files and will use those values to make a large set of queries to find a filed. for example there are 2000 values given in the text file, then i'm trying to read the values from the file and it will use each value to excute each query. So im not getting the concept how to write it, cause it is not possible to write 2000 queries separately, please help
var values = new Array();
$.get('UserFile.txt', function(data){
values = data.split('\n');
console.log(value);
db.students.find(CID:{$in[values]})
});
This is the concept I want to apply, the UserFile.txt got 20000 values and it will use those values to find it. Also it gives "$" is not defined in the shell
The concept is sound provided that you actually have a library that can load the contents of a file for you, so would likely be better suited to another language implementation or something like nodejs if you want JavaScript, as opposed to doing this in the mongo shell.
Aside from that the only issue is your syntax for the use of the $in operator, which should rather be:
values = filecontent.split('\n'); // Somehow obtained from reading a file
db.students.find({ "CID": { "$in": values } });
That is because whatever went into values is already a native array to the language and will be passed in as such for an argument to $in which expects it's argument to be an array.
It probably isn't the best idea to pass in a very large array and your total request must be under the 16MB BSON limit. So you may need to split up those array elements into several queries, but in batch sizes rather than singular, so that is at least better.
I could be wrong, but I don't find it likely you are going to find a library you could load that allows this, you can load native JavaScript as a structure but that would be beside the point.
Use something else as your language implementation and you can get the desired result by following the correct syntax.
Related
I have MarkLogic 9 on my database.
I am trying to iterate through the results of a cts.search() using JavaScript.
I have tried using Array.from() and toArray(), and although they both work, they take quite some time for a large result set size.
I am looking for an iteration method that would be faster than the 2 mentioned above.
The way to iterate lazily through search results and Sequences in general is by using the for...of construct, as described in our JS Reference guide:
http://docs.marklogic.com/guide/jsref/api#id_59096
for (const doc of cts.search(cts.trueQuery())) {
//...
}
Mind though that might still not perform well if you are trying to query the entire database, and are simply returning too much output. It is probably wise to consider paging through your results.
A good way of doing so is by using our JSearch library. You can read more about it in our Search Developer guide:
https://docs.marklogic.com/10.0/guide/search-dev/javascript
HTH!
I'm making an application in which I need to generate Unique IDs.
When generating IDs the best way to avoid clashes, is it simply brute force generate-then-check'ing, or is there a way to garuantee unique generation.
I'm sure the brute force method would do well for a while however I have a feeling companies like Google aren't using this method.
The other things is how to actually generate them in node.js.
I thought of generating int's from '0 - 63' and then parsing to base-64 like this:
var map = {
0:0, 1:1 ... 16:"Q" ... 63:"/"
}; /* I used object so you can see the indexes,
this would be an array - or even a string? */
for (var i = IDLENGTH, id=""; i--;) id+=map[~~(Math.random*64)];
However this seems inneficient, especially having the map in the first place.
I saw this peice of code in another SO post
console.log(new Buffer("Hello World").toString('base64'));
> SGVsbG8gV29ybGQ=
Which seems to make more sense, but this doesn't seem to fit what I need, isn't this conversion of char-sets?
This is quite an open question, but you probably want to generate an UUID. Basically it's a random 128 bit number. If done correctly, the probability for a duplicate is extremely low. I believe for most applications it's not strictly needed to check, but you might still want to do it.
Use a well tested library instead of implementing this yourself, I recommend node-uuid. Generating good random numbers is very similar to crypto operations - it's not trivial to get right.
I'm saving in Couchbase a document which has javascript Date values, and wish to get it exactly the same, not as string '2016-01-02T12:13:14Z'.
Found a way to achieve this using plain Javascript, by using the second parameter of JSON.parse , but Couchbase does the deserialization internally and can't really use this.
Is there any way to disable the Couchbase deserialization, and to avoid doing JSON.stringify + JSON.parse and neither deep-walking the object?
bucket.get(key, (err, result) => {
if (err) {
//deal with error here
} else {
//here "result.value" is already deserialized
done(result.value);
}
});
As you probably know, JSON and Date object handling can be a bit specific depending on what you're trying to do. We tend to stick to the defaults. What you're looking for is a fairly advanced use case. At the moment, we don't have direct support for changing the way we do that parsing.
However, there is an interface for this. It's called a "transcoder" and it lets you be very specific about how you want to handle converting incoming data to what is stored. I don't have an example that shows quite what you want to do, but a good place to start looking for this at a lower layer is shown in the tests.
It might be easier, however, to just treat whatever you're storing differently at the application level. From my read of the revivier parameter you pointed to, that'd be only at time of retrieval. There's nothing stopping you from wrapping that get() and then mutating the object before passing it along to the next layer at very low cost I do believe.
I am getting results from a database with a simple php while loop, one of the pieces of information is a number that links to another table where the value is stored, I can think of plenty of ways to get this information linked and display the text related to the value but I want to know the fastest way to do it as I have a huge set of results so every bit of speed will make a difference. Is an array fastest, javascript? any advice you can give me would be great.
The schema would look something like this
col_table
colID(autonumber) colName(str) colState(int) colDate(date)
state_table
stateID(int) stateType(str)
I want to select the correct state type based on the colState matching a stateID and output the stateType while preserving the stateID for so I can edit the field and update the database using the number.
Using MySQL will be faster.
If you have to get through a PHP loop to read your results and make each time a new MySQL request, your script will take longer.
You can increase speed on MySQL by creating the right kind/amount of index, choosing wisely what is store in each field.
The later you parse content, the longer it will take. If you go for js, you will have to read a DB, loop trough it in PHP and do it again in JS, and making more request again ...
A join can be a good solution. A view can be even more easier to treat. Yuo can also consider caching results
Use a timer in php and try trial and error method. Use the time returned by the timer to evaluate speed and efficiency.
you should prepare your data on server side it is faster.
Whether you choose your server or database with a fast query it depends. If you have complex object graphs then the processing of results from db in order to create associations would be time consuming so an ORM is the way to go, otherwise as is your case with a simple join i would simply retrieve all data from db.
If you use php for rendering as well then render it using php no js.
If you use js for your ui then prepare data on server side and publish it via a REST webservice in json,i.e. usind json_encode functions of php, then retrieve it from js and output.
I have read a lot about parsing JSON with Actionscript. Originally it was said to use this library. http://code.google.com/p/as3corelib/ but it seems Flash Player 11 has native support for it now.
My problem is that I cannot find examples or help that takes you from beginning to end of the process. Everything I have read seems to start in the middle. I have no real experience with JSON so this is a problem. I don't even know how to point ActionScript to the JSON file it needs to read.
I have a project with a tight deadline that requires me to read twitter through JSON. I need to get the three most recent tweets, along with the user who posted it, their twitter name and the time those tweets were posted.
The back end to this is already set up I believe by the development team here, therefor my JSON files or XML just needs to be pointed to and then I need to display the values in the interface text boxes I have already designed and created.
Any help will be greatly appreciated...I do know that there are a lot of threads on here I just do not understand them as they all have some understanding of it to begin with.
You need to:
Load the data, whatever it is.
Parse the data from a particular format.
For this you would normally:
Use URLLoader class to load any data. (Just go to the language reference and look into example of how to use this class).
Use whatever parser to parse the particular format that you need. http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/JSON.html this is the reference to JSON API, it also shows usage examples. I'm not aware of these API being in production version of the player, still there might be quite a bit of FP 10.X players out there, so I'd have a fallback JSON parser, but I would recommend using this library: http://www.blooddy.by/en/crypto/ over as3corelib because it is faster. The built-in API are no different from those you would find in browser, so if you look up JSON JavaScript entries, the use should be in general similar to Flash.
After you parse JSON format, you will end up with a number of objects of the following types: Object, Array, Boolean, Number, String. It has also literals to mean null and undefined. Basically, you will be working with native to Flash data structures, you only should take extra care because they will be dynamically constructed, meaning you may not make assumption about existence of parts of the data - you must always check the availability.
wvxvw's answer is good, but I think skips over a to be desired explanation of what JSON itself is. JSON is plain text, javascript object notation, when you read the text on screen it looks something like this
http://www.json.org/example.html
you can see a side by side JSON and XML (both plain text formats) essentially JSON is a bunch of name value pairs.
When you use JSON.parse("your JSON string goes here") it will do the conversions to AS3 "dynamic objects" which are just plain objects (whose properties can be assigned without previously being defined, hence dynamic). But to make a long story short, take the example you see in the link above, copy and paste the JSON as a string variable in AS3, use
var str:String = '{"glossary": {"title": "example glossary","GlossDiv": {"title": "S","GlossList": {"GlossEntry": {"ID": "SGML","SortAs": "SGML","GlossTerm": "Standard Generalized Markup Language","Acronym": "SGML","Abbrev": "ISO 8879:1986","GlossDef": {"para": "A meta-markup language, used to create markup languages such as DocBook.","GlossSeeAlso": ["GML", "XML"]},"GlossSee": "markup"}}}}}';
var test:Object = JSON.parse(str);
method on the string, store it in a variable and use the debugger to see what the resulting object is. As far as I know there's really nothing else to JSON it's simply this format for storing data (you can't use E4X on it since it's not XML based and because of that it's slightly more concise than XML, no closing tags, but in my opionion slightly less readable... but is valid javascript). For a nice break-down of the performance gains/losses between AMF, JSON and XML check out this page: http://www.jamesward.com/census2/ Though many times you don't have a choice with regard to the delivery message format or protocol being used if you're not building the service, it's good to understand what the performance costs of them are.