At the top of a file can I put something like...
var collection = db.mongo.collection('test', function(err, collection){return collection});
and then in any of the files functions use collection.find() etc
I guess my question is... is collection a reference to the collection or a copy of the data?
If data in the collection changes will i still get up to date data by querying the collection variable?
Thanks!!
Collection is a reference for the collection object. Until you issue a find (or findOne) you don't have real data in your hands. And even then, it returns a Cursor object leaving the collection object always untouched.
Storing both Collections or cursors will not store your data. remember that you could be dealing with millions of records. dealing with data itself could be overwhelming for the server memory. Instead, mongo returns cursors and references for you to filter away. In PHP you have a function called iterator_to_array that you can pass it the cursor and it converts to an array of data. In javascript I don't know if there is such functions. But I guess it doesn't makes sense to be such functions. Filter the information until you have manageable data size, then iterate over the cursor and do your thing. If you have something like a config array or such, intead of several documents, try to store everything on one and fetch it with the findOne() function.
But in the end I guess that's just a design question whether your data is possible to filter or not.
Related
We have a MongoDB database in a development environment. There are a lot of collections that contain names of people. What we want to do is the following:
mask the names in each collection, the fields need to be updated directly in the database, cannot run them through some external pipeline
once masked, it is ok if we are unable to retrieve the original names (so one-way masking)
every unique name should result in the same mask
the masking script can be run on the mongodb cli or a MongoDB gui like Studio3T
I was thinking of maybe using MD5 or SHA, but I am not sure if either is available to use directly in mongo operations like update or even in javascript without external libraries.
Also, since MD5 always produces the same hash, if someone were to get access to the document, since we will not be masking the field name, it would be fairly easy to feed typical names into the algorithm until the hash matches to figure out the name, but I think we may be able to live with this.
An alternative I was thinking of was, to loop through the unique names we have, and create a map from names to UUIDs. Then, go through each collection and use this map to update the names with the UUIDs. The problem with this is that we'll need to keep this mapping dictionary for when we receive additional documents for an existing person.
I am having troubles with firebase using the cloudstore .where query.
I want to query a big collection of documents (in my case posts) but I only want to query the posts in which the groupId matches any of the groups that the user is in. The reason for this is that I want to query a combined feed for the user with all the latest relevant data (using orderBy and limit).
I know that I can use array-contains, so I could for instance query all of the posts for user where the user is a member.
firebase.db.collection('posts').where('members','array-contains',firebase.uid)
This would work if I decided to keep track of the members in a group. Problem is if I would change members in a group, I would have to loop through all posts and change the array of members (not really good practice). Better would then be to have the post contain and id of which group it was posted in.
So let's say the user has an array containing all the groups he is in
user.groups = ['companyGroup', '{id}', '{id2}']
I would then like to query through the whole posts collection and get all the documents where the field groupId matches any of the values in user.groups something like this:
firebase.db.collection('posts').where('groupId','==',[any of user.groups])
or maybe the reverse:
firebase.db.collection('posts').where(user.groups,'array-contains','groupId')
^ I have not tried this one but I am certain it doesn't work according to the docs according to
The where() method takes three parameters: a field to filter on, a comparison operation, and a value. The comparison can be <, <=, ==, >, >=, or array_contains.
Is there a possible way to do something like this? I can't really query multiple locations at once and combine them because that defeats the purpose of being able to limit the data and orderBy fields. I understand that I could put a new collection called feed under every user and for every post use a firebase function that pushes to post to the relevant members feed (only the id and latestActivity), but then as soon as that post changes (I am going to use a field called latestActivity to order data according to relevancy, but also when deleting a post) I would need to loop through all docs under every user affected and change the value/delete doc.
Any ideas are greatly appreciated!
Currently, there is no way to pass an array of ids to the where() function and expect to get all the documents that corresponde to each particular id:
firebase.db.collection('posts').where('groupId','==',[any of user.groups])
The option that you have, is to store in the array either the ids as strings or references (path to group documents). There is no real advantage of storing references rather than strings, so it's up to you to decide which one you feel more comfortable with.
To get the all group documents, you should get the array that contains those ids/references and for each one separately create a new database request. So unfortunately there is no other way to get those group documents at once using a single query.
However, creating extra database calls it doesn't mean that fetching, let's say 3 documents, will be 3x slower as fetching one document. You can make some tests yourself.
I quote #Frank van Puffelen, "Firebase has a history of performing fine in such cases, since it pipelines the requests."
use array contain it works perfectly.
firebase.db.collection('posts').where(user.groups,'array-contains','groupId')
it works pretty good for me . you should try this.
I'm using the Firebase Realtime Database to store measured weather data. (i.e. temperature, air pressure, etc.)
Let's say every 15 minutes a new value gets added to my db. I wanna use Firebase Functions to extract certain values automatically (maxima, minima, 24h high/low, etc.) because I want to display these values on my website. It seemed a good idea because in this way all the work would be done on the back-end and the web-js-code would just grab the values off of my db instead of querying endlessly.
Now I'm no expert on Firebase Functions and ran into some trouble trying to get Firebase to read and compare all these values. My db-tree looks something like this:
The idea is to use .onWrite to listen for new entries in, say, 'weather/temps' and compare each new entry with 'history/extrema/maximum/temp'. Now I dont really know how to read in the current maximum value inside the functions that would update it's value. So far, my code looks like this:
How can I read data in my function from any point in my db and use it for comparison etc.?
You need to asynchronously get the "history/extrema/maximum/temp" and then compare and set. But that would be a costly operation and takes time!
In cloud functions, you can store global variable, they will survive until the function timesout. So if your function is not called for some time, the system destroys your function , and you loose the variable.
So a good way to do is,if the global variable,say "maxTemp" is null, read the "history/extrema/maximum/temp" and store it in a global varaible, then proceed to compare with new value, and if it higher, then update the "maxTemp" variable and the "history/extrema/maximum/temp" also.
A quick overview of the situation:
For simplicity of complicated data, I've got a series of JavaScript objects (built using function Object Factories) that handle auditing data, so for example, you have a child object inside a parent object inside a user object (these are basically arrays). These objects contain functions that help process the data they contain, both in and out, when in the UI.
On the top level, a single JSON function call pulls the data the objects contain out, without the functions (which is what I want to happen, as only the data is stored in an external database).
The problem
However, on pulling the information back from the database, using JSON.parse and assigning the result into the top level object holder obviously only contains just the data, no functions.
Bearing in mind the object arrays won't be initialised (as they won't know the length until extracted from the JSON data), how can I reinsert the JSON data back into the objects, functions included?
Is there some way of saying to the parser 'for this level of data, use this object, for this level, use this one'?
[Note: due to the nature of the organisation and security, external libraries aren't permitted. No jQuery suggestions etc.]
I want to receive data from JavaScript file using PHP in Firebase in following structure.
Not like this.
From how I understood your question, I think you're looking to have your data to be added with an auto-generated Firebase id. So I think what you're looking for is the push() method:
Generates a new child location using a unique key and returns its Reference.
This is the most common pattern for adding data to a collection of items.
If you provide a value to push(), the value will be written to the generated location. If you don't pass a value, nothing will be written to the Database and the child will remain empty (but you can use the Reference elsewhere).
The unique key generated by push() are ordered by the current time, so the resulting list of items will be chronologically sorted. The keys are also designed to be unguessable (they contain 72 random bits of entropy).
Also see Firebase Database: Read and Write Data on the Web - Update specific fields.