Optimalization of firebase query. Getting data by ids - javascript

I'm new in Firebase. I would like to create an app (using Angular and AngularFire library), which shows current price of some wares. I have list all available wares in Firebase Realtime Database in the following format:
"warehouse": {
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id2": {
"id": "id2",
"name": "name2",
"price": "15.00"
},
... //much more stuff
}
}
I'm using ngrx with my app, so I think that I can load all wares to store as an object not list because normalizing state tree. I wanted load wares to store in this way:
this.db.object('warehouse/wares').valueChanges();
The problem is wares' price will be refresh every 5 minutes. The number og wares is huge (about 3000 items) so one response will be weight about 700kB. I know that I will exceed limit downloaded data in a short time, in this way.
I want limit the loading data to interesing for user, so every user will can choose wares. I will store this choices in following way:
"users": {
"user1": {
"id": "user1",
"wares": {
"id1": {
"order": 1
},
"id27": {
"order": 2
},
"id533": {
"order": 3
}
},
"waresIds": ["id1", "id27", "id533"]
}
}
And my question is:
Is there a way to getting wares based on waresIds' current user? I mean, does it exist way to get only wares, whose ids are in argument array? F.e.
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id27": {
"id": "id27",
"name": "name27",
"price": "0.19"
},
"id533": {
"id": "id533",
"name": "name533",
"price": "1.19"
}
}
for query like:
this.db.object('warehouse/wares').contains(["id1", "id27", "id533"]).valueChanges();
I saw query limits in Angular Fire like equalTo and etc. but every is for list. I'm totally confused. Is there anyone who can help me? Maybe I'm making mistakes in the design of the app structure. If so, I am asking for clarification.

Because you are saving the ids inside user try this way.
wares: Observable<any[]>;
//inside ngOnInit or function
this.wares = this.db.list('users/currentUserId/wares').snapshotChanges().map(changes => {
return changes.map(c => {
const id = c.payload.key; //gets ids under users/wares/ids..
let wares=[];
//now get the wares
this.db.list('warehouse/wares', ref => ref.orderByChild('id').equalTo(id)).valueChanges().subscribe(res=>{
res.forEach(data=>{
wares.push(data);
})
});
return wares;
});
});

There are two things you can do. I don't believe Firebase allows you to query for multiple equals values at once. You can however loop over the array of "ids" and query for each one directly.
I am assuming you already queried for "waresIds" and you've stored those ID's in an array named idArray:
for id in idArray {
database.ref('warehouse/wares').orderByChild('id').equalTo(id).once('value').then((snapshot) => {
console.log(snapshot.val());
})
}
In order to use the above query efficiently you'll have to index your data on id.
Your second option would be to use .childChanged to get only the updated data after your initial fetch. This should cut down drastically on the amount of data you need to download.

Yes , you can get exactly data that you want in firebase,
See official Firebase documents about filtering
You need to get each waresID
var waresID = // logic to get waresID
var userId = // logic to get userId
var ref = firebase.database().ref("wares/" + userId).child(waresID);
ref.once("value")
.then(function(snapshot) {
console.log(snapshot.val());
});
this will return only data related to that waresID or userId
Note: this is javascript code, i hope this will work for you.

Related

My Query Quagmire: My node.js program works just fine, except when I try to execute queries, or filtered HTTP requests. Why?

I have been working on the backend of my app. At this point, it can access all data in a data base, and output it. I'm trying to implement some queries, so that the user can filter out the content that is returned. My DAL/DAO, looks like this
let mflix //Creates a variable used to store a ref to our DB
class MflixDAO {
static async injectDB(conn){
if(mflix){
return
}
try{
mflix = await conn.db(process.env.JD_NS).collection("movies")
}catch(e){
console.error('Unable to establish a collection handle in mflixDAO: ' + e)
}
}
// Creates a query to fetch data from the collection/table in the DB
static async getMovies({
mflix.controller
filters = null,
page = 0,
moviesPerPage = 20,
} = {}) {
let query
if (filters){
// Code
if("year" in filters){
query = {"year": {$eq: filters["year"]}}
}
// Code
}
// Cursor represents the returned data
let cursor
try{
cursor = await mflix.find(query)
}catch(e){
console.error('Unable to issue find command ' + e)
return {moviesList: [], totalNumMovies: 0}
}
const displayCursor = cursor.limit(moviesPerPage).skip(moviesPerPage * page)
try{
const moviesList = await displayCursor.toArray() // Puts data in an array
const totalNumMovies = await mflix.countDocuments(query) // Gets total number of documents
return { moviesList, totalNumMovies}
} catch(e){
console.error('Unable to convert cursor to array or problem counting documents ' + e)
return{moviesList: [], totalNumMovies: 0}
}
}
}
export default MflixDAO
Just so you know, I am using a sample database from MongoDB Atlas. I am using Postman to test HTTP requests. All the data follows JSON format
Anyway, when I execute a basic GET request. The program runs without any problems. All the data outputs as expected. However, if I execute something along the lines of
GET http://localhost:5000/api/v1/mflix?year=1903
Then moviesList returns an empty array [], but no error message.
After debugging, I suspect the problem lies either at cursor = await mflix.find(query) or displayCursor = cursor.limit(moviesPerPage).skip(moviesPerPage * page), but the callstacks for those methods is so complex for me, I don't know what to even look for.
Any suggestions?
Edit: Here is an example of the document I am trying to access:
{
"_id": "573a1390f29313caabcd42e8",
"plot": "A group of bandits stage a brazen train hold-up, only to find a determined posse hot on their heels.",
"genres": [
"Short",
"Western"
],
"runtime": 11,
"cast": [
"A.C. Abadie",
"Gilbert M. 'Broncho Billy' Anderson",
"George Barnes",
"Justus D. Barnes"
],
"poster": "https://m.media-amazon.com/images/M/MV5BMTU3NjE5NzYtYTYyNS00MDVmLWIwYjgtMmYwYWIxZDYyNzU2XkEyXkFqcGdeQXVyNzQzNzQxNzI#._V1_SY1000_SX677_AL_.jpg",
"title": "The Great Train Robbery",
"fullplot": "Among the earliest existing films in American cinema - notable as the first film that presented a narrative story to tell - it depicts a group of cowboy outlaws who hold up a train and rob the passengers. They are then pursued by a Sheriff's posse. Several scenes have color included - all hand tinted.",
"languages": [
"English"
],
"released": "1903-12-01T00:00:00.000Z",
"directors": [
"Edwin S. Porter"
],
"rated": "TV-G",
"awards": {
"wins": 1,
"nominations": 0,
"text": "1 win."
},
"lastupdated": "2015-08-13 00:27:59.177000000",
"year": 1903,
"imdb": {
"rating": 7.4,
"votes": 9847,
"id": 439
},
"countries": [
"USA"
],
"type": "movie",
"tomatoes": {
"viewer": {
"rating": 3.7,
"numReviews": 2559,
"meter": 75
},
"fresh": 6,
"critic": {
"rating": 7.6,
"numReviews": 6,
"meter": 100
},
"rotten": 0,
"lastUpdated": "2015-08-08T19:16:10.000Z"
},
"num_mflix_comments": 0
}
EDIT: It seems to be a datatype problem. When I request a data with a string/varchar type, the program returns values that contain that value. Example:
Input:
GET localhost:5000/api/v1/mflix?rated=TV-G
Output:
{
"_id": "XXXXXXXXXX"
// Data
"rated" = "TV-G"
// Data
}
EDIT: The problem has nothing to do with anything I've posted up to this point it seems. The problem is in this piece of code:
let filters = {}
if(req.query.year){
filters.year = req.query.year // This line needs to be changed
}
const {moviesList, totalNumMovies} = await MflixDAO.getMovies({
filters,
page,
moviesPerPage,
})
I will explain in the answer below
Ok so the problem, as it turns out, is that when I make an HTTP request, the requested value is passed as a string. So in
GET http://localhost:5000/api/v1/mflix?year=1903
the value of year is registered by the program as a string. In other words, the DAO ends up looking for "1903" instead of 1903. Naturally, year = "1903" does not exist. To fix this, the line filters.year = req.query.year must be changed to filters.year = parseInt(req.query.year).

Error getting a single value from a collection

I have a collection called notification and i am trying to get a single value with findOne()
var allnotices = Notifications.findOne({eventownernumber:"2"},{sort: {noticedate: -1, limit: 1}}).noticemessage;
I want to get the value where the eventownernumber is 2 and i want to get the latest record and i only want one record.
Even though noticemessage is part of the row fields,i get the error that noticemessage is undefined.
This is the schema
{
"_id": "tmkWCydSKZtYdrKTZ",
"eventoriginalid": "3bXvARk6K6yhee6Hi",
"lat": "-1.851881824302658",
"lng": "96.987469482421875",
"eventownernumber": "1",
"eventownernames": "Test 1",
"eventtitle": "ci",
"eventtime": "08:05",
"invited": "0",
"eventduration": "21",
"eventtype": "notification",
"eventcategory": "hackathon",
"eventstatus": "11",
"createdAt": {
"$date": "2016-11-02T12:38:40.378Z"
},
"noticedate": {
"$date": "2016-11-02T16:50:53.394Z"
},
"noticenumber": "2",
"noticenames": "Test 2",
"noticemessage": "Test 2 has joined your event ci",
"noticestatus": "12"
}
Why is noticemessage undefined?.
There are four basic possibilities why Collection.findOne(query).key could yield an error:
There is no document matching the query therefore you're trying to reference undefined.key
The key in question doesn't exist in the returned document
The document exists in the database but isn't being published by the server and being subscribed to by the client
The document exists and is published and subscribed to but the subscription is not yet .ready(), i.e. you need to wait before you can access it.
A common defensive pattern is:
const oneDoc = myCollection.findOne(query);
let myVar = oneDoc && oneDoc.key;
if ( myVar ) {
// do the thing
} else {
// handle the error
}
You need to save the number as integer for eventownernnumber (and please write it like eventOwnerNumber, which is a good practice for readability), not string. Either use input type="number" or convert the value to integer like this:
Number(valueHere);
The rest of your query looks fine to me but you don't need limit since you do findOne() and you find the newest inserted doc with noticedate: -1
Another thing is, you need to save the date like this in your insert():
noticeDate: new Date() //your current query should give you the right document after this change

Firebase - how to find all the match items with auto-generated ID, where values are false

Following the techniques on https://www.firebase.com/docs/web/guide/structuring-data.html
Given the url root is: https://myApp.firebaseio.com
And the data is:
// Tracking two-way relationships between users and groups
{
"users": {
"mchen": {
"name": "Mary Chen",
"groups": {
"alpha": true,
"bob": false, // <- I want this
"charlie": true,
"dave": false // <- I want this
}
},
...
},
"groups": {
"alpha": {
"name": "Alpha Group",
"members": {
"mchen": true,
"hmadi": true
}
},
...
}
}
Is it possible to construct a Firebase query to find all the groups that have the value 'false' for user "mchen" (basically I want bob and dave)? How?
e.g. new Firebase('https://myApp.firebaseio.com/users/mchen/groups') ...
In my real code, alpha, bob, charlie and dave are Firebase auto-generated keys, but the same query should solve my problem. (I'm using Firebase Web version)
Thanks in advance.
Finally got it working. I'm not an expert, but I hope this helps anybody not familiar with Firebase:
If you only use Firebase:
var ref = new Firebase('https://myApp.firebaseio.com/users/mchen/groups');
ref.orderByValue().equalTo(false).on('value', function(data){
console.log(data.val());
})
If you use AngularFire:
var ref = new Firebase('https://myApp.firebaseio.com/users/mchen/groups');
$firebaseArray(ref.orderByValue().equalTo(false)).$loaded().then(function(data){
console.log(data);
});
Note:
orderByValue() is the one to use without knowing the keys
Use equalTo(false), not equalTo('false')
$firebaseArray is the one that returns multi items, not $firebaseObject
A rule is also required for performance:
{
"rules": {
"users": {
"$userName": { // A better way would be using an auto-generated ID in here
"groups": {
".indexOn" : ".value" // add index on the value - notice it's .value
}
}
}
}
}

IBM worklight JSON store remove array of documents

I am working in IBM worklight hybrid app,i am using JSON store to store data,to remove records from collection,i am using id and i could able delete single record using id,how to delete multiple records together from JSON store,if any example is there it will be useful,can anyone help me in doing this?Thanks in advance.
Delete function:
var id = JSON.parse(localStorage.getItem('jsonindex'));
var query = {
_id: id
};
var options = {
push: true
};
try {
WL.JSONStore.get(PEOPLE_COLLECTION_NAME).remove(query, options)
.then(function (res) {
console.log("REMOVE_MSG");
})
.fail(function (errorObject) {
console.log("Not Removed");
});
} catch (e) {
alert(INIT_FIRST_MSG);
}
JSON data
[{
"_id": 16,
"json": {
"name": " Debit",
"cardmonth": " 8",
"cardyear": " 2028",
"number": " 4216170916239547"
}
}, {
"_id": 17,
"json": {
"name": " Credit",
"cardmonth": " 7",
"cardyear": " 2027",
"number": " 4216170916239547"
}
}]
Try:
WL.JSONStore.get('collectionName').remove([...], options);
Replace ... with {_id: 1}, {_id: 2} or whatever query you want to use to remove documents.
If it doesn't work, please upgrade to the latest version of Worklight and try again.
Relevant:
PI10959: JSONSTORE FAILS TO REMOVE ALL DOCS IN THE DOC ARRAY WHEN A DOC ARRAY IS PASSED
IBM Worklight JSONStore | Remove Document from Collection and erase it from memory
If you are able to delete the single record. its easy to delete multiple record. but it raises some performance issues you have so many records.
var id="3"; If you are deleting this one by using Delete method. just do it for multiple records
var ids=[];
when user selects item ids.push(item.id);
for(i=0;i<ids.length;i++){
Delete(ids[i]); //its your Delete method
}

Iteration in handlebar using backbone

I'm using backbone and handlebars for templating and i'm new to this.
My current json is in the below format and the code works fine.
[
{
"id": "10",
"info": {
"name": "data10"
}
},
{
"id": "11",
"info": {
"name": "data11"
}
}
]
But when i change my json structure to something like shown below i'm having difficulty in getting things to be populated.
{
"total_count": "10",
"dataElements": [
{
"id": "10",
"info": {
"name": "data10"
}
},
{
"id": "11",
"info": {
"name": "data11"
}
}
]
}
How can i populate name, info and total_count keeping the current code structure ?
JSFiddle : http://jsfiddle.net/KTj2K/1/
Any help really appriciated.
A few things that you need to do in order for this to work.
Replace Backbone's core 'reset' on your collection with a custom one that understands the data you are passing to it. For example:
reset: function (data) {
this.totalCount = data.total_count;
Backbone.Collection.prototype.reset.call(this, data.dataElements);
}
Now when you reset your collection, it will pull the total_count out of the object you are resetting it with, and use Backbone's core reset with the dataElement array. Keep in mind you may have to do a similar thing with 'parse' if you're intending on pulling this from the server.
I'd recommend that (if your example looks anything like the real code you're working with) you reset your collection before getting to rendering.
var dataCollectionList = new dataCollection();
dataCollectionList.reset(jsonData);
var App = new AppView({model : dataCollectionList});
Now in your view's "render" method you can grab the 'totalCount' property off the collection -
render : function() {
//Should spit the total count into the element, just as an example
this.$el.append(this.model.totalCount);
//or console.log it
console.log(this.model.totalCount);
return this;
}
Voila. Side note - as someone who works with Backbone a lot, it drives me nuts when people set an attribute of something like "model" (i.e. peopleModel, itemModel, etc) and it ends up being a backbone collection. It's much clearer to name it after what it is - though some MVC purists may disagree a bit.
Also, in this code block:
_.each(this.model.models, function (myData) {
$(this.el).append(new ItemView({model:myData}).render().el);
}, this);
You don't need to do _.each(this.model.models.......). Since you're working with a collection, the collection has a built in 'each' method.
this.model.each(function (myData) { ..... } , this);
Quite a bit cleaner.

Categories

Resources