Develop Reference

Optimalization of firebase query. Getting data by ids

I'm new in Firebase. I would like to create an app (using Angular and AngularFire library), which shows current price of some wares. I have list all available wares in Firebase Realtime Database in the following format:
"warehouse": {
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id2": {
"id": "id2",
"name": "name2",
"price": "15.00"
},
... //much more stuff
}
}
I'm using ngrx with my app, so I think that I can load all wares to store as an object not list because normalizing state tree. I wanted load wares to store in this way:
this.db.object('warehouse/wares').valueChanges();
The problem is wares' price will be refresh every 5 minutes. The number og wares is huge (about 3000 items) so one response will be weight about 700kB. I know that I will exceed limit downloaded data in a short time, in this way.
I want limit the loading data to interesing for user, so every user will can choose wares. I will store this choices in following way:
"users": {
"user1": {
"id": "user1",
"wares": {
"id1": {
"order": 1
},
"id27": {
"order": 2
},
"id533": {
"order": 3
}
},
"waresIds": ["id1", "id27", "id533"]
}
}
And my question is:
Is there a way to getting wares based on waresIds' current user? I mean, does it exist way to get only wares, whose ids are in argument array? F.e.
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id27": {
"id": "id27",
"name": "name27",
"price": "0.19"
},
"id533": {
"id": "id533",
"name": "name533",
"price": "1.19"
}
}
for query like:
this.db.object('warehouse/wares').contains(["id1", "id27", "id533"]).valueChanges();
I saw query limits in Angular Fire like equalTo and etc. but every is for list. I'm totally confused. Is there anyone who can help me? Maybe I'm making mistakes in the design of the app structure. If so, I am asking for clarification.

Because you are saving the ids inside user try this way.
wares: Observable<any[]>;
//inside ngOnInit or function
this.wares = this.db.list('users/currentUserId/wares').snapshotChanges().map(changes => {
return changes.map(c => {
const id = c.payload.key; //gets ids under users/wares/ids..
let wares=[];
//now get the wares
this.db.list('warehouse/wares', ref => ref.orderByChild('id').equalTo(id)).valueChanges().subscribe(res=>{
res.forEach(data=>{
wares.push(data);
})
});
return wares;
});
});

There are two things you can do. I don't believe Firebase allows you to query for multiple equals values at once. You can however loop over the array of "ids" and query for each one directly.
I am assuming you already queried for "waresIds" and you've stored those ID's in an array named idArray:
for id in idArray {
database.ref('warehouse/wares').orderByChild('id').equalTo(id).once('value').then((snapshot) => {
console.log(snapshot.val());
})
}
In order to use the above query efficiently you'll have to index your data on id.
Your second option would be to use .childChanged to get only the updated data after your initial fetch. This should cut down drastically on the amount of data you need to download.

Yes , you can get exactly data that you want in firebase,
See official Firebase documents about filtering
You need to get each waresID
var waresID = // logic to get waresID
var userId = // logic to get userId
var ref = firebase.database().ref("wares/" + userId).child(waresID);
ref.once("value")
.then(function(snapshot) {
console.log(snapshot.val());
});
this will return only data related to that waresID or userId
Note: this is javascript code, i hope this will work for you.

Failing to add a key and value to a javascript object

I'm new to JavaScript and Node etc but enjoying the experience of developing - last serious development i did was in the 90s - I used to be a 370 assembler programmer back 30 years ago!
I've been stuck on this all day.
My issue is that I am trying to add a new element to the entries in an array of objects. I have simplified my code here in the hope that I am doing something blindingly stupid and obviously wrong though I have tried this in javascript in the browser and I know I am barking up the right tree.
What I am trying to do is add a new key/value pair to the collectibles returned. I have simplified the code here but in essence the value is coming for a different collection. the issue is that the lines that set the new keys are having no effect on my objects in the array collectibles. I have tried several methods all of which I believe should work.
I can read the key/values in the returned array of objects and I can also set existing keys to new values, but setting a new key is not working.
Here is my code:
var sendJsonResponse = function (res, status, content) {
res.status(status);
res.json(content);
};
var mongoose = require("mongoose");
var Coll = mongoose.model("collectible");
var Img = mongoose.model("image");
module.exports.collectiblesList = function (req, res) {
var firstImage_ids = [];
var imageDir = "https://s3.amazonaws.com/xxxxxxxxxxx/images/";
var defaultImage = imageDir + "default.jpg";
// get the collectibles
Coll.find(req.body).exec(function (err, collectibles) {
if (!collectibles) {
// not found
sendJsonResponse(res, 404, {"message": "no collectibles found"});
return;
}
if (err) {
console.log(collectible);
sendJsonResponse(res, 400, err);
return;
}
//found the collectibles
// now find the list of images we need to pull (each collectible can have multiple images
collectibles.forEach(function (collectible, index) {
if (collectible.image_ids) {
firstImage_ids.push(collectible.image_ids[0]);
collectibles[index].imageUrl = defaultImage; // this is the line that has no effect.
collectible.imageUrl = defaultImage; // tried this too.
collectible["imageUrl"] = defaultImage; // and this. :-(
console.log(collectible);
}
});
sendJsonResponse(res, 200, collectibles);
});
};
Using Postman to test this API. This is what is being returned:
[
{
"_id": "5a43c61134aaea2025158cab",
"name": "A-0_5",
"title": "Occoneechee Lodge 104: A-0_5",
"issueDate": null,
"quantity": null,
"event_id": null,
"type": "A - Arrowhead patches",
"tag_ids": [
"5a401a17b720186ab61c649e"
],
"description": "Elangomat, no name",
"organisation_id": "5a413340b720186ab61c661e",
"supercedes_id": null,
"sortName": "A-000.5",
"image_ids": [
"5a41685ab720186ab61c663d"
],
"deleted": false,
"schemaVersion": "1.0"
},
{
"_id": "5a43c63334aaea2025158cac",
"name": "A-1",
"title": "Occoneechee Lodge 104: A-1",
"issueDate": null,
"quantity": null,
"event_id": null,
"type": "A - Arrowhead patches",
"tag_ids": [
"5a401a17b720186ab61c649e"
],
"description": "Service",
"organisation_id": "5a413340b720186ab61c661e",
"supercedes_id": null,
"sortName": "A-001",
"image_ids": [
"5a41685ab720186ab61c663e"
],
"deleted": false,
"schemaVersion": "1.0"
}]
i.e. it contains no imageUrl keys.
thanks for the assist.
WWW
GT

I might have misunderstood. But if your issue is that collectibles[i].imageUrl is later on returning undefined/null, then i believe it's simply because you need to add
collectibles.save();
after setting the collectibles[i].imageUrl value. you can add that line after the foreach loop, before the json response.
collectibles.forEach(function (collectible, index) {
if (collectible.image_ids) {
firstImage_ids.push(collectible.image_ids[0]);
collectible.imageUrl = defaultImage;
}
});
collectibles.save(); //here, this saves the changes done to the mongoose object.
sendJsonResponse(res, 200, collectibles);

I solved it in the end after a lot of effort. This article explains (but i don't fully yet follow the explanation)
I found that the object has more metadata in it and the actual object is in _doc, so the first (and i suspect frowned upon solution) is to update the code to:
collectible._doc.imageUrl = defaultImage;
The better way was to simply add imageUrl to the mongoose model and leave _doc out of it. The issue there is that this is not a field i intend to populate in the mongoDB.

IBM worklight JSON store remove array of documents

I am working in IBM worklight hybrid app,i am using JSON store to store data,to remove records from collection,i am using id and i could able delete single record using id,how to delete multiple records together from JSON store,if any example is there it will be useful,can anyone help me in doing this?Thanks in advance.
Delete function:
var id = JSON.parse(localStorage.getItem('jsonindex'));
var query = {
_id: id
};
var options = {
push: true
};
try {
WL.JSONStore.get(PEOPLE_COLLECTION_NAME).remove(query, options)
.then(function (res) {
console.log("REMOVE_MSG");
})
.fail(function (errorObject) {
console.log("Not Removed");
});
} catch (e) {
alert(INIT_FIRST_MSG);
}
JSON data
[{
"_id": 16,
"json": {
"name": " Debit",
"cardmonth": " 8",
"cardyear": " 2028",
"number": " 4216170916239547"
}
}, {
"_id": 17,
"json": {
"name": " Credit",
"cardmonth": " 7",
"cardyear": " 2027",
"number": " 4216170916239547"
}
}]

Try:
WL.JSONStore.get('collectionName').remove([...], options);
Replace ... with {_id: 1}, {_id: 2} or whatever query you want to use to remove documents.
If it doesn't work, please upgrade to the latest version of Worklight and try again.
Relevant:
PI10959: JSONSTORE FAILS TO REMOVE ALL DOCS IN THE DOC ARRAY WHEN A DOC ARRAY IS PASSED
IBM Worklight JSONStore | Remove Document from Collection and erase it from memory

If you are able to delete the single record. its easy to delete multiple record. but it raises some performance issues you have so many records.
var id="3"; If you are deleting this one by using Delete method. just do it for multiple records
var ids=[];
when user selects item ids.push(item.id);
for(i=0;i<ids.length;i++){
Delete(ids[i]); //its your Delete method
}

Backbone and best practice getting config JSON

I've got a JSON file that looks like this.
{
"config": {
"setting1": 'blabla',
"setting2": 'blablabla'
},
"content": {
"title": "Title of an exercise.",
"author": "John Doe",
"describtion": "Exercise content."
},
"answers": [
{
"id": "1",
"content": "Dog",
"correct": true
},
{
"id": "2",
"content": "Fish",
"correct": false
}
]
}
Than, I create a Backbone View, combined from content model, and answers (which are randomly selected, but It's not most important now).
I've also got a config, which has settings that will determinate which view and collection methods to use.
It seems like a simple task, but as I'm new to Backbone, I'm wondering which is the best way to fetch JSON file, creating one model with url to JSON and than using parse and initialize creating another models and collections (with answers), or using $.getJSON method that will create exactly the models that I need?
I was trying using $.getJSON
$.getJSON(source, function(data) {
var contentModel = new ContentModel(data.content);
var contentView = new ExerciseView({ model: contentModel });
var answerCollection = new AnswersCollection();
_.each(data.answers, function(answer) {
answerCollection.add(answer);
});
var answersView = new AnswersView({collection: answerCollection});
$(destination).html( contentView.render().el );
$('.answers').append( answersView.el );
)};
But It doesn't seem very elegant solution, I know that this application needs good architecture, cause It will be developed with many other Views based on 'config'.
Hope you guys give me some suggestions, have a good day!

I think what you've done works fine and is correct. But you may need to refactor a little bit since "it will be developed with many other Views based on 'config'".
IMHO, the first thing you need to do is to handle failure in your getJson callback to make the process more robust.
Second, it is useful to create a Factory to generate your views because your logic is to generate different views based on the config data from server. So the factory maybe:
contentViewFactory.generate = function(data) {
var config = data.config;
....
var ActualContentView = SomeContentView;
var contentModel = new ContentModel(data.content);
return = new ActualContentView({ model: contentModel });
}
If your logic is simple, you can have a dict map from config to view class like:
var viewMaps = {
"exercise" : ExerciseView,
"other": SomeOtherView,
//....
}
And if every workflow has a AnswersView you can keep that in your getJSON callback. So maybe now your getJSON looks like this:
$.getJSON(source, function(data) {
// keep the config->view logic in the factory
var contentView = contentViewFactory.generate(data);
var answerCollection = new AnswersCollection();
_.each(data.answers, function(answer) {
answerCollection.add(answer);
});
var answersView = new AnswersView({collection: answerCollection});
$(destination).html( contentView.render().el );
$('.answers').append( answersView.el );
})
.fail(){
//some failure handling
};
Furthermore, if you have common logics in you "ContentView"s, it's natural that you can have a "BaseContentView" or "ContentViewMixin" to extract the common logic and use extends to make your code more OO:
Backbone.View.extend(_.extend({}, ContentViewMixin, {
//.....
}
So if someone is trying to add a new ContentView, he/she just needs to add some code in the factory to make the new View be generated by config. Then extends the ContentViewMixin to implement the new View.

Looking for design pattern to create multiple models/collections out of single JSON responses

I have a Backbone application where the JSON I get from the server isn't exactly 1 on 1 with how I want my models to look. I use custom parse functions for my models, ex:
parse: function(response) {
var content = {};
content.id = response.mediaId;
content.image = response.image.url;
return content;
}
This works. But, in some cases I have an API call where I get lots of information at once, for instance, information about an image with its user and comments:
{
"mediaId": "1",
"image": {
"title": "myImage",
"url": "http://image.com/234.jpg"
},
"user": {
"username": "John"
},
"comments": [
{
"title": "Nice pic!"
},
{
"title": "Great stuff."
}
]
}
How would I go about creating a new User model and a Comments collection from here? This is an option:
parse: function(response) {
var content = {};
content.id = response.mediaId;
content.image = response.image.url;
content.user = new User(response.user);
content.comments = new Comments(response.comments);
return content;
}
The trouble here is, by creating a new User or new Comments with raw JSON as input, Backbone will just add the JSON properties as attributes. Instead, I'd like to have an intermediate parse-like method to gain control over the objects' structure. The following is an option:
parse: function(response) {
// ...
content.user = new User({
username: response.user.username
});
// ...
}
...but that's not very DRY-proof.
So, my question is: what would be a nice pattern to create several models/collections out of 1 JSON response, with control over the models/collections attributes?
Thanks!

It may not be the nicest way possible, but this is how I do it:
content.user = new User(User.prototype.parse(response.user));
The only problem is that the this context in User.parse will be wrong. If you don't have any specific code in the User constructor, you can also do:
content.user = new User();
content.user.set(user.parse(response.user));
I also noticed an interesting note in the Backbone version 0.9.9 change log:
The parse function is now always run if defined, for both collections and models — not only after an Ajax call.
And looking at the source code of Model and Collection constructor, they do it like so:
if (options && options.parse) attrs = this.parse(attrs);
Maybe upgrading to 0.9.9 will give you what you need? If upgrade is not an option, you can of course implement the same in your own constructor.