Develop Reference

Optimalization of firebase query. Getting data by ids

I'm new in Firebase. I would like to create an app (using Angular and AngularFire library), which shows current price of some wares. I have list all available wares in Firebase Realtime Database in the following format:
"warehouse": {
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id2": {
"id": "id2",
"name": "name2",
"price": "15.00"
},
... //much more stuff
}
}
I'm using ngrx with my app, so I think that I can load all wares to store as an object not list because normalizing state tree. I wanted load wares to store in this way:
this.db.object('warehouse/wares').valueChanges();
The problem is wares' price will be refresh every 5 minutes. The number og wares is huge (about 3000 items) so one response will be weight about 700kB. I know that I will exceed limit downloaded data in a short time, in this way.
I want limit the loading data to interesing for user, so every user will can choose wares. I will store this choices in following way:
"users": {
"user1": {
"id": "user1",
"wares": {
"id1": {
"order": 1
},
"id27": {
"order": 2
},
"id533": {
"order": 3
}
},
"waresIds": ["id1", "id27", "id533"]
}
}
And my question is:
Is there a way to getting wares based on waresIds' current user? I mean, does it exist way to get only wares, whose ids are in argument array? F.e.
"wares": {
"id1": {
"id": "id1",
"name": "name1",
"price": "0.99"
},
"id27": {
"id": "id27",
"name": "name27",
"price": "0.19"
},
"id533": {
"id": "id533",
"name": "name533",
"price": "1.19"
}
}
for query like:
this.db.object('warehouse/wares').contains(["id1", "id27", "id533"]).valueChanges();
I saw query limits in Angular Fire like equalTo and etc. but every is for list. I'm totally confused. Is there anyone who can help me? Maybe I'm making mistakes in the design of the app structure. If so, I am asking for clarification.

Because you are saving the ids inside user try this way.
wares: Observable<any[]>;
//inside ngOnInit or function
this.wares = this.db.list('users/currentUserId/wares').snapshotChanges().map(changes => {
return changes.map(c => {
const id = c.payload.key; //gets ids under users/wares/ids..
let wares=[];
//now get the wares
this.db.list('warehouse/wares', ref => ref.orderByChild('id').equalTo(id)).valueChanges().subscribe(res=>{
res.forEach(data=>{
wares.push(data);
})
});
return wares;
});
});

There are two things you can do. I don't believe Firebase allows you to query for multiple equals values at once. You can however loop over the array of "ids" and query for each one directly.
I am assuming you already queried for "waresIds" and you've stored those ID's in an array named idArray:
for id in idArray {
database.ref('warehouse/wares').orderByChild('id').equalTo(id).once('value').then((snapshot) => {
console.log(snapshot.val());
})
}
In order to use the above query efficiently you'll have to index your data on id.
Your second option would be to use .childChanged to get only the updated data after your initial fetch. This should cut down drastically on the amount of data you need to download.

Yes , you can get exactly data that you want in firebase,
See official Firebase documents about filtering
You need to get each waresID
var waresID = // logic to get waresID
var userId = // logic to get userId
var ref = firebase.database().ref("wares/" + userId).child(waresID);
ref.once("value")
.then(function(snapshot) {
console.log(snapshot.val());
});
this will return only data related to that waresID or userId
Note: this is javascript code, i hope this will work for you.

How to insert new child data in Firebase web app?

I am building a simple car database app. When a user adds a new car it creates the below JSON data with blank fields for the gallery:
{
"name": "Cars",
"gallery": [{
"img": "http://",
"title": "BMW"
}, {
"img": "http://",
"title": "Lexus"
}, {
"img": "http://",
"title": "Ferrari"
}],
"pageId": "23",
"storeURL": "/app/cars/gallery"
}
Now what I want to do in the app is if a user adds a new gallery image it should add new child after the last gallery picture. So after Ferrari it should insert a new object. I cannot seem to do this as every tutorial leads to a dead end and also the official Google docs specify that set() will replace all data so if you use update() it will add and in this case it does not add.
This code deletes all data:
function add(){
var data = [
{
title: 'Mercedes',
img: 'http://'
}
]
var postData = {
pageGallery: data
};
var updates = {};
updates['/app/cars/'] = postData;
firebase.database().ref().update(updates);
console.log('New car added');
}
And if I change and add the child:
firebase.database().ref().child('gallery').update(updates);
It does not do anything to do the data. Please can you help?

See this blog post on the many reasons not use arrays in our Firebase Database. Read it now, I'll wait here...
Welcome back. Now that you know why your current approach will lead to a painful time, let's have a look at how you should model this data. The blog post already told you about push IDs, which are Firebase's massively scalable, offline-ready approach to array-like collections.
As our documentation on reading and writing lists of data explains, you can add items to a list by calling push().
So if this creates a car:
function addStore(){
var rootRef = firebase.database().ref();
var storesRef = rootRef.child('app/cars');
var newStoreRef = storesRef.push();
newStoreRef.set({
name: "Cars",
"pageId": "23",
"storeURL": "/app/cars/gallery"
});
}
Then to add an image to the gallery of that store:
var newCarRef = newStoreRef.child('gallery').push();
newCarRef.set({
title: 'Mercedes',
img: 'http://'
})
This will add a new car/image to the end of the gallery (creating the gallery if it doesn't exist yet).

If somebody wants it With Error Handling :
var rootRef = firebase.database().ref();
var storesRef = rootRef.child('Lorelle/Visitors');
var newStoreRef = storesRef.push();
newStoreRef.set($scope.CarModal,
function(error) {
//NOTE: this completion has a bug, I need to fix.
if (error) {
console.log("Data could not be saved." + error);
Materialize.toast('Error: Failed to Submit' + error, 2000);
} else {
console.log("Data saved successfully.");
Materialize.toast('Data Submitted, Thank You.', 2000);
}
});
}

Firebase - how to find all the match items with auto-generated ID, where values are false

Following the techniques on https://www.firebase.com/docs/web/guide/structuring-data.html
Given the url root is: https://myApp.firebaseio.com
And the data is:
// Tracking two-way relationships between users and groups
{
"users": {
"mchen": {
"name": "Mary Chen",
"groups": {
"alpha": true,
"bob": false, // <- I want this
"charlie": true,
"dave": false // <- I want this
}
},
...
},
"groups": {
"alpha": {
"name": "Alpha Group",
"members": {
"mchen": true,
"hmadi": true
}
},
...
}
}
Is it possible to construct a Firebase query to find all the groups that have the value 'false' for user "mchen" (basically I want bob and dave)? How?
e.g. new Firebase('https://myApp.firebaseio.com/users/mchen/groups') ...
In my real code, alpha, bob, charlie and dave are Firebase auto-generated keys, but the same query should solve my problem. (I'm using Firebase Web version)
Thanks in advance.

Finally got it working. I'm not an expert, but I hope this helps anybody not familiar with Firebase:
If you only use Firebase:
var ref = new Firebase('https://myApp.firebaseio.com/users/mchen/groups');
ref.orderByValue().equalTo(false).on('value', function(data){
console.log(data.val());
})
If you use AngularFire:
var ref = new Firebase('https://myApp.firebaseio.com/users/mchen/groups');
$firebaseArray(ref.orderByValue().equalTo(false)).$loaded().then(function(data){
console.log(data);
});
Note:
orderByValue() is the one to use without knowing the keys
Use equalTo(false), not equalTo('false')
$firebaseArray is the one that returns multi items, not $firebaseObject
A rule is also required for performance:
{
"rules": {
"users": {
"$userName": { // A better way would be using an auto-generated ID in here
"groups": {
".indexOn" : ".value" // add index on the value - notice it's .value
}
}
}
}
}

#Sequelize while adding retrieve all attributes

I would like to find a way to retrieve all my attributes while inserting in my Database.
models.association.build(associationParams)
.save()
.then(function(assoAdded){
return next(assoAdded);
}).catch(function(err){
// # TODO : implement error Handler
return next(err);
});
I got this :
{
"idAssoParente": null,
"id": 420,
"name": "a",
"email": "aa#aa.aa",
"updated_at": "2015-07-29T17:12:47.000Z",
"created_at": "2015-07-29T17:12:47.000Z"
}
But I want to return all my fields like description , phone , city from my database even if they are empty. Should I necessarily do a find after adding to get all my fields or does it exist a way to retrieve my fields without doing an other request ?
Thanks

In short, yes you would need to query your db to return the info. I just started using Sequelize but I found the following worked for me.
// if all the info you need is in your user
Users.build({req.body})
.save()
.then(function(newUser){
Users.find({where: {UserID: newUser.UserID}}).then(function(user){
//resolve your promise as you please.
});
// or if address info is in another model you can use eager loading.
Users.build({req.body})
.save()
.then(function(newUser){
Users.find({where: {UserID: newUser.UserID},
include: [{
model: address
}]
}).then(function(user){
//resolve your promise as you please.
});

Firebase: How do I retrieve records from my data for which a specific key exists?

I have data in firebase that looks like this:
"application": {
"companies": {
"firebase": {
"creation": {
"name": "Firebase Inc",
"location": "USA"
},
"google": {
"creattion": {
"name": "Google Inc",
"location": "USA"
}
}
"facebook": {
},
"apple": {
}
}
}
}
There are tens of thousands of records under companies key. How do i efficiently execute following queries?
How do I query only the records for which key creation is present under their name?
How do I query only the records that DO NOT have key creation present under their name?
I also want to call .on('child_added') on the returned result set so that I can process only those specific records later on. Is it possible?

EDIT: Simpler way without using an extra parameter
Queries
Here are the queries to do this without having to use an extra parameter:
Find the companies without creation:
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").equalTo(null);
Find the companies with creation:
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").startAt(!null);
You would add ".indexOn": "creation" to the rules.
Edit 2: I was curious, so I pushed 11,000 records to /companies2 (half with creation children, half without). I was able to retrieve 5500 matching records in ~4 seconds using the above queries (or one of the variants I've shown below).
Edit 3: If you're running these queries frequently, it might be worth it to separate children of /companies into two bins based the presence of creation. That way, you can read the two segments separately without having to rely on queries.
Factory
Here is what the revised factory would look like (I've revised the PLNKR to match):
app.factory("CompaniesFactory",function($q, fbUrl){
return function(hasCreation){
var deferred = $q.defer();
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation");
var query;
if (hasCreation) {
query = ref.startAt(!null);
// or:
// query = ref.startAt(true);
} else {
query = ref.equalTo(null);
// or:
// query = ref.endAt(!null);
// query = ref.endAt(true);
}
query.once("value", function(dataSnapshot){
deferred.resolve(dataSnapshot.val());
}, function (error) {
deferred.reject(error);
});
return deferred.promise;
}
});
And yes, it is possible to call .on('child_added') on the returned dataSnapshot. See DataSnapshot.ref().
Original answer using an extra parameter:
(Keeping this for reference)
Another way to do it would be by adding another parameter called hasCreation to children of companies that have creation, and query by that.
Data
The query would then be var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
If hasCreation in the query is null, the query will return the companies without a hasCreation child.
If hasCreation in the query is true, the query will return the companies with hasCreation===true.
{
"company1" : {
"creation" : {
"name" : "company1"
},
"hasCreation" : true
},
"company2" : {
"name" : "company2"
},
"company3" : {
"name" : "company3"
},
"company4" : {
"creation" : {
"name" : "company4"
},
"hasCreation" : true
}
}
Rules
You would add the ".indexOn" : "hasCreation" to your rules like so:
"so:29179389":{
".read" : true,
".write" : true,
"companies" : {
".indexOn" : "hasCreation"
}
}
Companies Factory
app.factory("CompaniesFactory",function($q, fbUrl){
return function(hasCreation){
var deferred = $q.defer();
if (!hasCreation) {
hasCreation = null;
}
var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
ref.once("value", function(dataSnapshot){
deferred.resolve(dataSnapshot.val());
});
return deferred.promise;
}
});
Controller
app.controller('HomeController',function($scope,fbUrl,CompaniesFactory) {
$scope.getCompanies = function(hasCreation) {
var companies = new CompaniesFactory(hasCreation).then(function(data){
console.log(data);
$scope.companies = data;
});
}
});
HTML
<body ng-app="sampleApp">
<div ng-controller="HomeController">
<button ng-click="getCompanies(true)">Find with creation</button>
<button ng-click="getCompanies(false)">Find without creation</button>
<h2>Companies:</h2>
{{companies}}
</div>
</body>

What I would do, is I would set a condition to verify if your xxx.firebaseio.com/Application/companies/______/creation exists. In a empty blank, you can set for loop to irritate over the array of companies.Then, you can create two arrays with angular.forEach: one including those companies which do have 'creation', and the other array in which the elements do not include the 'creation'.
Hope that helps :)
Edit:
There is another approach to this question, in this thread:
Angularfire: how to query the values of a specific key in an array?