Develop Reference

Writing to same document in a WriteBatch causes multiple writes in Firebase?

I want to know if, in Firebase, I perform multiple updates to the same document in a batch, will it cause a single write or multiple writes?
For instance, in this example code (in javascript):
const doc = getDocumentReference();
batch.update(doc, { foo: "a" });
batch.update(doc, { bar: "b" });
batch.commit();
Will there be a single write to the document updating everything or there will be two writes, one for each update?
Does the answer change if one of the two operations is a "set" instead of an "update"?
Thanks in advance

I haven't found any reference how the writes are counted, but I found it interesting and tried to figure it out by myself. I prepared test and looked on Metrics explorer on Google Cloud Platform (Firebase database are visible there as well) and it can show the writes_count.
I prepared following codes:
adding whole object at once:
let test = db.collection("test").doc("sdaasd3")
test.set({})
let update = {}
for (let i = 0; i < 100; i++) {
update[i.toString()] = i;
}
test.update(update);
Then updating 100 times:
let test = db.collection("test").doc("sdaasd2")
test.set({})
for (let i = 0; i < 100; i++) {
let update = {}
update[i.toString()] = i;
test.update(update)
}
And in by batch commit:
let test = db.collection("test").doc("sdaasd")
test.set({})
const batch = db.batch();
for (let i = 0; i < 100; i++) {
let update = {}
update[i.toString()] = i;
batch.update(test, update)
}
batch.commit();
I performed them and it seems that batch commit is exactly the same as updating whole object at once and it takes 2 writes (I think one is set which I used to clear object in Firebase). While updating 100 times took 101 writes. Here as it looks (executed in order like in the post 12:57,01:01,01:07)
I am not sure, if this test was reliable enough to your needs, but you can use the Matrix explorer on GCP to analyze that on your own.

How to perform fast search on JSON file?

I have a json file that contains many objects and options.
Each of these kinds:
{"item": "name", "itemId": 78, "data": "Some data", ..., "option": number or string}
There are about 10,000 objects in the file.
And when part of item value("ame", "nam", "na", etc) entered , it should display all the objects and their options that match this part.
RegExp is the only thing that comes to my mind, but at 200mb+ file it starts searching for a long time(2 seconds+)
That's how I'm getting the object right now:
let reg = new RegExp(enteredName, 'gi'), //enteredName for example "nam"
data = await fetch("myFile.json"),
jsonData = await data.json();
let results = jsonData.filter(jsonObj => {
let item = jsonObj.item,
itemId = String(jsonObj.itemId);
return reg.test(item) || reg.test(itemId);
});
But that option is too slow for me.
What method is faster to perform such search using js?

Looking up items by item number should be easy enough by creating a hash table, which others have already suggested. The big problem here is searching for items by name. You could burn a ton of RAM by creating a tree, but I'm going to go out on a limb and guess that you're not necessarily looking for raw lookup speed. Instead, I'm assuming that you just want something that'll update a list on-the-fly as you type, without actually interrupting your typing, is that correct?
To that end, what you need is a search function that won't lock-up the main thread, allowing the DOM to be updated between returned results. Interval timers are one way to tackle this, as they can be set up to iterate through large, time-consuming volumes of data while allowing for other functions (such as DOM updates) to be executed between each iteration.
I've created a Fiddle that does just that:
// Create a big array containing items with names generated randomly for testing purposes
let jsonData = [];
for (i = 0; i < 10000; i++) {
var itemName = '';
jsonData.push({ item: Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15) });
}
// Now on to the actual search part
let returnLimit = 1000; // Maximum number of results to return
let intervalItr = null; // A handle used for iterating through the array with an interval timer
function nameInput (e) {
document.getElementById('output').innerHTML = '';
if (intervalItr) clearInterval(intervalItr); // If we were iterating through a previous search, stop it.
if (e.value.length > 0) search(e.value);
}
let reg, idx
function search (enteredName) {
reg = new RegExp(enteredName, 'i');
idx = 0;
// Kick off the search by creating an interval that'll call searchNext() with a 0ms delay.
// This will prevent the search function from locking the main thread while it's working,
// allowing the DOM to be updated as you type
intervalItr = setInterval(searchNext, 0);
}
function searchNext() {
if (idx >= jsonData.length || idx > returnLimit) {
clearInterval(intervalItr);
return;
}
let item = jsonData[idx].item;
if (reg.test(item)) document.getElementById('output').innerHTML += '<br>' + item;
idx++;
}
https://jsfiddle.net/FlimFlamboyant/we4r36tp/26/
Note that this could also be handled with a WebWorker, but I'm not sure it's strictly necessary.
Additionally, this could be further optimized by utilizing a secondary array that is filled as the search takes place. When you enter an additional character and a new search is started, the new search could begin with this secondary array, switching to the original if it runs out of data.

Looping through a series of GET requests

I have a get request that looks for a specific ID within a tree, and then pulls back the value from that ID. I need to loop through a series of these get requests, each with a similar ID (each ID increases in value by one).
I have created a standard loop using hard coded values but I'm struggling to set the variable based on dynamic values coming out of the tree.
For example, I can set a variable like this:
var cars = [entry.get('sub_menu.sub_menu_link.0.title'), entry.get('sub_menu.sub_menu_link.1.title'), entry.get('sub_menu.sub_menu_link.2.title')];
This grabs all the values from these areas of the tree.
But I don't know how many of these there will be so I can't hard code it in this way. I need to be able to replace 0, 1 and 2 in those values with a loop that adds a new get request and increases the integer between "link." and ".title" each time.
Expected result would be to add as many get requests in to the variable as it finds, with the integer increased for each request, until it finds no more.
Full example code with hard coded get requests is below (which won't actually work because the tree isn't being pulled in. For example purposes only):
Query.fetch()
.then(function success(entry) {
var subMenu = [entry.get('sub_menu.sub_menu_link.0.title'), entry.get('sub_menu.sub_menu_link.1.title'), entry.get('sub_menu.sub_menu_link.2.title')];
var text = "";
var i;
for (i = 0; i < subMenu.length; i++) {
text += subMenu[i] + "<br>";
}
document.getElementById("subMenu-container").innerHTML = text;
},
function error(err) {
// err object
});

I'm answering based on three assumptions:
entry.get is synchronous
entry.get returns null or undefined if it couldn't get the string matching the argument we pass it
Although your code is using promises, you want to keep it to ES5-level language features (not ES2015+)
See inline comments:
Query.fetch()
.then(function success(entry) {
// Loop getting titles until we run out of them, put them in `titles`
var titles = [];
var title;
while (true) { // Loop until `break;`
// Use string concat to include the value of `titles.length` (0, 1, ...) in the string
title = entry.get('sub_menu.sub_menu_link.' + titles.length + '.title');
if (title === null || title === undefined) {
break;
}
titles.push(title);
}
// Use `Array.prototype.join` to join the titles together with <br> in-between
document.getElementById("subMenu-container").innerHTML = titles.join("<br>");
}, function error(err) {
// Handle/report error
});

how to work with a large array in javascript [duplicate]

This question already has answers here:
Best way to iterate over an array without blocking the UI
(4 answers)
Closed 6 years ago.
In my application I have a very big array (arround 60k records). Using a for loop I am doing some operations on it as shown below.
var allPoints = [];
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
if (allPoints.indexOf(this._clusterData[i].attributes.PropertyAddress) == -1) {
allPoints.push(this._clusterData[i].attributes.PropertyAddress);
this._DistClusterData.push(this._clusterData[i])
}
}
When I run this loop the browser hangs as it is very big & in Firefox is shows popup saying "A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete". What can I do so that browser do not hang?

You need to return control back to the browser in order to keep it responsive. That means you need to use setTimeout to end your current processing and schedule it for resumption sometime later. E.g.:
function processData(i) {
var data = clusterData[i];
...
if (i < clusterData.length) {
setTimeout(processData, 0, i + 1);
}
}
processData(0);
This would be the simplest thing to do from where you currently are.
Alternatively, if it fits what you want to do, Web Workers would be a great solution, since they actually shunt the work into a separate thread.
Having said this, what you're currently doing is extremely inefficient. You push values into an array, and consequently keep checking the ever longer array over and over for the values it contains. You should be using object keys for the purpose of de-duplication instead:
var allPoints = {};
// for (...) ...
if (!allPoints[address]) { // you can even omit this entirely
allPoints[address] = true;
}
// later:
allPoints = allPoints.keys();

First of all, avoid the multiple this._clusterData[i] calls. Extract it to a variable like so:
var allPoints = [];
var current;
for (var i = 0, cLength = this._clusterData.length; i < cLength; i+=1) {
current = this._clusterData[i];
if (allPoints.indexOf(current.attributes.PropertyAddress) == -1) {
allPoints.push(current.attributes.PropertyAddress);
this._DistClusterData.push(current)
}
}
This should boost your performance quite a bit :-)

As others already pointed out, you can do this asynchronously, so the browser remains responsive.
It should be noted however that the indexOf operation you do can become very costly. It would be better if you would create a Map keyed by the PropertyAddress value. That will take care of the duplicates.
(function (clusterData, batchSize, done) {
var myMap = new Map();
var i = 0;
(function nextBatch() {
for (data of clusterData.slice(i, i+batchSize)) {
myMap.set(data.attributes.PropertyAddress, data);
}
i += batchSize;
if (i < clusterData.length) {
setTimeout(nextBatch, 0);
} else {
done(myMap);
}
})();
})(this._clusterData, 1000, function (result) {
// All done
this._DistClusterData = result;
// continue here with other stuff you want to do with it.
}.bind(this));

Try considering adding to the array asynchronously with a list, for a set of 1000 records at a time, or for what provides the best performance. This should free up your application while a set of items is added to a list.
Here is some additional information: async and await while adding elements to List<T>

Generating set of random elements with further generation of 'one by one'

I am generating elements that have random data attributes like so :
generateCards : function(n)
{
var actions = ['press', 'blue-right', 'blue-left', 'red-right', 'red-left'],
i = n,
ran,
actions_cpy = actions.slice();
for (; i--;) {
ran = (Math.random() * actions_cpy.length)|0;
$('#container-game-mobile').prepend(
$('<div>', {
// remove and return a random string from the array
'class': 'game-card-mobile',
'data-action': actions_cpy.splice(ran, 1)[0]
})
);
// load the array backup with values when it is empty
if (actions_cpy.length === 0) {
actions_cpy = actions.slice();
}
}
}
Function works in a way where there are more or less equal amounts of each data atribute from actions array. I initially generate 10 elements so n = 10 due to the nature of application every time an action is performed on a .game-card-mobile I destroy it and need to generate new one so, call above function but now with n = 1 . Issue here is that, I somehow need to call elements that will still differ, so prevent 'blue-right' appearing over and over again.

It's a bit confusing, but I think I get the problem.
You need a function to get your actions, that function will return a random of this array, to this function, pass a param of your last actions randomised and if your random is the same as your last random, call the same function again, it will prevent your problem.