Develop Reference

Override Mongoose save method to retry on `duplicate key error`

My Mongoose schema uses a custom _id value and the code I inherited does something like this
const sampleSchema = new mongoose.Schema({
_id: String,
key: String,
});
sampleSchema.statics.generateId = async function() {
let id;
do {
id = randomStringGenerator.generate({length: 8, charset: 'hex', capitalization: 'uppercase'});
} while (await this.exists({_id: id}));
return id;
};
let SampleModel = mongoose.model('Sample', sampleSchema);
A simple usage looks like this:
let mySample = new SampleModel({_id: await SampleModel.generateId(), key: 'a' });
await mySample.save();
There are at least three problems with this:
Every save will require at least two trips to the database, one to test for a unique id and one to save the document.
For this to work, it is necessary to manually call generateId() before each save. An ideal solution would handle that for me, like Mongoose does with ids of type ObjectId.
Most significantly, there is a potential race condition that will result in duplicate key error. Consider two clients running this code. Both coincidentally generate the same id at the same time, both look in the database and find the id absent, both try to write the record to the database. The second will fail.
An ideal solution would, on save, generate an id, save it to the database and on duplicate key error, generate a new id and retry. Do this in a loop until the document is stored successfully.
The trouble is, I don't know how to get Mongoose to let me do this.
Here's what I tried: Based on this SO Question, I found a rather old sample (using a very old mongoose version) of overriding the save function to accomplish something similar and based this attempt off it.
// First, change generateId() to force a collision
let ids = ['a', 'a', 'a', 'b'];
let index = 0;
let generateId = function() {
return ids[index++];
};
// Configure middleware to generate the id before a save
sampleSchema.pre('validate', function(next) {
if (this.isNew)
this._id = generateId();
next();
});
// Now override the save function
SampleModel.prototype.save_original = SampleModel.prototype.save;
SampleModel.prototype.save = function(options, callback) {
let self = this;
let retryOnDuplicate = function(err, savedDoc) {
if (err) {
if (err.code === 11000 && err.name === 'MongoError') {
self.save(options, retryOnDuplicate);
return;
}
}
if (callback) {
callback(err, savedDoc);
}
};
return self.save_original(options, retryOnDuplicate);
}
This gets me close but I'm leaking a promise and I'm not sure where.
let sampleA = new SampleModel({key: 'a'});
let sampleADoc = await sampleA.save();
console.log('sampleADoc', sampleADoc); // prints undefined, but should print the document
let sampleB = new SampleModel({key: 'b'});
let sampleBDoc = await sampleB.save();
console.log('sampleBDoc', sampleBDoc); // prints undefined, but should print the document
let all = await SampleModel.find();
console.log('all', all); // prints `[]`, but should be an array of two documents
Output
sampleADoc undefined
sampleBDoc undefined
all []
The documents eventually get written to the database, but not before the console.log calls are made.
Where am I leaking a promise? Is there an easier way to do this that addresses the three problems I outlined?
Edit 1:
Mongoose version: 5.11.15

I fixed the problem by changing the save override. The full solution looks like this:
const sampleSchema = new mongoose.Schema({
_id: String,
color: String,
});
let generateId = function() {
return randomStringGenerator.generate({length: 8, charset: 'hex', capitalization: 'uppercase'});
};
sampleSchema.pre('validate', function() {
if (this.isNew)
this._id = generateId();
});
let SampleModel = mongoose.model('Sample', sampleSchema);
SampleModel.prototype.save_original = SampleModel.prototype.save;
SampleModel.prototype.save = function(options, callback) {
let self = this;
let isDupKeyError = (error, field) => {
// Determine whether the error is a duplicate key error on the given field
return error?.code === 11000 && error?.name === 'MongoError' && error?.keyValue[field];
}
let saveWithRetries = (options, callback) => {
// save() returns undefined if used with callback or a Promise otherwise.
// https://mongoosejs.com/docs/api/document.html#document_Document-save
let promise = self.save_original(options, callback);
if (promise) {
return promise.catch((error) => {
if (isDupKeyError(error, '_id')) {
return saveWithRetries(options, callback);
}
throw error;
});
}
};
let retryCallback;
if (callback) {
retryCallback = (error, saved, rows) => {
if (isDupKeyError(error, '_id')) {
saveWithRetries(options, retryCallback);
} else {
callback(error, saved, rows);
}
}
}
return saveWithRetries(options, retryCallback);
}
This will generate an _id repeatedly until a successful save is called and addresses the three problems outlined in the original question:
The minimum trips to the database has been reduced from two to one. Of course, if there are collisions, more trips will occur but that's the exceptional case.
This implementation takes care of generating the id itself with no manual step to take before saving. This reduces complexity and removes the required knowledge of prerequisites for saving that are present in the original method.
The race condition has been addressed. It won't matter if two clients attempt to use the same key. One will succeed and the other will generate a new key and save again.
To improve this:
There ought to be a maximum number of save attempts for a single document followed by failure. In this case, you've perhaps used up all the available keys in whatever domain you're using.
The unique field may not be named _id or you might have multiple fields that require a unique generated value. The embedded helper function isDupKeyError() could be updated to look for multiple keys. Then on error you could add logic to regenerate just the failed key.

How To Store A Element Inside A Map Using Dynamic Variable In Firebase

I Wanted To Know How We Can Store Elements(Map) Inside A Map With Dynamic Element Names.
I Tried The Following
var activequestionid = "12201"
var aqid = "answers."+activequestionid
await updateDoc((doc(db, "tests", testid), {
[`${aqid}`]: {ans:ans,type:type,time:serverTimestamp()}
}))
var activequestionid = "12201"
var aqid = "answers."+activequestionid
await updateDoc((doc(db, "tests", testid), {
[aqid]: {ans:ans,type:type,time:serverTimestamp()}
}))
Also My Map Is As Follows:
testid(document)
answers(map)
12201(map)
But Both Give The Same Error: Expected type 'Pc', but it was: a custom Object object
Any Help On This Issue Is Appreciated!

The updateDoc() function takes 2 parameters - a DocumentReference and the data. You have () around both of them so essentially that's 1 parameter. Try removing them:
var activequestionid = "12201"
var aqid = "answers."+activequestionid
await updateDoc(
doc(db, "tests", testid), {
[aqid]: {
ans: ans,
type: type,
time: serverTimestamp()
}
})

How do I change an object's value in an IndexedDB index?

Is it possible to update an object's value within an IndexedDB index without cloning, deleting, or putting a new entry? Theoretically something like the following snippet would do the trick, though it probably would not delete until the put was confirmed. But it looks like overkill to me. It looks like it would be a nightmare to do any error handling on.
const objectStore = db.transaction([objectStoreName], 'readwrite')
.objectStore(objectStoreName);
const requestGet = objectStore.get(index);
requestGet.onsuccess = (event: any) => {
const value = event.target.result.value // Store old value
const requestDelete = objectStore.delete(index);
requestDelete.onsuccess = (event: any) => {
const requestPut = objectStore
.put({index: 'New Index Value', value: value}); // Put back using new index
};
};

You cannot directly change values in an object store's index. You can change the values of an object in an object store, and IndexedDB will propagate your changes to related indices. Indices are essentially read-only.

It is possible since you specify your index, otherwise, an other logic may be necessary.
As you should know, the IDBObjectStore has a method .put() which it will receive two params. With it you can either PUT a new value or UPDATE a value.
IDBObjectStore.put(item, key)
item: The item you want to put/update
key: opcional: Your primary object store key (such as an uuid, a random number, in short...) for that item you would like to update.
Code:
//This is an example only.
//Let's think that we have an object store into our IndexDB 'user', where object store is called by user-data:
//# Key Value
//0 1 { username: 'John Doe' }
//Here, we are receiving the 'success' result from an indexedDB.open(), and using its result with a promise.
dbPromise.then(db => {
//Getting the transaction
const transaction = db.transaction('user-data', 'readwrite')
//Getting the objectStore with the data, the same object store before.
const store = transaction.objectStore('user-data')
//Getting the key's object store, in the other other words, this is the key you define when you create you objectStore, with createObjectStore. In this example, I've used 'autoIncrement: true'
const query = store.get(1)
//Getting the query result with a success listener.
query.addEventListener('success', event => {
const { ['result']: user } = event.target
user.productsIntoCart.push(newItem)
//With this, we will be able to change the object store value.
user.username = 'Jane Doe'
store.put(user, 1)
})
query.addEventListener('error', event => console.error(event))
transaction.addEventListener('complete', () => db.close())
})
//# Key Value
//0 1 { username: 'Jane Doe' }
You can see more details you want in the MDN IDBObjectStore.put documentation.
IDBObjectStore

In Firebase when using push() How do I get the unique ID and store in my database [duplicate]

This question already has answers here:
When utilizing the .push method can I write a copy of the id to the object?
(2 answers)
Closed 7 months ago.
I am pushing data in firebase, but i want to store unique id in my database also .
can somebody tell me,how to push the data with unique id.
i am trying like this
writeUserData() {
var key= ref.push().key();
var newData={
id: key,
websiteName: this.webname.value,
username: this.username.value,
password : this.password.value,
websiteLink : this.weblink.value
}
firebase.database().ref().push(newData);
}
error is "ReferenceError: ref is not defined"

You can get the key by using the function key() of any ref object
There are two ways to invoke push in Firebase's JavaScript SDK.
using push(newObject). This will generate a new push id and write the data at the location with that id.
using push(). This will generate a new push id and return a reference to the location with that id. This is a pure client-side
operation.
Knowing #2, you can easily get a new push id client-side with:
var newKey = ref.push().key();
You can then use this key in your multi-location update.
https://stackoverflow.com/a/36774761/2305342
If you invoke the Firebase push() method without arguments it is a
pure client-side operation.
var newRef = ref.push(); // this does *not* call the server
You can then add the key() of the new ref to your item:
var newItem = {
name: 'anauleau'
id: newRef.key()
};
And write the item to the new location:
newRef.set(newItem);
https://stackoverflow.com/a/34437786/2305342
in your case :
writeUserData() {
var myRef = firebase.database().ref().push();
var key = myRef.key();
var newData={
id: key,
Website_Name: this.web_name.value,
Username: this.username.value,
Password : this.password.value,
website_link : this.web_link.value
}
myRef.push(newData);
}

Firebase v3 Saving Data
function writeNewPost(uid, username, picture, title, body) {
// A post entry.
var postData = {
author: username,
uid: uid,
body: body,
title: title,
starCount: 0,
authorPic: picture
};
// Get a key for a new Post.
var newPostKey = firebase.database().ref().child('posts').push().key;
// Write the new post's data simultaneously in the posts list and the user's post list.
var updates = {};
updates['/posts/' + newPostKey] = postData;
updates['/user-posts/' + uid + '/' + newPostKey] = postData;
return firebase.database().ref().update(updates);
}

You can get last inserted item id using Promise like this
let postRef = firebase.database().ref('/post');
postRef.push({ 'name': 'Test Value' })
.then(res => {
console.log(res.getKey()) // this will return you ID
})
.catch(error => console.log(error));

Try this . It worked for me
this.addressRef.push(addressObj).then(res => {
console.log("address key = " + res.key) ;
});
Here res.getKey() will not work but use res.key to get the latest push ID

It looks like now .push() always returns an Observable. When applied the solutions above I had the following error: "Type 'String' has no compatible call signatures".
That being said, I refer what did work in my case for this new version:
Type 'String' has no compatible call signatures

This worked for me:
var insertData = firebase.database().ref().push(newData);
var insertedKey = insertData.getKey(); // last inserted key
See Here: Saving data with firebase.

FireBase - angularfire setting object key as variable

I am using firebase(angularfire) in my angularjs app to store and process my message system but can't seem to figure out how to replicate the example data from the firebase docs
// room members are easily accessible (or restricted)
// we also store these by room ID
"members": {
// we'll talk about indices like this below
"one": {
"mchen": true,
"hmadi": true
}
}
Here the members.one contains the user name as a key and I am trying to do this for my data as well but can't seem to figure out a solution.
The members portion of my firebase data is like so:
members { one: { } }
I have two variables set in the $scope.
user_name = kep; //person chatting with name
sender_name = pek; //current user name
So I want to use the set function to insert data into members.one or in this case members.user_name + ':' + sender_name but where I am having trouble is how to actually insert the data without creating a parent object.
ref.child('members').child(user_name + ':' + sender_name).set({
user_name: true, sender_name: true
});
The problem arises when I try to pass user_name and sender_name into the set() function below is the result it gets.
members { "kep:pek": { user_name: true, sender_name: true }}
where as I want it to be:
members { "kep:pek": { kep: true, pek: true }}
If I put user_name and sender_name into an object and then run the set() function with the object passed it will create the following structure which is not what I am looking for:
members { "kep:pek": { newObject: { kep: true, pek: true }}}

Firebase team member here.
The Firebase Database is a just a JSON document.
So let's say you want to structure your data this way:
{
"members" : {
"kep:pek" : {
"kep" : true,
"pek" : true
}
}
}
A custom key is created by using the .child() method, or by creating a key in the JavaScript Object.
JSBin Demo
var rootRef = new Firebase('<my-firebase-app>');
var membersRef = rootRef.child('members');
var user_name = 'kep';
var sender_name = 'pek';
// child object to store custom keys
var objectToSave = {};
// set keys in [] syntax
objectToSave[user_name] = true;
objectToSave[sender_name] = true;
// use .child() with a formatted string to save the object
membersRef.child(user_name + ':' + sender_name).set(objectToSave);