Initially I had this code:
const newUserObject = { name: "username", uid: "FirebaseUID", posts: [] }
await Promise.all([
admin.firestore().collection("users").doc("FirebaseUID").set(newUserObject),
mongoDb.collection("users").insertOne(newUserObject)
])
For some reason, this doesn't work and returns the error "cannot encode value". Here is the complete stack trace:
Cannot encode value: 60bb87e2cb98774f01d8bb0b
at Serializer.encodeValue (~/node_modules/#google-cloud/firestore/build/src/serializer.js:168:15)
at Serializer.encodeFields (~/node_modules/#google-cloud/firestore/build/src/serializer.js:57:30)
at Function.fromObject (~/node_modules/#google-cloud/firestore/build/src/document.js:99:53)
at Object.op (~/node_modules/#google-cloud/firestore/build/src/write-batch.js:267:58)
at ~/node_modules/#google-cloud/firestore/build/src/write-batch.js:457:44
at Array.map (<anonymous>)
at WriteBatch._commit (~/node_modules/#google-cloud/firestore/build/src/write-batch.js:457:31)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Promise.all (index 0)
Caused by: Error
at WriteBatch.commit (~/node_modules/#google-cloud/firestore/build/src/write-batch.js:426:23)
at DocumentReference.set (~/node_modules/#google-cloud/firestore/build/src/reference.js:343:14)
at ~/dist/handlers/users.js:75:68
at Generator.next (<anonymous>)
at fulfilled (~/dist/handlers/users.js:5:58)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
The value looks similar to MongoDB ObjectID but I am sure I am not passing that in Firestore object in any way. I checked an answer to a similar question which said you cannot set empty arrays in a new document but that doesn't seem right to me.
If I rewrite my code as follows, it works like charm.
await admin.firestore().collection("users").doc("FirebaseUID").set(newUserObject)
await mongoDb.collection("users").insertOne(newUserObject)
To confirm, nothing is undefined in the object I am trying to set. The issue occurs only when I use it with MongoDB's insertOne in a Promise.all().
Steps to clone the issue:
npm init -y
npm i firebase-admin mongodb
Then copy this code:
const admin = require("firebase-admin")
const mongo = require("mongodb")
const client = mongo.MongoClient("mongodb://localhost:27017/dbname", {
useNewUrlParser: true,
useUnifiedTopology: true,
auth: {
user: "mongodbusername",
password: "mongodbpassword"
}
})
client.connect().then(async () => {
console.log("MongoClient Connected");
admin.initializeApp({
credential: admin.credential.cert("serviceAccountKey.json"),
databaseURL: "https://<project-id>.firebaseio.com"
})
const newUserObject = {name: "NewUser", uid: "userUID", posts: ["PostID1"]}
// UPDATING INDIVIDUALLY
// await admin.firestore().collection("users").doc(newUserObject.uid).set(newUserObject)
// await client.db("lasergg").collection("users").insertOne(newUserObject)
// Works perfectly
// UPDATING IN A PROMISE.ALL()
await Promise.all([
admin.firestore().collection("users").doc(newUserObject.uid).set(newUserObject),
client.db("local").collection("users").insertOne(newUserObject)
])
// Throws error Error: Cannot encode value: 60bdee0a77659f27d493dec8
console.log("All Documents Added!");
})
I modified the source code of Firestore Node SDK here and logged the 'val' parameter [console.log(val, typeof val)] that encodeValue function takes and the output was:
60be04dd53e7965bf4a23173 object
Edit: I logged values being passed in encodeValue() when running promises individually and the output is:
I ran the same when running in Promise.all():
This is what I edited in 'serialier.js':
Fix
You should be able to fix this problem by passing a copy of newUserObject into MongoDB, like this:
await Promise.all([
admin.firestore().collection("users").doc("FirebaseUID").set(newUserObject),
mongoDb.collection("users").insertOne({ ...newUserObject })
])
Note this: insertOne({ ...newUserObject }).
Explanation
When you try to insert a document into MongoDB without an _id field, MongoDB creates one automatically by generating an ObjectId. The operation basically looks like this: newUserObject._id = new ObjectId().
It seems that when you run firestore.set() and mongo.insertOne() in parallel, this happens before the serializer step in Firestore.
So whenever the serializer starts encoding, it sees this new _id field and tries to encode it, and that's where it fails. Note that it sees the _id field because you are passing the same object reference (newUserObject) to both firestore.set() and mongo.insertOne().
This explains why executing the operations sequentially works. Here you first insert the document into Firestore and only then into MongoDB, so the _id field is no longer a problem:
await admin.firestore().collection("users").doc("FirebaseUID").set(newUserObject)
await mongoDb.collection("users").insertOne(newUserObject)
However, if you were to reverse the order here and inserted into MongoDB first, you'd encounter the same problem.
Related
I'm saving a document into a mongodb collection and upon success im returning the document.
ie:
ProductHistory.findOneAndUpdate(
{productId: product._id},
$push: {'history': historyData },
{upsert: true, new: true})
.exec((err, success) => {
if (err){
console.log('Error: ', err)
}
console.log('success is: ', success)
}
The result is successfully saved entry and a console logged object which looks like the following:
{
_id: 616eb07b1e7edf6e9b035707,
history: [
{
_id: 61b7cdb160854a0564c15f25,
value: 114.5,
},
//...repeated objects x times
]
}
The problem is I want to access history so I can perform a forEach on it but I'm having trouble selecting it.
If i do
console.log('success is:' , success.history) it returns undefined.
Likewise if i do success.history.map(). I've also tried destructing is using:
const {history} = success
I just can't seem to grab that array for some reason! any ideas please?
Thanks
Without viewing your Mongoose schema, I can't say for certain. However, more than likely you haven't added the history array type to your schema.
Solutions
Add this array to your schema (preferred method).
You should be able to bypass the schema by accessing success._doc object. This is not recommended and should only be used for testing and whatnot during development.
It was working fine earlier, but now it's just being weird.
module.exports = {
commands: 'test',
callback: (message, args, client) => {
const guild = client.guilds.cache.get('ID') // 'ID' is a replacement for the actual guild ID I'm using.
console.log(guild) // client returns undefined
}
}
(dumbed down to exclude code that is not relevant.)
It was working before, I changed nothing regarding fetching the guild. It's been the same for weeks now, somehow it just doesn't work.*
UPDATE: Found the issue, originally, callback() accepted three params, I edited it to include text (Everything after the command, e.g. !test test would only pick up test and not !test)
So, in reality, client has been arguments.join() all along.
I figured this out after logging client, something I should have done before I posted it here.
I would suggest changing your code to this.
module.exports = {
name: "test",
description: "test",
execute(message, args, client){
const guild = client.guilds.cache.get('ID') // 'ID' is a replacement for the actual guild ID I'm using.
console.log(guild) // client returns undefined
}
}
I am having troubles running queries from Cloud Functions using the request parameters to build the query form HTTP calls. In the past, I have ran queries from cloud functions fine with no error. My problem arises when I try to run the query using parameters gotten from the request.
When I hardcode the location of the document in the function, it works fine but when I try to build a query, it returns status code of 200. I have also logged the the built query and it is logging out the right thing but no data is being returned. It only returns data when the document path is hardcoded. See code below.
Query looks like this
https://us-central1-<project-id>.cloudfunctions.net/getData/CollectionName/DocumentName
export const getData = functions.https.onRequest((request, response) => {
const params = request.url.split("/");
console.log("the params 0 "+params[0]);
console.log("the params 1 "+params[1]);
console.log("the params 2 "+params[2]);
//Build up the document path
const theQuery = "\'"+params[1]+"\/"+params[2]+"\'";
console.log("the query "+theQuery); <-- logs out right result in the form 'Collection/Document'
//Fetch the document
const promise = admin.firestore().doc("\'"+params[1]+"\/"+params[2]+"\'").get() <---- This doesnt work, building the query
//const promise = admin.firestore().doc('collectionName/DocID').get() <---- This hard coded and it works
promise.then(snapshot => {
const data = snapshot.data()
response.send(data)
}).catch(error => {
console.log(error)
response.status(500).send(error);
})
});
I tried using a different approach and giving the datafields a names as seen below
Query looks like this
https://us-central1-<project-id>.cloudfunctions.net/getData?CollectionName=CName&DocumentID=Dname
export const getData = functions.https.onRequest((request, response) => {
const collectName = request.query.CollectionName;
const DocId = request.query.DocumentName;
//Build up the document path
const theQuery = "'"+collectName+"\/"+collectName+"'";
console.log("the query "+theQuery); <---Logs out correct result
//Fetch the document
const promise = admin.firestore().doc(theQuery).get() <-- Building the query does not work
//const promise = admin.firestore().doc('collectionName/DocID').get() <---- This hard coded and it works
promise.then(snapshot => {
const data = snapshot.data()
response.send(data)
}).catch(error => {
console.log(error)
response.status(500).send(error);
})
});
In both cases, when the request is build from the URL, it does not return any data and it does not return any errors. And I am sure the documents I am trying to fetch exsist in the database. Am I missing anything ?
Try request.path. Then you can obtain the path components, e.g. request.path.split("/")[1]
The syntax for request.query is valid when using Express. This is referenced in some of the docs, but not made explicit that Express is required. It's confusing.
To properly handle the dynamic inputs, you may have more luck working with Express and creating routes and handlers. This Firebase page has links to some projects using it.
Walkthough set-up using Express on Firebase.
Attempting to build a basic API to interact with a MSSQL v12 database using Nodejs. I have been able to connect to the database using the mssql/msnodesqlv8 package but parameterized queries are failing with the following.
code: 'EREQUEST',
number: 102,
state: undefined,
originalError:
{ Error: [Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near ''. sqlstate: '42000', code: 102 },
name: 'RequestError' }
Debug: internal, implementation, error
I used SQL Server Profiler and saw that the query was coming in as such
exec sp_executesql N'declare #SecurityKey nvarchar (MAX);set #SecurityKey=#P1;exec database.getSecurityBySecurityId #SecurityKey;',N'#P1 nvarchar(20)',N'XXXXXXXX'
and failing. After some investigation it seems to be an issue with the semicolons after the declare and set statements as it is not allowed in TSQL (very new to MSSql, will need to read up). Removing the semicolons did indeed fix the issue when I ran the query manually.
So my question is this.. is there a way to get msnodesqlv8 to work with my version on |Mssql and if yes, how so? Is there a way to omit these semicolons.
If you think there is a better way, i would like to hear it as I am new to Nodejs + MSSql.
Contents of getSecurity.sql
exec database.getSecurityBySecurityId #SecurityKey
contents of index.js
"use strict";
const utils = require("../utils");
const api = async ({ sql, getConnection }) => {
const sqlQueries = await utils.loadSqlQueries("events");
const getSecurity = async SecurityKey => {
const cnx = await getConnection();
const request = await cnx.request();
request.input('SecurityKey', SecurityKey);
return request.query(sqlQueries.getSecurity);
};
return {
getSecurity
};
};
module.exports = { api };
I was able to work around this by editing the library.
In ./lib/msnodesqlv8.js you can find where is concatenates the query string
...
}
if (input.length) command = `declare ${input.join(',')} ${sets.join(';')};${command};`
if (output.length) {
command += `select ${output.join(',')};`
handleOutput = true
}
....
Editing this will allow you to control the flow.
How can I get ids of affected records inside database when executing mongoose update command with multi:true option?
MyModel.update({ age: { $gt: 18 } }, { oldEnough: true }, { multi: true }, function (err, raw) {
if (err) return handleError(err);
console.log('The raw response from Mongo was ', raw);
});
This will return number of affected documents, but how can I know their ids without running another query inside database?
The update operation with or without the multi: true option returns a WriteResult which does not have a property containing such details. Only various counts.
When it comes to mongoose as per their documentation you only get and err and rawResponse in their callback on success:
The callback function receives (err, rawResponse).
err is the error if any occurred rawResponse is the full response from
Mongo
So you basically get the WriteResult from mongo nothing more nothing less.
So no you can not get a convenient list of the updated _id although you could probably do something with the post update middleware where you can run the filter part of the update and store the _ids somewhere I guess for audit etc.