I extracted data from database in excel format using sql join query. I modified few columns data in to that exce sheet. I want update modified data into database using respective row ids.
When I iterate stack: Error: aborted is thrown.
below is my code
sql.connect(db, async (err)=> {
if (err)
console.log(err);
var request = new sql.Request();
var workbook = XLSX.readFile('2021-08-30-202129.xlsx');
var sheet_name_list = workbook.SheetNames;
var xlData = XLSX.utils.sheet_to_json(workbook.Sheets[sheet_name_list[1]]);
console.log(xlData);
xlData.forEach(async (item)=>{
let returnquery = await ` UPDATE [billing] SET notes='${item.notes}' WHERE id='${item.id}'`
request.query(returnquery, function (err, result) {
if (err) {
console.log(err)
}
else{
console.log('result', result);
}
sql.close();
});
})
});
I typically use the workbench app, and open a query window, to ensure I have the correct SQL syntax. (console.log your sql statement right before you call the DB, then copy it over to the workbench).
I usually have some small typo issue, and this allows me to easily figure that out, then go back to the code and fix the error.
I also think you have issues with having ' by your variables. those should be removed since you're already in `` style. SQL will misinterpret those.
Related
I've had a look around but can't find an obvious solution.
I have a collection with 130k documents. I need to export these as a CSV file. (The CSV part I have sorted I think).
My code works fine with smaller collection but when trying it on the 130k documents in a collection it hangs and I get "Request Aborted". What would be the best way to handle this?
My code:
db.collection("games")
.doc(req.params.docid)
.collection("players")
.onSnapshot(snapshot => {
console.log("On Snapshot")
snapshot.docs.forEach(data => {
const doc = data.data();
downloadArray.push(doc);
});
jsonexport(downloadArray, function(err, csv) {
if (err) return console.log(err);
fs.writeFile("out.csv", csv, function() {
res.sendFile(path.join(__dirname, "../out.csv"), err => {
console.log(err);
});
});
});
});
I'm trying out pagination as suggested, however I'm having trouble understanding how to keep calling the next batch until the loop is done, as sometimes I won't know the collection size, and querying such a large collection size takes over 1-2 minutes.
let first = db
.collection("games")
.doc(req.params.docid)
.collection("players")
.orderBy("uid")
.limit(500);
let paginate = first.get().then(snapshot => {
// ...
snapshot.docs.map(doc => {
console.log(doc.data());
});
// Get the last document
let last = snapshot.docs[snapshot.docs.length - 1];
// Construct a new query starting at this document.
let next = db
.collection("games")
.doc(req.params.docid)
.collection("players")
.orderBy("uid")
.startAfter(last.data())
.limit(500);
You could paginate your query with cursors to reduce the size of the result set to something more manageable, and keep paging forward until the collection is fully iterated.
Also, you will want to use get() instead of onSnapshot(), as an export process is probably not interested in receiving updates for any document in the set that might be added, changed, or deleted.
I have just learnt about SQL injection so I am making my queries a little more safe, everything has worked out so far except when I use update.
var sql = 'UPDATE Clients SET LastAccessDate= ? WHERE ClientID= ?';
var values = [
[timeStamp[1], ClientID[1]],//timeStamp=1473674800015, ClientID= 123
[timeStamp[2], ClientID[2]],
[timeStamp[3], ClientID[3]],
[timeStamp[4], ClientID[4]]
];
connection.query(sql, [values], function(err) {
if (err) {
throw err;
console.log("Error updating Last Access times");
}
else if(!err){
connection.end();
}
});
The connection to the database was done correctly, all column names are correct and arrays for the variables is correct, but I get an error that the syntax is wrong for update and I don't know where to look for the correct answer as my searches have been unsuccessful so far. What is the correct syntax for update and where can I look to find the syntax for other queries? The timestamp is a Bigint and the clientID is a unsigned int. Also ClientID is unique. Using nodejs and mysql database.
I have been going around in circles with this. I'm trying to execute an existing stored procedure using the node js documentdb library.
var sproc = self.client.queryStoredProcedures(collection._self, "select * from root r WHERE r.id = 'helloWorld'");
self.client.executeStoredProcedure(sproc._self, function (err, res) {
if(err){
console.log(err);
}else{
console.log(res);`
}
});
Not entirely sure queryStoredProcedures (Seems to be no async version of this) is the correct way of retrieving the uri for the store procedure, I haven't managed to get this to work. I'm also trying to avoid too many round trips to the database, but from what I gather I either hard code the store procedure's uri or have to make at least two requests just to execute the stored procedure.
queryStoredProcedures (along with all query and read functions) return a QueryIterator rather than an actual result. The methods you call on the returned QueryIterator are async. So, following the approach of your example (minus error handling), you would do this:
var queryIterator = self.client.queryStoredProcedures(collection._self, "select * from root r WHERE r.id = 'helloWorld'");
queryIterator.toArray(function(err, result) {
var sproc = result[0];
self.client.executeStoredProcedure(sproc._self, function (err, res) {
console.log(res);`
});
});
However, since the introduction of id-based routing, you can short hand the above like this:
var sprocLink = "dbs/myDatabase/colls/myCollection/sprocs/helloWorld";
self.client.executeStoredProcedure(sprocLink, function (err, res) {
console.log(res);`
});
I have used tedious to connect to sql server and restify for restful api
here is the server.js
server.get('/getInvoiceData', function (req, res, next) {
repository.GetInvoiceData(function(data){
res.send(data);
next();
});
});
and the invoice.js
exports.GetInvoiceData = function(callback){
var query = "SELECT * FROM [Snapdeal].[dbo].[tbl_Configuration]";
var req = new request(query,function(err,rowcount){
if (err)
{
console.log(err.toString());
}else{
console.log(rowcount+ " rows");
}
});
req.on('row',function(){
callback({customernumber:123});
});
connection.execSql(req);
}
I am getting the error as Cant set the headers after they are sent.
I am not 100% sure as I am not familiar with the SQL lib you are using, however, it looks to me like the problem is your row event would be raised per row, rather than per transaction.
You are ending the response after the first row event therefore if there is more than one row being returned the response will already have been closed (hence the error).
One way of dealing with this is to accumulate the row data as it's being retrieved and then raise the callback after your done
Now that you have stated the lib you are using (Tedius), it would appear my hunch was correct. Looking at the library, here is the simplest approach you can take to returning all the rows in a single callback
exports.GetInvoiceData = function(callback){
var query = "SELECT * FROM [Snapdeal].[dbo].[tbl_Configuration]";
var req = new request(query,function(err, rowcount, rows){
if (err) {
console.log(err.toString());
} else{
callback(rows);
}
});
connection.execSql(req);
}
Note - remember to set config.options.rowCollectionOnRequestCompletion to true otherwise the rows parameter will be empty.
My issue, using mssql, was that I had a default value or binding set (in this case, (getdate())) to one of my columns (modified date column). However, the data I was trying to retrieve had preset NULL values for this particular column.
I put data in those rows and I was good to go.
I am wanted to display every documents stored in my mongodb. I tried following code which simply get collection.find() and display through res.send()
router.get('/index', function(req,res){
var db = req.db
var collection = db.get('usercollection')
var display = util.inspect(collection.find()));
res.send(display);
});
I expected it to display the actual document stored in mongodb. But instead, it displayed this object format:
{cold:{manager:{driver:[Object], helper:[Object], collection:[Object].....
Is there any other steps needed to display raw mongodb document?
If the library you are using is the official 10gen library, then you can't simply output collection.find() without unwinding it. The easiest way to do that for smaller datasets is
collection.find().toArray(function(err, results) {
if (err) {
// do something error-y
} else {
res.send( results );
}
});
If you post more of your code, and tag your question with the libraries you are using, you'll be able to get more targeted help. If the library you are using returns a promise, this is probably how you'd unwind it:
collection.find().then(function(results){
res.send(results);
}).catch(function(err){
console.error(err);
});