I'm uploading some Excel files to the server to do something with the data. I'm using xlsx to read and Tabulator to present the uploaded data in the frontend.
This is the loop in which i read the data and push it into an array:
for (const k of raw) {
let columns: Column[] = []
//new Array each loop
let tmpData: any[] = []
//read data from Excel files
const wb = read(await k.file.arrayBuffer(), {
sheets: k.type.sheet
})
tmpData = utils.sheet_to_json(wb.Sheets[k.type.sheet], {
range: k.type.row
})
//do something with the data of the files
tmpData.forEach((item, index) => {
if (Object.entries(item).length < 2) {
data.splice(index, 1);
}
});
tmpData.forEach((item) => {
Object.keys(item).forEach((key) => {
if (key.includes('EMPTY')) {
delete item[key]
}
})
})
//seperate the first row to extract columns
Object.keys(tmpData[0]).forEach((key) => {
columns.push(new Column(key, key, 'input', true, 300))
})
//create columns Array containing the headers of all files
columnsObject.push(new ColumnsType(columns, k.type.name))
//push the data into an Array
data.push(new TablesType(tmpData, k.type.name))
//troubleshooting
data.forEach(item => {
console.log(item.vendor)
})
console.log('\n ----- \n')
}
This is an example of the output of the troubleshooting where the problem accurs:
Ciena
-----
Ciena
PaloAlto
-----
PaloAlto
Infinera
-----
PaloAlto
Infinera
Arista
-----
In the third loop the Ciena object is missing. I tried to do it with a traditional for loop already, but the same issue accured with some constellation of uploaded files.
The data is reading just fine for all files, just the array is throwing it away.
//do something with the data of the files
tmpData.forEach((item, index) => {
if (Object.entries(item).length < 2) {
data.splice(index, 1);
}
});
Why even write the comment if it is as meaningless as this? What is this supposed to do? Something?!
This is btw. also the place where your mistake most likely lies. You're splicing your data array. (Meaning deleting stuff from your data array) This would explain how your data array loses some of it's elements. As I don't know what this part of your code is actually supposed to to, I can't tell you how to fix it, just that this is most likely the origin of your troubles.
Related
I am trying to create an add to cart button which fetches the data from product database using the id of specific product which I selected. I am trying to push the object found using the same Id into a normal javascript array and then to display it using ejs methods. While I was tring I found I am unable to push the data in object form.
Summary:
On 7th line I have declared an array and in that array I want to store some objects which I have fetched frome a db model.
On 15th line I am trying to push the object form into my array so that I could iterate through the objects to display them on my page using ejs. But I am unable to do that.
screenshots:
Here's the final result I'm getting even after trying to push objects in array:
empty array logged
Here are the objects I'm trying to push:
Objects
Code:
app.get("/cart", (req, res) => {
if (req.isAuthenticated()) {
const findcartdata = req.user.username;
userData.findOne({email: findcartdata}, (err, BookId) => {
// console.log(BookId.cartItemId);
const idArray = BookId.cartItemId;
var bookArray = [];
idArray.forEach((data) => {
productData.findOne({_id: data}, (err, foundBookData) =>{
// console.log(foundBookData);
if(err){
console.log(err);
}
else{
bookArray.push(foundBookData);
}
})
});
console.log(bookArray);
// res.render("cart", {
// cartBookArray: BookId.cartItemId
// })
});
} else {
res.redirect("/login");
}
})
In above code i found the user's email using passport authentication user method and using that email I wanted to add the products in a different javascript array (which I am goint to pass to my ejs file of cart and then iterate it on list) using those array of Id which I got from another model called userData. The problem is I am able to find userData of each Id but unable to store them as an array of objects.
Looks like a timing issue, your code completes before the database downloads the objects and pushes them to your array.
This should fix your issue:
// ...
const idArray = BookId.cartItemId;
var bookArray = [];
for (const data of idArray) {
const foundBookData = await productData.findOne({_id: data}).catch(console.error);
if (!foundBookData) continue;
bookArray.push(foundBookData);
}
console.log(bookArray);
// ...
By the way, make sure to make the whole function asynchronous as well, which would be done by changing this line:
userData.findOne({email: findcartdata}, async (err, BookId) => { // ...
There are two things I want to do:
I want to create a new array of objects from an existing object,
And increment the object so each object can have a count id of 1,2,3 etc
My issue is that when I write to the file it writes only 1 random object to the file and the rest don't show. There are so errors and all the objects have the same increment value. Please explain what I am doing wrong. Thanks.
Code:
data.json:
{
"users":[
{
"name":"mike",
"category":[
{
"title":"cook",
}
],
"store":{
"location":"uptown",
"city":"ulis"
},
"account":{
"type":"regular",
"payment":[
"active":false
]
}
}
]
}
index.js:
const appData = ('./data.json')
const fs = require('fs');
let newObject = {}
appData.forEach(function(items){
let x = items
let numincrement = 1++
newObject.name = x.name
newObject.count = numincrement
newObject.categories = x.categories
newObject.store = x.store
newObject.account = x.account
fs.writeFile('./temp.json', JSON.stringify(newObject, null, 2),'utf8' , function(err, data) {
// console.log(data)
if(err) {
console.log(err)
return
} else{
console.log('created')
}
})
})
There are a whole bunch of problems here:
You're just rewriting the same object over and over to the file. fs.writeFile() rewrites the entire file. It does not append to the file. In addition, you cannot append to the JSON format either. So, this code will only every write one object to the file.
To append new JSON data to what's in the existing file, you would have to read in the existing JSON, parse it to convert it to a Javascript array, then add new items onto the array, then convert back to JSON and write out the file again. For more efficient appending, you would need a different data format (perhaps comma delimited lines).
Your loop has all sorts of problems. You're assigning to the same newObject over and over again.
Your numincrement is inside the loop so it will have the same value on every invocation of the loop. You can also just use the index parameter passed to the forEach() callback instead of using your own variable.
If what you're trying to iterate over is the users array in your data, then you may need to be iterating over appData.users, not just appData.
If you really just want to append data to a text file, the JSON is not the easiest format to use. It might be easier to just use comma delimited lines. Then, you can just append new lines to the file. Can't really do that with JSON.
If you're willing to just overwrite the file with the current data, you can do this:
const appData = ('./data.json').users;
const fs = require('fs');
// create an array of custom objects
let newData = appData.map((item, index) => {
return {
name: item.name,
count: index + 1,
categories: item.categoies,
store: item.store,
account: item.account
};
});
// write out that data to a file as JSON (overwriting existing file)
fs.writeFile('./temp.json', JSON.stringify(newData, null, 2),'utf8' , function(err, data) {
if (err) {
console.log(err);
} else {
console.log("data written");
}
});
I am writing a script which at its core parses a .csv file for certain columns storing them in an array and then writes the contents to another .csv file. I am able to parse the file using fast-csv and have confirmed in the terminal that my array is in the correct format. However, when I attempt to write this array using the fast-csv to a .csv file, the contents never appear in the file and no errors are thrown. I have validated that the array is being passed all the way through to the callback. In addition I have gone so far as to replace that variable in the writeToPath function with a simple array and still no luck. Any assistance would be appreciated.
const processFile = (fileName, file, cb) => {
let writeData = []
let tempArray = []
csv.fromPath(basePath + file, {ignoreEmpty: false, headers: false})
.on("data", function(data){
if (data[0] != ''){
[startDate, endDate] = fileName
tempArray[0] = data[0]
tempArray[1] = data[1]
tempArray[2] = data[2]
tempArray[3] = data[3]
tempArray[4] = data[4]
tempArray[5] = data[8]
tempArray[6] = ""
tempArray[7] = ""
tempArray[8] = ""
tempArray[9] = startDate
tempArray[10] = endDate
writeData[i] = tempArray
writeData.shift()
tempArray = []
i++
}
})
.on("end", () => {
console.log('end')
}).on('finish', (() => {
cb(writeData)
}));
}
processFile(fileName, file, (csvData) => {
console.log(csvData)
csv.writeToPath('./working-files/top.csv', {headers: false}, csvData).on("finish", () => {
console.log('done')
})
Unfortunately, without any context to the dataset you are using, there is only so much I can suggest. The variables needed to debug this properly would be: the file, the file names used and whatever 'i' is. If you can update this then I'll be happy to take another look.
I would suggest going back and logging the variables after each step that would modify them, hopefully then you'll get a better picture as what is going wrong.
I understand this isn't a complete answer and it will probably get removed but I don't have the 50 needed reputation to make a comment.
I'm trying to pass a property, that is inside the first position of an array of objects, to another module so I can use this value later. I've tried to pass it as module(args), but it keeps reading the default value which is 0. Is there a way to do this?
I tried to implement some React.context but the Bot framework Emulator is refusing it.
/////////////////Module that ll acquire the value/////////////////////////////
getCard(bot, builder, params) {
let configValues = { ...params[0] }
bot.dialog(`${configValues.path}`, function (session) {
var msg = new builder.Message(session);
const cardItem = (obj) => {
return (new builder.HeroCard(session)
.title(`${obj.title}`)
.text(`R$ ${obj.price}`)
.images([builder.CardImage.create(session, `${obj.img}`)])
.buttons([
builder.CardAction.imBack(session, `${obj.price} Item adicionado!`, 'add to cart')
// !onClick event must add the current obj.price to
// the configValues.total(Ex: configValues.total += obj.price)!
])
)
}
msg.attachmentLayout(builder.AttachmentLayout.carousel)
msg.attachments(
eval(params.map(obj => cardItem(obj)))
);
//!in here before end the dialog is where i want to update
// the configValues.total so i can show it in the -> Checkout module
session.send(msg).endDialog()
}).triggerAction({ matches: configValues.regex });
}
}
//////////////CheckOut.Module///////////////////////////////
{...}
let configValues = { ...params[0] }
let state = {
nome: "",
endereco: "",
pagamento: "",
total: configValues.total // this is the value to be read
}
bot.dialog('/intent', [
{...},
(session, results) => {
state.pagamento = results.response
session.send(
JSON.stringify(state) // here is the place to be printed
)
{...}
]
).triggerAction({ matches: /^(finalizar|checar|encerrar|confirmar pedido|terminar)/i })
Since you solved your original problem, I'll answer the one in your comment.
Your problem is here:
cartId.map((obj, i , arr) => {
// if (!obj.total) {
// obj.total.reduce(i => i += i)
// }
const newtotal = new total
newtotal.getTotals(bot, builder, obj, arr)
})
cartId contains the totals for each of your items. When you call map on it, you're passing each item individually to getTotals, which passes each item to checkout()
The reason you can't sum all of the totals and can only sum one item's total is that you pass cartId to checkout and cartId has been changed to just a single item. Instead, there's a couple of different things you could do:
Pass the whole cartId from cartItems and use something like for (var key in cartItems) in totalConstructor() and checkoutConstructor(). This is probably the easiest, but not very memory efficient.
Use BotBuilder's State Storage to store your totals array in userData, then sum that at the end. This might be more difficult to implement, but would be a much better route to go. Here's a sample that can help you get started.
I've used the 'fast-csv' module to parse the csv file for other manipulations, but that returns data row-wise. I want to read the first 2 columns of a csv file. Can someone please help?
I see two options.
One is do specify which headers you want in fast-csv and discard the rest. This approach will return an object which may suit your needs or you can then turn that into an array afterwards.
const csv = require('fast-csv')
const CSV_STRING = 'a,b,c\n' +
'a1,b1,c1\n' +
'a2,b2,c2\n'
let filtered = []
csv
.fromString(CSV_STRING, { headers: ['column_1', 'column_2'], renameHeaders: true, discardUnmappedColumns: true }) // I give arbitrary names to the first two columns - use whatever make sense
// .fromString(CSV_STRING, { headers: ['column_1', undefined, 'column_3'], discardUnmappedColumns: true }) // I could use undefined if I wanted to say skip column 2 and just want 1 and 3
.on('data', function (data) {
// console.log([data.column_1, data.column_2])
filtered.push([data.column_1, data.column_2]) // or you can push to an array
})
.on('end', function () {
console.log('done')
console.log(filtered)
})
The other is to return as an array (default) and filter what you need using the transform method
const csv = require('fast-csv')
const CSV_STRING = 'a,b,c\n' +
'a1,b1,c1\n' +
'a2,b2,c2\n'
csv
.fromString(CSV_STRING)
.transform(function (data) {
return [data[0], data[1]]
})
.on('data', function (data) {
console.log(data)
})
.on('end', function () {
console.log('done')
})