I have some xlsx files, and besides parsing this file in a "classic" way, I also need to do it in a "smart" way.
var filePath = process.env.PWD + '/testdata/unfried-xlsx-1.xlsx';
var inboundWorkbook = new ExcelJS.Workbook();
inboundWorkbook.xlsx.readFile(filePath).then(function() {
var inboundWorksheet = inboundWorkbook.getWorksheet(1); //or name of the worksheet
inboundWorksheet.eachRow({ includeEmpty: true }, function(row, rowNumber) {
console.log(row.values);
});
});
example of xlsx file
My file is made of a header, a table, and a footer.
Problem is, I need to "isolate" informations. For example I need a customer number located either in the header or in the footer, and of course the datas in the table.
I could be easy if the file was constant, but the feature includes a variety of files that I dont know in advanced (different headers and footers, different columns names ect...)
I know I wont have the most precise feature I can imagine, but if I can get close to it, it could be great :)
Any idea how to proceed ?
Regards,
Related
Please excuse the rookie question as I'm not a programmer :)
We're using Pentaho 8
I'm looking for a way to have Javascript or Java read a directory and return the file names of any files that are older than a date that will be provided by a Pentaho parameter.
Here is what I currently have using a Modified Java Script Value step that only lists the directory contents:
var _getAllFilesFromFolder = function(dir) {
var filesystem = require("fs");
var results = [];
filesystem.readdirSync(dir).forEach(function(file) {
file = dir+'\'+file;
var stat = filesystem.statSync(file);
if (stat && stat.isDirectory()) {
results = results.concat(_getAllFilesFromFolder(file))
} else results.push(file);
});
return results;
};
Is Javascript/Java the right way to do this?
There's a step called "Get file names". You just need to provide the path you want to poll. It also allows doing so recursively, only showing filenames that match a given filter, and in the filters tab allow you to show only folders, only files, or both.
nsousa's answer would be the easiest, then after you get your file list you can use a filter rows step on the lastmodifiedtime returned from the Get file names. 2 -steps, 3 if you want to format the date/time returned to something easier to sort/filter through. This is the approach I use and its is faster then the transformations can keep up with generally.
I'm using a JavaScript lib here to select multiple image and save them in database through a form, and when i edit the form i get the images from data and populate like this.
<script>
var upload = new FileUploadWithPreview('myUniqueUploadId', {
showDeleteButtonOnImages: true,
text: {
chooseFile: 'Įkelkite nuotrauką',
browse: 'naršyti',
selectedCount: 'Pasirinkti failai',
},
presetFiles: [
<?php
foreach($Workimages as $Workimg){
echo $website_url.'master_image/'.$Workimg.','
}
],
})
</script>
The Images are coming as Blob and are being populated but when i update the data in database the old images are disregard but the new selected images get uploaded. I also checked $_FILES['images']. it don't have my old image files.
I have also tried to used methods upload.cachedFilesArray.push and upload.addFiles() but i wasn't successful at that. Is there any other way to populate multiple file input?
I'm the author of this library.
It seems as though you've uncovered a problem with a Blob type being accepted by your PHP server. This may be something that I just never ran into before, because I feel like I've uploaded Blob types in the past to my PHP servers - but maybe I'm just imagining things and they've always been File types and that's why I never had any issue.
Regardless, to make your situation work in the meantime, I made a simple codepen that loops over the upload.cachedFileArray array and converts each Blob (or File) into a File type, and appends it to a new temp array that's ready for uploading. See the comments in the codepen for more.
At this point, you can just upload the new temp array and you'll be all set. In the future I'll look at making sure all the presetFiles are of the type File instead of Blob.
I have another interesting case which I have never faced before, so I'm asking help from SO community and also share my experience with it.
The case || What we have:
A csv file (exported from other SQL DB) with such structure
(headers):
ID,SpellID,Reagent[0],Reagent[1..6]Reagent[7],ReagentCount[0],ReagentCount[1..6],ReagentCount[7]
You could also check a full -csv data file here, at my
dropbox
My gist from Github, which helps you to understand how MongoImport works.
What we need:
I'd like to receive such structure(schema) to import it into MongoDB collection:
ID(Number),SpellID(Number),Reagent(Array),ReagentCount(Array)
6,898,[878],[1]
with ID, SpellID, and two arrays, in first we store all Reagent IDs, like [0,1,2,3,4,5,6,7] from all Reagent[n] columns, and in the second array we have the array with the same length that represent quantity of ReagentIDs, from all ReagentCount[n]
OR
A transposed objects with such structure (schema):
ID(Number),SpellID(Number),ReagentID(Number),Quantity/Count(Number)
80,2675,1,2
80,2675,134,15
80,2675,14,45
As you may see, the difference between the first example and this one, that every document in the collection represents each ReagentID and it's quantity to SpellID. So if one Spell_ID have N different reagents it will be N documents in the collection, cause we all know, that there can't be more then 7 unique Reagent_ID belonging to one Spell_ID according to our -csv file.
I am working on this problem right now, with the help of node js and npm i csv (or any other modules for parsing csv files). Just to make my csv file available for importing to my DB via mongoose. I'll be very thankful for all those, who could provide any relevant contribution to this case. But anyway, I will solve this problem eventually and share my solution in this question.
As for the first variant I guess there should be one-time script for MongoImport that could concat all columns from Reagent[n] & ReagentCount[n] to two separate arrays like I mentioned above, via -fields but unfortunately I don't know it, and there are no examples on SO or official Mongo docs relevant to it. So if you have enough experience with MongoImport feel free to share it.
Finally I solve my problem as I want it to, but without using mongoimport
I used npm i csv and write function for parsing my csv file. In short:
async function FuncName (path) {
try {
let eva = fs.readFileSync(path,'utf8');
csv.parse(eva, async function(err, data) {
//console.log(data[0]); we receive headers, if they exist
for (let i = 1; i < data.length; i++) { //we start from 1, because 0 is headers, if we don't have it, then we start from 0
console.log(data[i][34]); //where i is row number and j(34) is a header address
}
});
} catch (err) {
console.log(err);
}
}
It loops over csv file and shows data in array that allows you to operate with them as you want it to.
I am reading in a csv file and i am in the process of parsing through it but having some trouble.
1 - Before parsing the file i already have an array with strings of headers that i want to pull the data for from the csv file.
2- I want to parse the file so i can also display the data into a table with my predefined headers. And any extra headers they will be ignored from being displayed in the table.
Here is my code:
this.predefinedHeaders = ["Name", "Age", "Gender"];
readCSV(event: Event) {
const file = (event.target as HTMLInputElement).files![0];
var reader = new FileReader();
let text = (reader.result as string).split(/\r\n|\r|\n/);
let lines = [];
for( var i=1; i<text.length; i++) {
var data = text[i].split(',');
var tarr=[];
for(var j=0; j<this.predefinedHeaders.length; j++) {
tarr.push(data[j]);
}
lines.push(tarr);
}
this.tableData = lines;
}
reader.readAsText(file);
What is currently happening is that data is being populated to the table but not under the right headers.. How can i bind the data to my headers... NOTE: the predefined are guaranteed to be part of the original headers from the file. the difference is that it doesn't show data for all the columns such several of them.
HTML View:
table
thead
tr
th(v-for='column in predefinedColumns) {{column.name}}
tbody
tr(v-for='(a, index)in data')
td(v-for='(b, index2) in a') {{data[index][index2]}}
You might want to use most popular CSV parser which is PapaParse.
URLs for the deep documentation:
https://www.papaparse.com/demo
this library has various configuration options and one of them is 'Header row' which is exact solution you need.
to use the predefined header you can supply header argument as true to have all the data parsed as key-value pairs.
example: { data: Papa.parse(reader.result, { header: true }) }
with 'header: true', it will take first row of CSV file as key value for all the row in CSV file
NPM package for easiest implementation in javaScript app:
https://www.npmjs.com/package/papaparse
If you want to have the predefined headers and display only the table with only needed columns.
checkout this one of my example on codesandbox,
https://codesandbox.io/embed/llqmrp96pm
sample CSV file is already uploaded in the same directory so you would be able to upload it and see magic
this CSV file in there has 7 or 8 column but I am displaying only 4 columns, I assume that's what you are looking for
I see you are looking for JavaScript solution, my example is created with ReactJS and couple of NPM libraries but it is almost same as you are looking for, I believe it would be easier than anything to replicate in your code.
I've got an update to my question.
What I really wanted to know was this:
How do I get csv data into netsuite?
Well, it seems I use the csv import tool to create a mapping and use this call to import the csv nlapiSubmitCSVImport(nlobjCSVImport).
Now my question is: How do I iterate through the object?!
That gets me half way - I get the csv data but I can't seem to find out how I iterate through it in order to manipulate the date. This is, of course, the whole point of a scheduled script.
This is really driving me mad.
#Robert H
I can think of a million reasons why you'd want to import data from a CSV. Billing, for instance. Various reports on data any company keeps and I wouldn't want to keep this in the file cabinet nor would I really want to keep the file at all. I just want the data. I want to manipulate it and I want to enter it.
Solution Steps:
To upload a CSV file we have to use a Suitelet script.
(Note: file - This field type is available only for Suitelets and will appear on the main tab of the Suitelet page. Setting the field type to file adds a file upload widget to the page.)
var fileField = form.addField('custpage_file', 'file', 'Select CSV File');
var id = nlapiSubmitFile(file);
Let's prepare to call a Restlet script and pass the file id to it.
var recordObj = new Object();
recordObj.fileId = fileId;
// Format input for Restlets for the JSON content type
var recordText = JSON.stringify(recordObj);//stringifying JSON
// Setting up the URL of the Restlet
var url = 'https://rest.na1.netsuite.com/app/site/hosting/restlet.nl?script=108&deploy=1';
// Setting up the headers for passing the credentials
var headers = new Array();
headers['Content-Type'] = 'application/json';
headers['Authorization'] = 'NLAuth nlauth_email=amit.kumar2#mindfiresolutions.com, nlauth_signature=*password*, nlauth_account=TSTDRV****, nlauth_role=3';
(Note: nlapiCreateCSVImport: This API is only supported for bundle installation scripts, scheduled scripts, and RESTlets)
Let's call the Restlet using nlapiRequestURL:
// Calling Restlet
var output = nlapiRequestURL(url, recordText, headers, null, "POST");
Create a mapping using Import CSV records available at Setup > Import/Export > Import CSV records.
Inside the Restlet script Fetch the file id from the Restlet parameter. Use nlapiCreateCSVImport() API and set its mapping with mapping id created in step 3. Set the CSV file using the setPrimaryFile() function.
var primaryFile = nlapiLoadFile(datain.fileId);
var job = nlapiCreateCSVImport();
job.setMapping(mappingFileId); // Set the mapping
// Set File
job.setPrimaryFile(primaryFile.getValue()); // Fetches the content of the file and sets it.
Submit using nlapiSubmitCSVImport().
nlapiSubmitCSVImport(job); // We are done
There is another way we can get around this although neither preferable nor would I suggest. (As it consumes a lot of API's if you have a large number of records in your CSV file.)
Let's say that we don't want to use the nlapiCreateCSVImport API, so let's continue from the step 4.
Just fetch the file Id as we did earlier, load the file, and get its contents.
var fileContent = primaryFile.getValue();
Split the lines of the file, then subsequently split the words and store the values into separate arrays.
var splitLine = fileContent.split("\n"); // Splitting the file on the basis of lines.
for (var lines = 1,count=0; lines < splitLine.length; lines++)
{
var words = (splitLine[lines]).split(","); // words stores all the words on a line
for (var word = 0; word < words.length; word++)
{
nlapiLogExecution("DEBUG", "Words:",words[word]);
}
}
Note: Make sure you don't have an additional blank line in your CSV file.
Finally create the record and set field values from the array that we created above.
var myRec = nlapiCreateRecord('cashsale'); // Here you create the record of your choice
myRec.setFieldValue('entity', arrCustomerId[i]); // For example, arrCustomerId is an array of customer ID.
var submitRec = nlapiSubmitRecord(myRec); // and we are done
fellow NetSuite user here, I've been using SuiteScripts for a while now but never saw nlobjCSVImport object nor nlapiSubmitCSVImport .. I looked in the documentation, it shows, but there is no page describing the details, care to share where you got the doc from?
With the doc for the CSVImport object I might be able to provide some more help.
P.S. I tried posting this message as a comment but the "Add comment" link didn't show up for some reason. Still new to SOF
CSV to JSON:
convert csv file to json object datatable
https://code.google.com/p/jquery-csv/
If you know the structure of the CSV file, just do a for loop and map the fields to the corresponding nlapiSetValue.
Should be pretty straightforward.