Appending new Google Sheets Data into BigQuery Table - javascript

So I'm new to all of this, both BigQuery and AppScript (coding in general..) and I'm learning as I go, so maybe to some my question may seem stupid. Please just hear me out.
I have created a script that loads 10 of the most recent data points into a Google Sheets doc from one of my BigQuery tables. Now, when I manually add new data points to the bottom of this table, I would like to have a load script run that uploads that new data back into BigQuery, and appends it to my original table. I read somewhere that just by inserting a new table the data is automatically appended if the table mentioned already exists. However, I haven't tested that part yet since I get stuck on an error earlier up the line..
Below is the load script I have loosely copied from https://developers.google.com/apps-script/advanced/bigquery
function loadCsv() {
var projectId = 'XX';
var datasetId = 'YY';
var tableId = 'daily_stats_testing';
var file = SpreadsheetApp.getActiveSpreadsheet();
var sheet = file.getActiveSheet().getRange("A12:AK10000").getValues();
var data = sheet.getBlob().setContentType('application/octet-stream');
var job = {
configuration: {
load: {
destinationTable: {
projectId: projectId,
datasetId: datasetId,
tableId: tableId
},
skipLeadingRows: 1
}
}
};
job = BigQuery.Jobs.insert(job, projectId, data);
var Msg = "Load started. Check status of it here:" +
"https://bigquery.cloud.google.com/jobs/%s", projectId
Logger.log(Msg);
Browser.msgBox(Msg);
return;
}
Now the error I get (in a variety of forms, since I've been testing stuff out) is that the BigQuery.Jobs function only accepts Blob data, and that the data from my current sheet (with rage A12 marking the first manually imputed row of data) is not a Blob recognizable data set.
Is there any way (any function?) I can use that will directly convert the selected data range and make it Blob compatible?
Does anyone have any recommendation on how to do this more efficiently?
Unfortunately the script has to load directly out of the current, open Spreadsheet sheet, since it is part of a larger script I will be running. Hopefully this isn't too much of a hinder!
Thanks in advance for the help :)

Is there any way (any function?) I can use that will directly convert the selected data range and make it Blob compatible?
There is actually a function that does convert objects into blob type, namely newBlob(data).
To test this I got a range from my spreadsheet and used it.
function blobTest(){
var sheet = SpreadsheetApp.getActiveSheet();
var range = sheet.getRange("A1:C1");
Logger.log(range); //here, range data is of type Range object
var blob = Utilities.newBlob(range);
Logger.log(blob); //here, range data is now of type Blob object
}

Related

How to pull data validation from another sheet in Google App Scripts

I'm trying to build a data validation for a cell from a range in a different sheet. I've tried the below code, and I continue to get this error "". Any help would be greatly appreciated.
function dataValidations(){
var builder = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Basic Info");
var invMgmt = SpreadsheetApp.openById("1eGNsJ2iB_IOzcfJe7DyFV_0WhzdQhvW2Lqdzfta2eV4");
var partners = invMgmt.getSheetByName("Partners and Locations").getRange(2,1,75,1).getValues();
var partnerRule = SpreadsheetApp.newDataValidation().requireValueInRange(partners).build();
var origin = builder.getRange("B4");
origin.setDataValidation(partnerRule);
}
In the case of requireValueInRange(range), range is the range object. In your script, the values retrieved by getValues() is 2 dimensional array. I think that this might be the reason of your issue. So in order to remove this issue, how about the following modification?
From:
var partners = invMgmt.getSheetByName("Partners and Locations").getRange(2,1,75,1).getValues();
var partnerRule = SpreadsheetApp.newDataValidation().requireValueInRange(partners).build();
To:
var partners = invMgmt.getSheetByName("Partners and Locations").getRange(2,1,75,1).getValues();
var partnerRule = SpreadsheetApp.newDataValidation().requireValueInList(partners.flat()).build(); // Modified
Note:
When SpreadsheetApp.openById("###").getSheetByName("###").getRange(###) is used for requireValueInRange, no error occurs and the retrieved values are the correct from other Spreadsheet. But when I confirmed the data validation in the cell, it is found that SpreadsheetApp.getActiveSpreadsheet() is used instead of SpreadsheetApp.openById("###"). So in this case, it seems that using requireValueInList is suitable.
References:
requireValueInRange(range)
requireValueInList(values)
flat()

JS Array not recognized as an array in Google Apps script

I am writing what should be a simple HTML form for users to make a reservation. However, having difficulty doing something I've done many times before - getting data from a GSheet and presenting it in an HTML form. Code snippets:
**Server side**
//Getting data from spreadsheet; expecting it to be an array of arrays.
function getMembers(){
const url = "https://my gsheet url"
const ss = SpreadsheetApp.openByUrl(url);
const ws = ss.getSheetByName("Members");
const row = ws.getLastRow()
const memberData = ws.getRange(2,1,row-1,7).getValues();
const array = JSON.stringify(memberData); //EDIT...helps get data to client side
return //array; //this data to be passed to browser when called. Type Error?
//Log(memberData) = //Looks like an array to me...
//[1,"Simmie","Smith","Simmie Smith","2020-01-01T05:00:00.000Z","temp","53120-1"],
//[2,"Fred","Williams","Fred Williams","2020-01-01T05:00:00.000Z","temp","53120-2"],
//[3,"Carleton","Johnson","Carleton Johnson","2020-01-01T05:00:00.000Z","temp","53120-3"],
//[4,"Caroll","Williams","Caroll Williams","2020-01-01T05:00:00.000Z","temp","53120-4"],
//[5,"Gabby","Williams","Gabby Williams","2020-01-01T05:00:00.000Z","temp","53120-5"]
//EDIT HtmlService code
function doGet(e){
return HtmlService.createTemplateFromFile('Reservations HTML')
.evaluate().setXFrameOptionsMode(HtmlService.XFrameOptionsMode.ALLOWALL)
.setSandboxMode(HtmlService.SandboxMode.IFRAME);
}
Calling the above server function when page loads so that member name can be presented in a select dropdown and other info from the array used elsewhere.
**Browser JS**
function getMemberData(){
google.script.run.withSuccessHandler(populateMembersDD).getMembers();
}
function populateMembersDD(data){
console.log(data) //undefined - same result FFox or Chrome..
memberInfo = data.filter(function(row){return true});
not getting any further than that because data is not passing to browser in a format with which I can apply necessary methods to get data into dropdown. Have tried various methods (slice, filter etc) to convert data into an array but none have worked. I suspect a type error, as though server function is not producing an object in array format. But I cannot figure out why. Have read many similar questions/answers on Stack but none have solved my problem. Help much appreciated as I am 24 hours in on solution experiments. Have many other apps where this basic code pattern works totally fine...

Google Script - importing file data from a file name listed in a spreadsheet

Very new to this, but have been pretty lucky with my tinkering in the past. Really stuck on this one, however.
Looking to import file data to specific sheets. With a specific file name, I've been successful with this script:
function importVauto() {
var app = SpreadsheetApp;
var import1 = app.getActiveSpreadsheet().getSheetByName('Import1');
var file = DriveApp.getFilesByName("samplefilename").next();
var csvData = Utilities.parseCsv(file.getBlob().getDataAsString());
import1.getRange(1, 1, csvData.length, csvData[0].length).setValues(csvData);
}
The challenge is, the file names change weekly. My thought was to have the required file names listed in a spreadsheet, reference the cell containing the file name, and import the required file. Tried this:
function importVauto() {
var app = SpreadsheetApp;
var import1 = app.getActiveSpreadsheet().getSheetByName('Import1');
var data = app.getActiveSpreadsheet().getSheetByName('Data');
var name = data.getRange("C24")
name.getValues()
Logger.log(name)
var file = DriveApp.getFilesByName(name).next();
var csvData = Utilities.parseCsv(file.getBlob().getDataAsString());
import1.getRange(1, 1, csvData.length, csvData[0].length).setValues(csvData);
}
Appears there is a problem with using the var 'name' as the file name for DriveApp.getFilesByName(), but chances are, there's a ton I'm missing (don't know what you don't know).
Hopefully this question makes sense (and even more hopefully, there is a simple solution). Again, very new to this. Appreciate any feedback.
How about this modification?
Modification points :
getValues() returns 2 dimensional array. If you want to retrieve the value of one cell "C24", you can use getValue(). In this case, you can directly retrieve the string value of the cell.
Modified script :
From :
name.getValues();
To :
name = name.getValue();
OR
name = name.getValues()[0][0];
Note :
When you retrieve file using DriveApp.getFilesByName(name).next(); if there are several files with the same filename, this can retrieve only one of them. Please be careful about this.
References :
getValues()
getValue()
If this was not what you want, please tell me. I would like to modify my answer.

Unable to perform functions on google sheets data

I have a script that pulls in data from different endpoints from an API.
The script then puts the data in a google sheet. In this sheet I want to chart the data or perform a math/sum operation. I am able to do this with the data from one endpoint just fine. But the other endpoints (using the same code) does not allow me to do this. I can do =A1+A2 and the value will add. But if I try to do =sum(A1:A2) the result is 0. No matter what I do. I am able to do the sum() operation on the other endpoint. Its very peculiar.
Here is some of the code from my apps script:
var response = UrlFetchApp.fetch(requestUrl, params);
var json = JSON.parse(response.getContentText());
var rows = [], jsondata;
for (i = 0; i < json.length; i++) {
jsondata = json[i];
rows.push([jsondata.currency, jsondata.amount, jsondata.fee, jsondata.date]);
}
return rows;
}
Your problem is with number formats, change the cells by highlighting them go to 'Format' -> 'Number' -> 'Automatic' and then it will work with =SUM(A1:A2)
To fix this issue in your script, use .setNumberFormat('0') on the range to set the format you require. The number format may need to be '0.00' if you are working with decimals.

How go I get csv data into netsuite?

I've got an update to my question.
What I really wanted to know was this:
How do I get csv data into netsuite?
Well, it seems I use the csv import tool to create a mapping and use this call to import the csv nlapiSubmitCSVImport(nlobjCSVImport).
Now my question is: How do I iterate through the object?!
That gets me half way - I get the csv data but I can't seem to find out how I iterate through it in order to manipulate the date. This is, of course, the whole point of a scheduled script.
This is really driving me mad.
#Robert H
I can think of a million reasons why you'd want to import data from a CSV. Billing, for instance. Various reports on data any company keeps and I wouldn't want to keep this in the file cabinet nor would I really want to keep the file at all. I just want the data. I want to manipulate it and I want to enter it.
Solution Steps:
To upload a CSV file we have to use a Suitelet script.
(Note: file - This field type is available only for Suitelets and will appear on the main tab of the Suitelet page. Setting the field type to file adds a file upload widget to the page.)
var fileField = form.addField('custpage_file', 'file', 'Select CSV File');
var id = nlapiSubmitFile(file);
Let's prepare to call a Restlet script and pass the file id to it.
var recordObj = new Object();
recordObj.fileId = fileId;
// Format input for Restlets for the JSON content type
var recordText = JSON.stringify(recordObj);//stringifying JSON
// Setting up the URL of the Restlet
var url = 'https://rest.na1.netsuite.com/app/site/hosting/restlet.nl?script=108&deploy=1';
// Setting up the headers for passing the credentials
var headers = new Array();
headers['Content-Type'] = 'application/json';
headers['Authorization'] = 'NLAuth nlauth_email=amit.kumar2#mindfiresolutions.com, nlauth_signature=*password*, nlauth_account=TSTDRV****, nlauth_role=3';
(Note: nlapiCreateCSVImport: This API is only supported for bundle installation scripts, scheduled scripts, and RESTlets)
Let's call the Restlet using nlapiRequestURL:
// Calling Restlet
var output = nlapiRequestURL(url, recordText, headers, null, "POST");
Create a mapping using Import CSV records available at Setup > Import/Export > Import CSV records.
Inside the Restlet script Fetch the file id from the Restlet parameter. Use nlapiCreateCSVImport() API and set its mapping with mapping id created in step 3. Set the CSV file using the setPrimaryFile() function.
var primaryFile = nlapiLoadFile(datain.fileId);
var job = nlapiCreateCSVImport();
job.setMapping(mappingFileId); // Set the mapping
// Set File
job.setPrimaryFile(primaryFile.getValue()); // Fetches the content of the file and sets it.
Submit using nlapiSubmitCSVImport().
nlapiSubmitCSVImport(job); // We are done
There is another way we can get around this although neither preferable nor would I suggest. (As it consumes a lot of API's if you have a large number of records in your CSV file.)
Let's say that we don't want to use the nlapiCreateCSVImport API, so let's continue from the step 4.
Just fetch the file Id as we did earlier, load the file, and get its contents.
var fileContent = primaryFile.getValue();
Split the lines of the file, then subsequently split the words and store the values into separate arrays.
var splitLine = fileContent.split("\n"); // Splitting the file on the basis of lines.
for (var lines = 1,count=0; lines < splitLine.length; lines++)
{
var words = (splitLine[lines]).split(","); // words stores all the words on a line
for (var word = 0; word < words.length; word++)
{
nlapiLogExecution("DEBUG", "Words:",words[word]);
}
}
Note: Make sure you don't have an additional blank line in your CSV file.
Finally create the record and set field values from the array that we created above.
var myRec = nlapiCreateRecord('cashsale'); // Here you create the record of your choice
myRec.setFieldValue('entity', arrCustomerId[i]); // For example, arrCustomerId is an array of customer ID.
var submitRec = nlapiSubmitRecord(myRec); // and we are done
fellow NetSuite user here, I've been using SuiteScripts for a while now but never saw nlobjCSVImport object nor nlapiSubmitCSVImport .. I looked in the documentation, it shows, but there is no page describing the details, care to share where you got the doc from?
With the doc for the CSVImport object I might be able to provide some more help.
P.S. I tried posting this message as a comment but the "Add comment" link didn't show up for some reason. Still new to SOF
CSV to JSON:
convert csv file to json object datatable
https://code.google.com/p/jquery-csv/
If you know the structure of the CSV file, just do a for loop and map the fields to the corresponding nlapiSetValue.
Should be pretty straightforward.

Categories

Resources