I am writing about a problem my org is having, as a result I have to be a bit vague (though I have permission to post this).
I am writing a script parse a google-sheet that has a column with numerical data in it. I need the script to return a set of ranges (strings in A1 notation for use elsewhere). The ranges have to correspond to real ranges in the script for which the numerical data sums to as close to 0.25 as possible but not more than .25, and return all ranges in the sheet for which this is true. E.g. if my sheet data was
A (COLUMN HEADER)
DataHeader (A1)
.1 (A2)
.01 (A3)
.04 (A4)
.1 (A5)
.03 (A6)
.02 (A7)
.1 (A8)
.05 (A9)
.04 (A10)
.07 (A11)
my script should return the array [A2:A4,A5:A10,A11], because A2:A4 sum to exactly .25, while A5:A10 sum to below (but close to) .25 but including A11 would get us above .25. A11 is included because I still need the data there.
In reality, my script will return larger ranges in A1 notation, but the problem I'm having is occuring in my attempt to properly construct a running sum and return the rows of interest.
My script is broken into 2 functions at the moment, because I was tackling the problems I was having in pieces.
The first script IDs the following info: The data to examine, as well as the number of rows from the top of the sheet that the start of the data
var data
and
var A1offset
respectively
The second function does a running sum of an array of my data and build an array of row number pairs representing the ranges at which the sum is just below .25. I have pasted the portion throwing an error below.
var result = [] //empty array to store row number pairs.
//data is a previously obtained array of my data of interest
//A1offset is the number of rows between the top of the sheet and the start of data
for (var i = 0;i<data.length;i++){
cursum += data[i][0]//add current value to cursum
if( cursum >= .25 && result.length<1){//if we are # the beginning add first rownum (A1 Notation) of data to result
i=i-1 //Go back one so that i++ runs, we don't miss this datapoint.
cursum = 0 //reset
result.push([(A1offset),(i + A1offset )])//
}
else if( cursum >= .25 && result.length>=1){
i=i-1//go back one
cursum = 0//reset
result.push([(result[result.length-1][1]+1),(i+A1offset)]) //Add to result a pair of values, the
//row after the end of the last range and the current row.
}
}
The issue:
Previously, I did not have the
i = i-1
statement but I realized I needed that or
i--
However after adding these two statements, upon running my code I get an 'Out of Memory error' error.
However, this does not happen if I loop over a smaller dataset. Experimentally, I have found that if I use
for (var i=0;i<.85*data.length;i++){
it runs fine.
data has 4978 elements in it.
Question: Is there an upper limit to the number of iterations a google script can run? I was under the impression that if there was, it was much higher than 4978 (or closer to 5050, since I'm using the i=i-1 statement)
Or, did I simply write poor code? Or both?
Thank you for any insights you can provide, I don't have a CS degree so frequently I run into issues where I don't know what I don't know.
Please let me know if I have not provided enough information.'
Edited to add a link to a test version of my sheet as well as the entire script below (overcommented to be certain):
function IDDATA() {
//ID the data to sum
var spsheet = SpreadsheetApp.getActive(); //Get spreadsheet
var tab = spsheet.getSheetByName("Test") //Get data sheet
var Drows = tab.getRange("A:A").getValues() //All values in Column D
var filledvals = Drows.filter(String).length //Length of filled in rows in Column D
//Subtract out the header row from the length of values
// Logger.log(datarows)
var offset = 0 //Initialize at 0. Offset is the distance (ZERO INDEXED <-THIS IS IMPORTANT!) between row 1 and the data
var ct1 = 0//initialize counter 1 at 0
while(Drows[ct1][0].indexOf("DATA")<0){ //Look at the ct1 Row of the 0th column of Column D
ct1++//Count until a the row with DATA in it. indexOf(String) returns -1 if the string is not found.
}
offset = ct1 //set offset to ct1.
var A1offset = ct1 + 2//Assume that all data has a header row, and create an offset variable that points to the first row of NUMERICAL data.
var datarows = filledvals-2 //Gets us the ROW NUMBER of the last filled in row in D:D, which should be the LAST number in the data.
return([datarows,offset,A1offset])
}
function RUNSUM(){
var firstcol = "A"
var lastcol = "A"
var spsheet = SpreadsheetApp.getActive(); //Get spreadsheet
var tab = spsheet.getSheetByName("Test") //Get sheet
var vararray = IDDATA()
var len = vararray[0] //Length of data
var offset = vararray[1] //Unused?
var A1offset = vararray[2]//The offset from 0, but to be used in A1 Notation
// Logger.log(typeof(A1offset))
var startrow = "A"+A1offset//The first row to get data from in the column that has system size data.
var endrow = "A"+(len+A1offset)//The last row to get data from that has system size data.
var cursum = 0 //Initialize # 0, the current sum of system size data.
var range = tab.getRange(startrow+":"+endrow) //Range to examine, in A1 notation Could probably use other forms of getRange()....
var data = range.getValues() //Values in range to exampine
var testmax = Math.floor(.85*data.length)
var result = [] //Positions, initialize as empty array
//var exceptions = [] //Rows that contain non-numbers
for (var i = 0;i<.8*data.length;i++){//.8* because of memory error
/* if(isNaN(data[i][0])){ //if data[i][0] is Not a Number
exceptions.push((i+A1offset).toFixed(0))//push the row number to exclude from eventual csv
continue //skip to next i
}*/
cursum += data[i][0]//add current value to cursum
if( cursum > .25 && result.length<1){//if we are # the beginning, we need to know, see results.push statement below
i=i-1
cursum = 0 //reset
result.push([(A1offset),(i + A1offset )])//
}
else if( cursum > .25 && result.length>=1){
i=i-1//go back one
cursum = 0//reset
result.push([(result[result.length-1][1]+1),(i+A1offset)])
}
}
var rangearray = []
var intarray = [] //declare an empty array to intermediate array vals todo: remove in favor of single loop
for(var k = 0; k < result.length; k++){
intarray.push([result[k][0].toFixed(0),result[k][1].toFixed(0)])//build array of arrays of result STRING vals w/out decimal pts.
}
for (var j = 0;j<result.length;j++){
rangearray.push(firstcol+intarray[j][0]+":"+lastcol+intarray[j][1])
}
Logger.log(rangearray)
return rangearray
}
In the Sheet you provided, the cell 4548 contains the number 0.2584.
Your algorithm, upon finding that number, will insert it into the result array and decrement i, creating an infinite loop that appends data into your array.
In order to fix your issue, in the RUNSUM() function right after the for loop declaration (for (var i = 0;i<data.length;i++)), you should handle this situation (or otherwise, make your data "valid" by removing that outlier value).
Furthermore, I have rewritten the algorithm in the hopes of making it more clear and stable (see that it now handles the last range, and ignores values over 0.25). I believe this may be interesting to you:
function myAlgorithm() {
var sheet = SpreadsheetApp.getActive().getSheetByName('Test');
var values = sheet.getRange("A6:A").getValues();
var l = 0;
var results = []; // 1-indexed results
var currentSum = 0;
var offset = 5;
for (var i=0; i<values.length && typeof values[i][0] == 'number'; i++) {
if (values[i][0] >= 0.25) {
// Ignore the value larger or equal than 0.25
Logger.log('Found value larger or equal than 0.25 at row %s', i+1+offset);
if (l<i) {
results.push([l+1+offset, i+offset, currentSum]);
}
l = i+1;
currentSum = 0;
} else {
if (currentSum + values[i][0] > 0.25) {
results.push([l+1+offset, i+offset, currentSum]);
l = i;
currentSum = values[i][0];
} else {
currentSum += values[i][0];
}
}
}
// handle last range after exitig the for-loop
if (l<i) {
results.push([l+1+offset, i+offset, currentSum]);
}
for (var i=0; i<results.length; i++) {
Logger.log(results[i]);
}
}
Finally, if you decide to modify your code in order to handle this situation, I suggest you check out the information in the following link:
Troubleshooting in GAS
Related
I wrote the code below to grab a portion of a column of numbers and add them up. However, the function is concatenating numbers rather than adding.
I'm getting this result:
0AMOUNT120123126129132135138141144147
But if I run it from Dec on the sum should be:
432
My code in Google Scripts:
//add sum and input it into the total column
function sum(startMonth, startColumn, lastRow){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[0];
startColumn = startColumn +1;
var sum = 0;
var currAmount = 0;
var k = 0;
for(k = startMonth; k <= lastRow; k++){
currAmount = sheet.getRange(k, startColumn).getValue();
sum += currAmount; //currAmount + sum;
Logger.log(sum);
}
SpreadsheetApp.getActiveSheet().getRange(k, startColumn).setValue(sum);
SpreadsheetApp.getActiveSheet().getRange(k, startColumn).setBackground("Yellow");
return sum;
}
And a snapshot of my sheet:
Your result is the hint: That's not a number being added. That's strings being concatenated.
currAmount = sheet.getRange(k, startColumn).getValue();
sum += currAmount; //currAmount + sum;
Specifically, this is the main problem. Regardless of whether the number returned by getValue() is a number or not, you add it to sum.
A few ways to fix this would be to adjust the value of k, to check the typeof the value, or coerce the value into a number, depending on exactly what you're trying to do. This is essentially a javascript problem, with the canonical answer here
Also, generally you should batch operations where possible; e.g., instead of running getRange().getValue() every go through a for-loop, you are much better using (as an example, may need tweaking)
var amounts = sheet.getRange(startMonth, startColumn, (lastRow - startMonth), 1);
for(var k = startMonth; k <= lastRow; k++){
sum += amounts[k][0]
Logger.log(sum);
}
So I have a column of data in a sheet, that I am iterating over with an apps script function.
The problem is that my sheet has a number of empty cells at the end, which are rendered as [] when I use .getRange().
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("AUD Data");
var sheetRange = sheet.getRange("F2:F").getValues();
function logRange() {
Logger.log(sheetRange)
}
Returns
I want to get all these blank cells at the end, out of my cell array, as it is messing with results when I loop over this array.
Have tried filter() and isNaN but those solutions didn't work.
Ideally I would like to remove the empty arrays, before looping, but a way of ignoring them in a condition would be fine too.
I find it interesting that even though these empty cells are displaying as empty arrays, Apps Script seems to treat them as having a value. My conditions should ignore imo... eg.
else if (i > 0 && sheetRange[i] <= 0 && sheetRange[i-1] <= 0)
How can an empty array be treated as a number?
NOTE: I can fix this by deleting the empty rows in the sheet, but would rather do this with code.
How about this answer?
Issue 1:
In your spreadsheet, if the last row of all columns is the same or the last row of the column "F" is most bottom, how about this modification?
From:
var sheetRange = sheet.getRange("F2:F").getValues();
To:
var sheetRange = sheet.getRange(2, 6, sheet.getLastRow() - 1, 1).getValues();
Or if the last row of the column "F" is different from the last row of other columns, how about the following modified script?
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("AUD Data");
var sheetRange = sheet.getRange("F2:F").getValues();
for (var i = sheetRange.length - 1; i >= 0; i--) {
if (sheetRange[i][0]) {
sheetRange.splice(i + 1, sheetRange.length - (i + 1));
break;
}
}
Logger.log(sheetRange)
Or if you want to also remove the empty elements of the middle of array, how about the following script?
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("AUD Data");
var sheetRange = sheet.getRange("F2:F").getValues().filter(String);
Logger.log(sheetRange)
Issue 2:
About else if (i > 0 && sheetRange[i] <= 0 && sheetRange[i-1] <= 0), from your script, I thought that you might use 2 dimensional array is used for this situation. If sheetRange is the value retrieved by getValues(), it is 2 dimensional array. And getRange("F2:F") is used for this. So how about this modification?
From:
else if (i > 0 && sheetRange[i] <= 0 && sheetRange[i-1] <= 0)
To:
else if (i > 0 && sheetRange[i][0] <= 0 && sheetRange[i-1][0] <= 0)
References:
getLastRow()
splice()
getValues()
If I misunderstood your question and this was not the direction you want, I apologize.
All of the cases in the filter should be eliminated with a simple truthy statement, but this will explicitly check to make sure none of those conditions happen.
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("AUD Data");
var sheetRange = sheet.getRange("F2:F").getValues();
sheetRange = sheetRange.filter(sheet => sheet != null && sheet != 'undefined' && sheet.trim() != '');
function logRange() {
Logger.log(sheetRange)
}
I'm trying to make a google script to automate some calculations in a google spreadsheet.
I have several things in this sheet and what i need is, for all the date values i have on A column (between rows 3 and 20) i need to search on the column T (between rows 17 and 50) and when a date match i need to grab the value in the cell next to the data in column T.
Then with this sum i need to go to the B move one row down and substract that sum from the above cell.
example:
A
17/8/2017
18/8/2017
19/8/2017
B
100
empty
empty
T
empty
empty
18/8/2017
18/8/2017
18/8/2017
19/8/2017
U
empty
empty
5
5
2
1
After running the script B should be:
100
88
87
My code:
function burnDown(){
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var data = sheet.getDataRange().getValues();
var completed = sheet.getRange(20, 20, 40);
for(var i = 3; i<20;i++){
for(var j = 1; i<50; j++){
var sum = 0;
if(data[i][0] == completed[j][0])
{
sum = sum + completed[j][20];
Logger.log(completed[j][21] + "" + sum);
}
j++;
}
i++;
}
}
I'm stuck at this point where I have to make the match but i'm getting the error: TypeError: Cannot read property "0" from undefined. (line 9, file "BurnDown"
Thanks,
No values in completed. Perhaps you need a getValues() somewhere
function burnDown()
{
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var data = sheet.getDataRange().getValues();
//var completed = sheet.getRange(20, 20, 40);//this is a range
var completed = sheet.getRange(20,20,40).getValues();//this creates a 2d array of values
for(var i = 3;i<20;i++)
{
for(var j = 1; j<50; j++)
{
var sum = 0;
if(data[i][0]==completed[j][0])
{
sum=sum + completed[j][20];
Logger.log(completed[j][21] + "" + sum);
}
//j++;//not sure why these are here the loop increments
}
//i++;//same question as above
}
}
Thank you in advance for your help.
I have a google sheet that contains header values in the first row. I have a script that is looking through the remainder of the sheet (row by row) and if a cell is a certain color the script keeps a count. At the end, if the count number is greater than a variable I set in the sheet the script will trigger an email.
What I am looking at trying to do, is to also capture the column header value if the script finds a cell with the set color? I'm sure I need to create an array with the header values and then compare the positions, I'm just not sure how to do so efficiently.
function sendEmails() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var dataSheet = ss.getSheets()[0];
var lastRow = dataSheet.getLastRow();
var lastColumn = dataSheet.getLastColumn();
//Project Range Information
var projectRange = dataSheet.getRange(6,3,lastRow-5,lastColumn);
var projectRangeValues = projectRange.getValues()[0];
var cellColors = projectRange.getBackgrounds();
//Student Information Range
var studentRange = dataSheet.getRange(6,1,lastRow-5,lastColumn);
var studentRangeValues = studentRange.getValues();
//Pull email template information
var emailSubject = ss.getRange("Variables!B1").getValue();
var emailText = ss.getRange("Variables!B2").getValue();
var triggerValue = ss.getRange("Variables!B4").getValue();
var ccValue = ss.getRange("Variables!B5").getValue();
//Where to Start and What to Check
var colorY = ss.getRange("Variables!B6").getValue();
var count = 0;
var startRow = 6;
//Loop through sheet and pull data
for(var i = 0; i < cellColors.length; i++) {
//Pull some information from the rows to use in email
var studentName = studentRangeValues[i][0];
var studentBlogUrl = studentRangeValues[i][1];
var studentEmail = studentRangeValues[i][2];
var studentData = [studentName,studentBlogUrl];
//Loop through cell colors and count them
for(var j = 0; j < cellColors[0].length ; j++) {
if(cellColors[i][j] == colorY) {
/*This is where I feel I need to add the array comparisons to get the header values */
count = count + 1;
};//end if statement
};//end for each cell in a row
//If the count is greater than trigger, send emails
if (count >= triggerValue) {
//A call to another function that merges the information
var emailBody = fillInTemplateFromObject(emailText, studentData);
MailApp.sendEmail({
to: studentEmail,
cc: ccValue,
subject: emailSubject,
htmlBody: emailBody,
});
} else {};
//reset count to 0 before next row
count = 0;
};//end for each row
};
EDIT:
I have updated the above sections of the code to based on the responses:
//Header Information
var headers = dataSheet.getRange(4,4,1,lastColumn);
var headerValues = headers.getValues();
var missingAssignments = new Array();
In the for loop I added:
//Loop through cell colors and count them
for(var j = 0; j < cellColors[0].length ; j++) {
if(cellColors[i][j] == colorY) {
//This pushes the correct information into the array that matches up with the columns with a color.
missingAssignments.push(headervalues[i][j]);
count = count + 1;
};//end if statement
};//end for each cell in a row
The issue I am running into is that I am getting an error - TypeError: Cannot read property "2" from undefined. This is being caused by the push in the for loop as the script moves to the next row. I am unsure why I am getting this error. From other things I have read, the array is set as undefined. I have tried to set the array to empty and set it's length to 0, but it does not help. I don't think I understand the scoping of the array as it runs through.
EDIT:
Figured it out, the "i" should not iterate. It should read:
missingAssignments.push(headervalues[0][j]);
The end of the first for loop I clear the array for the next row.
missingAssignments.length = 0;
You should get the values of the entire sheet. Then use the shift method to get just the headers. It is hard for me to completely understand your intent without more information about your sheet. Let me know if I can provide more information.
function sendEmails() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var dataSheet = ss.getSheets()[0];
var lastRow = dataSheet.getLastRow();
var lastColumn = dataSheet.getLastColumn();
//below gets the whole sheet and shifts off the first row as headers
var fullSheet = dataSheet.getDataRange().getValues();
var headers = fullSheet.shift();
//then in your loops you can check against the index of the headers array
Spreadsheets with Apps Scripts is really slow especially if you have lots of data to read.
Check these tips from Apps doc:
Use batch operations
Scripts commonly need to read in data from a spreadsheet, perform
calculations, and then write out the results of the data to a
spreadsheet. Google Apps Script already has some built-in
optimization, such as using look-ahead caching to retrieve what a
script is likely to get and write caching to save what is likely to be
set.
You can write scripts to take maximum advantage of the built-in
caching, by minimizing the number of reads and writes. Alternating
read and write commands is slow. To speed up a script, read all data
into an array with one command, perform any operations on the data in
the array, and write the data out with one command.
Here's an example — an example you should not follow or use. The
Spreadsheet Fractal Art script in the Gallery (only available in the
older version of Google Sheets) uses the following code to set the
background colors of every cell in a 100 x 100 spreadsheet grid:
// DO NOT USE THIS CODE. It is an example of SLOW, INEFFICIENT code.
// FOR DEMONSTRATION ONLY
var cell = sheet.getRange('a1');
for (var y = 0; y < 100; y++) {
xcoord = xmin;
for (var x = 0; x < 100; x++) {
var c = getColor_(xcoord, ycoord);
cell.offset(y, x).setBackgroundColor(c);
xcoord += xincrement;
}
ycoord -= yincrement;
SpreadsheetApp.flush();
}
The script is inefficient: it loops through 100 rows and 100 columns,
writing consecutively to 10,000 cells. The Google Apps Script
write-back cache helps, because it forces a write-back using flush at
the end of every line. Because of the caching, there are only 100
calls to the Spreadsheet.
But the code can be made much more efficient by batching the calls.
Here's a rewrite in which the cell range is read into an array called
colors, the color assignment operation is performed on the data in the
array, and the values in the array are written out to the spreadsheet:
// OKAY TO USE THIS EXAMPLE or code based on it.
var cell = sheet.getRange('a1');
var colors = new Array(100);
for (var y = 0; y < 100; y++) {
xcoord = xmin;
colors[y] = new Array(100);
for (var x = 0; x < 100; x++) {
colors[y][x] = getColor_(xcoord, ycoord);
xcoord += xincrement;
}
ycoord -= yincrement;
}
sheet.getRange(1, 1, 100, 100).setBackgroundColors(colors);
The
inefficient code takes about 70 seconds to run. The efficient code
runs in just 1 second!
If you're looking at the Spreadsheet Fractal Art script (only
available in the older version of Google Sheets), please be aware that
we made a minor change to it to make this example easier to follow.
The script as published uses the setBackgroundRGB call, rather than
setBackgroundColor, which you see above. The getColor_ function was
changed as follows:
if (iteration == max_iteration) {
return '#000000';
} else {
var c = 255 - (iteration * 5);
c = Math.min(255, Math.max(0, c));
var hex = Number(c).toString(16);
while (hex.length < 2)
hex = '0' + hex;
return ('#' + hex + '3280');
}
I have a trouble executing the following snippet. It is callback code for a larger python Bokeh plotting. Since my experience with javascript is limited I was drawing from this question on how to loop over an array in javascript.
This is the modified objects2 = ColumnDataSource(data=dict(x=[],y1=[],y2=[],y3=[], y4=[]))
var inds = cb_obj.get('selected')['1d'].indices; // Obtains column numbers
var d1 = m1.get('data'); // Obtains matrix with numbered columns that correspond to inds
var d2 = s2.get('data'); // Plotted data columns that get modified according to inds
var ys = ['y1','y2','y3','y4']; // Array of names of columns in d2/s2
var j, y, select= ys.slice(0,inds.length+1), // Limit number of columns to number of inds
//This piece is from the linked question - a loop through select (e.g only 'y1' and 'y2')
var len = select.length;
for(j=0; j<len; ++j) {
if (j in select) {
y = selected[j];
// For each selected name of a column
d2[y] = []; // Erase current value
// Fill the column from d1 (12 is just the length of the column)
for (var i = 0; i < 12; i++) {
d2[y].push(d1[inds[j]][i]),
}
}
}
s2.trigger('change');
The second loop was working just fine for a predefined number of y1,y2, when only two indices were passed to it. But my goal is to make the selection size dynamic i.e for a varying number of indices like [1,5,65,76] or [1,8] which is passed down from outside to inds, the code is to fill up appropriate number of ys so either y1,y2,y3,y4 or just y1,y2 with data from d1/m1.
From what I can tell the code "should" be doing just that , but I am obviously not seeing something.
For comparison here is the previous version of code that worked, but was limited to exactly two indices.
var inds = cb_obj.get('selected')['1d'].indices;
var d1 = m1.get('data');
var d2 = s2.get('data');
d2['y'] = []
d2['y2'] = []
for (i = 0; i < 12; i++) {
d2['y1'].push(d1[inds['0']][i]),
d2['y2'].push(d1[inds['1']][i])
}
s2.trigger('change');
Here is the whole thing (Python) with sample data for better visualization.
I figured it out. There was an extra comma at the end of this line
d2[y].push(d1[inds[j]][i])