Develop Reference

Copy filtered range from one spreadsheet to another - Google app script

I have a large google sheet with 30275 rows and 133 columns in a google sheet. I want to filter the data and copy column AZ to another spreadsheet.
Link to spreadsheet: https://docs.google.com/spreadsheets/d/1aiuHIYzlCM7zO_5oZ0aOCKDwPo06syXhWvhQMKgJE2I/edit?usp=sharing
I have been trying to follow this link
I am not that familiar with javascript and the code is designed to exclude items from filter rather than including items on filter. I have 500+ items to exclude so need to work out something that will be efficient in filtering large dataset in short time before execution limit is reached.
Here is my code so far. Any help to get this working would be appreciated.
NOTE: Filter/ Query with importrange formulas dont work due to the large volume of data. So I need an efficient script to filter large dataset and move them to another sheet before execution time limit.
function filtered() {
var ss = SpreadsheetApp.openById('1u9z_8J-tvTZaW4adO6kCk7bkWeB0pwPcZQdjBazpExI');
var sheet = ss.getSheetByName('Sheet1');
var destsheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('JockeyList');
var demosheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Betting data - Demo');
var jockey = demosheet.getRange('L14').getValues();
// Get full (non-filtered) data
var values = sheet.getRange('A:EC').getValues();
// Apply filter criteria here
//Logger.log(jockey);
var hiddenValues = jockey;
values = values.filter(function(v) {
return hiddenValues.indexOf(v[51]) == 1;
});
Logger.log(values.length);
// Set filtered data on the target sheet
destsheet.getRange(10, 5, values.length, 2).setValues(values);
}

Ok so it seems like you want to copy only the values from AZ in 'Sheet1' that are equal to whatever string value is contained in cell L14 of sheet 'Betting data - Demo.' If that is the case, then here is a change to your original code that will accomplish that:
function filtered() {
var ss = SpreadsheetApp.openById('1u9z_8J-tvTZaW4adO6kCk7bkWeB0pwPcZQdjBazpExI');
var sheet = ss.getSheetByName('Sheet1');
// this assumes that your destsheet and demosheet are in the same spreadsheet. Keep in mind that opening spreadsheets with SpreadsheetApp is costly so you want to minimize your calls to new sheets.
var destsheet = ss.getSheetByName('JockeyList');
var demosheet = ss.getSheetByName('Betting data - Demo');
var jockey = demosheet.getRange('L14').getValue();
var searchTerm = new RegExp(jockey);
// Get full (non-filtered) data
var values = sheet.getRange('A:EC').getValues();
// Apply filter criteria here and return ONLY THE VALUE IN COLUMN AZ
var filteredValues = values.reduce(function(resultArray, row) {
if (searchTerm.test(row[51])) resultArray.push([row[51]]);
return resultArray;
}, [])
// Set filtered data on the target sheet
// Note* not clear why you are starting at row 10, but as is this will overwrite all of the data in column 5 of destsheet starting from row 10 every time this function runs
destsheet.getRange(10, 5, filteredValues.length, 1).setValues(filteredValues);
}
As it says in the code sample, this will only copy and paste the value in column AZ of 'sheet1'. It does not alter 'sheet1' in any way. If that is really all you want to do, then this function works, but it's overkill. Since you are just filtering values in AZ against a single string value, it would be more efficient to simply count the number of times that string occurs in column AZ and then add the string that number of times to your destination sheet.
Also note that your original function is pasting values into destsheet into a constant row (row 10). That means that every time your function runs, the data from row 10 on will be overwritten by whatever data you are copying from 'sheet1'

How can i combine rows range into json format?

I have a google sheet like this
The number of columns and rows may be different (not static). I want my google script to look for last row and column, then run a for loop and combine data of each row into this json format like this.
[["1","Lee","Blue","Active"],["2","Mike","Green","Disabled"],["3","Chan","Yellow","Active"]]
I learned about the JSON.stringify() function that helps and I am able to combine data or each row into json format BUT i want each row data separated by a comma (,), not sure how to add that in the logic since I DO NOT want comma to be added after LAST row.
I have tried creating this code and need help with the logic to dynamically run the loop based on number of columns and rows instead of hard-coding any range and add comma.
var mysheet = SpreadsheetApp.getActive().getSheetByName('mysheet');
function combine_val() {
GetRowsCount();
var startRow = 2; // First row of data to process
var numRows = LastAudienceRow; // Number of rows to process
for (var i = 0; i < numRows; ++i) {
var dataRange = mysheet.getRange(startRow, 1, 1, 4);
var data = dataRange.getValues();
startRow = startRow+1;
var data2= JSON.stringify(data);
}
SpreadsheetApp.getUi().alert(data2);
}
function GetRowsCount() {
LastAudienceRow = mysheet.getLastRow() - 1;
return LastAudienceRow;
}

You've rather over complicated the whole situation, as much of this is done automatically. This script is also not very efficient and when you add more runs, will take too long to execute, eventually hitting the App script limit for maximum execution time.
Here you are trying to get the data from each row individually, and add it to an array. This is very inefficient, and it's much better/faster to just specify the range of data you want. Happily, this also formats it's pretty much as you like.
Here's a sample script:
var mysheet = SpreadsheetApp.getActive().getSheetByName('mysheet');
function combine_val() {
var startRow = 1; // First row of data to process.
var startColumn = 1; //First Column to process, in case that changes.
var numRows = mysheet.getLastRow(); // Number of rows to process
var numCols = mysheet.getLastColumn(); //Also the number of columns to process, again in case that changes.
var dataRange = mysheet.getRange(startRow, startColumn, numRows, numCols);//Get the full range of data in the sheet dynamically.
var data = JSON.stringify(dataRange.getValues());//Get the value of the range, AND convert it to a JSON string in one line.
Logger.log(data); //Use the in built logger to read the values that are returned. You can read this by pressing 'ctrl+enter'.
}
I've also added in some sample dynamic improvements which will mean you are hard coding less of your values into your scripts, preventing it from breaking if the data changes.
Additionally, I have added in a sample of how to read the data via the inbuilt logging tool, rather then via a cumbersome pop up after every run.

JSON.stringify already returns row data separated by commas.
var arr = [["1","Lee","Blue","Active"],["2","Mike","Green","Disabled"],["3","Chan","Yellow","Active"]];
console.info(JSON.stringify(arr))
The above example prints the following string to the console
[["1","Lee","Blue","Active"],["2","Mike","Green","Disabled"],["3","Chan","Yellow","Active"]]
The following example prints each row values separated by comma.
var arr = [["1","Lee","Blue","Active"],["2","Mike","Green","Disabled"],["3","Chan","Yellow","Active"]];
for(var i = 0; i < arr.length; i++){
console.info(JSON.stringify(arr[i]).replace(/[\[\]]/g,''));
}
To use it on as a Google Sheets / Apps Script instead of assigning a literal to arr use getValues() . You could replace console.info by SpreadsheetApp.getUi.alert(message) too.

Javascript array grid object property serialized by Kendo is null on server

I have a Kendo grid that I have an editable detail template. In the detail template, I have a multiselect.
When I save, I am manually updating the dataItem to grab the most up to date options in the detail template's multiselect. There is no problem with grabbing the data from the multiselect. Everything actually looks pretty good until I trigger the grid to save.
I add the multiselect items to the grid's dataItem with this code:
function SaveRefItemSubCategories() { //Called on Save button click
var grid = $("#RefItemSubCategoryGrid").data("kendoGrid");
for (var i = 0; i < grid.dataSource._data.length; i++) {
var dataItem = grid.dataSource._data[i];
dataItem.RefItemCategoryServiceIDs = [];
var ms = $("#MultiSelect_" + dataItem.uid).data("kendoMultiSelect");
var values = ms.value();
for(var x = 0; x < values.length; x++) {
dataItem.RefItemCategoryServiceIDs.push(values[i]);
}
}
grid.dataSource.sync();
}
At that point, when I look at what is posted to the server, the grid's dataSource looks like this for the serialized multiselect items:
models[0].RefItemCategoryServiceIDs[] 15
&models%5B0%5D.RefItemCategoryServiceIDs%5B%5D=15
It should look like this:
models[0].RefItemCategoryServiceIDs[0] 15
&models%5B0%5D.RefItemCategoryServiceIDs%5B 0%5D=15
Sorry for extra space in there, I couldn't get it to bold otherwise
The lack of the array's index in the serialized data results in that property being null when it hits the server. this property's type is List
I have tried a couple different approaches, but the end up creating new properties on the dataItem that stick around and cause problems after I save for the first time.
Some examples I've used instead of the push() method are:
This one does send the data back, but it ignores the existing list and creates a new dataSource property for each array index. This would work if I was only saving once, but if I save the grid multiple times, then the properties added stick around and mess with data integrity. I would entertain this option if a way to 'reset' the dataSource was possible after saving (without reloading the grid).
dataItem["RefItemCategoryServiceIDs[" + i + "]"] = values[i];
This one had the same effect as the .push(), which is the property on my object on the server being null
dataItem.RefItemCategoryServiceIDs[i] = values[i];
Please let me know if I can provide any more information and thanks in advance!

I'm not confident that this is the best way to handle it, but I have puzzled out a way that works.
I changed adding to the data item's properties from this:
for(var x = 0; x < values.length; x++) { //values is array from multiselect
dataItem.RefItemCategoryServiceIDs.push(values[i]);
}
to this:
for (var i = 0; i < values.length; i++) { //values is array from multiselect
dataItem["RefItemCategoryServiceIDs[" + i + "]"] = values[i];
}
Just this change works for every scenario EXCEPT this:
Save row with 2 options chosen from multiselect, resulting in 2 properties being added to the dataItem. We'll say the ID's of those items are 1 and 21. Posted data looks like this:
models[0].RefItemCategoryServiceIDs[0] = 1
models[0].RefItemCategoryServiceIDs[1] = 21
After saving, if we open that multiselect again and delete the first option (the one with ID 1), we get the following data posted:
models[0].RefItemCategoryServiceIDs[0] = 21
models[0].RefItemCategoryServiceIDs[1] = 21
Which is obviously not right (should just be 1 entry with the ID set to 21). This is happening because the RefItemCategoryServiceIDs isn't really an array, just properties whose names look like array entries so that they serialize into a list on the server (as I understand it). To mitigate this issue, I preceded the code that maps the current multiselect values to delete the existing properties associate with this array:
function CleanDataItemLists(dataItem) {
if (dataItem.RefItemCategoryCarrierIDs != undefined && dataItem.RefItemCategoryCarrierIDs.length > 0) {
var i = 0;
//Continues to remove properties by index until none are left
while (dataItem.RefItemCategoryCarrierIDs[i] != undefined) {
delete dataItem["RefItemCategoryCarrierIDs[" + i + "]"];
i++;
}
//Cleans up empty array property
delete dataItem.RefItemCategoryCarrierIDs;
}
return dataItem;
}
After this function is called, you can execute the for loop code referenced above and the data serializes with an index present (See initial post for example of missing index in serialized data), allowing the server to map it to the list object. Like I said, I'm not convinced that this is the best way to do this, but it does work for me. Hopefully, this can help someone else or lead to an even better answer.

indexOf returning -1 despite object being in the array - Javascript in Google Spreadsheet Scripts

I am writing a script for a Google Docs Spreadsheet to read a list of directors and add them to an array if they do not already appear within it.
However, I cannot seem to get indexOf to return anything other than -1 for elements that are contained within the array.
Can anyone tell me what I am doing wrong? Or point me to an easier way of doing this?
This is my script:
function readRows() {
var column = SpreadsheetApp.getActiveSpreadsheet().getRangeByName("Director");
var values = column.getValues();
var numRows = column.getNumRows();
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[0];
var directors = new Array();
for (var i = 0; i <= numRows - 1; i++) {
var row = values[i];
if (directors.indexOf(row) == -1) {
directors.push(row);
} else {
directors.splice(directors.indexOf(row), 1, row);
}
}
for (var i = 2; i < directors.length; i++) {
var cell = sheet.getRange("F" + [i]);
cell.setValue(directors[i]);
}
};

When you retrieve values in Google Apps Script with getValues(), you will always be dealing with a 2D Javascript array (indexed by row then column), even if the range in question is one column wide. So in your particular case, and extending +RobG's example, your values array will actually look something like this:
[['fred'], ['sam'], ['sam'], ['fred']]
So you would need to change
var row = values[i];
to
var row = values[i][0];
As an aside, it might be worth noting that you can use a spreadsheet function native to Sheets to achieve this (typed directly into a spreadsheet cell):
=UNIQUE(Director)
This will update dynamically as the contents of the range named Director changes. That being said, there may well be a good reason that you wanted to use Google Apps Script for this.

It sounds like an issue with GAS and not the JS. I have always had trouble with getValues(). Even though the documentation says that it is a two dimensional array, you can't compare with it like you would expect to. Although if you use an indexing statement like values[0][1] you will get a basic data type. The solution (I hope there is a better way) is to force that object into a String() and then split() it back into an array that you can use.
Here is the code that I would use:
var column = SpreadsheetApp.getActiveSpreadsheet().getRangeByName("Director");
var values = column.getValues();
values = String(values).split(",");
var myIndex = values.indexOf(myDirector);
If myDirector is in values you will get a number != -1. However, commas in your data will cause problems. And this will only work with 1D arrays.
In your case: var row = values[i]; row is an object and not the string that you want to compare. Convert all of your values to an array like I have above and your comparison operators should work. (try printing row to the console to see what it says: Logger.log(row))

I ran into a similar problem with a spreadsheet function that took a range as an object. In my case, I was wanting to do a simple search for a fixed set of values (in another array).
The problem is, your "column" variable doesn't contain a column -- it contains a 2D array. Therefore, each value is it's own row (itself an array).
I know I could accomplish the following example using the existing function in the spreadsheet, but this is a decent demo of dealing with the 2D array to search for a value:
function flatten(range) {
var results = [];
var row, column;
for(row = 0; row < range.length; row++) {
for(column = 0; column < range[row].length; column++) {
results.push(range[row][column]);
}
}
return results;
}
function getIndex(range, value) {
return flatten(range).indexOf(value);
}
So, since I wanted to simply search the entire range for the existance of a value, I just flattened it into a single array. If you really are dealing with 2D ranges, then this type of flattening and grabbing the index may not be very useful. In my case, I was looking through a column to find the intersection of two sets.

Because we are working with a 2D array, 2dArray.indexOf("Search Term") must have a whole 1D array as the search term. If we want to search for a single cell value within that array, we must specify which row we want to look in.
This means we use 2dArray[0].indexOf("Search Term") if our search term is not an array. Doing this specifies that we want to look in the first "row" in the array.
If we were looking at a 3x3 cell range and we wanted to search the third row we would use 2dArray[2].indexOf("Search Term")
The script below gets the current row in the spreadsheet and turns it into an array. It then uses the indexOf() method to search that row for "Search Term"
//This function puts the specified row into an array.
//var getRowAsArray = function(theRow)
function getRowAsArray()
{
var ss = SpreadsheetApp.getActiveSpreadsheet(); // Get the current spreadsheet
var theSheet = ss.getActiveSheet(); // Get the current working sheet
var theRow = getCurrentRow(); // Get the row to be exported
var theLastColumn = theSheet.getLastColumn(); //Find the last column in the sheet.
var dataRange = theSheet.getRange(theRow, 1, 1, theLastColumn); //Select the range
var data = dataRange.getValues(); //Put the whole range into an array
Logger.log(data); //Put the data into the log for checking
Logger.log(data[0].indexOf("Search Term")); //2D array so it's necessary to specify which 1D array you want to search in.
//We are only working with one row so we specify the first array value,
//which contains all the data from our row
}

If someone comes across this post you may want to consider using the library below. It looks like it will work for me. I was getting '-1' return even when trying the examples provide (thanks for the suggestions!).
After adding the Array Lib (version 13), and using the find() function, I got the correct row!
This is the project key I used: MOHgh9lncF2UxY-NXF58v3eVJ5jnXUK_T
And the references:
https://sites.google.com/site/scriptsexamples/custom-methods/2d-arrays-library#TOC-Using
https://script.google.com/macros/library/d/MOHgh9lncF2UxY-NXF58v3eVJ5jnXUK_T/13
Hopefully this will help someone else also.

I had a similar issue. getValues() seems to be the issue. All other methods were giving me an indexOf = -1
I used the split method, and performed the indexOf on the new array created. It works!
var col_index = 1;
var indents_column = main_db.getRange(1,col_index,main_db.getLastRow(),1).getValues();
var values = String(indents_column).split(","); // flattening the getValues() result
var indent_row_in_main_db = values.indexOf(indent_to_edit) + 1; // this worked

I ran into the same thing when I was using
let foo = Sheet.getRange(firstRow, dataCol, maxRow).getValues();
as I was expecting foo to be a one dimensional array. On research for the cause of the apparently weird behavior of GAS I found this question and the explanation for the always two dimensional result. But I came up with a more simple solution to that, which works fine for me:
let foo = Sheet.getRange(firstRow, dataCol, maxRow).getValues().flat();

Google Apps Scripting - Generic Headers in Spreadsheet

Quick look where I normally bother people tells me here is the new place to ask questions!
I have been making a script to create documentation generated from spreadsheet data which was in turn generated from a Google form. (Hope that makes sense...)
Anyway, I have been very successful with a lot of searching and bit of help but now I want to make my script homogeneous so I don't need to tinker with it when I want to setup new forms etc.
I have the getRowData function going from Googles script tutorials but instead of calling the row data from the normalised Headers i would like these to be generic i.e. Column1, Column2 etc.
I've pasted the tutorial function below, it passes the data to another function which normalizes the headers to use as objects, I was thinking thats where I could make them generic but I'm not sure how to get started on it...
Any help would be greatly appreciated!!
Thanks,
Alex
// getRowsData iterates row by row in the input range and returns an array of objects.
// Each object contains all the data for a given row, indexed by its normalized column name.
// Arguments:
// - sheet: the sheet object that contains the data to be processed
// - range: the exact range of cells where the data is stored
// - columnHeadersRowIndex: specifies the row number where the column names are stored.
// This argument is optional and it defaults to the row immediately above range;
// Returns an Array of objects.
function getRowsData(sheet, range, columnHeadersRowIndex) {
columnHeadersRowIndex = columnHeadersRowIndex || range.getRowIndex() - 1;
var numColumns = range.getEndColumn() - range.getColumn() + 1;
var headersRange = sheet.getRange(columnHeadersRowIndex, range.getColumn(), 1, numColumns);
var headers = headersRange.getValues()[0];
// Browser.msgBox(headers.toSource());
return getObjects(range.getValues(), normalizeHeaders(headers));
// return getObjects(range.getRowIndex);
}

If you want to get the columns using their index, why parse them to object at all? Just use the plain getValues!
var values = sheet.getDataRange().getValues();
var row2 = values[1];
var cell_in_col4 = row2[3];

It looks like you are missing "var" when declaring your columnHeadersRowIndex variable.