Develop Reference

Need script to loop through stock ticker list, copy resulting output, and paste the output of multiple rows to separate sheet within workbook

My question is similar to Need script to loop through stock ticker list, copy resulting output, and paste output to separate sheet within workbook. However, instead of copying and pasting a single row (for each ticker) below the last non-empty row, I need to copy and paste 100 rows (for each ticker) below the last non-empty row.
I have three sheets named Tickers , Data, and Results within a single workbook.
If done manually, the job would be the following: Enter a stock symbol into cell A2 in the Data sheet. The sheet then retrieves the historical time series data from google finance, runs formulas and returns 100 rows and 7 columns of data (Ticker, Date, Open, High, Low, Close, Volume) into range A5:G254 within the same Data sheet. Copy the 100 rows of returned data and paste the data on the Results sheet below the last non-empty row. Then repeat the process with each ticker.
Request: Create a script to loop through a list of 50 stock symbols from the Ticker sheet (range B2:B51), paste 1 symbol at a time into cell A2 in the Data sheet, wait for google finance API to run, and then copy all the resulting rows generated in range A5:G254, and pasting those results into the Results sheet, below the data generated from previous ticker, until there is a "long format" table of time series data of each of the 50 tickers.
The issue with my current script is that it pastes only the first row of the wanted range A5:G254.
function getNewPrices() {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const tickerSheet = ss.getSheetByName('Tickers');
const dataSheet = ss.getSheetByName('Data');
const resultsSheet = ss.getSheetByName('Results');
let data;
let myListOfTickers = tickerSheet.getRange('B2:B51').getValues().filter(String).sort();
myListOfTickers.forEach(ticker => {
dataSheet.getRange('A2').setValue(ticker[0]);
data = dataSheet.getRange('A5:G254').getValues()[0];
pasteData(resultsSheet, data);
});
};
function pasteData(resultsSheet,data){
let nextRow = resultsSheet.getLastRow() + 1;
data.forEach((datum,index) => {
resultsSheet.getRange(nextRow,index + 1,1,1).setValue(datum);
});
};

I'm not completely sure but I think this is what you want
function getNewPrices() {
const ss = SpreadsheetApp.getActive();
const tsh = ss.getSheetByName('Tickers');
let tvs = tsh.getRange('B2:B51').getValues().filter(String).sort();
const dsh = ss.getSheetByName('Data');
const rsh = ss.getSheetByName('Results');
dsh.getRange(2,1,tvs.length,tvs[0].length).setValues(tvs);
let dvs = dsh.getRange('A5:G254').getValues();
rsh.getRange(rsh.getLastRow() + 1, 1, dvs.length, dvs[0].length).setValues(dvs);
}
You can goto to Google Apps Script Reference and using the search box find any function that you don't understand. If it's a pure JavaScript function the go here

Problem solved. Here is the final code if anyone wants to use it later on. Thank you for the initial code Cooper.
function getNewPrices() {
const ss = SpreadsheetApp.getActive();
const tsh = ss.getSheetByName('Tickers');
let tvs = tsh.getRange('B2:B51').getValues().filter(String).sort();
const dsh = ss.getSheetByName('Data');
const rsh = ss.getSheetByName('Results');
tvs.forEach(ticker =>{
dsh.getRange('A2').setValue(ticker[0]);
let dvs = dsh.getRange('A5:G254').getValues();
rsh.getRange(rsh.getLastRow() + 1, 1, dvs.length, dvs[0].length).setValues(dvs);
});
}

How to efficiently remove rows that match criteria on another sheet?

I have a requirement to remove a list of exceptions/exclusions from a large (115244 rows) data set.
The total number of exclusions is 1133.
I have the following piece of code:
function removeExclusions() {
const ss = SpreadsheetApp.getActive();
const exclusionSheet = ss.getSheetByName("Exclusion List");
const excludedAccounts = exclusionSheet
.getRange("C2:C" + exclusionSheet.getLastRow())
.getValues()
.reduce(
(o, [c]) =>
Object.assign(o, {
[c]: true
}),
{}
);
Logger.log("Total accounts to remove: " + excludedAccounts);
const dataSheet = ss.getSheetByName("Data Sheet");
const dataSheetMatches = dataSheet
.getRange("A2:A" + dataSheet.getLastRow())
.getValues()
.reduce((ar, [a], i) => {
if (excludedAccounts[a]) ar.push(i + 2);
return ar;
}, [])
.reverse();
Logger.log("Accounts left to remove: " + dataSheetMatches.length);
dataSheetMatches.forEach((r) => {
dataSheet.deleteRow(r);
Logger.log("Row:" + r + " has been deleted");
});
}
However, due to the size of the dataset/number of accounts/rows, this needs to run against - it takes an eternity and hits the timeout that Google has on Apps Script runtimes.
I need a much more efficient way to do the following:
Check the "Exclusion List" sheet (column C), then remove the row in the "Data Sheet" when it matches against column A.
Rows A-O can be cleared. Columns P-S header row (1) contains some formulas that I need to keep.
Any suggestions?
EDIT
So I have amended my code to the following:
function removeExclusions() {
const ss = SpreadsheetApp.getActive();
const exclusionSheet = ss.getSheetByName("Exclusion List");
const exclusionRange = exclusionSheet.getRange("C2:C");
const exclusionVals = exclusionRange.getDisplayValues();
const dataSheet = ss.getSheetByName("Data Sheet");
const dataSheetRange = dataSheet.getRange("A2:O");
let dataSheetVals = dataSheetRange.getValues();
dataSheetVals = dataSheetVals.filter((data) => {
return !exclusionVals.includes(data[0]);
});
Logger.log(dataSheetVals);
}
However, it's still showing the rows I would expect it to exclude...

it takes an eternity and hits the timeout that Google has on Apps
Script runtimes.
The cause of this is
dataSheet.deleteRow(r);
Even if you try it on 20 rows sheet, it's slow, you can almost observe rows being deleted one by one.
Besides that, 100K+ rows is sometimes too much for Google Sheets, the response time is slower.
The strategy you could use
Pull all data from "Data Sheet" into variable, using getValues(). (exclude formula columns if possible)
Pull "Exclusion List" column C into variable just like you did in the code
Use .filter on "Data Sheet" values array to create new array by removing the unwanted rows
Clear "Data Sheet" values (exclude formula columns if possible)
Write reduced data into "Data Sheet", using setValues().
Notes
This approach can take a long time too. It's much faster than deleteRow, but getValues and setValues on 100K+ rows and I guess 15-20 columns will take some time. The rough estimation of the execution time would be to do manual select all/copy/paste tests on the "Data Sheet" and see how long you have to wait.
If some person/process is adding/removing/moving/editing "Data Sheet" at the same time your function runs, you may have data loses. If you can prevent this by using some overnight once per day trigger to run your code you should be fine.
In general your code can break, you could make a duplicate of the sheet (to serve as a backup) at the beginning and then if everything is executed without an error delete the duplicate sheet at the end.
In general you should consider moving your data from Google Sheets to some platform that can handle larger amounts of records, I use BigQuery for such scenarios.
Edit: response to your amended code
Assuming that values you are comparing are strings:
function somethingLikeThis() {
const ss = SpreadsheetApp.getActive();
// Load "Exclusion List" column C to array of strings
const exclusionSheet = ss.getSheetByName("Exclusion List");
const lastRowExclusionSheet = exclusionSheet.getLastRow();
const exlusionList = exclusionSheet
.getRange(2, 3, lastRowExclusionSheet - 2 + 1) // Data starts at row 2, column is C
.getValues()
.map(row => row[0].toString()) // Convert array of arrays to array of strings
.filter(el => el.length > 0); // Remove empty rows if any
// Read values from "Data Sheet"
const dataSheet = ss.getSheetByName("Data Sheet");
const lastRowDataSheet = dataSheet.getLastRow();
const oldDataRange = dataSheet
.getRange(2, 1, lastRowDataSheet - 2 + 1, 15) // Data starts at row 2, columns A-O
const oldDataValues = oldDataRange.getValues();
// Clear "Data Sheet"
oldDataRange.clearContent();
// Keep rows where column A value is not on the "Exclusion List"
const newDataValues = oldDataValues
.filter(row => exlusionList.indexOf(row[0].toString()) < 0);
// Write reduced rows to "Data Sheet"
if (newDataValues.length > 0) {
dataSheet.getRange(2, 1, newDataValues.length, newDataValues[0].length)
.setValues(newDataValues);
}
}
Code is not tested, I don't have actual sheets. Try using getLastRow(), ranges like "C2:C" can pick empty rows at the end of the sheet.

Move Specific Rows depending on Filtering Keywords within unknown amount of rows using Google Sheets Apps Scripts

I do SEO, and therefore I have a lot of keywords flowing around in different spreadsheets. I'd like a way to filter these into seperate sheets based on specific filters, but I can't for the life of me, figure out how to do this in Google Apps Script.
Criteria I set myself for this to work out:
A list of strings and their corresponding volumes are entered in column 1+2.
A list of filter-words are written in column 3.
The script has to create a new sheet for each of the filter words and move the strings + volumes into these different sheets if the string contains a filter word.
Example:
Filter words: Apple, Banana, Pineapple
String: "The Apple Was Big", Volume: "100"
The script would move the string and volume into the sheet called "Apple" on row 1
(Beware, I'm in no means experienced in coding)
I believe you can use the following structure:
for(let i = 0; i <= column3RowAmount; i++){ //Run as long as there are more filter words
create(column3Row[i]); //create a new sheet with the name of the filter word
for(let j = 0; j <= column1RowAmount; j++){ //Run as long as there are more keywords
if(column1Row[j].indexOf(column3Row[i]) >= 0){ //If the Row in column 1 contains the filter word
column1Row[j].moveToSheet(column3Row[i]); // Make sure not to move Column 3, but only 1+2
}
}
}
Example sheet: https://docs.google.com/spreadsheets/d/15YIMyGmmfZdy094gwuJNxFmTd8h7NOLnA8KevZrGtdU/edit?usp=sharing

Explanation:
Your goal is to create a sheet for every filter-word in column C. Then copy the data in columns A, B but only the rows that include the filter-word to the corresponding sheet.
For starters, you need to get the filter-word list. You can get the full range of column C and filter out the empty cells:
const sh_names = sh.getRange('C1:C').getValues().flat().filter(r=>r!='');
Similarly, you need to get the data in columns A and B:
const data = sh.getRange('A1:B'+sh.getLastRow()).getValues();
The next step is to iterate over sh_names and for every element / filter-word, check if a sheet with that name exists. If it does not exist, then create a sheet with that name, if it exists then skip the creation part:
if(!ss.getSheetByName(s)){
ss.insertSheet().setName(s);}
The next step is to filter data on the rows that include the filter-word:
let f_data = data.filter(r=>r[0].includes(s));
Finally, check if the length of the data is bigger than 0, otherwise there is not data to use and set the values of data to the corresponding sheet:
sheet.getRange(sheet.getLastRow()+1,1,f_data.length,f_data[0].length).setValues(f_data)
Solution
function myFunction() {
const ss = SpreadsheetApp.getActive();
const sh = ss.getSheetByName('Ark1');
const filter_sh = ss.getSheetByName('Filter');
const data = sh.getRange('A1:B'+sh.getLastRow()).getValues();
const sh_names = filter_sh.getRange('A1:A'+filter_sh.getLastRow()).getValues().flat();
sh_names.forEach(s=>{
if(!ss.getSheetByName(s)){
ss.insertSheet().setName(s);}
let sheet = ss.getSheetByName(s);
let f_data = data.filter(r=>r[0].includes(s));
if(f_data.length>0){
sheet.getRange(sheet.getLastRow()+1,1,f_data.length,f_data[0].length).setValues(f_data);}
});
}

This function will place all of your results into column 4 next to the appropriate word rather than creating a page for each word. So it runs much faster.
function stringswords() {
const ss=SpreadsheetApp.getActive();
const sh=ss.getSheetByName('Sheet1');
const sr=2;
const rgd=sh.getRange(sr,1,sh.getLastRow()-sr+1,2);
const data=rgd.getDisplayValues();
const rgw=sh.getRange(sr,3,sh.getLastRow()-sr+1,1);
const words=rgw.getDisplayValues().flat();
const wiObj={};
words.forEach(function(w,i){wiObj[w]=i});
const rgr=sh.getRange(sr,4,sh.getLastRow()-sr+1,1);
rgr.clearContent();
var results=rgr.getValues();
words.forEach(function(w,i,A){
data.forEach(function(r,j,D) {
if(data[j][0] && data[j][0].indexOf(w)!=-1) {
results[wiObj[w]][0]+=Utilities.formatString('String:%s Vol:%s\n',data[j][0],data[j][1]);
}
});
});
rgr.setValues(results);
}
Image of Data and output:

How do I group multiple tab ranges in array literal dynamically(by using wildcard)?

I have one spreadsheet with multiple tabs, e.g.
"Summary"
"Week 1"
"Week 2"
"Backlog"
The number of Weeks is not fixed. In the future, I may add more tabs as time goes on, for example
"Week 3"
"Week 4"
...
In the "summary" tab, I want to list all the data from Column A (remove title) from all "Week" tabs.
Currently, I have to add all the tab names of "Week" manually in the query range parameter, e.g.
=query({'Week 1'!A2:A; 'Week 2'!A2:A}, "select *")
But I want to make it automatically if I add more tabs in the future by doing something like this:
=query({"Week *"}!A2:A, "select *")
How do I do that?

An option would be to get rid of the QUERY formula altogether and use an Apps Script Custom Function instead.
First, open a bound script by selecting Tools > Script editor, and copy the following functions to the script:
function SUMMARIZE_WEEKTABS() {
const ss = SpreadsheetApp.getActive();
const weekSheets = ss.getSheets().filter(sheet => sheet.getName().startsWith("Week "));
const summarySheet = ss.getSheetByName("Summary");
let weekData = weekSheets.map(weekSheet => {
return weekSheet.getRange(2, 1, weekSheet.getLastRow()).getValues().flat();
});
weekData = weekData[0].map((_, colIndex) => weekData.map(row => row[colIndex]));
return weekData;
}
The function SUMMARIZE_TABS returns the data from column A from all sheets whose name starts with "Week ". Once it is defined in your script, you can use it the same way you would use any sheets built-in function. See, for example, this:
Update:
If you want all data to be written on the same column, use this instead:
function SUMMARIZE_WEEKTABS() {
const ss = SpreadsheetApp.getActive();
const weekSheets = ss.getSheets().filter(sheet => sheet.getName().startsWith("Week "));
const summarySheet = ss.getSheetByName("Summary");
let weekData = weekSheets.map(weekSheet => {
return weekSheet.getRange(2, 1, weekSheet.getLastRow() - 1).getValues();
}).flat();
return weekData;
}
Reference:
Custom Functions in Google Sheets

Explanation:
You can use Google Apps Script to accomplish this task.
The following script:
gets all the sheets of your spreadsheet file that contain the name
Week,
iterates over these sheets and construct a string object which can be
used inside the query formula,
sets the resulting formula to cell A1 of the Summary sheet. Feel free to modify A1 in the code below.
I also added a function that creates a macro button on top of your sheet, so you can use this functionality from the sheet itself (see instructions below).
Solution:
Please follow the instructions (gif) on how to use the following code:
function getQuery() {
const ss = SpreadsheetApp.getActive();
const summary_sheet = ss.getSheetByName('Summary');
const sheet_names = ss.getSheets().filter(fsh=>fsh.getName().includes("Week")).map(sh=>sh.getName())
var weeks = [];
sheet_names.forEach(wk =>
weeks.push(`'${wk}'!A2:A`)
)
summary_sheet.getRange('A1').setFormula(`=query({${weeks.join(";")}}, "select *")`)
}
function onOpen() {
SpreadsheetApp.getUi()
.createMenu('Macros')
.addItem('Get Query', 'getQuery')
.addToUi();
}
Instructions:
After you save the code snippet in the script editor, refresh your sheet/browser.
References:
Google Apps Script libraries:
Class Spreadsheet
setFormula()
onOpen trigger
JavaScript libraries:
forEach()
map()

How to compare two sheets and delete/add any column with a distinct value in row 1? Google Script

I want to compare two sheets (based on header values in row 1) and delete any column with a unique value (without a match). For example, Assuming Sheet1, Row 1 data and Sheet 2, Row 1 are uniform, if a user adds/deletes a column within any sheet, I want to always match the number of columns in both sheets with their values
Screenshots of sheets headings.
IF both sheets looks like this
And a user adds a new Column N
Or delete column N
How can I ensure that both sheet matches by deleting the odd/distinct column in Sheet 1?
I have tried modifying this code below but I can't just get the unique one out. This code only look for headers with a defined value.
function deleteAloneColumns(){
var sheet = SpreadsheetApp.getActiveSheet();
var lastColumnPos = sheet.getLastColumn();
var headers = sheet.getRange( 1 ,1, 1, lastColumnPos ).getValues()[0];
for( var i = lastColumnPos ; i < 1; i--){
if( headers[i] === "alone" ) sheet.deleteColumn(i);
}
SpreadsheetApp.getUi().alert( 'Job done!' );
}
Any help to compare and delete the column with the unique value will be appreciated.

Problem
Balancing sheets based on header row values mismatch.
Solution
If I understood you correctly, you have a source sheet against which validation is run and two primary use cases: user adds a new column named differently than any other column (if you want to check that the column strictly matches the one in sheet1, it is easy to modify) in source sheet or deletes one that should be there.
const balanceSheets = (sourceShName = 'Sheet1',targetShName = 'Sheet2') => {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const s1 = ss.getSheetByName(sourceShName);
const s2 = ss.getSheetByName(targetShName);
const s2lcol = s2.getLastColumn();
//keep all vals from source to reduce I/O
const s1DataVals = s1.getDataRange().getValues();
const s2Vals = s2.getRange(1, 1, 1, s2lcol).getValues();
const h1Vals = s1DataVals[0];
const h2Vals = s2Vals[0];
//assume s1 is source (validation) sheet
//assume s2 is target sheet that a user can edit
//case 1: target has value not present in source -> delete column in target
let colIdx = 0;
h2Vals.forEach(value => {
const isOK = h1Vals.some(val => val===value);
isOK ? colIdx++ : s2.deleteColumn(colIdx+1);
});
//case 2: target does not have values present in source -> append column from source
h1Vals.forEach((value,index) => {
const isOK = h2Vals.some(val => val===value);
!isOK && s2.insertColumnAfter(index);
const valuesToInsert = s1DataVals.map(row => [row[index]]);
const numRowsToInsert = valuesToInsert.length;
s2.getRange(1,index+1, numRowsToInsert,1).setValues(valuesToInsert);
});
};
Showcase
Here is a small demo of how it works as a macros:
Notes
Solving your problem with two forEach is suboptimal, but I kept number of I/O low (it can be lowered further by, for example, moving deleteColum out of the loop while only keeping track of column indices).
The script uses ES6 capabilities provided by V8, so please, be careful (although I would recommend migrating as soon as possible - even if you encounter bugs / inconsistencies , it is worth more than it costs.
UPD made script more flexible by moving sheet names to parameter list.
UPD2 after discussing the issue with deleteColumn() behaviour, the answer is updated to keep column pointer in bounds (for those curious about it - forEach kept incrementing the index, while deleteColumn reduced bounds for any given index).
Reference
insertColumnAfter() method reference