Optimization of code while working with range object in excel - javascript

I have recently moved for office add-in from vb.net to JavaScript and Office.js. In General, I have observed that JavaScript add-in is way too fast than to vb.net.
In one of the operations, I am not able to get benefit of speed in JavaScript add-in. Maybe it is due to inadequate code that I am using!
I have one master sheet which contains a large table, first column of table is name of every other sheets of that workbook and first row of table have address of cells. Rest of data in table are values to be transferred at the sheet name defined in the first column and cell adress defined in the first row.
In coding, I am creating an array of range object for each value indicated in table and then I run context. Sync () function to restore values in every sheets.
In my real-life application data in table can be 10K to 50K, I can see time taken for this operation is about approx. one minute (for 10K). In contrast of this, I can create the table (master sheet) within few seconds (5 -6 sec) only.
Is there any other workaround or suggestion to reduce time?
/* global Excel, console*/
export default async function restoreData() {
var allRollback = false;
await Excel.run(async (context) => {
var sheets = context.workbook.worksheets.load("items/name");
var wsdb = context.workbook.worksheets.getItem("db");
const arryRange = wsdb.getUsedRange();
var addRow = 0;
var sheetName = [];
var rangeObj = [];
//Get last row/column from used range
arryRange.load(["rowCount", "columnCount", "values", "address"]);
await context.sync();
sheets.items.forEach((sheet) => sheetName.push(sheet.name));
for (let iRow = 0; iRow < arryRange.rowCount; iRow++) {
if (arryRange.values[iRow][0] == "SheetName/CellAddress") {
addRow = iRow;
}
if (sheetName.indexOf(arryRange.values[iRow][0]) != -1) {
for (let iCol = 1; iCol < arryRange.columnCount; iCol++) {
if (arryRange.values[addRow][iCol]) {
const sheet = context.workbook.worksheets.getItem(arryRange.values[iRow][0]);
const range = sheet.getRange(arryRange.values[addRow][iCol]);
range.values = arryRange.values[iRow][iCol];
rangeObj.push(range);
}
}
} else {
// code for higlight Row in db
console.log("Y");
}
}
console.log("Range object created");
await context.sync();
// console.log(arryRange.rowCount);
// console.log(arryRange.columnCount);
console.log("done");
// Copy a range starting at a single cell destination.
});
allRollback = true;
return allRollback;
}

First, base on your statement, I assume you have a table in master sheet. This table's heading is like ["sheetName","A13","D23",...] ("A13" and "D23" are examples of cell address). In each row of this table, contain sheet's name and some values. The sheet's name may not related to a real sheet's name(not exist), and values may contain some blank. And you want to set values on other sheets based on the information given by master sheet's table.
Then I have some suggestions based on my assumptions and your code.
Move unchanged value out of loops.
For example, you called const sheet = context.workbook.worksheets.getItem(arryRange.values[iRow][0]);. We can move context.workbook.worksheets out of loops by define var sheets = context.workbook.worksheets and const sheet = sheets.getItem(arryRange.values[iRow][0]). Which could increase the performance.
Also some reused values like arryRange.values[iRow][0], arryRange.values[0][iCol] can be moved out of loop.
Seems you use arryRange.values[addRow][iCol] only for get the address in table's heading. You can replace it by arryRange.values[0][iCol].
Below is the code I rewrite, just for reference, it may not fully satisfy what you need.
export default async function restoreData() {
var allRollback = false;
await Excel.run(async (context) => {
var sheets = context.workbook.worksheets.load("items/name");
var wsdb = context.workbook.worksheets.getItem("db");
const arryRange = wsdb.getUsedRange();
//var addRow = 0;
var sheetName = [];
var rangeObj = [];
//Get last row/column from used range
arryRange.load(["rowCount", "columnCount", "values", "address"]);
await context.sync();
sheets.items.forEach((sheet) => sheetName.push(sheet.name));
var cellAddress, curSheetName;
const mySheets = context.workbook.worksheets;
for (let iRow = 0; iRow < arryRange.rowCount; iRow++) {
curSheetName = arryRange.values[iRow][0]
if (sheetName.indexOf(curSheetName) != -1) {
for (let iCol = 1; iCol < arryRange.columnCount; iCol++) {
cellAddress = arryRange.values[0][iCol];
if (cellAddress) {
const sheet = mySheets.getItem(curSheetName);
const range = sheet.getRange(cellAddress);
range.values = arryRange.values[iRow][iCol];
rangeObj.push(range);
}
}
} else {
// code for higlight Row in db
console.log("Y");
}
}
console.log("Range object created");
await context.sync();
// console.log(arryRange.rowCount);
// console.log(arryRange.columnCount);
console.log("done");
// Copy a range starting at a single cell destination.
});
allRollback = true;
return allRollback;
}
More references:
https://learn.microsoft.com/en-us/office/dev/add-ins/excel/performance?view=excel-js-preview
https://learn.microsoft.com/en-us/office/dev/add-ins/concepts/correlated-objects-pattern

with refer of you assumption, please note that master sheet was crated based on actual workbook with user selected area + deletion of empty value columns in master sheet (I,e empty cell at the adress at each sheet). sheet's name will be as same as actual sheet name unless user don’t change value in master sheet by accident.
With reference to yours,
Suggestions 1) I believe that Move unchanged value out of loops will be the key to my problem. I will refabricate one range and get data comparison for changes. I believe speed will be approved drastically in best case scenario. (I will also have some worst-case scenario (less than 5% case), where I will be required to write every values of master sheet).
Suggestions 2) I am planning to have a new functionality, which may have more rows as address in, that is the reason I have I am keep looking for address rows.
Thanks for your reply.

Related

How to update a custom formula inside google spredsheets automatically without loosing cells around?

Hello guys I am trying to receive crypto prices by using Googles Spreadsheet Formula, putting a Formula into google spreadsheets works just fine and I am getting the prices without any problems.
=IMPORTXML("https://www.marketscreener.com/quote/cryptocurrency/BITCOIN-EURO-45553925/";"//td[#class='fvPrice colorBlack']")
The Problem: Custom Formulas like IMPORTXML do not update automatically in the spreadsheet, so I grabbed every method from google which does this task, none of them which worked. Until I found one that works partially, it does the job but unfortunately deletes the cells around my formula.
This is my spreadsheet before running the method Before
This is my spreadsheet after running the method After
And this is the method that partially works:
/**
* Get all sheet names from active spreadsheet
*/
function sheetNames() {
var sheetNames = [];
var sheets = SpreadsheetApp.getActiveSpreadsheet().getSheets();
for (var i = 0; i < sheets.length; i++) sheetNames.push( sheets[i].getName() )
return sheetNames;
}
/**
* Update all formula data in cell range
*/
function updateData() {
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet(),
activeSheet = spreadsheet.getActiveSheet()
activeSheetName = activeSheet.getSheetName(),
activeRange = activeSheet.getActiveRange(),
sheets = sheetNames() // OR define an array of sheet names: ['Sheet1', 'Sheet2'],
// cellRange = (activeRange.getValues().length > 1) ? activeRange.getA1Notation() : Browser.inputBox('Enter your range'),
range = '';
// loop through all sheets
sheets.forEach(function(sheet) {
var originalFormulas = [],
range = activeRange,
defaultRanges = {
'europeList': 'A1:F9',
'europeList': 'A1:F9'
},
// set cell range if entered, ortherwise fall back to default
cellRange = defaultRanges[sheet];
// set range to active range, otherwise fall back to defined cell range
range = spreadsheet.getSheetByName(sheet).getRange(cellRange);
originalFormulas = range.getFormulas();
// modify formulas
const modified = originalFormulas.map(function (row) {
return row.map(function (col) {
return col.replace('=', '?');
});
});
// update formulas
range.setFormulas(modified);
SpreadsheetApp.flush();
// return formulas to original
range.setFormulas(originalFormulas);
});
}
Anyone got a idea, how to fix this?

Copy Data to another sheet and Calculate Totals

I want to copy the names to another tab as well as their costs, and then do the total cost of all similar items.
My logic for the script is to compare each value to the previous copied value and if it is different then the output should be "Total" but if it is the same then the "Name" should be copied. (I basically want to copy the data row by row, value by value, so that I can put certain conditions based on which the copying should happen or not.
The script I have written so far is only for copying the names to the target sheet but even that isn't working out for me. My knowledge is clearly limited, if anyone could help out, that would be great!
This is the doc that I am using, if anyone wants to refer to.
Attached images for those who can't access it:
Source Data:
How I want the target sheet to look like
function myFunction() {
Copy();
}
function Copy() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var ts = ss.getSheetByName("Total");
for (var n=1;n<10;n++) {
var namesSourceRange = ss.getSheetByName("Breakdown").getRange(1,2,n,1);
var namesTargetRange = ss.getSheetByName("Total").getRange(1,2,n,1);
var sourceNames = namesSourceRange.getValues();
var targetNames = namesTargetRange.getValues();
if (n=1){
namesTargetRange.setValues(sourceNames);
}
else{
if (sourceNames[n] === targetNames[n-1]){
namesTargetRange.setValues(sourceNames);
}
else {
namesTargetRange.setValue("Total");
}
}
}
}
You could do the following:
Sort the source data according to the Names column, using sort.
Iterate through the source data, for example, using forEach. Push each row values to another array destData which will be used to copy the data to the destination.
If the row's name is not the same as the next one (that means it's the last row corresponding to this Name), or if there's no next one (that is to say, it's the last row in the source data), calculate the Total for this name, using filter and reduce, and then push that total to destData.
Use setValues(values) to copy destData to the destination sheet.
Code sample:
function Copy() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var ts = ss.getSheetByName("Totals");
var sourceSheet = ss.getSheetByName("Breakdown");
var sourceData = sourceSheet.getRange(2,2,sourceSheet.getLastRow()-1,2).getValues();
sourceData.sort((a,b) => {
if (a[0] < b[0]) return 1;
else return -1;
});
const destData = [["Names", "Cost"]];
sourceData.forEach((row, i) => {
destData.push(row);
if (sourceData.length === i + 1 || row[0] !== sourceData[i+1][0]) {
const total = sourceData.filter(rowData => rowData[0] === row[0]).map(row => row[1]).reduce((acc,current) => acc+current);
destData.push(["Total", total], ["", ""])
}
});
ts.getRange(1,2,destData.length, destData[0].length).setValues(destData);
}

Move Row to Other Sheet Based on Cell Value

I have a sheet with rows I'd like to move to another sheet based on a cell value. I tried following this post's solution (refer below), but I'm having trouble editing the script towards what I want it to do.
I'm doing Check-Ins for an event. I would like to be able to change the value in Column F, populate Column G with the time the status changed, and for the row data to migrate to the Attendee Arrived sheet.
I believe the script already does this, but one has to run it manually. It also takes care of deleting the row data in A (Event) after migrating it to B (Attendee Arrived).
My question is could someone please help me set it up in order for script to run continuously (on edit), and also accomplish all of the above if I missed something?
I don't believe the script will respect the drop down format as it runs so I'm willing to manually type in something. It'd be cool if it could stay that way though - makes it easier for one.
Here's the sheet I'm testing on.
https://docs.google.com/spreadsheets/d/1HrFnV2gFKj1vkw_UpJN4tstHVPK6Y8XhHCIyna9TLJg/edit#gid=1517587380
Here's the solution I tried following. All credit to Jason P and Ritz for this.
Google App Script - Google Spreadsheets Move Row based on cell value efficiently
Thank you D:
function CheckIn() {
// How Many Columns over to copy
var columsCopyCount = 7; // A=1 B=2 C=3 ....
// What Column to Monitor
var columnsToMonitor = 6; // A=1 B=2 C=3 ....
//TARGET SPREAD SHEETS
var target1 = "Attendee Arrived";
//Target Value
var cellvalue = "Attendee Arrived";
//SOURCE SPREAD SHEET
var ss = SpreadsheetApp.openById('1HrFnV2gFKj1vkw_UpJN4tstHVPK6Y8XhHCIyna9TLJg');
var sourceSpreadSheetSheetID = ss.getSheetByName("Event");
var sourceSpreadSheetSheetID1 = ss.getSheetByName(target1);
var data = sourceSpreadSheetSheetID.getRange(2, 1, sourceSpreadSheetSheetID.getLastRow() - 1, sourceSpreadSheetSheetID.getLastColumn()).getValues();
var attendee = [];
for (var i = 0; i < data.length; i++) {
var rValue = data[i][6];
if (rValue == cellvalue) {
attendee.push(data[i]);
} else { //Fail Safe
attendee.push(data[i]);
}
}
if(attendee.length > 0){
sourceSpreadSheetSheetID1.getRange(sourceSpreadSheetSheetID1.getLastRow() + 1,
1, attendee.length, attendee[0].length).setValues(attendee);
}
//Will delete the rows of importdata once the data is copided to other
sheets
sourceSpreadSheetSheetID.deleteRows(2,
sourceSpreadSheetSheetID.getLastRow() - 1);
}
Try this:
I imagine that you already know that you'll need an installable onEdit Trigger and that you can't test a function of this nature by running it without the event object.
function checkIn(e) {
var sh=e.range.getSheet();
if(sh.getName()!="Event") return;
if(e.range.columnStart==6) {
if(e.value=="Attendee Arrived"){
e.range.offset(0,1).setValue(Utilities.formatDate(new Date(), Session.getScriptTimeZone(), "M/d/yyyy HH:mm:ss"));
var row=sh.getRange(e.range.rowStart,1,1,sh.getLastColumn()).getValues()[0];
e.source.getSheetByName("Attendee Arrived").appendRow(row);
sh.deleteRow(e.range.rowStart);
}
}
}

JS / GS Simple Google Sheets Script

Im trying to create a simple script that firstly checks (all cells in row 3 starting from column 3) for whether they contain a name different from the available sheets and if so create a new one. If not go to the next cell down. Preferably until the row is empty but I didnt get that far. Currently I have
var row = 3; //Global Variable
function Main() { // Main Function
Activate();
GetNames();
}
function Activate() { // Initialize
var get = SpreadsheetApp.getActiveSpreadsheet();
var import = get.getSheetByName("Import");
}
function GetNames() { // Get Names and check for existing Sheets
var ss = SpreadsheetApp.getActiveSpreadsheet();
var import = ss.getSheetByName("Import");
var sheet = ss.getSheets()[0];
for (i = 0; i < 1000; i++) { // Loop which I think is broken
var names = import.getRange(row,3).getValue();
if (import.getSheetByName(names) == null) {
import.insertSheet(names);
import.activate();
}
row + 1;
}
}
And here is the Data
It succeeds to add the first sheet but fails to continue in the loop I think.
As you will probably see I'm very new to this and any help would be appreciated.
You probably wanted to increase row by 1 here:
row + 1;
But you're not. row + 1 is just an expression with a value (4, in your example, because row remains 3 throughout). What you would need is the statement
row = row + 1;
But if this is all that you're using the global variable row for, you don't need it at all. Just use the loop variable i that's already counting from 0 to 1000. You probably want something like import.getRange(i+3,3).

Going from a google spreadsheet to a JS array

I have a google spreadsheet which i've made public and all and want to go from that to a JS array. I've been playing with it and trying to convert it into a JSON then into an array but i'm having no luck. Is there a way to go from my spreadsheet to a 2d JS array which I can then run functions on.
So if the spreadsheet looks like this:
NAME COLOR SIZE
MARK BLUE 6
DAVE RED 8
The array will be a 2d array such that value [0][0] will be MARK, [1][0] will be BLUE and [0][1] will be DAVE etc.
Any help would be much appreciated
Edit: since google API4 is our, and the module was updated accordingly, and since API3 will be discontinued on September, I updated the code accordingly.
I know this is old now, but in case any one else searching for an answer:
I played around with manipulating google spreadsheet in JS, and wanted to go from a worksheet with several sheets, to an array I can easily use in other functions.
So I wrote a program to access a Worksheet, get all of the sheets (tab) from within, reduce the all-empty rows and columns, and return the a new worksheet array.
Each element in the worksheet array represents the sheet from the Excel worksheet.
Also, each element has a 2d array of its cells, as such:
cellsArr:[[row],[col]]
To run the program, simply replace YOUR_CLIENT_SECRET.json to your client secret json file, and ENTER_GOOGLE_SPREADSHEET_ID_FROM_URL to the ID of the spreadsheet from the URL. For example:
docs.google.com/spreadsheets/d/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/edit#gid=01
and then run:
getWorksheet(ENTER_GOOGLE_SPREADSHEET_ID_FROM_URL);
This returns a promise, which is resolved into the revised worksheet (as described above).
Please note, reduceRowsCols() deletes empty rows and columns from each sheet. It deletes each row where the first column cell is empty, and every column where the first row cell is empty, so every row where [i][0]==null and every column where [0][i]==null.
Simply remove reduceRowsCols() from the code to negate that
const fs = require('fs');
const {GoogleSpreadsheet} = require('google-spreadsheet'),
creds = require('./client_secret.json'),
/**
* #Description Authenticate with the Google Spreadsheets API using credentials json.
* #param {string} spreadsheet_id - spreadsheet id to get. Get this from the URL of the spreadsheet
* Returns the entire raw worksheet.
* */
async function accessSpreadsheet(spreadsheet_id) {
const doc = new GoogleSpreadsheet(spreadsheet_id);
await doc.useServiceAccountAuth(creds);
await doc.loadInfo();
console.log(`${getCurrentTime()}: Loaded raw doc ${doc.title}`);
return doc;
}
/**
* #Description This is an async function to retrieve the raw worksheet
* #param {string} spreadsheet_id - spreadsheet id to get. Get this from the URL of the spreadsheet
* */
async function getWorksheet(spreadsheet_id) {
try {
let res = await accessSpreadsheet(spreadsheet_id);
console.log(`${getCurrentTime()}: Organizing sheets for extraction...`);
res = await getCondensedWorksheets(res.sheetsByIndex);
createSheetsCopy(res);
return res;
} catch (e) {
throw (e);
}
}
/**
* #param worksheetsArr
*/
async function getCondensedWorksheets(worksheetsArr) {
if (!Array.isArray(worksheetsArr)) {
throw `getCondensedWorksheets: worksheets variable passed is not an array. Object passed:\n${worksheetsArr}`;
}
let revisedArr = [];
for (let i = 0; i < worksheetsArr.length; i++) {
// for (let i = 0; i < 1; i++) {
revisedArr.push(await worksheetsArr[i].getRows().then((res)=>{
let thisRevised = {
id: worksheetsArr[i]._rawProperties.sheetId,
title: worksheetsArr[i].title,
rowCount: worksheetsArr[i].rowCount,
colCount: worksheetsArr[i].columnCount,
getRows: worksheetsArr[i].getRows,
getHeaders: worksheetsArr[i].headerValues,
getCells : res.map( row => {return(row._rawData);}),
resize: worksheetsArr[i].resize,
}
return getCells2dArray(thisRevised);
}))
}
return Promise.all(revisedArr).then(()=>{
return revisedArr;
})
}
/**
* #param {array} thisSheet - a single sheet (tab)
*/
function getCells2dArray(thisSheet) {
let sheetCellsArr = [];
sheetCellsArr.push(thisSheet.getHeaders);
thisSheet.getCells.map(row => {
sheetCellsArr.push(row);
})
thisSheet.cellsArr = sheetCellsArr;
delete thisSheet.getCells;
reduceRowsCols(thisSheet);
return thisSheet;
}
function reduceRowsCols(thisSheet) {
for (let i = 0; i < thisSheet.cellsArr.length; i++) {
if (thisSheet.cellsArr[i][0] == null) {
thisSheet.cellsArr.slice(0, i);
break;
}
}
for (let i = 0; i < thisSheet.cellsArr[0].length; i++) {
if (thisSheet.cellsArr[0][i] == null) {
thisSheet.cellsArr[0].slice(0, i);
break;
}
}
}
const getCurrentTime = function (){
var date = new Date();
return `${date.getHours()}:${date.getMinutes()}:${date.getSeconds()}:${date.getMilliseconds()}`
}
module.exports = {getWorksheet};
There are two methods in GAS that allow for reading values in cells. They are .getValue() and .getValues(). As it is implied, .getValue() retrieves only the value of the top left cell in a range. .getValues() retrieves a 2D array of all values in your range.
The code below defines the variables necessary to read the range that follows. The range is defined (based off your example) as starting at the second row (ignores your header row), continuing for 3 columns and down to the last row with content.
You'll notice that we find .getLastRow() but then define the number of rows to read as lastRow - 1. This is because .getLastRow() gives us the position of the last row with data; or more simply, an integer of how many rows have data. If we have 20 rows with data, that includes the header row which we would like to ignore. Reading 20 rows after shifting our range down a row will include the first empty row. Reading 19 rows instead stops after reading the last data-filled row.
Note that .getRange() requires the input of .getRange(startRow, startColumn, numRows, numColumns). This can be found with more detail and explanation in the GAS Reference sheet here
function myFunction() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSpreadsheet();
var lastRow = sheet.getLastRow();
var range = sheet.getRange(2, 1, (lastRow - 1), 3);
var dataArray = range.getValues();
}
Values in this array can then be called, defined, read, etc. as you would expect for Javascript. You will find in the GAS Reference that many methods have a singular and plural version such as .getBackground() and .getBackgrounds(). These will act in the same comparative way as .getValue() vs. .getValues().
It may also be helpful to note that even if your range is a singular row but many columns, .getValues() gives you a 2D array where all values will start with [0] as the only possible value for row is 0.

Categories

Resources