I've been writing a script to copy some data from an input sheet to a database to keep track of some data. I've managed to successfully write the code for linear arrays (only one row) but when I try to copy an entire 15x15 cells range I get an error stating the parameters are not correct (suggesting the dimension of the arrays are not correct) but I can't seem to understand why.
I tried both copying directly the entire 15x15 range and creating a for loop to copy row by row 15 times but I can't mangage to make it work.
Here is the main structure:
// active spreadsheet and source + target sheets
const activeSheet = SpreadsheetApp.getActiveSpreadsheet();
const srcSheet = activeSheet.getSheetByName('Data Entry');
const dstTOTSheet = activeSheet.getSheetByName('DataBaseTOT');
var firstEmptyRowTOT = dstTOTSheet.getLastRow()+1;
For loop test:
for (var i=0; i=14;i=i+1) {
// source cells
var RoundInfo = srcSheet.getRange(10+i, 2, 1, 15); // 15x15 B10:P24
// target cells
var dstTOTRoundInfo = dstTOTSheet.getRange(firstEmptyRowTOT + i, 21, 1, 15); // I am starting from column 21 because of some other data
// set value
dstTOTRoundInfo.setValues(RoundInfo);
}
Direct 15x15 test:
// source cells
var RoundInfo = srcSheet.getRange("B10:P24"); // 15x15 B10:P24
// target cells
var dstTOTRoundInfo = dstTOTSheet.getRange(firstEmptyRowTOT, 21, 15, 15);
// set value
dstTOTRoundInfo.setValues(RoundInfo);
It is actually not that difficult to understand the 2D array concept for spreadsheet, it works like this:
// inner arrary is always one row of values:
// the 1st value (array index: 0) of the inner-array is always the cell value of the 1st column of the given row.
const row_1 = ['row_value_1-1','row_value_1-2','row_value_1-3','row_value_1-4'];
const row_2 = ['row_value_2-1','row_value_2-2','row_value_2-3','row_value_2-4'];
const row_3 = ['row_value_3-1','row_value_3-2','row_value_3-3','row_value_3-4'];
// the outter-array is always the container of every rows you have in range,
// the leangth of rows (which is the leangth of each inner-array) must be the same for this structure to work in spreadsheet.
const data = [
row_1, // the 1st value (array index: 0) of the outter-array is always the 1st row,
row_2, // the 2nd value (array index: 1) of the outter-array is always the 2nd row, etc.
row_3
];
// when you loop over such kind of 2D array, it is easier to understand with a "for... of" loop than the classic "for" loop:
for (const row of data) { // << this means "for each row of the given data"
for (const col of row) { // << this means "for each column of the given row"
// this structure will go through each cell of row 1, than each cell of row 2, etc., until all rows of the given data are iterated.
console.log(col);
}
}
/**output:
row_value_1-1
row_value_1-2
row_value_1-3
row_value_1-4
row_value_2-1
row_value_2-2
row_value_2-3
row_value_2-4
row_value_3-1
row_value_3-2
row_value_3-3
row_value_3-4
*/
// and if you need to works with the indexes of some cell, you can get the index by this:
for (const [rIndex,row] of data.entries()) {
for (const [cIndex,col] of row.entries()) {
console.log(rIndex,cIndex,col);
}
}
/** output:
0 0 row_value_1-1
0 1 row_value_1-2
0 2 row_value_1-3
0 3 row_value_1-4
1 0 row_value_2-1
1 1 row_value_2-2
1 2 row_value_2-3
1 3 row_value_2-4
2 0 row_value_3-1
2 1 row_value_3-2
2 2 row_value_3-3
2 3 row_value_3-4
*/
I'm trying to find the most efficient way of breaking a two-dimensional array into different two-dimensional pieces, all stored into a new array which will be returned as the result.
I have a 2D array consisting of a string "N M" in the cell at row N and column M. For example, the value in row 2, column 3 would be "2 3".
Now given some rows and column indexes, I want to remove those rows and columns, and group each "piece" into its own 2D array. A "piece" is a set of orthogonally connected cells, without any intervening cells from the rows and columns removed above.
e.g. if I have an array with 8 rows and 7 columns, and I try to remove the rows [2, 3, 5], and columns [2, 6], the resulting array of "pieces" should be
[
[["0, 0", "0, 1"], ["1, 0", "1, 1"]],
[["0, 3", "0, 4", "0, 5"], ["1, 3", "1, 4", "1, 5"]],
[["4, 0", "4, 1"]],
[["4, 3", "4, 4", "4, 5"]],
[["6, 0", "6, 1"], ["7, 0", "7, 1"]],
[["6, 3", "6, 4", "6, 5"], ["7, 3", "7, 4", "7, 5"]]
]
In case the description is unclear, I went ahead and made a visual example of the process in mind here to detail the steps, problems and goals involved.
This is what I was able to come up with, but it's producing arrays that include exception columns. I know why, but I don't know how to get around it short of adding extra complexity which is not conducive to efficiency.
const SourceArray = [ ['R0·C0', 'R0·C1', 'R0·C2', 'R0·C3', 'R0·C4', 'R0·C5', 'R0·C6'],
['R1·C0', 'R1·C1', 'R1·C2', 'R1·C3', 'R1·C4', 'R1·C5', 'R1·C6'],
['R2·C0', 'R2·C1', 'R2·C2', 'R2·C3', 'R2·C4', 'R2·C5', 'R2·C6'],
['R3·C0', 'R3·C1', 'R3·C2', 'R3·C3', 'R3·C4', 'R3·C5', 'R3·C6'],
['R4·C0', 'R4·C1', 'R4·C2', 'R4·C3', 'R4·C4', 'R4·C5', 'R4·C6'],
['R5·C0', 'R5·C1', 'R5·C2', 'R5·C3', 'R5·C4', 'R5·C5', 'R5·C6'],
['R6·C0', 'R6·C1', 'R6·C2', 'R6·C3', 'R6·C4', 'R6·C5', 'R6·C6'],
['R7·C0', 'R7·C1', 'R7·C2', 'R7·C3', 'R7·C4', 'R7·C5', 'R7·C6'] ];
function split2DArray_test() {
let arrObjs = {
oldArr: {
obj: SourceArray,
bounds: { row: 8, col: 7, },
setBounds: function() {
this.bounds.row = this.obj.length;
this.bounds.col = this.obj[0].length;
},
},
newArr: { collection: [], component: [], design: { rInit: 0, rEnd: 0, cInit: 0, cEnd: 0 } },
splits: { atRows: [2, 3, 5], atCols: [2, 5] }
};
arrObjs.oldArr.setBounds();
let i = { lv1_R: 0, lv2_C: 0 , lv3_R: 0, lv4_C: 0, slicer: 0 }; let escape = false;
/*
1. Iterate through rows. (1st level)
2. If row excepted, continue iterating through rows. If not excepted, iterate through columns. (2nd level)
3. If column not excepted, RECORD ADDRESS and continue iterating through column.
4. When excepted column found, RECORD COLUMN - 1, and regin iterating through rows again (3rd level). When excepted row found, RECORD ROW - 1.
5. Record data (SliceArr): [RECORD ADDRESS[R] = RowA, RECORD ADDRESS[C] = ColA, RECORD ROW - 1 = RowB, RECORD COLUMN - 1 = ColB]
6. Create NewArr[] and SliceArrObj[]
7. Iterate through rows (r) from new SliceArr (4th level), and do SliceArrObj.push(OldArr[r].slice(ColA, ColB - ColA))
8. NewArr.push(SliceArrObj)
9. Set 2nd-level iterator to RECORD COLUMN + 1
10. Loop should work theoretically. Untested.
*/
Logger.log(`Excepting rows: ${arrObjs.splits.atRows.join(', ')}`);
Logger.log(`Excepting columns: ${arrObjs.splits.atCols.join(', ')}`);
console.time('slicer');
for (i.lv1_R = 0; i.lv1_R < arrObjs.oldArr.bounds.row; i.lv1_R++) {
if (arrObjs.splits.atRows.includes(i.lv1_R)) { continue; } else {
// We've arrived at a non-excepted row.
for (i.lv2_C = 0; i.lv2_C < arrObjs.oldArr.bounds.col; i.lv2_C++) {
if (arrObjs.splits.atCols.includes(i.lv2_C)) { continue; } else {
arrObjs.newArr.design.rInit = i.lv1_R; arrObjs.newArr.design.cInit = i.lv2_C;
Logger.log(`Found origin cell '${arrObjs.oldArr.obj[i.lv1_R][i.lv2_C]}'`);
for (i.lv3_R = arrObjs.newArr.design.rInit; i.lv3_R < arrObjs.oldArr.bounds.row; i.lv3_R++) {
if (arrObjs.splits.atRows.includes(i.lv3_R)) {
arrObjs.newArr.design.rEnd = i.lv3_R - 1;
for (i.lv4_C = arrObjs.newArr.design.cInit; i.lv4_C < arrObjs.oldArr.bounds.col; i.lv4_C++) {
if (arrObjs.splits.atCols.includes(i.lv4_C)) {
arrObjs.newArr.design.cEnd = i.lv4_C - 1;
for (i.slicer = arrObjs.newArr.design.rInit; i.slicer < arrObjs.newArr.design.rEnd + 1; i.slicer++) {
arrObjs.newArr.component.push([arrObjs.oldArr.obj[i.slicer].slice(arrObjs.newArr.design.cInit, arrObjs.newArr.design.cEnd + 1)]);
};
arrObjs.newArr.collection.push('Split'); // For the sake of output logging.
arrObjs.newArr.collection.push(arrObjs.newArr.component);
arrObjs.newArr.component = [];
i.lv2_C += 1 + arrObjs.newArr.design.cEnd - arrObjs.newArr.design.cInit;
arrObjs.newArr.design.rInit = 0; arrObjs.newArr.design.rEnd = 0; arrObjs.newArr.design.cInit = 0; arrObjs.newArr.design.cEnd = 0;
escape = true; break;
};
};
};
if (escape) { escape = false; break; };
};
};
};
i.lv2_R += 1 + arrObjs.newArr.design.rEnd - arrObjs.newArr.design.rInit;
};
};
console.timeEnd('slicer');
Logger.log(`Splitting 2D Array into:\n\n${(arrObjs.newArr.collection.join('\n\n'))}`); // The output logging.
/* Idea Space:
===== PROPOSED METHOD, INCOMPLETE =====
1. Save the origin cell of the first valid slice bounds.
2. Iterate (FOR) through columns, starting at the Bound Minimum, and going to the Bound Maximum.
3. If an exception row is found, save slice col bound max as (i.c - 1).
4. Now, iterate through rows from slice origin to next exception or bound. Once this is met, save slice row bound max as (i.r - 1).
5. Iterate through slice row bounds, pushing slices with slice column bounds as Array.slice() arguments.
6. Push new 2D array to new array collection.
=== CONCEPTS WORTH TESTING ===
I. Iterative Cell Search
Consider moving forward by setting exception or bound limits. Ex: Column iteration runs into a bound limit: slices.resume.c = false. If it's an exception, slices.resume.c = true. This way,
any time a full 2D slice is pushed to the collection of arrays, it can take the existing current slice data and either iterate across to the next horizontal array (Resume = true), or
iterate down to the next vertical array (Resume = false). If both Column and Row resume are false, then we are done adding sliced 2D arrays to the array collection.
If at least one Resume is true, then another resume can be made true. Once both are true, then a new slice origin point can be made.
II. Iterative Bounds & Exception Search.
1. Table Bounds defined and adjacent exceptions trimmed (Done).
> The current exceptions are now the top-left-side borders of the desired slices.
2. Iterate through exception indices column first (On first loop, operate from top-left bound), then row. On each, set slice start to (i.r + 1, i.c + 1), where i.? = the exception column or
row. Set the end bounds to the index represented by the next element in the list (column or row).
*/
};
In the approach I take below, I split up the problem in two main steps:
const ranges = (splits, length) => [...splits, length].reduce(
(r, e, i, a) => (s = i ? a[i - 1] + 1 : 0, e - s ? [...r, {s, e}] : r),
[]
);
const blocks = (array, rowRanges, colRanges) => rowRanges.map(
({s, e}) => array.slice(s, e)
).map(block => colRanges.map(
({s, e}) => block.map(row => row.slice(s,e))
)).flat();
The ranges() function takes an input array containing "splits" information (i.e. rowSplits or colSplits), and then uses that information to create an array of ranges for the row blocks or column blocks. Each range is defined as an object of type {s: number, e: number}, where s is the start of the range (inclusive) and e is the end (exclusive). The reduce operation used in this function makes it so that empty ranges are filtered out.
The blocks() function takes the 2D array and the row and column ranges as input. It first uses the row ranges to split the 2D array into row blocks, and then uses the column ranges to split those blocks down to the regions defined in the problem statement. The splitting operations are all implemented by mapping the ranges to their corresponding array slices. Finally, the entire result is flattened to arrive at the required output.
Complete snippet:
const rows = 8;
const cols = 7;
const arrOld = Array.from(
{ length: rows}, (_, y) => Array.from(
{ length: cols }, (_, x) => `${y}, ${x}`
)
);
const rowSplits = [2, 3, 5];
const colSplits = [2, 6];
const ranges = (splits, length) => [...splits, length].reduce(
(r, e, i, a) => (s = i ? a[i - 1] + 1 : 0, e - s ? [...r, {s, e}] : r),
[]
);
const blocks = (array, rowRanges, colRanges) => rowRanges.map(
({s, e}) => array.slice(s, e)
).map(block => colRanges.map(
({s, e}) => block.map(row => row.slice(s,e))
)).flat();
const arrNew = blocks(
arrOld,
ranges(rowSplits, rows),
ranges(colSplits, cols)
);
console.log(arrNew);
I am currently attempting to calculate the growth percentage change of the properties in an object. I have the calculation that I am able to successfully do with an array of integers, but I am having a hard time applying the same logic to the properties of an object.
My expected outcome is to perform a calculation to data similar to this: year2011-year2010 /year2010; year2012-year2011 /year2011 etc.. in an array such as this:
calculationData = [14.9274 , -0.1875, 0.6122]
Here is an example of a manual way of solving this:
let calc2010 = data.year2011 - data.year2010 / data.year2010
let calc2011 = data.year2011 - data.year2012 / data.year2011
calculationData.push(calc2010, calc2011)
const data = [{'year2010': 4323, 'year2011': 64532, 'year2012': 52432, 'year2013': 84532, 'year2014': 63232, 'year2015': 49332 }]
How can I accomplish this?
In the code snippet below; I provided two examples First, is the properties of objects I am attempting to calculate, and second is showing the calculations working for an array of integers
const data = [{'year2010': 4323, 'year2011': 64532, 'year2012': 52432, 'year2013': 84532, 'year2014': 63232, 'year2015': 49332 }]
let calculationData = data.map(function (stPer, i) {
return 100 * (stPer - data[i - 1]) / (data[i - 1]);
});
console.log(calculationData)
const dataArr = [ [312], [4342], [4343] ]
let calculationDataTwo = dataArr.map(function (stPer, i) {
return 100 * (stPer - dataArr[i - 1]) / (dataArr[i - 1]);
});
console.log(calculationDataTwo)
Here is a solution which converts your object into an array of data in ascending order by year.
const data = {'year2010': 4323, 'year2011': 64532, 'year2012': 52432, 'year2013': 84532, 'year2014': 63232, 'year2015': 49332 };
const calculationData = Object.entries(data).sort(([year1], [year2]) => {
return Number(year1.slice(4)) - Number(year2.slice(4));
}).map(([year, value]) => value);
This is using the Object.entries function to create an array which looks like: [['year2010', 4323], ['year2011', 64532], etc...]
Next, we can sort it and map it easily. Sorting may not be needed if you guarantee the data is pre-sorted, in which case you only need to use map.
The result is an array like this: [4323, 64532, 52432, 84532, 63232, 49332]
You should be able to plug that into your existing code.
EDIT: I see there is still some additional processing to do after you get to this point. Do you need any help after this point?
You need to put each year in a different array element, not all years in the same object.
And when you use map, you need to start with array index 1, because there's no previous element before index 0 to compare with. You can do this by mapping over a slice.
const data = [{
year: 2010,
value: 4323
},
{
year: 2011,
value: 64532
},
{
year: 2012,
value: 52432
},
{
year: 2013,
value: 84532
},
{
year: 2014,
value: 63232
},
{
year: 2015,
value: 49332
}
]
let calculationData = data.slice(1).map(function(stPer, i) {
return 100 * (stPer.value - data[i].value) / data[i].value;
});
console.log(calculationData)
Presentation :
I am working on a piece of code that can compare two javascript Object by looping into the first one (called A) to perform a key lookup in the second one called B (I put value as key and occurence as value).
But when I am measuring the performance of the subkey key lookup of Object A (10 times per amount of data, with data as changing parameters for each 10 times the program runs (100 per row, 200...) I get high timing for the smallest amount of data (so potentially less key in the dict B)
Objects layout :
Object A looks like below:
{
SQL_TABLE_1:
{
column_1:
[
'2jmj7l5rSfeb/vlWAYkK/YBwIDk=',
'3MaRDFGBKvsLLhrLUdplz3wUiOI=',
'PEvUFHDR4HbOYXcj7danOvyRJqs='
'XHvERAKZ4AqU+iWlx2scZXdem80=',
'nSG0lvwlkIe5YxZGTo5binr3pAw=',
'Swuc/7YCU9Ptfrff+KHaJJ1+b7U=',
'N28qqdfezfZbPmK7CaGmj7m7IOQ=',
'ffc7skeffegT1ZytAqjco3EpwUE=',
'2XldayBefzBxsUuu6tMFYHVZEEY=',
'5rC2y6IzadQ1aEy7CvNyr30JJ2k='
]
},
SQL_TABLE_2:
{
column_1:[......]
}
}
Object B field have various size but this size never change in our tests
And Object B looks like:
[
field_1:
{
'2jmj7l5rSfeb/vlWAYkK/YBwIDk=': 1,
'3MaRDFGBKvsLLhrLUdplz3wUiOI=': 1,
'PEvUFHDR4HbOYXcj7danOvyRJqs=': 1,
'XHvERAKZ4AqU+iWlx2scZXdem80=': 4,
'nSG0lvwlkIe5YxZGTo5binr3pAw=': 1,
'Swuc/7YCU9Ptfrff+KHaJJ1+b7U=': 1,
'N28qqdfezfZbPmK7CaGmj7m7IOQ=': 27,
'ffc7skeffegT1ZytAqjco3EpwUE=': 1,
'2XldayBefzBxsUuu6tMFYHVZEEY=': 18,
'5rC2y6IzadQ1aEy7CvNyr30JJ2k=': 1 },
field_2:{......}
]
Timing measurement in the code is structured like this:
sql_field_1:
{
mongo_field_1: 0.003269665241241455, mongo_field_2: 0.0015446391105651855, mongo_field_3: 0.0009834918975830079, mongo_field_4: 0.0004488091468811035,
},
sql_field_2:
{
....
}
Goal
The goal is to perform for each sub-subkey of Object A a key lookup on the Object B subkeys.
Code
Here's the code that cause this behavior:
Object A is called sql_dict_array
Object B is called hash_mongo_dict
for(var field_name in hash_mongo_dict)
{
performance_by_mongo_field[field_name] = {};
result_by_mongo_field[field_name] = {};
// LOOP ON OBJECT A KEYS
for(var table_name in sql_dict_array)
{
// Start of time measurement
var start_time = performance.now();
// there is only one column in our test data
for(var column_name in sql_dict_array[table_name])
{
found_count = 0;
for(var value of sql_dict_array[table_name][column_name])
{
// **SUBKEY LOOPKUP HERE WITH VALUE**
var results = hash_mongo_dict[field_name][value];
// IF NOT UNDEFINED THEN IT HAS BEEN FOUND
// THIS CHECK IS NOT THE BOTTLENECK
if(results != undefined)
{
found_count+=results;
}
}
if(found_count > limit_parameter)
{
console.log("error: too many match between hashes")
process.exit(0)
}
// PERFORMANCE CALCULATION
performance_by_mongo_field[field_name][table_name] = (performance.now() - start_time)/1000;
result_by_mongo_field[field_name][table_name+'.'+column_name] = (found_count/verif_hash_count*100);
}
}
return some results...
}
Testing:
With this code, I expect to have almost constant time whatever the size of the Object B (amount of subkey) but in my code I have higher time when I have only 10 subkeys in the nested object A, and it become stable when reaching 100 keys or more (tested with 6000 keys)
Here's 10 runs for the key lookup code of one key of Object A containing 10 subkeys with 300.000+ data from Object B:
0.2824700818061829 0.2532380700111389 0.2455208191871643 0.2610406551361084 0.2840422649383545 0.2344329071044922 0.2375670108795166 0.23545906591415405 0.23111085414886476 0.2363566837310791
Here's the same comparison but with 4000+ subkeys:
0.0027927708625793456 0.0018292622566223144 0.015235211849212647 0.0036304402351379395 0.002919149875640869 0.004972007751464844 0.014655702114105225 0.003572652339935303 0.0032280778884887697 0.003232938766479492
I will appreciate every advice you can provide me,
I am using JavaScript for this.
This is what I have currently in my array
0 "mariam"
1 "jibrin"
2 "maths"
3 "Primary_1"
4 "period_3"
5 "Monday"
6 "subject=maths?name=mariam&jibrin "
7 "zainab"
8 "usman"
9 "maths"
10 "Nursery_2"
11 "period_3"
12 "Wednesday"
13 "subject=maths?name=zainab&usman "
Snapshot for better view can be found here
What I want to do is loop through this array split it where there is a value containing "subject=" and move the all the values up to that point into a new array.
So everything from index 0 to 6 will be moves to a new array from here since the value at index 6 will match the condition.
This will apply for the rest.
So this is the code I have so far:
var your_object = JSON.parse(response); // here I will have a JSON
var splitter = String(your_object).split(",");
var returnArray = [];
var tmp = [];
$(splitter).each(function (i,v) {
console.log(i,v);
if (v.includes("subject=")) {
//console.log("FOUND", i);
}
});
I am not sure how to go about moving it into a new array and dynamically handle this. Json at the minute only passed a full total size of 14 however it can be higher later on.
If a condition matches (i.e. value contains "subject=" then everything from the index at the start at the point where it matched needs moving to a new array which have to be dynamic.
I know how to remove from the current array but not sure how to dynamically move it into a new one.
If I did not made myself clear please ask and thank You
You could check if the item contains the subject substring and change the array for the result.
var data = ["mariam", "jibrin", "maths", "Primary_1", "period_3", "Monday", "subject=maths?name=mariam&jibrin ", "zainab", "usman", "maths", "Nursery_2", "period_3", "Wednesday", "subject=maths?name=zainab&usman "],
result = data.reduce(function (r, a, i, aa) {
r[r.length - 1].push(a);
if (a.slice(0, 8) === 'subject=' && i + 1 < aa.length) {
r.push([]);
}
return r;
}, [[]]);
console.log(result);
.as-console-wrapper { max-height: 100% !important; top: 0; }