Develop Reference

Why is my code behaving like I have duplicate keys in a dictionary despite using unique strings? Javascript / Appscript

I am trying to loop through a dictionary of customers and save energy usage data, but for some customers when I try to change the values in their usage dictionary it will also change a completely different customer's value. I have a nested dictionary with customer utility information, the top-level key being a unique internal ID.
I stripped my code down to a single loop, looping through the top-level keys and setting the same month's usage for all customers in the dictionary to be the value of the iterator. After that, as shown in the code sample below, I log the values for three customers. After that, I increment only one of those customer's usage, and log the values again. The console shows that two over the customer's have dictionaries that are tied together somehow, but I can't figure out why or how to solve this. I can't discern any pattern in the keys of the linked customers, either.
Structure of the nested dictionary:
CustDict =
{"N0100000XXXXXX" =
{"name" = "XXXX"},
{"address" = "XXXX"},
{"meter_read_dates" =
{"2021-05-13" =
{"usage" = "XXXX"}
}
}
}
Stripped down code I used to demonstrate what is happening as simply as possible (real ID values):
Logger.log(custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000000419887"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000012580668"]["meter_read_dates"]["2021-05-13"]["usage"])
custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"] =
custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"] + 1
Logger.log(custDict["N01000009700816"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000000419887"]["meter_read_dates"]["2021-05-13"]["usage"])
Logger.log(custDict["N01000012580668"]["meter_read_dates"]["2021-05-13"]["usage"])
Console Output:
11:54:56 AM Info 346.0
11:54:56 AM Info 346.0
11:54:56 AM Info 322.0
11:54:56 AM Info 347.0
11:54:56 AM Info 347.0
11:54:56 AM Info 322.0
Code used to create the CustDict dictionary:
stmtCR = conn.prepareStatement('SELECT cust_id, utility_account, cycle_id, read_cycle FROM customers')
results = stmtCR.executeQuery()
resultsMetaData = results.getMetaData()
numCols = resultsMetaData.getColumnCount();
results.last();
numRows = results.getRow();
i = 0
results.first()
var custDict = {}
while (i < numRows)
{
custDict[results.getString(1)] = {}
custDict[results.getString(1)]["id"] = results.getString(1)
custDict[results.getString(1)]["utility_account"] = results.getString(2)
custDict[results.getString(1)]["cycle_id"] = results.getString(3)
custDict[results.getString(1)]["read_cycle"] = results.getString(4)
results.next()
i++;
}
for (i = 0; i < Object.keys(custDict).length; i++)
{
tempCust = custDict[Object.keys(custDict)[i]]
tempCycleId = tempCust["cycle_id"]
tempReadCycle = tempCust["read_cycle"]
tempCust["meter_read_dates"] = cycleIdShdDict[tempCycleId][tempReadCycle]
custDict[Object.keys(custDict)[i]] = tempCust
}
cycleIdShdDict is a seperate dictionary that contains a set of dates associated with each cycle_id and read_cycle

I suspect the problem is that Object.keys(custDict) is returning the keys in a different order at different places in the for loop. So you're getting the object from one key, and then assigning it to a different key.
There's no need to assign back to custDict[Object.keys(custDict)[i]] since you're modifying the object in place, not a copy.
But instead of looping through the keys, loop through the values and modify them.
Object.values(custDict).forEach(tempCust => {
let tempCycleId = tempCust["cycle_id"];
let tempReadCycle = tempCust["read_cycle"];
tempCust["meter_read_dates"] = cycleIdShdDict[tempCycleId][tempReadCycle];
});

d3.js pass user select input value as csv column name in d3.csvParse callback

I am working on this visualization at
https://observablehq.com/d/88fa7afb1d32ebc8
below is the spot that I am trying to flesh out with user input from a drop down menu
dataRaw = Object.assign(
new Map(d3.csvParse(await FileAttachment("DATA.csv").text(),
//want to make csvColumnName on line below dynamic
({objKey, csvColumnName}) => [objKey, +csvColumnName])),
{title: " "+SelectOptions.filter( (stat) => stat.value.key === mySelect.key)[0].label})
I want the csv to be parsed based on what the user selects.
I can get csv column names back from the select, but as strings.
Below is what I have tried
mySelect.key = "userInputCsvCol1Name";
....
d3.csvParse(await FileAttachment("covid19-dec16#4.csv").text(),
({objKey, csvColumnName}) => [objKey, +(mySelect.key)] //<-------can't be a string
)
I am pretty new to Javascript, is there a way to rewrite this so I can build the object dynamically?

I solved this by going step by step in the notebook and breaking up the function
first I got the file and parsed the csv
x = await FileAttachment("covid19-dec16#4.csv").text()
y = d3.csvParse(x)
then I created 2 arrays, with the second being the user input from a select dropdown and finally iterated over them to create the final map to use in the chart
map1 = y.map(d=> d["objKey"] )
map2 = y.map(d=> +d[mySelect.key])
let obj =new Map();
let k
for (let i = 0; i < map1 .length; i++) {
obj.set(yy[i], xx[i]);
}
see it in action here thanks
https://observablehq.com/#josephhailu/untitled/3

How to compare two sheets and delete/add any column with a distinct value in row 1? Google Script

I want to compare two sheets (based on header values in row 1) and delete any column with a unique value (without a match). For example, Assuming Sheet1, Row 1 data and Sheet 2, Row 1 are uniform, if a user adds/deletes a column within any sheet, I want to always match the number of columns in both sheets with their values
Screenshots of sheets headings.
IF both sheets looks like this
And a user adds a new Column N
Or delete column N
How can I ensure that both sheet matches by deleting the odd/distinct column in Sheet 1?
I have tried modifying this code below but I can't just get the unique one out. This code only look for headers with a defined value.
function deleteAloneColumns(){
var sheet = SpreadsheetApp.getActiveSheet();
var lastColumnPos = sheet.getLastColumn();
var headers = sheet.getRange( 1 ,1, 1, lastColumnPos ).getValues()[0];
for( var i = lastColumnPos ; i < 1; i--){
if( headers[i] === "alone" ) sheet.deleteColumn(i);
}
SpreadsheetApp.getUi().alert( 'Job done!' );
}
Any help to compare and delete the column with the unique value will be appreciated.

Problem
Balancing sheets based on header row values mismatch.
Solution
If I understood you correctly, you have a source sheet against which validation is run and two primary use cases: user adds a new column named differently than any other column (if you want to check that the column strictly matches the one in sheet1, it is easy to modify) in source sheet or deletes one that should be there.
const balanceSheets = (sourceShName = 'Sheet1',targetShName = 'Sheet2') => {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const s1 = ss.getSheetByName(sourceShName);
const s2 = ss.getSheetByName(targetShName);
const s2lcol = s2.getLastColumn();
//keep all vals from source to reduce I/O
const s1DataVals = s1.getDataRange().getValues();
const s2Vals = s2.getRange(1, 1, 1, s2lcol).getValues();
const h1Vals = s1DataVals[0];
const h2Vals = s2Vals[0];
//assume s1 is source (validation) sheet
//assume s2 is target sheet that a user can edit
//case 1: target has value not present in source -> delete column in target
let colIdx = 0;
h2Vals.forEach(value => {
const isOK = h1Vals.some(val => val===value);
isOK ? colIdx++ : s2.deleteColumn(colIdx+1);
});
//case 2: target does not have values present in source -> append column from source
h1Vals.forEach((value,index) => {
const isOK = h2Vals.some(val => val===value);
!isOK && s2.insertColumnAfter(index);
const valuesToInsert = s1DataVals.map(row => [row[index]]);
const numRowsToInsert = valuesToInsert.length;
s2.getRange(1,index+1, numRowsToInsert,1).setValues(valuesToInsert);
});
};
Showcase
Here is a small demo of how it works as a macros:
Notes
Solving your problem with two forEach is suboptimal, but I kept number of I/O low (it can be lowered further by, for example, moving deleteColum out of the loop while only keeping track of column indices).
The script uses ES6 capabilities provided by V8, so please, be careful (although I would recommend migrating as soon as possible - even if you encounter bugs / inconsistencies , it is worth more than it costs.
UPD made script more flexible by moving sheet names to parameter list.
UPD2 after discussing the issue with deleteColumn() behaviour, the answer is updated to keep column pointer in bounds (for those curious about it - forEach kept incrementing the index, while deleteColumn reduced bounds for any given index).
Reference
insertColumnAfter() method reference

How can I parse a .frd file with Javascript?

A .frd file is a type of multi-column numeric data table used for storing information about the frequency response of speakers. A .frd file looks something like this when opened in a text editor:
2210.4492 89.1 -157.7
2216.3086 88.99 -157.7
2222.168 88.88 -157.6
2228.0273 88.77 -157.4
Using javascript, is there a way that I can parse this data in order to return each column separately?
For example, from the .frd file above, I would need to return the values like so:
var column1 = [2210.4492, 2216.3086, 2222.168, 2228.0273];
var column2 = [89.1, 88.99, 88.88, 88.77];
var column3 = [-157.7, -157.7, -157.6, -157.4];
I'm not exactly sure where to begin in trying to achieve this, so any step in the right direction would be helpful!

I found the following description of the FRD file format and I will follow it.
Let's assume that the content of your .frd file is in the variable called content (the following example is for Node.js):
const fs = require('fs');
const content = fs.readFileSync('./input.frd').toString();
Now if content has your FRD data, it means it's a set of lines, each line contains exactly three numbers: a frequency (Hz), a level (dB), and a phase (degrees). To split your content into lines, we can just literally split it:
const lines = content.split(/\r?\n/);
(normally, splitting just by '\n' would've worked, but let's explicitly support Windows-style line breaks \r\n just in case. The /\r?\n/ is a regular expression that says "maybe \r, then \n")
To parse each line into three numbers, we can do this:
const values = line.split(/\s+/);
If the file can contain empty lines, it may make sense to double check that the line has exactly three values:
if (values.length !== 3) {
// skip this line
}
Given that we have three values in values, as strings, we can assign the corresponding variables:
const [frequency, level, phase] = values.map(value => Number(value));
(.map converts all the values in values from strings to Number - let's do this to make sure we store the correct type).
Now putting all those pieces together:
const fs = require('fs');
const content = fs.readFileSync('./input.frd').toString();
const frequencies = [];
const levels = [];
const phases = [];
const lines = content.split(/\r?\n/);
for (const line of lines) {
const values = line.split(/\s+/);
if (values.length !== 3) {
continue;
}
const [frequency, level, phase] = values.map(value => Number(value));
frequencies.push(frequency);
levels.push(level);
phases.push(phase);
}
console.log(frequencies);
console.log(levels);
console.log(phases);
The main code (the one that works with content) will also work in browser, not just in Node.js, if you need that.
This code can be written in a tons of different ways, but I tried to make it easier to explain so did something very straightforward.
To use it in Node.js (if your JavaScript file is called index.js):
$ cat input.frd
2210.4492 89.1 -157.7
2216.3086 88.99 -157.7
2222.168 88.88 -157.6
2228.0273 88.77 -157.4
$ node index.js
[ 2210.4492, 2216.3086, 2222.168, 2228.0273 ]
[ 89.1, 88.99, 88.88, 88.77 ]
[ -157.7, -157.7, -157.6, -157.4 ]

JS and ExpressionEngine - Remove KV pairs by duplicate values only?

We're building a site with ExpressionEngine. We are running a SQL query to gather up all member IDs for a specific member group. After that, we are using EE tags to get data from a custom member field for each member ID.
The ID and field data need to stay paired, as we will be populating a drop-down so that the ID is the value and the field data is the text, so we are currently putting them into a JS array as key/value pairs. The call is as follows:
var array= [
{exp:query sql="SELECT * FROM exp_members WHERE group_id = 5"}
{exp:member:custom_profile_data
member_id="{member_id}"}
{if company != ''}
{{member_id}:"{company}"},
{/if}
{/exp:member:custom_profile_data}
{/exp:query}
};
This gives us the output:
var array = [
{1:"name01"},
{2:"name02"},
{3:"name01"},
{4:"name03"}
];
Now, our problem. We need to remove objects based on duplicate field data (values) only, so the above array would look like this:
var array = [
{1:"name01"},
{2:"name02"},
{4:"name03"}
];
None of these IDs (keys) will ever be the same, but the field data (values) can be. So we want to keep the first KV pair that comes through with a unique value, but remove any subsequent dupes of that value - despite the fact that they will not be true "duplicate values" due to a different ID (key).
Keeping in mind that the KV pairs are all dynamic, is there any possible way to do this via JS so we can create a new array for the cleaned data to pass to the drop-down?

You could handle the duplications by modifying your MySQL query. (In my example, my custom field ID was 1.)
var myArray = [];
{exp:query sql="SELECT MIN(m.member_id) AS co_member_id, d.m_field_id_1 AS company FROM exp_members m INNER JOIN exp_member_data d ON m.member_id = d.member_id WHERE d.m_field_id_1 != '' AND m.group_id > 0 GROUP BY d.m_field_id_1;"}
myArray.push({{co_member_id}: "{company}"});
{/exp:query}
This query would use the first (in the ordinal sense) member_id found; you could also change the MIN to MAX and get the last.
This will give you a clean output in your source, without the need for any additional JS processing. I'd also recommend changing the names of the variables you're outputting as to not conflict in EE's parsing.

I would do it like...
function removeDups(arry){
var tmp = {}, retainIdx=[], newArry=[];
arry.forEach(function(obj, idx){
var val = obj[Object.keys(obj)[0]];
if(val && !tmp[val]){
retainIdx.push(idx);
tmp[val] = true;
}
});
retainIdx.forEach(function(i){
newArry.push(arry[i]);
});
return newArry;
};