I have a google sheet with several named ranges.
I would like to import every single named range in the sheet and save them as individual objects named after the named range.
Furthermore, there are 4 similar blocks of data. Therefore the names of the ranges are structured like "block1_name1" "block1_name2" "block2_name1" "block2_name2" etc.
With the following code I can enter every range manually, but there are too many to enter them all manually:
const API_KEY = "###"; // Please set your API key.
const ID = "###"; // Please set your Spreadsheet ID.
const RANGE = ["Range1", "Range2"]; // Named ranges
const ranges = RANGE.map(e => `ranges=${encodeURIComponent(e)}`).join("&");
const response = await fetch(`https://sheets.googleapis.com/v4/spreadsheets/${ID}/values:batchGet?key=${API_KEY}&${ranges}`);
const { valueRanges } = await response.json();
const obj = valueRanges.reduce((o, { values }, i) => (o[RANGE[i]] = values, o), {});
console.log(obj);
How can I import every named range automatically?
How can I save them as a different objects for each datablock like block1.name1 etc?
I believe your goal is as follows.
You want to retrieve the named range list from Google Spreadsheet, and want to retrieve the values from the named ranges.
And, you want to export the values as {namedRange1: values1, namedRange2: values2,,,}.
You want to achieve this by directly requesting the endpoint of Sheets API with fetch API of Javascript.
In this case, how about the following modification?
Modified script:
const API_KEY = "###"; // Please set your API key.
const ID = "###"; // Please set your Spreadsheet ID.
// 1. Retrieve the named range list.
const base = `https:\/\/sheets.googleapis.com/v4/spreadsheets/${ID}`;
const res1 = await fetch(`${base}?key=${API_KEY}&fields=namedRanges(name)`);
const { namedRanges } = await res1.json();
// 2. Retrieve values from named ranges.
const ranges = namedRanges.map(({ name }) => `ranges=${encodeURIComponent(name)}`).join("&");
const res2 = await fetch(`${base}/values:batchGet?key=${API_KEY}&${ranges}`);
const { valueRanges } = await res2.json();
// 3. Create an output object.
const res = valueRanges.reduce((o, { values }, i) => (o[namedRanges[i].name] = values, o), {});
console.log(res);
// For your 2nd question.
const res3 = valueRanges.reduce((o, { values }, i) => {
const [k1, k2] = namedRanges[i].name.split("_");
if (o[k1]) {
o[k1][k2] = values;
} else {
o[k1] = { [k2]: values };
}
return o;
}, {});
console.log(res3);
Testing:
When this script is run, the following result can be seen at the console.
{
"block1_name2":[###values###],
"block2_name2":[###values###],
,
,
,
}
Note:
When you have a lot of named ranges, it is required to separate the request for retrieving the values, because of the limitation of the length of the URL. Please be careful about this.
About your 2nd question, when you want to convert from {"block1_name2":[###values###],,,} to {"block1": {"name2":[###values###]},,,}, as a premise, I think that it is required to decide the format of the name of the named ranges. In this case, from your showing sample named ranges, it supposes that your format of the name of all named ranges is like block#_name#. Please be careful about this.
References:
Method: spreadsheets.get
Method: spreadsheets.values.batchGet
You can use the Method: spreadsheets.get to list the ranges of a Google Spreadsheet. This is returned as an object.
{
"namedRangeId": "xxxxxxxxxxx",
"name": "test",
"range": {
"startRowIndex": 4,
"endRowIndex": 10,
"startColumnIndex": 0,
"endColumnIndex": 3
}
},
A simple sample using the method, and limiting the field for name ranges only:
function execute() {
return gapi.client.sheets.spreadsheets.get({
"spreadsheetId": "Google_Sheet_ID",
"fields": "namedRanges"
})
You can read more information about this method in the Google Documentation here
And there is a sample complete code on the same documentation. You can then use the name in "name": "test", to rename the objects.
Related
I am using the Google API to fetch data from a Google Spreadsheet via Node.js.
In Excel VBA it is possible to either use
Range("A3") or Cells(3,1)
to reference the range of the cell.
And for a multiple cell range it would be like:
Range(Cells(1,1), Cells(2,3))
instead of:
Range("A1:C2")
I could not find out how to do that via the Google Sheets API.
Tried following this guideline, but I think it's not the thing I need here:
https://googlesheets4.tidyverse.org/articles/range-specification.html
My example code in Node.js looks like this:
async function getHelperData(cl){
const gsapi = google.sheets({version:'v4', auth: cl});
const opt = {
spreadsheetId: gs.gs_ID,
range: '_helperSheet!A1:C2'
};
let data = await gsapi.spreadsheets.values.get(opt);
let dataArray = data.data.values;
return dataArray;
};
Is it actually possible?
About Use Cell Range instead of A1-notation in Google Sheets and the use of a value like Range(Cells(1,1), Cells(2,3)), I thought that in the current stage, when Sheets API is used, the a1Notation is used for retrieving values from Google Spreadsheet. So, in this case, I thought that it is required to convert a value like Range(Cells(1,1), Cells(2,3)) to the a1Notation.
When this is reflected in your script, how about the following modification?
Modified script:
const gsapi = google.sheets({version:'v4', auth: cl});
const sheetName = "_helperSheet"; // Please set sheet name.
const rangeObj = { start: [1, 1], end: [2, 3] }; // This is from `Range(Cells(1,1), Cells(2,3))` of "A1:C2" in your question.
const columnIndexToLetter_ = (index) => (a = Math.floor(index / 26)) >= 0 ? columnIndexToLetter_(a - 1) + String.fromCharCode(65 + (index % 26)) : ""; // Ref: https://stackoverflow.com/a/53678158
const a1Notation = rangeObj.end ? `'${sheetName}'!${columnIndexToLetter_(rangeObj.start[1] - 1)}${rangeObj.start[0]}:${columnIndexToLetter_(rangeObj.end[1] - 1)}${rangeObj.end[0]}` : `'${sheetName}'!${columnIndexToLetter_(rangeObj.start[1] - 1)}${rangeObj.start[0]}`;
const opt = {
spreadsheetId: gs.gs_ID,
range: a1Notation,
};
let data = await gsapi.spreadsheets.values.get(opt);
let dataArray = data.data.values;
console.log(dataArray);
In this modification, it supposes that the value of gs.gs_ID has already been declared elsewhere. Please be careful about this.
In this modification, a1Notation returns '_helperSheet'!A1:C2.
If const rangeObj = { start: [1, 1] }; is used, a1Notation returns '_helperSheet'!A1.
I have two google sheets forst one is https://docs.google.com/spreadsheets/d/1PJtjlkxCDFOIhJxpMJl4PcpJKL4nvGVtk2LDgVhbuNQ/edit#gid=0
and the second is https://docs.google.com/spreadsheets/d/1ZGw6dHpYE4ABvsE8S6dIPe7kLQ1eZW95Xp2F1oPHX5c/edit#gid=0
I want to filer data in first sheet base on the criteria which presents in the second sheet "Automate!D3" and then copy the filtered data to second sheet in "Filtered_Data". and want to this process automate so in future when i add more data to sheet one so that can be copy to below data in sheet two.
i can not able to filter throw app script, so i want to help in this.
Something like this should work:
const dsid = "1PJtjlkxCDFOIhJxpMJl4PcpJKL4nvGVtk2LDgVhbuNQ";
const tssid = "1ZGw6dHpYE4ABvsE8S6dIPe7kLQ1eZW95Xp2F1oPHX5c"
function filterByDate() {
const values = SpreadsheetApp.openById(dsid).getSheetByName('DT').getDataRange().getValues();
const tss = SpreadsheetApp.openById(tssid);
const criteria = tss.getSheetByName('Automate').getRange('D3').getValue();
const ts = tss.getSheetByName('Filtered_Data');
const results = values
.filter(row => row[1].getDate == criteria.getDate)
.map(row => [row[1],row[2],row[3],row[4]]);
ts.getRange(2,1,results.length,results[0].length).setValues(results);
}
I would suggest to place the criteria value in the daily sheet, and install the function into daily sheet as an edit trigger.
Make sure that the 'criteria' and the 'date' column are both formatted as 'Date' format.
The follow code has changed the way to get values.
Insead of .getValues(), it now uses .getDisplayValues(), and convert the string back into date obj afterwards.
const dsid = "1PJtjlkxCDFOIhJxpMJl4PcpJKL4nvGVtk2LDgVhbuNQ";
const tssid = "1ZGw6dHpYE4ABvsE8S6dIPe7kLQ1eZW95Xp2F1oPHX5c";
function filterByDate() {
const values = SpreadsheetApp.openById(dsid).getSheetByName('DT').getDataRange().getDisplayValues();
const tss = SpreadsheetApp.openById(tssid);
const criteria = tss.getSheetByName('Automate').getRange('D3').getDisplayValues();
const ts = tss.getSheetByName('Filtered_Data');
const results = values
.filter(row => new Date(row[1]).setHours(0,0,0,0) == new Date(criteria).setHours(0,0,0,0))
.map(row => [row[1],row[2],row[3],row[4]]);
ts.getRange(2,1,results.length,results[0].length).setValues(results);
}
in JavaScript I am trying to use Google Sheets API.
What I am trying to do is to append a row if it doesn't already exists in the spreadsheets, but update it if it is duplicated.
My sheets :
IDS
Status
A1
Status1
A2
Status2
What I want to push is : [A1, Status2];[A3, Status1]
So in this case, A1 Status is gonna change, and A3 is added as a new row.
(I already get in the array I want to push the IDs with (and the Status in another way):
const getIds = googleSheets.spreadsheets.values.get({
spreadsheetId,
range: "'Sheet 1'!A3:A23",
auth,
});
I believe your goal is as follows.
Your "Sheet1" of Spreadsheet is in the table of your question.
In your script, you are using the range as 'Sheet 1'!A3:A23. But, in your table, it seems that the data is shown from row 2.
You have the input value like [["A1", "Status2"], ["A3", "Status1"]].
You want to update "Sheet1" using the input value. When your showing table and your input value are used, you want to achieve the following situation of "Sheet1".
IDS Status
A1 Status2
A2 Status2
A3 Status1
From your added tag, you want to achieve this using googleapis for Node.js.
You have already been able to get and put values to the Spreadsheet using Sheets API.
In this case, how about the following sample script?
Sample script:
const sheets = google.sheets({ version: "v4", auth }); // Please use your authorization.
const spreadsheetId = "###"; // Please set your Spreadsheet ID.
const inputValues = [["A1", "Status2"], ["A3", "Status1"]]; // This is from your question.
sheets.spreadsheets.values.get({spreadsheetId, range: "'Sheet 1'!A2:B"},
(err, { data: { values } }) => {
if (err) {
console.log(err);
return;
}
const obj1 = inputValues.reduce((o, e) => ((o[e[0]] = e), o), {});
const obj2 = values.reduce((o, e) => ((o[e[0]] = e), o), {});
const res = [
...values.map((e) => obj1[e[0]] || e),
...inputValues.reduce((ar, e) => (obj2[e[0]] || ar.push(e), ar), []),
];
sheets.spreadsheets.values.update(
{
spreadsheetId,
range: "'Sheet 1'!A2",
resource: { values: res },
valueInputOption: "USER_ENTERED",
},
(err, { data }) => {
if (err) {
console.log(err);
return;
}
console.log(data);
}
}
);
}
);
In this sample script, first, the values are retrieved from the columns "A" and "B" of "Sheet1". And, the retrieved values are updated using the input value. And, the updated values are put to "Sheet1".
Note:
In your showing script, it seems that the sheet name is Sheet 1 which is not Sheet1. So, please confirm your sheet name again.
References:
Method: spreadsheets.values.get
Method: spreadsheets.values.update
I want to update automatically the value of comments_list with the values in the comments JSON object
const tweet = JSON.stringify({"tweet_id":1,"created_at":"2022-06-28","comments_list":[]})
const comments = JSON.stringify({"tweet_id":1,"commenter_id": 2"commenter_first_name":"tito","commenter_username":"tito_lulu"})
The final output should look like this
{"tweet_id":1,"created_at":"2022-06-28","comments_list":[{"commenter_id": 2"commenter_first_name":"tito","commenter_username":"tito_lulu"}]}
I'd work with those strings in an object form, otherwise string-manipulation could be slow in some cases.
This is by no means the fastest solution but perhaps the idea behind it can be helpful.
const tweet = [{
"tweet_id": 1,
"created_at": "2022-06-28",
"comments_list": []
}]; // There could be many tweet objects so wrap it in an array
const comments = [{
"tweet_id": 1,
"commenter_id": 2,
"commenter_first_name": "tito",
"commenter_username": "tito_lulu"
},
{
"tweet_id": 1,
"commenter_id": 5,
"commenter_first_name": "me-too",
"commenter_username": "me294"
}
]; // Same here, could be many comments right?
let UpdatedTweets = [];
// There are faster ways to do this, but for your question
tweet.forEach((tweet, tweetIndex) => {
// Loop each tweet
let post = tweet;
comments.forEach((comment, commentIndex) => {
if (comment.tweet_id == tweet.tweet_id) {
// we have a match lets combine them
tweet.comments_list.push({
commenter_id: comment.comment_id,
commenter_first_name: comment.commenter_first_name,
commenter_username: comment.commenter_username
});
}
});
UpdatedTweets.push(post);
});
console.log(JSON.stringify(UpdatedTweets));
The general idea is:
Parse the JSON into JS objects
Update the target object with the complementary information
Stringify the target object into JSON (only if you need to, eg. send the data to some other machine)
In your case:
const tweet = JSON.stringify({"tweet_id":1,"created_at":"2022-06-28","comments_list":[]});
const comments = JSON.stringify({"tweet_id":1,"commenter_id": 2,
"commenter_first_name":"tito","commenter_username":"tito_lulu"});
let o_tweet = JSON.parse(tweet)
, o_comments = JSON.parse(comments)
;
if (Array.isArray(comments)) { // Test whether that is a single or multiple comments
comments.forEach( c => { o_tweet.comments_list.push(c); });
} else {
o_tweet.comments_list.push(o_comments);
}
console.log(o_tweet);
// Only if needed:
// let newtweet = JSON.stringify(o_tweet)
I've got an array of rows that I've parsed out of a table from html, stored in a list. Each of the rows in the list is a string that looks (something) like this:
["<td headers="DOCUMENT" class="t14data"><a target="6690-Exhibit-C-20190611-1" href="http://www.fara.gov/docs/6690-Exhibit-C-20190611-1.pdf" class="doj-analytics-processed"><span style="color:blue">Click Here </span></a></td><td headers="REGISTRATIONNUMBER" class="t14data">6690</td><td headers="REGISTRANTNAME" class="t14data">SKDKnickerbocker LLC</td><td headers="DOCUMENTTYPE" class="t14data">Exhibit C</td><td headers="STAMPED/RECEIVEDDATE" class="t14data">06/11/2019</td>","<td headers="DOCUMENT" class="t14data"><a target="5334-Supplemental-Statement-20190611-30" href="http://www.fara.gov/docs/5334-Supplemental-Statement-20190611-30.pdf" class="doj-analytics-processed"><span style="color:blue">Click Here </span></a></td><td headers="REGISTRATIONNUMBER" class="t14data">5334</td><td headers="REGISTRANTNAME" class="t14data">Commonwealth of Dominica Maritime Registry, Inc.</td><td headers="DOCUMENTTYPE" class="t14data">Supplemental Statement</td><td headers="STAMPED/RECEIVEDDATE" class="t14data">06/11/2019</td>"]
The code is pulled from the page with the following page.evaluate function using puppeteer.
I'd like to then parse this code with cheerio, which I find to be simpler and more understandable. However, when I pass each of the strings of html into cheerio, it fails to parse them correctly. Here's the current function I'm using:
let data = res.map((tr) => {
let $ = cheerio.load(tr);
const link = $("a").attr("href");
const number = $("td[headers='REGISTRATIONNUMBER']").text();
const name = $("td[headers='REGISTRANTNAME']").text();
const type = $("td[headers='DOCUMENTTYPE']").text();
const date = $("td[headers='STAMPED/RECEIVEDDATE']").text();
return { link, number, name, type, date };
});
For some reason, only the "a" tag is working correctly for each row. Meaning, the "link" variable is correctly defined, but none of the other ones are. When I use $("*") to return a list of what should be all of the td's, it returns an unusual node list:
What am I doing wrong, and how can I gain access to the td's with the various headers, and their text content? Thanks!
It usually looks more like this:
let data = res.map((i, tr) => {
const link = $(tr).find("a").attr("href");
const number = $(tr).find("td[headers='REGISTRATIONNUMBER']").text();
const name = $(tr).find("td[headers='REGISTRANTNAME']").text();
const type = $(tr).find("td[headers='DOCUMENTTYPE']").text();
const date = $(tr).find("td[headers='STAMPED/RECEIVEDDATE']").text();
return { link, number, name, type, date };
}).get();
Keep in mind that cheerio map has the arguments reversed from js map.
I found the solution. I'm simply returning the full html through puppeteer instead of trying to get individual rows, and then using the above suggestion (from #pguardiario) to parse the text:
const res = await page.evaluate(() => {
return document.body.innerHTML;
});
let $ = cheerio.load(res);
let trs = $(".t14Standard tbody tr.highlight-row");
let data = trs.map((i, tr) => {
const link = $(tr).find("a").attr("href");
const number = $(tr).find("td[headers='REGISTRATIONNUMBER']").text();
const registrant = $(tr).find("td[headers='REGISTRANTNAME']").text();
const type = $(tr).find("td[headers='DOCUMENTTYPE']").text();
const date = moment($(tr).find("td[headers='STAMPED/RECEIVEDDATE']").text()).valueOf().toString();
return { link, number, registrant, type, date };
});