Develop Reference

Google Sheets API - Import all named Ranges

I have a google sheet with several named ranges.
I would like to import every single named range in the sheet and save them as individual objects named after the named range.
Furthermore, there are 4 similar blocks of data. Therefore the names of the ranges are structured like "block1_name1" "block1_name2" "block2_name1" "block2_name2" etc.
With the following code I can enter every range manually, but there are too many to enter them all manually:
const API_KEY = "###"; // Please set your API key.
const ID = "###"; // Please set your Spreadsheet ID.
const RANGE = ["Range1", "Range2"]; // Named ranges
const ranges = RANGE.map(e => `ranges=${encodeURIComponent(e)}`).join("&");
const response = await fetch(`https://sheets.googleapis.com/v4/spreadsheets/${ID}/values:batchGet?key=${API_KEY}&${ranges}`);
const { valueRanges } = await response.json();
const obj = valueRanges.reduce((o, { values }, i) => (o[RANGE[i]] = values, o), {});
console.log(obj);
How can I import every named range automatically?
How can I save them as a different objects for each datablock like block1.name1 etc?

I believe your goal is as follows.
You want to retrieve the named range list from Google Spreadsheet, and want to retrieve the values from the named ranges.
And, you want to export the values as {namedRange1: values1, namedRange2: values2,,,}.
You want to achieve this by directly requesting the endpoint of Sheets API with fetch API of Javascript.
In this case, how about the following modification?
Modified script:
const API_KEY = "###"; // Please set your API key.
const ID = "###"; // Please set your Spreadsheet ID.
// 1. Retrieve the named range list.
const base = `https:\/\/sheets.googleapis.com/v4/spreadsheets/${ID}`;
const res1 = await fetch(`${base}?key=${API_KEY}&fields=namedRanges(name)`);
const { namedRanges } = await res1.json();
// 2. Retrieve values from named ranges.
const ranges = namedRanges.map(({ name }) => `ranges=${encodeURIComponent(name)}`).join("&");
const res2 = await fetch(`${base}/values:batchGet?key=${API_KEY}&${ranges}`);
const { valueRanges } = await res2.json();
// 3. Create an output object.
const res = valueRanges.reduce((o, { values }, i) => (o[namedRanges[i].name] = values, o), {});
console.log(res);
// For your 2nd question.
const res3 = valueRanges.reduce((o, { values }, i) => {
const [k1, k2] = namedRanges[i].name.split("_");
if (o[k1]) {
o[k1][k2] = values;
} else {
o[k1] = { [k2]: values };
}
return o;
}, {});
console.log(res3);
Testing:
When this script is run, the following result can be seen at the console.
{
"block1_name2":[###values###],
"block2_name2":[###values###],
,
,
,
}
Note:
When you have a lot of named ranges, it is required to separate the request for retrieving the values, because of the limitation of the length of the URL. Please be careful about this.
About your 2nd question, when you want to convert from {"block1_name2":[###values###],,,} to {"block1": {"name2":[###values###]},,,}, as a premise, I think that it is required to decide the format of the name of the named ranges. In this case, from your showing sample named ranges, it supposes that your format of the name of all named ranges is like block#_name#. Please be careful about this.
References:
Method: spreadsheets.get
Method: spreadsheets.values.batchGet

You can use the Method: spreadsheets.get to list the ranges of a Google Spreadsheet. This is returned as an object.
{
"namedRangeId": "xxxxxxxxxxx",
"name": "test",
"range": {
"startRowIndex": 4,
"endRowIndex": 10,
"startColumnIndex": 0,
"endColumnIndex": 3
}
},
A simple sample using the method, and limiting the field for name ranges only:
function execute() {
return gapi.client.sheets.spreadsheets.get({
"spreadsheetId": "Google_Sheet_ID",
"fields": "namedRanges"
})
You can read more information about this method in the Google Documentation here
And there is a sample complete code on the same documentation. You can then use the name in "name": "test", to rename the objects.

Cheerio Not Parsing HTML Correctly

I've got an array of rows that I've parsed out of a table from html, stored in a list. Each of the rows in the list is a string that looks (something) like this:
["<td headers="DOCUMENT" class="t14data"><a target="6690-Exhibit-C-20190611-1" href="http://www.fara.gov/docs/6690-Exhibit-C-20190611-1.pdf" class="doj-analytics-processed"><span style="color:blue">Click Here </span></a></td><td headers="REGISTRATIONNUMBER" class="t14data">6690</td><td headers="REGISTRANTNAME" class="t14data">SKDKnickerbocker LLC</td><td headers="DOCUMENTTYPE" class="t14data">Exhibit C</td><td headers="STAMPED/RECEIVEDDATE" class="t14data">06/11/2019</td>","<td headers="DOCUMENT" class="t14data"><a target="5334-Supplemental-Statement-20190611-30" href="http://www.fara.gov/docs/5334-Supplemental-Statement-20190611-30.pdf" class="doj-analytics-processed"><span style="color:blue">Click Here </span></a></td><td headers="REGISTRATIONNUMBER" class="t14data">5334</td><td headers="REGISTRANTNAME" class="t14data">Commonwealth of Dominica Maritime Registry, Inc.</td><td headers="DOCUMENTTYPE" class="t14data">Supplemental Statement</td><td headers="STAMPED/RECEIVEDDATE" class="t14data">06/11/2019</td>"]
The code is pulled from the page with the following page.evaluate function using puppeteer.
I'd like to then parse this code with cheerio, which I find to be simpler and more understandable. However, when I pass each of the strings of html into cheerio, it fails to parse them correctly. Here's the current function I'm using:
let data = res.map((tr) => {
let $ = cheerio.load(tr);
const link = $("a").attr("href");
const number = $("td[headers='REGISTRATIONNUMBER']").text();
const name = $("td[headers='REGISTRANTNAME']").text();
const type = $("td[headers='DOCUMENTTYPE']").text();
const date = $("td[headers='STAMPED/RECEIVEDDATE']").text();
return { link, number, name, type, date };
});
For some reason, only the "a" tag is working correctly for each row. Meaning, the "link" variable is correctly defined, but none of the other ones are. When I use $("*") to return a list of what should be all of the td's, it returns an unusual node list:
What am I doing wrong, and how can I gain access to the td's with the various headers, and their text content? Thanks!

It usually looks more like this:
let data = res.map((i, tr) => {
const link = $(tr).find("a").attr("href");
const number = $(tr).find("td[headers='REGISTRATIONNUMBER']").text();
const name = $(tr).find("td[headers='REGISTRANTNAME']").text();
const type = $(tr).find("td[headers='DOCUMENTTYPE']").text();
const date = $(tr).find("td[headers='STAMPED/RECEIVEDDATE']").text();
return { link, number, name, type, date };
}).get();
Keep in mind that cheerio map has the arguments reversed from js map.

I found the solution. I'm simply returning the full html through puppeteer instead of trying to get individual rows, and then using the above suggestion (from #pguardiario) to parse the text:
const res = await page.evaluate(() => {
return document.body.innerHTML;
});
let $ = cheerio.load(res);
let trs = $(".t14Standard tbody tr.highlight-row");
let data = trs.map((i, tr) => {
const link = $(tr).find("a").attr("href");
const number = $(tr).find("td[headers='REGISTRATIONNUMBER']").text();
const registrant = $(tr).find("td[headers='REGISTRANTNAME']").text();
const type = $(tr).find("td[headers='DOCUMENTTYPE']").text();
const date = moment($(tr).find("td[headers='STAMPED/RECEIVEDDATE']").text()).valueOf().toString();
return { link, number, registrant, type, date };
});

Get first word of string inside array - from return REST

I try get the sessionid before REST function, but in the case if I does not convert toString(); show only numbers (21 22 2e ...).
See this image:
1º:
Obs.: Before using split.
!!xxxxxxx.xxxxx.xxxxxxx.rest.schema.xxxxResp {error: null, sessionID: qdaxxxxxxxxxxxxxj}
My code:
var Client = require('./lib/node-rest-client').Client;
var client = new Client();
var dataLogin = {
data: { "userName":"xxxxxxxx","password":"xxxxxxxx","platform":"xxxxx" },
headers: { "Content-Type": "application/json" }
};
client.registerMethod("postMethod", "xxxxxxxxxxx/login", "POST");
client.methods.postMethod(dataLogin, function (data, response) {
// parsed response body as js object
// console.log(data); all return, image 1
// raw response
if(Buffer.isBuffer(data)){
data = data.toString('utf8'); // if i does not convert to string, return numbers, see image 1..
console.log(data); //all inside image 2, and i want just value from sessionid
var output = data;
var res = output.split(" "); // using split
res = res[4].split("}", 1);
}
console.log(res); //image 3
});
I tested with JSON.parse and JSON.stringify and it did not work, show just 'undefined' for all. After convert toString();, And since I've turned the values ​​into string, I thought of using split to get only the value of sessionid.
And when I used split, all transform to array and the return is from console.log(data), see image 2:
2º:
Obs.: After use split and convert to array automatically.
And the return after use split is with the conditions inside my code:
3º:
And the return after use split is with the conditions inside my code:
[ 'bkkRQxxxxxxxxxxxxx' ]
And I want just:
bkkRQxxxxxxxxxxxxx
I would like to know how to solve this after all these temptations, but if you have another way of getting the sessionid, I'd be happy to know.
Thanks advance!

After converting the Buffer to a string, remove anything attached to the front with using data.substr(data.indexOf('{')), then JSON.parse() the rest. Then you can just use the object to get the sessionID.
if(Buffer.isBuffer(data)){
data = data.toString('utf8');
data = data.substr(data.indexOf('{'));
obj = JSON.parse(data);
console.log(obj.sessionID);
}
EDIT:
The issue you are having with JSON.parse() is because what is being returned is not actually JSON. The JSON spec requires the properties to be quoted ("). See this article
If the string looked like this, it would work: {"error": null, "sessionID": qdaxxxxxxxxxxxxxj}
Because the json is not really json, you can use a regular expression to get the info you want. This should get it for you.
re = /(sessionID: )([^,}]*)/g;
match = re.exec(data);
console.log(match[2]);
EDIT 2: After fully reading the article that I linked above (oops haha), this is a more preferable way to deal with unquoted JSON.
var crappyJSON = '{ somePropertyWithoutQuotes: "theValue!" }';
var fixedJSON = crappyJSON.replace(/(['"])?([a-zA-Z0-9_]+)(['"])?:/g, '"$2": ');
var aNiceObject = JSON.parse(fixedJSON);

Javascript from Buffer to JSON

I'm using bleno (A node js BLE package) and it uses Buffer to send and receive data. How will I go about getting a Buffer object and converting it into JSON? This is what i have now:
bufferToJson = buffer.toString();
bufferToJson = JSON.stringify(bufferToJson)
bufferToJson = JSON.parse(bufferToJson)
buffer is where the data is. An example of what buffer can be is {cmd:'echo'} I have tried bufferToJson.cmd and only get undefine. Thanks.

If your buffer object contains a valid representation of a JSON, then the easiest way to convert it would be like so:
const json = JSON.parse(buffer);

Following should work:
var bufferToJson = JSON.parse(myBuffer.toString());

You can use TextDecoder as in following fragment:
const buffer = await characteristic.readValue();
const decoder = new TextDecoder('utf8');
const text = decoder.decode(buffer);
console.log(JSON.parse(text));

For nodejs apps, I found String Decoder to work out great.
https://nodejs.org/api/string_decoder.html
// API for decoding Buffer objects into strings
const { StringDecoder } = require('string_decoder');
const decoder = new StringDecoder('utf8');
let body = Buffer.from(response.body);
let json = decoder.write(body);
let foo = JSON.parse(json);

How to concat buffers with delimiter in node.js?

I'm trying to concat two buffers with a space in between them in Node.js.
Here is my code.
var buff1 = new Buffer("Jumping");
var buff2 = new Buffer("Japang");
var buffSpace = new Buffer(1);
buffSpace[0] = "32";
var newBuff = Buffer.concat([buff1, buffSpace, buff2], (buff1.length + buff2.length + buffSpace.length));
console.log(newBuff.toString());
As per official doc, the first argument will be the Array list of Buffer objects. Hence I've created buffSpace for space.
Class Method: Buffer.concat(list[, totalLength])
list : Array List of Buffer objects to concat
totalLength: Number Total length of the buffers when concatenated
I'm getting the result as expected but not sure whether it is right way to do so. Please suggest if any better solution to achieve the same.

There are three changes I would suggest.
First, if you are using Node v6, use Buffer.from() instead of new Buffer(), as the latter is deprecated.
Second, you don't need to pass an argument for totalLength to Buffer.concat(), since it will be calculated automatically from the length of all of the buffers passed. While the docs note it will be faster to pass a total length, this will really only be true if you pass a constant value. What you are doing above is computing the length and then passing that, which is what the concat() function will do internally anyway.
Finally, I would recommend putting this in a function that works like Array.prototype.join(), but for buffers.
function joinBuffers(buffers, delimiter = ' ') {
let d = Buffer.from(delimiter);
return buffers.reduce((prev, b) => Buffer.concat([prev, d, b]));
}
And you can use it like this:
let buf1 = Buffer.from('Foo');
let buf2 = Buffer.from('Bar');
let buf3 = Buffer.from('Baz');
let joined = joinBuffers([buf1, buf2, buf3]);
console.log(joined.toString()); // Foo Bar Baz
Or set a custom delimiter like this:
let joined2 = joinBuffers([buf1, buf2, buf3], ' and ');
console.log(joined2.toString()); // Foo and Bar and Baz

Read the Buffer stream and save it to file as:
const data = [];
req.on('data', stream => {
data.push(stream);
});
req.on('close', () => {
const parsedData = Buffer.concat(data).toString('utf8');
fs.writeFileSync('./test.text', parsedData);
});