I want to execute this node js line in google apps script.
How do I use this line in google apps script:
const payload = new Buffer(JSON.stringify(obj)).toString('base64');
When I run it I got this error:
ReferenceError: Buffer is not defined
I believe your goal as follows.
You want to convert const payload = new Buffer(JSON.stringify(obj)).toString('base64'); in Node.js to Google Apps Script.
Unfortunately, in the current stage, new Buffer() and Buffer.from() cannot be used with Google Apps Script. So in this case, I think that Utilities.base64Encode can be used for your situation. The sample script is as follows.
Sample script:
const obj = {key: "value"};
const payload = Utilities.base64Encode(JSON.stringify(obj));
console.log(payload) // eyJrZXkiOiJ2YWx1ZSJ9
Result:
When above script is run, eyJrZXkiOiJ2YWx1ZSJ9 is retrieved. In this case, I could confirm that the result value is the same with the following Node.js script.
const obj = {key: "value"};
const payload = new Buffer(JSON.stringify(obj)).toString('base64');
// or const payload = Buffer.from(JSON.stringify(obj)).toString('base64');
console.log(payload) // eyJrZXkiOiJ2YWx1ZSJ9
Reference:
base64Encode(data)
Buffer is used in nodeJS
The equivilant in client side JavaScript is array buffer
In order to make one from a string, you have to make a uint8array or Uint16Array from a new array buffer object with the length of the string, then loop through the array and add the char code values from the string at the index, and return the buffer
A function
function Buffer(str) {
var buf= new ArrayBuffer(str.length/*multiply by 2 for higher chars*/)
var ar = new Uint8Array(buf) //use Uint16Array etc for larger, i=0
for(i=0;i< ar.length;i++) ar[i] = str.charCodeAt(I)
return buf
}
Related
I have a google apps script function that is pulling data from a spreadsheet. It is able to log the information happily and then I return the data to a javascript function. But when I try to stringify and alert the information it alerts "null".
Not sure where I am going wrong!
Javascript calling the GAS function:
google.script.run.withSuccessHandler(getNews).getTheNews();
GAS function:
function getTheNews(){
var ss = SpreadsheetApp.openByUrl(url);
var ws = ss.getSheetByName("NewsFeed");
var theNews = ws.getRange(3, 2, ws.getLastRow()-2, 1).getValues();
Logger.log(theNews);
return theNews;
}
The GAS log:
Javascript function receiving the info:
function getNews(theNews){
var mystr = JSON.stringify(theNews);
alert(mystr);
}
Issue and workaround:
Unfortunately, it seems that in the current stage, the date object cannot be directly returned from Google Apps Script side to Javascript side. Also I could replicate your situation. I think that this is the reason of your issue. So as the current workaround, how about the following modification?
Modified script:
In this modification, the date object is converted to the string and sent to Javascript side. At Javascript side, the date string is converted to the date object.
HTML & Javascript side:
google.script.run.withSuccessHandler(getNews).getTheNews();
function getNews(theNews){
theNews = theNews.map(r => r.map(c => new Date(c))); // Added
var mystr = JSON.stringify(theNews);
alert(mystr);
}
Google Apps Script side:
function getTheNews(){
var ss = SpreadsheetApp.openByUrl(url);
var ws = ss.getSheetByName("NewsFeed");
var theNews = ws.getRange(3, 2, ws.getLastRow()-2, 1).getValues();
return theNews.map(r => r.map(c => c.toISOString())); // Modified
}
I think that in this case, c.getTime() instead of c.toISOString() can be also used.
Note:
For example, if you want to retrieve the display values on the cells, you can also modify as follows. In this case, Javascript is not required to be modified.
From
var theNews = ws.getRange(3, 2, ws.getLastRow()-2, 1).getValues();
To
var theNews = ws.getRange(3, 2, ws.getLastRow()-2, 1).getDisplayValues();
Also I found this issue at the Google issue tracker. Ref But when I saw the status, it seems "Won't Fix (Intended behavior)".
References:
toISOString()
getDisplayValues()
I am trying to create an HMAC signature in Postman using a pre-request script. Without going too far into the details of implementation,
I have confirmed that my means for generating the signature is messed up. I can see what the expected result should be with a proof of concept example but I’m missing something somewhere and cannot tell if it is in the conversion. I’ve read around from other questions on SO that binary is the default provided by cryptojs internally and that simply calling for the hash is the equivalent of asking for the digest with conversions completed for you. Here is the code I’m trying to run in postman and the working implementation code as shown in nodeJS.
var CryptoJS = require("crypto-js");
const d = new Date();
const timestamp = d.getTime();
const postData = {};
postData.nonce = 100; //timestamp * 1000; //nanosecond
postman.setEnvironmentVariable('nonce', postData.nonce);
const secret = CryptoJS.enc.Base64.parse(pm.environment.get("apiSecret"));
const path = pm.globals.get("balanceMethod");
const message = CryptoJS.SHA256( encodeURI(postData.nonce + postData)) ; // ...
const hmacDigest = CryptoJS.HmacSHA512(path + message, secret);
postman.setEnvironmentVariable('API-Signature', CryptoJS.enc.Base64.stringify(hmacDigest));
console.log(CryptoJS.enc.Base64.stringify(hmacDigest));
Does this apply to my situation in that I’d need to convert my sha256 message into a bytes array in order to work?
Reference code for building implementation that does work with nodeJS:
const getMessageSignature = (path, request, secret, nonce) => {
const message = qs.stringify(request);
const secret_buffer = new Buffer(secret, 'base64');
const hash = new crypto.createHash('sha256');
const hmac = new crypto.createHmac('sha512', secret_buffer);
const hash_digest = hash.update(nonce + message).digest('binary');
const hmac_digest = hmac.update(path + hash_digest, 'binary').digest('base64');
return hmac_digest;
};
Same reference code for building implementation in python3:
req['nonce'] = 100 #int(1000*time.time())
postdata = urllib.parse.urlencode(req)
# Unicode-objects must be encoded before hashing
encoded = (str(req['nonce']) + postdata).encode()
message = urlpath.encode() + hashlib.sha256(encoded).digest()
signature = hmac.new(base64.b64decode(self.secret),
message, hashlib.sha512)
sigdigest = base64.b64encode(signature.digest())
The only post data I'm sending is the Nonce at this time and I've purposely set it to 100 to be able to replicate the result to fix the generated signature. Seems close but not matching result. The python and nodeJS do match expected results and work properly.
Check out the answer in this thread. It helped me with the my problem and may be what is happening in your case also. All it is necessary is break the input of the HMAC into two parts.
I'm processing a very large amount of data that I'm manipulating and storing it in a file. I iterate over the dataset, then I want to store it all in a JSON file.
My initial method using fs, storing it all in an object then dumping it didn't work as I was running out of memory and it became extremely slow.
I'm now using fs.createWriteStream but as far as I can tell it's still storing it all in memory.
I want the data to be written object by object to the file, unless someone can recommend a better way of doing it.
Part of my code:
// Top of the file
var wstream = fs.createWriteStream('mydata.json');
...
// In a loop
let JSONtoWrite = {}
JSONtoWrite[entry.word] = wordData
wstream.write(JSON.stringify(JSONtoWrite))
...
// Outside my loop (when memory is probably maxed out)
wstream.end()
I think I'm using Streams wrong, can someone tell me how to write all this data to a file without running out of memory? Every example I find online relates to reading a stream in but because of the calculations I'm doing on the data, I can't use a readable stream. I need to add to this file sequentially.
The problem is that you're not waiting for the data to be flushed to the filesystem, but instead keep throwing new and new data to the stream synchronously in a tight loop.
Here's an piece of pseudocode that should work for you:
// Top of the file
const wstream = fs.createWriteStream('mydata.json');
// I'm no sure how're you getting the data, let's say you have it all in an object
const entry = {};
const words = Object.keys(entry);
function writeCB(index) {
if (index >= words.length) {
wstream.end()
return;
}
const JSONtoWrite = {};
JSONtoWrite[words[index]] = entry[words[index]];
wstream.write(JSON.stringify(JSONtoWrite), writeCB.bind(index + 1));
}
wstream.write(JSON.stringify(JSONtoWrite), writeCB.bind(0));
You should wrap your data source in a readable stream too. I don't know what is your source, but you have to make sure, it does not load all your data in memory.
For example, assuming your data set come from another file where JSON objects are splitted with end of line character, you could create a Read stream as follow:
const Readable = require('stream').Readable;
class JSONReader extends Readable {
constructor(options={}){
super(options);
this._source=options.source: // the source stream
this._buffer='';
source.on('readable', function() {
this.read();
}.bind(this));//read whenever the source is ready
}
_read(size){
var chunk;
var line;
var lineIndex;
var result;
if (this._buffer.length === 0) {
chunk = this._source.read(); // read more from source when buffer is empty
this._buffer += chunk;
}
lineIndex = this._buffer.indexOf('\n'); // find end of line
if (lineIndex !== -1) { //we have a end of line and therefore a new object
line = this._buffer.slice(0, lineIndex); // get the character related to the object
if (line) {
result = JSON.parse(line);
this._buffer = this._buffer.slice(lineIndex + 1);
this.push(JSON.stringify(line) // push to the internal read queue
} else {
this._buffer.slice(1)
}
}
}}
now you can use
const source = fs.createReadStream('mySourceFile');
const reader = new JSONReader({source});
const target = fs.createWriteStream('myTargetFile');
reader.pipe(target);
then you'll have a better memory flow:
Please note that the picture and the above example are taken from the excellent nodejs in practice book
EDIT
I found this:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays
Basically if I have something like this:
struct someStruct {
unsigned long id;
char username[16];
float amountDue;
};
on client side I can do:
var ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = function (e) {
var buffer = e.data;
var data_view = new DataView(buffer);
// ... read the data into the buffer ...
var idView = data_view.getUint32(0);
var usernameView = data_view.getUint32(4);
var amountDueView = data_view.getFloat32(20);
};
The problem is that I want to convert them to normal Javascript objects (numbers, strings etc).
Original question
I would send data via websocket packed using a C struct, and unpack on browser using Javascript.
I know modules exists for node.js, but I can't find nothing client-side.
If you're familiar with Python struct, then you may like structjs. It's my attempt at porting Python struct to javascript. As it is, it's for Node, but a client port should be easy.
You won't have issues with alignment or padding (you can specify those explicitly though) for that structure, but you may need to indicate little-endian (by the '<' in the format string) if that's your flavour. You might do like so (I haven't tested this example in any way):
let struct = require("./struct") // Node specific, you need to wrap it.
let someStruct = struct('<I16sf') // This is your struct definition
let ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = e => {
// Unpack using the structure definition. Unpack takes an ArrayBuffer.
let [id, username, amountDue] = someStruct.unpack(e.data);
// Use data...
};
Ok,
https://www.npmjs.com/package/c-struct looks like what you want.
Good luck!
Ok, after some researching, I finally decided this could not be a good idea:
https://justin.harmonize.fm/development/2013/04/28/a-slower-js-msgpack.html
Shortly: javascript is slow in decoding.
Probably it's just more simple to use JSON and Content-Encoding: gzip, if it does not slow down your web app.
I'm trying to access a named pipe in a Firefox add-on. My code, based on solution 2 to this CodeProject question, is:
var file = Components.classes["#mozilla.org/file/local;1"].createInstance(Components.interfaces.nsILocalFile);
file.initWithPath("\\\\.\\pipe\\test");
var text = "Some text to be written";
var writer = Components.classes["#mozilla.org/network/file-output-stream;1"].createInstance(Components.interfaces.nsIFileOutputStream);
// Open file for read/write access, and create it if it does not exist.
writer.init (file, 0x04 | 0x08, -1, 0);
writer.write (text, text.length);
writer.flush ();
writer.close ();
When I run this is Firefox Scratchpad, I get:
/*
Exception: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFileOutputStream.init]
#6
*/
Line 6 is the line where I call writer.init.
I've played with passing different flags to writer.init, but no luck. I'm able to write to a normal file path with this code, just not the named pipe path.
I've been searching for more information for most of a day. The only other relevant thing I've found is this Bugzilla bug that mentions the same problem, but it's dated 2009.
Thank you in advance for any suggestions.
Edit: Based on the replies I tried the following:
var encoder = new TextEncoder();
var array = encoder.encode("This is some text");
var path = "\\\\.\\pipe\\test";
Task.spawn(function() {
let pfh = yield OS.File.open(path, {write: true});
yield pfh.write(array);
yield pfh.close();
});
which lets me write to a file (if I change the path accordingly) but does not seem to send anything to the pipe.
Here is the (admittedly crude) code I'm using to read the pipe on the .NET side:
let server = new NamedPipeServerStream ("\\\\.\\pipe\\test", PipeDirection.InOut)
let reader = new StreamReader (server)
do
printfn "Listening..."
server.WaitForConnection ()
printfn "Connected."
while true do
while reader.EndOfStream = false do reader.ReadLine () |> printfn "%s"
And the code I threw together to verify that I could at least write to the pipe in .NET:
let client = new NamedPipeClientStream ("\\\\.\\pipe\\test")
do client.Connect ()
let writer = new StreamWriter (client)
do writer.AutoFlush <- true
// Wait for connection
System.Threading.Thread.Sleep (1000)
let message = "hello"
do message |> writer.WriteLine
Try OS.File:
let encoder = new TextEncoder(); // This encoder can be reused for several writes
let array = encoder.encode("This is some text"); // Convert the text to an array
let promise = OS.File.writeAtomic("\\\\.\\pipe\\test", array, // Write the array atomically to "file.txt", using as temporary
{tmpPath: "file.txt.tmp"}); // buffer "file.txt.tmp".