I tried to extract feature of an audio file using Meyda. But feature exytracted values are different when extracted using command line meyda and through javascript file
Command line
meyda 1600153162.63571.wav mfcc
Result
0.057628476337413304,0.025103534166687494,-0.0033402018588145435,-0.0050914619032203364,-0.005969043107840086,-0.009452339628697326,-0.0077781823184730455,-0.0024033016732702066,-0.0033368248853514206,-0.0012372591871309979,0.00003587458387239886,0.00293735474906295,0.0033156231532707036
1.4306079393227265,1.3713938566968262,1.2306487569743767,1.0515807610571597,0.8414768385049735,0.6060415036630535,0.37326507523088365,0.1654296077929904,-0.015392187854876595,-0.1652509593657213,-0.27571446879975764,-0.340564139593766,-0.36505384389267703
9.282664122642018,6.880236029290547,3.3870794551074606,1.7170683624244123,-0.29657914336587826,-3.18544774959499,-4.433789386396694,-3.7101490665746217,-2.816738558429225,-2.2736417577176273,-1.6636438686466657,-0.8537746809738949,-0.5116463089021152
25.65632094501052,22.93621824737729,16.08611763344924,7.490656903754756,-0.4358888007314062,-5.641656791337735,-7.229285832273481,-5.635856022395436,-2.1681330598896817,1.5575050036371694,4.296381195632149,5.45750656573637,5.059966218569768
23.469876801827922,20.984500104582974,15.80549272827184,10.064116650359527,3.957435073588796,-1.75506262168123,-5.904087102697238,-8.29320895086182,-9.037630914839253,-8.230609293960022,-6.604159793447832,-4.8505887091085365,-3.2390627275186175
29.14206006610766,25.50517483703385,19.279159148659456,13.806566615476791,7.715092034538,2.2364700767576764,-0.8129542016331001,-2.16772108652068,-2.6257181917727093,-2.1347711722007845,-1.0689231846996041,-0.6059541306434987,-0.9955650805848651
12.251538716256618,9.479093353756676,5.26726325506769,2.7893618666871762,0.10288164291841367,-3.121859159330943,-4.446188894370852,-3.5959072906303837,-2.5699522942015007,-1.8300687753240275,-0.7140618177046422,0.3644613187160053,0.8353185065087604
18.42772721964866,15.990459244575096,11.993313228734504,9.088259023158539,6.172351883886699,2.6632784721745213,-0.2683186290158647,-1.948275236251193,-2.793271841187546,-3.305172382572076,-3.6707865100252994,-3.638999985131219,-2.863263822827715
15.915610973257571,13.737073178518493,9.801934620786648,5.779403187264773,1.358952752885622,-2.5086411759816647,-5.187471901991046,-7.0034704192058985,-7.837140523422115,-7.437282553994893,-6.058991955183858,-4.360246975428363,-2.680629805396066
15.75090683856979,13.497089898444463,9.489154494388005,5.578283842168272,1.4567586737610156,-2.141429668615224,-4.380728646793593,-5.364310654901947,-5.049848052960211,-4.214979808517224,-3.9745754014376224,-4.076571890925421,-4.020714303609834
23.361729244701564,20.83103985201436,16.592090051349242,12.885538444506535,8.373777248668826,2.9768227085972145,-1.399644048138661,-4.090450595665753,-5.574775136428788,-5.787457864519816,-4.693018758118676,-3.100993535744204,-1.5625546377399475
12.793707716744393,10.197116076581045,5.819041457288568,2.4679725003955064,-0.33041998215142393,-2.607817847380143,-4.086120143731154,-4.779218992884254,-4.374465394096245,-3.118504640745296,-1.9666020102081847,-1.5140789377241373,-1.5130647010950313
3.2732387410942465,2.2453286293698977,0.9749483649128511,0.472715919767034,0.11249067642678866,-0.19927487640318992,-0.24286230087509772,-0.3645480814147444,-0.5060378572115204,-0.4793996411313405,-0.45934442588699886,-0.4680761055711004,-0.2967506177492105
Through javscript file
var express = require('express');
var app = express();
const fs = require('fs');
var Meyda = require('meyda');
var load = require('audio-loader')
let filename = "1600153162.63571.wav"
load(filename).then(function (buffer) {
const channelData = buffer.getChannelData(0)
const PaddingLength = (Math.pow(2,Math.round(Math.log2(channelData.length)+1)) - channelData.length)
let halfPaddingLength = parseInt(PaddingLength/2)
const pad1 = new Array(halfPaddingLength).fill(0);
const pad2 = new Array(PaddingLength - halfPaddingLength).fill(0);
let finalBbuffer = [...pad1,...channelData,...pad2]
console.log(finalBbuffer.length)
let mfccData = Meyda.extract('mfcc', finalBbuffer)
console.log("mfccData : ",mfccData);
});
Result :
[
249.11783051490784,
-90.61751411189829,
-12.253094024524968,
19.88245460444982,
-11.661965456271869,
-14.795375019626466,
7.19298966815922,
8.362884489124907,
3.9941283332736557,
-3.2158388656478287,
-2.0395393071161063,
-0.48849176751482837,
-3.6199623273626695
]
Why this difference . How to make them same ?
They're different because you're padding the buffer in your code, whereas the command line tool is splitting your file into multiple buffers with the default buffer size and returning MFCCs for each segment.
To make them the same, you could replicate the buffer chunking that the CLI performs in your code, rather than zero-padding the whole buffer, or you could make a CLI that zero pads the input. If you do the latter, we would certainly be interested in adding that functionality to the bundled CLI, so feel free to open an issue on our issue tracker to discuss if you go that route and wish to contribute.
Thanks for using Meyda!
I want to execute this node js line in google apps script.
How do I use this line in google apps script:
const payload = new Buffer(JSON.stringify(obj)).toString('base64');
When I run it I got this error:
ReferenceError: Buffer is not defined
I believe your goal as follows.
You want to convert const payload = new Buffer(JSON.stringify(obj)).toString('base64'); in Node.js to Google Apps Script.
Unfortunately, in the current stage, new Buffer() and Buffer.from() cannot be used with Google Apps Script. So in this case, I think that Utilities.base64Encode can be used for your situation. The sample script is as follows.
Sample script:
const obj = {key: "value"};
const payload = Utilities.base64Encode(JSON.stringify(obj));
console.log(payload) // eyJrZXkiOiJ2YWx1ZSJ9
Result:
When above script is run, eyJrZXkiOiJ2YWx1ZSJ9 is retrieved. In this case, I could confirm that the result value is the same with the following Node.js script.
const obj = {key: "value"};
const payload = new Buffer(JSON.stringify(obj)).toString('base64');
// or const payload = Buffer.from(JSON.stringify(obj)).toString('base64');
console.log(payload) // eyJrZXkiOiJ2YWx1ZSJ9
Reference:
base64Encode(data)
Buffer is used in nodeJS
The equivilant in client side JavaScript is array buffer
In order to make one from a string, you have to make a uint8array or Uint16Array from a new array buffer object with the length of the string, then loop through the array and add the char code values from the string at the index, and return the buffer
A function
function Buffer(str) {
var buf= new ArrayBuffer(str.length/*multiply by 2 for higher chars*/)
var ar = new Uint8Array(buf) //use Uint16Array etc for larger, i=0
for(i=0;i< ar.length;i++) ar[i] = str.charCodeAt(I)
return buf
}
I'm trying to write in a text file, but not at the end like appendFile() do or by replacing the entiere content...
I saw it was possible to chose where you want to start with start parameter of fs.createwritestream() -> https://nodejs.org/api/fs.html#fs_fs_createwritestream_path_options
But there is no parameter to say where to stop writting, right ? So it remove all the end of my file after I wrote with this function.
const fs = require('fs');
var logger = fs.createWriteStream('result.csv', {
flags: 'r+',
start: 20 //start to write at the 20th caracter
})
logger.write('5258,525,98951,0,1\n') //example a new line to write
Is there a way to specify where to stop writting in the file to have something like:
....
data from begining
....
5258,525,98951,0,1
...
data till the end
...
I suspect you mean, "Is it possible to insert in the middle of the file." The answer to that is: No, it isn't.
Instead, to insert, you have to:
Determine how big what you're inserting is
Copy the data at your insertion point to that many bytes later in the file
Write your data
Obviously when doing #2 you need to be sure that you're not overwriting data you haven't copied yet (either by reading it all into memory first or by working in blocks, from the end of the file toward the insertion point).
(I've never looked for one, but there may be an npm module out there that does this for you...)
You could read/parse your file at first. Then apply the modifications and save the new file.
Something like:
const fs = require("fs");
const fileData = fs.readFileSync("result.csv", { encoding: "utf8" });
const fileDataArray = fileData.split("\n");
const newData = "5258,525,98951,0,1";
const index = 2; // after each row to insert your data
fileDataArray.splice(index, 0, newData); // insert data into the array
const newFileData = fileDataArray.join("\n"); // create the new file
fs.writeFileSync("result.csv", newFileData, { encoding: "utf8" }); // save it
I am trying to create an HMAC signature in Postman using a pre-request script. Without going too far into the details of implementation,
I have confirmed that my means for generating the signature is messed up. I can see what the expected result should be with a proof of concept example but I’m missing something somewhere and cannot tell if it is in the conversion. I’ve read around from other questions on SO that binary is the default provided by cryptojs internally and that simply calling for the hash is the equivalent of asking for the digest with conversions completed for you. Here is the code I’m trying to run in postman and the working implementation code as shown in nodeJS.
var CryptoJS = require("crypto-js");
const d = new Date();
const timestamp = d.getTime();
const postData = {};
postData.nonce = 100; //timestamp * 1000; //nanosecond
postman.setEnvironmentVariable('nonce', postData.nonce);
const secret = CryptoJS.enc.Base64.parse(pm.environment.get("apiSecret"));
const path = pm.globals.get("balanceMethod");
const message = CryptoJS.SHA256( encodeURI(postData.nonce + postData)) ; // ...
const hmacDigest = CryptoJS.HmacSHA512(path + message, secret);
postman.setEnvironmentVariable('API-Signature', CryptoJS.enc.Base64.stringify(hmacDigest));
console.log(CryptoJS.enc.Base64.stringify(hmacDigest));
Does this apply to my situation in that I’d need to convert my sha256 message into a bytes array in order to work?
Reference code for building implementation that does work with nodeJS:
const getMessageSignature = (path, request, secret, nonce) => {
const message = qs.stringify(request);
const secret_buffer = new Buffer(secret, 'base64');
const hash = new crypto.createHash('sha256');
const hmac = new crypto.createHmac('sha512', secret_buffer);
const hash_digest = hash.update(nonce + message).digest('binary');
const hmac_digest = hmac.update(path + hash_digest, 'binary').digest('base64');
return hmac_digest;
};
Same reference code for building implementation in python3:
req['nonce'] = 100 #int(1000*time.time())
postdata = urllib.parse.urlencode(req)
# Unicode-objects must be encoded before hashing
encoded = (str(req['nonce']) + postdata).encode()
message = urlpath.encode() + hashlib.sha256(encoded).digest()
signature = hmac.new(base64.b64decode(self.secret),
message, hashlib.sha512)
sigdigest = base64.b64encode(signature.digest())
The only post data I'm sending is the Nonce at this time and I've purposely set it to 100 to be able to replicate the result to fix the generated signature. Seems close but not matching result. The python and nodeJS do match expected results and work properly.
Check out the answer in this thread. It helped me with the my problem and may be what is happening in your case also. All it is necessary is break the input of the HMAC into two parts.
EDIT
I found this:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays
Basically if I have something like this:
struct someStruct {
unsigned long id;
char username[16];
float amountDue;
};
on client side I can do:
var ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = function (e) {
var buffer = e.data;
var data_view = new DataView(buffer);
// ... read the data into the buffer ...
var idView = data_view.getUint32(0);
var usernameView = data_view.getUint32(4);
var amountDueView = data_view.getFloat32(20);
};
The problem is that I want to convert them to normal Javascript objects (numbers, strings etc).
Original question
I would send data via websocket packed using a C struct, and unpack on browser using Javascript.
I know modules exists for node.js, but I can't find nothing client-side.
If you're familiar with Python struct, then you may like structjs. It's my attempt at porting Python struct to javascript. As it is, it's for Node, but a client port should be easy.
You won't have issues with alignment or padding (you can specify those explicitly though) for that structure, but you may need to indicate little-endian (by the '<' in the format string) if that's your flavour. You might do like so (I haven't tested this example in any way):
let struct = require("./struct") // Node specific, you need to wrap it.
let someStruct = struct('<I16sf') // This is your struct definition
let ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = e => {
// Unpack using the structure definition. Unpack takes an ArrayBuffer.
let [id, username, amountDue] = someStruct.unpack(e.data);
// Use data...
};
Ok,
https://www.npmjs.com/package/c-struct looks like what you want.
Good luck!
Ok, after some researching, I finally decided this could not be a good idea:
https://justin.harmonize.fm/development/2013/04/28/a-slower-js-msgpack.html
Shortly: javascript is slow in decoding.
Probably it's just more simple to use JSON and Content-Encoding: gzip, if it does not slow down your web app.