Develop Reference

JSON/EJSON Error w/ Stripe API - Invoice Webhook Event

Update w/ Additional Info - 18 Dec.
As I test further it appears that the error is isolated to the "this.response.end(json);" See notes below:
stripe non-invoice event:
a) "this.response.end(EJSON.stringify(obj, {indent: true}));" works & returns 200
b) "this.response.end(EJSON.stringify(obj));" no error on local server, error on stripe dashboard with 'unable to connect' and does not return code 200
stripe invoice event:
a) "this.response.end(EJSON.stringify(obj, {indent: true}));" throws errors - see details below.
b) "this.response.end(EJSON.stringify(obj));" no error on local server, error on stripe dashboard with 'unable to connect' and does not return code 200
Any insight would be appreciated.
I am having some issues with the Stripe API and am seeking some help or insight. I am using ngrok to test the Stripe webhooks locally and everything works fine until I make a request to any Stripe 'invoice' event type (e.g. invoice.payment_succeeded). When I run a test on any 'invoice' event type I get several errors:
my running application breaks (i.e. requires me to type meteor run in the terminal to restart the app)
I receive this server side error message in my terminal:
///error message start///
events.js:183
throw er; // Unhandled 'error' event
^
Error: write after end
at write_ (_http_outgoing.js:622:15)
at ServerResponse.write (_http_outgoing.js:617:10)
at IncomingMessage.ondata (_stream_readable.js:639:20)
at emitOne (events.js:116:13)
at IncomingMessage.emit (events.js:211:7)
at IncomingMessage.Readable.read (_stream_readable.js:475:10)
at flow (_stream_readable.js:846:34)
at resume_ (_stream_readable.js:828:3)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
///error message end///
On my Stripe dashboard I get an error: 'Test webhook error: Unable to connect'
Again this only breaks when I make a request to any Stripe 'invoice' event type (e.g. invoice.payment_succeeded).
I contacted Stripe to see if there was anything else I should consider but they said things were fine on their end.
One final point, to get ngrok running I use 'ngrok http 3000'.
With that said my server side webhook code is below. If anyone has any insight on what would be causing this error, any insight perspective would be appreciated.
///server side webhook code start///
Router.route('/webhooks/stripe', { where: 'server' })
.get(function () {
console.log('getter');
this.response.end('closing...');
})
.post(function () {
console.log('post function initiated');
// stores payload as string
var obj = this.request.body;
console.log("print obj var");
console.log(obj);
// saves as indented string to send as response
var json = EJSON.stringify(obj, {indent: true});
this.response.writeHead(200, {
'Content-Length': json.length,
'Content-Type': 'application/json'
});
this.response.end(json);
})
.put(function () {
console.log('put');
});
///server side webhook code end///

I was able to solve this issue. For reference and posterity I removed the commented code below and manually added my status code (i.e. this.response.statusCode = 200;). This got things working and resolved the issue.
///code snippet start///
var json = EJSON.stringify(obj, {indent: true});
//this is the code i removed
// this.response.writeHead(200, {
// 'Content-Length': json.length,
// 'Content-Type': 'application/json'
// });
//this is the code i added to send a manual status code
this.response.statusCode = 200;
this.response.end(json);
///code snippet end///

Redis connection is lost after multiple calls to function

The program I am writing is a status display screen for alarms, each of which is represented by a channel.
When the server is started (run on a vagrant virtual machine), an Influx database is accessed, the data (comprising of 1574 'channels') is processed and put into a Redis database. This runs fine and the GUI is displayed with no issues when the webpage is refreshed, although it takes a long time to load (up to 20s), and nearly all of this time is spent in the method below.
However, after a few refreshes/moving around the site, it often crashes with the following error:
{ AbortError: Redis connection lost and command aborted. It might
have been processed.
at RedisClient.flush_and_error (/vagrant/node_modules/redis/index.js:362:23)
at RedisClient.connection_gone (/vagrant/node_modules/redis/index.js:664:14)
at RedisClient.on_error (/vagrant/node_modules/redis/index.js:410:10)
at Socket. (/vagrant/node_modules/redis/index.js:279:14)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at onwriteError (_stream_writable.js:417:12)
at onwrite (_stream_writable.js:439:5)
at _destroy (internal/streams/destroy.js:39:7)
at Socket._destroy (net.js:568:3) code: 'UNCERTAIN_STATE', command: 'HGETALL', args: [
'vista:hash:Result:44f59707-c873-11e8-93b9-7f551d0bdd1f' ], origin:
{ Error: Redis connection to 127.0.0.1:6379 failed - write EPIPE
at WriteWrap.afterWrite (net.js:868:14) errno: 'EPIPE', code: 'EPIPE', syscall: 'write' } }
This error is displayed 1574 times (once for each channel), and occurs when the program reaches this function:
Result.getFormattedResults = async function (cycle) {
const channels = await Channel.findAndLoad()
const formattedResults = await mapAsyncParallel(channels, async channel => {
const result = await this.findAndLoadByChannel(channel, cycle)
const formattedResult = await result.format(channel)
return formattedResult
})
return formattedResults
}
mapAsyncParallel() is as follows:
export const mapAsyncParallel = (arr, fn, thisArg) => {
return Promise.all(arr.map(fn, thisArg))
}
findAndLoadByChannel() finds the channel and loads it with this line:
const resultModel = await this.load(resultId)
And format() takes the model and outputs the data as in a JSON format
There are two 'fetch(...)' commands (which are needed and cannot be combined) in the front end, and the problem rarely occurs when I comment out one of them (either one). This is making me think it could be a max memory or max connections problem? (increasing maxmemory in the config file didn't help). Or a problem with using so many promises (a concept I am fairly new to).
This has only started to occur as I have added more functionality and I assume the function needs optimizing but I have taken over this project from someone else and am still quite new to node.js and redis.
Versions:
Vagrant: 2.0.1
Ubuntu: 16.04.5
Redis: 4.0.9
Node: 8.12.0
npm: 5.7.1

I've now moved all the 'getting' of the data (from redis) to the server side channels.controller file.
So, where before I would have:
renderPage: async (req, res) => {
res.render('page')
},
I now have a method like:
renderPage: async (req, res) => {
const data1 = getData1()
const data2 = getData2()
res.render('page', {data1, data2})
},
(Don't worry, these aren't my actual variable names)
Where the two 'data' variables were previously retrieved using the 'fetch' method.
I export the data once it's loaded into redis, and import it in the controller file, where I have the getters to combine it all into one return array.
The pages now take milliseconds to refresh and I haven't had any crashes

Error _LAMBDA_LOG_FD in AWS lambda function with nodejs 4.3

i have an error with AWS lambda function with nodejs 4.3,
[ERROR] (get_int_from_env#src/lambda/runtime.c:493 errno: None) temp_str = getenv(env_name) failed.
[ERROR] (get_int_from_env#src/lambda/runtime.c:493 errno: None) temp_str = getenv(env_name) failed.
[ERROR] (runtime_init#src/lambda/runtime.c:532 errno: None) ((fd =get_int_from_env("_LAMBDA_LOG_FD", 1))) >= 0 failed. invalid file descriptor -1
[ERROR] (runtime_init#src/lambda/runtime.c:560 errno: None) Runtime internal error
I think _LAMBDA_LOG_FD is environment variable set internally that indicates the file descriptor for logging.
I did not found anything in the documetation
here!
For information, my lambda function uses cluster with fork process.
Someone can give a clue to resolve this issue ? Do you need more informations? and if yes , which one?
Thanks a lot

I was getting this issue in python flask.
I have resolved it by changing main function.
Previously main function was defined as
app = Flask(__name__)
app.run()
I have replaced it with
if __name__=='__main__':
app.run()
one more thing parameter_depth variable added in "zappa_settings.json" file.
{
"dev": {
"app_function": "run.app",
"s3_bucket": "Your_bucket_name",
"parameter_depth":1
}
}

Node.js - HTTPS PFX Error: Unable to load BIO

I'm trying to make a HTTPS request-promise. I already know that the PFX is good and that is not the issue (I have a similar sample app working).
I am doing the following:
var request = require('request-promise');
...
options.pfx = fs.readFileSync('myfile.pfx');
options.passphrase = 'passphrase';
I am passing my options into an request.
request.post(options);
I then try to build the request I get the following error:
_tls_common.js:130
c.context.loadPKCS12(pfx, passphrase);
^
Error: Unable to load BIO
at Error (native)
at Object.createSecureContext (_tls_common.js:130:17)
at Object.exports.connect (_tls_wrap.js:955:21)
at Agent.createConnection (https.js:73:22)
at Agent.createSocket (_http_agent.js:174:16)
at Agent.addRequest (_http_agent.js:143:23)
at new ClientRequest (_http_client.js:133:16)
at Object.exports.request (http.js:31:10)
at Object.exports.request (https.js:163:15)
at Request.start (/Users/filomeno/workspace/sla-crawler/node_modules/request/request.js:747:30)
at Request.write (/Users/filomeno/workspace/sla-crawler/node_modules/request/request.js:1369:10)
at end (/Users/filomeno/workspace/sla-crawler/node_modules/request/request.js:561:16)
at Immediate._onImmediate (/Users/filomeno/workspace/sla-crawler/node_modules/request/request.js:589:7)
at processImmediate [as _immediateCallback] (timers.js:374:17)
I have a sample app where the same code works.
I've tried to convert to .p12 without success.
Does anyone have an idea what this error might refer to?
Edit:
I'm using lodash to do a merge of 2 objects with dinamic properties and static properties
_.merge(options, _this.requestOptions);
And that was causing the problem

Looking at the nodejs source code (specifically this file https://github.com/nodejs/node/blob/master/src/node_crypto.cc)
the error is thrown by this function
// Takes .pfx or .p12 and password in string or buffer format
void SecureContext::LoadPKCS12(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
...
In line 964
in = LoadBIO(env, args[0]);
if (in == nullptr) {
return env->ThrowError("Unable to load BIO");
}
Where the LoadBIO returns null
// Takes a string or buffer and loads it into a BIO.
// Caller responsible for BIO_free_all-ing the returned object.
static BIO* LoadBIO(Environment* env, Local<Value> v) {
HandleScope scope(env->isolate());
if (v->IsString()) {
const node::Utf8Value s(env->isolate(), v);
return NodeBIO::NewFixed(*s, s.length());
}
if (Buffer::HasInstance(v)) {
return NodeBIO::NewFixed(Buffer::Data(v), Buffer::Length(v));
}
return nullptr;
}
Perhaps the buffer is somehow not readable? Also it seems that the function is expecting an utf-8 encoded string.
Some ideas:
Are you sure the path to the file is correct?
Maybe encoding issue? Did you try to set fs.readFileSync() encoding explicitly?
Try with fs.readFile(<filename>, <encoding>, function(error, data){}) to see if it throws an error?

Node js + Sails js + websocket app crashing every few hours

I am having a problem similar to socket.io issue using sails.js. Every once in a while (once per day, or even few hours, it varies), a visitor to the web site/app will crash Node, seemingly due to the way his websocket client tries to connect. Anyway, here's the crash log:
debug: Lowering sails...
/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/express/node_modules/connect/lib/utils.js:216
return 0 == str.indexOf('s:')
^
TypeError: Cannot call method 'indexOf' of undefined
at exports.parseSignedCookie (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/express/node_modules/connect/lib/utils.js:216:19)
at Manager.socketAttemptingToConnect (/Volumes/Two/Sites/lsdfinder/node_modules/sails/lib/hooks/sockets/authorization.js:35:26)
at Manager.authorize (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:910:31)
at Manager.handleHandshake (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:786:8)
at Manager.handleRequest (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:593:12)
at Server.<anonymous> (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:119:10)
at Server.EventEmitter.emit (events.js:98:17)
at HTTPParser.parser.onIncoming (http.js:2076:12)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:120:23)
at Socket.socket.ondata (http.js:1966:22)
9 Oct 10:42:24 - [nodemon] app crashed - waiting for file changes before starting...
In config/sockets.js, authorization is set to true. Not sure what else to do, where to fix this. Any suggestions? I can read the Sails docs too, but this appears to be a problem in Express/Connect, no? Thanks.
...René

The problem is that once every so often, a client will connect that has no cookies. Sails.js is using util.parseSignedCookie() from Connect without checking for errors, and therefore an error is thrown. This is what it looks like in Sails:
if (handshake.headers.cookie) {
handshake.cookie = cookie.parse(handshake.headers.cookie);
handshake.sessionID = parseSignedCookie(handshake.cookie[sails.config.session.key], sails.config.session.secret);
}
If you take a look into the cookieParser() middleware of Connect, you can see error checking is required:
if (cookies) {
try {
req.cookies = cookie.parse(cookies);
if (secret) {
req.signedCookies = utils.parseSignedCookies(req.cookies, secret);
req.signedCookies = utils.parseJSONCookies(req.signedCookies);
}
req.cookies = utils.parseJSONCookies(req.cookies);
} catch (err) {
err.status = 400;
return next(err);
}
}
I've created a Gist here that fixes the problem, and will submit a pull request to Sails.js when I have the time. The Gist uses Connect's cookieParser() middleware to automatically handle errors. If you want to use this, modify this file in your modules folder:
node_modules/sails/lib/hooks/sockets/authorization.js

If you are doing a crossdomain request, you could turn off authorization.
In *site_dir/config/sockets.js* set authorization to false. One way of doing it. You can also call your api with something like this
bash
**http://localhost:1337?cookie=smokeybear**
Its is in the comments on the sockets.js file.