Develop Reference

Dialogflow Webhook (Webhook call failed. Error: 500 Internal Server Error)

I've followed this tutorial's code (https://dialogflow.com/docs/getting-started/basic-fulfillment-conversation) to return results of an API to dialog flow. However my webhook keeps failing. Can someone help me figure out why?
Here's one of the failed conversations:
Here's my code:
'use strict';
const http = require('http');
exports.Hadoop = (req, res) => {
// Get name node server from the request
let nameNodeServer = req.body.queryResult.parameters['nameNodeServer']; // nameNodeServer is a required param
// Call the Hadoop API
getNameNodeInfo(nameNodeServer).then(function(output) {
res.json({ 'fulfillmentText': output }); // Return the results to Dialogflow
}).catch(() => {
res.json({ 'fulfillmentText': 'getNameNodeInfo() Error'- });
});
};
function getNameNodeInfo (nameNodeServer) {
return new Promise((resolve, reject) => {
// Create url for the HTTP request to get the name node info
let url = 'http://' + nameNodeServer + '[rest of url]';
// Make the HTTP request to get the name node info
http.get(url, (res) => {
let body = ''; // var to store the response chunks
res.on('data', (chunk) => {body += chunk; });
res.on('end', () => {
// After all the data has been received, parse the JSON for desired data
let response = JSON.parse(body);
let beans = response['beans'][0];
// Create response
let output = `Percent Used: ${beans['PercentUsed']}`;
// Resolve the promise with the output text
console.log(output);
resolve(output);
});
res.on('error', (error) => {
console.log(`Error calling the Hadoop API: ${error}`);
reject();
});
});
});
}
I believe the getNameNodeInfo function and the retrieval of the name node server are correct, as they logged the correct output in debugging.
Diagnostic Info:

I contacted someone at Dialogflow and this was their response.
Thank you for providing all the information. I have observed in your
code that you have used http requests instead of https. The service
must use HTTPS and the URL must be publicly accessible in order for
the fulfillment to function. Dialogflow does not support self-signed
SSL certs. For information on SSL setup, please refer to this :
https://developers.google.com/web/fundamentals/security/encrypt-in-transit/enable-https

We've had a somewhat different, but related, issue:
Internal Server Error when running an agent.
“status”: {
“code”: 500,
“errorType”: “internal_server_error”,
“errorDetails”: “Internal Server Error”
},
This error was not caused by any changes we introduced. We are using that agent in a dev version of an app and one morning it stopped working.
We tested by creating a .zip and restoring into a new agent. The new agent would work properly, but we would continue to get the 500 error on the agent hooked into our dev app. We submitted a help request and overnight the error got resolved. We suspect that DialogFlow team had to manually reboot the server or something similar.

Refresh api query with Node.JS & Heroku

I'm building a messenger bot that queries an API once a day and sends the information contained there to users. it's hosted on Heroku and uses Node.js
I'm currently using this to query the API:
var request = require('request');
//url for classes JSON
var url = 'https://example.api';
//get JSON, parse it and store it in jsonstorage variable
request(url, (error, response, body)=> {
if (!error && response.statusCode === 200) {
jsonstorage = JSON.parse(body)
console.log("Got a response")
} else {
console.log("Got an error: ", error, ", status code: ", response.statusCode)
}
})
The problem with this is that it doesn't refresh the API query ever, so the values returned are always the same. If I try and put it within the functions that send the data once a day though, it claims the jsonstorage variable is 'undefined'.
How can I refresh the query regularly?
Thanks!

Found an answer to this myself - the issue I was running into was that request is asynchronous, so I needed to wait for request to receive a result before running the rest of the script.

Serving New Data With Node.js

There may already by an answer to this question but I was unable to find it.
Let's say I have a Node.js webpage doing somewhat time-consuming API calls and computations:
var request = require('request'),
Promise = require('es6-promise').Promise,
is_open = require('./is_open');
// Fetch the name of every eatery
var api_url = 'url of some api';
request(api_url, function (error, response, body) {
if (error) {
console.log(error);
} else if (!error && response.statusCode == 200) {
// Good to go!
var results = JSON.parse(body).events;
results.(function (result) {
// This line makes its own set of API calls
is_open(result
.then(function (output) {
console.log(output);
if (output == false) {
console.log('CLOSED\n');
} else {
console.log(output);
console.log();
}
})
.catch(console.error);
});
} else {
console.log('Returned an unknown error.');
console.log(error);
console.log(response);
console.log(body);
}
});
(I haven't yet created an actual web server, I'm just running the app locally through the command line.)
I want the web server to serve a loading page first to every user. Then, once the API calls are complete and the data is ready, it should send that data in a new webpage to the same user.
The reason I think there's an issue is because in order to serve a webpage, you must end with:
res.end();
Therefore ending the connection to that specific user.
Thanks for the help!

You must conceptually separate static content from dynamic content (later you will serve static with nginx or apache leaving only dynamic to node)
The best solution to your "problem" is to make the first webpage ask the data via AJAX once loaded. Ideally, your node app will return JSON to an ajax query from the first page, and js on the page will format the result creating DOM nodes.

How to handle ETIMEDOUT error?

How to handle etimedout error on this call ?
var remotePath = "myremoteurltocopy"
var localStream = fs.createWriteStream("myfil");;
var out = request({ uri: remotePath });
out.on('response', function (resp) {
if (resp.statusCode === 200) {
out.pipe(localStream);
localStream.on('close', function () {
copyconcurenceacces--;
console.log('aftercopy');
callback(null, localFile);
});
}
else
callback(new Error("No file found at given url."), null);
})
There are a way to wait for longer? or to request the remote file again?
What exactly can cause this error? Timeout only?

This is caused when your request response is not received in given time(by timeout request module option).
Basically to catch that error first, you need to register a handler on error, so the unhandled error won't be thrown anymore: out.on('error', function (err) { /* handle errors here */ }). Some more explanation here.
In the handler you can check if the error is ETIMEDOUT and apply your own logic: if (err.message.code === 'ETIMEDOUT') { /* apply logic */ }.
If you want to request for the file again, I suggest using node-retry or node-backoff modules. It makes things much simpler.
If you want to wait longer, you can set timeout option of request yourself. You can set it to 0 for no timeout.

We could look at error object for a property code that mentions the possible system error and in cases of ETIMEDOUT where a network call fails, act accordingly.
if (err.code === 'ETIMEDOUT') {
console.log('My dish error: ', util.inspect(err, { showHidden: true, depth: 2 }));
}

In case if you are using node js, then this could be the possible solution
const express = require("express");
const app = express();
const server = app.listen(8080);
server.keepAliveTimeout = 61 * 1000;
https://medium.com/hk01-tech/running-eks-in-production-for-2-years-the-kubernetes-journey-at-hk01-68130e603d76

Try switching internet networks and test again your code. I got this error and the only solution was switching to another internet.
Edit: I now know people besides me that have had this error and the solution was communicating with the ISP and ask them to chek the dns configuration because the http request were failing. So switching networks definitely could help with this.
That is why I will not delete the post. I could save people a few days of headaches (especially noobs like me).

Simply use a different network. Using a different network solved this issue for me within seconds.

JSONP call with server-side language as Javascript

I've been trying to use JSONP to get a JSON object from a server via a client-side call (on a different port). However, because my server is implemented in javascript using Node.js and Express, I haven't been able to find much on JSONP with Javascript on the server as most sites I found used php for server-side code.
I believe the issue is with how I set up the url with respect to the callback, which I'm a bit fuzzy on cause it's new to me.
On the server:
//Setting up server stuff
var express = require('express'),
app = express.createServer();
app.use(express.logger());
//Making a connection to the mongoDB to get the data that I want to display in the JSON object
new Db('prism',
new Server("127.0.0.1", 27017, {auto_reconnect: false}), {}).open(function(err, db) {
app.get('/getData', function(req, res) {
console.log('JSONPCALLBACK CALLED WITH RETURNDATA PASSED IN; SERVER SIDE');
if (typeof callback == 'function') {
console.log('callback is defined');
callback(returnData);
}
else {
console.log('callback is not defined');
}
}
});
And on the client:
$.ajaxSetup({ cache : false });
$.getJSON('http://127.0.0.1:1337/getData&callback=?', function(rtndata) {
console.log('SUCCESS');
console.log(rtndata);
});
embedded by the standard tags.
But I get the error:
GET http://127.0.0.1:1337/getData&callback=jQuery16108897686484269798_1311007334273?_=1311007334342 404 (Not Found)
The server is on port 1337 while the client is run through MAMP on localhost:8888. I'm not sure if its even a localhost related issue as I've been trying to get this setup running for a few days now.
I believe the issue has something to do with not writing this line, which is in php, into my server-side Javascript. Most of the JSONP examples I found had something like this. But I'm not sure.
if ($GET['callback'] != '')
$json = $GET['callback']."( $json )";
return $json;
Any help would be greatly appreciated. I apologize ahead of times for being super verbose.
Bests,
Cong

I think you have two problems. First is the 404. Completely separate from getting the client-side jQuery code to work, you need to make sure that you can issue a regular browser request (i.e. paste in that URL) and get back what you expect. I haven't used express, so it's hard for me to comment on why you'd be getting that, except to say that I don't see 1337 anywhere in your server-side code, just what appears to be the port number 27017.
The second problem is that you don't actually want to execute the callback on the server, just build the JSON response including the callback (string) prefixed to it.
So instead of this ...
if (typeof callback == 'function') {
console.log('callback is defined');
callback(returnData);
}
else {
console.log('callback is not defined');
}
try this:
if (callback) {
console.log('callback is defined');
res.write(callback + '(' + JSON.stringify(returnData) + ')');
}
else {
console.log('callback is not defined');
}
Hope this helps!

From http://api.jquery.com/jQuery.getJSON/ there is an example that includes 2 '?' in the URL.
you only have one, so try
$.getJSON('http://127.0.0.1:1337/getData?callback=?', function(rtndata) {
and see if that gets rid of your 404
then look #jimbojw suggestion for returning a proper jsonp formated responce.

Use this:
var express = require("express");
var server = express.createServer();
server.enable("jsonp callback");
server.get("/foo", function(req, res) {
// this is important - you must use Response.json()
res.json("hello");
});
jsonp with node.js express