Develop Reference

Dialogflow Fulfilment webhook call failed

I am new to dialogflow fulfillment and I am trying to retrieve news from news API based on user questions. I followed documentation provided by news API, but I am not able to catch any responses from the search results, when I run the function in console it is not errors. I changed the code and it looks like now it is reaching to the newsapi endpoint but it is not fetching any results. I am utilizing https://newsapi.org/docs/client-libraries/node-js to make a request to search everything about the topic. when I diagnoise the function it says " Webhook call failed. Error: UNAVAILABLE. "
'use strict';
const functions = require('firebase-functions');
const {WebhookClient} = require('dialogflow-fulfillment');
const {Card, Suggestion} = require('dialogflow-fulfillment');
const http = require('http');
const host = 'newsapi.org';
const NewsAPI = require('newsapi');
const newsapi = new NewsAPI('63756dc5caca424fb3d0343406295021');
process.env.DEBUG = 'dialogflow:debug';
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((req, res) =>
{
// Get the city
let search = req.body.queryResult.parameters['search'];// search is a required param
// Call the weather API
callNewsApi(search).then((response) => {
res.json({ 'fulfillmentText': response }); // Return the results of the news API to Dialogflow
}).catch((xx) => {
console.error(xx);
res.json({ 'fulfillmentText': `I don't know the news but I hope it's good!` });
});
});
function callNewsApi(search)
{
console.log(search);
newsapi.v2.everything
(
{
q: 'search',
langauge: 'en',
sortBy: 'relevancy',
source: 'cbc-news',
domains: 'cbc.ca',
from: '2019-12-31',
to: '2020-12-12',
page: 2
}
).then (response => {console.log(response);
{
let articles = response['data']['articles'][0];
// Create response
let responce = `Current news in the $search with following title is ${articles['titile']} which says that
${articles['description']}`;
// Resolve the promise with the output text
console.log(output);
}
});
}
Also here is RAW API response
{
"responseId": "a871b8d2-16f2-4873-a5d1-b907a07adb9a-b4ef8d5f",
"queryResult": {
"queryText": "what is the latest news about toronto",
"parameters": {
"search": [
"toronto"
]
},
"allRequiredParamsPresent": true,
"fulfillmentMessages": [
{
"text": {
"text": [
""
]
}
}
],
"intent": {
"name": "projects/misty-ktsarh/agent/intents/b52c5774-e5b7-494a-8f4c-f783ebae558b",
"displayName": "misty.news"
},
"intentDetectionConfidence": 1,
"diagnosticInfo": {
"webhook_latency_ms": 543
},
"languageCode": "en"
},
"webhookStatus": {
"code": 14,
"message": "Webhook call failed. Error: UNAVAILABLE."
},
"outputAudio": "UklGRlQqAABXQVZFZm10IBAAAAABAAEAwF0AAIC7AAACABAAZGF0YTAqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA... (The content is truncated. Click `COPY` for the original JSON.)",
"outputAudioConfig": {
"audioEncoding": "OUTPUT_AUDIO_ENCODING_LINEAR_16",
"synthesizeSpeechConfig": {
"speakingRate": 1,
"voice": {}
}
}
}
And Here is fulfillment request:
{
"responseId": "a871b8d2-16f2-4873-a5d1-b907a07adb9a-b4ef8d5f",
"queryResult": {
"queryText": "what is the latest news about toronto",
"parameters": {
"search": [
"toronto"
]
},
"allRequiredParamsPresent": true,
"fulfillmentMessages": [
{
"text": {
"text": [
""
]
}
}
],
"intent": {
"name": "projects/misty-ktsarh/agent/intents/b52c5774-e5b7-494a-8f4c-f783ebae558b",
"displayName": "misty.news"
},
"intentDetectionConfidence": 1,
"diagnosticInfo": {
"webhook_latency_ms": 543
},
"languageCode": "en"
},
"webhookStatus": {
"code": 14,
"message": "Webhook call failed. Error: UNAVAILABLE."
},
"outputAudio": "UklGRlQqAABXQVZFZm10IBAAAAABAAEAwF0AAIC7AAACABAAZGF0YTAqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA... (The content is truncated. Click `COPY` for the original JSON.)",
"outputAudioConfig": {
"audioEncoding": "OUTPUT_AUDIO_ENCODING_LINEAR_16",
"synthesizeSpeechConfig": {
"speakingRate": 1,
"voice": {}
}
}
}
Also here is the screenshot from the firebase console.
Can anyone guide me what is that I am missing in here?

The key is the first three lines in the error message:
Function failed on loading user code. Error message: Code in file index.js can't be loaded.
Did you list all required modules in the package.json dependencies?
Detailed stack trace: Error: Cannot find module 'newsapi'
It is saying that the newsapi module couldn't be loaded and that the most likely cause of this is that you didn't list this as a dependency in your package.json file.
If you are using the Dialogflow Inline Editor, you need to select the package.json tab and add a line in the dependencies section.
Update
It isn't clear exactly when/where you're getting the "UNAVAILABLE" error, but one likely cause if you're using Dialogflow's Inline Editor is that it is using the Firebase "Spark" pricing plan, which has limitations on network calls outside Google's network.
You can upgrade to the Blaze plan, which does require a credit card on file, but does include the Spark plan's free tier, so you shouldn't incur any costs during light usage. This will allow for network calls.
Update based on TypeError: Cannot read property '0' of undefined
This indicates that either a property (or possibly an index of a property) is trying to reference against something that is undefined.
It isn't clear which line, exactly, this may be, but these lines all are suspicious:
let response = JSON.parse(body);
let source = response['data']['source'][0];
let id = response['data']['id'][0];
let name = response['data']['name'][0];
let author = response['author'][0];
let title = response['title'][0];
let description = response['description'][0];
since they are all referencing a property. I would check to see exactly what comes back and gets stored in response. For example, could it be that there is no "data" or "author" field in what is sent back?
Looking at https://newsapi.org/docs/endpoints/everything, it looks like none of these are fields, but that there is an articles property sent back which contains an array of articles. You may wish to index off that and get the attributes you want.
Update
It looks like that, although you are loading the parameter into a variable with this line
// Get the city and date from the request
let search = req.body.queryResult.parameters['search'];// city is a required param
You don't actually use the search variable anywhere. Instead, you seem to be passing a literal string "search" to your function with this line
callNewsApi('search').then((output) => {
which does a search for the word "search", I guess.
You indicated that "it goes to the catch portion", which indicates that something went wrong in the call. You don't show any logging in the catch portion, and it may be useful to log the exception that is thrown, so you know why it is going to the catch portion. Something like
}).catch((xx) => {
console.error(xx);
res.json({ 'fulfillmentText': `I don't know the news but I hope it's good!` });
});
is normal, but since it looks like you're logging it in the .on('error') portion, showing that error might be useful.

The name of the intent and the variable I was using to make the call had a difference in Casing, I guess calls are case sensitive just be aware of that

pouchDB query() bug when updating documents

Let's say I have these three documents:
{ "_id": "11111", "type": "template", "name": "person" }
{ "_id": "22222", "type": "template", "name": "place" }
{ "_id": "33333", "type": "template", "name": "thing" }
I have a cloud database and then I have a device with pouchDB syncing from that database.
These are the steps that I do:
I sync both databases together. So now I have the most recent versions of this document on my device.
I run the below query and I get back all three templates like so:
Code
var template_obj = {};
return device_db.query('filters/templates')
.then((templates) => {
for (let t of templates.rows) templates_obj[t.id] = true;
return templates_obj;
});
filters/templates
function (doc) {
if(doc.type == "template")
emit(doc._id);
}
return
{ "11111": true, "22222": true, "33333": true }
I update template: person on cloud. And then I update it again. So 2 revisions have gone by without syncing to my device.
I sync with my device.
Now when I run the same query and I only get back the document I edited. Which is weird because I haven't touched any of the other documents. The same view returns the expected results on the cloud but not on the device.
return
{"11111": true}
If I do the following code however, all templates come back as normal and the same _rev from the cloud show up on the device. Meaning the sync was successful and view is getting confused.
new code
return device_db.allDocs({conflicts: true})
.then((data) => {
for (let d of data.rows) {
if(d.doc.type == "template") {
templates_obj[d.doc._id] = true;
}
}
return templates_obj;
}).catch((err) => {
console.log(JSON.stringify(err));
})
I'm starting to believe this is a bug because if I destroy my database and do these steps again, I can reproduce this issue.

After realizing you are using React Native, I think this actually has to do with PouchDB in React Native, and it's indeed a bug. There are several reports of that behavior:
https://github.com/pouchdb/pouchdb/issues/7219
https://github.com/pouchdb/pouchdb/issues/7188
https://github.com/pouchdb/pouchdb/issues/7293

[edit: Seems to be a bug in PouchDB with React Native. I leave this answer because it might be helpful in other ways.]
I suspect it's some side effect with the global variable template_obj you are using. Try to console.log(templates.rows) directly instead of storing it in a variable in the top scope, or use Array.reduce() to avoid side effects. Then you'd always get the correct view results.
This is step by step code:
return device_db.query('filters/templates')
.then(templates => templates.rows) // Take only the rows into account.
.then(rows => rows.map(row => row.id) // Extract the id. If you wanted the name instead this would be possible with a slightly different view.
// I think it would suffice to log the result right now,
// but if you really want to have a single object with boolean values,
// you can do the following:
.then(ids => ids.reduce((asObject, id) => { // Use Array.reduce() here to avoid any potential side effects.
asObject[id] = true;
return asObject;
}, {})
.then(asObject => { console.log(asObject); }; // Debug the result.
Or more concise with ES2015+:
return device_db.query('filters/templates')
.then(({rows}) => rows.reduce((acc, {id}) => ({...acc, [id]: true }), {}))
.then(result => console.log(result))
By the way: You could also use other strategies to "filter" your documents, as it's not necessary to emit the _id. Instead you can use the key and/or value for "secondary indexes":
{
"_id": "_design/docs",
"views": {
"byType": "function(doc) { emit(doc.type); }",
"templatesByName": "function(doc) { if (doc.type === 'template') emit(doc.name); }",
"byTypeAndName": "function(doc) { emit([doc.type, doc.name], doc.name); }
}
}
you can use docs/byType as an universal view for other doc types too. Just call it with db.query('docs/byType', { key: 'template' })
If you want the templates sorted by name, use db.query('docs/templatesByName') or db.query('docs/byTypeAndName', { startkey: ['template'], endkey: ['template', {}]}).
A word of caution: This is all untested and just from memory, so some brackets might be missing in the code, or some bugs might hide in there.

It's not a bug in PDB, it's about outdated unfortunately components in pouchdb-react-native.
Confirmed way is to combine pouchdb-react-native yourself like this - then queries work as expected:
import PouchDB from 'pouchdb-core';
PouchDB
.plugin(require('pouchdb-adapter-asyncstorage').default)
.plugin(require('pouchdb-adapter-http'))
.plugin(require('pouchdb-mapreduce'))
.plugin(require('pouchdb-replication'))
.plugin(require('pouchdb-authentication'));
const localDB = new PouchDB(localDBname, {adapter: 'asyncstorage', auto_compaction: true});
This way one can be sure that all components are the latest.

Firebase multi location update, delete redundant data as part of update

I'm using firebase multi location updates to update a number of locations in my database.
One of the update paths contains a value that could be changed as part of the update therefore creating a new record at the new path location which is correct. This then leaves a redundant data at the original location that I need to delete as part of the update. Can someone give me a pointer on how to delete the redundant data as part of the update?
I am using the code below to create my update: -
var updatedUserDataContent = {
title: $scope.postToUpdate.title
commenter: $scope.postToUpdate.commenter
};
updatedUserData["Posts/" + $scope.postToUpdate.$id] = updatedUserDataContent;
updatedUserData["UserPosts/" + $scope.postToUpdate.commenter + "/" + $scope.postToUpdate.$id] = updatedUserDataContent;
The commenter could be changed by a user on the record before saving leaving the redundant data in the original commenter node. I have tried using the security rules but these stop data being created rather than deleted.
Edit 1
JSON as requested.
Before the update I have
{
"Not assigned" : {
"-KNfs3OhBmbb93w9VEW-" : {
"title": "vytg",
"commenter": "Not assigned"
}
}
}
After the update (after changing commenter from "Not assigned" to "User A") I have
{
"Not assigned": {
"-KNfs3OhBmbb93w9VEW-": {
"title": "vytg",
"commenter": "Not assigned"
}
},
"User A" : {
"-KNfs3OhBmbb93w9VEW-" : {
"title": "vytg",
"commenter": "User A"
}
}
}
but I want just
{
"User A" : {
"-KNfs3OhBmbb93w9VEW-" : {
"title": "vytg",
"commenter": "User A"
}
}
}

You're trying to move/rename a node, which is not an operation the database supports.
Since the new data is written correctly, all that is needed is that you also clear out the old node. Putting null in for that location will take care of that:
updatedUserData["Not assigned"] = null;

Assistance needed with JSON

I'm creating a JSON file to hold information for commands.
The JSON file holds the trigger and repsonse for the commands, I have one in the file, but I want to add more, but when I do I get an error
{
"trigger": "twitter",
"repsonse": "https://www.twitter.com/Fhaelin"
}
{
"trigger": "test",
"repsonse": "This is a test command"
}
I get thrown errors and i dont know why
Here's the code I'm using to read it:
bot.on("message", function(message) {
var input = message.content.toLowerCase();
if (input === prefix + Commands.trigger)
{
bot.sendMessage(message, message.author + " : " + Commands.repsonse)
}
})
Whole code
http://hastebin.com/punabobisu.coffee

You get errors because that's an invalid JSON document. There can only be one top-level value in a JSON document.
To have a list of objects, put them in an array: [..., ...]:
[
{
"trigger": "twitter",
"response": "https://www.twitter.com/Fhaelin"
},
{
"trigger": "test",
"response": "This is a test command"
}
]
Side note: You have a consistent misspelling in your question: It's "response", not "repsonse". Only mentioning it because it'll come back to bite you at some stage if you actually put it in your code.

how to write json files in javascript

Ok, so I am programming a web operating system using js. I am using JSON for the file system. I have looking online for tutorials on JSON stuff for about a week now, but I cannot find anything on writing JSON files from a web page. I need to create new objects in the file, not change existing ones. Here is my code so far:
{"/": {
"Users/": {
"Guest/": {
"bla.txt": {
"content":
"This is a test text file"
}
},
"Admin/": {
"html.html": {
"content":
"yo"
}
}
},
"bin/": {
"ls": {
"man": "Lists the contents of a directory a files<br/>Usage: ls"
},
"cd": {
"man": "Changes your directory<br/>Usage: cd <directory>"
},
"fun": {
"man": "outputs a word an amount of times<br/>Usage: fun <word> <times>"
},
"help": {
"man": "shows a list of commands<br/>Usage: help"
},
"clear": {
"man": "Clears the terminal<br/>Usage: clear"
},
"cat": {
"man": "prints content of a file<br/>Usage: cat <filename>"
}
},
"usr/": {
"bin/": {
},
"dev/": {
}
}
}}

I think the better solution is to stringify your JSON, encode with base64 encoding and then send it to a server-side script (a PHP page, for instance) which could save this file. See:
var json = JSON.stringify(myJson);
var encoded = btoa(json);
You can use ajax for sending:
var xhr = new XMLHttpRequest();
xhr.open('POST','myServerPage.php',true);
xhr.setRequestHeader('Content-type','application/x-www-form-urlencoded');
xhr.send('json=' + encoded);
And in the server-side:
$decoded = base64_decode($_POST['json'])
$jsonFile = fopen('myJson.json','w+');
fwrite($jsonFile,$decoded);
fclose($jsonFile);

I'd take off the "/"'s from the keys then could split on "/" and walk the tree by shifting values off the result. For example, the following code will create the full path if it doesn't already exist, but preserving the folder & contents if it does.
var fs = {
"bin": {
"mkdir": function(inPath) {
// Gets rid of the initial empty string due to starting /
var path = inPath.split("/").slice(1);
var curFolder = fs;
while(path.length) {
curFolder[path[0]] = curFolder[path[0]] || {};
curFolder = curFolder[path.shift()];
}
}
}
}
fs.bin.mkdir("/foo/bar");
console.log(JSON.stringify(fs, function(key,val) {
if(key === 'mkdir') return undefined;
return val;
}, 2));
Output:
{
"bin": {},
"foo": {
"bar": {}
}
}
As others have mentioned, rather than building the JSON object by hand with strings, to avoid syntax errors (and frustration), building it through code then using JSON.stringify to get the final result would likely be simpler.