I am using TranscriptLoggerMiddleware and CosmosDB to log my chatbot transcripts. We are trying to capture the user state information (user name, account number, account type, etc) as top level attributes in the transcript so that specific customers can easily be queried in the DB (if that information is just in the individual timestamp attributes of the document, they can't be queried).
Ideally I would just add the user state when I'm building the file, but I can't figure any way to access it since the logger is defined in index.js and TranscriptLoggerMiddleware only provides the activity to my function, not the full context. If anyone has a way to get the user state data via TranscriptLoggerMiddleware, let me know, that would solve this issue. Here is the customLogger code. Note that due to the function receiving both the user query and bot response, I couldn't get retrieving and resaving the transcript to work, so I'm overwriting the transcript from a local log object. Not trying to come up with a new approach here but if one would solve the overall issue I'd like to hear it.
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
const { CosmosDbPartitionedStorage } = require('botbuilder-azure');
const path = require('path');
/**
* CustomLogger, takes in an activity and saves it for the duration of the conversation, writing to an emulator compatible transcript file in the transcriptsPath folder.
*/
class CustomLogger {
/**
* Log an activity to the log file.
* #param activity Activity being logged.
*/
// Set up Cosmos Storage
constructor(appInsightsClient) {
this.transcriptStorage = new CosmosDbPartitionedStorage({
cosmosDbEndpoint: process.env.COSMOS_SERVICE_ENDPOINT,
authKey: process.env.COSMOS_AUTH_KEY,
databaseId: process.env.DATABASE,
containerId: 'bot-transcripts'
});
this.conversationLogger = {};
this.appInsightsClient = appInsightsClient;
this.msDelay = 250;
}
async logActivity(activity) {
if (!activity) {
throw new Error('Activity is required.');
}
// Log only if this is type message
if (activity.type === 'message') {
if (activity.attachments) {
try {
var logTextDb = `${activity.from.name}: ${activity.attachments[0].content.text}`;
} catch (err) {
var logTextDb = `${activity.from.name}: ${activity.text}`;
}
} else {
var logTextDb = `${activity.from.name}: ${activity.text}`;
}
if (activity.conversation) {
var id = activity.conversation.id;
if (id.indexOf('|') !== -1) {
id = activity.conversation.id.replace(/\|.*/, '');
}
// Get today's date for datestamp
var currentDate = new Date();
var day = currentDate.getDate();
var month = currentDate.getMonth()+1;
var year = currentDate.getFullYear();
var datestamp = year + '-' + month + '-' + day;
var fileName = `${datestamp}_${id}`;
var timestamp = Math.floor(Date.now()/1);
// CosmosDB logging (JK)
if (!(fileName in this.conversationLogger)) {
this.conversationLogger[fileName] = {};
this.conversationLogger[fileName]['userData'] = {};
this.conversationLogger[fileName]['botName'] = process.env.BOTNAME;
}
this.conversationLogger[fileName][timestamp] = logTextDb;
let updateObj = {
[fileName]:{
...this.conversationLogger[fileName]
}
}
// Add delay to ensure messages logged sequentially
await this.wait(this.msDelay);
try {
let result = await this.transcriptStorage.write(updateObj);
} catch(err) {
console.log(err);
this.appInsightsClient.trackTrace({message: `Logger Error ${err.code} - ${path.basename(__filename)}`,severity: 3,properties: {'botName': process.env.BOTNAME, 'error':err.body}});
}
}
}
}
async wait(milliseconds) {
var start = new Date().getTime();
for (var i = 0; i < 1e7; i++) {
if ((new Date().getTime() - start) > milliseconds) {
break;
}
}
}
}
exports.CustomLogger = CustomLogger;
Not being able to get user state in this function, I decided to try a few other approaches. The most promising was creating a separate "updateTranscript" function to grab the transcript, add user state, and save it back. But I think it was catching it only on user request and getting overidden again by local object on bot response. I added a delay to try to combat this, but it still didn't work. On my very first prompt of providing customer number user state data is getting stored on transcript, but at the next activity it is gone and never comes back (even though I can see it is supposedly getting written to DB). Here is that update function.
const { CosmosDbStorage } = require('botbuilder-azure');
var updateTranscript = async (context, userData, appInsightsClient) => {
const transcriptStorage = new CosmosDbStorage({
serviceEndpoint: process.env.COSMOS_SERVICE_ENDPOINT,
authKey: process.env.COSMOS_AUTH_KEY,
databaseId: process.env.DATABASE,
collectionId: 'bot-transcripts',
partitionKey: process.env.BOTNAME
});
var id = context.activity.conversation.id;
if (id.indexOf('|') !== -1) {
id = context.activity.conversation.id.replace(/\|.*/, '');
}
// Get today's date for datestamp
var currentDate = new Date();
var day = currentDate.getDate();
var month = currentDate.getMonth()+1;
var year = currentDate.getFullYear();
var datestamp = year + '-' + month + '-' + day;
var filename = `${datestamp}_${id}`;
var msDelay = 500;
await new Promise(resolve => setTimeout(resolve, msDelay));
var transcript = await transcriptStorage.read([filename]);
transcript[filename]['userData'] = userData
try {
await transcriptStorage.write(transcript);
console.log('User data added to transcript');
} catch(err) {
console.log(err);
appInsightsClient.trackTrace({message: `Log Updater Error ${err.code} - ${path.basename(__filename)}`,severity: 3,properties: {'botName': process.env.BOTNAME, 'error':err.body}});
}
return;
}
module.exports.updateTranscript = updateTranscript
I realize this approach is a bit of a cluster but I've been unable to find anything better. I know the Microsoft COVID-19 bot has a really nice transcript retrieval function, but I haven't been able to get any input from them on how that was accomplished. That aside, I'm quite happy to continue with this implementation if someone can help me figure out how to get that user state into the transcript without being overwritten or running into concurrency issues.
As to why I can't query an account number even via substring() function, here's an example of the documents data object. I have no idea which string to check for a substring, in this case 122809. I don't know what that timestamp could be. If this is stored at the top level (e.g. userData/accountNumber) I know exactly where to look for the value. For further context, I've displayed what I see after the first prompt for account number, where userData is populated. But it gets overidden on subsequent writes and I can't seem to get it back even with a delay in my updateTranscript function.
"document": {
"userData": {},
"botName": "AveryCreek_OEM_CSC_Bot_QA",
"1594745997562": "AveryCreek_OEM_CSC_Bot_QA: Hi! I'm the OEM CSC Support Bot! Before we get started, can you please provide me with your 6-digit Vista number? If you don't have one, just type \"Skip\".",
"1594746003973": "You: 122809",
"1594746004241": "AveryCreek_OEM_CSC_Bot_QA: Thank you. What can I help you with today? \r\nYou can say **Menu** for a list of common commands, **Help** for chatbot tips, or choose one of the frequent actions below. \r\n \r\n I'm still being tested, so please use our [Feedback Form](https://forms.office.com/Pages/ResponsePage.aspx?id=lVxS1ga5GkO5Jum1G6Q8xHnUJxcBMMdAqVUeyOmrhgBUNFI3VEhMU1laV1YwMUdFTkhYVzcwWk9DMiQlQCN0PWcu) to let us know how well I'm doing and how I can be improved!",
"1594746011384": "You: what is my account number?",
"1594746011652": "AveryCreek_OEM_CSC_Bot_QA: Here is the informaiton I have stored: \n \n**Account Number:** 122809 \n\n I will forget everything except your account number after the end of this conversation.",
"1594746011920": "AveryCreek_OEM_CSC_Bot_QA: I can clear your information if you don't want me to store it or if you want to reneter it. Would you like me to clear your information now?",
"1594746016034": "You: no",
"1594746016301": "AveryCreek_OEM_CSC_Bot_QA: OK, I won't clear your information. You can ask again at any time."
},
"document": {
"userData": {
"accountNumber": "122809"
},
"botName": "AveryCreek_OEM_CSC_Bot_QA",
"1594746019952": "AveryCreek_OEM_CSC_Bot_QA: Hi! I'm the OEM CSC Support Bot! What can I help you with today? \r\nYou can say **Menu** for a list of common commands, **Help** for chatbot tips, or choose one of the frequent actions below. \r\n \r\n I'm still being tested, so please use our [Feedback Form](https://forms.office.com/Pages/ResponsePage.aspx?id=lVxS1ga5GkO5Jum1G6Q8xHnUJxcBMMdAqVUeyOmrhgBUNFI3VEhMU1laV1YwMUdFTkhYVzcwWk9DMiQlQCN0PWcu) to let us know how well I'm doing and how I can be improved!"
},
You had said you were encountering concurrency issues even though JavaScript is single-threaded. As strange as that sounds, I think you're right on some level. TranscriptLoggerMiddleware does have its own buffer that it uses to store activities throughout the turn and then it tries to log all of them all at once. It could easily have provided a way to get that whole buffer in your own logger function, but instead it just loops through the buffer so that you still only get to log them each individually. Also, it allows logActivity to return a promise but it never awaits it, so each activity will get logged "simultaneously" (it's not really simultaneous but the code will likely jump between function calls before waiting for them to complete). This is a problem for any operation that isn't atomic, because you'll be modifying state without knowing about its latest modifications.
while (transcript.length > 0) {
try {
const activity: Activity = transcript.shift();
// If the implementation of this.logger.logActivity() is asynchronous, we don't
// await it as to not block processing of activities.
// Because TranscriptLogger.logActivity() returns void or Promise<void>, we capture
// the result and see if it is a Promise.
const logActivityResult = this.logger.logActivity(activity);
// If this.logger.logActivity() returns a Promise, a catch is added in case there
// is no innate error handling in the method. This catch prevents
// UnhandledPromiseRejectionWarnings from being thrown and prints the error to the
// console.
if (logActivityResult instanceof Promise) {
logActivityResult.catch(err => {
this.transcriptLoggerErrorHandler(err);
});
}
} catch (err) {
this.transcriptLoggerErrorHandler(err);
}
}
All in all, I don't think transcript logger middleware is the way to go here. While it may purport to serve your purposes, there are just too many problems with it. I would either write my own middleware or just put the middleware code directly in my bot logic like this:
async onTurn(turnContext) {
const activity = turnContext.activity;
await this.logActivity(turnContext, activity);
turnContext.onSendActivities(async (ctx, activities, next) => {
for (const activity of activities) {
await this.logActivity(ctx, activity);
}
return await next();
});
// Bot code here
// Save state changes
await this.userState.saveChanges(turnContext);
}
async logActivity(turnContext, activity) {
var transcript = await this.transcriptProperty.get(turnContext, []);
transcript.push(activity);
await this.transcriptProperty.set(turnContext, transcript);
console.log('Activities saved: ' + transcript.length);
}
Since your transcript would be stored in your user state, that user state would also have the account number you need and hopefully you'd be able to query for it.
Kyle's answer did help me solve the issue, and I think that will be the most reusable piece for anyone experiencing similar issues. The key takeaway is that, if you're using nodejs, you should not be using TranscriptLoggerMiddleware and instead use Kyle's function in your onTurn handler (repeated here for reference):
// Function provided by Kyle Delaney
async onTurn(turnContext) {
const activity = turnContext.activity;
await this.logActivity(turnContext, activity);
turnContext.onSendActivities(async (ctx, activities, next) => {
for (const activity of activities) {
await this.logActivity(ctx, activity);
}
return await next();
});
// Bot code here
// Save state changes
await this.userState.saveChanges(turnContext);
}
You need to note, though, that his logActivity function is just storing the raw activities to the user state using a custom transcriptProperty. As of yet I haven't found a good method to give business/admin users access to this data in a way that is easily readable and searchable, nor construct some sort of file out output to send to a customer requesting a transcript of their conversation. As such, I continued using my CustomLogger instead. Here is how I accomplished that.
First, you must create the transcriptLogger in the constructor. If you create it inside your turn handler, you will lose the cache/buffer and it will only have the latest activity instead of the full history. May be common sense but this tripped me up briefly. I do this in the constructor via this.transcriptLogger = new CustomerLogger(appInsightsClient);. I also modified my logActivity function to accept the userData (my state object) as a second, optional parameter. I have successfully been able to use that userData object to add the required customer information to the bot transcript. To modify Kyle's function above you just need to replace this.logActivity with your function call, in my case this.transcriptLogger.logActivity(context, userData);.
While there are still some other issues with this approach, it does solve the title question of how to get user state data into the transcript.
I am trying to set up (Facebook) Webhook. My databse structure is like this:
{ "users": [
"1" : {"FacebookID" : "1234567"},
"2" : {"FacebookID" : "9876554"}
]}
I reduced the code for the webhook to the minimum to show where I have my problems. I go through all entries from the request and check what uid has the change. Before I want to change my data, I would like to see if the user with the id is available in my database. Facebook sends me the FacebookID of course, therefore I need to check, if there is an user account with that specific facebook id.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.webhooks = functions.https.onRequest((req, res) => {
var i = 0;
for (i = 0; i < req.body.entry.length; i++)
{
//strCurrentUser = req.body.entry[i].uid;
strCurrentUser = "2357295025722342"; //For debug
if (checkIfUserAvailable(strCurrentUser))
{
console.log("User is available " + strCurrentUser);
}
else
{
console.log("User is not available " + strCurrentUser);
}
}
return res.send(200);
});
function checkIfUserAvailable(userFacebookID)
{
const rootRef = admin.database().ref();
return rootRef.child('users').orderByChild('FacebookID').equalTo(userFacebookID).once('value').then( snap =>
{
console.log("snap is : " + (snap.val() !== null));
return (snap.val() !== null);
});
}
Now the problem is, that the checkIfUserAvailable always returns true and does not wait till the read is done to return the correct boolean. How can I make sure, that it waits with the return until the read has been finished to get the entire code working?
What I get as result ist:
11:34:58.057 PM / webhooks / snap is : false
11:34:31.665 PM / webhooks / Function execution took 1880 ms, finished with status code: 200
11:34:31.569 PM / webhooks / User is available 2357295025722342
11:34:29.786 PM / webhooks / Function execution started
As you can see the res.send(200) is fired earlier and the snap is: false will be logged later. (I am also not sure why it takes 20 seconds to read data from a almost empty database)
Thanks for the help!
You're not using promises correctly. The call to once() is asynchronous and returns immediately before the query is finished. This means the response is sent before the query is done, and Cloud Functions shuts down that async work. Instead, you should be sending that response only after the query is complete.
You might want to watch my videos on using promises in Cloud Functions in order to learn how best to deal with situations like this. Also be sure you understand the documentation.
I am running Node.js server with express. I'd also like the server to accept IceCast audio stream.
I could use another port, sure, but not all hostings (like Heroku) allow that. Ice cast's stream request looks like this:
SOURCE /mountpoint ICE/1.0\n
content-type: audio/mpeg\n
Authorization: Basic USER+PASS base64encoded\n
ice-name: This is my server name\n
ice-url: http://www.oddsock.org\n
ice-genre: Rock\n
ice-bitrate: 128\n
ice-private: 0\n
ice-public: 1\n
ice-description: This is my server description\n
ice-audio-info: ice-samplerate=44100;ice-bitrate=128;ice-channels=2\n
\n
After that, audio stream follows. I wrote a separate server that handles this on another port and it works fine.
var headers = "";
var headersEnd = false;
var mp3;
const audioServer = net.createServer(function (socket) {
if (mp3) {
socket.write("HTTP/1.0 403 Client already connected\r\n\r\n");
socket.end();
socket.on("error", (e) => {});
return;
}
mp3 = fs.createWriteStream("test.mp3", { encoding: null, flags: "a" });
socket.on("data", (data) => {
if (!headersEnd) {
var tmp = "";
for (let i = 0, l = data.byteLength; i < l; ++i) {
const item = data[i];
if (item == CR_NUMBER)
continue;
const character = String.fromCharCode(item);
tmp += character;
headers += character;
if (headers.endsWith("\n\n")) {
headersEnd = true;
console.log("ICE CAST HEADERS: \n", headers.replace(/\n/g, "\\n\n").replace(/\r/g, "\\r"));
break;
}
}
}
else {
mp3.write(data);
}
});
socket.on("close", () => {
console.log("ICE CAST: END");
if (mp3) {
mp3.close();
mp3 = null;
}
});
socket.on("error", (e) => {
console.log("ICE CAST: ERROR" + e.message);
socket.end();
});
});
audioServer.listen(11666);
What I'd like is to somehow bootstrap node's HTTP server so that I can stream over the same port.
I tried to access the req connection info, that doesn't really work, because the server does not even let the SOURCE /mountpoint ICE/1.0 through.
const server = http.createServer(function (req, res) {
/// does not happen, server closes the connection from icecast
if (handleAudioStream(req, res)) {
return;
}
else {
return expressApp(req, res);
}
});
So I'd need to go deeper. I tried to inspect the net and http code, but didn't fund anything useful.
How can I do this? I really need to use same port, and since icecast DOES send the HTTP-like headers, it should be possible.
This isn't trivial, but possible. You can do some duck punching/monkey patching. See this answer: https://stackoverflow.com/a/24298059/362536
Also, it may be possible to get official support some day, but we're a ways off from that. The first blocker was the non-standard SOURCE method. I sponsored a bounty on that and Ben Noordhuis was kind enough to implement last week: https://github.com/nodejs/http-parser/issues/405 It should land in Node.js eventually.
The next issue is the ICE/1.0. I've opened an issue for that here: https://github.com/nodejs/http-parser/issues/410 There hasn't been any objection to adding it to the parser yet, but if you want to add a pull request, that might help a chance of approval.
You'll find other compatibility issues as well as you continue down this road, but all I've hit I've been able to overcome with various solutions. The trick is, maintaining strict compatibility with the Node.js core as it is updated.
I am currently try to get the data from cloudmqtt by subscribe to a topic. But when I try to convert it to string so that my Alexa could read it the test service of Alexa tell me "The remote endpoint could not be called, or the response it returned was invalid."
My Subscribing code is as follow
var mqtt = require('mqtt');
function DHTSensorInSession(intent, session, callback) {
const cardTitle = intent.name;
let repromptText = '';
let sessionAttributes = {};
const shouldEndSession = true;
let speechOutput = '';
let Tempe = '';
var mqttpromise = new Promise( function(resolve,reject){
var client = mqtt.connect({port:19546,host:'m14.cloudmqtt.com',username:'jkgnbggq',password:'VbG2-RvZTEt-'})
client.on('connect', function() { // When connected
// publish a message to any mqtt topic
client.publish('main/light/esp', 'connected')
client.subscribe('main/sensor/temp')
});
client.on('message',function(topic,message){
if(topic.toString() === 'main/sensor/temp')
{
Tempe += message.toString();
}
client.end();
resolve('Done sending');
})
});
mqttpromise.then(
function(data) {
console.log('Function called succesfully:', data);
sessionAttributes = createSensorAttributes(Tempe);
speechOutput = "That wire less light has been turned " + Tempe;
repromptText = "Ok, turning the light " + Tempe;
callback(sessionAttributes,buildSpeechletResponse(cardTitle, speechOutput, repromptText, shouldEndSession));
},
function(err) {
console.log('An error occurred:', err);
}
);
}
Refer to the logs for your lambda to locate any error messages. These can be found in the AWS console, or using the ASK CLI command "ask lambda log ...".
These may appear a bit cryptic at first, but they can be very helpful when something goes wrong. Look for the entries with the words "error" or "fail". Often they will include a line number of your index.js (or other) file showing you exactly which line is causing the problem.
Alternatively, create unit tests to test your code locally before uploading.
There are lots of articles and posts online about doing this.
MQTT doesn't work that way.
Normally messages are only delivered to already connected clients that are subscribed to a topic at the instant they the message is published. The exception to this is if the message is flagged as retained, in this case the LAST message published will be delivered to a new client at the point where they subscribe to the topic.
Im using an external API. I'd like to show logged in user's nickname (steam API)
server main.js
Meteor.startup(function () {
ServiceConfiguration.configurations.upsert(
{ service: 'steam' },
{
$set: {
loginStyle: 'redirect',
timeout: 10000 // 10 seconds
}
}
);
Accounts.onLogin(function() {
var steam64Id = Meteor.user().profile.id;
console.log(steam64Id + " is logged in.");
// To retrieve more details about user
var steamApiKey = ("XXXXXXXXXXXXX");
var result = Meteor.http.get('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=xxxxxxxxx&steamids=76561197977045111');
console.log(result.data.response.players[0].personaname);
});
});
This returns "My Nickname" in the console log, but I'd like to retrieve this variable in client side so I can add {{Nickname}} to client js template and show user's Nickname in the template
The most obvious way would be to add that result to the user's profile in their user record. Then it would automatically be synchronized to the client and available for use there.
var steamApiKey = ("XXXXXXXXXXXXX");
var result = Meteor.http.get('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=xxxxxxxxx&steamids=76561197977045111');
Meteor.users.update(Meteor.userId(),
{ $set: { 'profile.personaname': result.data.response.players[0].personaname }}
);
On the client side, make yourself a helper (either a global one or one specific to the template(s) where you need it) to get this value:
Nickname(){
return Meteor.user().profile.personaname;
}
Note that the profile object under the user is automatically published to the client so you don't have to make a special publication for it.