I'll try to keep this simple. I'm working on a paging app.
What I have: A terminal without internet, running a desktop app made from node.js using express,ejs, ect. I also have a physical server with internet access and a mssql database on this machine I have a node.js server that interfaces with the database, collects the data and send sends it over to terminal via websocket when the app is launched, the node.js app gets this data and is rendered to the web interface using ejs. This data is displayed in a form with a button that when clicked fires the app.post route. In the app.post that data is packaged in an array, and sent back to the server using a second websocket connection. I can then take that form data (a name, phone#, and a radio button value) and form a SMS message using and using twilio send a message to that number (with addition info)
Yes, I have two websocket connections one Server > client serves data to web app
and another client > server serves form data to the server.
What's working: I get data from the boh server/database > webserver/client and data is displayed correctly when I hit the button data is packaged and send back to the boh server.
On the boh server i have a function that get the data array and parses it out, and sends an SMS message using twilio
My issue: I have to restart the server app to get it to process the data and send the message, if I hit the page button, it does all the stuff in the background it should, packages the array and sends the info to the server. The server is waiting for the data to be sent and client has sent the data, however it will only send off the text message is a stop and restart the node.js server, if i do that, the server starts and runs through the initial process of getting the sql data, and sets it up to be called when the app launches, then continues on to read that data was sent from the client, received and it will parse the data from the SMS message through a function send the message, wait a few seconds and then grab the response and confirm delivery of message. I am quite sure I am missing something basic here, but I have different functions but nothing I do will seem to get it to fire when the data is received.
I'm new at node and not very advanced in js, but I understand on some level why its not firing as is right, script is running, data has not been sent, so it stops when it gets here, then i hit the button, and it does nothing, because the script is stopped, and it doesn't has know way of knowing that data was sent, but when i rerun the script, the data that was sent is still sitting there wait to be received, so it recognizes the open websocket, and the sent data and work appropriately, I feel like a am missing something on the server side that tell it to wait for the data send but have not been able to make it work
twilws.onopen = async() => {//when the page button is presses, it starts a websocket server on the client, if that makes sense
twilws.send('test')
if (twilws.readyState === WebSocket.OPEN) {
logger.info('Twilio Sockpuppet Connected')
} else {
logger.error('Twilio Sockpuppet Fail')
}
twilws.onmessage = async (e) => {//after i hit the page button the server is on and the array is sent, i am trying to get this to be waiting for data and when it arrives go, but it will only do that when i restart the script
try {
data = JSON.parse(e.data);//parse the data
logger.info('got the edata ' + e.data)
//main(data) this will fire but not when the button is pressed.
} catch (er) {
logger.error('socket parse error: ' + e.data);
}
}
}
twilws.onclose = () => {//close the connection, purges that data so that the websocket can be recreated and an array with new data sent.
logger.info('Web Socket Connection Closed');
twilws.close(1000, 'all done');
};
Here is the whole server
I believe i may be blocking something, sorry about the bad formatting i have been changing and trying different things for a couple weeks off and on now and have not had a chance to clean thing up.
const express = require("express");
const cors = require("cors");
const app = express();
const db = require("./app/models");
const twilioconfig = require('./configs/twilioConfig.js');
const path = require('path');
const WebSocket = require('ws');
const sql = require('mssql');
const WebSocketServer = require('websocket').server;
const WebSocketClient = require('websocket').client;
const WebSocketFrame = require('websocket').frame;
/*const WebSocketRouter = require('websocket').router;*/
const W3CWebSocket = require('websocket').w3cwebsocket;
const http = require('http');
const https = require('https');
const twilio = require('twilio');
const { Console } = require("console");
app.disable('view cache');
const pino = require('pino')
const SonicBoom = require('sonic-boom')
const logger = require('pino')()
const transport = pino.transport({
target: 'pino/file',
options: { destination: './logs/logs.txt', level: 'info', mkdir: true, append: true }
})
pino(transport)
/*const dbConfig2 = require("./app/config/db.config.js");*/
/*const config = require("./app/config/config.js");*/
const { client } = require("websocket");
const { err } = require("./node_modules/pino-std-serializers/index");
const { setInterval } = require("node:timers/promises");
app.use(pino)
//const webserver = app.listen(8080, function () {
// console.log('Node WEb Server is running..');
//});
var params = {
autoReconnect: false, //Enable/Disable reconnect when the server closes connection (boolean)
autoReconnectInterval: 1000, //Milliseconds to wait between reconnect attempts (number)
autoReconnectMaxRetries: 600, //Max number of reconnect attempts to allow (number)
requestTimeout: 30000, //Milliseconds to wait for a response before resending the request (number)
requestRetryInterval: 5000, //Milliseconds between request retry checks. This garbage collects the retry queue (number)
requestRetryQueueMaxLength: 10 //Max queue length of retry queue before old messages start getting dropped (number)
}
var wss = new WebSocket.Server({ port: 8081 })
sql.connect(config, function (err) {
if (err)
logger.error(err);
const sqlRequest = new sql.Request();
const sqlQuery = "SELECT TOP 5 guest_name,guest_phone_number,CONVERT(varchar,creation_time, 126) AS creation_time,CONVERT(varchar,last_modified_timestamp, 126) AS last_modified_timestamp,party_size from dbo.WaitList where status = '4' AND CAST(creation_time as date) = CAST( GETDATE() AS Date ) ORDER BY creation_time ASC";
logger.info("query passes preflight.....lets get data")
sqlRequest.query(sqlQuery, function (err, data) {
if (err) {
logger.error(err)
} else {
logger.info("We do preliminary query now.")
}
//console.table(data.recordset);
//logger.info('rows affected ' + data.rowsAffected);
//console.log(data.recordset[0]);
var array = [];
for (let i = 0; i < data.rowsAffected; i++) {
var a = data.recordset[i];
array.push(a);
}
wss.on('connection', ws => {
logger.info('Client connection established')
ws.on('message', function () {
sqlRequest.query(sqlQuery, function (err, data) {
if (err) {
logger.error(err)
}
//console.table(data.recordset);
//logger.info(data.rowsAffected);
//console.log(data.recordset[0]);
var array = [];
for (let i = 0; i < data.rowsAffected; i++) {
var a = data.recordset[i];
array.push(a);
}
})
wss.clients
.forEach(client => {
logger.info('sending data')
client.send(JSON.stringify(array))
})
})
})
})
});
const twilws = new W3CWebSocket('ws://172.16.0.101:8082', params);
twilws.onopen = () => {//when the page button is presses, it starts a websocket server on the client, if that makes sense
twilws.send('test')
if (twilws.readyState === WebSocket.OPEN) {
logger.info('Twilio Sockpuppet Connected')
} else {
logger.error('Twilio Sockpuppet Fail')
}
twilws.onmessage = async (e) => {//after i hit the page button the server is on and the array is sent, i am trying to get this to be waiting for data and when it arrives go, but it will only do that when i restart the script
try {
data = JSON.parse(e.data);//parse the data
logger.info('got the edata ' + e.data)
//main(data) this will fire but not when the button is pressed.
} catch (er) {
logger.error('socket parse error: ');
}
}
}
twilws.onclose = () => {//close the connection, purges that data so that the websocket can be recreated and an array with new data sent.
logger.info('Web Socket Connection Closed');
twilws.close(1000, 'all done');
};
//async function main(s) {
// /* these settings are loaded from configs/twilioConfig.js, go here to set the store account info and edit message body.*/
// const TWILIO_ACCOUNT_SID = twilioconfig.twilioOptions.TWILIO_ACCOUNT_SID;
// const TWILIO_AUTH_TOKEN = twilioconfig.twilioOptions.TWILIO_AUTH_TOKEN;
// const STORE_TWILIO_NUMBER = twilioconfig.twilioOptions.STORE_TWILIO_NUMBER;
// const TEXT_TWILIO_BODY = twilioconfig.twilioOptions.TEXT_TWILIO_BODY;
// var messarray = new Array([s])
// var num = JSON.stringify(messarray[0][0][0])
// var state = JSON.stringify(messarray[0][0][1])
// logger.info(num + ', ' + state)
// var readyin = state
// var custnum = num
// /*logger.info(TWILIO_ACCOUNT_SID + ' ' + TWILIO_AUTH_TOKEN)*/
// var customer = new twilio(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN);//these are our twilio account sid and token, set in twilioConfig, get from oneNote
// if (readyin == 0) {
// waitmsg = TEXT_TWILIO_BODY + "Your Table is Ready!";
// } else if (readyin == 15) {
// waitmsg = TEXT_TWILIO_BODY + "Your Table will be ready in about 15 minutes!";
// }
// logger.info(`Recieved Form data.....Twilio data read successfully....`);//log that we got the data from the gui form
// // send the message to the customer number through twilio, custnum from form, from is the store number, alson in twilioConfig
// customer.messages.create({
// to: custnum,
// from: STORE_TWILIO_NUMBER,
// body: waitmsg
// })
// .then(message => {
// var messageid = message.sid
// logger.info(messageid + ' Hey i found this, we might need it in a sec')
// const msid = JSON.stringify({ "MessageSid": messageid })
// const twilid = messageid
// setTimeout(getStatus, 5000);
// function getStatus() {
// customer.messages(messageid).fetch()
// .then(call => {
// const d = new Date(call.dateCreated).toLocaleString();
// const messageStatus = call.status
// var twil_response_array = [twilid, messageStatus, d]
// logger.info(twil_response_array)
// wss.on('connection', ws => {
// ws.on('message', function () {
// })
// wss.clients
// .forEach(client => {
// client.send(twil_response_array)
// })
// })
// })
// }
// //})
app.listen(8180, function () {
logger.info('Server is running..');
});
});
I want to send a audio file to a server (in my case discord) easly as if it was comming from the microphone
I found this code at Send sound through microphone in javascript and modified it to try to fit my use case, but I still cannot get it to work.
navigator.mediaDevices.getUserMedia = () => {
const audioContext = new AudioContext();
return fetch('http://127.0.0.1:8000/enemey.ogg',{mode: 'no-cors'})
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => {
const audioBufferSourceNode = audioContext.createBufferSource();
const mediaStreamAudioDestinationNode = audioContext.createMediaStreamDestination();
audioBufferSourceNode.buffer = audioBuffer;
// Maybe it makes sense to loop the buffer.
audioBufferSourceNode.loop = true;
audioBufferSourceNode.start();
audioBufferSourceNode.connect(mediaStreamAudioDestinationNode);
return mediaStreamAudioDestinationNode.stream;
});
};
any Ideas? I cannot find a fix for this, and the error is
[AudioActionCreators] unknown getUserMedia error: EncodingError
by discord
(all of this is done with the console, not a external program)
Goal: Objects will be pushed to a readable stream and then saved in a separate .csv depending on what channel (Email, Push, In-App) they come from.
Problem: I am unable to separate out the streams in to different .pipe() "lines" so all .csv logs receive only their channel specific event objects. But in the current iteration all of the .csv files created by the Writestream are receiving the event objects from all channels.
Questions:
Can I dynamically create the multiple channel "pipe() lines" in the setup() function programmatically or is the current way I am approaching this correct?
Is this manual creation of the "pipe() lines" the reason all of the .csv's are being populated with events? Can this be solved with one "pipe() line" and dynamic routing?
A brief explanation of the code below:
setup() calls makeStreams() - creates an object with a Readable and a Writable (rotating file system Writable stream) (setup() is an unnecessary function right now but will hold more setup tasks later.)
pushStream() is called when an inbound event occurs and pushes an object like: {Email: {queryParam:1, queryParam:2, etc.}} The event is sorted by the highest level obj (in this case "Email") and then is pushed to the correct writable stream which in theory should be ported to the correct writable stream.
Unfortunately this isn't the case, it's sending the event object to all of the writable streams. How can I send it to only the correct stream?
CODE:
const Readable = require('stream').Readable
const Json2csvTransform = require('json2csv').Transform;
var rfs = require("rotating-file-stream");
const channelTypes = ['Push Notification', 'Email', 'In-app Message']
var streamArr = setup(channelTypes);
const opts = {};
const transformOpts = {
objectMode: true
};
const json2csv = new Json2csvTransform(opts, transformOpts);
function setup(list) {
console.log("Setting up streams...")
streamArr = makeStreams(list) //makes streams out of each endpoint
return streamArr
}
//Stream Builder for Logging Based Upon Channel Name
function makeStreams(listArray) {
listArray = ['Push Notification', 'Email', 'In-app Message']
var length = listArray.length
var streamObjs = {}
for (var name = 0; name < length; name++) {
var fileName = listArray[name] + '.csv'
const readStream = new Readable({
objectMode: true,
read() {}
})
const writeStream = rfs(fileName, {
size: "50M", // rotate every 50 MegaBytes written
interval: "1d" // rotate daily
//compress: "gzip" // compress rotated files
});
var objName = listArray[name]
var obj = {
instream: readStream,
outstream: writeStream
}
streamObjs[objName] = obj
}
return streamObjs
}
function pushStream(obj) {
var keys = Object.keys(obj)
if (streamArr[keys]) {
streamArr[keys].instream.push(obj[keys])
} else {
console.log("event without a matching channel error")
}
}
//Had to make each pipe line here manually. Can this be improved? Is it the reason all of the files are receiving all events?
streamArr['Email'].instream.pipe(json2csv).pipe(streamArr['Email'].outstream)
streamArr['In-app Message'].instream.pipe(json2csv).pipe(streamArr['In-app Message'].outstream)
streamArr['Push Notification'].instream.pipe(json2csv).pipe(streamArr['Push Notification'].outstream)
module.exports = {
makeStreams,
pushStream,
setup
}
I am streaming audio data in chunks through web-Socket from server
ws.on('message', function incoming(message) {
var readStream = fs.createReadStream("angular/data/google.mp3",
{
'flags': 'r',
'highWaterMark': 128 * 1024
}
);
readStream.on('data', function(data) {
ws.send(data);
});
readStream.on('end', function() {
ws.send('end');
});
readStream.on('error', function(err) {
console.log(err)
});
});
on the client side
var chunks = [];
var context = new AudioContext();
var soundSource;
var ws = new WebSocket(url);
ws.binaryType = "arraybuffer";
ws.onmessage = function(message) {
if (message.data instanceof ArrayBuffer) {
chunks.push(message.data)
} else {
createSoundSource(chunks);
}
};
function createSoundSource(audioData) {
soundSource = context.createBufferSource();
for (var i=0; i < audioData.length;i++) {
context.decodeAudioData(audioData[i], function(soundBuffer){
soundSource.buffer = soundBuffer;
soundSource.connect(context.destination);
soundSource.start(0);
});
}
}
But setting buffer soundSource.buffer = soundBuffer; for the second time causing an error
Uncaught DOMException: Failed to set the 'buffer' property on 'AudioBufferSourceNode': Cannot set buffer after it has been already been set
Any advice or insight into how best to update Web Audio API playback with new audio data would be greatly appreciated.
You cannot reset a buffer on an AudioBufferSourceNode once it's been set. It's like fire-and-forget. Each time you want to play a different buffer, you have to create a new AudioBufferSourceNode to continue playback. Those are very lightweight nodes so don't worry about the performance even when creating tons of them.
To account for this, you can modify your createSoundSource function to simply create an AudioBufferSourceNode for each chunk inside the cycle body, like that:
function createSoundSource(audioData) {
for (var i=0; i < audioData.length;i++) {
context.decodeAudioData(audioData[i], function(soundBuffer){
var soundSource = context.createBufferSource();
soundSource.buffer = soundBuffer;
soundSource.connect(context.destination);
soundSource.start(0);
});
}
}
I tried to keep the code style as close to original as possible, but it's 2020, and a function taking advantage of modern features could actually look like this:
async function createSoundSource(audioData) {
await Promise.all(
audioData.map(async (chunk) => {
const soundBuffer = await context.decodeAudioData(chunk);
const soundSource = context.createBufferSource();
soundSource.buffer = soundBuffer;
soundSource.connect(context.destination);
soundSource.start(0);
})
);
}
If you want to stop the old nodes as soon as new data arrives (it looks like you wanted that by resetting the .buffer but I'm not sure), you'll have to store them and call disconnect on all of them when it's time.
Not positive, but I think you have to handle your streaming websocket buffer a bit differently. Maybe websocket-streaming-audio package source code can give you better clues on how to handle your scenario.
I'm attempting to setup a node js server to stream some events via SSE and I cannot get it to work unless I start the stream with the data field.
When i try and include other fields for example id or event they dont show in the chrome inspector?
If I attempt to put these field before data no events other than the connection opening occurs.
Here is the route that I am playing around with.
router.get('/stream', function * (){
let stream = new PassThrough();
let send = (message, id) => stream.write(`id: ${JSON.stringify(id)}\n data: ${JSON.stringify(message)}\n\n`);
let finish = () => dispatcher.removeListener('message', send);
this.socket.setTimeout(Number.MAX_VALUE);
this.type = 'text/event-stream;charset=utf-8';
this.set('Cache-Control', 'no-cache');
this.set('Connection', 'keep-alive');
this.body = stream;
stream.write(': open stream\n\n');
dispatcher.on('message', send);
this.req.on('close', finish);
this.req.on('finish', finish);
this.req.on('error', finish);
setInterval(function(){
dispatcher.emit('message', {date: Date.now()}, '12345')
}, 5000)
});
I am not too certain if I am writing to the stream correctly.
Thanks
Silly me, it was indeed a malformed event stream. There isnt supposed to be a space between the fields. It should instead be:
id: ${JSON.stringify(id)}\ndata: ${JSON.stringify(message)}\n\n