I am attempting to use a Zebra DS9208 scanner to capture barcode data into a web page. For some reason, when I first select the serial port and connect to the scanner, the very first scan causes the barcode reader to crash (it locks into read mode, refuses to read additional scans, then disconnects from the computer and reboots itself). When I establish the connection a second time, the scanner continuously reads scans without issues. Anyone able to spot anything in my code that could cause this?
Connect Script:
async function connect() {
// - Request a port and open a connection.
port = await navigator.serial.requestPort();
// - Wait for the port to open.
await port.open({ baudRate: 9600 });
// Send a bunch of Bell characters so we can hear that the scanner understands us.
await quadBeep();
// Read the stream let textDecoder = new TextDecoderStream();
inputDone = port.readable.pipeTo(textDecoder.writable);
reader = textDecoder.readable.getReader();
readScans();
}
Beep Script (used to make the scanner beep by sending a BELL character)
function quadBeep() {
console.log('Quad Beep Requested');
//Write to output stream
const writer = port.writable.getWriter();
const data = new Uint8Array([07, 07, 07, 07]);
writer.write(data);
//allow the serial port to be closed later
writer.releaseLock();
return;
}
Read Loop to source data from the scanner:
`async function readScans() {
//Listen to data coming from the serial device
async function readScans() {
while (true) {
try {
const { value, done } = await reader.read();
await saveScan(value); //process the scan
console.log('readScans Barcode: ' + value);
document.getElementById('scan').innerHTML += value;
if (done) {
//Allow the serial port to be closed later.
console.log('readScans done value: ' + done);
reader.releaseLock();
break;
}
} catch(error) {
console.log('readScans Error: ' + error);
break;
}
}
}
Save Function, to write the data (eventually will submit the barcode via AJAX)
function saveScan(barcode) {
var session = document.getElementById('sessionID').getAttribute('data-value');
if (barcode == previousBarcode) {
//duplicate scan.
console.log('saveScan Duplicate');
return;
} else {
//Submit the scan
previousBarcode = barcode; //store the barcode so it doesn't get rescanned
console.log('saveScan Previous set to: ' + barcode);
//future AJAX FUNCTION GOES HERE
return;
}
}
Related
I'll try to keep this simple. I'm working on a paging app.
What I have: A terminal without internet, running a desktop app made from node.js using express,ejs, ect. I also have a physical server with internet access and a mssql database on this machine I have a node.js server that interfaces with the database, collects the data and send sends it over to terminal via websocket when the app is launched, the node.js app gets this data and is rendered to the web interface using ejs. This data is displayed in a form with a button that when clicked fires the app.post route. In the app.post that data is packaged in an array, and sent back to the server using a second websocket connection. I can then take that form data (a name, phone#, and a radio button value) and form a SMS message using and using twilio send a message to that number (with addition info)
Yes, I have two websocket connections one Server > client serves data to web app
and another client > server serves form data to the server.
What's working: I get data from the boh server/database > webserver/client and data is displayed correctly when I hit the button data is packaged and send back to the boh server.
On the boh server i have a function that get the data array and parses it out, and sends an SMS message using twilio
My issue: I have to restart the server app to get it to process the data and send the message, if I hit the page button, it does all the stuff in the background it should, packages the array and sends the info to the server. The server is waiting for the data to be sent and client has sent the data, however it will only send off the text message is a stop and restart the node.js server, if i do that, the server starts and runs through the initial process of getting the sql data, and sets it up to be called when the app launches, then continues on to read that data was sent from the client, received and it will parse the data from the SMS message through a function send the message, wait a few seconds and then grab the response and confirm delivery of message. I am quite sure I am missing something basic here, but I have different functions but nothing I do will seem to get it to fire when the data is received.
I'm new at node and not very advanced in js, but I understand on some level why its not firing as is right, script is running, data has not been sent, so it stops when it gets here, then i hit the button, and it does nothing, because the script is stopped, and it doesn't has know way of knowing that data was sent, but when i rerun the script, the data that was sent is still sitting there wait to be received, so it recognizes the open websocket, and the sent data and work appropriately, I feel like a am missing something on the server side that tell it to wait for the data send but have not been able to make it work
twilws.onopen = async() => {//when the page button is presses, it starts a websocket server on the client, if that makes sense
twilws.send('test')
if (twilws.readyState === WebSocket.OPEN) {
logger.info('Twilio Sockpuppet Connected')
} else {
logger.error('Twilio Sockpuppet Fail')
}
twilws.onmessage = async (e) => {//after i hit the page button the server is on and the array is sent, i am trying to get this to be waiting for data and when it arrives go, but it will only do that when i restart the script
try {
data = JSON.parse(e.data);//parse the data
logger.info('got the edata ' + e.data)
//main(data) this will fire but not when the button is pressed.
} catch (er) {
logger.error('socket parse error: ' + e.data);
}
}
}
twilws.onclose = () => {//close the connection, purges that data so that the websocket can be recreated and an array with new data sent.
logger.info('Web Socket Connection Closed');
twilws.close(1000, 'all done');
};
Here is the whole server
I believe i may be blocking something, sorry about the bad formatting i have been changing and trying different things for a couple weeks off and on now and have not had a chance to clean thing up.
const express = require("express");
const cors = require("cors");
const app = express();
const db = require("./app/models");
const twilioconfig = require('./configs/twilioConfig.js');
const path = require('path');
const WebSocket = require('ws');
const sql = require('mssql');
const WebSocketServer = require('websocket').server;
const WebSocketClient = require('websocket').client;
const WebSocketFrame = require('websocket').frame;
/*const WebSocketRouter = require('websocket').router;*/
const W3CWebSocket = require('websocket').w3cwebsocket;
const http = require('http');
const https = require('https');
const twilio = require('twilio');
const { Console } = require("console");
app.disable('view cache');
const pino = require('pino')
const SonicBoom = require('sonic-boom')
const logger = require('pino')()
const transport = pino.transport({
target: 'pino/file',
options: { destination: './logs/logs.txt', level: 'info', mkdir: true, append: true }
})
pino(transport)
/*const dbConfig2 = require("./app/config/db.config.js");*/
/*const config = require("./app/config/config.js");*/
const { client } = require("websocket");
const { err } = require("./node_modules/pino-std-serializers/index");
const { setInterval } = require("node:timers/promises");
app.use(pino)
//const webserver = app.listen(8080, function () {
// console.log('Node WEb Server is running..');
//});
var params = {
autoReconnect: false, //Enable/Disable reconnect when the server closes connection (boolean)
autoReconnectInterval: 1000, //Milliseconds to wait between reconnect attempts (number)
autoReconnectMaxRetries: 600, //Max number of reconnect attempts to allow (number)
requestTimeout: 30000, //Milliseconds to wait for a response before resending the request (number)
requestRetryInterval: 5000, //Milliseconds between request retry checks. This garbage collects the retry queue (number)
requestRetryQueueMaxLength: 10 //Max queue length of retry queue before old messages start getting dropped (number)
}
var wss = new WebSocket.Server({ port: 8081 })
sql.connect(config, function (err) {
if (err)
logger.error(err);
const sqlRequest = new sql.Request();
const sqlQuery = "SELECT TOP 5 guest_name,guest_phone_number,CONVERT(varchar,creation_time, 126) AS creation_time,CONVERT(varchar,last_modified_timestamp, 126) AS last_modified_timestamp,party_size from dbo.WaitList where status = '4' AND CAST(creation_time as date) = CAST( GETDATE() AS Date ) ORDER BY creation_time ASC";
logger.info("query passes preflight.....lets get data")
sqlRequest.query(sqlQuery, function (err, data) {
if (err) {
logger.error(err)
} else {
logger.info("We do preliminary query now.")
}
//console.table(data.recordset);
//logger.info('rows affected ' + data.rowsAffected);
//console.log(data.recordset[0]);
var array = [];
for (let i = 0; i < data.rowsAffected; i++) {
var a = data.recordset[i];
array.push(a);
}
wss.on('connection', ws => {
logger.info('Client connection established')
ws.on('message', function () {
sqlRequest.query(sqlQuery, function (err, data) {
if (err) {
logger.error(err)
}
//console.table(data.recordset);
//logger.info(data.rowsAffected);
//console.log(data.recordset[0]);
var array = [];
for (let i = 0; i < data.rowsAffected; i++) {
var a = data.recordset[i];
array.push(a);
}
})
wss.clients
.forEach(client => {
logger.info('sending data')
client.send(JSON.stringify(array))
})
})
})
})
});
const twilws = new W3CWebSocket('ws://172.16.0.101:8082', params);
twilws.onopen = () => {//when the page button is presses, it starts a websocket server on the client, if that makes sense
twilws.send('test')
if (twilws.readyState === WebSocket.OPEN) {
logger.info('Twilio Sockpuppet Connected')
} else {
logger.error('Twilio Sockpuppet Fail')
}
twilws.onmessage = async (e) => {//after i hit the page button the server is on and the array is sent, i am trying to get this to be waiting for data and when it arrives go, but it will only do that when i restart the script
try {
data = JSON.parse(e.data);//parse the data
logger.info('got the edata ' + e.data)
//main(data) this will fire but not when the button is pressed.
} catch (er) {
logger.error('socket parse error: ');
}
}
}
twilws.onclose = () => {//close the connection, purges that data so that the websocket can be recreated and an array with new data sent.
logger.info('Web Socket Connection Closed');
twilws.close(1000, 'all done');
};
//async function main(s) {
// /* these settings are loaded from configs/twilioConfig.js, go here to set the store account info and edit message body.*/
// const TWILIO_ACCOUNT_SID = twilioconfig.twilioOptions.TWILIO_ACCOUNT_SID;
// const TWILIO_AUTH_TOKEN = twilioconfig.twilioOptions.TWILIO_AUTH_TOKEN;
// const STORE_TWILIO_NUMBER = twilioconfig.twilioOptions.STORE_TWILIO_NUMBER;
// const TEXT_TWILIO_BODY = twilioconfig.twilioOptions.TEXT_TWILIO_BODY;
// var messarray = new Array([s])
// var num = JSON.stringify(messarray[0][0][0])
// var state = JSON.stringify(messarray[0][0][1])
// logger.info(num + ', ' + state)
// var readyin = state
// var custnum = num
// /*logger.info(TWILIO_ACCOUNT_SID + ' ' + TWILIO_AUTH_TOKEN)*/
// var customer = new twilio(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN);//these are our twilio account sid and token, set in twilioConfig, get from oneNote
// if (readyin == 0) {
// waitmsg = TEXT_TWILIO_BODY + "Your Table is Ready!";
// } else if (readyin == 15) {
// waitmsg = TEXT_TWILIO_BODY + "Your Table will be ready in about 15 minutes!";
// }
// logger.info(`Recieved Form data.....Twilio data read successfully....`);//log that we got the data from the gui form
// // send the message to the customer number through twilio, custnum from form, from is the store number, alson in twilioConfig
// customer.messages.create({
// to: custnum,
// from: STORE_TWILIO_NUMBER,
// body: waitmsg
// })
// .then(message => {
// var messageid = message.sid
// logger.info(messageid + ' Hey i found this, we might need it in a sec')
// const msid = JSON.stringify({ "MessageSid": messageid })
// const twilid = messageid
// setTimeout(getStatus, 5000);
// function getStatus() {
// customer.messages(messageid).fetch()
// .then(call => {
// const d = new Date(call.dateCreated).toLocaleString();
// const messageStatus = call.status
// var twil_response_array = [twilid, messageStatus, d]
// logger.info(twil_response_array)
// wss.on('connection', ws => {
// ws.on('message', function () {
// })
// wss.clients
// .forEach(client => {
// client.send(twil_response_array)
// })
// })
// })
// }
// //})
app.listen(8180, function () {
logger.info('Server is running..');
});
});
I am struggling through learning JQuery/Javascript and have a web application using the chrome "experimental" web serial API. When I enter a command and get a response back, this string is broken into 2 pieces in a random place, usually in the first third:
<p0><iDCC-EX V-0.2.1 / MEGA / STANDARD_MOTOR_SHIELD G-9db6d36>
All the other return messages are shorter and also wrapped in "<" and ">" brackets.
In the code below. The log window only ever shows the second chunk, even in the "ChunkTransformer() routine that simultaneously displays it properly in the devtools console log.
How can I get all my return messages to appear as one string? It is ok if the chunks are split as separate return values by the brackets as long as they display in the log. I think the <p0> is not displaying because the log window thinks it is a special character. It would not even display here until I wrapped in in a code tag. So I think I have at least two issues.
async function connectServer() {
try{
port = await navigator.serial.requestPort(); // prompt user to select device connected to a com port
await port.open({ baudRate: 115200 }); // open the port at the proper supported baud rate
// create a text encoder output stream and pipe the stream to port.writeable
const encoder = new TextEncoderStream();
outputDone = encoder.readable.pipeTo(port.writable);
outputStream = encoder.writable;
// send a CTRL-C and turn off the echo
writeToStream('\x03', 'echo(false);');
let decoder = new TextDecoderStream();
inputDone = port.readable.pipeTo(decoder.writable);
inputStream = decoder.readable
// test why only getting the second chunk in the log
.pipeThrough(new TransformStream(new ChunkTransformer()));
// get a reader and start the non-blocking asynchronous read loop to read data from the stream.
reader = inputStream.getReader();
readLoop();
return true;
} catch (err) {
console.log("User didn't select a port to connect to")
return false;
}
}
async function readLoop() {
while (true) {
const { value, done } = await reader.read();
if (value) {
displayLog(value);
}
if (done) {
console.log('[readLoop] DONE'+done.toString());
displayLog('[readLoop] DONE'+done.toString());
reader.releaseLock();
break;
}
}
}
class ChunkTransformer {
transform(chunk, controller) {
displayLog(chunk.toString()); // only shows last chunk!
console.log('dumping the raw chunk', chunk); // shows all chunks
controller.enqueue(chunk);
}
}
function displayLog(data){
$("#log-box").append("<br>"+data+"<br>");
$("#log-box").animate({scrollTop: $("#log-box").prop("scrollHeight"), duration: 1}, "fast");
}
First Step:
Modify the displayLog() function in one of the following ways
With Animate:
function displayLog(data){
$("#log-box").append("<br>"+data+"<br>");
$("#log-box").animate({scrollTop: $("#log-box").prop("scrollHeight")}, "fast");
}
Without Animate:
function displayLog(data){
$("#log-box").append("<br>"+data+"<br>");
$("#log-box").scrollTop( $("#log-box").prop("scrollHeight"));
}
OR Just for your understanding:
function displayLog(data){
$("#log-box").append("<br>"+data+"<br>");
scrollHeight = $("#log-box").prop("scrollHeight");
$("#log-box").scrollTop(scrollHeight);
}
I'm trying to connect the Muse headband in chrome so i can stream live brainwave data.
I can connect and stream data fine in python using ports, but am having trouble streaming the data via web BLE.
With the code bellow, I can connect to the device and startNotifications(), but the event listener for "characteristicvaluechanged" is not being triggered. I've also tried connecting with other bluetooth devices, and no values are read so I don't think this is a Muse problem.
<h1>Test HTML</h1>
<button id="read" onclick="onButtonClick(event)">Connect with the BLE device</button>
<script>
function onButtonClick(event){
connectBLE()
}
async function connectBLE() {
let serviceUuid = '5052494d-2dab-0341-6972-6f6861424c45'
let characteristicUuid = '43484152-2dab-3141-6972-6f6861424c45'
try {
console.log('Requesting Bluetooth Device...');
const device = await navigator.bluetooth.requestDevice({
filters: [{services: [serviceUuid]}]});
console.log('Connecting to GATT Server...');
const server = await device.gatt.connect();
console.log('Getting Service...');
const service = await server.getPrimaryService(serviceUuid);
console.log('Getting Characteristic...');
myCharacteristic = await service.getCharacteristic(characteristicUuid);
await myCharacteristic.startNotifications();
console.log(myCharacteristic)
console.log('> Notifications started');
myCharacteristic.addEventListener('characteristicvaluechanged',
handleNotifications);
} catch(error) {
console.log('Argh! ' + error);
}
}
function handleNotifications(event) {
console.log('YEAH!')
let value = event.target.value;
let a = [];
// Convert raw data bytes to hex values just for the sake of showing something.
// In the "real" world, you'd use data.getUint8, data.getUint16 or even
// TextDecoder to process raw data bytes.
for (let i = 0; i < value.byteLength; i++) {
a.push('0x' + ('00' + value.getUint8(i).toString(16)).slice(-2));
}
console.log('> ' + a.join(' '));
}
</script>
Anyone ever had a similar issue? I can't find any solutions online so this is kinda my last resort.
PS: I'm running Ubuntu 18.04 and Chromium 83.0.4103.61
first is length
second is command - to start streaming it is letter "d" ascii
third is "LF" - dunno if in JS it is not CRLF or whatever - you can use 0x0a...
example (which actually starts sending the raw data you've subcribed to) in swift:
let array: [UInt8] = [2, 0x64, 0x0a]
let data: Data = Data(array)
peripheral.writeValue(data, for: characteristic, type: .withoutResponse)
I am building a simple VoIP app using asterisk-manager module on nodeJS. The asterisk server is installed on centos 7 (basic install) and is hosted on a virtual machine. The code below, listens for agent login event, and popups a url when it receives dtmf key:
var port = 5038,
host = 'asteriskIP',
username = 'popup',
password = 'popup',
open = require('open'),
mysql = require('mysql'),
ami = new require('asterisk-manager')(port, host, username, password, true);
ami.keepConnected();
//Mysql server connection pool
var pool = mysql.createPool({
host: host,
user: 'user',
password: 'password',
database: 'db'
});
ami.on('newstate', function (stateEvent) {
var channelState = stateEvent.channelstate;
if (channelState === '6') {
return false;
}
/*
Listen for new channel after agent login
*/
ami.on('newchannel', function (e) {
/* Check if caller id number is empty (This is necessary owning to new channel created as a result of
DTMF. If this returns true, return false else execute mysql query.
*/
if (e.calleridnum === '' && isNaN(e.calleridnum)) {
return false;
} else if (e.calleridnum !== '' && !isNaN(e.calleridnum)) {
var callerId = e.calleridnum;
sql = "INSERT INTO dtmf (caller_id) VALUES ?",
values = [[callerId]];
pool.query(sql, [values], function (error) {
if (error) throw error;
});
/*
Listen for DTMF on 'end' and update current caller table
*/
ami.on('dtmf', function (evt) {
var end = evt.end;
if (end === 'Yes') {
var digit = evt.digit;
sql = `UPDATE dtmf SET caller_lang = ${digit} WHERE caller_id = ?`,
values = [[callerId]];
pool.query(sql, [values], function (error) {
if (error) throw error;
});
/*
This piece of code retrieves DTMF code input and popsup
a url in the agents browser window.
*/
ami.on('bridge', function (evt) {
var state = evt.bridgestate;
if (state === 'Link') {
switch (digit) {
case '1':
open('http://someurl?' + digit);
break;
case '2':
open('http://someurl?' + digit);
break;
default:
}
}
})
}
});
}
return false;
});
});
Everything works fine when I run this code on my mac. However, when I deployed the code to the virtual machine, it inserts and updates the database normally, but no url pops up in my browser. Please is there a way nodeJS app deployed on a virtual machine, can open a window on a users local browser? Thanks.
For security reason there is no browser or OS that will let you pop up a browser window on someone else computer without first being connected.
For that i think you would have to build a client app for example a widget, service or browser extension that would be running on the person computer ... This client could use Socket.io to listen and react to event happening on the Node.js end.
That could be one solution.
I have a situation with a React-based app where I have an input for which I wanted to allow voice input as well. I'm okay making this compatible with Chrome and Firefox only, so I was thinking of using getUserMedia. I know I'll be using Google Cloud's Speech to Text API. However, I have a few caveats:
I want this to stream my audio data live, not just when I'm done recording. This means that a lot of solutions I've found won't work very well, because it's not sufficient to save the file and then send it out to Google Cloud Speech.
I don't trust my front end with my Google Cloud API information. Instead, I already have a service running on the back end which has my credentials, and I want to stream the audio (live) to that back end, then from that back end stream to Google Cloud, and then emit updates to my transcript as they come in back to the Front End.
I already connect to that back end service using socket.io, and I want to manage this entirely via sockets, without having to use Binary.js or anything similar.
Nowhere seems to have a good tutorial on how to do this. What do I do?
First, credit where credit is due: a huge amount of my solution here was created by referencing vin-ni's Google-Cloud-Speech-Node-Socket-Playground project. I had to adapt this some for my React app, however, so I'm sharing a few of the changes I made.
My solution here was composed of four parts, two on the front end and two on the back end.
My front end solution was of two parts:
A utility file to access my microphone, stream audio to the back
end, retrieve data from the back end, run a callback function each
time that data was received from the back end, and then clean up
after itself either when done streaming or when the back end threw
an error.
A microphone component which wrapped my React
functionality.
My back end solution was of two parts:
A utility file to handle the actual speech recognize stream
My main.js file
(These don't need to be separated by any means; our main.js file is just already a behemoth without it.)
Most of my code will just be excerpted, but my utilities will be shown in full because I had a lot of problem with all of the stages involved. My front end utility file looked like this:
// Stream Audio
let bufferSize = 2048,
AudioContext,
context,
processor,
input,
globalStream;
//audioStream constraints
const constraints = {
audio: true,
video: false
};
let AudioStreamer = {
/**
* #param {function} onData Callback to run on data each time it's received
* #param {function} onError Callback to run on an error if one is emitted.
*/
initRecording: function(onData, onError) {
socket.emit('startGoogleCloudStream', {
config: {
encoding: 'LINEAR16',
sampleRateHertz: 16000,
languageCode: 'en-US',
profanityFilter: false,
enableWordTimeOffsets: true
},
interimResults: true // If you want interim results, set this to true
}); //init socket Google Speech Connection
AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();
processor = context.createScriptProcessor(bufferSize, 1, 1);
processor.connect(context.destination);
context.resume();
var handleSuccess = function (stream) {
globalStream = stream;
input = context.createMediaStreamSource(stream);
input.connect(processor);
processor.onaudioprocess = function (e) {
microphoneProcess(e);
};
};
navigator.mediaDevices.getUserMedia(constraints)
.then(handleSuccess);
// Bind the data handler callback
if(onData) {
socket.on('speechData', (data) => {
onData(data);
});
}
socket.on('googleCloudStreamError', (error) => {
if(onError) {
onError('error');
}
// We don't want to emit another end stream event
closeAll();
});
},
stopRecording: function() {
socket.emit('endGoogleCloudStream', '');
closeAll();
}
}
export default AudioStreamer;
// Helper functions
/**
* Processes microphone data into a data stream
*
* #param {object} e Input from the microphone
*/
function microphoneProcess(e) {
var left = e.inputBuffer.getChannelData(0);
var left16 = convertFloat32ToInt16(left);
socket.emit('binaryAudioData', left16);
}
/**
* Converts a buffer from float32 to int16. Necessary for streaming.
* sampleRateHertz of 1600.
*
* #param {object} buffer Buffer being converted
*/
function convertFloat32ToInt16(buffer) {
let l = buffer.length;
let buf = new Int16Array(l / 3);
while (l--) {
if (l % 3 === 0) {
buf[l / 3] = buffer[l] * 0xFFFF;
}
}
return buf.buffer
}
/**
* Stops recording and closes everything down. Runs on error or on stop.
*/
function closeAll() {
// Clear the listeners (prevents issue if opening and closing repeatedly)
socket.off('speechData');
socket.off('googleCloudStreamError');
let tracks = globalStream ? globalStream.getTracks() : null;
let track = tracks ? tracks[0] : null;
if(track) {
track.stop();
}
if(processor) {
if(input) {
try {
input.disconnect(processor);
} catch(error) {
console.warn('Attempt to disconnect input failed.')
}
}
processor.disconnect(context.destination);
}
if(context) {
context.close().then(function () {
input = null;
processor = null;
context = null;
AudioContext = null;
});
}
}
The main salient point of this code (aside from the getUserMedia configuration, which was in and of itself a bit dicey) is that the onaudioprocess callback for the processor emitted speechData events to the socket with the data after converting it to Int16. My main changes here from my linked reference above were to replace all of the functionality to actually update the DOM with callback functions (used by my React component) and to add some error handling that wasn't included in the source.
I was then able to access this in my React Component by just using:
onStart() {
this.setState({
recording: true
});
if(this.props.onStart) {
this.props.onStart();
}
speechToTextUtils.initRecording((data) => {
if(this.props.onUpdate) {
this.props.onUpdate(data);
}
}, (error) => {
console.error('Error when recording', error);
this.setState({recording: false});
// No further action needed, as this already closes itself on error
});
}
onStop() {
this.setState({recording: false});
speechToTextUtils.stopRecording();
if(this.props.onStop) {
this.props.onStop();
}
}
(I passed in my actual data handler as a prop to this component).
Then on the back end, my service handled three main events in main.js:
// Start the stream
socket.on('startGoogleCloudStream', function(request) {
speechToTextUtils.startRecognitionStream(socket, GCSServiceAccount, request);
});
// Receive audio data
socket.on('binaryAudioData', function(data) {
speechToTextUtils.receiveData(data);
});
// End the audio stream
socket.on('endGoogleCloudStream', function() {
speechToTextUtils.stopRecognitionStream();
});
My speechToTextUtils then looked like:
// Google Cloud
const speech = require('#google-cloud/speech');
let speechClient = null;
let recognizeStream = null;
module.exports = {
/**
* #param {object} client A socket client on which to emit events
* #param {object} GCSServiceAccount The credentials for our google cloud API access
* #param {object} request A request object of the form expected by streamingRecognize. Variable keys and setup.
*/
startRecognitionStream: function (client, GCSServiceAccount, request) {
if(!speechClient) {
speechClient = new speech.SpeechClient({
projectId: 'Insert your project ID here',
credentials: GCSServiceAccount
}); // Creates a client
}
recognizeStream = speechClient.streamingRecognize(request)
.on('error', (err) => {
console.error('Error when processing audio: ' + (err && err.code ? 'Code: ' + err.code + ' ' : '') + (err && err.details ? err.details : ''));
client.emit('googleCloudStreamError', err);
this.stopRecognitionStream();
})
.on('data', (data) => {
client.emit('speechData', data);
// if end of utterance, let's restart stream
// this is a small hack. After 65 seconds of silence, the stream will still throw an error for speech length limit
if (data.results[0] && data.results[0].isFinal) {
this.stopRecognitionStream();
this.startRecognitionStream(client, GCSServiceAccount, request);
// console.log('restarted stream serverside');
}
});
},
/**
* Closes the recognize stream and wipes it
*/
stopRecognitionStream: function () {
if (recognizeStream) {
recognizeStream.end();
}
recognizeStream = null;
},
/**
* Receives streaming data and writes it to the recognizeStream for transcription
*
* #param {Buffer} data A section of audio data
*/
receiveData: function (data) {
if (recognizeStream) {
recognizeStream.write(data);
}
}
};
(Again, you don't strictly need this util file, and you could certainly put the speechClient as a const on top of the file depending on how you get your credentials; this is just how I implemented it.)
And that, finally, should be enough to get you started on this. I encourage you to do your best to understand this code before you reuse or modify it, as it may not work 'out of the box' for you, but unlike all other sources I have found, this should get you at least started on all involved stages of the project. It is my hope that this answer will prevent others from suffering like I have suffered.