Develop Reference

Web Serial API crashes Zebra DS9208 on initial connection

I am attempting to use a Zebra DS9208 scanner to capture barcode data into a web page. For some reason, when I first select the serial port and connect to the scanner, the very first scan causes the barcode reader to crash (it locks into read mode, refuses to read additional scans, then disconnects from the computer and reboots itself). When I establish the connection a second time, the scanner continuously reads scans without issues. Anyone able to spot anything in my code that could cause this?
Connect Script:
async function connect() {
// - Request a port and open a connection.
port = await navigator.serial.requestPort();
// - Wait for the port to open.
await port.open({ baudRate: 9600 });
// Send a bunch of Bell characters so we can hear that the scanner understands us.
await quadBeep();
// Read the stream let textDecoder = new TextDecoderStream();
inputDone = port.readable.pipeTo(textDecoder.writable);
reader = textDecoder.readable.getReader();
readScans();
}
Beep Script (used to make the scanner beep by sending a BELL character)
function quadBeep() {
console.log('Quad Beep Requested');
//Write to output stream
const writer = port.writable.getWriter();
const data = new Uint8Array([07, 07, 07, 07]);
writer.write(data);
//allow the serial port to be closed later
writer.releaseLock();
return;
}
Read Loop to source data from the scanner:
`async function readScans() {
//Listen to data coming from the serial device
async function readScans() {
while (true) {
try {
const { value, done } = await reader.read();
await saveScan(value); //process the scan
console.log('readScans Barcode: ' + value);
document.getElementById('scan').innerHTML += value;
if (done) {
//Allow the serial port to be closed later.
console.log('readScans done value: ' + done);
reader.releaseLock();
break;
}
} catch(error) {
console.log('readScans Error: ' + error);
break;
}
}
}
Save Function, to write the data (eventually will submit the barcode via AJAX)
function saveScan(barcode) {
var session = document.getElementById('sessionID').getAttribute('data-value');
if (barcode == previousBarcode) {
//duplicate scan.
console.log('saveScan Duplicate');
return;
} else {
//Submit the scan
previousBarcode = barcode; //store the barcode so it doesn't get rescanned
console.log('saveScan Previous set to: ' + barcode);
//future AJAX FUNCTION GOES HERE
return;
}
}

How do I stream live audio from the browser to Google Cloud Speech via socket.io?

I have a situation with a React-based app where I have an input for which I wanted to allow voice input as well. I'm okay making this compatible with Chrome and Firefox only, so I was thinking of using getUserMedia. I know I'll be using Google Cloud's Speech to Text API. However, I have a few caveats:
I want this to stream my audio data live, not just when I'm done recording. This means that a lot of solutions I've found won't work very well, because it's not sufficient to save the file and then send it out to Google Cloud Speech.
I don't trust my front end with my Google Cloud API information. Instead, I already have a service running on the back end which has my credentials, and I want to stream the audio (live) to that back end, then from that back end stream to Google Cloud, and then emit updates to my transcript as they come in back to the Front End.
I already connect to that back end service using socket.io, and I want to manage this entirely via sockets, without having to use Binary.js or anything similar.
Nowhere seems to have a good tutorial on how to do this. What do I do?

First, credit where credit is due: a huge amount of my solution here was created by referencing vin-ni's Google-Cloud-Speech-Node-Socket-Playground project. I had to adapt this some for my React app, however, so I'm sharing a few of the changes I made.
My solution here was composed of four parts, two on the front end and two on the back end.
My front end solution was of two parts:
A utility file to access my microphone, stream audio to the back
end, retrieve data from the back end, run a callback function each
time that data was received from the back end, and then clean up
after itself either when done streaming or when the back end threw
an error.
A microphone component which wrapped my React
functionality.
My back end solution was of two parts:
A utility file to handle the actual speech recognize stream
My main.js file
(These don't need to be separated by any means; our main.js file is just already a behemoth without it.)
Most of my code will just be excerpted, but my utilities will be shown in full because I had a lot of problem with all of the stages involved. My front end utility file looked like this:
// Stream Audio
let bufferSize = 2048,
AudioContext,
context,
processor,
input,
globalStream;
//audioStream constraints
const constraints = {
audio: true,
video: false
};
let AudioStreamer = {
/**
* #param {function} onData Callback to run on data each time it's received
* #param {function} onError Callback to run on an error if one is emitted.
*/
initRecording: function(onData, onError) {
socket.emit('startGoogleCloudStream', {
config: {
encoding: 'LINEAR16',
sampleRateHertz: 16000,
languageCode: 'en-US',
profanityFilter: false,
enableWordTimeOffsets: true
},
interimResults: true // If you want interim results, set this to true
}); //init socket Google Speech Connection
AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();
processor = context.createScriptProcessor(bufferSize, 1, 1);
processor.connect(context.destination);
context.resume();
var handleSuccess = function (stream) {
globalStream = stream;
input = context.createMediaStreamSource(stream);
input.connect(processor);
processor.onaudioprocess = function (e) {
microphoneProcess(e);
};
};
navigator.mediaDevices.getUserMedia(constraints)
.then(handleSuccess);
// Bind the data handler callback
if(onData) {
socket.on('speechData', (data) => {
onData(data);
});
}
socket.on('googleCloudStreamError', (error) => {
if(onError) {
onError('error');
}
// We don't want to emit another end stream event
closeAll();
});
},
stopRecording: function() {
socket.emit('endGoogleCloudStream', '');
closeAll();
}
}
export default AudioStreamer;
// Helper functions
/**
* Processes microphone data into a data stream
*
* #param {object} e Input from the microphone
*/
function microphoneProcess(e) {
var left = e.inputBuffer.getChannelData(0);
var left16 = convertFloat32ToInt16(left);
socket.emit('binaryAudioData', left16);
}
/**
* Converts a buffer from float32 to int16. Necessary for streaming.
* sampleRateHertz of 1600.
*
* #param {object} buffer Buffer being converted
*/
function convertFloat32ToInt16(buffer) {
let l = buffer.length;
let buf = new Int16Array(l / 3);
while (l--) {
if (l % 3 === 0) {
buf[l / 3] = buffer[l] * 0xFFFF;
}
}
return buf.buffer
}
/**
* Stops recording and closes everything down. Runs on error or on stop.
*/
function closeAll() {
// Clear the listeners (prevents issue if opening and closing repeatedly)
socket.off('speechData');
socket.off('googleCloudStreamError');
let tracks = globalStream ? globalStream.getTracks() : null;
let track = tracks ? tracks[0] : null;
if(track) {
track.stop();
}
if(processor) {
if(input) {
try {
input.disconnect(processor);
} catch(error) {
console.warn('Attempt to disconnect input failed.')
}
}
processor.disconnect(context.destination);
}
if(context) {
context.close().then(function () {
input = null;
processor = null;
context = null;
AudioContext = null;
});
}
}
The main salient point of this code (aside from the getUserMedia configuration, which was in and of itself a bit dicey) is that the onaudioprocess callback for the processor emitted speechData events to the socket with the data after converting it to Int16. My main changes here from my linked reference above were to replace all of the functionality to actually update the DOM with callback functions (used by my React component) and to add some error handling that wasn't included in the source.
I was then able to access this in my React Component by just using:
onStart() {
this.setState({
recording: true
});
if(this.props.onStart) {
this.props.onStart();
}
speechToTextUtils.initRecording((data) => {
if(this.props.onUpdate) {
this.props.onUpdate(data);
}
}, (error) => {
console.error('Error when recording', error);
this.setState({recording: false});
// No further action needed, as this already closes itself on error
});
}
onStop() {
this.setState({recording: false});
speechToTextUtils.stopRecording();
if(this.props.onStop) {
this.props.onStop();
}
}
(I passed in my actual data handler as a prop to this component).
Then on the back end, my service handled three main events in main.js:
// Start the stream
socket.on('startGoogleCloudStream', function(request) {
speechToTextUtils.startRecognitionStream(socket, GCSServiceAccount, request);
});
// Receive audio data
socket.on('binaryAudioData', function(data) {
speechToTextUtils.receiveData(data);
});
// End the audio stream
socket.on('endGoogleCloudStream', function() {
speechToTextUtils.stopRecognitionStream();
});
My speechToTextUtils then looked like:
// Google Cloud
const speech = require('#google-cloud/speech');
let speechClient = null;
let recognizeStream = null;
module.exports = {
/**
* #param {object} client A socket client on which to emit events
* #param {object} GCSServiceAccount The credentials for our google cloud API access
* #param {object} request A request object of the form expected by streamingRecognize. Variable keys and setup.
*/
startRecognitionStream: function (client, GCSServiceAccount, request) {
if(!speechClient) {
speechClient = new speech.SpeechClient({
projectId: 'Insert your project ID here',
credentials: GCSServiceAccount
}); // Creates a client
}
recognizeStream = speechClient.streamingRecognize(request)
.on('error', (err) => {
console.error('Error when processing audio: ' + (err && err.code ? 'Code: ' + err.code + ' ' : '') + (err && err.details ? err.details : ''));
client.emit('googleCloudStreamError', err);
this.stopRecognitionStream();
})
.on('data', (data) => {
client.emit('speechData', data);
// if end of utterance, let's restart stream
// this is a small hack. After 65 seconds of silence, the stream will still throw an error for speech length limit
if (data.results[0] && data.results[0].isFinal) {
this.stopRecognitionStream();
this.startRecognitionStream(client, GCSServiceAccount, request);
// console.log('restarted stream serverside');
}
});
},
/**
* Closes the recognize stream and wipes it
*/
stopRecognitionStream: function () {
if (recognizeStream) {
recognizeStream.end();
}
recognizeStream = null;
},
/**
* Receives streaming data and writes it to the recognizeStream for transcription
*
* #param {Buffer} data A section of audio data
*/
receiveData: function (data) {
if (recognizeStream) {
recognizeStream.write(data);
}
}
};
(Again, you don't strictly need this util file, and you could certainly put the speechClient as a const on top of the file depending on how you get your credentials; this is just how I implemented it.)
And that, finally, should be enough to get you started on this. I encourage you to do your best to understand this code before you reuse or modify it, as it may not work 'out of the box' for you, but unlike all other sources I have found, this should get you at least started on all involved stages of the project. It is my hope that this answer will prevent others from suffering like I have suffered.

node.js - pngjs error: "Stream not writable" randomly

I am working with pngjs through many of it's methods. Most of the time, they work fine. However, like in the following example, I get an error: "Stream is not writable"
var fs = require('fs'),
PNG = require('pngjs').PNG;
var dst = new PNG({width: 100, height: 50});
fs.createReadStream('http://1.1m.yt/hry7Eby.png') //download this picture in order to examine the code.
.pipe(new PNG())
.on('parsed', function(data) {
console.log(data);
});
This case is not singular, I get this error on 1 random png image once a day, through all of pngjs methods, and that error obviously crashes my app.
(note: you can't use the http link I gave you with a readStream, you will have to download & rename it and do something like):
fs.createReadStream('1.png')
Thank you for your time and effort.

This seems to be a bug in the library, though I'm wary of saying so as I'm no expert in PNGs. The parser seems to complete while the stream is still writing. It encounters the IEND, and so calls this:
ParserAsync.prototype._finished = function() {
if (this.errord) {
return;
}
if (!this._inflate) {
this.emit('error', 'No Inflate block');
}
else {
// no more data to inflate
this._inflate.end();
}
this.destroySoon();
};
If you comment out the this.destroySoon(); it finishes the image correctly, instead of eventually calling this function:
ChunkStream.prototype.end = function(data, encoding) {
if (data) {
this.write(data, encoding);
}
this.writable = false;
// already destroyed
if (!this._buffers) {
return;
}
// enqueue or handle end
if (this._buffers.length === 0) {
this._end();
}
else {
this._buffers.push(null);
this._process();
}
};
...which would otherwise end up setting the stream.writeable to false, or, if you comment that out, to pushing a null value into the _buffers array and screwing up the ChunkStream._processRead.
I'm fairly certain this is a synchronicity problem between the time the zlib parser takes to complete and the time the stream takes to complete, since if you do this synchronously it works fine:
var data = fs.readFileSync('pic.png');
var png = PNG.sync.read(data);
var buff = PNG.sync.write(png);
fs.writeFileSync('out2.png', buff);

Update HTML object with node.js and javascript

I'm new to nodejs and jquery, and I'm trying to update one single html object using a script.
I am using a Raspberry pi 2 and a ultrasonic sensor, to measure distance. I want to measure continuous, and update the html document at the same time with the real time values.
When I try to run my code it behaves like a server and not a client. Everything that i console.log() prints in the cmd and not in the browesers' console. When I run my code now i do it with "sudo node surveyor.js", but nothing happens in the html-document. I have linked it properly to the script. I have also tried document.getElementsByTagName("h6").innerHTML = distance.toFixed(2), but the error is "document is not defiend".
Is there any easy way to fix this?
My code this far is:
var statistics = require('math-statistics');
var usonic = require('r-pi-usonic');
var fs = require("fs");
var path = require("path");
var jsdom = require("jsdom");
var htmlSource = fs.readFileSync("../index.html", "utf8");
var init = function(config) {
usonic.init(function (error) {
if (error) {
console.log('error');
} else {
var sensor = usonic.createSensor(config.echoPin, config.triggerPin, config.timeout);
//console.log(config);
var distances;
(function measure() {
if (!distances || distances.length === config.rate) {
if (distances) {
print(distances);
}
distances = [];
}
setTimeout(function() {
distances.push(sensor());
measure();
}, config.delay);
}());
}
});
};
var print = function(distances) {
var distance = statistics.median(distances);
process.stdout.clearLine();
process.stdout.cursorTo(0);
if (distance < 0) {
process.stdout.write('Error: Measurement timeout.\n');
} else {
process.stdout.write('Distance: ' + distance.toFixed(2) + ' cm');
call_jsdom(htmlSource, function (window) {
var $ = window.$;
$("h6").replaceWith(distance.toFixed(2));
console.log(documentToSource(window.document));
});
}
};
function documentToSource(doc) {
// The non-standard window.document.outerHTML also exists,
// but currently does not preserve source code structure as well
// The following two operations are non-standard
return doc.doctype.toString()+doc.innerHTML;
}
function call_jsdom(source, callback) {
jsdom.env(
source,
[ 'jquery-1.7.1.min.js' ],
function(errors, window) {
process.nextTick(
function () {
if (errors) {
throw new Error("There were errors: "+errors);
}
callback(window);
}
);
}
);
}
init({
echoPin: 15, //Echo pin
triggerPin: 14, //Trigger pin
timeout: 1000, //Measurement timeout in µs
delay: 60, //Measurement delay in ms
rate: 5 //Measurements per sample
});

Node.js is a server-side implementation of JavaScript. It's ok to do all the sensors operations and calculations on server-side, but you need some mechanism to provide the results to your clients. If they are going to use your application by using a web browser, you must run a HTTP server, like Express.js, and create a route (something like http://localhost/surveyor or just http://localhost/) that calls a method you have implemented on server-side and do something with the result. One possible way to return this resulting data to the clients is by rendering an HTML page that shows them. For that you should use a Template Engine.
Any DOM manipulation should be done on client-side (you could, for example, include a <script> tag inside your template HTML just to try and understand how it works, but it is not recommended to do this in production environments).
Try searching google for Node.js examples and tutorials and you will get it :)

Serial port not working?

I made a program that sends out data to my arduino which detects what was sent and then turns on the correct pin according to what key is pressed.
When using the arduino software from my windows computer the arduino sketch works fine, I can make each pin turn on and off by sending either W A S Or D.
When sending via node the RX light on the arduino flashes but nothing else happens.
Can anyone help?
Node.js program:
var httpServer = require('http').createServer(function(req, response){ /* Serve your static files */ })
httpServer.listen(8080);
var nowjs = require("now");
var everyone = nowjs.initialize(httpServer);
everyone.now.logStuff = function(msg){
console.log(msg);
}
var SerialPort = require('serialport2').SerialPort;
var assert = require('assert');
var portName;
if (process.platform == 'win32') {
portName = 'COM4';
} else if (process.platform == 'darwin') {
portName = '/dev/cu.usbserial-A800eFN5';
} else {
portName = '/dev/ttyUSB0';
}
var readData = '';
var sp = new SerialPort();
sp.on('close', function (err) {
console.log('port closed');
});
sp.on('error', function (err) {
console.error("error", err);
});
sp.on('open', function () {
console.log('port opened... Press reset on the Arduino.');
});
sp.open(portName, {
baudRate: 9600,
dataBits: 8,
parity: 'none',
stopBits: 1,
flowControl: false
});
everyone.now.forward = function() {
sp.write("w");
}
everyone.now.back = function() {
sp.write("s");
}
everyone.now.left = function() {
sp.write("a");
}
everyone.now.right = function() {
sp.write("d");
}
sp.on('data', function(data) {
console.log(data.toString());
});
Arduino Program:
void setup(){
Serial.begin(9600);
Serial.write("READY");
//Set all the pins we need to output pins
pinMode(8, OUTPUT);
pinMode(9, OUTPUT);
pinMode(10, OUTPUT);
pinMode(11, OUTPUT);
}
void loop (){
if (Serial.available() > 0) {
//read serial as a character
char ser = Serial.read();
Serial.write(ser);
//NOTE because the serial is read as "char" and not "int", the read value must be compared to character numbers
//hence the quotes around the numbers in the case statement
switch (ser) {
case 'w':
move(8);
break;
case 's':
move(9);
break;
case 'a':
move(10);
break;
case 'q':
move(10);
move(8);
break;
case 'd':
move(11);
break;
case 'e':
move(11);
move(8);
break;
}
}
}
void move(int pin){
Serial.print(pin);
digitalWrite(pin, HIGH);
delay(1);
digitalWrite(pin, LOW);
}

I recently dabbled into this. The Arduino automatically resets when it receives serial communication from most things other than the Arduino IDE. This is why you can send from the IDE but not node.js.
I have an Uno and put a capacitor between Reset and Ground.Here's a page with some good info on the subject.
Good luck. http://arduino.cc/playground/Main/DisablingAutoResetOnSerialConnection

I use node on a daily basis to send actions to my Arduino via usb or via bt and it works great in both cases.
I think your problem comes from sending letters. You should send a buffer instead, with the ascii value of the letter, just like that:
myPort.write(Buffer([myValueToBeSent]));
also, for this, I think you would be better with some "logic" interface, with data headers, number of actions, stuff like that. It is no required for you but it will make your code more robust and easier to modify in the future.
Here is an example of how I do it. First, Node:
var dataHeader = 0x0f, //beginning of the data stream, very useful if you intend to send a batch of actions
myFirstAction = 0x01,
mySecondAction = 0x02,
myThirdAction = 0x03;
You then call them like you did:
everyone.now.MyBatchOfActions = function() {
sp.write(Buffer([dataHeader]));
sp.write(Buffer([0x03])); // this is the number of actions for the Arduino code
sp.write(Buffer([myFirstAction]));
sp.write(Buffer([mySecondAction]));
sp.write(Buffer([myThirdAction]));
}
This way it is easy on the Arduino to Serial.read() the data: (Note that you need to define data header and data footer somewhere)
void readCommands(){
while(Serial.available() > 0){
// Read first byte of stream.
uint8_t numberOfActions;
uint8_t recievedByte = Serial.read();
// If first byte is equal to dataHeader, lets do
if(recievedByte == DATA_HEADER){
delay(10);
// Get the number of actions to execute
numberOfActions = Serial.read();
delay(10);
// Execute each actions
for (uint8_t i = 0 ; i < numberOfActions ; i++){
// Get action type
actionType = Serial.read();
if(actionType == 0x01){
// do you first action
}
else if(actionType == 0x02{
// do your second action
}
else if(actionType == 0x03){
// do your third action
}
}
}
}
}
I hope I'm clear and I hope it helps!
Cheers!

On the capacitor and reset issue...
There is a small capacitor between one of the serial control lines and reset on the Arduino in the later models. This capacitor causes the Arduino to reset when the port is opened but otherwise does not interfere with normal serial operation.
This reset trick allows the code upload to reset the Arduino as part of the upload process. When the Arduino starts up the code boot loader runs first for a short time before the loaded code runs.
The upload process is: Reset the Arduino which starts the boot loader, start the upload process in the Arduino IDE, establish communications, upload, then run the loaded code. When the Arduino starts up it waits for uploads for a short period of time, if none are received, it moves on to running the code.
I find this very useful as it allows us to effectively reset the Arduino just by closing and opening the port. In the old Arduino's, without this capacitor, you had to press the reset button at the right time to get the code to upload. And the timing was such that the Arduino spent much more time waiting before it started with the uploaded code.
In the problem described here, I do not believe he was having any troubles due to the reset trick used. It should have had only the effect of resetting the Arduino when he opened the serial port, and from the looks of his information, this is a desired side-effect.

In my case the issue was the reset, but that the serial port was opened - but not available for write until the reset has finished. Putting a 3s delay before writing to the port fixed the issue. Writing ASCII was not an issue.