Develop Reference

ML5 FeatureExtractor addImage not working

I'm following the basic example of the ML5.js featureExtractor. I am not using video. After loading a model I am adding new images to it, and then training again. I get the following error:
Mobilenet.js:323 Uncaught (in promise) Error: Batch size is 0 or NaN. Please choose a non-zero fraction.
at t. (Mobilenet.js:323)
My code seems correct, according to the documentation - using video is optional, so I expect I should be able to re-train the model after just adding images manually. I used the callback for the addImage function, to make sure the images are really added before calling train
let added = 0
let classifier
let featureExtractor = ml5.featureExtractor('MobileNet', modelLoaded)
function modelLoaded() {
classifier = featureExtractor.classification()
classifier.addImage(document.getElementById('person1'), 'nomask', addedImage)
classifier.addImage(document.getElementById('mask1'), 'mask', addedImage)
}
// this gets called twice, but then train goes wrong
function addedImage(){
added++
if(added == 2){
classifier.train((lossValue) => {
console.log('Loss is', lossValue);
})
}

You need to add at least 3 images for training to work.
The following code should work.
let added = 0;
let classifier;
let featureExtractor = ml5.featureExtractor('MobileNet', modelLoaded);
function modelLoaded() {
classifier = featureExtractor.classification()
classifier.addImage(document.getElementById('person1'), 'nomask', addedImage);
classifier.addImage(document.getElementById('person2'), 'nomask', addedImage);
classifier.addImage(document.getElementById('mask1'), 'mask', addedImage);
}
function addedImage(){
added++;
if(added == 3){
classifier.train((lossValue) => {
console.log('Loss is', lossValue);
});
}
A working example: https://glitch.com/edit/#!/ml5-feature-extractor-addimage

Tone.js Tone.BufferSource: buffer is either not set or not loaded

Tone.BufferSource: buffer is either not set or not loaded. This error occurs in try/catch block. It only occurs, when I trigger update function constantly or sometimes randomly.
When this error occurs my audio just turns off for a brief moment.
The logic behind my code. When program starts create function is invoked in the constructor creating Tone.sequence later on when I change/update track parameters i call update fuction,
which calls loopprocessor with new/updated tracks. But when i trigger update which triggers loopprocessor function it runs into tone.sourcebuffer is either not set ir loaded. How can i work around this problem?
My code:
import Tone from "tone";
export function create(tracks, beatNotifier){
const loop = new Tone.Sequence(
loopProcessor(tracks, beatNotifier),
[...new Array(16)].map((_, i) => i),
"16n"
);
Tone.Transport.bpm.value = 120;
Tone.Transport.start();
return loop;
}
export function update(loop, tracks, beatNotifier){
loop.callback = loopProcessor(tracks, beatNotifier);
return loop;
}
function loopProcessor (tracks, beatNotifier) {
const urls = tracks.reduce((acc, {name}) => {
return {...acc, [name]: `http://localhost:3000/src/sounds/${name}.[wav|wav]`};
}, {});
const keys = new Tone.Players(urls, {
fadeOut: "64n"
}).toMaster();
return (time, index) => {
beatNotifier(index);
tracks.forEach(({name, vol, muted, note, beats}) => {
if (beats[index]) {
try {
var vel = Math.random() * 0.5 + 0.5;
keys
.get(name)
.start(time, 0, note, 0, vel);
keys
.get(name).volume.value = muted
? -Infinity
: vol;
} catch(e) {
console.log("error", e);
}
}
});
};
}

I had this problem recently and found a solution that worked for my case.
Tone.js doesn't like it when you initialise an audio buffer inside a function (what you're doing when you call new Tone.Players inside loopprocessor).
To get around this at the top of your code declare a new global variable buffer1 = new Tone.Buffer(url1) for each url that you need. https://tonejs.github.io/docs/r13/Buffer
Then inside loopprocessor just replace urls with each buffer and a name tag and you shouldn't have any problems. So new Tone.Players({"name1": buffer1, "name2": buffer2, ...})

Failed to execute 'start' on 'SpeechRecognition': recognition has already started

I am using a wrapper of Web Speech API for Angular6. I am trying to implement a system of starting-stopping after each 3.5s in order to be able to manipulate the results for these small parts.
Even though I stop the recognition, before starting it again, I keep getting this error Failed to execute 'start' on 'SpeechRecognition': recognition has already started.
As suggested in this post, I first verify whether the speech recognition is active or not and only if not active, I try to start it. https://stackoverflow.com/a/44226843/6904971
Here is the code:
constructor( private http: Http, private service: SpeechRecognitionService, private links: LinksService) {
var recognizing; // will get bool values to verify if recognition is active
this.service.onresult = (e) => {
this.message = e.results[0].item(0).transcript;
};
this.service.onstart = function () {
recognizing = true;
};
this.service.onaudiostart = function () {
recognizing = true;
};
this.service.onerror = function (event) {
recognizing = false;
};
this.service.onsoundstart = function () {
recognizing = true;
};
this.service.onsoundstart = function () {
recognizing = true;
};
this.record = () => {
this.service.start();
setInterval(root.ongoing_recording(), 3500);
};
var root = this;
var speech = '';
this.stop_recording = () => {
this.service.stop();
};
this.ongoing_recording = ()=> {
setTimeout(function(){
if( recognizing === true){
root.service.stop();
root.service.onend = (e) => {
recognizing = false;
speech = root.message;
var sentence = document.createElement('span');
sentence.innerHTML = speech + " ";
document.body.appendChild(sentence);
}
}
}, 3500);
setTimeout(function(){
if(recognizing === false){
root.service.start();
}
}, 3510);
};
}
start() {
this.service.start();
}
stop() {
this.service.stop();
}
record(){
this.record();
}
stop_recording(){
this.stop_recording();
}
ongoing_recording(){
this.ongoing_recording();
}
I think that the timing might not be good (with the setTimeout and interval). Any help would be much appreciated. Thank you! :)

I used Web Speech API for voice search functionality in my site and I was facing a similar sort of situation. It has one microphone icon which toggles the speech recognition on and off. It was working fine in the normal on and off of the button that started speech recognition but was breaking only if you test it rigorously with a continuous button toggle.
Solution:
The thing that worked for me is:
try{
//line of code to start the speech recognition
}
catch{
//line of code to stop the speech recognition
}
So I wrapped the .start() method which was breaking the application in a try block and then added the catch block to stop it. And even if it comes across this problem, on the next button click to turn on the speech recognition, it works. I hope you would be able to extract something from it.

one observation:
you run setInterval() every 3500 ms to invoke ongoing_recording(), but then use setTimeout() with 3500 ms again within ongoing_recording().
Besides that, maybe logging the error handler --where recognizing is also set to false-- could help finding a solution:
in past versions of the SpeechRecognition implementation, not every error did actually stop the recognition (I don't know if that is still the case).
So it might be the case, that recognizing is reset due to an error that did not actually stop the recognition; if this is really the cause of the error when restarting recognition, it could be just catched & ignored.
Also it might be worth trying to re-start the recognition in the onend handler (and onerror).

I am not sure what is the reason that is causing it in your code, but i had the same error and what caused it in my case was that I was calling start() twice in a row, so what fixed it was adding a variable to check if the recognition has started or stopped, so if it has started and I clicked it again it would return speach.stop() to avoid using start() again.
let recognition = new SpeechRecognition();
let status = 0;
document.querySelector(".mic").addEventListener("click",() => {
if (status == 1) {
status = 0;
return recognition.stop();
}
recognition.start();
status = 1;
recognition.onresult = function (event) {
status=0;
var text = event.results[0][0].transcript;
recognition.stop();
};
recognition.onspeechend = function () {
status = 0;
recognition.stop();
};
});

opponent moves promise in multiplayer game

I have spent the last four days studying promises, coroutines, fibers, continuations, etc.
I am still unable to see how to resolve my multiplayer turn-based card game moves, in which the starting player is effectively the game 'controller' of up to five, either AI or human players.
The code below works but has one problem:-
it cannot detect human oppo's card moves and therefore continues playing without them, which, of course makes a mess.
Can anyone please suggest either a change to my overall concept or a way to use promise or one of any other 'synchronising' constructs?
Here are the four key areas of my code:
function oppoPlays () {
// can only go through here if game starter
if (joiner!="") {return;}
for (pp=1; pp<numberofplayers; pp++) {
if (oppoType[pp] == "AI") {
// an AI player moves
.
.
} else {
// non-AI player
var yourTurnmsg="It's "+playerNames[pp]+"'s turn";
// tell human player that it's their turn
$("#text_message").val(yourTurnmsg).trigger(jQuery.Event('keypress', { keyCode: 13, which: 13 }));
// how to detect human oppo's card moved?
}
}
}
// chat functionality
$("#text_message").on("keypress", function(e) {
if (e.keyCode == 13){
payload = new Object();
payload.action = 'chat_text';
payload.chat_text = tmsg; // It's michael29's turn
payload.user_id = playerNames[pp];
payload.game_no = game_no;
socket.send(JSON.stringify(payload));
}
});
// socket gets oppo's response
function checkJson(res, sttr_id, game_no) {
if(res.action=="game_move"){
// find player
var pp=playerNames.indexOf(res.user_id);
cpos=res.cardno;
playCard_oppo(pp, cpos);
}
}
// turn an oppo's card face up and update scores
function playCard_oppo(pp, cardno) {
// and move it to the stack
topoc= parseInt($("#oppo_card" + cardno).css('top'));
leftoc=parseInt($("#oppo_card" + cardno).css('left'));
$("#oppo_card" + cardno).css({ top: topoc, left: leftoc, opacity: "50%" });
.
.
if (joiner=="") {
// tell oppoPlays fn that the card has moved
}
}
The game is similar to uno in concept but with a scoring component
(aimed to help children with basic arithmetic).

Consider having a board state that is a global and player/AI moves modify it. Then, when it is time for the AI opponent to make a move, it consults the current board state and decides the move.
If your board state is just represented by elements on the page you'll need a way to scan it and calculate a useful in-memory representation of the board state. Without details of your implementation it's hard to be more specific.

Maybe start thinking in terms of a play-cycle, something like this :
(host) Broadcasts game state.
(host) Awaits confirmation from all clients that broadcast was received.
(clients) Render game state.
(host) Receives confirmations then informs next client (and hence its player) that it is his/her/its turn.
(host) Awaits player's move
(client) Unlocks UI allowing player to make move.
(player) Makes move.
(client) Sends move command and re-locks UI.
(host) Receives move command and modifies game state accordingly.
Then back to 1.
Also, think of AI players as just a special case of human players. If you can get it right for humans, then (re)introducing AI should be fairly simple.

The solution hinged around two things:-
separating the AI player code from the human player code;
adding and removing a window event that's triggered after a human's move is detected.
The condensed code now looks like this:-
// if this is game starter give each player a turn
if (joiner == "") {
// there's always at least one
pp = 1;
if (oppoType[pp] == "AI") { AIplays(); } else { humanPlays(); }
}
function humanPlays () {
// tell human player that it's their turn
var yourTurnmsg="It's "+playerNames[pp]+"'s turn"
$("#text_message").val(yourTurnmsg).trigger(jQuery.Event('keypress', { keyCode: 13, which: 13 }));
//window.addEventListener("humanPlayed", function(evnt) {
$(window).on("humanPlayed", function(evnt) {
endOfTurn();
});
}
function endOfTurn () {
if (!(winner)) {
if (pp++ != numberofplayers) {
if (oppoType[pp] == "AI") {
setTimeout(function (){ $("#clickForNextPlayer").show(); }, 1000);
} else {
$("#clickForNextPlayer").trigger('click');
}
}
}
}
// click for next player
$("#clickForNextPlayer").on('click', function() {
$("#clickForNextPlayer").hide();
$(window).off("humanPlayed");
if (pp == numberofplayers) {
// uncover outStack for game starter to play
$("#outStackcover").hide();
return;
}
if (oppoType[pp] == "AI") { AIplays(); } else { humanPlays(); }
});
function AIplays () {
AIcardno = chooseCard(pp, diffLevel);
.
.
if ($("#chatWindow").is(":visible")) {
payload = new Object();
payload.action="game_move";
payload.game_no=gamestarted;
payload.user_id=playerNames[pp];
payload.cardno=AIcardno;
socket.send(JSON.stringify(payload));
}
$("#oppo_card" + cc).css('background-image', "url(JerseyTeam" + playerNumbers[(pp == numberofplayers ? 1 : pp)] + ".gif)");
outStackturn();
endOfTurn();
}

Simplest possible irrevocable timout / cooldown function?

I'm trying to add a 1 second cooldown to my send-message system (as in, you can send 1 message per second max). So my initial thought was simply to create a timeout, and before attempting in sending to check if it exists still. That turned out to take more line of code than I anticipated initially.
Is there something I'm missing here? Isn't there something as simple as:
//inside some message sending function
if(!mySuperCooldown)
{
//send message
mySuperCooldown = cooldown(1000);
}
Everything else I construct successfully ends up taking loads of lines, and it appears to me as something someone thought of before. Thank you, and excuse my illiteracy.

Have a flag that allows messages, and set it to false when a message is sent. Then set a timeout for 1000 milliseconds that resets the flag to true.
var allowMessage = true;
function sendMessage(msg) {
if (allowMessage) {
//do something
allowMessage = false;
setTimeout(() => allowMessage = true, 1000);
}
}

Make a higher order function that turns a normal function into one that is rate limited:
function rate_limit(delay, func) {
var last_call = null;
return function() {
if (last_call && (Date.now() - last_call <= delay)) {
return;
}
last_call = Date.now();
return func();
};
}
You can then rate limit any function:
var my_function = rate_limit(1000, function() {
console.log('foo');
});
Running my_function() will only call your original function once per second.