Record audio, sync to loop, offset latency and export portion - javascript

I am building a web app which allows users to listen to a loop of instrumental music and then record vocals on top. This is all working using Recorder.js however there are a few problems:
There is latency with recording, so this needs to be set by the user before pressing record.
The exported loop is not always the same length as the sample rate might not match the time needed exactly
However since then I went back to the drawing board and asked: What's best for the user?. This gave me a new set of requirements:
Backing loop plays continuously in the background
Recording starts and stops whenever the user chooses
Recording then plays back in sync with loop (the dead time between loops is automatically filled with blank audio)
User can slide an offset slider to adjust for small timing issues with latency
User can select which portion of the recording to save (same length as original backing loop)
Here's a diagram of how that would look:
Logic I have so far:
// backing loop
a.startTime = 5
a.duration = 10
a.loop = true
// recording
b.startTime = 22.5
b.duration = 15
b.loop = false
// fill blank space + loop
fill = a.duration - (b.duration % a.duration) // 5
c = b.buffers + (fill * blankBuffers)
c.startTime = (context.currentTime - a.startTime) % a.duration
c.duration = 20
c.loop = true
// user corrects timing offset
c.startTime = ((context.currentTime - a.startTime) % a.duration) - offset
// user choose favourite loop
? this is where I start to lose the plot!
Here is an example of chopping the buffers sent from Recorder.js:
// shorten the length of buffers
start = context.sampleRate * 2; // start at 2 seconds
end = context.sampleRate * 3; // end at 3 seconds
buffers.push(buffers.subarray(start, end));
And more example code from the previous versions i've been working on:
https://github.com/mattdiamond/Recorderjs/issues/105
Any help in working out how to slice the buffers for the exported loop or improving this logic would be greatly appreciated!
UPDATE
Using this example I was able to find out how to insert blank space into the recording:
http://mdn.github.io/audio-buffer/
I've now managed to almost replicate the functionality I need, however the white noise seems off. Is there a miscalculation somewhere?
http://kmturley.github.io/Recorderjs/loop.html

I managed to solve this by writing the following logic
diff = track2.startTime - track1.startTime
before = Math.round((diff % track1.duration) * 44100)
after = Math.round((track1.duration - ((diff + track2.duration) % track1.duration)) * 44100)
newAudio = [before data] + [recording data] + [after data]
and in javascript code it looks like this:
var i = 0,
channel = 0,
channelTotal = 2,
num = 0,
vocalsRecording = this.createBuffer(vocalsBuffers, channelTotal),
diff = this.recorder.startTime - backingInstance.startTime + (offset / 1000),
before = Math.round((diff % backingInstance.buffer.duration) * this.context.sampleRate),
after = Math.round((backingInstance.buffer.duration - ((diff + vocalsRecording.duration) % backingInstance.buffer.duration)) * this.context.sampleRate),
audioBuffer = this.context.createBuffer(channelTotal, before + vocalsBuffers[0].length + after, this.context.sampleRate),
buffer = null;
// loop through the audio left, right channels
for (channel = 0; channel < channelTotal; channel += 1) {
buffer = audioBuffer.getChannelData(channel);
// fill the empty space before the recording
for (i = 0; i < before; i += 1) {
buffer[num] = 0;
num += 1;
}
// add the recording data
for (i = 0; i < vocalsBuffers[channel].length; i += 1) {
buffer[num] = vocalsBuffers[channel][i];
num += 1;
}
// fill the empty space at the end of the recording
for (i = 0; i < after; i += 1) {
buffer[num] = 0;
num += 1;
}
}
// now return the new audio which should be the exact same length
return audioBuffer;
You can view a full working example here:
http://kmturley.github.io/Recorderjs/loop.html

Related

Is it possible to specify a range for the volume from AudioWorkletProcessor?

I followed the W3C's Documentation to implement a "vumeter" (code) using JavaScript's Web Audio API specifically the AudioWorkletProcessor and AudioWorkletNode interfaces. I would like to know why is the RMS level is compared with the previous volume multiplied by a "smoothing factor" (sorry I just started learning this topic so it's all new for me):
// Calculate the RMS level and update the volume.
rms = Math.sqrt(sum / samples.length);
this._volume = Math.max(rms, this._volume * SMOOTHING_FACTOR);
Also what is the range that the volume's variable can take? Is it possible to know so that we can assume that the volume will be a value in a 0...100 range?
Process method code:
process (inputs, outputs, parameters) {
const input = inputs[0];
// Note that the input will be down-mixed to mono; however, if no inputs are
// connected then zero channels will be passed in.
if (input.length > 0) {
const samples = input[0];
let sum = 0;
let rms = 0;
// Calculated the squared-sum.
for (let i = 0; i < samples.length; ++i)
sum += samples[i] * samples[i];
// Calculate the RMS level and update the volume.
rms = Math.sqrt(sum / samples.length);
this._volume = Math.max(rms, this._volume * SMOOTHING_FACTOR);
// Update and sync the volume property with the main thread.
this._nextUpdateFrame -= samples.length;
if (this._nextUpdateFrame < 0) {
this._nextUpdateFrame += this.intervalInFrames;
this.port.postMessage({volume: this._volume});
}
}
// Keep on processing if the volume is above a threshold, so that
// disconnecting inputs does not immediately cause the meter to stop
// computing its smoothed value.
return this._volume >= MINIMUM_VALUE;
}

Html5 interpolation-latency problems

I have a node.js server which receives the client position and emits it to everyone once every 100ms. On the client side I have already implemented client prediction, reconciliation and entity interpolation. The problem is that if the latency gets higher, the interpolation is not smooth. There is a constant jitter and the player has different velocities. This is my interpolation code:
function updatePlayers(player) {
var t1 = previousLastUpdate; // timestamp of the previous postion
var t2 = lastUpdate; // timestamp of the actual position
var tickRate = t2 - t1;
if (tickRate < 100)
tickRate = 100;
// sometimes the difference is lower than 100(which is the server update rate) and
//this also creates jitter
var renderTime = Date.now() - tickRate;
if(renderTime >= t1 && renderTime <= t2) {
var portion = renderTime - t1;
var percentage = portion / tickRate;
// interpolate from last known position to actual position
var interpX = lerp(player.lastX, player.getX(), percentage);
var interpY = lerp(player.lastY, player.getY(), percentage);
player.setPrevX(interpX);
player.setPrevY(interpY);
}
}
So, my question is: how to get rid of this problem? I can't use dead reckoning because the players change their direction fast.

Capture photos in interval from canvas

I have a script that allows me to show in canvas the webcam and 'download' a specific frame within some intervals.
I am having trouble when time parameters are big (30 minutes of captures every 2 seconds). It works smoothly for about 15 minutes and then crashes (firefox closes with out of memory error). Also, after restarting firefox sometimes many 0 byte photos are taken during a 3-4 mins and then starts working again. I am running this in an old 2 GB RAM machine placed in the lab, is there a way to reduce memory usage?
Here is the piece of code with parameters and the function realizarCapturas.
I can add the resting code but I think the part to optimize should be this one.
var frecuenciaComienzoCaptura = 1; // how long till next capture
var frecuenciaCaptura = 3; //seconds between photos
var duracion = 5; // amount of photos to capture
function realizarCapturas(){
var i = 1;
var interval = setInterval(function(){
if(i <= duracion){
context.drawImage(video, 0, 0, 640, 480);
var imagen = document.getElementById("imagen");
imagen.href = canvas.toDataURL("image/png");
var now = new Date();
var filename = formatNumber(now.getHours()) + "-" + formatNumber(now.getMinutes()) + "-" + formatNumber(now.getSeconds());
imagen.download = filename + ".png"; // Make sure the browser downloads the image
imagen.click(); // Trigger the click
i = i+1;
}else{
clearInterval(interval);
}
}, frecuenciaCaptura * 1000);
}
setInterval(function(){
realizarCapturas();
}, frecuenciaComienzoCaptura * 1000 * 60 * 60);
realizarCapturas();
}, false);
As a rule NEVER use setInterval as it can be a source of call stack overflows which are very difficult to detect in code.
Your problem is that you are not clearing all the intervals you are generating and thus every 3 seconds you are creating a new interval event. Eventually the time it takes to run the little bit of code will be longer than than can be managed by all the interval events you have created and thus each interval will continue to push their events onto the call stack but will not get a chance to be run until more intervals have been place on the stack eventually causing the crash. Nor does setInterval guarantee the time between events are accurate.
Use setTimeout instead. That way you will only ever generate event as needed and you do not have to keep a handle to turn off events.
Below is your code written so that you will never have a call stack overflow.
var frecuenciaComienzoCaptura = 1 * 1000* 60 * 60; // how long till next capture
var frecuenciaCaptura = 3 * 1000; //seconds between photos
var duracion = 5; // amount of photos to capture
var counter = 0;
// the capture function
var captura = function () {
counter = counter + 1;
if(counter < duracion){ // do we need more images?
// only create timer events as needed.
setTimeout(captura, frecuenciaCaptura); //set time till next image
}
context.drawImage(video, 0, 0, 640, 480);
var imagen = document.getElementById("imagen");
imagen.href = canvas.toDataURL("image/png");
var now = new Date();
var filename = formatNumber(now.getHours()) + "-" + formatNumber(now.getMinutes()) + "-" + formatNumber(now.getSeconds());
imagen.download = filename + ".png"; // Make sure the browser downloads the image
imagen.click(); // Trigger the click
}
function realizarCapturas() {
// request next batch of captures by only creating one timer event as we need
setTimeout(realizarCapturas,frecuenciaComienzoCaptura);
counter = 0; // reset counter
captura(); // capture timages
}
// start captures.
realizarCapturas();

How do I determine the efficiency of mouse movement from Point A to Point B in Javascript?

I am trying to create an analytical program which keeps track of user mouse movement on a website and stores the data in a DB. Here is where I am stuck:
Assuming the mouse is always starting at the middle of the screen, and the user is instructed to move it to a particular element, how do I determine the efficiency and accuracy of that movement. I need to keep in mind the duration from start of hovering till the click, but I want to also include the hovering path of the mouse.
A perfect score would be a perfect line from Point A to Point B in x seconds, how do I determine the score of a curved path in 2x seconds, or an instance where the path goes in the wrong direction before proceeding to Point B? Are there any algorithms in existence?
Thanks for your help!
Here is a JSFiddle that I created. Click on the START box and then click on the FINISH box. Hopefully this will help you get started.
var start = false;
var start_time,end_time;
var points = [];
$("#start").click(function() {
start = true;
points = [];
start_time = Date.now();
});
$("#finish").click(function() {
start = false;
distance = travelledDistance();
time = (Date.now() - start_time)/1000;
var center_x_start = $("#start").offset().left + $("#start").width() / 2;
var center_y_start = $("#start").offset().top + $("#start").height() / 2;
var center_x_finish = $("#finish").offset().left + $("#finish").width() / 2;
var center_y_finish = $("#finish").offset().top + $("#finish").height() / 2;
var straight_distance = Math.round(Math.sqrt(Math.pow(center_x_finish - center_x_start, 2) + Math.pow(center_y_finish - center_y_start, 2)));
$("#time").text(+time+"s");
$("#distance").text(distance+"px");
$("#straight_distance").text(straight_distance+"px");
});
$(document).mousemove(function( event ) {
if(!start)
return;
points.push(event.pageX + "," + event.pageY);
});
function travelledDistance(){
var distance = 0;
for (i = 0; i < points.length - 1; i++) {
start_point = points[i].split(",");
end_point = points[i+1].split(",");
distance += Math.round(Math.sqrt(Math.pow(end_point[0] - start_point[0], 2) + Math.pow(end_point[1] - start_point[1], 2)));
}
return distance;
}
UPDATE
I made a new version here. Now you can drag the targets to check the different results.

Incrementing a number smoothly with a variable time period in JS

I have a really simple JS counter which I display on a dashboard like screen which does the following:
Every 5 minutes it makes an jsonp call and retrieves a "total" number
It then displays this number to the screen by incrementing the last total displayed till it is equal to the new total. (the number can only ever increase)
I'm having some trouble with making the number increment smoothly. What I would like to do is find a delta (i.e. New total - old total) and increment the number gradually over the 5 minutes till the next call so it looks like a nice smooth transition.
Any ideas on how I can do this?
Currently some of my code looks like this (This block get's called every 5mins. And yes, it's in dire need of a refactor...)
var LAST_NUMBER_OF_SESSIONS = null;
var five_minutes_in_seconds = 300;
var new_number_of_sessions;
$.getJSON('http://blah.com/live_stats/default_jsonp.aspx?callback=?', function(data) {
if(LAST_NUMBER_OF_SESSIONS === null){
LAST_NUMBER_OF_SESSIONS = data.total_sessions;
}
new_number_of_sessions = data.total_sessions;
var delta = Math.floor(new_number_of_sessions - LAST_NUMBER_OF_SESSIONS);
var time_interval = (five_minutes_in_seconds / delta) * 1000;
var old_value = LAST_NUMBER_OF_SESSIONS;
var new_value = null;
sessions_interval = setInterval(function (){
new_value = parseInt(old_value, 10) + 1;
$('#stats').text(new_value);
old_value = new_value;
if(new_value >= new_number_of_sessions){
clearInterval(sessions_interval);
}
}, time_interval);
LAST_NUMBER_OF_SESSIONS = new_value;
});
}
This code it seems to increment the number very quickly at the start of the 5min period and then stop so it's not exactly right...
Try this:
var total = 0,
delta = 0,
stats = $('#stats').text( total );
function increment() {
var v = +stats.text();
if ( v < total ) {
stats.text( v + 1 );
} else {
$.getJSON('http://...', function(data) { // added data here
delta = Math.floor( 300000 / ( data.total_sessions - total ) );
total = data.total_sessions;
});
}
setTimeout(increment, delta);
}
Update:
In order to test my code, I had to simulate the JSON reponse - I used an array of numbers. See here: http://jsfiddle.net/simevidas/MwQKM/
(In the demo, I use an interval of 5 seconds instead of 5 minutes.)
I am not exactly sure why your code doesn't work as expected, although I suspect that it has to do with line LAST_NUMBER_OF_SESSIONS = new_value;. I wrote something similar and it works fine. It's not that different from what you have, minus that last line of code.

Categories

Resources