Title is preety much self explanitory on what
I want to achieve here...
I wrote a little script to loop
between 2 drum loops.
There is a tiny delay between the files.
Now I know that there is a hard-disk reading resorce
here that come into the considuration and that is
excatly what I thought (at first) that makes the
tiny delay.
BUT
I even tried to convert to audio to BASE64 and so
all audio files already cached into memory
BUT
STILL!!!
Even with fully base64 cached files the audio is
played with tiny latency between them.
I literraly need 0 latency between the audio files
so technically it would would sound like an endless
loop.
The drum loops are perfectly in sync to a specific
BPM so I know the latency is not coming from the
audio files.
Here is my code:
let validate=0;
let audio=
[
new Audio(),
new Audio()
];
let audioSrc=
[
'drumLoop1.wav',
'drumLoop2.wav'
];
audio[0].src=audioSrc[0];
audio[1].src=audioSrc[1];
audio[0].oncanplaythrough=()=>
{
validate++;
}
audio[1].oncanplaythrough=()=>
{
validate++;
}
let myTimer=setInterval(function()
{
if(validate===2)
{
clearInterval(myTimer);
audio[0].play();
};
},1000);
audio[0].onended=function()
{
audio[1].play();
};
audio[1].onended=function()
{
audio[0].play();
};
I did not paste the base64 code here since the
string is HUGE.
Hope I explained myself clearly on what I am
about to achieve here))
The code is working perfectly except the latency but that is exactly why I am here))
Related
I am currently trying to figure how to play chunked audio with the web audio API, right off the bat everything does work.. however most transitions between chunks aren't as smooth as I want them to be, there's a very very brief moment of silence between most of them.
My current loading and playback code:
const response = await fetch(`${this.src}`)
const reader = response.body.getReader()
let timestamptowaituntil = 0
let tolog = []
let tolog2 = []
while (true) {
const { done, value } = await reader.read()
if (done) {
console.log(tolog)
console.log(tolog2)
console.log(this.ctx)
break
} else {
let audiodata = await this.ctx.decodeAudioData(value.buffer)
let source = this.ctx.createBufferSource()
source.buffer = audiodata
source.connect(this.ctx.destination)
source.start(timestamptowaituntil, 0, audiodata.duration)
timestamptowaituntil +=audiodata.duration
tolog.push(audiodata)
tolog2.push(source)
}
}
How could I go about eliminating these little moments of silence (or overlap)?
Edit: So far I've tried the following
Removing some milliseconds off the waiting time.
Removing the amount of time that is in the latency properties of the AudioContext.
Making a function to get the playback length of the UInt8Array form data using its bitrate (this indeed got me a slightly different result than the .duration property of an audioBuffer, but there still is tiny gaps)
After trying a ton of different approaches, I finally got a thought that solved the issue in the end.
My new idea was to simply play the first chunk when it arrives, and meanwhile collect as many chunks as possible, whenever a chunk is collected, its chained with the previous chunk to make one bigger chunk (this way also makes it works in firefox which requires the chunk to have a header for decoding). The playback of the first chunk is stopped 0.5-1 second before the .duration property claims it would end, this way any anomalies in detecting length are avoided. At that same time, the next chunk is played.
A few things I added to my code for this is the following:
A function to concat two chunks:
const concat = (arrayOne, arrayTwo) => {
let mergedArray = new Uint8Array(arrayOne.length + arrayTwo.length)
mergedArray.set([...arrayOne, ...arrayTwo])
return mergedArray
}
Extra offset when timing:
source.start(timestamptowaituntil, 0, audiodata.duration - .75)
timestamptowaituntil += (audiodata.duration - .75 + this.ctx.currentTime)
This along with some more minor edits has brought me to a solution that makes the chunk-swap impossible to hear (every now and then it is when the cpu is overloaded and the timing slowed).
I'm trying to add filter effects to an audio stream I have playing on my website. I'm able to connect the Tone.js library to the audio stream but I'm not hearing any changes in the audio stream playing on the website. I'm not seeing any errors in the console and I've tried adjusting the filter from 50 to 5000 but nothing seems to have any effect on the audio. Do I need to set up the new Tone.Player() to actually hear the audio? If so, how do you go about setting up the Player if there is no src for the existing audio element.
$('#testBtn').click(async function () {
const audioElement = document.getElementById('theAudioStream');
const mediaElementSource = Tone.context.createMediaElementSource(audioElement);
const filter = new Tone.Filter(50, 'lowpass').toDestination();
Tone.connect(mediaElementSource, filter);
await Tone.start();
console.log("Started?");
});
The audio stream I'm trying to modify is set up from a JsSip call. The code to start the stream is as follows:
var audioStream = document.getElementById('theAudioStream')
//further down in code
currentSession.answer(options);
if (currentSession.connection) {
currentSession.connection.ontrack = function (e) {
audioStream.srcObject = e.streams[0];
audioStream.play();
}
}
I click the test button after the call has started so I know the audio stream is present before initializing the Tone.js Filters
Working solution:
Removing the audioStream.play() from where the JsSIP call is answered solves the issue.
I don't know the exact reason why this solves the issue (it might even be a workaround) but after much trial and error this way allows the audio to be available to ToneJS for effecting.
Any other solutions are welcome.
So what I want is to have constant looping interchanging from different audio sources. For demo purpose I made a little puzzle game - you align numbers in order from 0 to 8 and depending on how you align them different loops are playing. I managed to get the result I want on Chrome Browser, but not on Safari or Firefox. I tried adding a different audio destination or multiple audio contexts but no matter what loop just stops after one iteration in Safari and other browsers except for Chrome.
Here is a link to the demo on code-pen Demo Puzzle with music
please turn down your sound as music might be a little too loud, I didn't master it. And here is basic code I have for Web Audio Api manipulation.
Thanks
*Also it does not work for mobile at all.
const AudioContext = window.AudioContext || window.webkitAudioContext;
var audioContext = new AudioContext();
const audio1 = document.getElementById("aud1");
const audio2 = document.getElementById("aud2");
const audio3 = document.getElementById("aud3");
const audio4 = document.getElementById("aud4");
var chosenTrack = audio2;
let gameStarted = false;
function startGame() {
document.getElementById("sHold").style.display = "none";
document.getElementById("container").style.display = "block";
gameStarted = true;
audioContext.resume();
audioContext = new AudioContext();
audio1.pause();
audio1.play();
audio1.currentTime = 0;
}
setInterval(function() {
if (gameStarted) {
//console.log(audioContext.currentTime );
if (audioContext.currentTime >= 6.4) {
audioContext = new AudioContext();
chosenTrack.pause();
chosenTrack.play();
chosenTrack.currentTime = 0;
}
}
}, 5);
Some thoughts:
You're not really using Web Audio this way, you're still using audio elements as the source which doesn't help if you want to be able to achieve precise timing. You should load them into AudioBuffers and play them using an AudioBufferSourceNode.
If you absolutely want to use audio elements (because the files you use are really massive and you want to stream them) you probably want to use the loop property on it although i doubt if that ends up being precise and gapless.
Never use setInterval to get a callback every frame, use requestAnimationFrame
Don't use setInterval OR requestAnimationFrame to be able to achieve precise audio looping, the javascript thread is not precise enough to do that AND can be held up when other things take a bit more time, too many enemies in screen for example. You should be scheduling ahead of time now and then: https://www.html5rocks.com/en/tutorials/audio/scheduling/
AudioBufferSourceNodes have a loop boolean property which will loop them as precise as possible
Do realise that different audio-decoders (so: different browsers) MIGHT decode audiofiles slightly differently: some may have a few more ms on the start for example. This might become an issue when using multiple looping AudioBufferSourceNodes, which may all be running out of sync after an x amount of time. I always reschedule something on the exact time needed instead of using the loop property.
I making a game and I want to load 798 sound files, but there is a problem only in Chrome, Firefox fine. Sample code: https://jsfiddle.net/76zb42ag/, see the console (press F12).
Sometimes script loads only 100, 500, 700 files, sometimes is fine. If i reduce the number of files to ex. 300 is ok (always). How can I solve this problem? I need a callback or any ideas? The game will be offline (node webkit).
Javascript code :
var total = 0;
// sample file to download: http://www.sample-videos.com/audio/mp3/crowd-cheering.mp3
// sounds.length = 798 files
var sounds = [
(...limit character, see https://jsfiddle.net/76zb42ag/...)
];
for (var i in sounds) {
load(sounds[i]);
}
function load(file) {
var snd = new Audio();
snd.addEventListener('canplaythrough', loadedAudio, false);
snd.src = file;
}
function loadedAudio() {
total++;
console.log(total);
if (total == sounds.length){
console.log("COMPLETE");
}
}
This isn't really a code problem. It's a general architecture problem.
Depending not only on the number, but also the size of the samples, it's going to be unlikely you can get them all loaded at once. Even if you can, it'll run very poorly because of the high memory use and likely crash the tab after a certain amount of time.
Since it's offline, I would say you could even get away with not pre-loading them at all, since the read speed is going to be nearly instantaneous.
If you find that isn't suitable, or you may need like 5 at once and it might be too slow, I'd say you'll need to architect your game in a way that you can determine which sounds you'll need for a certain game state, and just load those (and remove references to ones you don't need so they can be garbage collected).
This is exactly what all games do when they show you a loading screen, and for the same reasons.
If you want to avoid "loading screens", you can get clever by working out a way to know what is coming up and load it just ahead of time.
I'm trying to stream a large video file to the browser in a <video> tag using websockets.
The video plays fine, but it always waits until it's downloaded the entire video before playing, resulting in a large delay. Setting autoplay = true and preload="none" seems to have no effect on this. So I've looked into chunking the video out and then sending it to the browser as a blob URL. For the chunking I'm using Node-Chunking-Streams
My code so far:
var chunkingStreams = require('chunking-streams');
var SizeChunker = chunkingStreams.Chunker;
var input = fs.createReadStream('src-videos/redcliff450.webm'),
chunker = new SizeChunker({
chunkSize: 2000000
}),
output;
chunker.on('chunkStart', function(id, done) {
output = fs.createWriteStream('src-videos/output/' + id + '.webm');
done();
});
chunker.on('chunkEnd', function(id, done) {
output.end();
done();
});
chunker.on('data', function(chunk) {
output.write(chunk.data);
});
input.pipe(chunker);
//test out the video using just the first chunk
var smallChunk = fs.createReadStream('src-videos/output/0.webm');
client.send(smallChunk);
My plan is to make the chunks small enough to load quickly - say ~2MB - and then send the next one when the clients ready. My issue is though that the first chunk (0) only plays for 3 seconds or so, before skipping straight to the end and stopping. This happens in Chrome and FF.
Increasing the chunk size until it encompasses the whole video still only results in the first 3 seconds playing.
If I play the chunked video 0.webm directly from the HDD in VLC, it plays fine. If I download the stream from within the browser and play it in VLC, it only plays the first 3 seconds. This article describes what I'm looking to do, but over HTTP. Anyone have any pointers for websockets?
removing input.pipe(chunker); solved this. I'm not quite sure the reason for this though, so will investigate as to why.