Develop Reference

Offscreen-Canvas not rendering some of the time

I am trying to implement a in browser raster drawing plugin for the leaflet library that that extends the leaflets GridLayer api. Essentially for every tile there is function createTile that returns a canvas with some drawing on it. and leaflet shows the tile in correct position.
initialize: function(raster_data){
this.raster_data = raster_data;
},
createTile: function (tile_coords) {
let _tile = document.createElement('canvas');
let _tile_ctx = _tile.getContext('2d');
// do some drawing here with values from this.raster_data
return _tile;
}
This implementation is so far working fine. Than I thought of offloading drawing with offscreen-canvas in a webworker. so I restructured the code like this
initialize: function(raster_data){
this.raster_data = raster_data;
this.tile_worker = new Worker('tile_renderer.js')
},
createTile: function (tile_coords) {
let _tile = document.createElement('canvas').transferControlToOffscreen();
this.tile_worker.postMessage({
_tile: _tile,
_raster_data: this.raster_data
},[_tile])
return _tile;
}
This works but every now and then i see a canvas that is just blank. That thing is quite random I don't know start from where and how should I debug this. can this be a problem that I am using a single worker for rendering every tile? any help is appreciated. Here is an example of a blank canvas.

This a known bug: https://crbug.com/1202481
The issue appears when too many OffscreenCanvases are sent to the Worker serially.
The workaround is then to batch send all these OffscreenCanvases in a single call to postMessage().
In your code you could achieve this by storing all the objects to be sent and use a simple debouncing strategy using a 0 timeout to send them all at once:
createTile: function (tile_coords) {
let _tile = document.createElement('canvas');
_tile.setAttribute('width', 512);
_tile.setAttribute('height', 512);
let _off_tile = _tile.transferControlToOffscreen();
this.tiles_to_add.push( _off_tile ); // store for later
clearTimeout( this.batch_timeout_id ); // so that the callback is called only once
this.batch_timeout_id = setTimeout( () => {
// send them all at once
this.tileWorker.postMessage( { tiles: this.tiles_to_add }, this.tiles_to_add );
this.tiles_to_add.length = 0;
});
return _tile;
}
Live example: https://artistic-quill-tote.glitch.me/

How to export an audio file byte frequency data to a JSON object?

I'm using ReactJS to develop a web page (.html + .js) that will be bundled in a USB drive and shipped to customers. This USB drive contains some audio (.wav) files that are played through an HTML5 audio element in the web page. Customers will open the HTML file through their browser and listen to the songs available inside the USB drive.
I used the recent Web Audio API (specifically the analyser node) to analyze the frequency data of the current playing audio and then draw a sort of visual audio spectrum on an HTML5 canvas element.
Sadly, I was using a NodeJS local webserver during the development. Now, I prepared everything for production, just to discover that due to CORS-related restrictions my JS code can't access the audio file through the Web Audio API.
(This is because the URL protocol would be "file://", and there is no CORS policy defined for this protocol – This is the behaviour on Chrome and Firefox, using Safari it just works.)
The visual audio spectrum is an essential part of the design of this web page, and I'd hate to throw it away just because of the CORS policy.
My idea is to embed inside the JS code a JSON representation of the frequency data for the audio file, and then to use the JSON object in sync with the playing audio file to draw a fake (not in real-time) spectrum.
I tried – modifying the original code I was using to draw the spectrum – to use the JS requestAnimationFrame loop to get the frequency data for each frame and save it to a JSON file, but the JSON data seems to be incomplete and some frames (a lot) are missing.
this.audioContext = new AudioContext();
// this.props.audio is a reference to the HTML5 audio element
let src = this.audioContext.createMediaElementSource(this.props.audio);
this.analyser = this.audioContext.createAnalyser();
src.connect(this.analyser);
this.analyser.connect(this.audioContext.destination);
this.analyser.smoothingTimeConstant = 0.95;
this.analyser.fftSize = 64;
this.bufferLength = this.analyser.frequencyBinCount;
this.frequencyData = new Uint8Array(this.bufferLength);
[...]
const drawSpectrum = () => {
if (this.analyser) {
this.analyser.getByteFrequencyData(this.frequencyData);
/*
* storing this.frequencyData in a JSON file here,
* this works but I get sometimes 26 frames per seconds,
* sometimes 2 frames per seconds, never 60.
*/
}
requestAnimationFrame(drawSpectrum);
};
drawSpectrum();
Do you have a better idea to fake the visual audio spectrum? How would you go to "circumvent" the CORS-related restrictions in this case?
What could be a solid method to export audio frequency data to JSON (and then access it)?

This is one of the only cases where a data:// URL will come handy.
You can bundle your media file directly in your js or html file, as a base64 string and load it from there:
// a simple camera shutter sound
const audio_data = 'data:audio/mpeg;base64,SUQzAwAAAAAfdlRJVDIAAAABAAAAVFBFMQAAABsAAABTb3VuZEpheS5jb20gU291bmQgRWZmZWN0c1RBTEIAAAABAAAAVFlFUgAAAAEAAABUQ09OAAAAAQAAAFRSQ0sAAAABAAAAQ09NTQAAAB8AAABlbmcAb25saW5lLWF1ZGlvLWNvbnZlcnRlci5jb20AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/7UMAAAAAAAAAAAAAAAAAAAAAAAEluZm8AAAAPAAAAGQAAFTgAExMTHR0dHScnJycxMTExOzs7O0REREROTk5OWFhYWGJiYmJsbGxsdnZ2dn9/f3+JiYmJk5OTk52dnZ2np6ensbGxsbu7u7vExMTEzs7OztjY2Nji4uLi7Ozs7Pb29vb/////AAAAAExhdmM1OC41NAAAAAAAAAAAAAAAACQCTAAAAAAAABU4n9z9QQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+1DEAAAGCBL/IYxCAU+HoZSWDBEABebsn6sAAxbiwgggHy5Cypw3CzikmzW+7x3t1O5z26fpqR5/X0uoJkCX/ptAn+UBAACwhBmaqlL0FMK512J0Su9FoXRfEDkmpGmT8OVLCJJ3JPz6qlPfuR3+Zn//yTjv9x8tMhkdzhHf9wf4j5M+v1+v/NtO2ktwP7Y/L9n/9fX7c6OqAAAa/lAbDBZhjpdDVHEtOwypFSU87nTbLo6BNsyK1cUlG2Fg+Tkc2iY+sWmDptdFm2YhS4K1//tSxBsCDpzvFSekycmek+NYl5gBk1mYQpDzl9e/ZgrLGf27vk2+2UlOL7eMtmf37Z07566kMDx76S/wBCk3/A++ra9h0kks0mZ+xCnyMADI92/62qUAEBKJucwJLAaiQDMZc6Eg6iz477ruLJnEDO2QYRkU/LXkGnEFqi3MNtjn7rVUP21mWzYXpgGUDZiuRg/Zx0LEvyR+U53lTjn2d1dR7pBx9aqH5bagP/yJaZXT/RUuE2ddT7/eARoICFtQAAgGgCgIARJq08gQ7XJ7b9v/+1LECoAMAGsfNMMAAZIxqyMUoAF1OgdOIyk0ddyJoWUikvOuc8O0HZFKDJ4TvGLGjROKAS0BuaFSZ4DMUfccQXScTUMnBgiWTXcoNqmnMnBULvT22sLr2reLpyGl9ZMooooEpgAHrfdxEBdbE4D4DmLZAJQH+zHiJAVEr5xjj8MAzEB9XowjiKLA3BS/v7ERMOC2IIVf559jLiQhERk4mE3/fbfkRU1SBDiv/232V3mqjHK41Lj43////889i500hTcRWt75qnUiIEAOWVCCMP/7UsQGAAs0eV38wwABkjFtvJGKKvzojAkDYShCHBWnFxiaoek2gYcdinx7pjv7vcJvJdaVmf7afJrz/uTNNkEIBAg/Mnwq0XOuYjswwTDxd6VzzkvTQ1+sN8Tl3zFft8ocSzy7IrGYohFSSeIFvFI+oDrRuWZER2gR6m+FiT/rfhAERIYnMvuhO55rhUajF1qqzi6a/U/Y6zm8ok4uCIAMEIjHVWRhJzO7HdN9dbHNOlHnod31m7VRNOcdkf9ERk3+QGaShMlVm9y2qEQ0ABJY//tSxASAixS5b+eE1Qlaku45rAzgOfgsEFXFgOdeLACkgLkdwS8SI4DfVdsySxol94j0/jc464NyKIQj+r+XI2S+uiIcnXQCOykA9uMEsHYaHPV/0bv1jaYs68MiRhkFqv/iz91XZDu4iK1wAtFZ5dg0c8R6jjCgMKMQksoHwqwtIhq0ysPLN29V4QcZ06BsI3nn5UtHiPw+xM/Lii+tTBuPrWLBEGQbAFDZpNnxzmfr+QHWo9hMz6f8uhW/ImEEAgAEiFgkTAbDWTD4OgbKT2z/+1LECoANFJlnpLDBSXkXq36SYAC1ocQYUkwU9SfDpVheXMv4EklH8ud8HgyCUX3+Nl5tVNdYw+DSkkVbro8EKy8SL02ldwXVu/9I/jRb+/X/xlXrLNV//67T50cj6y3sif5/y6ymVJREqKOhmIAAAKhaxgAQCEIGATA0ZITIrAREZNyNMDnA1kUH8He0P2XaNF1RBO+uUVCh6P/mt/2ZbTcv15w7X/7V/Gsl6rlHlnXSSx7NbpGx/trfFh4bahikpQZ+ipPW2uctdc0di1VZkP/7UsQFAAuwu4W4kwARdRtttx6AAMZDIhmiJE2xSCpDbxgSdstgreZu8EHPhGCGGgOKwSvIA9c9zKMCtSDyi8YT3s9fKKbN9R8fUnz57/UZbWK6B1B4P9SVGD6L2/vRpMrRT/ocvLocu22WyttOJMpgEBIJuwDX+vsaTk/wyxTyRyHYZ5FBaVMyKiGa7LXkkC5m0cKdiwtLY2p+ahfiA5pP4/+vEiQVUqzN/xzz6iorCpCk/+wuGho1jRaqkYiQMA8QBnEogv7yun2/mUpG4CiC//tSxAWAC2jFZ7iUAAF3oW1XFoAAUgAM6NYUZASqI2m0oE+yNHlUWIIaFTbFg8FDhGBZEbUH/CDGHqVay/XqI6Dl+7ji37vpz3f+eu08yos/k8cv+cg1DQZUy74HioHvZ//8ZOLn++YAAtH/YAScOpSJiMeKmSIkB0IzKLA2JqTamnDwJ4gPGFZiZDsOJGn1JDmlK/YoONFhM1wxUf/WLNDB8NThr5X//eHkYXaT/w1bf//0SUe0FULOnf8PrNCUBJbLVYq5VVZCIAAFJ20YgD3/+1LEBoAMMHVf/MMAAW8drL6YUAAGR2WREJfnpTNzJQYsnR0jd7VT3uPnzJKOaqkDk4o5wINpBV5p+hhjRRxIXLhYyLNQXQtIYwHbGE1FC8fahhcjsvgYcCY+9iqEhdiKljKFdC3xAGKarJd1VFNEAp10VhURB7Fatwej4rKTZDiWMmbZxd2FA6PLIHCjezoszKszbavRA4LO608ykURM5eiG0CYsPe1nL0/f2mYXKWIljwKHpUuq4eDwSSHTv/Sj6fybxVX//v+fzeOCMsCQQP/7UsQFgAuVg3u4koABd5etu5hgAANAnzhJFbDBQNtKIplGCYwsAwOAIDNa5A+PPO6SDRdSqJVVIudR0eQeN6LfufVgEAYTs98TIi3pWdE9/P/memrG/+c//VH67F//+//6FQe/I3bVSrKyGaAKkVQGjSIokAXHUWgjccVAjGYkiSuMDYAV+SxgVvpKmqyt36ZkXH80jlz+dbNM+Jh87HLcvPP3Nf/7P7zr6i54M9wiKPOpQ+21T0Ffyrv5IDI/3eLB1ralmZM4MzEAAABAFhQK//tSxAYACqEPYeSssUFJD+18wJrIUYbFcSw8ubqa6KCGTJlnKubutjOLuZbVJHHTu3Np8+/tE3BhCo46IWdDsymfdsruoq5HKX+0/1bR5UBgVDJK5LmpGIVUXNvjK5v3VkZUNpEEtyWg9YHsRla84bYYiUPBOho4Su2due5VzWqtRTSn00IbiCdNRpU8wHw3FozAK8qISxUIhO7/xUwMJLtVb6jzlORrmt2h931lbcWVzbdGRDIokAkrMJ4m6EsrpUJ5tjQU9Kwp2ZDldgIQUw//+1LEEAAKPPNn54zvAUcSrHzzCXjCNoDmJKPt0OUdmejrp5gFHb6KyTHmfa1mSQOn/sn/7NTHnI/+0LCUVIgAa59Dvm6/1zuOzMaIJCALDv0Ji0Np+ObAola8ZSOBaWVaJFHyxn/petTnrtnTx8o16NOEkAtClVnFN9bVoLZXWdEs976je69Z3//LVdtLkKbK0nkhvRAQ4KqMmJZoVCAALcntkmPZh7R06XTHSZGTihG+KInZxTWjX9qrAxJUqu37MarotzE2TSUqNK35jtlosv/7UsQcAAng9VvkjFLBSZcp/MKOmP+3p/2VjBnCf/7f5JdTz3dFU3lg6nOrSaiJdzQiAFJaJ32V9grDgaRQUj0LFx4SizezTa1F+t3ylVtxFELLsiqsxnLVpVKVwFZPJjux0qWcjWscD1BwDLLQaY+1cFSgINA2hyF3VnEJ/TVXmWh1VkSQklw9ezunJesI2pDdxONRPURvMwNqHyCKJon2MGd6y4pfeR1U35NAhlf3M/zEUrkq72Rpy36ou66bm6NZxIBxwWye/xj0Cr+ajBSI//tSxCkACgD1TeYMsYFAHqh8kYowdmQyAEBJKjLuKAKjhMLhpMQhsnUQF24YoVwZ7t3lUpAWDUt2XsNWceCd8Lcv+HIQNbdE0bFOr1skqb7UXX/YiwZUrkfnZs657VS5lbVqW3+shIIBTkH1Hgu7AWw5eILY4jomrWyOCza6CGBqRAy4MzEGe2ldigpYZ2g2CAwopXXu0u07fo+xos7Hf1YhiplM13vIkMC5sCqL03/ev06gjGi2ADmp34tqkE8PgpMeMtIRfMTWp+2c7T2TLXf/+1LENwAKEPE/pgxRAUMe5izBi4jd+Nc0+5MzR/mmn8NmMrbWWd1DfJ5aaxzMx72FSxRHxHDPaaGj6Zpt3or50b0iXCi3WVqVEm2jrSACcoHeX1y6qIOTR5SWBzNU6nj7mThCuuagPYetZqHGYsKtYiqttzCMdA0RUOl/pkIpbH9GU+YWtq6oitRJWSc/RzsUCgw64jYX+9+V9Ef5AAGQTJkE6flYnLKNYfCShkETVEZ2sXKKBi3Rh8cx20VtlWqvB0yhanTnCcULDwLzRDTBUf/7UsREAAo4/zemDLSJThblZMMh4NaTK2sNx0zdsdbg6GDYOCwWQ0EmItLPZvTqmf/61QQi1GQHzJ+yLuaH45PeHSw5PnofJTFaIdGXYoDdgK+yFbUHbnLVRW5aE8DxsgF5Q4e/K+1X1JRlLj1hkYNN7zSNo0ctZQlWm/Ndf////6wFalJgRguDZKRzIWw4kqvMhQExouywQNTSZVEWsyk7HqRdplHRStqjSKJFKEjVpgcmqPOY8MCJRgPrQOLlEmNdBiZcbC4jYzSv+3Z///+l//tSxE8ACdiVKWYZD0E8D+SgkyKQAVqgAZABSx4iaVFSEcXTkNyCwRRvomCyTI0JuM8vM/CCy9bY8EpqJgqJbB9Bi+D0awBB4gXoKg8FGg0KsGD5il3HRtWtn/XtX/oFV6vp7Q7JokZkusvWqBBdt0uLcoLBMEC0ARDNwdNIirUBOFektvZ4a8AywFBEoAxQRHzc4WGgcQOAo1JSUWPGka+dT//tqW1m2pUBCVVhkBOTRJD2jakvQsuwsDz6GxQAyqEHqw2FbM6ua7n7gBk1Cjn/+1LEXgAJSH0lIyTBQSgPJSSRlbA56gZ2eErlB02OQ1TjI8RAqEwSizBjBRK3Lbt+t3Ry1z8s+dTuv1AADVi8AIZ2fC1y+MJ96YzSpyh4EhCR7AWrH4mjZFtu95bSEcqVZ3FDgW1sPCnRmqzsWuwsa9DBxS5d5retejXV1d1iK9rkW9fVySoAAJOWRCr6cPT5zb1h8Kd2bqwBivjn7G1JSavR5KBDORZaSqp3hgxJCymhN8rAggsYSyJ4il1u/ku8O20uqVKf/VVyzv4ox/3BgP/7UsRxgAm0eSMmIGiBKhWkJMMI+AhpACYGA2ioQIGEiV6GwjlosMjhgTKBi8zMtXQE8zCXSVOPNqJFOLAc2UNngchBQ6qi+lN3KAjNb6y39oq5tDGU1f56pDUX1SraqgAASqpULMhkhqZjWa4x27swqGQqMqi4sJKdL90XbIj5aCRdbmpUi6fg1g/uWxy2vd0yBTay2mjLWLHsf/2M1/zrn2KRtupQMGiE4bQnJPTKbqDtehQyx8SpNJ+Sa8TUhzRf7Trv29pdsuWeUih0BrCY//tSxIMCCTCTHyYYZ8ErD6OkkY0ocFmOrSjct2hAy5W92ef2f7f9ikZn9nTVAAAZRpEgF4J2ZEgqZTfJO3Fal0aKIYWKQMRd8iiiQKgECY65p0hLEECbhI4UNQsKkXGD2Bcj0Vs3sq8t+7+3tq793WxnswAxBIcismMkzT2ZAtRLUh9hqDVfPOOz6LQEtLhVLXmwKJQmpKBjwKWrMFTDdqi+kYxE0xuNIkyRVDbmbKK8t62/hL7WHaz0NAQ9alkaTbP0Ic0iFULmU4jO2hW6+0T/+1LEloMImJUfJJhjgQ2Ro4iRmThMoocwHDDDIdPGy4jTKWb/R/Sz+t05bXNd+/71qH+xl3/Ycp6jhQICk8Hl5iS7VoSRIK6YgjIDBByJy4gewGCMxZspTn/mjcnVcx5yHUL5UZctofG+J6NG3/v2DtdIbsN5q6M7X+K+qCKd7tKeXVtZuQ/Rk/7oepkc8E/Vui+1apXyYx8bLPugNStLXmGPW0gdqNZ7Pk1L+qsZyOQ3RmdFfUiVhRGZQodh2wsp3Uv05OJln+eq7HmpHznp0v/7UsSwAgi4dR9EiGoBIIwixMKM0O0p92N5f2tbvaqmZLPPKt1y4IO1Tp9oVD8CDwHfUoNZUVaQRZcqjPXtn7/dd2fG7t4TScAjutljTFhJ5zTMiLhmilieML3o6cU8QjOfNL1D2yu1yIH2X4WPm9VjxZQ3PxSDdaNwEMKmureTdVRjuJvJB1QGBDaPwOnp6tp7R9Py895JLMgHAYkj9ajluacuSKOTjQC+dIVSdRMX6THFPZZDKt5Zr+eaGpbamsOa94BNqFIZ/nMppgoaEipa//tSxMcABvxVI0MERYFyHSGA9IxZDqVSYK0Gcl61FdCf4UWf077Rj5eumYWy72FylBdRYR3B1Ik6UTiTUclOcsAnhM4ckWjRSRqKw/ljKq81Kk214aqTGvtSY1WiYc1jF+uzqFJv1+7VYx36uxr9XP2Wl/w1/lWch/DXU1L2NVpfqTN/tVL/hqTCgrJ/skjmAqKSKoiqTEFNRTMuMTAwqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqr/+1LE2oALZYMQQzBgSUqh4lhgjn2qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqv/7UsThg4zQ6PYGGHDJlK9bxPMMuaqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq';
const button = document.getElementById( 'btn' );
const audio_ctx = new AudioContext();
// if you wish to use a MediaElementSource node:
function initMediaElementNode() {
const audio_el = new Audio();
audio_el.src = audio_data;
document.body.append( audio_el );
audio_el.controls = true;
const node = audio_ctx.createMediaElementSource( audio_el );
node.connect( audio_ctx.destination );
// to prove the data passes through the AudioContext
const analyser = audio_ctx.createAnalyser();
analyser.fftSize = 32;
node.connect( analyser );
const arr = new Uint8Array( 32 );
audio_el.onplay = (evt) => {
setTimeout( ()=> {
analyser.getByteFrequencyData( arr );
console.log( 'analyser data', [...arr] );
}, 150 );
};
}
// if you wish to use an AudioBuffer:
async function initAudioBuffer() {
const data_buf = dataURLToArrayBuffer( audio_data );
const audio_buf = await audio_ctx.decodeAudioData( data_buf );
button.onclick = (evt) => {
const source = audio_ctx.createBufferSource();
source.buffer = audio_buf;
source.connect( audio_ctx.destination );
source.start( 0 );
};
button.textContent = "play audio buffer";
}
button.onclick = (evt) => {
initMediaElementNode();
initAudioBuffer();
};
function dataURLToArrayBuffer( data_url ) {
const byte_string = atob( data_url.split( ',' )[ 1 ] );
return Uint8Array.from(
{ length: byte_string.length },
(_, i) => byte_string.charCodeAt(i)
).buffer;
}
button { vertical-align: top; }
<button id="btn">click to start</button>

Playing Audio Base64s With No Delay

I have an Object with couple of base64s (Audio) inside. The base64s will start to play with a keydown event. In some situations (when the Base64 size is a little high), a delay will occur before playing. Is there any way to remove this delay, or at least reduce it?
App Witten in JavaScript And Running On Electron
//audio base64s object
var audio = {A: new Audio('base64[1]'), B: new Audio('base64[2]'), C: new Audio('base64[3]')};
//audio will start plying with key down
function keydown(ev) {
if (audio[String.fromCharCode(ev.keyCode)].classList.contains('holding') == false) {
audio[String.fromCharCode(ev.keyCode)].classList.add('holding');
if (audio[String.fromCharCode(ev.keyCode)].paused) {
playPromise = audio[String.fromCharCode(ev.keyCode)].play();
if (playPromise) {
playPromise.then(function() {
setTimeout(function() {
// Follow up operation
}, audio.duration * 1000);
}).catch(function() {
// Audio loading failure
});
} else {
audio[String.fromCharCode(ev.keyCode)].currentTime = 0;
}
}
}

I wrote up a complete example for you, and annotated below.
Some key takeaways:
If you need any sort of expediency or control over timing, you need to use the Web Audio API. Without it, you have no control over the buffering or other behavior of audio playback.
Don't use base64 for this. You don't need it. Base64 encoding is a method for encoding binary data into a text format. There is no text format here... therefore it isn't necessary. When you use base64 encoding, you add 33% overhead to the storage, you use CPU, memory, etc. There is no reason for it here.
Do use the appropriate file APIs to get what you need. To decode an audio sample, we need an array buffer. Therefore, we can use the .arrayBuffer() method on the file itself to get that. This retains the content in binary the entire time and allows the browser to memory-map if it wants to.
The code:
const audioContext = new AudioContext();
let buffer;
document.addEventListener('DOMContentLoaded', (e) => {
document.querySelector('input[type="file"]').addEventListener('change', async (e) => {
// Start the AudioContext, now that we have user ineraction
audioContext.resume();
// Ensure we actually have at least one file before continuing
if ( !(e.currentTarget.files && e.currentTarget.files[0]) ) {
return;
}
// Read the file and decode the audio
buffer = await audioContext.decodeAudioData(
await e.currentTarget.files[0].arrayBuffer()
);
});
});
document.addEventListener('keydown', (e) => {
// Ensure we've loaded audio
if (!buffer) {
return;
}
// Create the node that will play our previously decoded buffer
bufferSourceNode = audioContext.createBufferSource();
bufferSourceNode.buffer = buffer;
// Hook up the buffer source to our output node (speakers, headphones, etc.)
bufferSourceNode.connect(audioContext.destination);
// Adjust pitch based on the key we pressed, just for fun
bufferSourceNode.detune.value = (e.keyCode - 65) * 100;
// Start playing... right now
bufferSourceNode.start();
});
JSFiddle: https://jsfiddle.net/bradisbell/sc9jpxvn/1/

Exporting a video in p5.js

I am creating a simple animation program in p5.js. When a user clicks the save button, I want to download a video of the animation.
I have an object called frames where each key is labelled frame_1, frame_2 and so on. The value associated with each key is an array of line segments that makes up that frame.
I am trying to think of an approach to take this data and create an mp4 video. p5.js has a built in save function that I thought might be helpful but it is not a full solution on its own. I could save each frame as an individual image and then somehow stitch those images together on the client side but I have yet to find a solution to this.
Any other approaches would be great as well. The only requirement is that it is done client side.

Since p5.js is built on the Canvas API, in modern browsers, you can use a MediaRecorder to do this job.
const btn = document.querySelector('button'),
chunks = [];
function record() {
chunks.length = 0;
let stream = document.querySelector('canvas').captureStream(30),
recorder = new MediaRecorder(stream);
recorder.ondataavailable = e => {
if (e.data.size) {
chunks.push(e.data);
}
};
recorder.onstop = exportVideo;
btn.onclick = e => {
recorder.stop();
btn.textContent = 'start recording';
btn.onclick = record;
};
recorder.start();
btn.textContent = 'stop recording';
}
function exportVideo(e) {
var blob = new Blob(chunks);
var vid = document.createElement('video');
vid.id = 'recorded'
vid.controls = true;
vid.src = URL.createObjectURL(blob);
document.body.appendChild(vid);
vid.play();
}
btn.onclick = record;
// taken from pr.js docs
var x, y;
function setup() {
createCanvas(300, 200);
// Starts in the middle
x = width / 2;
y = height;
}
function draw() {
background(200);
// Draw a circle
stroke(50);
fill(100);
ellipse(x, y, 24, 24);
// Jiggling randomly on the horizontal axis
x = x + random(-1, 1);
// Moving up at a constant speed
y = y - 1;
// Reset to the bottom
if (y < 0) {
y = height;
}
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.7/p5.min.js"></script>
<button>start recording</button><br>

ccapture works well with p5.js to achieve the goal of recording what's displaying on a canvas.
Here is a demo of ccapture working with p5.js. The source code comes with the demo.
This method won't output laggy videos because it is not recording what you see on the screen, which can be laggy. Instead, it writes every frame into the video and tells the videos to play at a fixed frame rate. So even if it takes seconds to calculate just one frame, the output video will play smoothly without showing any delay between frames.
However, there is one caveat though. This method only works with Chrome.

As you specified in the comments that a gif would also work, here is a solution:
Below is a sample p5 sketch that records canvas animation and turns it into a gif, using gif.js.
Works in browsers supporting: Web Workers, File API and Typed Arrays.
I've provided this code so you can get an idea of how to use this library because not much documentation is provided for it and I had a hard time myself figuring it out.
var cnv;
var gif, recording = false;
function setup() {
cnv = createCanvas(400, 400);
var start_rec = createButton("Start Recording");
start_rec.mousePressed(saveVid);
var stop_rec = createButton("Stop Recording");
stop_rec.mousePressed(saveVid);
start_rec.position(500, 500);
stop_rec.position(650, 500);
setupGIF();
}
function saveVid() {
recording = !recording;
if (!recording) {
gif.render();
}
}
var x = 0;
var y = 0;
function draw() {
background(51);
fill(255);
ellipse(x, y, 20, 20);
x++;
y++;
if (recording) {
gif.addFrame(cnv.elt, {
delay: 1,
copy: true
});
}
}
function setupGIF() {
gif = new GIF({
workers: 5,
quality: 20
});
gif.on('finished', function(blob) {
window.open(URL.createObjectURL(blob));
});
}
More Info :
This sketch starts recording frames when you click start_rec and stops when you hit stop_rec, in your sketch you might want to control things differently, but keep in mind that addFrame only adds one frame to the gif so you need to call it in the draw function to add multiple frames, you can pass in an ImageElement, a CanvasElement or a CanvasContext along with other optional parameters.
In the gif.on function, you can specify a callback function to do whatever you like with the gif.
If you want to fine tune settings of the gif, like quality, repeat, background, you can read more here. Hope this helps!

Resize & Compress Image in Web Worker. Can I use a canvas?

Backgroud
I'm now processing on the client select image.
I want to do two actions on that image, and outputs the base64-encoded string.
If the image size has a width or height larger than 1000, resize it.
Compress the image with jpeg of quality 0.5.
So now I will do the below in the main script:
$(function() {
$('#upload').change(function() {
var URL = window.URL || window.webkitURL;
var imgURL = URL.createObjectURL(this.files[0]);
var img = new Image();
img.onload = function() {
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d');
var w0 = img.width;
var h0 = img.height;
var w1 = Math.min(w0, 1000);
var h1 = h0 / w0 * w1;
canvas.width = w1;
canvas.height = h1;
ctx.drawImage(img, 0, 0, w0, h0,
0, 0, canvas.width, canvas.height);
// Here, the result is ready,
var data_src = canvas.toDataURL('image/jpeg', 0.5);
$('#img').attr('src', data_src);
URL.revokeObjectURL(imgURL);
};
img.src = imgURL;
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<input id="upload" type="file" accept="image/*" />
<img id="img" />
The Problem
But still, my code will work on a mobile, where the above procedure(resize and compress) can not work out soon. It causes the GUI stop response for a moment.
I want the procedure work in another thread, using web worker. So it won't block the UI, so the user experience would be better.
Now comes the problem, it seemed the web worker cannot deal with a canvas, how can I work around this?

Some Event driven coding
Saddly Web workers are not yet ready with browser support.
Limited support for toDataURL in web workers means another solution is needed. See MDN web worker APIs (ImageData) mid way down the page, only for Firefox at the moment.
Looking at your onload you have all the heavy duty work done in one blocking call to onload. You are blocking the UI during the process of creating the new canvas, getting its context, scaling, and toDataURL (don't know what revokeObjectURL does). You need to let the UI get a few calls in while this is happening. So a little event driven processing will help reduce the glitch if not make it unnoticeable.
Try rewriting the onload function as follows.
// have added some debugging code that would be useful to know if
// this does not solve the problem. Uncomment it and use it to see where
// the big delay is.
img.onload = function () {
var canvas, ctx, w, h, dataSrc, delay; // hosit vars just for readability as the following functions will close over them
// Just for the uninitiated in closure.
// var now, CPUProfile = []; // debug code
delay = 10; // 0 could work just as well and save you 20-30ms
function revokeObject() { // not sure what this does but playing it safe
// as an event.
// now = performance.now(); // debug code
URL.revokeObjectURL(imgURL);
//CPUProfile.push(performance.now()-now); // debug code
// setTimeout( function () { CPUProfile.forEach ( time => console.log(time)), 0);
}
function decodeImage() {
// now = performance.now(); // debug code
$('#img').attr('src', dataSrc);
setTimeout(revokeObject, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
function encodeImage() {
// now = performance.now(); // debug code
dataSrc = canvas.toDataURL('image/jpeg', 0.5);
setTimeout(decodeImage, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
function scaleImage() {
// now = performance.now(); // debug code
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
setTimeout(encodeImage, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
// now = performance.now(); // debug code
canvas = document.createElement('canvas');
ctx = canvas.getContext('2d');
w = Math.min(img.width, 1000);
h = img.height / img.width * w;
canvas.width = w;
canvas.height = h;
setTimeout(scaleImage, delay); // gives the UI a chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
};
setTimeout allows the current call to exit, freeing up the call stack and allowing the UI to get its mitts on the DOM. I have given 10ms, personally I would start with 0ms as call stack access is not blocked, but I am playing it safe
With luck the problem will be greatly reduced. If it still remains an unacceptable delay then I can have a look at the CPU profile and see if a solution can not be found by targeting the bottle neck. My guess is the toDataURL is where the load is. If it is, a possible solution is to find a JPG encoder written in javascript that can be converted to an event driven encoder.
The problem is not how long it takes to process the data, but how long you block the UI.