Develop Reference

Exporting a video in p5.js

I am creating a simple animation program in p5.js. When a user clicks the save button, I want to download a video of the animation.
I have an object called frames where each key is labelled frame_1, frame_2 and so on. The value associated with each key is an array of line segments that makes up that frame.
I am trying to think of an approach to take this data and create an mp4 video. p5.js has a built in save function that I thought might be helpful but it is not a full solution on its own. I could save each frame as an individual image and then somehow stitch those images together on the client side but I have yet to find a solution to this.
Any other approaches would be great as well. The only requirement is that it is done client side.

Since p5.js is built on the Canvas API, in modern browsers, you can use a MediaRecorder to do this job.
const btn = document.querySelector('button'),
chunks = [];
function record() {
chunks.length = 0;
let stream = document.querySelector('canvas').captureStream(30),
recorder = new MediaRecorder(stream);
recorder.ondataavailable = e => {
if (e.data.size) {
chunks.push(e.data);
}
};
recorder.onstop = exportVideo;
btn.onclick = e => {
recorder.stop();
btn.textContent = 'start recording';
btn.onclick = record;
};
recorder.start();
btn.textContent = 'stop recording';
}
function exportVideo(e) {
var blob = new Blob(chunks);
var vid = document.createElement('video');
vid.id = 'recorded'
vid.controls = true;
vid.src = URL.createObjectURL(blob);
document.body.appendChild(vid);
vid.play();
}
btn.onclick = record;
// taken from pr.js docs
var x, y;
function setup() {
createCanvas(300, 200);
// Starts in the middle
x = width / 2;
y = height;
}
function draw() {
background(200);
// Draw a circle
stroke(50);
fill(100);
ellipse(x, y, 24, 24);
// Jiggling randomly on the horizontal axis
x = x + random(-1, 1);
// Moving up at a constant speed
y = y - 1;
// Reset to the bottom
if (y < 0) {
y = height;
}
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.7/p5.min.js"></script>
<button>start recording</button><br>

ccapture works well with p5.js to achieve the goal of recording what's displaying on a canvas.
Here is a demo of ccapture working with p5.js. The source code comes with the demo.
This method won't output laggy videos because it is not recording what you see on the screen, which can be laggy. Instead, it writes every frame into the video and tells the videos to play at a fixed frame rate. So even if it takes seconds to calculate just one frame, the output video will play smoothly without showing any delay between frames.
However, there is one caveat though. This method only works with Chrome.

As you specified in the comments that a gif would also work, here is a solution:
Below is a sample p5 sketch that records canvas animation and turns it into a gif, using gif.js.
Works in browsers supporting: Web Workers, File API and Typed Arrays.
I've provided this code so you can get an idea of how to use this library because not much documentation is provided for it and I had a hard time myself figuring it out.
var cnv;
var gif, recording = false;
function setup() {
cnv = createCanvas(400, 400);
var start_rec = createButton("Start Recording");
start_rec.mousePressed(saveVid);
var stop_rec = createButton("Stop Recording");
stop_rec.mousePressed(saveVid);
start_rec.position(500, 500);
stop_rec.position(650, 500);
setupGIF();
}
function saveVid() {
recording = !recording;
if (!recording) {
gif.render();
}
}
var x = 0;
var y = 0;
function draw() {
background(51);
fill(255);
ellipse(x, y, 20, 20);
x++;
y++;
if (recording) {
gif.addFrame(cnv.elt, {
delay: 1,
copy: true
});
}
}
function setupGIF() {
gif = new GIF({
workers: 5,
quality: 20
});
gif.on('finished', function(blob) {
window.open(URL.createObjectURL(blob));
});
}
More Info :
This sketch starts recording frames when you click start_rec and stops when you hit stop_rec, in your sketch you might want to control things differently, but keep in mind that addFrame only adds one frame to the gif so you need to call it in the draw function to add multiple frames, you can pass in an ImageElement, a CanvasElement or a CanvasContext along with other optional parameters.
In the gif.on function, you can specify a callback function to do whatever you like with the gif.
If you want to fine tune settings of the gif, like quality, repeat, background, you can read more here. Hope this helps!

ExtJS how to set icon of button to image in memory (not on disk)?

Long story short in another portion of the program I make canvases, convert them to DataURLs, then pass them over to the following portion to use as the icon image of the buttons. Whenever I set this.icon = "/path/to/image.jpg", it pulls it correctly, but since these images are not on disk, I am unsure how to
arrowHandler: function (arrow) {
var list = [];
var library = Ext.getCmp("library");
var buttons = Ext.getCmp("numbered").menu.buttons; //where the dataURLs are pushed in another portion of the program
function btn(num) {
var image = new Image;
image.src = buttons[num].dataURL;
this.xtype = "button";
this.height = 50;
this.width = 50;
this.icon = image; //where putting an actual path works correctly, but this code doesn't
this.num = num;
this.handler = function (btn) {
btn.up("button").menu.Style = this.num;
btn.up("button").fireEvent("selected", this.num);
};
}
for (var i = 0; i <= 0; i++)
library.items.items.push(new btn(i));
},
I am aware the loop is only going thru index 0 - it's like that purposefully for testing.
SOLUTION
The selected correct answer did provide the right way to set the icon with a DataURI, but it wasn't the fix to my issue. Turns out instead of doing
library.items.items.push(new btn(i));
I needed to be doing
library.add(new btn(i));
The error I kept encountering with pushing was "c.render() is not a function". I mention that solely to make it hopefully searchable for anyone else who encounters that error.

Should be the same as data uri, you'll have to convert it first.
https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL
var dataURL = canvas.toDataURL();
Here is a button fiddle:
https://fiddle.sencha.com/#view/editor&fiddle/1og6

Resize & Compress Image in Web Worker. Can I use a canvas?

Backgroud
I'm now processing on the client select image.
I want to do two actions on that image, and outputs the base64-encoded string.
If the image size has a width or height larger than 1000, resize it.
Compress the image with jpeg of quality 0.5.
So now I will do the below in the main script:
$(function() {
$('#upload').change(function() {
var URL = window.URL || window.webkitURL;
var imgURL = URL.createObjectURL(this.files[0]);
var img = new Image();
img.onload = function() {
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d');
var w0 = img.width;
var h0 = img.height;
var w1 = Math.min(w0, 1000);
var h1 = h0 / w0 * w1;
canvas.width = w1;
canvas.height = h1;
ctx.drawImage(img, 0, 0, w0, h0,
0, 0, canvas.width, canvas.height);
// Here, the result is ready,
var data_src = canvas.toDataURL('image/jpeg', 0.5);
$('#img').attr('src', data_src);
URL.revokeObjectURL(imgURL);
};
img.src = imgURL;
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<input id="upload" type="file" accept="image/*" />
<img id="img" />
The Problem
But still, my code will work on a mobile, where the above procedure(resize and compress) can not work out soon. It causes the GUI stop response for a moment.
I want the procedure work in another thread, using web worker. So it won't block the UI, so the user experience would be better.
Now comes the problem, it seemed the web worker cannot deal with a canvas, how can I work around this?

Some Event driven coding
Saddly Web workers are not yet ready with browser support.
Limited support for toDataURL in web workers means another solution is needed. See MDN web worker APIs (ImageData) mid way down the page, only for Firefox at the moment.
Looking at your onload you have all the heavy duty work done in one blocking call to onload. You are blocking the UI during the process of creating the new canvas, getting its context, scaling, and toDataURL (don't know what revokeObjectURL does). You need to let the UI get a few calls in while this is happening. So a little event driven processing will help reduce the glitch if not make it unnoticeable.
Try rewriting the onload function as follows.
// have added some debugging code that would be useful to know if
// this does not solve the problem. Uncomment it and use it to see where
// the big delay is.
img.onload = function () {
var canvas, ctx, w, h, dataSrc, delay; // hosit vars just for readability as the following functions will close over them
// Just for the uninitiated in closure.
// var now, CPUProfile = []; // debug code
delay = 10; // 0 could work just as well and save you 20-30ms
function revokeObject() { // not sure what this does but playing it safe
// as an event.
// now = performance.now(); // debug code
URL.revokeObjectURL(imgURL);
//CPUProfile.push(performance.now()-now); // debug code
// setTimeout( function () { CPUProfile.forEach ( time => console.log(time)), 0);
}
function decodeImage() {
// now = performance.now(); // debug code
$('#img').attr('src', dataSrc);
setTimeout(revokeObject, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
function encodeImage() {
// now = performance.now(); // debug code
dataSrc = canvas.toDataURL('image/jpeg', 0.5);
setTimeout(decodeImage, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
function scaleImage() {
// now = performance.now(); // debug code
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
setTimeout(encodeImage, delay); // gives the UI a second chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
}
// now = performance.now(); // debug code
canvas = document.createElement('canvas');
ctx = canvas.getContext('2d');
w = Math.min(img.width, 1000);
h = img.height / img.width * w;
canvas.width = w;
canvas.height = h;
setTimeout(scaleImage, delay); // gives the UI a chance to get something done.
//CPUProfile.push(performance.now()-now); // debug code
};
setTimeout allows the current call to exit, freeing up the call stack and allowing the UI to get its mitts on the DOM. I have given 10ms, personally I would start with 0ms as call stack access is not blocked, but I am playing it safe
With luck the problem will be greatly reduced. If it still remains an unacceptable delay then I can have a look at the CPU profile and see if a solution can not be found by targeting the bottle neck. My guess is the toDataURL is where the load is. If it is, a possible solution is to find a JPG encoder written in javascript that can be converted to an event driven encoder.
The problem is not how long it takes to process the data, but how long you block the UI.

Optimizing easeljs line drawing for mobile

I'm trying to make a simple application where the user can draw lines by click-dragging the mouse or through gestures on their touch device screen.
It's perfectly fine on my desktop machine, but on my phone its very slow and jerky. It's not that performance degrades over time, it's immediately noticable.
I'm using easeljs and I extended shape. On mouse movements it records the points and on tick it draws them. The stage's autoClear is set to false and the graphics object clears before it draws so it doesn't redraw anything from the previous tick.
(function (window) {
function LineDrawer() {
this.initialize();
}
//Inheritance from Container
LineDrawer.prototype = new createjs.Shape();
LineDrawer.prototype.Shape_initialize = LineDrawer.prototype.initialize;
LineDrawer.prototype.Shape_tick = LineDrawer.prototype._tick;
LineDrawer.prototype.initialize = function () {
//call to initialize() method from parent class
this.Shape_initialize();
this.points = [];
this.mouseMoveEventListener = $.proxy(this.onMouseMove, this);
}
LineDrawer.prototype._tick = function (e) {
//call to _tick method from parent class
this.Shape_tick();
var points = this.points;
if (points.length > 0) {
var graphics = this.graphics.clear()
.setStrokeStyle(3, 'round', 'round')
.beginStroke("#000000")
.moveTo(points[0].x, points[0].y)
var pt;
for (var i = 1; i < points.length; i = i + 1) {
pt = points[i];
graphics.lineTo(pt.x, pt.y);
}
points.length = 0;
if (typeof pt !== 'undefined') {
points.push(new createjs.Point(pt.x, pt.y));
}
}
}
LineDrawer.prototype.onMouseDown = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
this.parent.addEventListener("stagemousemove", this.mouseMoveEventListener);
}
LineDrawer.prototype.onMouseMove = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
}
LineDrawer.prototype.onMouseUp = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
this.parent.removeEventListener("stagemousemove", this.mouseMoveEventListener);
}
window.LineDrawer = LineDrawer;
}(window));
Here's the code for setting up the stage:
var stage,
lineDrawer;
$(document).ready(function () {
lineDrawer = new LineDrawer();
var $canvas = $('#canvasMain');
stage = new createjs.Stage($canvas[0]);
createjs.Touch.enable(stage);
stage.addChild(lineDrawer);
stage.autoClear = false;
stage.addEventListener("stagemousedown", $.proxy(lineDrawer.onMouseDown, lineDrawer));
stage.addEventListener("stagemouseup", $.proxy(lineDrawer.onMouseUp, lineDrawer));
createjs.Ticker.timingMode = createjs.Ticker.RAF;
createjs.Ticker.addEventListener("tick", stage);
})
Here's a fiddle for everything. Other information: I'm using jquery-1.8.3 and a RAF polyfill, these are my phone specs. I also got somebody to try with a much better Samsung phone with the same results.
Although admittedly my phone is on the low end of the spectrum, its not a dinosaur of a phone. It is android 4.0+ and what I'm doing really isn't that complicated as far as I can tell. Am I doing anything wrong and/or is there anything I can do to improve this? I wonder if maybe its a problem with the touch events and not the drawing speed as well.
EDIT: the other phone with laggy drawing was a Samsung S3

Answering my own question.
The problem was a combination of me not using the graphic's endStroke method after all the lines causing the drawing to be one tick behind, and a logic error where I would only draw lines if there was one or more points, but really it was supposed to be 2 or more points.
The second part was causing the most lag. Interestingly there was no error since the array index is never out of bounds because of the for loop condition, so I guess just having an impossible for condition was really slow on the Chrome for android browsers, without crashing.

image loading loop IE7 Stack Overflow at line 0

Im trying to load in jpeg images, frame by frame to create an sequence animation of jpeg images. I'm attempting to load them in a recursive loop using javascript. I need to load images in linearly to achieve progressive playback of the animation. (start playback before all frames are loaded) I get a Stack overflow at line: 0 error from IE due to the natural recursion of the function. (My real code loads in over 60+ frames)
Here is a basic example of how I'm doing this:
var paths = ['image1.jpg', 'image2.jpg', 'image3.jpg']; //real code has 60+ frames
var images = [];
var load_index = 0;
var load = function(){
var img = new Image();
img.onload = function(){
if(load_index<=paths.length){
load_index++;
load();
}else{
alert('done loading');
}
}
img.src = paths[load_index];
images.push(img);
}
It seems I can avoid this error by using a setTimeout with an interval of 1 when calling the next step of the load. This seems to let IE "breathe" before loading the next image, but decreases the speed at which the images load dramatically.
Any one know how to avoid this stack overflow error?
http://cappuccino.org/discuss/2010/03/01/internet-explorer-global-variables-and-stack-overflows/
The above link suggests that wrapping the function to remove it from the window object will help avoid stack overflow errors. But I then see strangeness with it only getting about 15 frames through the sequence and just dies.

Put simply, don't use a recursive function for this situation, there isn't any need:
var paths = ['image1.jpg', 'image2.jpg', 'image3.jpg'];
var images = [];
var loads = [];
/// all complete function, probably should be renamed to something with a
/// unique namespace unless you are working within your own function scope.
var done = function(){
alert('all loaded');
}
var loaded = function(e,t){
/// fallbacks for old IE
e = e||Event; t = e.target||e.srcElement;
/// keep a list of the loaded images, you can delete this later if wanted
loads.push( t.src );
if ( loads.length >= paths.length ) {
done();
}
}
var load = function(){
var i, l = paths.length, img;
for( i=0; i<l; i++ ){
images.push(img = new Image());
img.onload = loaded;
img.src = paths[i];
}
}
In fact, as you are finding, the method you are using currently is quite intensive. Instead, the above version doesn't create a new function for each onload listener (saves memory) and will trigger off as many concurrent loads as your browser will allow (rather than waiting for each image load).
(the above has been manually typed and not tested, as of yet)
update
Ah, then it makes more sense as to why you are doing things this way :) In that case then your first approach using the setTimeout would probably be the best solution (you should be able to use a timeout of 0). There is still room for rearranging things to see if you can avoid that though. The following may get around the problem...
var paths = ['image1.jpg', 'image2.jpg', 'image3.jpg'];
var images = []; /// will contain the image objects
var loads = []; /// will contain loaded paths
var buffer = []; /// temporary buffer
var done = function(){ alert('all loaded'); }
var loaded = function(e,t){
e = e||Event; t = e.target||e.srcElement; loads.push( t.src );
/// you can do your "timing/start animation" calculation here...
/// check to see if we are complete
if ( loads.length >= paths.length ) { done(); }
/// if not fire off the next image load
else { next(); }
}
var next = function(){
/// current will be the next image
var current = buffer.shift();
/// set the load going for the current image
if ( current ) { current.img.src = current.path; }
}
var load = function(){
var i, l = paths.length, img;
for( i=0; i<l; i++ ){
img = new Image();
img.onload = loaded;
/// build up a list of images and paths to load
buffer.push({ img: img, path: paths[i] });
}
/// set everything going
next();
}
If the above doesn't do it, another way of getting around the issue would be to step through your list of paths, one at a time, and append a string of image markup (that would render off-screen) to the DOM with it's own onload="next()" handler... next() would be responsible for inserting the next image. By doing this it would hand off the triggering of the load and the subsequent load event to outside of your code, and should get around stacking calls.