Develop Reference

Offscreen-Canvas not rendering some of the time

I am trying to implement a in browser raster drawing plugin for the leaflet library that that extends the leaflets GridLayer api. Essentially for every tile there is function createTile that returns a canvas with some drawing on it. and leaflet shows the tile in correct position.
initialize: function(raster_data){
this.raster_data = raster_data;
},
createTile: function (tile_coords) {
let _tile = document.createElement('canvas');
let _tile_ctx = _tile.getContext('2d');
// do some drawing here with values from this.raster_data
return _tile;
}
This implementation is so far working fine. Than I thought of offloading drawing with offscreen-canvas in a webworker. so I restructured the code like this
initialize: function(raster_data){
this.raster_data = raster_data;
this.tile_worker = new Worker('tile_renderer.js')
},
createTile: function (tile_coords) {
let _tile = document.createElement('canvas').transferControlToOffscreen();
this.tile_worker.postMessage({
_tile: _tile,
_raster_data: this.raster_data
},[_tile])
return _tile;
}
This works but every now and then i see a canvas that is just blank. That thing is quite random I don't know start from where and how should I debug this. can this be a problem that I am using a single worker for rendering every tile? any help is appreciated. Here is an example of a blank canvas.

This a known bug: https://crbug.com/1202481
The issue appears when too many OffscreenCanvases are sent to the Worker serially.
The workaround is then to batch send all these OffscreenCanvases in a single call to postMessage().
In your code you could achieve this by storing all the objects to be sent and use a simple debouncing strategy using a 0 timeout to send them all at once:
createTile: function (tile_coords) {
let _tile = document.createElement('canvas');
_tile.setAttribute('width', 512);
_tile.setAttribute('height', 512);
let _off_tile = _tile.transferControlToOffscreen();
this.tiles_to_add.push( _off_tile ); // store for later
clearTimeout( this.batch_timeout_id ); // so that the callback is called only once
this.batch_timeout_id = setTimeout( () => {
// send them all at once
this.tileWorker.postMessage( { tiles: this.tiles_to_add }, this.tiles_to_add );
this.tiles_to_add.length = 0;
});
return _tile;
}
Live example: https://artistic-quill-tote.glitch.me/

Firebase Javascript + P5.js: Asynchronous function getting in the way of redrawing the canvas

I'm trying to create an application where circles are drawn onto the canvas through reading information from a Firebase database that stores the x and y coordinates of the circles. Executing the code below however, simply produces nothing, without any sign of the circles, because the function drawCricles runs asynchronously, and thus the command background(40) clears everything before the circles can be drawn.
Here is my code:
function setup() {
createCanvas(windowWidth, windowHeight);
background(40);
stroke(80);
smooth();
frameRate(60);
}
function drawCircles() {
firebase.database().ref("circles").once("value", function(snapshot) {
var snapshotVal = snapshot.val();
var circleCount = snapshotVal.numCircles;
for (var j = 0; j < circleCount; j++) {
firebase.database().ref("circles" + j).once("value", function(snapshot) {
var snapshotValue = snapshot.val();
fill(143, 2, 2);
ellipse(snapshotValue.xPos, 50, 50);
});
}
});
}
function draw() {
stroke(80);
background(40);
stroke(0);
drawCircles();
}

It seems your problem is simply 60 frames per second, which is causing a foot-race condition. Firebase .once will execute async when it completes fetching, and P5 won't wait for it to fetch because it'll stick with its framerate timing.
In this specific case I have multiple recommendations which will hopefully get you very close to the result you desire.
1 - Restructure your code
There's two issues with the current structure of your code.
Case 1 : Your current code would lead me to think your circles are updated in realtime in the database, and you need to stay up to date, so you keep fetching their latest position. If this is the case, you should use .on("value") instead of .once("value") and let firebase send you the updates whenever circles change, instead of asking it 60 times a second to save roundtrip request time. If this is the case : See my solution 1 below.
Case 2 : If your circles aren't updated real-time in the database, and you just want the whole list of circles, then you're fetching the list 60 times a second for no reason. You should instead fetch the list using .once upon setup and iterate through that list in draw() later. See solution 2 below.
2 - Restructure your database
In either case, your current database model requires you to keep fetching in a loop. Which means you're making as many requests as your circleCount. This is bad for your usage, simply because each request takes additional trip time, and we're trying to reduce the time it takes, so that it would be closer to real-time. (or match the framerate)
Currently your circles are seemingly saved as circles1 circles2 etc all at root, because you're using .ref("circles" + j) to retrieve them. Make it so that you save your circles like this : .ref("circles/" + j) that would mean that each circle is now saved in circles. like circles/circle1 circles/circle2 etc.
The benefit of this is that now you don't need the additional requests to firebase get all circles. Firebase has incredibly convenient things like forEach to iterate through all children with a single request.
3 - Clear background in your firebase callback
Currently, you clear the background in a frame-rate specific manner. This means that if each of your firebase calls take longer than 1/60th of a second (16 miliseconds) you will have cleared the background and move on. Chances of you achieving this speed is very low even after we structure our database. so instead, I would recommend first using 30fps, which would also reduce the number of calls you will make to firebase to 30 calls per second.
Solution 1
If your circles are updated in the database (say for example by some other game-player or someone else, and you want your code to always display the latest xPos)
var latestCirclePositionsSnapshot;
function setup() {
createCanvas(windowWidth, windowHeight);
background(40);
stroke(80);
smooth();
frameRate(60);
firebase.database().ref("circles").on("value", function(snapshot) {
// got a new value from database, so let's save this in a global variable.
latestCirclePositionsSnapshot = snapshot;
// we will keep drawing this update until we get a new one from the database.
});
}
function draw() {
drawCircles();
}
function clearBackground () {
stroke(80);
background(40);
}
function drawCircles() {
clearBackground();
stroke(0);
latestCirclePositionsSnapshot.forEach(function(circleSnapshot) {
// circleData will be the actual contents of each circle
var circleData = circleSnapshot.val();
fill(143, 2, 2);
ellipse(circleData.xPos, 50, 50);
});
}
Basically this will keep drawing the last circle positions we got from firebase until we get a new one. (so P5 will keep refreshing at 60fps, but your firebase updates will be as realtime as firebase can run and fetch from firebase etc.)
Solution 2
If you don't have real-time updates in your database, and all you'd like is to draw circles by getting data from firebase once (say for example to plot some dots based on some data)
var circlePositions;
var gotPositions = false;
function setup() {
createCanvas(windowWidth, windowHeight);
background(40);
stroke(80);
smooth();
frameRate(60);
firebase.database().ref("circles").once("value", function(snapshot) {
// got the circle values from the database
// let's store them and we'll keep drawing them forever.
circlePositions = snapshot;
gotPositions = true;
});
}
function draw() {
drawCircles();
}
function clearBackground () {
stroke(80);
background(40);
}
function drawCircles() {
clearBackground();
stroke(0);
if (gotPositions) {
circlePositions.forEach(function(circleSnapshot) {
// circleData will be the actual contents of each circle
var circleData = circleSnapshot.val();
fill(143, 2, 2);
ellipse(circleData.xPos, 50, 50);
});
} else {
// Display some text here like "LOADING DATA FROM SERVERS..."
}
}
Hope these help :) It's good to see another fellow fan of Processing & Firebase.

I looked into the docs and here is an example of how they suggest to deal with a fetched data. In your case, try to separate fetching and drawing, cache your data using some global variable:
var circles = [];
function fetchData() {
firebase.database().ref("circles").once("value",
function(snapshot) {
var snapshotVal = snapshot.val();
var circleCount = snapshotVal.numCircles;
circles = [];
for (var j = 0; j < circleCount; j++) {
firebase.database().ref("circles" + j).once("value", function(snapshot) {
circles.push(snapshot.val());
});
}
});
}
function setup() {
createCanvas(windowWidth, windowHeight);
background(40);
stroke(80);
smooth();
frameRate(60);
fetchData();
}
function drawCircles() {
circles.forEach(function (snapshotValue) {
var snapshotValue = snapshot.val();
fill(143, 2, 2);
ellipse(snapshotValue.xPos, 50, 50);
});
}
function draw() {
stroke(80);
background(40);
stroke(0);
drawCircles();
}
If you need to display always relevant data, try to call fetchData function using setInterval, like:
function setup() {
createCanvas(windowWidth, windowHeight);
background(40);
stroke(80);
smooth();
frameRate(60);
setInterval(fetchData, 5000); //will call fetchData every 5000 ms
}

The problem is not that drawCircles() is asynchronous -- the problem is that draw() is called at frameRate(), and background() clears the screen with a solid color each fraction of a second when draw loops: see draw reference and background. If you remove the line background(40) from draw() then it will not clear the screen each frame, and drawn circles will accumulate as desired. This is simpler than redrawing all the Firebase data each frame.
The following sketch that demonstrates the concept: background() is only called during setup(), not during draw(), so the screen area is colored once, then progressively overwritten with circles that accumulate.
function setup() {
createCanvas(400, 200);
frameRate(5)
background(40);
}
function drawCircles() {
fill(143, 2, 2);
ellipse(random(width), 50, 50);
}
function draw() {
// background(40);
drawCircles();
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.16/p5.js"></script>
<html>
<head>
</head>
<body>
</body>
</html>
If you want to accumulate some things and wipe away others each frame, then you need to accumulate your circles on a createGraphics buffer. Each frame redraw that buffer of circles onto the canvas, then draw ephemeral elements (like a mouse indicator etc.) on top.
Here is an example: each frame the canvas is cleared with background(), then the pg buffer is drawn onto the canvas, then a white circle is drawn at the mouse. Because background clears the screen, the white circle does not leave trails behind from frame to frame -- but the red circles are drawn to an un-cleared graphics buffer, so they persist.
var pg;
function setup() {
createCanvas(400, 200);
pg = createGraphics(400, 200);
background(40);
}
function drawCircles() {
pg.fill(143, 2, 2);
pg.ellipse(random(pg.width), 50, 50);
}
function draw() {
background(40);
drawCircles();
image(pg,0,0);
fill(255);
ellipse(mouseX,mouseY,50,50);
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.16/p5.js"></script>
<html>
<head>
</head>
<body>
</body>
</html>

Exporting a video in p5.js

I am creating a simple animation program in p5.js. When a user clicks the save button, I want to download a video of the animation.
I have an object called frames where each key is labelled frame_1, frame_2 and so on. The value associated with each key is an array of line segments that makes up that frame.
I am trying to think of an approach to take this data and create an mp4 video. p5.js has a built in save function that I thought might be helpful but it is not a full solution on its own. I could save each frame as an individual image and then somehow stitch those images together on the client side but I have yet to find a solution to this.
Any other approaches would be great as well. The only requirement is that it is done client side.

Since p5.js is built on the Canvas API, in modern browsers, you can use a MediaRecorder to do this job.
const btn = document.querySelector('button'),
chunks = [];
function record() {
chunks.length = 0;
let stream = document.querySelector('canvas').captureStream(30),
recorder = new MediaRecorder(stream);
recorder.ondataavailable = e => {
if (e.data.size) {
chunks.push(e.data);
}
};
recorder.onstop = exportVideo;
btn.onclick = e => {
recorder.stop();
btn.textContent = 'start recording';
btn.onclick = record;
};
recorder.start();
btn.textContent = 'stop recording';
}
function exportVideo(e) {
var blob = new Blob(chunks);
var vid = document.createElement('video');
vid.id = 'recorded'
vid.controls = true;
vid.src = URL.createObjectURL(blob);
document.body.appendChild(vid);
vid.play();
}
btn.onclick = record;
// taken from pr.js docs
var x, y;
function setup() {
createCanvas(300, 200);
// Starts in the middle
x = width / 2;
y = height;
}
function draw() {
background(200);
// Draw a circle
stroke(50);
fill(100);
ellipse(x, y, 24, 24);
// Jiggling randomly on the horizontal axis
x = x + random(-1, 1);
// Moving up at a constant speed
y = y - 1;
// Reset to the bottom
if (y < 0) {
y = height;
}
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.7/p5.min.js"></script>
<button>start recording</button><br>

ccapture works well with p5.js to achieve the goal of recording what's displaying on a canvas.
Here is a demo of ccapture working with p5.js. The source code comes with the demo.
This method won't output laggy videos because it is not recording what you see on the screen, which can be laggy. Instead, it writes every frame into the video and tells the videos to play at a fixed frame rate. So even if it takes seconds to calculate just one frame, the output video will play smoothly without showing any delay between frames.
However, there is one caveat though. This method only works with Chrome.

As you specified in the comments that a gif would also work, here is a solution:
Below is a sample p5 sketch that records canvas animation and turns it into a gif, using gif.js.
Works in browsers supporting: Web Workers, File API and Typed Arrays.
I've provided this code so you can get an idea of how to use this library because not much documentation is provided for it and I had a hard time myself figuring it out.
var cnv;
var gif, recording = false;
function setup() {
cnv = createCanvas(400, 400);
var start_rec = createButton("Start Recording");
start_rec.mousePressed(saveVid);
var stop_rec = createButton("Stop Recording");
stop_rec.mousePressed(saveVid);
start_rec.position(500, 500);
stop_rec.position(650, 500);
setupGIF();
}
function saveVid() {
recording = !recording;
if (!recording) {
gif.render();
}
}
var x = 0;
var y = 0;
function draw() {
background(51);
fill(255);
ellipse(x, y, 20, 20);
x++;
y++;
if (recording) {
gif.addFrame(cnv.elt, {
delay: 1,
copy: true
});
}
}
function setupGIF() {
gif = new GIF({
workers: 5,
quality: 20
});
gif.on('finished', function(blob) {
window.open(URL.createObjectURL(blob));
});
}
More Info :
This sketch starts recording frames when you click start_rec and stops when you hit stop_rec, in your sketch you might want to control things differently, but keep in mind that addFrame only adds one frame to the gif so you need to call it in the draw function to add multiple frames, you can pass in an ImageElement, a CanvasElement or a CanvasContext along with other optional parameters.
In the gif.on function, you can specify a callback function to do whatever you like with the gif.
If you want to fine tune settings of the gif, like quality, repeat, background, you can read more here. Hope this helps!

Phaserio Dynamic Sprite Creation

I am using PhaserIO in conjunction with Meteor to create a multiplayer html5 game and have run into a snag I cannot seem to figure out in a networking prototype I was making. First, the relevant code (also available as a gist):
if(Meteor.isClient) {
var game,
mainState,
mainStateInitialized,
characterData;
Template.game.game = function() {
characterData = Character.find().fetch();
if(!mainStateInitialized)
{
game = new Phaser.Game(500, 600, Phaser.AUTO, 'gameScreen');
createMainState()
}
}
}
function createMainState()
{
mainState = {
sprites: null,
playerLastFrame: characterData.length,
playerCurrentFrame: null,
charData: characterData,
preload: function() {
this.sprites = game.add.group();
game.stage.disableVisibilityChange = true;
game.load.image('hello', 'resources/hello.png');
},
create: function() {
$.each(characterData, function(index) {
var sprite = mainState.sprites.create(this.x, this.y, 'hello');
sprite.angle = this.angle;
});
},
update: function() {
this.charData = characterData;
this.playersCurrentFrame = this.charData.length;
if(this.playersLastFrame > this.playersCurrentFrame) {
//todo: remove player that left
}
else if(this.playersLastFrame < this.playersCurrentFrame) {
for(var i = playersLastFrame; i < playersCurrentFrame; i++) {
var thisData = this.charData[i],
sprite = null;
sprite = mainState.sprites.create(thisData.x, thisData.y, 'hello');
sprite.angle = thisData.angle;
}
}
for(var j = 0; j < mainState.sprites.length; j++) {
mainState.sprites.getAt(j).angle = this.charData[j].angle;
}
playersLastFrame = this.charData.length;
}
}
game.state.add('main', mainState);
game.state.start('main');
mainStateInitialized = true;
}
The idea of this prototype is to have a sprite shown in the canvas for each account in the DB. The main features I am testing are:
Dynamically adding sprites/player data seamlessly (as all proper multiplayer online games should be capable of. This will eventually pave the way for a proper join/leave system)
And to mess with creating efficient packets.
Right now, I am running into an issue with dynamically creating a new sprite when a player creates a new account. About 75% of the time, when a player makes a new account nothing happens. Meteor correctly pushes down the character data, which I can query, and mainState.sprites correctly shows the sprite data. However, nothing is rendered on the canvas.
The other 25% of the time, it works fine. In addition, if I have the code break-pointed it works 100% of the time as far as I can tell.
So, something intermittent is obviously occurring here but I can't figure out what the issue is. Is there something I am missing when adding a sprite during the update loop? Is there a better way to approach this?
I have my code on Nitrous.io, so I can run a localhost instance for you to hit if it would help in solving the problem.

The issue was I was not setting playersLastFrame = playersCurrentFrame.
I feel silly now considering this is a basic loop/compare structure. last = current at end of loop.
Sigh : (.

Optimizing easeljs line drawing for mobile

I'm trying to make a simple application where the user can draw lines by click-dragging the mouse or through gestures on their touch device screen.
It's perfectly fine on my desktop machine, but on my phone its very slow and jerky. It's not that performance degrades over time, it's immediately noticable.
I'm using easeljs and I extended shape. On mouse movements it records the points and on tick it draws them. The stage's autoClear is set to false and the graphics object clears before it draws so it doesn't redraw anything from the previous tick.
(function (window) {
function LineDrawer() {
this.initialize();
}
//Inheritance from Container
LineDrawer.prototype = new createjs.Shape();
LineDrawer.prototype.Shape_initialize = LineDrawer.prototype.initialize;
LineDrawer.prototype.Shape_tick = LineDrawer.prototype._tick;
LineDrawer.prototype.initialize = function () {
//call to initialize() method from parent class
this.Shape_initialize();
this.points = [];
this.mouseMoveEventListener = $.proxy(this.onMouseMove, this);
}
LineDrawer.prototype._tick = function (e) {
//call to _tick method from parent class
this.Shape_tick();
var points = this.points;
if (points.length > 0) {
var graphics = this.graphics.clear()
.setStrokeStyle(3, 'round', 'round')
.beginStroke("#000000")
.moveTo(points[0].x, points[0].y)
var pt;
for (var i = 1; i < points.length; i = i + 1) {
pt = points[i];
graphics.lineTo(pt.x, pt.y);
}
points.length = 0;
if (typeof pt !== 'undefined') {
points.push(new createjs.Point(pt.x, pt.y));
}
}
}
LineDrawer.prototype.onMouseDown = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
this.parent.addEventListener("stagemousemove", this.mouseMoveEventListener);
}
LineDrawer.prototype.onMouseMove = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
}
LineDrawer.prototype.onMouseUp = function (e) {
this.points.push(new createjs.Point(e.stageX, e.stageY));
this.parent.removeEventListener("stagemousemove", this.mouseMoveEventListener);
}
window.LineDrawer = LineDrawer;
}(window));
Here's the code for setting up the stage:
var stage,
lineDrawer;
$(document).ready(function () {
lineDrawer = new LineDrawer();
var $canvas = $('#canvasMain');
stage = new createjs.Stage($canvas[0]);
createjs.Touch.enable(stage);
stage.addChild(lineDrawer);
stage.autoClear = false;
stage.addEventListener("stagemousedown", $.proxy(lineDrawer.onMouseDown, lineDrawer));
stage.addEventListener("stagemouseup", $.proxy(lineDrawer.onMouseUp, lineDrawer));
createjs.Ticker.timingMode = createjs.Ticker.RAF;
createjs.Ticker.addEventListener("tick", stage);
})
Here's a fiddle for everything. Other information: I'm using jquery-1.8.3 and a RAF polyfill, these are my phone specs. I also got somebody to try with a much better Samsung phone with the same results.
Although admittedly my phone is on the low end of the spectrum, its not a dinosaur of a phone. It is android 4.0+ and what I'm doing really isn't that complicated as far as I can tell. Am I doing anything wrong and/or is there anything I can do to improve this? I wonder if maybe its a problem with the touch events and not the drawing speed as well.
EDIT: the other phone with laggy drawing was a Samsung S3

Answering my own question.
The problem was a combination of me not using the graphic's endStroke method after all the lines causing the drawing to be one tick behind, and a logic error where I would only draw lines if there was one or more points, but really it was supposed to be 2 or more points.
The second part was causing the most lag. Interestingly there was no error since the array index is never out of bounds because of the for loop condition, so I guess just having an impossible for condition was really slow on the Chrome for android browsers, without crashing.