Develop Reference

efficient way of streaming a html5 canvas content?

I'm trying to stream the content of a html5 canvas on a live basis using websockets and nodejs.
The content of the html5 canvas is just a video.
What I have done so far is:
I convert the canvas to blob and then get the blob URL and send that URL to my nodejs server using websockets.
I get the blob URL like this:
canvas.toBlob(function(blob) {
url = window.URL.createObjectURL(blob);
});
The blob URLs are generated per video frame (20 frames per second to be exact) and they look something like this:
blob:null/e3e8888e-98da-41aa-a3c0-8fe3f44frt53
I then get that blob URL back from the the server via websockets so I can use it to DRAW it onto another canvas for other users to see.
I did search how to draw onto canvas from blob URL but I couldn't find anything close to what i am trying to do.
So the questions I have are:
Is this the correct way of doing what i am trying to achieve? any
pros and cons would be appreciated.
Is there any other more efficient way of doing this or I'm on a right
path?
Thanks in advance.
EDIT:
I should have mentioned that I cannot use WebRTC in this project and I have to do it all with what I have.
to make it easier for everyone where I am at right now, this how I tried to display the blob URLs that I mentioned above in my canvas using websockets:
websocket.onopen = function(event) {
websocket.onmessage = function(evt) {
var val = evt.data;
console.log("new data "+val);
var canvas2 = document.querySelector('.canvMotion2');
var ctx2 = canvas2.getContext('2d');
var img = new Image();
img.onload = function(){
ctx2.drawImage(img, 0, 0)
}
img.src = val;
};
// Listen for socket closes
websocket.onclose = function(event) {
};
websocket.onerror = function(evt) {
};
};
The issue is that when I run that code in FireFox, the canvas is always empty/blank but I see the blob URLs in my console so that makes me think that what I am doing is wrong.
and in Google chrome, i get Not allowed to load local resource: blob: error.
SECOND EDIT:
This is where I am at the moment.
First option
I tried to send the whole blob(s) via websockets and I managed that successfully. However, I couldn't read it back on the client side for some strange reason!
when I looked on my nodejs server's console, I could see something like this for each blob that I was sending to the server:
<buffer fd67676 hdsjuhsd8 sjhjs....
Second option:
So the option above failed and I thought of something else which is turning each canvas frame to base64(jpeg) and send that to the server via websockets and then display/draw those base64 image onto the canvas on the client side.
I'm sending 24 frames per second to the server.
This worked. BUT the client side canvas where these base64 images are being displayed again is very slow and and its like its drawing 1 frame per second. and this is the issue that i have at the moment.
Third option:
I also tried to use a video without a canvas. So, using WebRTC, I got the video Stream as a single Blob. but I'm not entiely sure how to use that and send it to the client side so people can see it.
IMPORTANT: this system that I am working on is not a peer to peer connection. its just a one way streaming that I am trying to achieve.

The most natural way to stream a canvas content: WebRTC
OP made it clear that they can't use it, and it may be the case for many because,
Browser support is still not that great.
It implies to have a MediaServer running (at least ICE+STUN/TURN, and maybe a gateway if you want to stream to more than one peer).
But still, if you can afford it, all you need then to get a MediaStream from your canvas element is
const canvas_stream = canvas.captureStream(minimumFrameRate);
and then you'd just have to add it to your RTCPeerConnection:
pc.addTrack(stream.getVideoTracks()[0], stream);
Example below will just display the MediaStream to a <video> element.
let x = 0;
const ctx = canvas.getContext('2d');
draw();
startStream();
function startStream() {
// grab our MediaStream
const stream = canvas.captureStream(30);
// feed the <video>
vid.srcObject = stream;
vid.play();
}
function draw() {
x = (x + 1) % (canvas.width + 50);
ctx.fillStyle = 'white';
ctx.fillRect(0,0,canvas.width,canvas.height);
ctx.fillStyle = 'red';
ctx.beginPath();
ctx.arc(x - 25, 75, 25, 0, Math.PI*2);
ctx.fill();
requestAnimationFrame(draw);
}
video,canvas{border:1px solid}
<canvas id="canvas">75</canvas>
<video id="vid" controls></video>
The most efficient way to stream a live canvas drawing: stream the drawing operations.
Once again, OP said they didn't want this solution because their set-up doesn't match, but might be helpful for many readers:
Instead of sending the result of the canvas, simply send the drawing commands to your peers, which will then execute these on their side.
But this approach has its own caveats:
You will have to write your own encoder/decoder to pass the commands.
Some cases might get hard to share (e.g external media would have to be shared and preloaded the same way on all peers, and the worse case being drawing an other canvas, where you'd have to also have shared its own drawing process).
You may want to avoid intensive image processing (e.g ImageData manipulation) to be done on all peers.
So a third, definitely less performant way to do it, is like OP tried to do:
Upload frames at regular interval.
I won't go in details in here, but keep in mind that you are sending standalone image files, and hence a whole lot more data than if it had been encoded as a video.
Instead, I'll focus on why OP's code didn't work?
First it may be good to have a small reminder of what is a Blob (the thing that is provided in the callback of canvas.toBlob(callback)).
A Blob is a special JavaScript object, which represents binary data, generally stored either in browser's memory, or at least on user's disk, accessible by the browser.
This binary data is not directly available to JavaScript though. To be able to access it, we need to either read this Blob (through a FileReader or a Response object), or to create a BlobURI, which is a fake URI, allowing most APIs to point at the binary data just like if it was stored on a real server, even though the binary data is still just in the browser's allocated memory.
But this BlobURI being just a fake, temporary, and domain restricted path to the browser's memory, can not be shared to any other cross-domain document, application, and even less computer.
All this to say that what should have been sent to the WebSocket, are the Blobs directly, and not the BlobURIs.
You'd create the BlobURIs only on the consumers' side, so that they can load these images from the Blob's binary data that is now in their allocated memory.
Emitter side:
canvas.toBlob(blob=>ws.send(blob));
Consumer side:
ws.onmessage = function(evt) {
const blob = evt.data;
const url = URL.createObjectURL(blob);
img.src = url;
};
But actually, to even better answer OP's problem, a final solution, which is probably the best in this scenario,
Share the video stream that is painted on the canvas.

Html2canvas: facebook plugin comments

I have a website witch uses facebook plugin comments. I'm looking for a way to have those comments inside a screenshot. If I use the simple html2canvas I get a blank box instead of them. So I try to use html2canvasproxy but now it print some javascript console log instead of the facebook comments.
It shoud be like but I get . I noticed that the html2canvasproxy.php saves the facebook plugin html correctly.
I can't find any javascript error in the console log.
I'm using the following code to take the screenshot:
html2canvas(document.body, {
"logging": true, //Enable log (use Web Console for get Errors and Warnings)
"proxy":"js/html2canvasproxy.php",
"onrendered": function(canvas) {
var img = new Image();
img.onload = function() {
img.onload = null;
document.body.appendChild(img);
};
img.onerror = function() {
img.onerror = null;
if(window.console.log) {
window.console.log("Not loaded image from canvas.toDataURL");
} else {
alert("Not loaded image from canvas.toDataURL");
}
};
img.src = canvas.toDataURL("image/png");
}
});
And I have this settings in html2canvasproxy.php:
//Turn off errors because the script already own uses "error_get_last"
error_reporting(0);
//setup
define('JSLOG', 'console.log'); //Configure alternative function log, eg. console.log, alert, custom_function
define('PATH', '../screenshots');//relative folder where the images are saved
define('CCACHE', 60 * 5 * 1000);//Limit access-control and cache, define 0/false/null/-1 to not use "http header cache"
define('TIMEOUT', 30);//Timeout from load Socket
define('MAX_LOOP', 10);//Configure loop limit for redirect (location header)
define('CROSS_DOMAIN', 0);//Enable use of "data URI scheme"
//constants
define('EOL', chr(10));
define('WOL', chr(13));
define('GMDATECACHE', gmdate('D, d M Y H:i:s'));

First idea I got while reading is to include some timeout - waiting a bit longer (let's say 200ms) - so that you have more probability for things to get loaded.
But after reading this on plugin site: "The script allows you to take "screenshots" of webpages or parts of it, directly on the users browser. The screenshot is based on the DOM and as such may not be 100% accurate to the real representation as it does not make an actual screenshot, but builds the screenshot based on the information available on the page." it could not help.
Personally I would investigate using another solution - like for example PhantomJS:
"PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG."
It's easy like this:
var page = require('webpage').create();
page.open('http://github.com/', function() {
page.render('github.png');
phantom.exit();
});

How to capture a photo from the webcam in the browser and save it in the server?

I have seen this done by many websites, but I wonder how they do it. Some even allow one to crop the image. Is there a standard library or package for this?

You don't need any library, because It could be done in several steps. I assume you are familiar with webcam and able to show signal from it in the Video object. If you don't, in short It reads as:
var video: Video = new Video();
addChild(video);
video.smoothing = true;
video.attachCamera(camera); //Camera reference
video.width = someWidth;
video.height = someHeight;
Because Video object implements IBitmapDrawable you can draw it in the Bitmap, and do whatever you want.
var bitmapData : BitmapData = new BitmapData(_video.width, _video.height);
//Tada! You have screenshot of the current frame from video object
bitmapData.draw(cameraView);
//For testing, add as Bitmap
addChild(new Bitmap(bitmapData));
As for sending to the server, you need some server-side implementation

Here is a very usefull blog i came across (not mine)
http://matthewschrager.com/2013/05/25/how-to-take-webcam-pictures-from-browser-and-store-server-side/

Put text on an image and save as image

I have one problem with HTML5 Canvas.
I have one image. On this image I want to put text and display/save this as an image.
I have this code:
window.onload = function(){
var canvas = document.getElementById("myCanvas");
var context = canvas.getContext("2d");
var imageObj = new Image();
imageObj.onload = function(){
context.drawImage(imageObj, 10, 10);
context.font = "20px Calibri";
context.fillText("My TEXT!", 50, 200);
};
imageObj.src = "mail-image.jpg";
};
This works fine. There is my image and the text on it.
But it is still a canvas and no image.
Can anybody help me?

For security reasons, there's no convenient way of saving a canvas drawing to a user's local drive.
As a workaround, go "old school": Convert the canvas to an image and display it in a new window.
window.onload = function(){
var canvas = document.getElementById("myCanvas");
var context = canvas.getContext("2d");
var imageObj = new Image();
imageObj.onload = function(){
context.drawImage(imageObj, 10, 10);
context.font = "20px Calibri";
context.fillText("My TEXT!", 50, 200);
// open the image in a new browser tab
// the user can right-click and save that image
var win=window.open();
win.document.write("<img src='"+canvas.toDataURL()+"'/>");
};
imageObj.src = "mail-image.jpg";
};

Sand boxing
Browsers are sand-boxed when it deals with saving content to user's hard disk. This is for security (you don't want a bad hacker (or spy) to overwrite system files or plant a virus or a backdoor etc.). So direct access is prevented and local storage is isolated.
You always need to "bridge" the content by an user interaction that approves the operation and therefor the browser will request you to choose a location for the file by popping up a dialog to make the user aware of that the browser tries to deliver content to be saved (see demo below).
Invoking save dialogs
Here are a couple of other possibilities to enable download.
If a link for example under the image is ok then you can do:
/// create an anchor/link (or use an existing)
var lnk = document.createElement('a');
/// set your image as data-uri link
lnk.href = canvas.toDataURL();
/// and the key, when user click image will be downloaded
lnk.download = 'filename.png';
/// add lnk to DOM, here after the canvas
canvas.parentElement.appendChild(lnk);
The download attribute is a new HTML5 feature. Instead of "navigating" to this location the browser will show a save dialog instead and let the user save its content to disk.
You can also automate the whole clicking feature by generating an event for it.
For example:
function download(canvas, filename) {
if (typeof filename !== 'string' || filename.trim().length === 0)
filename = 'Untitled';
var lnk = document.createElement('a'),
e;
lnk.download = filename;
lnk.href = canvas.toDataURL();
if (document.createEvent) {
e = document.createEvent("MouseEvents");
e.initMouseEvent('click', true, true, window,
0, 0, 0, 0, 0, false, false,
false, false, 0, null);
/// send event
lnk.dispatchEvent(e);
} else if (lnk.fireEvent) {
lnk.fireEvent("onclick");
}
}
Saving to server
You can always go by the step of saving the file to a server. However, you will also have to go through the save dialog step when retrieving the file from server (the dialog).
If you want to store the file only to be shown in the browser this is perfect.
There are various ways to do this (there are many solutions on SO for this).
Local storage
And a different option is to store the file in the browser's local storage. You have Web Storage, however this is very limited (typically between 2.5 - 5 mb) and considering that each char stored takes two bytes the actual storage is just half of that (it can only store strings such as the data-uri and data-uris is about 33% larger than the original file). But if you save small icons, sprites etc. this might do.
In addition you can use Indexed DB (and the now deprectaed Web SQL) which can store larger data and you can also request user's permission to store x mb of dat.
The same goes with File API (which is currently only implemented in Chrome). This acts more like a file system and is intended to store huge files.
These might seem more complex if you are not familiar with them, but I mention them as possible options as these also saves you bandwidth communicating with a server and you move the "burden" to the client instead of the server.

How to save a png from javascript variable

I have an image encoded in base64 in a javascript variable : data:image/png;base64, base64 data
[EDIT]
I need to save that file to disk without asking to the visitor to do a right click
[/EDIT]
Is it possible ? How ?
Thanks in advance
Best regards

I know this question is 2 years old, but hopefully people will see this update.
You can prompt the user to save an image in a base64 string (and also set the filename), without asking the user to do a right click
var download = document.createElement('a');
download.href = dataURI;
download.download = filename;
download.click();
Example:
var download = document.createElement('a');
download.href = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==';
download.download = 'reddot.png';
download.click();
In order to trigger a click event using Firefox, you need to do what it is explained in this SO answer. Basically:
function fireEvent(obj,evt){
var fireOnThis = obj;
if(document.createEvent ) {
var evObj = document.createEvent('MouseEvents');
evObj.initEvent( evt, true, false );
fireOnThis.dispatchEvent( evObj );
} else if( document.createEventObject ) {
var evObj = document.createEventObject();
fireOnThis.fireEvent( 'on' + evt, evObj );
}
}
fireEvent(download, 'click')
As of 20/03/2013, the only browser that fully supports the download attribute is Chrome. Check the compatibility table here

... without asking to the visitor anyhing ... Is it possible?
No, that would have been a security hole. If it was possible, one would be able to write malware to the enduser's disk unaskingly. Your best bet may be a (signed) Java Applet. True, it costs a bit of $$$ to get it signed (so that it doesn't pop security warnings), but it is able to write data to enduser's disk without its permission.

I am surprised nobody here mentioned using HTML5 blobs together with a couple of nice libraries.
You first need https://github.com/eligrey/FileSaver.js/ and https://github.com/blueimp/JavaScript-Canvas-to-Blob.
Then you can load the image into a canvas
base_image = new Image();
base_image.src ='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==';
the canvas into a blob
var canvas = document.getElementById('YourCanvas');
context = canvas.getContext('2d');
// Draw image within
context.drawImage(base_image, 0,0);
and finally save it
x_canvas.toBlob(function(blob) {
saveAs(blob, "screenshot.png");
}, "image/png");
FF is not fully supported but at least you get a separate page with the image.
Check this out: http://jsfiddle.net/khhmm/9/
EDIT: this is not compatible with Safari / Mac.

As other answers already stated, you cannot do it only with javascript. If you want, you can send the data (using normal HTTP POST) to a PHP script, call header('Content-type: image/png') and output the decoded image data to the page using echo base64_decode($base64data).
This will work just as if user clicked on an image and open it or prompt him to save the file to disk (the normal browser's save file dialog).

It's not possible.
If it was, browsers would be massively insecure, being able to write random data to your hard disk without user interaction.

with javascript, you can't. the only real possibility i can think of will be a java-applet, but maybe (i don't know how long that image should be saved) you could simply add an img-tag with you png and force caching (but if the user deletes his cache, the image will be gone).

I think it's possible with JavaScript if you use ActiveX.
Another possibility is to make the server spit out that file with a different mime type so the browser asks the user to save it.

I think you can do it something(maybe not only with javascript...xul programming needed). There are Firefox addons that save images to a folder(check Firefox addons site)

You can make this file as blob on the server and use setTimeout function in order to fire the download.

The accepted solution seems to have a limitation for large data. If you're running into this (instead of the downloaded file's name, I see "download" and "Failed - Network error" in Chrome), here's what I did in order to download a 2mb file:
const blob = await (await fetch(document.getElementById('canvasID').toDataURL())).blob();
const file = new File([blob], {type:"image/png", lastModified: new Date()});
var a = document.createElement('a');
a.href = window.URL.createObjectURL(file);
a.download = 'image.png';
a.click();