Yes, I know about .getImageData()
I mean, let's say, I have to change some pixels:
var imageData = ctx.getImageData(...);
Seems, this method give me completely new copy of "real" (hiden somewhere deep from me) image-data.
I say that, because if you create new one:
var imgData2 = ctx.getImageData(.../*same parameters as before*/);
and compare two buffers:
imageData.data.buffer === imgData2.data.buffer; //false
So, each time it create new copy from it's bitmap. Oh my Gosh, Why? Okay, go further:
/*...apply some new changes to the imageData in a loop...*/
Nothing special above. But now, it's time to put this back:
ctx.putImageData(imageData, ...);
And this one inside itself run new loop and copy my imageData.
So much extra work! Is there a way to get actual imageData and manipulate it without get/put? And if no - I'm ask again - WHY? Is it security reasons? What they afraid I can do with that pixels?
Thank you!
Short answer :
No.
Not so long Answer :
You may be able to achieve it with some hacks but that would be such a pain.
Explanations :
According to the specs, getImageData returns a TypedArray which contains the canvas' image data, not a pointer to the live imageData as a modifiable object.
To paint on a canvas, you have to use methods such as fill, stroke, drawImage or putImageData ; that would make even less sense that each time you iterate through the array, the actual canvas image get modified.
Each time you call getImageData, a new TypedArray (note that the choice of the actual type is up to the UA) is created and filled with the current data of the canvasImage. This way, you can call this method, make different alterations on the ArrayBuffer, without modifying the actual image into the canvas (so you can store it, or call the method again).
As to why the buffer of the returned ImageData is not the same on each call, I think it is because "Pixels must be returned as non-premultiplied alpha values", but for performances, they do store it as premultiplied. You can see here the de-premultiplication operation from Firefox source code, which actually fills a new Uint8ClampedArray.
Also, it avoids to check if the canvasImage as been modified since the last call, and ensure you always get its current ImageData.
Related
Well, I fill ScreenBuffer:ImageData 480x360 and then want to draw it to the canvas 960x720. The task is to decrease the fillrate; the nowadays pixels are very small and we can make them bigger with some quality loss. I look for the operator with 2D-accelaration. But we can't write directly to js.html.Image and ImageData hasn't link to js.html.Image. I found an example for pure JS:
https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Pixel_manipulation_with_canvas
However, it doesn't want to work in Haxe because there isn't 'zoom' element. And there is some information about restrictions in HTML at copying from one image to another.
Many thanks for answers!
The compiler writes "js.html.Element has no field getContext"
getElementById()'s return type is the generic js.html.Element class. Since in your case, you know you're dealing with a <canvas>, you can safely cast it to the more specific CanvasElement. This then lets you call its getContext() method:
var canvas:CanvasElement = cast js.Browser.document.getElementById('zoom');
var zoomctx = canvas.getContext('2d');
I'm using Mapbox GL, and trying to get a snapshot of it, and merge the snapshot with another image overlaid for output.
I have a HTMLCanvasElement off screen, and I'm first writing the canvas returned from Map.getCanvas() to it, then writing the second (alpha transparent) canvas over that.
The problem is that, though I clearly see elements onscreen in the Map instance, the result only shows the second image/canvas written, and the rest is blank.
So I export just the map's canvas, and I see it is because the map canvas is blank, although a console.log() shows the image data from it to be a large chunk of information.
Here's my export function:
onExport(annotationCanvas: HTMLCanvasElement) {
const mergeCanvas: HTMLCanvasElement = document.createElement('canvas');
const mapCanvas: HTMLCanvasElement = this.host.map.getCanvas();
const mergeCtx: CanvasRenderingContext2D = mergeCanvas.getContext('2d');
mergeCanvas.height = annotationCanvas.height;
mergeCanvas.width = annotationCanvas.width;
mergeCtx.drawImage(mapCanvas, 0, 0);
mergeCtx.drawImage(annotationCanvas, 0, 0);
const mergedDataURL = mergeCanvas.toDataURL();
const mapDataURL = mapCanvas.toDataURL();
const annotationDataURL = annotationCanvas.toDataURL();
console.log(mapDataURL); // Lots of data
download(mapDataURL, 'map-data-only.png'); // Blank image # 1920x1080
download(mergedDataURL, 'annotation.png'); // Only shows annotation (the second layer/canvas) data
}
Is this a bug, or am I missing something?
UPDATE: I sort of figured out what this is about, and have possible options.
Upon stumbling upon a Mapbox feature request, I learned that if you instantiate your Map with the preserveDrawingBuffer option set to false (the default), you wont be able to get a canvas with usable image data. But setting this option to true degrades performance. But you can't change this setting after a Map is instantiated...
I want the Map to perform the best it possibly can!!!!
So, on this answer I stumbled on, regarding a question about three.js, I learned that if I take the screenshot immediately after rendering, I will get the canvas/data that I need.
I tried just calling this.host.map['_rerender']() right before I capture the canvas, but it still returned blankness.
Then searching around in the source code, I found a function called _requestRenderFrame, that looks like it might be what I need, because I can ask the Map to run a function immediately after the next render cycle. But as I come to find out, for some reason, that function is omitted in the compiled code, whilst present in the source, apparently because it is only in the master, and not part of the release.
So I don't really have a satisfactory solution yet, so please let me know of any insights.
As you mentioned in your updated question the solution is to set preserveDrawingBuffer: true upon Map initialisation.
To answer your updated question I think #jfirebaugh's answer at https://github.com/mapbox/mapbox-gl-js/issues/6448#issuecomment-378307311 sums it up very well:
preserveDrawingBuffer can't be modified on the fly. It has to be set at the time the WebGL context is created, and that can have a negative effect on performance.
It's rumored that you can grab the the canvas data URL immediately after rendering, without needing preserveDrawingBuffer, but I haven't verified that, and I suspect it's not guaranteed by the spec.
So although it might be possible to grab the canvas data URL immediately after rendering, it's not guaranteed by the spec.
Im using Pixi.js and trying to save a frame of the animation to an image. canvas.toDataUrl should work, but all i get is a black rectangle. See live example here
the code I use to extract the image data and set the image is:
var canvas = $('canvas')[0];
var context = canvas.getContext('2d');
$('button').click(function() {
var data = renderer.view.toDataURL("image/png", 1);
//tried var data = canvas.toDataURL();
$('img').attr('src', data);
})
I know this has been answered at least 5 other times on SO but ...
What Kaiido mentioned will work but the real issue is that canvas, when used with WebGL, by default has 2 buffers. The buffer you are drawing to and the buffer being displayed.
When you start drawing into a WebGL canvas, as soon as you exit the current event, for example your requestAnimationFrame callback, the canvas is marked for swapping those 2 buffers. When the browser re-draws the page it does the swap. The buffer that you were drawing to is swapped with the one that was being displayed. You're now drawing to other buffer. That buffer is cleared.
The reason it's cleared instead of just left alone is that whether the browser actually swaps buffers or does something else is up to the browser. For example if antialiasing is on (which is the default) then it doesn't actually do a swap. It does a "resolve". It converts the highres buffer you just drew to a normal res anti-aliased copy into the display buffer.
So, to make it more consistent, regardless of which way the browser does its default it just always clears whatever buffer you're about to draw to. Otherwise you'd have no idea if it had 1 frame old data or 2 frame old data.
Setting preserveDrawingBuffer: true tells the browser "always copy, never swap". In this case it doesn't have to clear the drawing buffer because what's in the drawing buffer is always known. No swapping.
What is the point of all that? The point is, if you want to call toDataURL or gl.readPixels you need to call it IN THE SAME EVENT.
So for example your code could work something like this
var capture = false;
$('button').click(function() {
capture = true;
});
function render() {
renderer.render(...);
if (capture) {
capture = false;
var data = renderer.view.toDataURL("image/png", 1);
$('img').attr('src', data);
}
requestAnimationFrame(render);
}
requestAnimationFrame(render);
In that case because you call toDataURL in the same javascript event as you rendered to it you'll get the correct results always regardless of wither or not preserveDrawingBuffer is true or false.
If you're writing app that is not constantly rendering you could also do something like
$('button').click(function() {
// render right now
renderer.render(...);
// capture immediately
var data = renderer.view.toDataURL("image/png", 1);
$('img').attr('src', data);
});
The reason preserveDrawingBuffer is false by default is because swapping is faster than copying so this allows the browser to go as fast as possible.
Also see this answer for one other detail
[NOTE]
While this answer is the accepted one, please do read the one by #gman just below, it does contain a way better way of doing.
Your problem is that you are using webGL context, then you need to set the preserveDrawingBuffer property of the webGL context to true in order to be able to call toDataURL() method.
Or alternatively, you can force pixi to use the 2D context, by using the CanvasRenderer Class
I'm trying to figure out how to add filters ti a video source with easljs. According to the documentation, I can just set filters to the bitmap.filtes property. This doesn't seem to work though. There is no error or anything, the filters are just not applied at all.
One confusion about the documentation, is that it mentions that cache() must be called on the display object in order for filters to be applied but for the bitmap class, it says that cache() should not be called.
Below is the code that I'm using as a test:
var stage = new createjs.Stage('screen');
var video = document.getElementById('source');
var bitmap = new createjs.Bitmap(video);
bitmap.scaleX = 0.44;
bitmap.scaleY = 0.44;
bitmap.y = 30;
bitmap.filters = [
new createjs.ColorFilter(0,0,0,1,255,0,0),
new createjs.BoxBlurFilter(5, 5, 10)
];
stage.addChild(bitmap);
The documentation regarding Bitmap and Filters is misleading - and I will make sure it is updated for the next release.
Caching of a bitmap should be avoided because it will NOT provide any performance benefit, which you will get from caching containers, shapes, and Text. In fact, it can even degrade performance instead because it will use additional memory to store the cached version
But you MUST cache a bitmap to apply filters to it. Add a cache call to your sample, and it should work.
Sorry for any confusion that is caused by the documentation.
I have the following code for writing draw calls to a "back buffer" canvas, then placing those in a main canvas using drawImage. This is for optimization purposes and to ensure all images get placed in sequence.
Before placing the buffer canvas on top of the main one, I'm using fillRect to create a dark-blue background on the main canvas.
However, the blue background is rendering after the sprites. This is unexpected, as I am making its fillRect call first.
Here is my code:
render: function() {
this.buffer.clearRect(0,0,this.w,this.h);
this.context.fillStyle = "#000044";
this.context.fillRect(0,0,this.w,this.h);
for (var i in this.renderQueue) {
for (var ii = 0; ii < this.renderQueue[i].length; ii++) {
sprite = this.renderQueue[i][ii];
// Draw it!
this.buffer.fillStyle = "green";
this.buffer.fillRect(sprite.x, sprite.y, sprite.w, sprite.h);
}
}
this.context.drawImage(this.bufferCanvas,0,0);
}
This also happens when I use fillRect on the buffer canvas, instead of the main one.
Changing the globalCompositeOperation between 'source-over' and 'destination-over' (for both contexts) does nothing to change this.
Paradoxically, if I instead place the blue fillRect inside the nested for loops with the other draw calls, it works as expected...
Thanks in advance!
Addenum: Changing the composite operation does behave as expected, but not for remedying this specific issue. Sorry for the ambiguity.
There's a lot that's suspect here.
First off double buffering a canvas does nothing but hurt performance by adding complication, all browsers do double buffering automatically, so if that's your goal here you shouldn't be drawing to a buffer.
Here's an example of why you don't need double buffering: http://jsfiddle.net/simonsarris/XzAjv/
So getting to the meat of the matter, lines of javascript inside a discrete function don't simply run out of order. Something else is wrong here.
Setting a breakpoint on the drawImage would solve this pretty much instantly, so if you aren't familiar with firebug or chrome developer tools I'd highly recommend giving them a look.
I'm guessing that the "blue" you're seeing is actually the only thing drawn to your "buffer" canvas and perhaps this.buffer is not actually the buffer context.
Another possibility is that this.w and this.h are accidentally very small, so that your initial clearRect and fillRect at the start of the method are doing nothing.
In any case speculation is nowhere near as good as opening up developer tools and actually looking at what's happening.
Generally speaking if you need things to be in order use an array not an object. Iterating over an object is not guaranteed to be in any particular order.
Use an array and for (var i=0; i