Develop Reference

Get pixels' colors from HTML Canvas WITHOUT getImageData()?

Task
I am currently trying to create a web extension for Firefox.
It should be able to read images' pixels.
For that purpose, I am rendering the image on an invsible canvas, and then I want to read it.
Example Code
function getImdata(reference) {
var canvas=document.createElement("canvas");
canvas.width=reference.naturalWidth;
canvas.height=reference.naturalHeight;
var context=canvas.getContext("2d");
context.drawImage(reference,0,0);
return context.getImageData(0,0,reference.naturalWidth,reference.naturalHeight); //Here I actually get the error...
}
Problem
However, I am getting a "Security Error" if I use "getImageData()".
Question
So I need a workaround, but couldn't find anything myself.
How can I read images' pixels without getImageData() ?
EDIT
Apparently it has something to do with CORS : HTML5 Canvas getImageData and Same Origin Policy
Thanks in advance!

There is. Since you're running from an extension your extension will have privileged access to cross-origin sources but only if loaded via fetch() and XMLHttpRequest() from a content script (or background script) - excerpt from that link:
Content scripts get the same cross-domain privileges as the rest of
the extension: so if the extension has requested cross-domain access
for a domain using the permissions key in manifest.json, then its
content scripts get access that domain as well.
This is accomplished by exposing more privileged XHR and fetch
instances in the content script [...]
Please note that these calls when called from a content script will not set origin and referer headers which sometimes can cause problems if the cross-origin site expects these to be set - for those cases you will need to use the non-privileged content.XMLHttpRequest or content.fetch() which will bring you back to square one.
The permissions in the manifest file (or if set permissions dynamically) must also allow access to these cross-origin sites.
This means however that you will have to "reload" the image source separately via these calls. You can do this the following way by first obtaining the original URL to the image you want to load, say, from a content script:
// example loading all images in current tab
let images = document.querySelectorAll("img");
for(let image of images) loadAsBitmap(image.src); // some sub-call using the url
Then load that source via the content script's fetch():
fetch(src).then(resp => { // load from original source
return resp.blob(); // obtain a blob
}).then(blob => { // convert blob, see below
// ...
};
When the blob is obtained you can convert it to an Object-URL and set that as source for an image and be able to go around the cross-origin restriction we otherwise face. In the content script, next steps would be:
let url = URL.createObjectURL(blob); // attach Object-URL to blob
let img = new Image(); // create image element *
img.onload = () => { // attach handler
let c = document.createElement("canvas"); // create canvas
let ctx = c.getContext("2d"); // get context
c.width = img.width; // canvas size = image
c.height = img.height;
ctx.drawImage(img, 0, 0); // draw in image
URL.revokeObjectURL(url); // remove reference.
let imageData =
ctx.getImageData(0,0,c.width,c.height); // get image data
// .. callback to a function that handles the image data
};
img.src = url; // start loading blob

Which way to create a canvas pattern from a dataToURL-image string as directly as possible?

I'm using an image that I much previously had made by
var patternImageAsDataURL= canvasObject.toDataURL('image/png');
In a later stage I want to make a canvas pattern object. The following code doesn't work - I assume the image is simply not loaded when going to the last line, where it is needed in the createPattern function.
var img = document.createElement('img');
img.src = patternImageAsDataURL;
// canvasctx was created somewhere else in the program
pattern = canvasctx.createPattern(img,'repeat');
I get the error: NS_ERROR_NOT_AVAILABLE: on the last line. (And when using console.log on width and heigth of img between the two last lines, I see when it's not working the dimensions are 0.)
When later on the same operation is done with the same dataURL, it does work. Though the image (img) should always be created anew. (Only reason I can see it's because of some internal optimization in Firefox. But that's offtopic here, unless someone does know the answer.) The width and height when printing them out to the console are correct then.
While I will quite soon program some pattern handling service, that should solve this, my question is in general and for speed concerns and for simplicity. (If I use some code with like 20 to 50 objects with patterns, I would prefer a lean solution over a memory or time saving function.)
Could I somehow use the dataURL more directly (and faster) for the
createPattern function?
And:
Could I force the program to wait after the img.src = patternImageAsDataURL; command until the image is loaded, and then to go on processing the code? (Like in the synchronous mode of the XMLrequests.)
(Using the onload event of the image isn't feasible in the current program flow.)
This is running on Firefox 32, Win 7.

A faster, more direct way to create a pattern
You can use a second canvas element as the source for a pattern.
This allows you to completely skip the interim step of creating an ImageURL and Image from your source canvas so your pattern creation will be faster.
var canvas=document.getElementById("canvas");
var ctx=canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
// Make a temporary canvas to be the template for a pattern
var pc=document.createElement('canvas');
var px=pc.getContext('2d');
pc.width=4;
pc.height=4;
px.fillStyle='palegreen';
px.fillRect(0,0,2,2);
px.fillRect(2,2,2,2);
// Use the temporary canvas as the image source for "createPattern"
var pattern=ctx.createPattern(pc,'repeat');
ctx.fillStyle=pattern;
ctx.fillRect(50,50,100,75);
ctx.strokeRect(50,50,100,75);
body{ background-color: ivory; }
#canvas{border:1px solid red;}
<h4>Using a temporary canvas as source for a Pattern.</h4>
<canvas id="canvas" width=300 height=300></canvas>

Option 1 - Canvas as image source
The obvious is of course to use the canvas itself as image source for the pattern.
createPattern() can take image, canvas, context (although not all browsers allow this) or even video as source.
CanvasPattern createPattern(CanvasImageSource image,
[TreatNullAs=EmptyString] DOMString repetition);
where CanvasImageSource is defined as:
typedef (HTMLImageElement or
HTMLVideoElement or
HTMLCanvasElement or
CanvasRenderingContext2D or
ImageBitmap) CanvasImageSource;
This is also the only way that will allow you to not use onload at some point later (provided the pattern is generated and not drawn in from an image/video source).
You cannot deal with asynchronous behavior without using callbacks (or promises), and expect the program to work properly. Period.
Option 2 - Data-URIs
If you for some reason cannot use the original canvas as source, you have to deal with the image asynchronously. Add a onload handler for it and continue from inside it:
var img = document.createElement('img');
img.onload = function() {
pattern = canvasctx.createPattern(this, 'repeat');
// continue from here..
};
img.src = patternImageAsDataURL;
Note that the process of this is relative slow due to the additional encoding/decoding process on top of the image handling itself. You can find more details about this in this answer.
Option 3 - Blob and object-URL
A Blob lets you store the data in binary form. This is preferred over storing the binary data as encoded string as with data-URIs. This will be faster to embed as well as extract compared to data-URIs.
You can use URL form with the Blob and use that as image source.
First create the Blob directly from canvas:
var patternImageAsBlob = canvas.toBlob(...); //IE: msToBlob()
This is also an asynchronous call so you need to take that into account.
For example:
var patternAsBlob;
canvas.toBlob(function(blob) {
patternAsBlob = blob;
// continue from here
}
Then when you need it as an image, generate an Object-URL for it like this:
var img = new Image(),
url = URL.createObjectURL(patternAsBlob);
img.onload = function() {
URL.revokeObjectURL(url); // clean up by removing the url object
pattern = canvasctx.createPattern(this, 'repeat');
// continue from here..
};
img.src = url;
Tips
If you have several images to load and set, it would be better to make an image loader to load in all resources to an array, when done create the patterns.
This will simplify the asynchronous chain-calling (optionally use promises, but this is not yet supported in IE without a polyfill).
You may need a polyfill for toBlob in older browser. One can be found here.
You may need to "unprefix" the createObjectURL(), here is one way:
var domURL = self.URL || self.webkitURL || self;
var url = domURL.createObjectURL( ... );

getImageData and .data() [duplicate]

I'm a newbie in HTML5+JS, I want to develop an hybrid app using ocrad.js.
The code given below, downloaded from github page is perfectly working for me(Chrome 32.0.1).
<html>
<head>
</head>
<body>
<script src="../ocrad.js"></script>
<script>
function OCRImage(image){
var canvas = document.createElement('canvas')
canvas.width = image.naturalWidth;
canvas.height = image.naturalHeight;
canvas.getContext('2d').drawImage(image, 0, 0)
return OCRAD(canvas)
}
function OCRPath(url, callback){
var image = new Image()
image.src = url;
image.onload = function(){ callback(OCRImage(image)) }
}
function OCRFile(file, callback){
var reader = new FileReader();
reader.onload = function(){ OCRPath(reader.result, callback); }
reader.readAsDataURL(file)
}
</script>
<input type="file" onchange="OCRFile(this.files[0], function(text){alert(text)})">
</body>
</html>
When I called OCRAD() API in my code its giving Uncaught SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': the canvas has been tainted by cross-origin data.
My CODE
<html>
<head>
<script src="../ocrad.js"></script>
<body>
<canvas id="cancan" width="800", height="500">Test image</canvas>
<script type="text/javascript">
function imageLoaded(ev) {
element = document.getElementById("cancan");
c = element.getContext("2d");
im = ev.target;
width = element.width;
height = element.height;
c.drawImage(im, 0, 0);
var data1=OCRAD(c);
console.log(data1);
}
im = new Image();
im.src = "message.png";
im.onload = imageLoaded;
</script>
</body>
</html>
I have seen similar Stackoverflow Q&A here but it didn't help me to solve the issue. Please answer if any one had any comment on this issue who have worked with Ocrad.js.
OR
Is there anyother way to pass my image file (here message.png in second code example) as an argument to OCRFile() function in first code example ? (Simply I want to pass an image stored in an local file URL to OCRAD() Call to return text. )
Thanks in advance.... :)

It is a cross-origin issue which is a security mechanism in browsers.
You will either need to:
Move image to same origin as the page (origin = domain, port and protocol)
Request CORS usage from the other origin if you can't move the image
Use a proxy page to load the image (see one in action here - note: I do not know this site so use only for testing with non-critical data).
A request can be made like this (assuming im contains the image you want to OCR treat):
function imageLoaded(ev) {
element = document.getElementById("cancan");
c = element.getContext("2d");
width = element.width;
height = element.height;
c.drawImage(this, 0, 0); // 'this' = current image loaded
var data1 = OCRAD(c);
console.log(data1);
}
var im = new Image();
im.onload = imageLoaded; // set onload before src
im.crossOrigin = 'anonymous'; // request CORS usage before setting src
im.src = "message.png";
If a request will work is entirely up to the server which may deny the request (which is default behavior in most cases).
In that case only moving the image or setting up a proxy page to load the external image will allow usage of it. Note that file:// or local files are considered different origins.
A proxy page is in essence a page you pass the image url to as an argument. The page will then, on server side, load the image and pass the data back to your first (requesting) page. This way you can "stream" the image through your own server removing CORS restrictions but at the expense of increased traffic on your own server. Some server may also block this approach by denying external access (ie. by referrer or IP etc.)
See Cross-Origin Resource Sharing for more details.

load an image via ajax into canvas

It relates to another question Cross Origin Resource Sharing Headers not working only for safari .
I am trying to load an image into canvas from s3.
Seems safari < 6.0 has a bug related to loading images via CORS. So the canvas get tainted though the image has cors enabled. So I was thinking if there is some way to make an ajax request and then load the response into a canvas ?
Note : Ajax request works properly with CORS. Just that while loading image safari doesn't respect the crossOrigin attribute and hence the request is made without cross-origin.
I have my images at s3 so there is no way to encode it to base64 and get it from amazon directly
I am preferring not to set up a proxy at my domain for the image
some javascript
var img_location = "//temp_upload_test.s3.amazonaws.com/IMG_0222.JPG"
var img = new Image();
img.onload = function(){
console.log("image loaded")
EXIF.getData(img,function(){
console.log("image data read");
var orientation = EXIF.getTag(img,'Orientation');
console.log("orientation"+orientation);
load_image_into_canvas_with_orientation(img,orientation);
})
console.log("image loaded function complete");
}
img.crossOrigin = "anonymous";
$(img).attr("crossOrigin","anonymous");
img.src = img_location;
One way I am trying to approach the problem is make an xhr request to s3. get the image as BinaryFile and then decode it to base64 and use it as img's src . But while decoding I get a DOM exception not sure if the idea itself is wrong

img.crossOrigin = "anonymous";
It does not works in all cases because of security issue.
You can load any image in canvas by converting that image to base64 string and then bind to canvas.
Example:
<img src="www.example.com/name.png" alt="test" />
to
<img src="data:image/png;base64,vnaXYZetc..." alt="test" />
then load it to in canvas.
Thank you.

JavaScript: how to force Image() not to use the browser cache?

If I load the nextimg URL manually in the browser, it gives a new picture every time I reload. But this bit of code shows the same image every iteration of draw().
How can I force myimg not to be cached?
<html>
<head>
<script type="text/javascript">
function draw(){
var canvas = document.getElementById('canv');
var ctx = canvas.getContext('2d');
var rx;
var ry;
var i;
myimg = new Image();
myimg.src = 'http://ohm:8080/cgi-bin/nextimg'
rx=Math.floor(Math.random()*100)*10
ry=Math.floor(Math.random()*100)*10
ctx.drawImage(myimg,rx,ry);
window.setTimeout('draw()',0);
}
</script>
</head>
<body onload="draw();">
<canvas id="canv" width="1024" height="1024"></canvas>
</body>
</html>

The easiest way is to sling an ever-changing querystring onto the end:
var url = 'http://.../?' + escape(new Date())
Some people prefer using Math.random() for that instead of escape(new Date()). But the correct way is probably to alter the headers the web server sends to disallow caching.

You can't stop it from caching the image altogether within Javascript. But, you can toy with the src/address of the image to force it to cache anew:
[Image].src = 'image.png?' + (new Date()).getTime();
You can probably take any of the Ajax cache solutions and apply it here.

That actually sounds like a bug in the browser -- you could file at http://bugs.webkit.org if it's in Safari or https://bugzilla.mozilla.org/ for Firefox. Why do i say potential browser bug? Because the browser realises it should not be caching on reload, yet it does give you a cached copy of the image when you request it programmatically.
That said are you sure you're actually drawing anything? the Canvas.drawImage API will not wait for an image to load, and is spec'd to not draw if the image has not completely loaded when you try to use it.
A better practice is something like:
var myimg = new Image();
myimg.onload = function() {
var rx=Math.floor(Math.random()*100)*10
var ry=Math.floor(Math.random()*100)*10
ctx.drawImage(myimg,rx,ry);
window.setTimeout(draw,0);
}
myimg.src = 'http://ohm:8080/cgi-bin/nextimg'
(You can also just pass draw as an argument to setTimeout rather than using a string, which will save reparsing and compiling the same string over and over again.)

There are actually two caches you need to bypass here: One is the regular HTTP cache, that you can avoid by using the correct HTTP headers on the image. But you've also got to stop the browser from re-using an in-memory copy of the image; if it decides it can do that it will never even get to the point of querying its cache, so HTTP headers won't help.
To prevent this, you can use either a changing querystring or a changing fragment identifier.
See my post here for more details.