I recently used canvas to conert images to webp, using :
const dataUrl = canvas.toDataURL('image/webp');
But this takes a lots of time for certain images, like 400ms.
I got a warning from Chrome, since it is blocking UI.
I would like to use an Offscreen Canvas to perform that conversion in background.
But :
1) I don't know which Offscreen Canvas I should use :
a] new OffscreenCanvas()
b] canvas.transferControlToOffscreen()
2) I load a local image url in an Image object (img.src = url) to get width and height of the local image. But I don't understand how to transfert the Image object to the offscreen Canvas, to be able to do in the worker :
ctx.drawImage(img, 0, 0)
Because If I don't transfert the image, worker doesn't know img.
You are facing an XY and even -Z problem here, but each may have an useful answer, so let's dig in.
X. Do not use the canvas API to perform image format conversion.
The canvas API is lossy, whatever you do, you will loose information from your original image, even if you do pass it lossless images, the image drawn on the canvas will not be the same as this original image.
If you pass an already lossy format like JPEG, it will even add information that were not in the original image: the compression artifacts are now part of the raw bitmap, and export algo will treat these as information it should keep, making your file probably bigger than the JPEG file you fed it with.
Not knowing your use case, it's a bit hard to give you the perfect advice, but generally, make the different formats from the version the closest to the raw image, and once it's painted in a browser, you are already at least three steps too late.
Now, if you do some processing on this image, you may indeed want to export the results.
But you probably don't need this Web Worker here.
Y. What takes the biggest blocking time in your description should be the synchronous toDataURL() call.
Instead of this historical error in the API, you should always be using the asynchronous and nonetheless more performant toBlob() method. In 99% of the cases, you don't need a data URL anyway, almost all you want to do with a data URL should be done with a Blob directly.
Using this method, the only heavy synchronous operation remaining would be the painting on canvas, and unless you are downsizing some huge images, this should not take the 400ms.
But you can anyway make it even better on newest canvas thanks to createImageBitmap method, which allows you to prepare asynchronously your image so that the image's decoding be complete and all that needs to be done is really just a put pixels operation:
large.onclick = e => process('https://upload.wikimedia.org/wikipedia/commons/c/cf/Black_hole_-_Messier_87.jpg');
medium.onclick = e => process('https://upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Black_hole_-_Messier_87.jpg/1280px-Black_hole_-_Messier_87.jpg');
function process(url) {
convertToWebp(url)
.then(prepareDownload)
.catch(console.error);
}
async function convertToWebp(url) {
if(!supportWebpExport())
console.warn("your browser doesn't support webp export, will default to png");
let img = await loadImage(url);
if(typeof window.createImageBitmap === 'function') {
img = await createImageBitmap(img);
}
const ctx = get2DContext(img.width, img.height);
console.time('only sync part');
ctx.drawImage(img, 0,0);
console.timeEnd('only sync part');
return new Promise((res, rej) => {
ctx.canvas.toBlob( blob => {
if(!blob) rej(ctx.canvas);
res(blob);
}, 'image/webp');
});
}
// some helpers
function loadImage(url) {
return new Promise((res, rej) => {
const img = new Image();
img.crossOrigin = 'anonymous';
img.src = url;
img.onload = e => res(img);
img.onerror = rej;
});
}
function get2DContext(width = 300, height=150) {
return Object.assign(
document.createElement('canvas'),
{width, height}
).getContext('2d');
}
function prepareDownload(blob) {
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = 'image.' + blob.type.replace('image/', '');
a.textContent = 'download';
document.body.append(a);
}
function supportWebpExport() {
return get2DContext(1,1).canvas
.toDataURL('image/webp')
.indexOf('image/webp') > -1;
}
<button id="large">convert large image (7,416 × 4,320 pixels)</button>
<button id="medium">convert medium image (1,280 × 746 pixels)</button>
Z. To draw an image on an OffscreenCanvas from a Web Worker, you will need the createImageBitmap mentioned above. Indeed, the ImageBitmap object produced by this method is the only image source value that drawImage() and texImage2D()(*) can accept which is available in Workers (all other being DOM Elements).
This ImageBitmap is transferable, so you could generate it from the main thread and then send it to you Worker with no memory cost:
main.js
const img = new Image();
img.onload = e => {
createImageBitmap(img).then(bmp => {
// transfer it to your worker
worker.postMessage({
image: bmp // the key to retrieve it in `event.data`
},
[bmp] // transfer it
);
};
img.src = url;
An other solution is to fetch your image's data from the Worker directly, and to generate the ImageBitmap object from the fetched Blob:
worker.js
const blob = await fetch(url).then(r => r.blob());
const img = await createImageBitmap(blob);
ctx.drawImage(img,0,0);
And note if you got the original image in your main's page as a Blob (e.g from an <input type="file">), then don't even go the way of the HTMLImageElement, nor of the fetching, directly send this Blob and generate the ImageBitmap from it.
*texImage2D actually accepts more source image formats, such as TypedArrays, and ImageData objects, but these TypedArrays should represent the pixel data, just like an ImageData does, and in order to have this pixel data, you probably need to already have drawn the image somewhere using one of the other image source formats.
Related
I have a page that load a random image as background image. Something like this:
<!-- some other codes -->
<style>
.random-background {
height: 100px;
width: 100px;
background-image: url("https://server-domain.com/random.php");
}
</style>
<div class="random-background"></div>
<!-- some other codes -->
https://server-domain.com/random.php will return a random image every time it is called, so using AJAX to call the link is not an option. I am writing a Chrome extension, so I have no control on the server behaviour. Is there a way, using JavaScript, I can get the exact image used as background as the user see on screen?
It may seems to be a duplicated with this question. The difference is in this question, the target element is a div instead of img, so in the suggested solution to that question,
context.drawImage(img, 0, 0, img.naturalWidth, img.naturalHeight);
won't work. This will be the error message
Uncaught TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLCanvasElement or HTMLImageElement or HTMLVideoElement or ImageBitmap or OffscreenCanvas or SVGImageElement or VideoFrame)'.
at :6:9
I'm posting this as a potentially viable solution, but ultimately this depends on the CORS policy of the server generating the random images.
This somewhat uses the concept from Javascript: how to get image as bytes from a page (without redownloading), however to answer the differences outlined by OP, this works a little different.
The idea is that the image is first downloaded using JavaScript and applied to an Image() object created by JavaScript. When the image is done loading, it is then loaded into a canvas object on which we can apply toDataURL(). Finally this new image data can be applied to the background-image property of the <div> element and also used anywhere else needed (it would just need to be saved as global variable).
The server CORS policy has not been disclosed by OP nor has what OP needs to do with the image data, so that is not being addressed in this answer.
const _Base64Image = url => {
const img = new Image();
img.setAttribute('crossOrigin', 'anonymous');
img.onload = () => {
const canvas = document.createElement("canvas");
canvas.width = img.width;
canvas.height = img.height;
const ctx = canvas.getContext("2d");
ctx.drawImage(img, 0, 0);
const dataURL = canvas.toDataURL("image/png");
_SetBG(document.querySelector("#imgTest"), dataURL, img.width, img.height);
}
img.src = url;
}
const _SetBG = (el, data, w, h) => {
el.style.width = `${w}px`;
el.style.height = `${h}px`;
el.style.backgroundImage = `url("${data}")`;
}
_Base64Image('https://nathanchapman.gallerycdn.vsassets.io/extensions/nathanchapman/javascriptsnippets/0.2.0/1510271465098/Microsoft.VisualStudio.Services.Icons.Default');
.random-background {
height: 100px;
width: 100px;
}
<div id="imgTest" class="random-background"></div>
Why you want to do this is still a mystery to me, and the following is surely a bad idea in most cases. It will cause performance issues. But if you really need it, here goes...
You can achieve this by intercepting each image request so that you can always store the response in JS before the browser can start using it. This could get very costly, especially if you need to perform this on every request.
All you have to do is make a web worker listen to the fetch event.
Something like this:
const images = {};
function doAnythingWith(request, response) {
// response.url would not work as it changes on redirects.
// request.url should have the URL from the CSS.
images[request.url] = response.body;
}
self.addEventListener("fetch", (event) => {
if (event.request.method !== "GET") return;
// Optionally include some exlusion logic here.
// Maybe by domain or file extension.
// Prevent the default, and handle the request ourselves.
event.respondWith(
(async () => {
const response = await fetch(event.request);
doAnythingWith(request, response);
return response;
})()
);
});
self.addEventListener("message", (event) => {
if (event.type !== "get-loaded-image") return;
return images[event.payload.url];
});
Then you should be able to get the image for any given URL, and it will be the one that the user got.
myWorker.postMessage({type: "get-loaded-image", payload: someUrl});
You can store this with the URL as a key. You can then send messages to this worker to retrieve the original content for a URL.
Or perhaps you don't need to send a message at all and execute all your logic directly in the worker.
Let's suppose I'm calling an API which serves images with random dimensions.
I need to know the width and the height for each image before rendering, because I need to do some computations for the precise layout I want.
With Web Apis, I've made this and it's working fine:
const res = await fetch("https://picsum.photos/200/300");
const blob = await res.blob();
const img = new Image();
img.src = URL.createObjectURL(blob);
console.log(img.src);
img.onload = () => {
console.log(img.width, img.height);
};
The problem is that with React Active, we don't have access to the Web Api Image Object.
Image is a component on React Native.
Here's where I'm at:
const res = await fetch("https://picsum.photos/200/300");
const blob = await res.blob();
So basically the binary data is stored in RAM right now, but I can't find any way to interpret this data into an image and get the dimensions.
React Native Image component has built-in getSize method. If you can transform your blob to URI than you can use the code below.
Image.getSize(myUri, (width, height) => { console.log(width, height)});
I'm trying to produce the same base64 data for an image file in both JavaScript and in Ruby. Unfortunately both are outputting two very different values.
In Ruby I do this:
Base64.encode64(File.binread('test.png'));
And then in JavaScript:
var image = new Image();
image.src = 'http://localhost:8000/test.png';
$(image).load(function() {
var canvas, context, base64ImageData;
canvas = document.createElement('canvas');
context = canvas.getContext('2d');
canvas.width = this.width;
canvas.height = this.height;
context.drawImage(this, 0, 0);
imageData = canvas.toDataURL('image/png').replace(/data:image\/[a-z]+;base64,/, '');
console.log(imageData);
});
Any idea why these outputs are different?
When you load the image in Ruby the binary file without any modifications will be encoded directly to base-64.
When you load an image in the browser it will apply some processing to the image before you will be able to use it with canvas:
ICC profile will be applied (if the image file contains that)
Gamma correction (where supported)
By the time you draw the image to canvas, the bitmap values has already been changed and won't necessarily be identical to the bitmap that was encoded before loading it as image (if you have an alpha channel in the file this may affect the color values when drawn to canvas - canvas is a little peculiar at this..).
As the color values are changed the resulting string from canvas will naturally also be different, before you even get to the stage of re-encoding the bitmap (as PNG is loss-less the encoding/compressing should be fairly identical, but factors may exist depending on the browser implementation that will influence that as well. to test, save out a black unprocessed canvas as PNG and compare with a similar image from your application - all values should be 0 incl. alpha and at the same size of course).
The only way to avoid this is to deal with the binary data directly. This is of course a bit overkill (in general at least) and a relative slow process in a browser.
A possible solution that works in some cases, is to remove any ICC profile from the image file. To save an image from Photoshop without ICC choose "Save for web.." in the file menu.
The browser is re-encoding the image as you save the canvas.
It does not generate an identical encoding to the file you rendered.
So I actually ended up solving this...
Fortunately I am using imgcache.js to cache images in the local filesystem using the FileSystem API. My solution is to use this API (and imgcache.js makes it easy) to get the base64 data from the actual cached copy of the file. The code looks like this:
var imageUrl = 'http://localhost:8000/test.png';
ImgCache.init(function() {
ImgCache.cacheFile(imageUrl, function() {
ImgCache.getCachedFile(imageUrl, function(url, fileEntry) {
fileEntry.file(function(file) {
var reader = new FileReader();
reader.onloadend = function(e) {
console.log($.md5(this.result.replace(/data:image\/[a-z]+;base64,/, '')));
};
reader.readAsDataURL(file);
});
});
});
});
Also, and very importantly, I had to remove line breaks from the base64 in Ruby:
Base64.encode64(File.binread('test.png')).gsub("\n", '');
I wanted to know if there was anyone out there that knows how
canvas.toDataURL("image/png");
works? I want to understand better because at times it seems to really slow my computer down.
Is there a way to optimize the base64 image before during or after to get better performance ?
function base64(url) {
var dataURL;
var img = new Image(),
canvas = document.createElement("canvas"),
ctx = canvas.getContext("2d"),
src = url;
img.crossOrigin = "Anonymous";
img.onload = function () {
canvas.height = img.height;
canvas.width = img.width;
ctx.drawImage(img, 0, 0);
var dataURL = canvas.toDataURL('image/png');
preload(dataURL);
canvas = null;
};
img.src = url;
}
Basically this is my function but I wanted to see if there was a way to make this process perform better or if there was an alternative to canvas.toDataURL('image/png');
thanks
toDataURL() does the following when called (synchronously):
Creates a file header based on the file type requested or supported (defaults to PNG)
Compresses the bitmap data based on file format
Encodes the resulting binary file to Base-64 string
Prepends the data-uri header and returns the result
When setting a data-uri as source (asynchronously):
String is verified
Base-64 part is separated and decoded to binary format
Binary file verified then parsed and uncompressed
Resulting bitmap set to Image element and proper callbacks invoked
These are time-consuming steps and as they are internal we cannot tap into them for any reason. As they are pretty optimized as they are given the context they work in there is little we can do to optimize them.
You can experiment with different compression schemes by using JPEG versus PNG. They are using very different compression techniques and depending on the image size and content one can be better than the other in various situations.
My 2 cents..
The high performance alternative is canvas.toBlob. It is extremely fast, asynchronous, and produces a blob which can also be swapped to disk, and is subjectly speaking simply far more useful.
Unfortunately it is implemented in Firefox, but not in chrome.
Having carefully bench-marked this, there is no way around because canvas.toDataURL itself is the bottleneck by orders of magnitude.
I am working on a FineUploader implementation. Special request is to create thumbnails on the fly client-side and then upload those with the original image-upload.
I have an implementation that works on FF, but does not seem to work on iOs. It looks like so:
var uploader = new qq.FineUploaderBasic({
button: document.getElementById(buttonID),
request: {
endpoint: '/up/load/a/' + $('section#ajax-viewport').data('albumid')
},
callbacks: {
onSubmit: function(id, fileName) {
// getFile obtains the file being uploaded
file = this.getFile(id);
// create a thumbnail & upload it:
ThumbDown(file, id, 200, fileName);
},
}
})
This code calls a function:
function ThumbDown(file, id, dimension, fileName) {
var reader = new FileReader();
reader.onload = function(e) {
var img = document.createElement("img");
img.onload = function (ev) {
var thumbnailDimensions; // object holding width & height of thumbnail
var c=document.getElementById("canvas-for-thumbnails"); // must be a <canvas> element
var ctx=c.getContext("2d");
// set thumbnail dimensions of canvas:
thumbnailDimensions = calcThumbnailDimension (img.width, img.height, dimension )
c.width = thumbnailDimensions.width;
c.height = thumbnailDimensions.height;
var ctx = c.getContext("2d");
ctx.drawImage(img, 0, 0, c.width, c.height);
uploadThumbnail(c.toDataURL('image/jpeg'), //a base64 encoded representation of the image
id,
fileName); // we need filename to combine with mother-image on the server
};
img.src = e.target.result;
}
reader.readAsDataURL(file);
} // end function
Finally the Thumbnail is uploaded with a dumb ajax-call:
function uploadThumbnail (base64encodedString, id, fileName) {
$.post('/up/thumb',
{
img : base64encodedString,
id: id,
fileName: fileName
},
function(data) {});
}
My questions:
1) Currently I have two uploads: one for mother-image and another for thumbnail. I would like to combine this in one FineUploader call. However, I do not see a way to do this, due to the asynchronous nature of my thumbnail creation.
Am I missing something? Is this possible to reduce this to one FineUploader call?
2) This code uploads the thumbnails as a base64 encoded string. I would like to upload the thumbnail as an image (or as a blob ?). Perhaps by following this recipe of Jeremy Banks. Would that work with FineUploader?
3) Are there other options/methods of FineUploader that I have missed but I should be using?
Any help is, as always, greatly appreciated.
So, it is already trivial to upload the original image. Fine Uploader takes care of that for you. If I understand correctly, you want to also upload a scaled version of the image (which you have already generated). I suggest you take the image you have drawn onto the canvas and convert it to a Blob. Then, you can submit this Blob directly to Fine Uploader, where it will upload it for you.
For example, change the value of uploadThumbnail to this:
function uploadThumbnail(thumbnailDataUri, id, filename) {
var imageBlob = getImageBlob(thumbnailDataUri, "image/jpeg"),
blobData = {
name: filename,
blob: imageBlob
};
// This will instruct Fine Uploader to upload the scaled image
uploader.addBlobs(blobData);
}
function getImageBlob(dataUri, type) {
var binary = atob(dataUri.split(',')[1]),
array = [];
for(var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
return new Blob([new Uint8Array(array)], {type: type});
}
Note: the getImageBlob function was adapted from this Stack Overflow answer. If this works for you, be sure to upvote the answer I've linked to.
Server-side note
A Blob is pretty much a File without a name property. Your server-side code will handle the upload of a Blob pretty much the same way as it does a File or form submit containing a <input type="file"> form field. The only noticeable difference to your server will be the filename parameter value in the Content-Disposition header of the multipart boundary containing the file. To put it another way, your server may think the image is named "blob" or perhaps some other generic name, due to the way most browsers generate multipart encoded requests that contain Blob objects. Fine Uploader should be able to get around that by explicitly specifying a file name for the browser to include in blob's Content-Disposition header, but this ability does not have wide browser support. Fine Uploader gets around this limitation, to some degree, by including a "qqfilename" parameter with the request containing the actual name of the Blob.
Future native support for thumbnail generation & scaling
The plan is to add native support for thumbnail previews to Fine Uploader. This is covered in feature requests #868 and #896. There are other related feature requests open, such as image rotation and validation related to images. These features and other image-related features will likely be added to Fine Uploader in the very near future. Be sure to comment on the existing feature requests or add additional requests if you'd like.
As of version 4.4 of FineUploader, as Ray Nicholus pointed out would eventually happen, this functionality has been baked into their framework.
Here is an example of setting the upload sizes when creating a FineUploader instance:
var uploader = new qq.FineUploader({
...
scaling: {
sizes: [
{name: "small", maxSize: 100},
{name: "medium", maxSize: 300}
]
}
});
See their page on uploading scaled images.