I'm trying to use images off the internet to try and train my network. I'm using an Image() object to create the images and pass them to tensorflow. According to my knowledge, Image() returns a HTMLImageElement, however, I'm still getting the following error:
Error: pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was Image
Below is the code I'm running:
const tf = require("#tensorflow/tfjs");
require("#tensorflow/tfjs-node")
const mobilenetModule = require("#tensorflow-models/mobilenet");
const knnClassifier = require("#tensorflow-models/knn-classifier");
const { Image } = require("canvas");
const classifier = knnClassifier.create();
const urls = ["https://upload.wikimedia.org/wikipedia/commons/thumb/7/70/Solid_white.svg/2048px-Solid_white.svg.png", "https://stone.co.nz/wp-content/uploads/2020/06/Iconic-Black.jpg", "https://media.tarkett-image.com/large/TH_25094225_25187225_001.jpg"];
async function start() {
const mobilenet = await mobilenetModule.load();
const pic0 = new Image();
pic0.src = urls[0];
pic0.onload = () => {
const img0 = tf.browser.fromPixels(pic0);
const logits0 = mobilenet.infer(img0, true);
classifier.addExample(logits0, 0);
}
const pic1 = new Image();
pic1.src = urls[1];
pic1.onload = () => {
const img1 = tf.browser.fromPixels(pic1);
const logits1 = mobilenet.infer(img1, true);
classifier.addExample(logits1, 1);
}
const checkPic = new Image();
checkPic.src = urls[2];
checkPic.onload = () => {
const x = tf.browser.fromPixels(checkPic);
const xlogits = mobilenet.infer(x, true);
const p = classifier.predictClass(xlogits);
console.log(p);
}
}
start();
Please note that I am a js/nodejs newbie 😊
Seems like an oversight by TFJS team so Image type is not recognized.
do this instead:
const pic0 = document.createElement('image')
This creates HTMLImageElement which is "almost" the same as Image element, but has a different signature that TF expects
And best to report on TFJS Git as an issue, it should be easy to resolve
Related
I am trying to create multiple blob images, and show them for a video timeline UI. To create the images, I take screenshots of the video for some certain time frames, and create img elements, then add img.src as blob images' url. The problem is blob images does not show up at UI, although I can see them on console. Any ideas?
Code snippet from creation of blob images:
// draw the video frame to canvas
const ctx = canvas.getContext("2d");
ctx.drawImage(videoPlayer, 0, 0, canvas.width, canvas.height);
ctx.canvas.toBlob(
blob => {
resolve(blob);
},
"image/jpeg",
0.75 /* quality */
);
Code Snippet from creation of images:
let img = '';
for (let i = 0; i < count / 3; i++) {
const cover = await GetVideoCover(item.video_url, i);
var url = URL.createObjectURL(cover);
var image = new Image();
image.src = url;
img += "<img src='"+image.src+"'>";
console.log(img)
}
return img;
The console log from console.log(img)line:
actual html img elements:
When I manually update the source as below, it works, but I did not understand why it does not get the source by itself. Any idea would be appreciated.
Try registering an onload event listener on the image element and then handle all code that depends on that image in the event listener, like so:
for (let i = 0; i < count / 3; i++) {
const cover = await GetVideoCover(item.video_url, i);
var url = URL.createObjectURL(cover);
var image = new Image();
image.src = url;
image.addEventListener('load', (event) => {
// perform some magic here e.g append to DOM or draw the image to canvas
}
}
Thank you for the contributions, the answers did not work but they helped me in the process. I solved the problem today and wanted to share it here.
I should have mentioned before as the created images are fed to vis.js.
When I experimented more, I saw vis.js does not append the created objects, but it changes the innerHTML of the timeline. (vis.js might not support blob images as src of img tag)
Then, I decided to give a shot to append element. To do that, I created a new div element, and appended the created images to that div.
I also added an additional promise level to the blob image creation process, since some files were missing.
Code snippet from newly added promise level:
const blobToImage = (itemVideoURL,captureSecond) => {
return new Promise(async (resolve,reject)=> {
let cover = await GetVideoCover(itemVideoURL,captureSecond);
const url = URL.createObjectURL(cover)
let img = new Image()
img.onload = () => {
URL.revokeObjectURL(url)
resolve(img)
}
img.src = url
img.height = '75'
})
}
Code snippet from new function:
let divElement = document.createElement('div');
for (let i = 0; i < count * 3; i++) {
let newImage = await blobToImage(item.video_url,i+1)
divElement.appendChild(newImage)
}
return divElement;
It magically worked.
You should use a single image rather than multiple images.
Just concatanate all images together :
let imgEle1 = document.querySelectorAll(".image")[0];
let imgEle2 = document.querySelectorAll(".image")[1];
let resEle = document.querySelector(".result");
var context = resEle.getContext("2d");
let BtnEle = document.querySelector(".Btn");
BtnEle.addEventListener("click", () => {
resEle.width = imgEle1.width;
resEle.height = imgEle1.height;
context.globalAlpha = 1.0;
context.drawImage(imgEle1, 0, 0);
context.globalAlpha = 0.5;
context.drawImage(imgEle2, 0, 0);
});
I have something i searched a lot about the solution but i don't find anything. Also i tried a lot of things but also no thing work with me. I hope to find something here.
I build a website, in some stages i allow the user to convert his images from any type to WEBP. So I made I constructor function to do this proccess:
async function loadImage(url) {
return await new Promise((resolve, reject) => {
let img = new Image();
img.addEventListener('load', e => resolve(img));
img.addEventListener('error', () => {
reject(new Error(`Failed to load image's URL: ${url}`));
});
img.src = url;
});
}
async function convert(image, options) {
let newImage = new Image();
let canvas = document.createElement('canvas');
let ctx = canvas.getContext('2d');
return await new Promise((resolve) => {
canvas.width = image.width;
canvas.height = image.height;
ctx.drawImage(image, 0, 0, image.width, image.height);
// convert canvas to webp image
newImage.src = canvas.toDataURL(`image/${options.to}`, 1);
resolve(newImage)
});
}
async function Converter(userImage, options) {
let src = URL.createObjectURL(userImage);
let image = new Image();
image.src = src;
image.style.width = "auto";
image.style.height = "auto";
let t;
await loadImage(src).then(img => {
return convert(img, options)
}).then(newImage => {
t = newImage;
})
return t;
}
export default Converter;
I allow him to load the images and converting it with Javascript without upload the images to server.
I am using this function in another js file like this:
formInputs.addEventListener('change', e => {
if (formInputs.files.length > 0) {
updateFilatorTable(filatorInput.files);
}
})
As you see i am using promises and async/wait, for waiting images to load and convert. but the problem also here in await. how can i show the user Progress bar when the image loading and after that showing another progress bar when the image converting, with real percent for example: 25% converting.
Every thing working good but the problem is how to show progress bar when i load the image and when i convert it?
can i make something like that with promise and await?
I have two functions,
One function converts images from whatever format to canvas, then to WEBP image format.
The other function receives the image and console.log the image.
The problem is that when I log the result of the image in the saveToBackend function, I get a result of undefined, but when I console.log the converted image in the convertImage function, I get the image. Please how can i solve the issue?
Below are the functions in my NUXT app
convertImage(file) {
// convert image
let src = URL.createObjectURL(file)
let canvas = document.createElement('canvas');
let ctx = canvas.getContext('2d')
let userImage = new Image();
userImage.src = src
userImage.onload = function() {
canvas.width = userImage.width;
canvas.height = userImage.height;
ctx.drawImage(userImage, 0, 0);
let webpImage = canvas.toDataURL("image/webp");
console.log(webpImage); // with this i get the image in the console
return webpImage
}
},
async saveToBackend(file, result) {
// convert images to webp
let webpFile = await this.convertImage(file)
console.log(webpFile); // with this i get undefined in the console
}
You can't return webpImage from inside the onload callback. It simply will not wait for onload to execute. You can instead create a Promise that resolves the value of webpImage when onload completes:
convertImage(file) {
return new Promise((resolve) => {
// convert image
let src = URL.createObjectURL(file);
let canvas = document.createElement("canvas");
let ctx = canvas.getContext("2d");
let userImage = new Image();
userImage.src = src;
userImage.onload = function() {
canvas.width = userImage.width;
canvas.height = userImage.height;
ctx.drawImage(userImage, 0, 0);
let webpImage = canvas.toDataURL("image/webp");
console.log(webpImage); // with this i get the image in the console
// resolve promise so that saveToBackend can await it
return resolve(webpImage);
};
});
}
Hopefully that helps!
i want to draw image next to another in this same div.
Generaly drawing works fine, but i have problem with placing one picture next to another.
Every time i draw image it is placend in this same div.
My HTML is:
<div id="thumbnailImages">
<canvas
id="thumbnailImage"
/>
</div>
And code:
function thumbnail(video) {
const canvasThumbnail = document.getElementById('thumbnailImage')
const cth = canvasThumbnail.getContext('2d')
const data = video.slice(8)
imgToHTML(data, e => {
const drawingThumbnail = new Image()
drawingThumbnail.src = e.target.result
drawingThumbnail.onload = function () {
canvasThumbnail.width = 128
canvasThumbnail.height = 128
cth.drawImage(drawingThumbnail, 0, 0)
document.getElementById('thumbnails').appendChild(canvasThumbnail)
}
})
}
I would like to have many 'canvas' in 'thumbnailImages' id.
Can you help me?
The issue is that you are reusing the same canvas over and over again. You need to create a new canvas to put next to it:
function thumbnail(video) {
const canvasThumbnail = document.createElement('canvas');
const cth = canvasThumbnail.getContext('2d')
const data = video.slice(8)
imgToHTML(data, e => {
const drawingThumbnail = new Image()
drawingThumbnail.src = e.target.result
drawingThumbnail.onload = function () {
canvasThumbnail.width = 128
canvasThumbnail.height = 128
cth.drawImage(drawingThumbnail, 0, 0)
document.getElementById('thumbnailImages').appendChild(canvasThumbnail)
}
})
}
I am trying to generate a pdf report including current map screenshot of ESRI map using JavaScript. Here is my code.
var getCurrentMapScreenShot = function (success, error) {
esriLoader.Config.defaults.io.proxyUrl = myAppSettingsModel.SettingsModel.MapSettings.AGSProxyURL;
esriLoader.Config.defaults.io.alwaysUseProxy = true;
var printTask = new esriLoader.PrintTask("myexportUrl");
var template = new esriLoader.PrintTemplate();
template.exportOptions = {
width: 600,
height: 600,
dpi: 96
};
template.format = "image/png";
template.layout = "MAP_ONLY",
template.preserveScale = true;
template.layoutOptions = {
legendLayers: [], // empty array means no legend
scalebarUnit: "Miles"
};
var params = new esriLoader.PrintParameters();
params.map = map;
params.template = template;
printTask.execute(params, success, error);
}
This function will give you an event that contains a url , Then I am passing this url to get map base64 data.
Here is the function calling.
map.GetCurrentMapScreenShot(function (event) {
var mapScreenShotURL = event.url;
Factory.GetBase64ForImgUrl(mapScreenShotURL,
function (mapImageBase64Encoded) {});
and here is the function that converts url to base64image
function getBase64ForImgUrl(url, callback, outputFormat) {
console.log("#################### Summary Report Image " + url);
var canvas = document.createElement('CANVAS');
var ctx = canvas.getContext('2d');
var img = new Image;
img.crossOrigin = 'Anonymous';
img.onload = function () {
canvas.height = img.height;
canvas.width = img.width;
ctx.drawImage(img, 0, 0);
var dataURL = canvas.toDataURL(outputFormat || 'image/png');
callback.call(this, dataURL);
// Clean up
canvas = null;
};
img.src = url;
}
I am getting base64 image data of the map , But the problem is that, I am getting blurred image of the map with no feature layers and no icons..And also map contains some junk texts.
Thanks
I resolved the issue almost, Now I am getting the map image that contains map Icons and layer graphics, But unfortunately still getting some junk texts.
I have changed the code as follows.
template.format = "JPG";
template.layout = "A4 Landscape",
template.preserveScale = false;
template.layoutOptions = {
"legendLayers": [], // empty array means no legend
"scalebarUnit": "Miles"
}