I draw my webcam stream on a canvas. I will apply my prediction rectangle bounding box on this canvas at each frame. Before starting prediction the canvas stream is prefect but as soon as predictions start the canvas starts to blink; I mean the whole canvas shows up and disappear, shows up and disappear very fast. do U have any idea how could I make it right? thanks a lot.
my prediction function looks like this:
let isPredicting = false;
async function predict() {
ui.isPredicting();
while (isPredicting) {
// Capture the frame from the webcam.
let video = document.getElementById('predwebcam');
let width=224,
height=224,
frameRate=10;
navigator.mediaDevices.getUserMedia({
video: {
width: width,
height:height,
frameRate:frameRate
}
}
).then(function(stream) {
video.srcObject = stream;
video.onloadedmetadata = function(e) {
video.play();
video.addEventListener('play', function() {
}, false);
};
}).catch(function(err) {
console.log('failed to load webcam')
});
var canvas = document.getElementById('predcanvas');
canvas.width = 500;
canvas.height = 500;
var context = canvas.getContext('2d');
/*
context.drawImage(video, 0, 0, canvas.width, canvas.height);
*/
// video 'play' event listener
video.addEventListener('play', function() {
//context.drawImage(this, 0, 0, canvas.width, canvas.height);
//draw(video, canvas, context, frameRate);
}, false);
function draw(video, canvas, context, frameRate) {
context.drawImage(video, 0, 0, canvas.width, canvas.height);
setTimeout(draw, 1/frameRate, video, canvas, context, frameRate);
}
//draw(video, canvas, context, frameRate);
const img = await getImage();
// Make a prediction through mobilenet, getting the internal activation of
// the mobilenet model, i.e., "embeddings" of the input images.
//const embeddings = truncatedMobileNet.predict(img);
console.log('starting to predict')
// Make a prediction through our newly-trained model using the embeddings
// from mobilenet as input.
const predictions = model.predict(img);
const values = predictions.dataSync();
const arr = Array.from(values);
console.log(arr)
let u = document.getElementById("1label").value;
dict.add(classindex1, u);
if (arr[0]>= 0){
arr[0]=1;
}
var Id = arr[0];
let ClassID = dict.findAt(Id);
//console.log(ClassID)
let startX = arr[1];
let startY = arr[2];
let w = arr[3];
let h = arr[4];
// Draw the bounding box.
context.strokeStyle = 'red';
//context.shadowColor = '#d53';
//context.shadowBlur = 20;
context.lineJoin = 'bevel';
context.lineWidth = 10;
context.strokeStyle = '#38f';
context.strokeRect(startX, startY, w, h);
// Draw the label background.
const font = "24px helvetica";
context.font = font;
context.textBaseline = "top";
context.fillStyle = "#2fff00";
const textWidth = context.measureText(ClassID).width;
const textHeight = parseInt(font, 10);
// draw top left rectangle
context.fillRect(startX, startY, textWidth + 10, textHeight + 10);
// draw bottom left rectangle
//context.fillRect(startX, startY + h - textHeight, textWidth + 15, textHeight + 10);
// Draw the text last to ensure it's on top.
context.fillStyle = "#000000";
context.fillText(ClassID, startX, startY);
//context.fillText(prediction.score.toFixed(2), startX, startY + h - textHeight);
// Returns the index with the maximum probability. This number corresponds
// to the class the model thinks is the most probable given the input.
//const predictedClass = predictions.as1D().argMax();
//const classId = (await predictedClass.data())[0];
img.dispose();
//ui.predictClass(metaClasses[classId]);
await tf.nextFrame();
}
ui.donePredicting();
}
Related
I am working on a project where user upload structural diagram(engineering diagram). When I user double click on the intended location on canvas the speech to text engine turns on and listen for user comments and then it draw a small circle with different colors and fill text (count) on that location. I am saving comments, counts, coordinates of arc and other things in react state and displaying the list in a component with edit and delete button. When user press the delete button. comment and other property gets deleted from the state.
I want to remove the drawn arc from the canvas. How can I do it?
I have tried clearRect. But it is not working in this case.
Please let me know.
componentDidMount() {
const img = this.refs.image;
const canvas = this.refs.canvas;
const ctx = canvas.getContext('2d');
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;
img.onload = () => {
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
ctx.font = "40px Courier";
ctx.fillText('Drawing 1', 200, 100);
}
}
drawCircle(x, y, getcolor) {
const canvas = this.refs.canvas;
const ctx = canvas.getContext("2d");
ctx.beginPath();
ctx.arc(x, y, 8, 0, Math.PI * 2, false);
ctx.strokeStyle = "black";
ctx.stroke();
ctx.fillStyle = getcolor;
ctx.fill();
ctx.closePath();
// Text
ctx.beginPath();
ctx.font = "20px Arial"
ctx.fillStyle = "#00538a";
ctx.textAlign = "center";
ctx.fillText(this.state.count, x , y - 20);
ctx.fill();
ctx.closePath();
}
remove(id) {
this.setState({
comments: this.state.comments.filter(c => c.id !== id)
});
const canvas = this.refs.canvas;
const ctx = canvas.getContext('2d');
const arc = this.state.comments.filter(c => c.id === id);
let x = arc[0].coordinates.x;
let y = arc[0].coordinates.y
console.log("TCL: Drawing -> remove -> arc", arc[0].coordinates);
ctx.beginPath();
ctx.arc(x, y, 8, 0, Math.PI * 2, false);
ctx.clip();
ctx.clearRect(x-8, y-8, 16,16);
}
Thanks
Meet
As I mentioned in my comments the way you're trying to remove a circle from the canvas ain't gonna work.
If you call clearRect() on the canvas, it will essentially overwrite the target area including your original background image.
Instead you need to keep track of the circles - more precisely the position at which those should be drawn - using an array.
If you click the canvas -> add a circle element to an array -> clear the canvas -> draw the diagram again -> loop over the array to draw the circles on top
If you click the remove button of a circle -> search the array for this particular circle -> remove it from the array -> clear the canvas -> draw the diagram again -> loop over the array to draw the circles on top
Here's an example to illustrate what I'm talking about:
var comments = new Array();
var canvas = document.createElement("canvas");
canvas.style="float:left;"
canvas.width = 400;
canvas.height = 200;
document.body.appendChild(canvas);
var ctx = canvas.getContext("2d");
function updateCanvas() {
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
ctx.font = "40px Courier";
ctx.fillText('Drawing 1', 200, 100);
for (var a = 0; a < comments.length; a++) {
ctx.beginPath();
ctx.arc(comments[a].x, comments[a].y, 8, 0, Math.PI * 2, false);
ctx.strokeStyle = "black";
ctx.stroke();
ctx.fillStyle = "black";
ctx.fill();
ctx.closePath();
}
}
var img = new Image();
img.onload = () => {
updateCanvas();
}
img.src = "https://picsum.photos/id/59/400/200";
function addCircle(e) {
var div = document.createElement("div");
div.innerHTML = "remove" + comments.length;
document.body.appendChild(div);
div.addEventListener("click", function(e) {
for (var a = 0; a < comments.length; a++) {
if (comments[a].div == e.target) {
comments.splice(a, 1);
break;
}
}
document.body.removeChild(e.target);
updateCanvas();
});
comments.push({
x: e.clientX,
y: e.clientY,
div: div
});
updateCanvas();
}
canvas.addEventListener("click", addCircle);
Everytime you click on the picture a 'remove' div will be created to the right of the canvas. if you click it, the associated circle will be removed.
I am using a face tracking library (tracking.js) to capture a video stream and place an image on top of the face.
The image is drawn on a canvas, which has the same width and height as the video therefore, the overlay.
I am trying to take a picture and video of the stream + canvas image, however O can only get a crude implementation of the stream and image that is distorted.
Here is a CodePen
const canvas = document.getElementById('canvas');
const context = canvas.getContext('2d');
const tracker = new tracking.ObjectTracker('face');
const flowerCrownButton = document.getElementById('flower-crown');
tracker.setInitialScale(1);
tracker.setStepSize(2.7);
tracker.setEdgesDensity(.2);
const img = document.createElement("img");
img.setAttribute("id", "pic");
img.src = canvas.toDataURL();
let filterX = 0;
let filterY = 0;
let filterWidth = 0;
let filterHeight = 0;
function changePic(x, y, width, height, src) {
img.src = src;
filterX = x;
filterY = y;
filterWidth = width;
filterHeight = height;
}
function flowerCrown() {
changePic(0, -0.5, 1, 1, 'https://s3-us-west-
2. amazonaws.com / s.cdpn.io / 450347 / flower - crown.png ')
}
flowerCrownButton.addEventListener('click', flowerCrown);
//listen for track events
tracker.on('track', function(event) {
//if (event.data.length === 0) {
//alert("No objects were detected in this frame.");
//} else {
context.clearRect(0, 0, canvas.width, canvas.height)
event.data.forEach(rect => {
context.drawImage(img, rect.x + (filterX * rect.width),
rect.y + (filterY * rect.height),
rect.width * filterWidth,
rect.height * filterHeight
)
})
//}// end of else
});
//start tracking
tracking.track('#video', tracker, {
camera: true
})
const canvas2 = document.getElementById('canvas2');
const context2 = canvas2.getContext('2d');
const video = document.getElementById("video");
video.addEventListener("loadedmetadata", function() {
ratio = video.videoWidth / video.videoHeight;
w = video.videoWidth - 100;
h = parseInt(w / ratio, 10);
canvas2.width = w;
canvas2.height = h;
}, false);
function snap() {
context2.drawImage(video, 10, 5);
context2.drawImage(img, 10, 10)
}
}
Any ideas? I prefer to use the media recorder API and have tried it, but again could not get a stream or picture with the image filter overlay.
Thanks and please don't be snarky :)
As an exercise, I would like to fill the background of the browser window with pixels of random color: https://jsfiddle.net/j8fay7bs/3/
My question is how to retrieve the current filling of the background image and randomly change the pixels? Currently the image seems to be reverted to black.
// redraw canvas pixels on resize --------------------------
var render = function resize() {
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
context.canvas.width = window.innerWidth;
context.canvas.height = window.innerHeight;
var imageData = context.getImageData(0, 0, canvas.width, canvas.height);
var t0 = performance.now();
for (var i = 0; i < canvas.width*canvas.height; i++) {
if (Math.random() < 0.5) {
imageData.data[i*4] = Math.round(Math.random()*255);
imageData.data[i*4+1] = Math.round(Math.random()*255);
imageData.data[i*4+2] = Math.round(Math.random()*255);
imageData.data[i*4+3] = 255;
}
}
context.putImageData(imageData, 0, 0);
$('#fps').text(1000/(performance.now() - t0) + " fps");
}
window.onresize = render;
// rendering loop ---------------------------------------
(function loop() {
setInterval(render, 10);
})();
You may retrieve the last image data before u touch the canvas width and height,
once u touched them, the canvas will be reset:
var render = function resize() {
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
var oldImageData = context.getImageData(0, 0, canvas.width, canvas.height); // <----
context.canvas.width = window.innerWidth;
context.canvas.height = window.innerHeight;
var imageData = context.getImageData(0, 0, canvas.width, canvas.height);
updated fiddle
I made a small program that:
changes the mouse cursor inside the canvas to a black square
gives the black square a nice trail that fades away over time (the point of the program)
Here's the code:
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
canvas.style.cursor = 'none'; // remove regular cursor inside canvas
function getMousePos(canvas, e) {
var rect = canvas.getBoundingClientRect();
return {
x: e.clientX - rect.left,
y: e.clientY - rect.top
};
}
function fadeCanvas() {
ctx.save();
ctx.globalAlpha = 0.1; // the opacity (i.e. fade) being applied to the canvas on each function re-run
ctx.fillStyle = "#FFF";
ctx.fillRect(0, 0, canvas.width, canvas.height); // area being faded (whole canvas)
ctx.restore();
requestAnimationFrame(fadeCanvas); // animate at 60 fps
}
fadeCanvas();
function draw(e) {
var pos = getMousePos(canvas, e);
ctx.fillStyle = "black";
ctx.fillRect(pos.x, pos.y, 8, 8); // the new cursor
}
addEventListener('mousemove', draw, false);
Here's a live example: https://jsfiddle.net/L6j71crw/2/
Problem
However the trail does not fade away completely, and leaves a ghosting trail.
Q: How can I remove the ghosting trail?
I have tried using clearRect() in different ways, but it just clears the entire animation leaving nothing to display. At best it just removes the trail and only fades the square cursor alone, but it still doesn't make the cursor completely transparent when the fading process is completed. I have tried finding posts about it, but I found nothing that gave a definitive answer and—most importantly—no posts with a working example.
Any ideas?
Try having a list of positions, this won't leave a ghost trail!
my code:
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
var Positions = [];
var maxlength = 20;
canvas.style.cursor = 'none'; // remove regular cursor inside canvas
var V2 = function(x, y){this.x = x; this.y = y;};
function getMousePos(canvas, e) {
// ctx.clearRect(0, 0, canvas.width, canvas.height);
var rect = canvas.getBoundingClientRect();
return {
x: e.clientX - rect.left,
y: e.clientY - rect.top
};
}
function fadeCanvas() {
ctx.clearRect(0, 0, canvas.width, canvas.height);
for(var e = 0; e != Positions.length; e++)
{
ctx.fillStyle = ctx.fillStyle = "rgba(0, 0, 0, " + 1 / e + ")";
ctx.fillRect(Positions[e].x, Positions[e].y, 8, 8);
}
if(Positions.length > 1)
Positions.pop()
//ctx.save();
//ctx.globalAlpha = 0.5; // the opacity (i.e. fade) being applied to the canvas on each function re-run
//ctx.fillStyle = "#fff";
//ctx.fillRect(0, 0, canvas.width, canvas.height); // area being faded (whole canvas)
//ctx.restore();
requestAnimationFrame(fadeCanvas); // animate at 60 fps
}
fadeCanvas();
function draw(e) {
var pos = getMousePos(canvas, e);
Positions.unshift(new V2(pos.x, pos.y));
if(Positions.length > maxlength)
Positions.pop()
//ctx.fillStyle = "black";
//ctx.fillRect(pos.x, pos.y, 8, 8); // the new cursor
}
addEventListener('mousemove', draw, false);
JSFiddle: https://jsfiddle.net/L6j71crw/9/
Edit: made the cursor constant.
Say we have a canvas:
<canvas id="one" width="100" height="200"></canvas>
And on a button click the canvas gets rotated 90 degrees clockwise (around the center) and the dimensions of the canvas get also updated, so in a sense it looks like this afterwards:
<canvas id="one" width="200" height="100"></canvas>
Note that the id of the canvas is the same.
Imagine simply rotating an image clockwise without it being cropped or being padded.
Any suggestions before I do it the long way of creating a new canvas and rotating and copying pixel by pixel?
UPDATE sample code with suggestion from comments still not working:
function imageRotatecw90(){
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
var myImageData = context.getImageData(0,0, cw,ch);
context.save();
context.translate(cw / 2, ch / 2);
context.rotate(Math.PI/2);
context.putImageData(myImageData, 0, 0);
context.restore();
canvas.width=ch;
canvas.height=cw;
}
FiddleJS
Look at this DEMO.
To achieve the results seen in demo, I made use of canvas.toDataURL to cache the canvas into an image, then reset the canvas to their new dimensions, translate and rotate the context properly and finally draw the cached image back to modified canvas.
That way you easily rotate the canvas without need to redraw everything again. But because anti-aliasing methods used by browser, each time this operation is done you'll notice some blurriness in result. If you don't like this behavior the only solution I could figure out is to draw everything again, what is much more difficult to track.
Here follows the code:
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw = canvas.width;
var ch = canvas.height;
// Sample graphic
context.beginPath();
context.rect(10, 10, 20, 50);
context.fillStyle = 'yellow';
context.fill();
context.lineWidth = 7;
context.strokeStyle = 'black';
context.stroke();
// create button
var button = document.getElementById("rotate");
button.onclick = function () {
// rotate the canvas 90 degrees each time the button is pressed
rotate();
}
var myImageData, rotating = false;
var rotate = function () {
if (!rotating) {
rotating = true;
// store current data to an image
myImageData = new Image();
myImageData.src = canvas.toDataURL();
myImageData.onload = function () {
// reset the canvas with new dimensions
canvas.width = ch;
canvas.height = cw;
cw = canvas.width;
ch = canvas.height;
context.save();
// translate and rotate
context.translate(cw, ch / cw);
context.rotate(Math.PI / 2);
// draw the previows image, now rotated
context.drawImage(myImageData, 0, 0);
context.restore();
// clear the temporary image
myImageData = null;
rotating = false;
}
}
}
Rotation
Note it is not possible to rotate a single element.
ctx.save();
ctx.rotate(0.17);
// Clear the current drawings.
ctx.fillRect()
// draw your object
ctx.restore();
Width/height adjustment
The only way I ever found to properly deal with display ratios, screen sizes etc:
canvas.width = 20;// DO NOT USE PIXELS
canvas.height = 40; // AGAIN NO PIXELS
Notice I am intentionally not using canvas.style.width or canvas.style.height. Also for an adjustable canvas don't rely on CSS or media queries to do the transformations, they are a headache because of the pixel ratio differences. JavaScript automatically accounts for those.
Update
You also have to update the width and the height before you draw. Not sure what you are trying to achieve, but I guess this isn't a problem:
Demo here
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw = canvas.width;
var ch = canvas.height;
canvas.width = 200;
canvas.height = 400;
// Sample graphic
context.beginPath();
context.rect(10,10,20,50);
context.fillStyle = 'yellow';
context.fill();
context.lineWidth = 7;
context.strokeStyle = 'black';
context.stroke();
var myImageData = context.getImageData(0, 0, cw, ch);
context.save();
context.translate(cw / 2, ch / 2);
context.putImageData(myImageData, 0, 0);
context.rotate(0.20);
If you want to rotate an image by 90 degrees this might be helpful:
export const rotateBase64Image = async (base64data: string) => {
const image = new Image();
image.src = base64data;
return new Promise<string>((resolve, reject) => {
image.onload = function () {
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
if (!ctx) throw new Error("cannnot get context '2d'");
canvas.width = image.height;
canvas.height = image.width;
ctx.setTransform(0, 1, -1, 0, canvas.width, 0); // overwrite existing transform
ctx!.drawImage(image, 0, 0);
canvas.toBlob((blob) => {
if (!blob) {
return reject("Canvas is empty");
}
const fileUrl = window.URL.createObjectURL(blob);
resolve(fileUrl);
}, "image/jpeg");
};
});
};
If you don't have image in base64 format you can do it like this:
const handleRotate = async () => {
const res = await fetch(link);
const blob = await res.blob();
const b64: string = await blobToB64(blob);
const rotatedImage = await rotateBase64Image(b64)
setLink(rotatedImage);
}
Here is my blobTob64 function:
export const blobToB64 = async (blob) => {
return new Promise((resolve, _) => {
const reader = new FileReader();
reader.onloadend = () => resolve(reader.result);
reader.readAsDataURL(blob);
});
};