Trying to achieve an effect of seeking through a video when the page is scrolled. This has been achieved by exporting all frames of the video to JPEG images, pre-loading the images, and rendering them to a canvas. However, this approach uses a lot of bandwidth and takes a long time to load on slow networks.
Trying to achieve the same effect by rendering a video to a canvas does not play as smoothly as the image-based approach.
Here is a working demo with the video-based approach:
https://codesandbox.io/s/infallible-chaum-grvi0r?file=/index.html
Since HTMLMediaElement.fastSeek() doesn't have widespread browser coverage, how can one achieve a realtime playback rate of 30-60 FPS?
Here is the relevant code for the effect (see CSB link above for the full code):
const video = document.querySelector("video");
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
(function tick() {
requestAnimationFrame(tick);
const { scrollHeight, clientHeight, scrollTop } = document.body;
const maxScroll = scrollHeight - clientHeight;
const scrollProgress = scrollTop / maxScroll;
canvas.width = document.body.clientWidth;
canvas.height = document.body.clientHeight;
// This is the line that causes the issue
video.currentTime = video.duration * scrollProgress;
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
})();
The problem is due to specifics of drawImage() as noted here, it should be called only after firing of seeked event. The code below solves the problem by using a sentinel value (seeked = false) which will block the execution of drawImage() until the next seeked event:
const video = document.querySelector("video");
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
let seeked;
(function tick() {
requestAnimationFrame(tick);
if (seeked) {
seeked = false;
const { scrollHeight, clientHeight, scrollTop } = document.body;
const maxScroll = scrollHeight - clientHeight;
const scrollProgress = scrollTop / Math.max(maxScroll, 1);
canvas.width = document.body.clientWidth;
canvas.height = document.body.clientHeight;
// This is the line that causes the issue
video.currentTime = video.duration * scrollProgress;
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
}
})();
video.addEventListener('seeked', () => {
seeked = true;
});
video.currentTime = .001;
Related
im trying to implement a scratch card in Reactjs, when the component mount I'm able to load an image on the canvas to scratch it, the issue is when I click to reload the scratch card to play again I'm not able to reload an image, only a background color. is there a way to reload the image
componentDidMount() {
const canvas = this.canvas;
canvas.width = canvas.parentElement.offsetWidth;
canvas.height = canvas.parentElement.offsetHeight;
canvas.addEventListener("mousedown", this.touchStart);
canvas.addEventListener("touchstart", this.touchStart);
canvas.addEventListener("mousemove", this.touchMove);
canvas.addEventListener("touchmove", this.touchMove);
canvas.addEventListener("mouseup", this.touchEnd);
canvas.addEventListener("touchend", this.touchEnd);
this.ctx = canvas.getContext("2d");
this.brush = new Image();
this.brush.src = require("../../../assets/images/brush.png");
this.cover = new Image();
this.cover.src = require("../../../assets/images/ae-unscratched.png");
this.cover.onload = () =>
this.ctx.drawImage(this.cover, 0, 0, canvas.width, canvas.height);
}
reloadCard() {
this.ctx = this.canvas.getContext("2d");
this.ctx.globalCompositeOperation = "destination-over";
this.ctx.fillStyle = "#ffecc0";
let width = 310;
let height = 350;
this.ctx.fillRect(0, 0, width, height);
this.ctx.save();
}
I tried to initialize a new image like on componentdidmount it didn't work, any idea how can I reload the image?
You could try and add a cache killer query string attribute to your src value:
this.cover.src = require(`../../../assets/images/ae-unscratched.png?${Date.now()}`);
I want to add a drawing canvas on top of a website that stretches for the whole height of a website.
I specified
canvas.height = window.innerHeight;
canvas.width = window.innerWidth;
and it seems to work for the viewport, but when I scroll, I cannot draw bellow it.
I assume I need to update the canvas.height every time I scroll down, but I cannot seem to find a solution.
Here is the code:
window.addEventListener ("load", () => {
const canvas = document.querySelector ("#canvas");
const ctx = canvas.getContext ("2d");
//Resizing
canvas.height = window.innerHeight;
canvas.width = window.innerWidth;
//Variables
let painting = false;
function startPosition (e){
painting = true;
draw (e);
}
function finishedPosition (){
painting = false;
ctx.beginPath();
}
function draw (e){
if (!painting) return;
ctx.lineWidth = 1;
ctx.lineCap = "round";
ctx.lineTo(e.clientX, e.clientY);
ctx.stroke();
ctx.beginPath();
ctx.moveTo(e.clientX, e.clientY);
}
//Event listeners
canvas.addEventListener ("mousedown", startPosition);
canvas.addEventListener ("mouseup", finishedPosition);
canvas.addEventListener ("mousemove", draw);
});
I would appreciate any help on this!
This appears to be due to the use of window.innerHeight. Try using document.querySelector('body').clientHeight as that should give you the height of the entire body of the page.
Also, use MouseEvent.pageX and .pageY because .clientY will not update when the user scrolls the page.
I have the webcam stream on a canvas and am having trouble displaying a rectangle on top. The webcam stream works.
The issue is that the rectangle does not appear, and sometimes, there is a quick flash of the rectangle (and pretty blurry too...)
webcam to canvas setup:
const canvas = document.querySelector('.photo');
const ctx = canvas.getContext('2d');
// gets the video stream from the user's webcam
function getVideo() {
navigator.mediaDevices.getUserMedia({
video: true,
audio: false
}).then(localMediaStream => {
video.src = window.URL.createObjectURL(localMediaStream);
video.play();
}).catch(err => {
console.error('oh no', err);
});
}
// Applies the webcam stream to a canvas
function paintToCanvas() {
const width = video.videoWidth;
const height = video.videoHeight;
canvas.width = width;
canvas.height = height;
return setInterval(() => {
ctx.drawImage(video, 0, 0, width, height);
}, 16);
}
draw rectangle on top:
function drawRectangle() {
ctx.rect(150, 150, 50, 30);
ctx.lineWidth = 2;
ctx.strokeStyle = '#FF0000';
ctx.stroke();
}
drawRectangle();
I have tried different strokes, linewidths, and areas but same results.
What am I doing wrong? why it sometimes flashing in and out (upon page load) and not staying?
You need to use requestAnimationFrame instead of setInterval
That way your drawing will happen when the browser updates it's viewport.
Drawing a shape in Canvas on repeat
function draw() {
requestAnimationFrame(draw);
ctx.drawImage(video, 0, 0, width, height);
}
function paintToCanvas() {
const width = video.videoWidth;
const height = video.videoHeight;
canvas.width = width;
canvas.height = height;
requestAnimationFrame(draw);
}
Here is what I added to fix it:
let startTime;
const interval=1;
requestAnimationFrame(animate);
function animate(t){
requestAnimationFrame(animate);
if(!startTime){startTime=t;}
if(t-startTime<interval){return;}
startTime=t;
drawRectangle();
}
every milli seconds (my choice) the shape is drawn. tested and works.
I stopped calling drawRectangle outside of a function and there is no more flash upon page load either.
As an exercise, I would like to fill the background of the browser window with pixels of random color: https://jsfiddle.net/j8fay7bs/3/
My question is how to retrieve the current filling of the background image and randomly change the pixels? Currently the image seems to be reverted to black.
// redraw canvas pixels on resize --------------------------
var render = function resize() {
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
context.canvas.width = window.innerWidth;
context.canvas.height = window.innerHeight;
var imageData = context.getImageData(0, 0, canvas.width, canvas.height);
var t0 = performance.now();
for (var i = 0; i < canvas.width*canvas.height; i++) {
if (Math.random() < 0.5) {
imageData.data[i*4] = Math.round(Math.random()*255);
imageData.data[i*4+1] = Math.round(Math.random()*255);
imageData.data[i*4+2] = Math.round(Math.random()*255);
imageData.data[i*4+3] = 255;
}
}
context.putImageData(imageData, 0, 0);
$('#fps').text(1000/(performance.now() - t0) + " fps");
}
window.onresize = render;
// rendering loop ---------------------------------------
(function loop() {
setInterval(render, 10);
})();
You may retrieve the last image data before u touch the canvas width and height,
once u touched them, the canvas will be reset:
var render = function resize() {
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
var oldImageData = context.getImageData(0, 0, canvas.width, canvas.height); // <----
context.canvas.width = window.innerWidth;
context.canvas.height = window.innerHeight;
var imageData = context.getImageData(0, 0, canvas.width, canvas.height);
updated fiddle
Say we have a canvas:
<canvas id="one" width="100" height="200"></canvas>
And on a button click the canvas gets rotated 90 degrees clockwise (around the center) and the dimensions of the canvas get also updated, so in a sense it looks like this afterwards:
<canvas id="one" width="200" height="100"></canvas>
Note that the id of the canvas is the same.
Imagine simply rotating an image clockwise without it being cropped or being padded.
Any suggestions before I do it the long way of creating a new canvas and rotating and copying pixel by pixel?
UPDATE sample code with suggestion from comments still not working:
function imageRotatecw90(){
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
var myImageData = context.getImageData(0,0, cw,ch);
context.save();
context.translate(cw / 2, ch / 2);
context.rotate(Math.PI/2);
context.putImageData(myImageData, 0, 0);
context.restore();
canvas.width=ch;
canvas.height=cw;
}
FiddleJS
Look at this DEMO.
To achieve the results seen in demo, I made use of canvas.toDataURL to cache the canvas into an image, then reset the canvas to their new dimensions, translate and rotate the context properly and finally draw the cached image back to modified canvas.
That way you easily rotate the canvas without need to redraw everything again. But because anti-aliasing methods used by browser, each time this operation is done you'll notice some blurriness in result. If you don't like this behavior the only solution I could figure out is to draw everything again, what is much more difficult to track.
Here follows the code:
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw = canvas.width;
var ch = canvas.height;
// Sample graphic
context.beginPath();
context.rect(10, 10, 20, 50);
context.fillStyle = 'yellow';
context.fill();
context.lineWidth = 7;
context.strokeStyle = 'black';
context.stroke();
// create button
var button = document.getElementById("rotate");
button.onclick = function () {
// rotate the canvas 90 degrees each time the button is pressed
rotate();
}
var myImageData, rotating = false;
var rotate = function () {
if (!rotating) {
rotating = true;
// store current data to an image
myImageData = new Image();
myImageData.src = canvas.toDataURL();
myImageData.onload = function () {
// reset the canvas with new dimensions
canvas.width = ch;
canvas.height = cw;
cw = canvas.width;
ch = canvas.height;
context.save();
// translate and rotate
context.translate(cw, ch / cw);
context.rotate(Math.PI / 2);
// draw the previows image, now rotated
context.drawImage(myImageData, 0, 0);
context.restore();
// clear the temporary image
myImageData = null;
rotating = false;
}
}
}
Rotation
Note it is not possible to rotate a single element.
ctx.save();
ctx.rotate(0.17);
// Clear the current drawings.
ctx.fillRect()
// draw your object
ctx.restore();
Width/height adjustment
The only way I ever found to properly deal with display ratios, screen sizes etc:
canvas.width = 20;// DO NOT USE PIXELS
canvas.height = 40; // AGAIN NO PIXELS
Notice I am intentionally not using canvas.style.width or canvas.style.height. Also for an adjustable canvas don't rely on CSS or media queries to do the transformations, they are a headache because of the pixel ratio differences. JavaScript automatically accounts for those.
Update
You also have to update the width and the height before you draw. Not sure what you are trying to achieve, but I guess this isn't a problem:
Demo here
var canvas = document.getElementById("one");
var context = canvas.getContext("2d");
var cw = canvas.width;
var ch = canvas.height;
canvas.width = 200;
canvas.height = 400;
// Sample graphic
context.beginPath();
context.rect(10,10,20,50);
context.fillStyle = 'yellow';
context.fill();
context.lineWidth = 7;
context.strokeStyle = 'black';
context.stroke();
var myImageData = context.getImageData(0, 0, cw, ch);
context.save();
context.translate(cw / 2, ch / 2);
context.putImageData(myImageData, 0, 0);
context.rotate(0.20);
If you want to rotate an image by 90 degrees this might be helpful:
export const rotateBase64Image = async (base64data: string) => {
const image = new Image();
image.src = base64data;
return new Promise<string>((resolve, reject) => {
image.onload = function () {
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
if (!ctx) throw new Error("cannnot get context '2d'");
canvas.width = image.height;
canvas.height = image.width;
ctx.setTransform(0, 1, -1, 0, canvas.width, 0); // overwrite existing transform
ctx!.drawImage(image, 0, 0);
canvas.toBlob((blob) => {
if (!blob) {
return reject("Canvas is empty");
}
const fileUrl = window.URL.createObjectURL(blob);
resolve(fileUrl);
}, "image/jpeg");
};
});
};
If you don't have image in base64 format you can do it like this:
const handleRotate = async () => {
const res = await fetch(link);
const blob = await res.blob();
const b64: string = await blobToB64(blob);
const rotatedImage = await rotateBase64Image(b64)
setLink(rotatedImage);
}
Here is my blobTob64 function:
export const blobToB64 = async (blob) => {
return new Promise((resolve, _) => {
const reader = new FileReader();
reader.onloadend = () => resolve(reader.result);
reader.readAsDataURL(blob);
});
};