Preload content before display for smoother experience? - javascript

My website https://www.dimovski.nu/ starts off with a scroll-triggered animation, displaying and updating a sequence of images on scroll. I have compressed the images as much as possible to optimize load time, but it's a total of 541 images.
Is it possible to preload all the images somehow? Should I have a loading page before the trigger animation in order to avoid a choppy experience?
Here's the JS code:
const canvas = document.querySelector(".canvas");
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;
const context = canvas.getContext("2d");
const frameCount = 541;
const currentFrame = (index) => `./seq/${(index + 1).toString()}.jpg`;
const images = [];
let ball = {frame : 0};
for (let i = 0; i < frameCount; i++){
const img = new Image();
img.src = currentFrame(i);
images.push(img);
}
gsap.to(ball, {
frame: frameCount - 1,
snap: "frame",
ease: "none",
scrollTrigger: {
scrub: true,
pin: "canvas",
end: "500%",
},
onUpdate: render,
})
images[0].onload = render;
function render(){
context.canvas.width = images[0].width;
context.canvas.height = images[0].height;
context.clearRect(0, 0, canvas.width, canvas.height);
context.drawImage(images[ball.frame], 0, 0);
}
I have tried to find a solution of preloaders, but uncertain on which solution works best for my situation.

Based on the number of request required to fetch these resources and the total size I wouldn't recommend doing this. Especially not on your landing page.
You could potentially reduce number of requests by baking all of the frames into one single image as i.e. sprite sheet and show a loading spinner, but the size, without any further compression, would be similar.
Another option could be to compile the frames into a video/animation, with better compression, and have an embedded video player rendering the frames based on scroll in the borderless player.

Related

I want smooth animation on live server for image sequence scroll

I'm using canvas for this animation. The animation is working fine on localhost but on the live servers it's taking too much time.
This is because I'm using almost 3000 frames for this animation, all frames are important. How can I increase the loading speed on the live server?
I have attached the code. Please review it and help me if I'm wrong somewhere.
const html = document.documentElement;
const canvas = document.getElementById("hero-lightpass");
const context = canvas.getContext("2d");
const frameCount = 2999;
const currentFrame = index => (`compressed/${index.toString().padStart(9, '720_0000')}.jpg`)
const preloadImages = () => {
for (let i = 1; i < frameCount; i++) {
const img = new Image();
img.src = currentFrame(i);
}
};
const img = new Image()
img.src = currentFrame(1);
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;
img.onload = function() {
scaleToFill(this);
}
function scaleToFill(img) {
var scale = Math.max(canvas.width / img.width, canvas.height / img.height);
var x = (canvas.width / 2) - (img.width / 2) * scale;
var y = (canvas.height / 2) - (img.height / 2) * scale;
context.drawImage(img, x, y, img.width * scale, img.height * scale);
}
const updateImage = index => {
img.src = currentFrame(index);
context.drawImage(img, 0, 0);
}
window.addEventListener('scroll', () => {
const scrollTop = html.scrollTop;
const maxScrollTop = html.scrollHeight - window.innerHeight;
const scrollFraction = scrollTop / maxScrollTop;
const frameIndex = Math.min(
frameCount - 1,
Math.ceil(scrollFraction * frameCount)
);
requestAnimationFrame(() => updateImage(frameIndex + 1))
});
preloadImages()
<canvas id="hero-lightpass"></canvas>
There might be some edge cases where you still need individual images, and the suggestions in the comments (using video or Sprite sheets) will not do, if you can use one of those option you should, it will simplify a lot...
I'm going to focus on that edge case where we have a ton of images
Great examples how to do things like that are online maps:
Google maps (https://www.google.com/maps)
OpenStreetMap (https://www.openstreetmap.org/)
Those maps have a lot of images, but they do not download all at once, they download and draw only what is needed, in your case you can preload maybe 50 (you have to experiment to see what best) and download the 51, 52 ... as you are drawing the first you have preloaded.
Back to the maps; They use "small" map tiles to paint one big mosaic, here are a couple of tiles:
https://a.tile.openstreetmap.org/6/16/25.png
https://b.tile.openstreetmap.org/6/15/26.png
You can see they are coming from different servers a & b that is to speed up download, browsers have limits on how many images are downloaded from one server at a time, for more details see:
http://kb.mozillazine.org/Network.http.max-connections-per-server
In a nutshell:
Download and draw only what is needed
Use multiple servers to host your images

Strange 700ms delay when using context.drawImage();

I'm working on a small canvas animation that requires me to step through a large sprite sheet png so I'm getting a lot of mileage out of drawImage(). I've never had trouble in the past using it, but today I'm running into an odd blocking delay after firing drawImage.
My understanding is that drawImage is synchronous, but when I run this code drawImage fired! comes about 700ms before the image actually appears. It's worth noting it's 700ms in Chrome and 1100ms in Firefox.
window.addEventListener('load', e => {
console.log("page loaded");
let canvas = document.getElementById('pcb');
let context = canvas.getContext("2d");
let img = new Image();
img.onload = function() {
context.drawImage(
img,
800, 0,
800, 800,
0, 0,
800, 800
);
console.log("drawImage fired!");
};
img.src = "/i/sprite-comp.png";
});
In the larger context this code runs in a requestAnimationFrame loop and I only experience this delay during the first execution of drawImage.
I think this is related to the large size of my sprite sheet (28000 × 3200) # 600kb though the onload event seems to be firing correctly.
edit: Here's a printout of the time (ms) between rAF frames. I get this result consistently unless I remove the drawImage function.
That's because the load event only is a network event. It only tells that the browser has fetched the media, parsed the metadata, and has recognized it is a valid media file it can decode.
However, the rendering part may still not have been made when this event fires, and that's why you have a first rendering that takes so much time. (Though it used to be an FF only behavior..)
Because yes drawImage() is synchronous, It will thus make that decoding + rendering a synchrounous operation too. It's so true, that you can even use drawImage as a way to tell when an image really is ready..
Note that there is now a decode() method on the HTMLImageElement interface that will tell us exactly about this, in a non-blocking means, so it's better to use it when available, and to anyway perform warming rounds of all your functions off-screen before running an extensive graphic app.
But since your source image is a sprite-sheet, you might actually be more interested in the createImageBitmap() method, which will generate an ImageBitmap from your source image, optionally cut off. These ImageBitmaps are already decoded and can be drawn to the canvas with no delay. It should be your preferred way since it will also avoid that you draw the whole sprite-sheet every time. And for browsers that don't support this method, you can monkey patch it by returning an HTMLCanvasElement with the part of the image drawn on it:
if (typeof window.createImageBitmap !== "function") {
window.createImageBitmap = monkeyPatch;
}
var img = new Image();
img.crossOrigin = "anonymous";
img.src = "https://upload.wikimedia.org/wikipedia/commons/b/be/SpriteSheet.png";
img.onload = function() {
makeSprites()
.then(draw);
};
function makeSprites() {
var coords = [],
x, y;
for (y = 0; y < 3; y++) {
for (x = 0; x < 4; x++) {
coords.push([x * 132, y * 97, 132, 97]);
}
}
return Promise.all(coords.map(function(opts) {
return createImageBitmap.apply(window, [img].concat(opts));
})
);
}
function draw(sprites) {
var delay = 96;
var current = 0,
lastTime = performance.now(),
ctx = document.getElementById('canvas').getContext('2d');
anim();
function anim(t) {
requestAnimationFrame(anim);
if (t - lastTime < delay) return;
lastTime = t;
current = (current + 1) % sprites.length;
ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height)
ctx.drawImage(sprites[current], 0, 0);
}
}
function monkeyPatch(source, sx, sy, sw, sh) {
return Promise.resolve()
.then(drawImage);
function drawImage() {
var canvas = document.createElement('canvas');
canvas.width = sw || source.naturalWidth || source.videoWidth || source.width;
canvas.height = sh || source.naturalHeight || source.videoHeight || source.height;
canvas.getContext('2d').drawImage(source,
sx || 0, sy || 0, canvas.width, canvas.height,
0, 0, canvas.width, canvas.height
);
return canvas;
}
}
<canvas id="canvas" width="132" height="97"></canvas>

Show webcam / camera stream with delay - webrtc

I've made a simple setup, getting the webcam / phone camera stream and the passing it on , drawing on a html 2d canvas.
But ive been having trouble figuring out how to show the stream with a delay of few seconds. Kinda like a delay mirror.
I tried playing with ctx.globalAlpha = 0.005; but this gives me a ghosting effect rather than 'delaying' the stream.
Any idea how this can be achieved?
The snippet below doesnt work here probably because of security issues apparently but here's a pen:
https://codepen.io/farisk/pen/LvmGGQ
var width = 0, height = 0;
var canvas = document.createElement('canvas'),
ctx = canvas.getContext('2d');
document.body.appendChild(canvas);
var video = document.createElement('video'),
track;
video.setAttribute('autoplay',true);
window.vid = video;
function getWebcam(){
navigator.mediaDevices.getUserMedia({ video: true }).then(function(stream) {
var videoTracks = stream.getVideoTracks();
var newStream = new MediaStream(stream.getVideoTracks());
video.srcObject = newStream;
video.play();
track = stream.getTracks()[0];
}, function(e) {
console.error('Rejected!', e);
});
}
getWebcam();
var rotation = 0,
loopFrame,
centerX,
centerY,
twoPI = Math.PI * 2;
function loop(){
loopFrame = requestAnimationFrame(loop);
// ctx.globalAlpha = 0.005;
ctx.drawImage(video, 0, 0, width, height);
ctx.restore();
}
function startLoop(){
loopFrame = requestAnimationFrame(loop);
}
video.addEventListener('loadedmetadata',function(){
width = canvas.width = video.videoWidth;
height = canvas.height = video.videoHeight;
centerX = width / 2;
centerY = height / 2;
startLoop();
});
canvas.addEventListener('click',function(){
if ( track ) {
if ( track.stop ) { track.stop(); }
track = null;
} else {
getWebcam();
}
});
video,
canvas {
max-width: 100%;
height: auto;
}
The snippet below doesnt work here probably because of security issues apparently but here's a pen:
https://codepen.io/farisk/pen/LvmGGQ
You might want to consider storing the video data you get in an array of sorts. It might mean delaying the playback for n seconds at first.
Basically on frame 1, you store the video feed into an array, and draw nothing. This happened until frame 1000 (1 second). At that point start drawing based on the first element of the array.
Once you draw that frame, remove it from the array and add the new frame.

How to cache image rasterized

I'm doing one of them stop-shot-scroll-controlled-playback sites like Sony's Be Moved.
The problem that I'm facing, considering the stop-shot technique, is the time that it takes for image to be rasterized before browser draws it on screen. It takes a lot on mobile. Probably resizing the image takes most of the cpu, but I'm not sure. This is how I show the frames:
<div
style="
position: fixed;
top:0; right:0; bottom:0; left:0;
background-image: url(...);
background-position: center;
background-size: cover;
"
></div>
The question:
Is there a way to cache a rasterized version of an image? Maybe canvas supports this? That way, when I decide to show it on screen, it'll be ready.
Right now, this is the only way I know how to cache an image.
var image = new Image();
image.src = '...';
Ref comments - there is a way to pre-cache video frames. Each frame will use a full memory block for the bitmap (which in any case also is the case with preloaded image sequences).
Cache Process
Create an "off-line" video element
Set video source with preload set to auto
You need to know the frame rate (typical: 30 fps for USA/Japan, 25 fps for Europe), calculate a time delta based on this, ie. 1 / FPS.
Use the timeupdate event for every currentTime update as setting current time is asynchronous.
Chose an in-point in the video, cache (this can take a while due to the event cycle), store to a frame buffer using a canvas element for each frame. Then playback the buffer when and as needed (this also gives you the ability to play video backwards as shown below, a feature not yet supported in the browsers).
Example
This example will load a video from net, cache 90 (3 sec # 30 fps) frames to memory, then play back the sequence ping-pong in the window (the images you see are from the cache obviously):
var canvas = document.querySelector("canvas"),
ctx = canvas.getContext("2d"),
video = document.createElement("video"),
frames = [],
w = canvas.width, h = canvas.height;
video.addEventListener("canplay", cache);
video.preload = "auto";
video.src = "http://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4";
function cache() {
this.removeEventListener("canplay", cache); // remove to avoid recalls
var fps = 30, // assuming 30 FPS
delta = 1 / fps, // time delta
count = 0, // current cached frame
max = fps * 3, // 3 seconds
div = document.querySelector("div"); // just for info
this.addEventListener("timeupdate", cacheFrame); // time update is aync
this.currentTime = 19; // start with initial time
function cacheFrame() {
div.innerHTML = "Caching frame: " + count;
if (count++ < max) {
// create canvas for frame-buffer;
var canvas = document.createElement("canvas"),
ctx = canvas.getContext("2d");
canvas.width = this.videoWidth; // canvas size = video frame
canvas.height = this.videoHeight;
ctx.drawImage(video, 0, 0); // draw current frame
frames.push(canvas); // store frame
this.currentTime += delta; // update time, wait..
}
else {
this.removeEventListener("timeupdate", cacheFrame); // remove!!
play(); // play back cached sequence
}
}
}
// to demo the cached frames
function play() {
var current = 0, max = frames.length, dlt = 1,
div = document.querySelector("div"),
toggle = false,
mem = max * video.videoWidth * video.videoHeight * 4; // always RGBA
mem = (mem / 1024) / 1024; //mb
ctx.fillStyle = "red";
(function loop() {
toggle = !toggle; // toggle FPS to 30 FPS
requestAnimationFrame(loop);
if (toggle) {
div.innerHTML = "Playing frame: " + current +
" (raw mem: " + mem.toFixed(1) + " mb)";
ctx.drawImage(frames[current], 0, 0, w, h); // using frame-buffer
ctx.fillRect(0, 0, current/max * w, 3);
current += dlt;
if (!current || current === max-1) dlt = -dlt; // pong-pong
}
})();
}
html, body {width:100%;height:100%}
body {margin:0; overflow:hidden;background:#aaa}
div {font:bold 20px monospace;padding:12px;color:#000}
canvas {z-index:-1;position:fixed;left:0;top:0;width:100%;height:100%;min-height:400px}
<div>Pre-loading video... wait for it, wait for it...</div>
<canvas width=600 height=360></canvas>
Canvas drawing sources are image & video objects (and some other sources which aren't relevant now). So if your unwanted delay is occurring during the initial download and rendering, then canvas will take longer because the incoming image must first be rendered onto an image object and then again rendered onto the canvas--two steps instead of one.
Your answer is not in the canvas element, so you're back to the usual solution: lessen the quantity of image bits being downloaded by lowering the quality of your images (jpg with less quality).
You could also (as you've indicated), preload & cache all your images in new Images so they can be used immediately when needed. The usual cost applies: increased memory usage for the cached images and a delay at the start of your app while all required images are downloaded.

Recording speed javascript

I am currently creating a project that supports video recording through my website.
I create a canvas and then push the recorded frames to it. The problem is, when I play the video after its recorded, it plays too fast. A 10 second long video plays in like 2 seconds. I have checked the playbackRate is set to 1. I save the recording to a database and its speeded up there aswell, so it has nothing to do with the browsers videoplayer.
I am relative new to AngularJS and javascript so im sorry if I left something important out.
I have tried changing alot of the values back and forth but I cant seem to find the cause for the problem. Any ideas?
Here is the code for the video recording:
scope.startRecording = function () {
if (mediaStream) {
var video = $('.video-capture')[0];
var canvas = document.createElement('canvas');
canvas.height = video.videoHeight;
canvas.width = video.videoWidth;
ctx = canvas.getContext('2d');
var CANVAS_WIDTH = canvas.width;
var CANVAS_HEIGHT = canvas.height;
function drawVideoFrame(time) {
videoRecorder = requestAnimationFrame(drawVideoFrame);
ctx.drawImage(video, 0, 0, CANVAS_WIDTH, CANVAS_HEIGHT);
recordedFrames.push(canvas.toDataURL('image/webp', 1));
}
videoRecorder = requestAnimationFrame(drawVideoFrame); // Note: not using vendor prefixes!
scope.recording = true;
}
};
scope.stopRecording = function () {
cancelAnimationFrame(videoRecorder); // Note: not using vendor prefixes!
// 2nd param: framerate for the video file.
scope.video.files = Whammy.fromImageArray(recordedFrames, 1000 / 30);
recordedVideoBlob = Whammy.fromImageArray(recordedFrames, 1000 / 30);
scope.videoMode = 'viewRecording';
scope.recording = false;
};
I am guess the culprit is requestAnimationFrame, left on it's own, you cannot tell at what intervals it keeps calling the callback, it can be as high as 60fps.
also looking at your code, I cannot tell how you came to the conclusion that frame rate = 1000/30
my advice( at least for your case) would be to go with $interval,
you can do something like:
scope.frameRate = 10, videoInterval; // the amount I consider ideal for client-side video recording.
scope.startRecording = function () {
if (mediaStream) {
var video = $('.video-capture')[0];
var canvas = document.createElement('canvas');
canvas.height = video.videoHeight;
canvas.width = video.videoWidth;
ctx = canvas.getContext('2d');
var CANVAS_WIDTH = canvas.width;
var CANVAS_HEIGHT = canvas.height;
function drawVideoFrame() {
ctx.drawImage(video, 0, 0, CANVAS_WIDTH, CANVAS_HEIGHT);
recordedFrames.push(canvas.toDataURL('image/webp', 1));
}
videoInterval = $interval(drawVideoFrame, 1000/scope.frameRate);
scope.recording = true;
}
};
scope.stopRecording = function () {
$interval.cancel(videoInterval);
// 2nd param: framerate for the video file.
scope.video.files = Whammy.fromImageArray(recordedFrames, scope.frameRate);
recordedVideoBlob = Whammy.fromImageArray(recordedFrames, scope.frameRate); // you can chage this to some file copy method, so leave out the duplicate processing of images into video.
scope.videoMode = 'viewRecording';
scope.recording = false;
};

Categories

Resources