I am trying to read pixels (frame by frame) from websocket and render it on to canvas (in grayscale). Data is being displayed properly, just that pixels of the current frame are displayed overlapped on previous frame. Thus picture gets smudged after few frames.
I am tying to clear out the previous frame before rendering current one. What would be the proper way to do it?
I am using putImageData() to render the frame. I have tried clearRect() before calling putImageData(). and have tried clearing Imgdata (Imgdata.data = [];) array before populating it again but none of these things worked.
` var canvas = document.querySelector("canvas");
var context = canvas.getContext("2d");
var Imgdata = context.createImageData(100,100);
var numPixel = Imgdata.data.length/4;
ws.onmessage = function (event) {
var bytes = new Uint8Array(event.data);
console.log("data: " + numPixel + " bytes: "+ bytes.length);
//Imgdata.data = [];
for (var i = 0; i < numPixel; i++) {
Imgdata.data[(i*4)+Math.round(bytes[i]/85)] = (bytes[i]%85)*3;
Imgdata.data[(i*4)+3] = 255;
Imgdata.data[(i*4)+0] = bytes[i];
Imgdata.data[(i*4)+1] = bytes[i];
Imgdata.data[(i*4)+2] = bytes[i];
Imgdata.data[(i*4)+3] = 255;
}
//context.clearRect(0, 0, canvas.width, canvas.height);
context.putImageData(Imgdata,0,0);
};
`
I would say the cause is most likely due to the first line inside the for loop, without knowing the data i dont know what it does but id does not look right.
// ???
Imgdata.data[(i*4)+Math.round(bytes[i]/85)] = (bytes[i]%85)*3;
If the data is random then this would randomly set one of the colour channels of each pixel.
Apart from that I can not see any errors that would cause what you describe.
I have rewritten the code only as a suggestion to improve the performance a little.
const canvas = document.querySelector("canvas");
const context = canvas.getContext("2d");
const imgdata = context.createImageData(100,100);
const pix = imgdata.data;
pix.fill(255); // set all to 255 To cover the alpha channel
const numPixel = pix.length; // best leave as byte count
ws.onmessage = function (event) {
const inData = event.data;
var inIndex = 0;
var i = 0;
while(i < numPixel){
pix[i ++] = pix[i ++] = pix[i ++] = inData[inIndex ++];
i++;
}
context.putImageData(imgdata,0,0);
};
That should work event if the incoming data is incomplete the function putImageData completely replaces the pixels. If any pixels are undefined from the socket data they will be set to zero and will show up as black.
Related
I'm making a web app for drawing pixel art. The app itself works, I can choose canvas size, colors and tools to draw. Now I want to generate an image from it.
My canvas is a grid of divs that change background color when you draw on them. I made a function that creates a 2D array with each item having 4 values (red, green, blue, alpha).
Here's a function I use:
function rgb2pngStart(val) {
let finish = val.length - 1;
let sliced = val.slice(4, finish)
let stringAr = sliced.split(', ');
let R = parseInt(stringAr[0]);
let G = parseInt(stringAr[1]);
let B = parseInt(stringAr[2]);
let pixelArray = [R, G, B, 255];
return pixelArray;
}
function createArray() {
let sideInPixels = document.documentElement.style.getPropertyValue('--number');
let canvasX = [];
let canvasY = [];
for (let i = 1; i <= sideInPixels; i++) {
for (let j = 1; j <= sideInPixels; j++) {
let pixel = document.getElementById(i + ' ' + j).style.backgroundColor;
canvasX[j - 1] = rgb2pngStart(pixel);
}
canvasY[i - 1] = canvasX;
}
return canvasY;
}
I was trying to create a canvas, using some old answers here, and put my data into imagedata and then show that image to user so that he could download it. But I don't know how do I do that...
So, how do I generate image from this array? Or what should I change for it to work?
The most convenient approach would be to use an actual <canvas> element: you can create ImageData from your RGBA pixel information and put it to the canvas context — and then generate a data URL or a blob, which can be used to create an object URL (both of which can be used as download targets).
I have a javascript function that calculates the area of a region drawn on a canvas, the function is extremely slow and makes the performance of the web app horrible, I was wondering if there is any way I can optimize it?
It works fine if a user triggers it once in a while, but there are scenarios where it will be rapidly triggered by the user, sometimes up to 200 times in 5 seconds, so it's not ideal.
Tegaki.refreshMeasurements = () => {
for (let layer of Tegaki.layers) {
if (layer.name == 'background') continue;
let canvas = layer.canvas
let i, ctx, dest, data, len;
ctx = canvas.getContext('2d')
dest = ctx.getImageData(0, 0, canvas.width, canvas.height);
data = dest.data,
len = data.length;
let totalPixels = 0;
let totalHU = 0
for (i = 3; i < len; i += 4) {
if (data[i] > 0) {
totalPixels++;
totalHU += (Tegaki.selectedSlicePixelData[Math.floor(i/4)] +
Tegaki.selectedSliceRescaleIntercept)
}
}
let area = totalPixels * Tegaki.selectedSlicePixelSpacing / 100;
let meanHU = totalHU / totalPixels
let layerType = _this.layerTypes.find(t => t.id == layer.name)
layerType.area = area;
layerType.meanHU = meanHU
}
}
I think the part causing the most performance drop is the for-loop because the length of the image data array sometimes is over 1 million so this thing probably loops for a while.
Is there any better way to implement this?
I need to analyze an image to find all colors in a PNG or GIF image. I currently load the image to a canvas, then get the image data, then loop through every pixel and check it against each color in the palette. It takes forever, the browser thinks the script has stopped and it sometimes just crashes. Hoping there is a better way.
//load file
var fileReader = new FileReader();
fileReader.onload = function(e) {
var img = new Image();
img.onload = function() {
//create a new pixel with the images dimentions
newPixel(this.width, this.height, []);
//draw the image onto the canvas
context.drawImage(img, 0, 0);
var colorPalette = [];
var imagePixelData = context.getImageData(0,0,this.width, this.height).data;
console.log(imagePixelData)
for (var i = 0; i < imagePixelData.length; i += 4) {
var color = rgbToHex(imagePixelData[i],imagePixelData[i + 1],imagePixelData[i + 2]);
if (colorPalette.indexOf(color) == -1) {
colorPalette.push(color);
//don't allow more than 256 colors to be added
if (colorPalette.length >= settings.maxColorsOnImportedImage) {
alert('The image loaded seems to have more than '+settings.maxColorsOnImportedImage+' colors.')
break;
}
}
}
createColorPalette(colorPalette, false);
//track google event
ga('send', 'event', 'Pixel Editor Load', colorPalette.length, this.width+'/'+this.height); /*global ga*/
};
img.src = e.target.result;
};
fileReader.readAsDataURL(this.files[0]);
This will speed things up a little.
The function indexof will search all of the array if it can not find an entry. This can be very slow, you can improve the speed of the search by adding to the top of the array moving down then searching for the index from the top most entry. This ensures that the search is only over added entries not the entire array.
Also use 32Bit words rather than 8bit bytes, use typed arrays as they are significantly quicker that pushing onto a array and don't convert to hex inside the loop it is redundant and can be done after on the smaller data set.
See comments for logic.
//draw the image onto the canvas
ctx.drawImage(image, 0, 0);
// get size
const size = image.width * image.height;
// create pallete buffer
const colors32 = new Uint32Array(256);
// current pos of palette entry
var palettePos = colors32.length - 1; // Start from top as this will speed up indexOf function
// pixel data as 32 bit words so you can handle a pixel at a time rather than bytes.
const imgData = new Uint32Array(ctx.getImageData(0, 0, image.width, image.height).data.buffer);;
// hold the color. If the images are low colour (less 256) it is highly probable that many pixels will
// be the same as the previous. You can avoid the index search if this is the case.
var color= colors32[palettePos --] = imgData[0]; // assign first pixels to ease logic in loop
for (var i = 1; i < size && palettePos >= 0; i += 1) { // loop till al pixels read if palette full
if(color !== imgData[i]){ // is different than previouse
if (colors32.indexOf(imgData[i], palettePos) === -1) { // is in the pallet
color = colors32[palettePos --] = imgData[i]; // add it
}
}
}
// all the performance issues are over so now convert to the palette format you wanted.
const colorPalette = [];
colors32.reverse();
const paletteSize = (255 - palettePos) * 4;
const colors8 = new Uint8Array(colors32.buffer);
for(i = 0; i < paletteSize; i += 4){
colorPalette.push(rgbToHex(colors8[i],colors8[i + 1],colors8[i + 2]));
}
I have binary nodejs Buffer object that contains bitmap information. How do make image from the buffer and save it to file?
Edit:
I tried using the file system package as #herchu said but if I do this:
let robot = require("robotjs")
let fs = require('fs')
let size = 200
let img = robot.screen.capture(0, 0, size, size)
let path = 'myfile.png'
let buffer = img.image
fs.open(path, 'w', function (err, fd) {
if (err) {
// Something wrong creating the file
}
fs.write(fd, buffer, 0, buffer.length, null, function (err) {
// Something wrong writing contents!
})
})
I get
Although solutions by #herchu and #Jake work, they are extremely slow (10-15s in my experience).
Jimp supports converting Raw Pixel Buffer into PNG out-of-the-box and works a lot faster (sub-second).
const img = robot.screen.capture(0, 0, width, height).image;
new Jimp({data: img, width, height}, (err, image) => {
image.write(fileName);
});
Note: I am editing my answer according to your last edits
If you are using Robotjs, check that its Bitmap object contains a Buffer to raw pixels data -- not a PNG or any other file format contents, just pixels next to each other (exactly 200 x 200 elements in your case).
I have not found any function to write contents in other format in the Robotjs library (not that I know it either), so in this answer I am using a different library, Jimp, for the image manipulation.
let robot = require("robotjs")
let fs = require('fs')
let Jimp = require('jimp')
let size = 200
let rimg = robot.screen.capture(0, 0, size, size)
let path = 'myfile.png'
// Create a new blank image, same size as Robotjs' one
let jimg = new Jimp(size, size);
for (var x=0; x<size; x++) {
for (var y=0; y<size; y++) {
// hex is a string, rrggbb format
var hex = rimg.colorAt(x, y);
// Jimp expects an Int, with RGBA data,
// so add FF as 'full opaque' to RGB color
var num = parseInt(hex+"ff", 16)
// Set pixel manually
jimg.setPixelColor(num, x, y);
}
}
jimg.write(path)
Note that the conversion is done by manually iterating through all pixels; this is slow in JS. Also there are some details on how each library handles their pixel format, so some manipulation was needed in the loop -- it should be clear from the embedded comments.
Adding this as an addendum to accepted answer from #herchu, this code sample processes/converts the raw bytes much more quickly (< 1s for me for a full screen). Hope this is helpful to someone.
var jimg = new Jimp(width, height);
for (var x=0; x<width; x++) {
for (var y=0; y<height; y++) {
var index = (y * rimg.byteWidth) + (x * rimg.bytesPerPixel);
var r = rimg.image[index];
var g = rimg.image[index+1];
var b = rimg.image[index+2];
var num = (r*256) + (g*256*256) + (b*256*256*256) + 255;
jimg.setPixelColor(num, x, y);
}
}
Four times faster!
About 280ms and 550Kb for full screen 1920x1080, if use this script.
I found this pattern when I compared 2 byte threads per byte to the forehead.
const robotjs = require('robotjs');
const Jimp = require('jimp');
const app = require('express').Router();
app.get('/screenCapture', (req, res)=>{
let image = robotjs.screen.capture();
for(let i=0; i < image.image.length; i++){
if(i%4 == 0){
[image.image[i], image.image[i+2]] = [image.image[i+2], image.image[i]];
}
}
var jimg = new Jimp(image.width, image.height);
jimg.bitmap.data = image.image;
jimg.getBuffer(Jimp.MIME_PNG, (err, result)=>{
res.set('Content-Type', Jimp.MIME_PNG);
res.send(result);
});
});
If you add this code before jimp.getBuffer you'll get about 210ms and 320Kb for full screen
jimg.rgba(true);
jimg.filterType(1);
jimg.deflateLevel(5);
jimg.deflateStrategy(1);
I suggest you to take a look on sharp as it has superior performance metrics over jimp.
The issue with robotjs screen capturing, which actually happened to be very efficient, is BGRA color model and not RGBA. So you would need to do additional color rotation.
Also, as we take screenshot from the desktop I can't imagine the case where we would need transperency. So, I suggest to ignore it.
const [left, top, width, height] = [0, 0, 100, 100]
const channels = 3
const {image, width: cWidth, height: cHeight, bytesPerPixel, byteWidth} = robot.screen.capture(left, right, width, height)
const uint8array = new Uint8Array(cWidth*cHeight*channels);
for(let h=0; h<cHeight; h+=1) {
for(let w=0; w<cWidth; w+=1) {
let offset = (h*cWidth + w)*channels
let offset2 = byteWidth*h + w*bytesPerPixel
uint8array[offset] = image.readUInt8(offset2 + 2)
uint8array[offset + 1] = image.readUInt8(offset2 + 1)
uint8array[offset + 2] = image.readUInt8(offset2 + 0)
}
}
await sharp(Buffer.from(uint8array), {
raw: {
width: cWidth,
height: cHeight,
channels,
}
}).toFile('capture.png')
I use intermediate array here, but you actually can just to swap in the result of the screen capture.
I've been developing a canvas application in javascript in which I need to define a 9 cell grid (3x3) and in each grid have a canvas element.
The thing is, I have already an array called grid which has the data to represent the size of each cell.
var grid = new Array();
grid[0] = {x:[0, elemWidth], y:[0, elemHeight]};
grid[1] = {x:[elemWidth, elemWidth*2], y:[0, elemHeight]};
grid[2] = {x:[(elemWidth*2), sWidth], y:[0, elemHeight]};
grid[3] = {x:[0, elemWidth], y:[elemHeight+1,elemHeight*2]};
grid[4] = {x:[elemWidth, elemWidth*2], y:[elemHeight, elemHeight*2]};
grid[5] = {x:[(elemWidth*2), sWidth], y:[elemHeight, elemHeight*2]};
grid[6] = {x:[0, elemWidth], y:[(elemHeight*2), sHeight]};
grid[7] = {x:[elemWidth, elemWidth*2], y:[(elemHeight*2), sHeight]};
grid[8] = {x:[(elemWidth*2), sWidth], y:[(elemHeight*2), sHeight]};
This produces the following output on Chrome Inspector:
As far as I can tell, I should be able to obtain the x and y values by using the following code
canvas[i].fillRect(grid[i].x[0], grid[i].y[0], grid[i].x[1], grid[i].y[1]);
canvas[i].strokeRect(grid[i].x[0], grid[i].y[0], grid[i].x[1], grid[i].y[1]);
and each should give me the corresponding values.
I can't understand why I can get everything OK when on canvas[0] (everything renders on the browser) but when it comes to the subsequent canvas[i](with i>0) won't render and I get an undefined value although the HTML produces the tags for each canvas element.
Am I doing anything wrong declaring the arrays?
EDIT
Made a few changes to the code just for the sake of simplicity and readability:
function createCanvas() {
/* criar grelha aqui */
var sWidth = window.innerWidth; //- 50;
var sHeight = window.innerHeight; //- 50;
var elemWidth = sWidth / 3;
var elemHeight = sHeight / 3;
var grid = new Array();
var row1 = [0, elemHeight];
var row2 = [elemHeight, elemHeight*2];
var row3 = [elemHeight*2, sHeight];
var col1 = [0, elemWidth];
var col2 = [elemWidth, elemWidth*2];
var col3 = [elemWidth*2, sWidth];
grid[0] = [col1, row1];
grid[1] = [col2, row1];
grid[2] = [col3, row1];
grid[3] = [col1, row2];
grid[4] = [col2, row2];
grid[5] = [col3, row2];
grid[6] = [col1, row3];
grid[7] = [col2, row3];
grid[8] = [col3, row3];
for (var i = 0; i < grid.length; i++) {
var c = document.getElementById('content');
var newCanvas = document.createElement('canvas');
newCanvas.setAttribute('id', 'canvas'+i);
c.appendChild(newCanvas);
canvas[i] = new Canvas('canvas'+i, 0, function() {});
canvas[i].clear();
canvas[i].setAutoResize(true);
canvas[i].canvasElement.width = elemWidth - 2;
canvas[i].canvasElement.height = elemHeight - 2;
canvas[i].fillStyle('#000000');
canvas[i].strokeStyle('#FFFFFF');
canvas[i].fillRect(grid[i][0], grid[i][0], grid[i][2], grid[i][3]);
canvas[i].strokeRect(grid[i][0], grid[i][0], grid[i][4], grid[i][5]);
This code is much more complete than the previous version which only had a little bit of the function
Meanwhile I managed to get 3 cells to render (although the 3 cell, which should be 1x3 is actually a 2x1 because somehow the rendering isn't happening properly, which I think might be a CSS problem)
EDIT2
Again I've made some changes to the code above. And new screenshot of the grid array expanded view.