Develop Reference

How do I insert my image data from array into canvas so that I can download it?

I'm making a web app for drawing pixel art. The app itself works, I can choose canvas size, colors and tools to draw. Now I want to generate an image from it.
My canvas is a grid of divs that change background color when you draw on them. I made a function that creates a 2D array with each item having 4 values (red, green, blue, alpha).
Here's a function I use:
function rgb2pngStart(val) {
let finish = val.length - 1;
let sliced = val.slice(4, finish)
let stringAr = sliced.split(', ');
let R = parseInt(stringAr[0]);
let G = parseInt(stringAr[1]);
let B = parseInt(stringAr[2]);
let pixelArray = [R, G, B, 255];
return pixelArray;
}
function createArray() {
let sideInPixels = document.documentElement.style.getPropertyValue('--number');
let canvasX = [];
let canvasY = [];
for (let i = 1; i <= sideInPixels; i++) {
for (let j = 1; j <= sideInPixels; j++) {
let pixel = document.getElementById(i + ' ' + j).style.backgroundColor;
canvasX[j - 1] = rgb2pngStart(pixel);
}
canvasY[i - 1] = canvasX;
}
return canvasY;
}
I was trying to create a canvas, using some old answers here, and put my data into imagedata and then show that image to user so that he could download it. But I don't know how do I do that...
So, how do I generate image from this array? Or what should I change for it to work?

The most convenient approach would be to use an actual <canvas> element: you can create ImageData from your RGBA pixel information and put it to the canvas context — and then generate a data URL or a blob, which can be used to create an object URL (both of which can be used as download targets).

Incomplete Sierpinski's Triangle Displaying

I am following the textbook The Nature of code's Example 7.1. (The original code is written in Java, but since the processing library is functionally identical to p5.js, I have rewritten it JavaScript out of convenience)
I believe that I have copied the examples code verbatim, yet somehow I have ended up with a result which I did not expect. There is an incomplete portion in the Sierpinski's triangle which is displayed.
I would like to know where I am going wrong in my code, or what I might be misunderstanding to cause this kind of issue.
Here's the Code for the image
class CA {
constructor(ca_width) {
this.generation = 0;
this.w = 5;
this.cells = new Array(1050 / this.w);
this.newcells = new Array(this.cells.length);
this.ruleset = [0, 1, 0, 1, 1, 0, 1, 0]; // [1,1,0,1,1,1,1,0]//
for (let i = 0; i < this.cells.length; i++) {
this.cells[i] = 0;
}
this.cells[this.cells.length / 2] = 1;
}
generate() {
let nextgen = new Array(this.cells.length);
for (let i = 1; i < this.cells.length - 1; i++) {
let left = this.cells[i - 1];
let me = this.cells[i];
let right = this.cells[i + 1];
nextgen[i] = this.rules(left, me, right);
}
this.cells = nextgen;
this.generation++;
}
rules(a, b, c) {
let s = "" + a + b + c;
let index = parseInt(s, 2);
return this.ruleset[index];
}
display() {
for (let i = 0; i < this.cells.length; i++) {
if (this.cells[i] == 0)
fill(255);
else fill(0);
stroke(0);
rect(i * this.w, this.generation * this.w,
this.w, this.w);
}
}
}
let cA;
function setup() {
createCanvas(windowWidth, windowHeight);
cA = new CA(1000);
}
function draw() {
// createCanvas(windowWidth, 400);
// background(150);
cA.display();
cA.generate();
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.1/p5.js"></script>

Based on your logic for computing the next generation, this.cells[0] and this.cells.at(-1) are always undefined, which is rendered as black. You might want to initialize these to 0, and possibly use a wraparound logic for computing their value (i.e. cells[0] = rule(cells.at(-1), cells[0], cells[1])).
I don't know what your Java code looks like, but if you're using a new int[] type, then that'll be zeroed out and probably work as expected, unlike JS Array().
In general, use caution with the Array() constructor in JS. It leaves the cells in an uninitialized state ("empty slots"), so you usually want to chain .fill(0) or spread/map it to make it a usable, unsurprising array that you'd expect such a constructor to emit, as it does in Java.
Keep in mind that the Nature of Code is available in a p5.js version, and there exist automatic translators from Processing to p5.js, like pde2js. Note that I haven't tried pde2js or the other translators but they seem worth a look.

HTML Canvas putImageData causing frame overlap

I am trying to read pixels (frame by frame) from websocket and render it on to canvas (in grayscale). Data is being displayed properly, just that pixels of the current frame are displayed overlapped on previous frame. Thus picture gets smudged after few frames.
I am tying to clear out the previous frame before rendering current one. What would be the proper way to do it?
I am using putImageData() to render the frame. I have tried clearRect() before calling putImageData(). and have tried clearing Imgdata (Imgdata.data = [];) array before populating it again but none of these things worked.
` var canvas = document.querySelector("canvas");
var context = canvas.getContext("2d");
var Imgdata = context.createImageData(100,100);
var numPixel = Imgdata.data.length/4;
ws.onmessage = function (event) {
var bytes = new Uint8Array(event.data);
console.log("data: " + numPixel + " bytes: "+ bytes.length);
//Imgdata.data = [];
for (var i = 0; i < numPixel; i++) {
Imgdata.data[(i*4)+Math.round(bytes[i]/85)] = (bytes[i]%85)*3;
Imgdata.data[(i*4)+3] = 255;
Imgdata.data[(i*4)+0] = bytes[i];
Imgdata.data[(i*4)+1] = bytes[i];
Imgdata.data[(i*4)+2] = bytes[i];
Imgdata.data[(i*4)+3] = 255;
}
//context.clearRect(0, 0, canvas.width, canvas.height);
context.putImageData(Imgdata,0,0);
};
`

I would say the cause is most likely due to the first line inside the for loop, without knowing the data i dont know what it does but id does not look right.
// ???
Imgdata.data[(i*4)+Math.round(bytes[i]/85)] = (bytes[i]%85)*3;
If the data is random then this would randomly set one of the colour channels of each pixel.
Apart from that I can not see any errors that would cause what you describe.
I have rewritten the code only as a suggestion to improve the performance a little.
const canvas = document.querySelector("canvas");
const context = canvas.getContext("2d");
const imgdata = context.createImageData(100,100);
const pix = imgdata.data;
pix.fill(255); // set all to 255 To cover the alpha channel
const numPixel = pix.length; // best leave as byte count
ws.onmessage = function (event) {
const inData = event.data;
var inIndex = 0;
var i = 0;
while(i < numPixel){
pix[i ++] = pix[i ++] = pix[i ++] = inData[inIndex ++];
i++;
}
context.putImageData(imgdata,0,0);
};
That should work event if the incoming data is incomplete the function putImageData completely replaces the pixels. If any pixels are undefined from the socket data they will be set to zero and will show up as black.

How to save binary buffer to png file in nodejs?

I have binary nodejs Buffer object that contains bitmap information. How do make image from the buffer and save it to file?
Edit:
I tried using the file system package as #herchu said but if I do this:
let robot = require("robotjs")
let fs = require('fs')
let size = 200
let img = robot.screen.capture(0, 0, size, size)
let path = 'myfile.png'
let buffer = img.image
fs.open(path, 'w', function (err, fd) {
if (err) {
// Something wrong creating the file
}
fs.write(fd, buffer, 0, buffer.length, null, function (err) {
// Something wrong writing contents!
})
})
I get

Although solutions by #herchu and #Jake work, they are extremely slow (10-15s in my experience).
Jimp supports converting Raw Pixel Buffer into PNG out-of-the-box and works a lot faster (sub-second).
const img = robot.screen.capture(0, 0, width, height).image;
new Jimp({data: img, width, height}, (err, image) => {
image.write(fileName);
});

Note: I am editing my answer according to your last edits
If you are using Robotjs, check that its Bitmap object contains a Buffer to raw pixels data -- not a PNG or any other file format contents, just pixels next to each other (exactly 200 x 200 elements in your case).
I have not found any function to write contents in other format in the Robotjs library (not that I know it either), so in this answer I am using a different library, Jimp, for the image manipulation.
let robot = require("robotjs")
let fs = require('fs')
let Jimp = require('jimp')
let size = 200
let rimg = robot.screen.capture(0, 0, size, size)
let path = 'myfile.png'
// Create a new blank image, same size as Robotjs' one
let jimg = new Jimp(size, size);
for (var x=0; x<size; x++) {
for (var y=0; y<size; y++) {
// hex is a string, rrggbb format
var hex = rimg.colorAt(x, y);
// Jimp expects an Int, with RGBA data,
// so add FF as 'full opaque' to RGB color
var num = parseInt(hex+"ff", 16)
// Set pixel manually
jimg.setPixelColor(num, x, y);
}
}
jimg.write(path)
Note that the conversion is done by manually iterating through all pixels; this is slow in JS. Also there are some details on how each library handles their pixel format, so some manipulation was needed in the loop -- it should be clear from the embedded comments.

Adding this as an addendum to accepted answer from #herchu, this code sample processes/converts the raw bytes much more quickly (< 1s for me for a full screen). Hope this is helpful to someone.
var jimg = new Jimp(width, height);
for (var x=0; x<width; x++) {
for (var y=0; y<height; y++) {
var index = (y * rimg.byteWidth) + (x * rimg.bytesPerPixel);
var r = rimg.image[index];
var g = rimg.image[index+1];
var b = rimg.image[index+2];
var num = (r*256) + (g*256*256) + (b*256*256*256) + 255;
jimg.setPixelColor(num, x, y);
}
}

Four times faster!
About 280ms and 550Kb for full screen 1920x1080, if use this script.
I found this pattern when I compared 2 byte threads per byte to the forehead.
const robotjs = require('robotjs');
const Jimp = require('jimp');
const app = require('express').Router();
app.get('/screenCapture', (req, res)=>{
let image = robotjs.screen.capture();
for(let i=0; i < image.image.length; i++){
if(i%4 == 0){
[image.image[i], image.image[i+2]] = [image.image[i+2], image.image[i]];
}
}
var jimg = new Jimp(image.width, image.height);
jimg.bitmap.data = image.image;
jimg.getBuffer(Jimp.MIME_PNG, (err, result)=>{
res.set('Content-Type', Jimp.MIME_PNG);
res.send(result);
});
});
If you add this code before jimp.getBuffer you'll get about 210ms and 320Kb for full screen
jimg.rgba(true);
jimg.filterType(1);
jimg.deflateLevel(5);
jimg.deflateStrategy(1);

I suggest you to take a look on sharp as it has superior performance metrics over jimp.
The issue with robotjs screen capturing, which actually happened to be very efficient, is BGRA color model and not RGBA. So you would need to do additional color rotation.
Also, as we take screenshot from the desktop I can't imagine the case where we would need transperency. So, I suggest to ignore it.
const [left, top, width, height] = [0, 0, 100, 100]
const channels = 3
const {image, width: cWidth, height: cHeight, bytesPerPixel, byteWidth} = robot.screen.capture(left, right, width, height)
const uint8array = new Uint8Array(cWidth*cHeight*channels);
for(let h=0; h<cHeight; h+=1) {
for(let w=0; w<cWidth; w+=1) {
let offset = (h*cWidth + w)*channels
let offset2 = byteWidth*h + w*bytesPerPixel
uint8array[offset] = image.readUInt8(offset2 + 2)
uint8array[offset + 1] = image.readUInt8(offset2 + 1)
uint8array[offset + 2] = image.readUInt8(offset2 + 0)
}
}
await sharp(Buffer.from(uint8array), {
raw: {
width: cWidth,
height: cHeight,
channels,
}
}).toFile('capture.png')
I use intermediate array here, but you actually can just to swap in the result of the screen capture.

HTML5 Canvas Drawing in Real Time

Question: How can I make putImageData() update the canvas in real time, as various parts of the image have been computed?
I am working on a JavaScript/TypeScript application to draw the Mandelbrot set on an HTML5 <canvas> element. Math and details aside, my application draws the set just fine. However, if you are familiar with visualizing the set, you know that it can take a long time to draw.
It will draw in a few seconds, but until then, the canvas is completely blank, then the image appears. I'm looking for a way to draw each row as it is computed using putImageData(). Here is what I am trying:
// part of the class definition
private Context: CanvasRenderingContext2D;
private ImageData: ImageData;
private Pixels: number[];
constructor() {
var c: HTMLCanvasElement = <HTMLCanvasElement>document.getElementById("can");
this.Context = c.getContext("2d");
this.ImageData = this.Context.createImageData(this.Size.Width, 1);
this.Pixels = this.ImageData.data;
}
public draw() {
// tried this... does not help
// var handler = function(m: Mandelbrot) {
// m.Context.putImageData(m.ImageData, 0, i)
// };
for(var i: number = 0; i < this.Size.Height; ++i) { // Loop over each row
for(var j: number = 0; j < this.Size.Width; ++j) { // Calc px. for one row
// all the math to compute the set... (works)
this.setPixelColor(j, color); // sets a color in this.Pixels (works)
}
// setTimeout(handler(this), 0); // does not help
this.Context.putImageData(this.ImageData, 0, i); // Draw the row on the canvas?
}
}
Somehow, the putImageData() function, which is called after a row in the image has been computed, only shows the image after the entire image has been generated.
How can I make putImageData() update the canvas in real time, as each row has been computed?
Latest update of non-working code:
var handler = function(m: Mandelbrot, i: number) {
for (var j: number = 0; j < m.Size.Width; ++j) {
// math
m.setPixelColor(j, color);
}
m.Context.putImageData(m.ImageData, 0, i);
};
var that: Mandelbrot = this;
for(var i: number = 0; i < this.Size.Height; ++i) {
setTimeout(function() {
handler(that, i)
}, 0);
}
Working code, thanks to ekuusela:
var handler = function(m: Mandelbrot, i: number) {
return function() {
for (var j: number = 0; j < m.Size.Width; ++j) {
// math
m.setPixelColor(j, color);
}
m.Context.putImageData(m.ImageData, 0, i);
}
};
for(var i: number = 0; i < this.Size.Height; ++i) {
setTimeout(handler(this, i), 0);
}

Try wrapping putImageData and the calculation for a single row to a setTimeout call to execute it asynchronously (post accept edit: see the final code in the question, this won't work since i will be undefined in the putImageData row)
public draw() {
var that = this;
var drawRow = function() {
for(var j: number = 0; j < that.Size.Width; ++j) { // Calc px. for one row
that.setPixelColor(j, color); // sets a color in this.Pixels (works)
}
// TODO specify the dirty region in this call
that.Context.putImageData(that.ImageData, 0, i); // Draw the row on the canvas?
};
for(var i: number = 0; i < this.Size.Height; ++i) { // Loop over each row
setTimeout(drawRow, 0);
}
}