How do we get the Pixel data from images in Elm?
Here in JavaScript, is code to get the color of a set of pixels in a figure (taken from here)
var image = new Image;
image.src = "starry-night.jpg";
var canvas = d3.select("body").append("canvas");
var context = canvas.node().getContext("2d");
context.drawImage(image, 0, 0);
// beware variable name "image" got used twice
image = context.getImageData(0, 0, width, height);
var x = Math.random()*width,
y = Math.random()*height,
i = (y * width + x) << 2;
pixelColor = d3.rgb(image.data[i + 0], image.data[i + 1], image.data[i + 2]) + "";
The code loads an image to a <canvas> element, then extracts the color of a pixel from the canvas using image.getImageData().
Can we interface the image.data object in Elm? Right now I don't think it's possible...
Right now Collage types are list of forms...
HTML is also a module that can put imags in the DOM.
SVG allows for some simple global image transformations but nothing at the pixel level
Elm has the Virtual Dom. In fact of problems like this, might be addressed in virtual-dom which is lower level so we are not encouraged to do this directly.
However, Elm makes a clear distinction between Collage elements and SVG elements, with no clear interface to the getImageData() function.
Do I write my own with Elm's new interOp feature?
Does a way already exist in Elm? Or a new one has to be written?
JavaScript
The << operator is called Left Shift
As suggested by #SimonH, use a port to JS until Elm provides a first-hand way to do so (if it ever does). The same approach would apply to anything you can't yet do in Elm.
I'm just answering as an answer rather than a comment for the sake of others who come here.
Related
I implemented a CNN that I use on a web application via Tensorflow.js.
I need to preprocess my webcam photos to be accepted by my CNN model. So I want to use OpenCV.js in my .js file but I can't figure out how to simply import this library into my .js file where I turn my canvasElement into a tensor using the tf.browser.fromPixels() function of Tensorflow.js.
The tutorials I see show the use of OpenCV.js in the .html file directly inside a <script>, whereas I would like to use it in my javascript file.
I would especially like to use the method cv.cvtColor(). If not, do you have another solution to convert my canvasElement to grayscale?
The script tag will import OpenCV into the webpage (be sure to load this before you load your code that needs to use it - order matters in HTML). You should then be able to access the OpenCV class / object to call its functions with your canvas data to do your pre processing, and then write that back out and convert to tensor in TF.js land.
If you want to quickly convert canvas to greyscale there are many ways to do this - eg how you average the colours etc will effect the greyscale image you get out.
Here is one method: http://www.vapidspace.com/coding/2012/02/26/converting-images-to-grayscale-using-the-canvas/
Here is the code from that site in case it gets removed:
function grayscale (input,output) {
//Get the context for the loaded image
var inputContext = input.getContext("2d");
//get the image data;
var imageData = inputContext.getImageData(0, 0, input.width, input.height);
//Get the CanvasPixelArray
var data = imageData.data;
//Get length of all pixels in image each pixel made up of 4 elements for each pixel, one for Red, Green, Blue and Alpha
var arraylength = input.width * input.height * 4;
//Go through each pixel from bottom right to top left and alter to its gray equiv
//Common formula for converting to grayscale.
//gray = 0.3*R + 0.59*G + 0.11*B
for (var i=arraylength-1; i>0;i-=4)
{
//R= i-3, G = i-2 and B = i-1
//Get our gray shade using the formula
var gray = 0.3 * data[i-3] + 0.59 * data[i-2] + 0.11 * data[i-1];
//Set our 3 RGB channels to the computed gray.
data[i-3] = gray;
data[i-2] = gray;
data[i-1] = gray;
}
//get the output context
var outputContext = output.getContext("2d");
//Display the output image
outputContext.putImageData(imageData, 0, 0);
}
Notice here how they use a formula to calc gray. Depending on your needs you may want to use different ratios of the RGB mix to get the grayscale image.
Personally I would strongly recommend using vanilla JS here as it's very easy to do and you dont need to include OpenCV just to do grayscale which is a massive overhead to include that file for such a task. If you are using some of the more advanced features of OpenCV too then maybe that is a reason to then use it.
I am trying to make a website which helps its users to create a page by dragging and dropping elements on the canvas. The user should be able to save the html file of the edited canvas. I cannot figure how to convert the changes made to the canvas to an html file.
I don't think it's possible to get Markup out of canvas. I've searched it for a month but can't find a valid solution. but may be some experts may know. Best of luck buddy.
Canvas is basically just a bit-map image. Whatever you draw on the canvas is stored as pixels not as elements. So changes to the canvas are just changes in pixel values. To do what you wish you would need to store your 'elements' as 'objects' within your code where each 'object' stores all the required data for your 'element'.
it would then be possible to open a new window and export code into it using document.writeln
The code below may give you an idea of what sort of thing would be needed
newwindow=window.open('','_blank');
newwindow.document.writeln('<!DOCTYPE HTML>');
newwindow.document.writeln('<html>');
newwindow.document.writeln('<head>');
newwindow.document.writeln('<style>');
newwindow.document.writeln('#element0 {');
newwindow.document.writeln('background:'+ obj0.background+';');
newwindow.document.writeln('width:'+ obj0.width+';');
newwindow.document.writeln ('}');
newwindow.document.writeln('</style>');
newwindow.document.writeln('</head>');
newwindow.document.writeln('<body>');
newwindow.document.writeln('<div id="element0"></div>');
newwindow.document.writeln('</body>');
newwindow.document.writeln('</html>');
newwindow.document.writeln('<html>');
newwindow.document.close();
Hope this helps
Canvas won't help you here for anything other than to visualize the objects you have dropped onto it.
You need to record the objects you drop in a "shadow" structure behind the scene sort of. That is to say: build a object list internally which you then can use as source data to render:
Canvas visualization of it
Raw HTML code from it.
You can for example drop an image to the canvas and your code will record a new object (intention with the following code is to show the principle not to provide a full working solution):
var myObjects = [];
/// a drop occurred
var o = new myElement(x, y, width, height, id, type);
myElement is a pre-defined object that you set up in advance to hold the given arguments.
Then push the object to your object stack and render it to canvas:
myObjects.push(o);
for(var i = 0, o; o = myObjects[i]; i++) {
/// draw the look of this object here to canvas
}
When you then need a HTML version of it you do the same:
for(var i = 0, o; o = myObjects[i]; i++) {
var el = '<' + o.type + ' id="' + o.id + ' .... other things here
}
This way you can produce canvas graphics, HTML, send data over a socket etc.
The key in these sort things is to keep raw base data available. In this case it would be the element type you want to drop, its position and dimension. For HTML you also have to consider things as nesting etc. but that would require a bit more code than shown here.
I am just getting started with box2d. I am able to create some objects and create a world in which to use those objects. For example, to create a box (taken from: http://box2d-js.sourceforge.net/index2.html) I can do so like this:
elements = getElementsByClass("box2d");
for ( var i = 0; i < elements.length; i ++ ) {
properties[i] = getElementProperties( elements[i] );
bodyDef.type = b2Body.b2_dynamicBody;
var data = { element: elements[i]};
bodyDef.userData = data;
fixDef.shape = new b2PolygonShape;
fixDef.shape.SetAsBox(2 //half width , 2);
bodyDef.position.x = Math.random() * 10;
bodyDef.position.y = Math.random() * 10;
world.CreateBody(bodyDef).CreateFixture(fixDef);
}
What I would like to do is take divs with text and turn those divs with text into shapes and use them in box2D. I really don't know where to start with this. I am using the userData property now thinking that this is at least a start.
I found a great example of this in action here: http://mrdoob.com/projects/chromeexperiments/google_gravity/
getElementProperties and getElementsByClass are defined. I was thinking about using getElementProperties to set the shape dimensions.
I am doing something similar with iOS, but a good way to approach it is to make images out of the information held in the div. Then you can make a sprite out of that image and add it as the userData for your box2d body. When you update (tick: method in iOS, I am not sure for javascript) then you iterate over all the bodies and adjust the sprite location. The box2d world should do everything else for you.
You can take reference this samples..
http://jsbin.com/efequk/5
http://www.jeremyhubble.com/box2d.html
I try to change Raphaël path elements essentially in the following way (please regard the code includes build up stuff for a complete example):
var n = 100,
i = 0;
var values = [n];
var panel = document.createElement("div");
var paper = null;
var path = null;
panel.id = "panel";
panel.style.top = '0px';
panel.style.left = '0px';
panel.style.width = '300px';
panel.style.height = '300px';
panel.style.background = 'black';
document.getElementsByTagName('body')[0].appendChild(panel);
paper = Raphael(panel, 0, 0);
path = paper.path('m0,0');
path.attr({ stroke: '#fff', 'stroke-width': 1 });
function test () {
i = n;
while (i--)
values[i] = Math.round(Math.random() * 3);
// perform change here!
path.attr({ path: 'm0,0l0,' + values.join('l3,') });
// just a test case!
setTimeout(test, 1);
};
test();
Unfortunately this approach leaks in memory. I've tested it in FF 4 and IE 7+ with Raphaël 1.5.2 and 2.0 beta. The only difference is that Raphaël 1.5.2 leaks much faster than 2.0 beta.
What am I doing wrong?
Update
To put this question into context: I want to implement a 'realtime' graph control with Raphaël. Therefore I use an array buffer for each series and render them when the buffer size is reached, so I need only to render a given fix length series.
The only way I saw to do this in Raphaël is a path element per series which gets an update of it's path attribute .attr({path: path.attr('path') + getSvgPath(buffer)}) if necessary, followed by an translation on the x axis depending on the buffer size .animate({translation: (bufferSize*valuesDistance*-1) + ',0'}, 500, '<', callback) - for a smooth animation of the updates - and at last a shift of the path attribute after the animation to prevent ever expanding path strings: .attr({path: shiftSvgPath(path.attr('path'))}).
The functions shiftSvgPath() and getSvgPath() just returning the appropriate svg path string. So that the result always consits of one moveTo command at the beginning and a constant number of lineTo commands, either equal to the number of displayed values or plus the buffer size.
I ran into a similar problem lately. I wouldn't call it a leak, but rather a huge memory consumption from Raphael, when drawing paths. I'm just guessing it uses some caching arrays internally that eat up a lot of memory.
My approach was to ditch Raphael and draw the svg elements with plain old javascript.
I want to take an irregularly shaped section from an existing image and render it as a new image in Javascript using HTML5 canvases. So, only the data inside the polygon boundary will be copied. The approach I came up with involved:
Draw the polygon in a new canvas.
Create a mask using clip
Copy the data from the original canvas using getImageData (a rectangle)
Apply the data to the new canvas using putImageData
It didn't work, the entire rectangle (e.g. the stuff from the source outside the boundary) is still appearing. This question explains why:
"The spec says that putImageData will not be affected by clipping regions." Dang!
I also tried drawing the shape, setting context.globalCompositeOperation = "source-in", and then using putImageData. Same result: no mask applied. I suspect for a similar reason.
Any suggestions on how to accomplish this goal? Here's basic code for my work in progress, in case it's not clear what I'm trying to do. (Don't try too hard to debug this, it's cleaned up/extracted from code that uses a lot of functions that aren't here, just trying to show the logic).
// coords is the polygon data for the area I want
context = $('canvas')[0].getContext("2d");
context.save();
context.beginPath();
context.moveTo(coords[0], coords[1]);
for (i = 2; i < coords.length; i += 2) {
context.lineTo(coords[i], coords[i + 1]);
}
//context.closePath();
context.clip();
$img = $('#main_image');
copy_canvas = new_canvas($img); // just creates a new canvas matching dimensions of image
copy_ctx = copy.getContext("2d");
tempImage = new Image();
tempImage.src = $img.attr("src");
copy_ctx.drawImage(tempImage,0,0,tempImage.width,tempImage.height);
// returns array x,y,x,y with t/l and b/r corners for a polygon
corners = get_corners(coords)
var data = copy_ctx.getImageData(corners[0],corners[1],corners[2],corners[3]);
//context.globalCompositeOperation = "source-in";
context.putImageData(data,0,0);
context.restore();
dont use putImageData,
just make an extra in memory canvas with document.createElement to create the mask and apply that with a drawImage() and the globalCompositeOperation function (depending on the order you need to pick the right mode;
I do something similar here the code is here (mind the CasparKleijne.Canvas.GFX.Composite function)