I implemented a CNN that I use on a web application via Tensorflow.js.
I need to preprocess my webcam photos to be accepted by my CNN model. So I want to use OpenCV.js in my .js file but I can't figure out how to simply import this library into my .js file where I turn my canvasElement into a tensor using the tf.browser.fromPixels() function of Tensorflow.js.
The tutorials I see show the use of OpenCV.js in the .html file directly inside a <script>, whereas I would like to use it in my javascript file.
I would especially like to use the method cv.cvtColor(). If not, do you have another solution to convert my canvasElement to grayscale?
The script tag will import OpenCV into the webpage (be sure to load this before you load your code that needs to use it - order matters in HTML). You should then be able to access the OpenCV class / object to call its functions with your canvas data to do your pre processing, and then write that back out and convert to tensor in TF.js land.
If you want to quickly convert canvas to greyscale there are many ways to do this - eg how you average the colours etc will effect the greyscale image you get out.
Here is one method: http://www.vapidspace.com/coding/2012/02/26/converting-images-to-grayscale-using-the-canvas/
Here is the code from that site in case it gets removed:
function grayscale (input,output) {
//Get the context for the loaded image
var inputContext = input.getContext("2d");
//get the image data;
var imageData = inputContext.getImageData(0, 0, input.width, input.height);
//Get the CanvasPixelArray
var data = imageData.data;
//Get length of all pixels in image each pixel made up of 4 elements for each pixel, one for Red, Green, Blue and Alpha
var arraylength = input.width * input.height * 4;
//Go through each pixel from bottom right to top left and alter to its gray equiv
//Common formula for converting to grayscale.
//gray = 0.3*R + 0.59*G + 0.11*B
for (var i=arraylength-1; i>0;i-=4)
{
//R= i-3, G = i-2 and B = i-1
//Get our gray shade using the formula
var gray = 0.3 * data[i-3] + 0.59 * data[i-2] + 0.11 * data[i-1];
//Set our 3 RGB channels to the computed gray.
data[i-3] = gray;
data[i-2] = gray;
data[i-1] = gray;
}
//get the output context
var outputContext = output.getContext("2d");
//Display the output image
outputContext.putImageData(imageData, 0, 0);
}
Notice here how they use a formula to calc gray. Depending on your needs you may want to use different ratios of the RGB mix to get the grayscale image.
Personally I would strongly recommend using vanilla JS here as it's very easy to do and you dont need to include OpenCV just to do grayscale which is a massive overhead to include that file for such a task. If you are using some of the more advanced features of OpenCV too then maybe that is a reason to then use it.
Related
I'm trying to write a script in Photoshop using JavaScript. The goal for this script is to automatically convert from RGB to Bitmap mode. I don't want use the manual way: image->mode->greyscale->image->mode->bitmap.
After reading the “JavaScript Scripting Reference” I understand that I need to do it the same as the manual way but using code.
I convert the RGB image to grayscale using the changeMode method. This works:
app.activeDocument.changeMode(ChangeMode.GRAYSCALE)
But, when I try to change from greyscale to bitmap I just can't. The document does not explain clearly how it works. I did try to follow the document but I don't have a lot of experience in JavaScript. Here is my current attempt:
```js
// Save the current preferences
var startRulerUnits = app.preferences.rulerUnits
var startTypeUnits = app.preferences.typeUnits
var startDisplayDialogs = app.displayDialogs
// Set Adobe Photoshop to use pixels and display no dialogs
app.preferences.rulerUnits = Units.PIXELS
app.preferences.typeUnits = TypeUnits.PIXELS
app.displayDialogs = DialogModes.NO
app.activeDocument.changeMode(ChangeMode.GRAYSCALE)
app.activeDocument.changeMode(ChangeMode.Bitmap)
// app.activeDocument.close()
Could you please provide a simple example about how to do it?
I am trying to detect some meter reading of an analogue meter. I am currently using the amazon recognition service to extract readings from a meter in a react-native app. The process did not work very well so as part of trying to fix this. I implemented a cropping functionality in the app so we send only relevant part of the image to the service. I run into another problem. The analogue separators on the meter are interspersed such that they are read as ones.
uncroppped meter image
uncroppped meter image
cropped image from the mobile app
cropped image from the mobile app
What I have tried. I created a simple server application to try to remove these lines before we send the image to rekognito
Converted the image to greyscale
Applied Gaussian blur to remove some of the noise.
Applied the [canny algortihm (https://en.wikipedia.org/wiki/Canny_edge_detector) to detect the edges.
using opencv for node
const { img } = req.params; // Mat
const grayWithGaussianBlur = img
.cvtColor(cv.COLOR_BGR2GRAY)
.gaussianBlur(new cv.Size(5, 5), 0, 0, cv.BORDER_DEFAULT)
.canny(30, 150);
The result look like this.
result
The output is as I expect. I have been trying to figure out how to remove the interspersed edges leaving the clearly defined edge.
I filtered the contours only leaving contours that meet specific criteria. Like area greater than a certain threshold,
const contours = grayWithGaussianBlur.copy().findContours(cv.RETR_TREE, cv.CHAIN_APPROX_NONE);
const viable = contours.filter(contour => {
const { width,height } = contour.boundingRect();
return width > 5 && width <= height; // example criteria
});
const newImage = new cv.Mat(grayWithGaussianBlur.rows, grayWithGaussianBlur.cols, 0);
newImage.drawContours(viable, new cv.Vec3(255, 255, 255), -1);
Can't get this to work.
My understanding of image processing concepts are very vague and I am unsure of this is a good way to fix this problem. I also don't know much about what I am doing :).
Sorry, I don't have enough reputations to embed the images directly.
Can anyone help or suggest a better approach to removing the lines. Thanks in advance.
How do we get the Pixel data from images in Elm?
Here in JavaScript, is code to get the color of a set of pixels in a figure (taken from here)
var image = new Image;
image.src = "starry-night.jpg";
var canvas = d3.select("body").append("canvas");
var context = canvas.node().getContext("2d");
context.drawImage(image, 0, 0);
// beware variable name "image" got used twice
image = context.getImageData(0, 0, width, height);
var x = Math.random()*width,
y = Math.random()*height,
i = (y * width + x) << 2;
pixelColor = d3.rgb(image.data[i + 0], image.data[i + 1], image.data[i + 2]) + "";
The code loads an image to a <canvas> element, then extracts the color of a pixel from the canvas using image.getImageData().
Can we interface the image.data object in Elm? Right now I don't think it's possible...
Right now Collage types are list of forms...
HTML is also a module that can put imags in the DOM.
SVG allows for some simple global image transformations but nothing at the pixel level
Elm has the Virtual Dom. In fact of problems like this, might be addressed in virtual-dom which is lower level so we are not encouraged to do this directly.
However, Elm makes a clear distinction between Collage elements and SVG elements, with no clear interface to the getImageData() function.
Do I write my own with Elm's new interOp feature?
Does a way already exist in Elm? Or a new one has to be written?
JavaScript
The << operator is called Left Shift
As suggested by #SimonH, use a port to JS until Elm provides a first-hand way to do so (if it ever does). The same approach would apply to anything you can't yet do in Elm.
I'm just answering as an answer rather than a comment for the sake of others who come here.
I'm trying to correctly save part of image which is highlighted with jcrop to a circle image.
I have canvas element which previews the selected area and how the image will look like, please check the screenshot below:
I also have hidden field which saves the value (example: "data:image/png;base64") which is displayed in the canvas.
I'm able to save image from the hidden field value with this code:
if (hfImageData.Value != string.Empty)
{
string value = hfImageData.Value;
if (value.Contains("jpeg"))
{
value = value.Replace("data:image/jpeg;base64,", "");
}
else if(value.Contains("png"))
{
value = value.Replace("data:image/png;base64,", "");
}
string path = Server.MapPath("/cropimages/");
string fileNameWitPath = path + DateTime.Now.ToString().Replace("/", "-").Replace(" ", "- ").Replace(":", "") + ".png";
using (FileStream fs = new FileStream(fileNameWitPath, FileMode.Create))
{
using (BinaryWriter bw = new BinaryWriter(fs))
{
byte[] data = Convert.FromBase64String(value);
bw.Write(data);
bw.Close();
}
}
}
This is the end result of that code:
What I really want to save is image in circle format as it is highlighted in the jcrop selection with jQuery/C#.
What do I need to modify in the existing code to make the image crop work as expected?
In general computer images are always stored as rectangular blocks of data. A "non-rectangular" images is a rectangular image with a non-rectangular mask or opacity "alpha" layer associated with it.
Per the jcrop online docs, jcrop doesn't do non-rectangular cropping-
Cropping Irregular Selections
If you actually want to crop a circle or an ellipse, you're on your
own. Jcrop will provide the rectangular coordinates for these crops,
and further processing can be done to extract the circle or ellipse
from the image.
If you're aiming to do the image manipulation on the client, then you would need to be working in an image format that supports an alpha channel (probably 32 bit: 8 bits for RGB and Alpha). You would need to apply the mask to the alpha channel in a canvas element. I think alpha support is fairly recent stuff in HTML5 so browser support is probably patchy.
You would then need to communicate that back to the host in a file format that supports alpha. JPEG doesn't, PNG (in 32 bit per pixel format) does.
Alternatively, if your server-side code "knows" the shape of the selection crop mask, you can ship the full (rectangular) image back to the server and have your server-side code apply the correct mask shape, using something like GD in PHP.
I'm building a simple game engine in javascript, and I'm currently building separate spritesheets for left and right animations, but that seems a bit of a pain... What I want to do is something like this:
function loadSprite(graphic)
{
var left_graphic = graphic;
var right_graphic = graphic.flip(); //Create a flipped copy of the graphic
}
// [...]
function update()
{
if(speed>0) context.drawImage(left_graphic);
if(speed<0) context.drawImage(right_graphic);
}
To clarify, I want to create a copy of an Image() object, and mirror flip all it's pixels, so I don't have to maintain two spritesheets.
Is it possible?
First off, you can copy the pixel data using the method described here: Get image data in JavaScript?
I assume you're using a sprite sheet with fixed-sized images. Then you'll have to write an algorithm to flip each image in your image sheet:
for each image in the sheet:
calculate sheet offset (xOffset, yOffset)
for each pixel in half the image:
read pixel ((x, y) + (xOffset, yOffset))
read opposite pixel ((width - x, y) + (xOffset, yOffset))
write on top of opposite pixel
write opposite pixel on top of current pixel