I'm trying to get rendering to a floating point texture working in WebGL on iOS Safari (not in a native app). I have managed to get iOS to read a manually (e.g. from JavaScript) created floating point texture, however when I create a framebuffer of floating point type and use the GPU to render into it, it does not work.
I've isolated the issue to code that renders to a floating point texture, which is then passed to another shader to be displayed. Here is what the result looks like applied to a cube:
The render to texture draws a green square, half the size of the texture, which is then applied to each side of the cube.
This all works perfectly fine on both desktop and iOS WebGL as long as the type of the texture that the green square is rendered to is the standard unsigned byte type. However, changing the type to floating point causes the render to texture to fail on iOS devices (while continuing to work on desktop browsers). The texture is empty, as if nothing had been rendered to it.
I have created an example project here to demonstrate the issue: https://github.com/felixpalmer/render-2-texture
Changing the precision of the shaders using the THREE.Renderer.precision setting does not make a difference
As far as I know no iOS device supports rendering to a floating point texture (nor do most mobile devices at this point in time 3/2015)
My understanding of the WebGL spec is
OES_texture_float: Allows you to create and read from 32bit float textures but rendering to a floating point is device dependent.
OES_texture_float_linear: Allows linear filter floating point textures. If this doesn't exist and OES_texture_float does then you can only use gl.NEAREST for floating point textures.
OES_texture_half_float and OES_texture_half_float_linear are the same as above except for half float textures.
The traditional way to see if you can render to a floating point texture in WebGL, assuming OES_texture_float exists, is to create a framebuffer, attach a floating point texture to it, then call gl.checkFramebufferStatus. If it returns gl.FRAMEBUFFER_COMPLETE then you can, if not then you can't. Note: This method should work regardless of the next paragraph.
The spec was updated so you could also check WebGL extensions to find out if it's possible to render to a floating point texture. The extension WEBGL_color_buffer_float is supposed to tell you you can render to floating point textures. The extension EXT_color_buffer_half_float is the same for half float textures. I know of no browser that actually shows these extensions though yet they support floating point rendering if the hardware supports it.
For example my 2012 Retina MBP on Chrome 41 reports
gl = document.createElement("canvas").getContext("webgl").getSupportedExtensions()
["ANGLE_instanced_arrays",
"EXT_blend_minmax",
"EXT_frag_depth",
"EXT_shader_texture_lod",
"EXT_sRGB",
"EXT_texture_filter_anisotropic",
"WEBKIT_EXT_texture_filter_anisotropic",
"OES_element_index_uint",
"OES_standard_derivatives",
"OES_texture_float",
"OES_texture_float_linear",
"OES_texture_half_float",
"OES_texture_half_float_linear",
"OES_vertex_array_object",
"WEBGL_compressed_texture_s3tc",
"WEBKIT_WEBGL_compressed_texture_s3tc",
"WEBGL_debug_renderer_info",
"WEBGL_debug_shaders",
"WEBGL_depth_texture",
"WEBKIT_WEBGL_depth_texture",
"WEBGL_lose_context",
"WEBKIT_WEBGL_lose_context"]
Firefox 36 reports
gl = document.createElement("canvas").getContext("webgl").getSupportedExtensions().join("\n")
"ANGLE_instanced_arrays
EXT_blend_minmax
EXT_frag_depth
EXT_sRGB
EXT_texture_filter_anisotropic
OES_element_index_uint
OES_standard_derivatives
OES_texture_float
OES_texture_float_linear
OES_texture_half_float
OES_texture_half_float_linear
OES_vertex_array_object
WEBGL_compressed_texture_s3tc
WEBGL_depth_texture
WEBGL_draw_buffers
WEBGL_lose_context
MOZ_WEBGL_lose_context
MOZ_WEBGL_compressed_texture_s3tc
MOZ_WEBGL_depth_texture"
The browser vendors are busy implementing WebGL 2.0 and given the gl.checkFramebufferStatus method works there's no pressure to spend time making the other extension strings appear.
Apparently some iOS devices support EXT_color_buffer_half_float so you could try creating a half float texture, attach it to a framebuffer and check its status then see if that works.
Here's a sample to check support. Running it on my iPadAir2 and my iPhone5s I get
can make floating point textures
can linear filter floating point textures
can make half floating point textures
can linear filter floating point textures
can **NOT** render to FLOAT texture
successfully rendered to HALF_FLOAT_OES texture
which is exactly what we expected.
"use strict";
function log(msg) {
var div = document.createElement("div");
div.appendChild(document.createTextNode(msg));
document.body.appendChild(div);
}
function glEnum(gl, v) {
for (var key in gl) {
if (gl[key] === v) {
return key;
}
}
return "0x" + v.toString(16);
}
window.onload = function() {
// Get A WebGL context
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");
if (!gl) {
return;
}
function getExt(name, msg) {
var ext = gl.getExtension(name);
log((ext ? "can " : "can **NOT** ") + msg);
return ext;
}
var testFloat = getExt("OES_texture_float", "make floating point textures");
getExt("OES_texture_float_linear", "linear filter floating point textures");
var testHalfFloat = getExt("OES_texture_half_float", "make half floating point textures");
getExt("OES_texture_half_float_linear", "linear filter half floating point textures");
gl.HALF_FLOAT_OES = 0x8D61;
// setup GLSL program
var program = webglUtils.createProgramFromScripts(gl, ["2d-vertex-shader", "2d-fragment-shader"]);
gl.useProgram(program);
// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");
var colorLoc = gl.getUniformLocation(program, "u_color");
// provide texture coordinates for the rectangle.
var positionBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
-1.0, -1.0,
1.0, -1.0,
-1.0, 1.0,
-1.0, 1.0,
1.0, -1.0,
1.0, 1.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
var whiteTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, whiteTex);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE,
new Uint8Array([255, 255, 255, 255]));
function test(format) {
var tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, format, null);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
var fb = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0);
var status = gl.checkFramebufferStatus(gl.FRAMEBUFFER);
if (status !== gl.FRAMEBUFFER_COMPLETE) {
log("can **NOT** render to " + glEnum(gl, format) + " texture");
return;
}
// Draw the rectangle.
gl.bindTexture(gl.TEXTURE_2D, whiteTex);
gl.uniform4fv(colorLoc, [0, 10, 20, 1]);
gl.drawArrays(gl.TRIANGLES, 0, 6);
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.clearColor(1, 0, 0, 1);
gl.clear(gl.COLOR_BUFFER_BIT);
gl.uniform4fv(colorLoc, [0, 1/10, 1/20, 1]);
gl.drawArrays(gl.TRIANGLES, 0, 6);
var pixel = new Uint8Array(4);
gl.readPixels(0, 0, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixel);
if (pixel[0] !== 0 ||
pixel[1] < 248 ||
pixel[2] < 248 ||
pixel[3] < 254) {
log("FAIL!!!: Was not able to actually render to " + glEnum(gl, format) + " texture");
} else {
log("succesfully rendered to " + glEnum(gl, format) + " texture");
}
}
if (testFloat) {
test(gl.FLOAT);
}
if (testHalfFloat) {
test(gl.HALF_FLOAT_OES);
}
}
canvas {
border: 1px solid black;
}
<script src="//webglfundamentals.org/webgl/resources/webgl-utils.js"></script>
<canvas id="c" width="16" height="16"></canvas>
<!-- vertex shader -->
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec4 a_position;
void main() {
gl_Position = a_position;
}
</script>
<!-- fragment shader -->
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
uniform vec4 u_color;
uniform sampler2D u_texture;
void main() {
gl_FragColor = texture2D(u_texture, vec2(0.5, 0.5)) * u_color;
}
</script>
Related
I tryed to put data in the fragment shader to color 2 triangles through textures, but the color given is black (maybe default?), not green or red as expected.
The source from where I tryed to do that is https://webglfundamentals.org/webgl/lessons/webgl-shaders-and-glsl.html#textures-in-fragment-shaders. I put everything from there in my source and I thought that if I put a green pixel with a red pixel in my texture, then it needs to have a different color than black, but was black.
That's the fragment shader:
precision mediump float;
uniform sampler2D u_texture;
void main() {
vec2 texcoord = vec2(0.5, 0.5) // get a value from the middle of the texture
gl_FragColor = texture2D(u_texture, texcoord);
}
There I created the texture and I put data in it.
var tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
var level = 0;
var width = 2;
var height = 1;
var data = new Uint8Array([
255, 0, 0, 255, // a red pixel
0, 255, 0, 255, // a green pixel
]);
gl.texImage2D(gl.TEXTURE_2D, level, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
I tried to put different data in the texture with even another colors, but the result was the same.
You've to set the texture minifying function to a "non mipmap" filter, else the texture is not mipmap complete. e.g.:
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
By default the minifing parameter is NEAREST_MIPMAP_LINEAR. This parameter value requires a mipmap for the texture, to be complete.
The other option is to generate mipmaps by:
gl.generateMipmap(gl.TEXTURE_2D)
But for that the width and the height of the texture have to be a power of two (for WebGL 1.0). See WebGL specification - 5.13.8 Texture objects.
Note, if a texture is not mipmap complete, then the return value of a texel fetch is (0, 0, 0, 1), this causes the "black" color.
I've created a canvas with width=16 and height=16. Then I used WebGL to render an image to it. This is what it looks like:
Afterwards, I scaled the canvas by using width: 256px and height: 256px. I also set image-rendering to pixelated:
canvas {
image-rendering: optimizeSpeed; /* STOP SMOOTHING, GIVE ME SPEED */
image-rendering: -moz-crisp-edges; /* Firefox */
image-rendering: -o-crisp-edges; /* Opera */
image-rendering: -webkit-optimize-contrast; /* Chrome (and eventually Safari) */
image-rendering: pixelated; /* Chrome */
image-rendering: optimize-contrast; /* CSS3 Proposed */
-ms-interpolation-mode: nearest-neighbor; /* IE8+ */
width: 256px;
height: 256px;
}
This is the result:
The image is blurred. Why? I'm using Safari 12.0.2 on OSX Mojave.
Safari does not yet support image-rendering: pixelated; on WebGL. Filed a bug
Also crisp-edges does not != pixelated. crisp-edges could be any number of algorithms. It does not mean pixelated. It means apply some algorithm that keeps crisp edges of which there are tons of algorithms.
The spec itself shows examples:
Given this image:
This is pixelated:
IMPORTANT: See update at bottom
Where as a browser is allowed to use a variety of algorithms for crisp-edges so for example the result could be
So in other words your CSS may not produce the results you expect. If a browser doesn't support pixelated but does support crisp-edges and if they use an algorithm like above then you won't a pixelated look.
The most performant way to draw pixelated graphics without image-rendering: pixelated is to draw to a small texture and then draw that texture to the canvas with NEAREST filtering.
const vs = `
attribute vec4 position;
void main() {
gl_Position = position;
}
`;
const fs = `
precision mediump float;
void main() {
gl_FragColor = vec4(1, 0, 0, 1);
}
`;
const screenVS = `
attribute vec4 position;
varying vec2 v_texcoord;
void main() {
gl_Position = position;
// because we know position goes from -1 to 1
v_texcoord = position.xy * 0.5 + 0.5;
}
`;
const screenFS = `
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_tex;
void main() {
gl_FragColor = texture2D(u_tex, v_texcoord);
}
`;
const gl = document.querySelector('canvas').getContext('webgl', {antialias: false});
// compile shaders, link programs, look up locations
const programInfo = twgl.createProgramInfo(gl, [vs, fs]);
const screenProgramInfo = twgl.createProgramInfo(gl, [screenVS, screenFS]);
const width = 16;
const height = 16;
const tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
const fb = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0);
// create buffers and put data in
const quadBufferInfo = twgl.createBufferInfoFromArrays(gl, {
position: {
numComponents: 2,
data: [
-1, -1,
1, -1,
-1, 1,
-1, 1,
1, -1,
1, 1,
],
}
});
render();
function render() {
// draw at 16x16 to texture
gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
gl.viewport(0, 0, width, height);
gl.useProgram(programInfo.program);
// bind buffers and set attributes
twgl.setBuffersAndAttributes(gl, programInfo, quadBufferInfo);
gl.drawArrays(gl.TRIANGLES, 0, 3); // only draw the first triangle
// draw texture to canvas
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
gl.useProgram(screenProgramInfo.program);
// bind buffers and set attributes
twgl.setBuffersAndAttributes(gl, screenProgramInfo, quadBufferInfo);
// uniforms default to 0 so in this simple case
// no need to bind texture or set uniforms since
// we only have 1 texture, it's on texture unit 0
// and the uniform defaults to 0
gl.drawArrays(gl.TRIANGLES, 0, 6);
}
<canvas width="256" height="256"></canvas>
<script src="https://twgljs.org/dist/4.x/twgl.min.js"></script>
Note: if you're rendering 3D or for some other reason need a depth buffer you'll need to add a depth renderbuffer attachment to the framebuffer.
Note that optimizeSpeed is not a real option either. It's been long deprecated and like crisp-edges is up to the browser to interpret.
Update
The spec changed in Feb 2021. crisp-edges now means "use nearest neighbor" and pixelated means "keep it looking pixelated" which can be translated as "if you want to then do something better than nearest neighbor that keeps the image pixelated". See this answer
This is a very old Webkit bug, from before the Blink fork happened.
Since then, Blink fixed it, Webkit still hasn't.
You may want to let them know it's still a problem by commenting on the still open issue.
As for a workaround, there are several, but no perfect one.
The first one would be to draw your scene at the correct size directly and make the pixelation yourself.
An other one would be to render your webgl canvas on a 2d canvas (that you would resize using your CSS trick, or render directly at the correct size using the 2d context imageSmoothingEnabled property.
CSS-Houdini will probably allow us to workaround this issue ourselves.
But the real problem here is to find out if you need this workaround. I don't see any mean to feature-test this case (at least without Houdini), so this means that you'd either have to do ugly user-agent detection, or to apply the workaround to everyone.
I want to save a calculated value from fragment shader in some variable ,so that I would be able to use it next time.
Currently, I am preparing a image using a huge algorithm and I want to save it to some vec4 and , once requested again , I want to just get that vec4 and should say
gl_FragColor = vec4(previously saved variable)
This question is related to another question here which is also asked by me , but I feel that if this question has a answer then I can easily crack the other one.
Any suggestions ?
Fragment shaders in WebGL write to 1 of 2 things. Either (1) the canvas to (2) the attachments of a framebuffer. The attachments of a framebuffer can be textures. Textures can be used as inputs to a shader. Therefore you can write to a texture and use that texture in your next draw.
Here's an example
var vs = `
attribute vec4 position;
void main() {
gl_Position = position;
}
`;
var fs = `
precision mediump float;
uniform sampler2D u_texture;
void main() {
// just grab the middle pixel(s) from the texture
// but swizzle the colors g->r, b->g, r->b
gl_FragColor = texture2D(u_texture, vec2(.5)).gbra;
}`;
var canvas = document.querySelector("canvas");
var gl = canvas.getContext("webgl");
var program = twgl.createProgramFromSources(gl, [vs, fs]);
var positionLocation = gl.getAttribLocation(program, "position");
// we don't need to look up the texture's uniform location because
// we're only using 1 texture. Since the uniforms default to 0
// it will use texture 0.
// put in a clipspace quad
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
-1, -1,
1, -1,
-1, 1,
-1, 1,
1, -1,
1, 1,
]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
// make 2 1x1 pixel textures and put a red pixel the first one
var tex1 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex1);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA,
gl.UNSIGNED_BYTE, new Uint8Array([255, 0, 0, 255]));
var tex2 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex2);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA,
gl.UNSIGNED_BYTE, null);
// make a framebuffer for tex1
var fb1 = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, fb1);
// attach tex1
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0,
gl.TEXTURE_2D, tex1, 0);
// check this will actually work
if (gl.checkFramebufferStatus(gl.FRAMEBUFFER) !==
gl.FRAMEBUFFER_COMPLETE) {
alert("this combination of attachments not supported");
}
// make a framebuffer for tex2
var fb2 = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, fb2);
// attach tex2
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0,
gl.TEXTURE_2D, tex2, 0);
// check this will actually work
if (gl.checkFramebufferStatus(gl.FRAMEBUFFER) !==
gl.FRAMEBUFFER_COMPLETE) {
alert("this combination of attachments not supported");
}
function render() {
gl.useProgram(program);
// render tex1 to the tex2
// input to fragment shader
gl.bindTexture(gl.TEXTURE_2D, tex1);
// output from fragment shader
gl.bindFramebuffer(gl.FRAMEBUFFER, fb2);
gl.viewport(0, 0, 1, 1);
gl.drawArrays(gl.TRIANGLES, 0, 6);
// render to canvas so we can see it
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
// input to fragment shader, the texture we just rendered to
gl.bindTexture(gl.TEXTURE_2D, tex2);
gl.drawArrays(gl.TRIANGLES, 0, 6);
// swap which texture we are rendering from and to
var t = tex1;
tex1 = tex2;
tex2 = t;
var f = fb1;
fb1 = fb2;
fb2 = f;
requestAnimationFrame(render);
}
requestAnimationFrame(render);
<script src="https://twgljs.org/dist/twgl-full.min.js"></script>
<canvas></canvas>
The sample above puts red in a texture. It then renders that texture by swizzling the color. Green goes to the red channel, Blue goes to the green channel, Red goes to the Blue channel.
I makes 2 textures and attaches them to 2 framebuffers.
First iteration
tex1 = red
tex2 = 0,0,0,0
render to fb2
tex2 is now blue (because red was copied to blue)
render tex2 to canvas (canvas is now green because blue is copied to green)
switch which textures we're rendering to
Second iteration
tex1 = blue (was tex2 last time)
tex2 = red (was tex1 last time)
render to fb2 (was fb1 last time)
tex2 = green (because blue is copied to green)
render tex2 to canvas (canvas is now red because green is copied to red)
switch which textures we're rendering to
Fragment shader executes per fragment(pixel).And as any other shader it cannot store values by default, as you would expect in regular programming language.
There are several ways to do what you want:
You can use imageLoad/Store ,which allows you to read and write data from shader into image.Image uses GL textures as memory storage.What is good about it is that you can store and load numeric data without losing precision when using images because texture filtering is disabled when accessing texture data via image.
Another way to store and read data in shaders is using buffers.Uniform buffers,or since GL4.3 Shader storage buffers..
SSBO allows to read and write huge amount of data.It is really up to you to decide which of those to use for storing and retrieving your data in the shaders.Some people say texture memory access is faster on some hardware.From my experience,using SSBO vs image load store,I haven't found significant difference in performance on Nvidia GPUs.
In your scenario I would probably go with Image Load/Store.Because you can use the same UV indexing into image data as you do into sampled texture.
Also,I don't really know what version of OpenGL you are using ,but to use these extensions you must use GL4.2 and GL4.3.
When using textures in WebGL, sometimes I need to make them larger than they were originally. When I do that, it causes the textures to appear differently, especially on lighter backgrounds.
I have the following image (256 x 256):
When rendered in WebGL, it is slightly larger than the original image. Here is how the image appears on two different backgrounds:
As you can see, the image appears correctly on the dark background, but when on the light background, has a white outline.
My setup code:
gl.clearColor(0x22 / 0xFF, 0x22 / 0xFF, 0x22 / 0xFF, 1); // set background color
gl.enable(gl.BLEND); // enable transparency
gl.disable(gl.DEPTH_TEST); // disable depth test (causes problems with alpha if enabled)
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA); //set up blending
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); //clear the gl canvas
gl.viewport(0, 0, canvas.width, canvas.height); //set the viewport
And this is the code called every time a texture is loaded:
function handleTextureLoaded(image, texture) {
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_NEAREST);
gl.generateMipmap(gl.TEXTURE_2D);
gl.bindTexture(gl.TEXTURE_2D, null);
loadCount++;
}
What is causing the outline to appear, and how do I fix it?
NOTE: When I put the original image on these same two backgrounds, this problem does not occur, even when I resize the image.
I tried disabling the alpha on the WebGL context (as told by #zfedoran):
gl = canvas.getContext('webgl', {antialias: false, alpha: false })
|| canvas.getContext('experimental-webgl', {antialias: false, alpha: false });
And a small blank border now appears around the image, like this (enlarged):
On top of the canvas's alpha as mentioned by #zfedoran how do you make the original image?
I believe the issue is as follows
Let's say you have an anti-aliased edge like this. What color is this pixel?
Assume the main color, the color of the pixels in the bottom right, was 1,0,0 (pure red). Ideally the pixel pointed to by the arrow would be (1,0,0,0.5). In other words, pure red with an alpha of 0.5. But, depending how on the image was created to generate that anti-aliased pixel it might have been blended with the purely transparent pixels next to it so it no longer pure red. Those purely transparent pixels are likely (0,0,0,0) which is transparent black.
Even if your drawing program handles this correctly, GL likely does not. When you draw an image with texture filtering on (gl.LINEAR etc) GL is going to average the pixels near each other, some of those pixels are transparent black. Blending black with red gives dark red. Hence you get a dark border.
Here you can see the issue
"use strict";
function main() {
var planeVertices = [
-1, -1,
1, -1,
-1, 1,
1, 1,
];
var texcoords = [
0, 1,
1, 1,
0, 0,
1, 0,
];
var indices = [
0, 1, 2,
2, 1, 3,
];
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl", {alpha:false});
var programs = {}
programs.normalProgram = twgl.createProgramFromScripts(
gl, ["2d-vertex-shader", "2d-fragment-shader"], ["a_position", "a_texcoord"]);
programs.preMultiplyAlphaProgram = twgl.createProgramFromScripts(
gl, ["2d-vertex-shader", "pre-2d-fragment-shader"], ["a_position", "a_texcoord"]);
var positionLoc = 0; // assigned in createProgramsFromScripts
var texcoordLoc = 1; // assigned in createProgramsFromScripts
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(
gl.ARRAY_BUFFER,
new Float32Array(planeVertices),
gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLoc);
gl.vertexAttribPointer(positionLoc, 2, gl.FLOAT, false, 0, 0);
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(
gl.ARRAY_BUFFER,
new Float32Array(texcoords),
gl.STATIC_DRAW);
gl.enableVertexAttribArray(texcoordLoc);
gl.vertexAttribPointer(texcoordLoc, 2, gl.FLOAT, false, 0, 0);
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, buffer);
gl.bufferData(
gl.ELEMENT_ARRAY_BUFFER,
new Uint16Array(indices),
gl.STATIC_DRAW);
var img = new Image();
img.onload = createTextures;
img.src = document.getElementById("i").text;
function createTexture() {
var tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);
gl.generateMipmap(gl.TEXTURE_2D); // assuming power-of-2
return tex;
}
var textures = {};
function createTextures() {
gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
textures.unpremultipliedAlphaTexture = createTexture();
gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true);
textures.premultipliedAlphaTexture = createTexture();
document.body.appendChild(document.createElement("hr"));
insert("original image");
document.body.appendChild(img);
render();
}
function insert(text) {
var pre = document.createElement("pre");
pre.appendChild(document.createTextNode(text));
document.body.appendChild(pre);
};
function grabImage(prg, blend, texName) {
document.body.appendChild(document.createElement("hr"));
insert(
"gl.useProgram(" + prg + ")\n" +
"gl.blendFunc(gl." + blend.src + ", gl." + blend.dst + ")\n" +
"gl.bindTexture(gl.TEXTURE2D, " + texName + ")");
var img = new Image();
img.src = gl.canvas.toDataURL();
document.body.appendChild(img);
};
function render() {
gl.enable(gl.BLEND);
Object.keys(programs).forEach(function(p, pndx) {
gl.useProgram(programs[p]);
[
{ src: "SRC_ALPHA", dst: "ONE_MINUS_SRC_ALPHA" },
{ src: "ONE", dst: "ONE_MINUS_SRC_ALPHA" },
].forEach(function(b, bndx) {
gl.blendFunc(gl[b.src], gl[b.dst]);
Object.keys(textures).forEach(function(texName, tndx) {
gl.bindTexture(gl.TEXTURE_2D, textures[texName]);
gl.clearColor(0x3D/0xFF, 0x87/0xFF, 0xEA/0xFF, 1);
gl.clear(gl.COLOR_BUFFER_BIT);
gl.drawElements(gl.TRIANGLES, 6, gl.UNSIGNED_SHORT, 0);
grabImage(p, b, texName);
});
});
});
}
}
main();
canvas {
border: 1px solid black;
display: none;
}
img {
background-color: #3D87EA;
border: 1px solid black;
width: 256px;
height: 256px;
}
<script src="https://twgljs.org/dist/3.x/twgl.min.js"></script>
<!-- vertex shader -->
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec4 a_position;
attribute vec2 a_texcoord;
varying vec2 v_texcoord;
void main() {
gl_Position = a_position;
v_texcoord = a_texcoord;
}
</script>
<!-- fragment shaders -->
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_texture;
void main() {
gl_FragColor = texture2D(u_texture, v_texcoord);
}
</script>
<script id="pre-2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_texture;
void main() {
vec4 textureColor = texture2D(u_texture, v_texcoord);
gl_FragColor = vec4(textureColor.rgb * textureColor.a, textureColor.a);
}
</script>
<canvas id="c" width="32" height="32"></canvas>
<script type="not-js" id="i">data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAA7dpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wUmlnaHRzPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvcmlnaHRzLyIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iIHhtcFJpZ2h0czpNYXJrZWQ9IkZhbHNlIiB4bXBNTTpPcmlnaW5hbERvY3VtZW50SUQ9InhtcC5kaWQ6RjgxNENDMDEzQjNGNjgxMTgyMkFCRTQ0RTFGNjIxOTciIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6Q0VDQTU5RDNGNjBEMTFFMjhFRUVEMkI5NkRDNTM4RDYiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6Q0VDQTU5RDJGNjBEMTFFMjhFRUVEMkI5NkRDNTM4RDYiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNiAoTWFjaW50b3NoKSI+IDx4bXBNTTpEZXJpdmVkRnJvbSBzdFJlZjppbnN0YW5jZUlEPSJ4bXAuaWlkOkRGODJCQjcxQkIyNDY4MTE4MjJBRkMwRDVCNTc4NTk3IiBzdFJlZjpkb2N1bWVudElEPSJ4bXAuZGlkOkY4MTRDQzAxM0IzRjY4MTE4MjJBQkU0NEUxRjYyMTk3Ii8+IDwvcmRmOkRlc2NyaXB0aW9uPiA8L3JkZjpSREY+IDwveDp4bXBtZXRhPiA8P3hwYWNrZXQgZW5kPSJyIj8+7i5UrAAAEShJREFUeNrsnQmQFcUZx2cVlVslFeKKIAYlWtbqeoEiiiKoFTVIQC7PBJUkSgxYiBfGVBCBEBOC0ZhEK4emSErBijEeiVhRVEQUqcXyQGC9o3J5BhTZ/D/fEDe6y5uZ995M97zfr+qrVvb1e/P69ffv7+uZ7q5pamoKAKA62Y4mAEAAAAABAAAEAAAQAABAAAAAAQAABAAAEAAAQAAAAAEAAAQAABAAAEAAAAABAAAEAAAQAABAAAAAAQAABAAAEAAAQAAAAAEAAAQAABAAAEAAAAABAAAEAAAQAABAAAAAAQAABAAAEAAAaJ02Ll5UTU0Nv0yFaGhosMbdW1Yv21+2j6yHrFbWRbbr56q8I/tQtkHWKFsd2jOyx+vq6tbRqtFoampyz9ecvCgEoNxOb05+kuxY2VEtOHkprJAtkt0r+7sEYQMtjgAgANk7/VdVnCEbJdsvpY/dLHtIdrvsNonBu/wSCAACkJ7T25zON2QXyAZlfDmWNsyV3SghWMKvgwAgAJVz/DbhaH+ZrLeDl3if7GoJwSIEAAFAAMrr/N9UMT0oTOa5jgnBZAnBMgQAAUAASnP8r6m4QTbQs0v/JLzuKRKCdxAABAABiOf429soKrtKtpPHX+XfsnESgb8iAAgAAhDN+XuquE3WL0df60bZRAnBRgQAAUAAWnf+r6u4NSjvPXxXsIeKhksEnkMA0oVHgf1w/ktU3JVT5zfsicRF+p4n8msTARAB/H++P0f23Sr5yjZBeL4igVuIAIgAcP4g+EMVOb9h3/lmffcJ9AAEAOcPgjFV2gTXqQ0upieQAlSj89uXv1n2LbpncJ7Sgd+SAhABVBNTcf7/cZME8WSagQigWkb/04PCrT74jPdlRygSWE4EgADk2fkPUPG4rC0+/wVWyuolAu8jAKQAeXT+jkFhDT3O3zK9gsL6AUAAcsnPAj9W82XJmRLKMTQDKUDeRv/BKu6nK0bCdhjqrVTgTVIAIoA8OH87FTfREpHpLJtFMyAAeWGSbC+aIRZnSDgH0AykAL6P/rYNt81ut6MbxuZp2cFKBZp8umhSAGjOlTh/YuxMgyE0AxGAr6N/t6BwuMYOdMHqiQKIAGArE3D+skQBp9AMCIBvo38HFefSEmXhQpoAAfCN0bKdaYayMCg8AQkQAG9gpV/5sMmicTQDAuBL+G+jVT9aoqyMogmS04YmSJWhNEHZ6SFhPaiurm5pC4JrOyt1C20PWYdmf94keyMonE+wWvU3IQBQabh3XRlOlS2Vw9vcip2U1F92mOwQWfsI9Terrm1N/pTsAdk9EoR1VZFD8RxAauG/Lfldj+hWhNdlL8n6BIWNRUtli+xhme1O/JdyHVrChiDVLQB2sMfd+Kp3mGjbgq1ZEoK1eRMAJgHT4wiawEvsMJZLZSsl4lfJ2ufpyyEA6dGHJvAam1/4kWxZnlYjkgLED+XtEd462UGyHrLdZF3DvPG90F4LCs/6P6GwcVVYz/5td/woN8yWTdLv+zFzADkXADmvOe5pspNkRwfxjuW22WSbUOIOQP54SDYi6g5FCIBnAhCGej8ICotOtqe/Qwu8IhssEXgeAciJAMjxLV+fLjuW/g0RWCMbKBFoQAA8FoDwXv20oLDKjCOKIQ6WBvTbOueDAHgmAOHBHPNlrC6DpLwg6ysR2OCLAHAbsOD8NsH3GM4PJdJbdmt4wKsXbIfzN5yvYm4Q7ZlxgGLYnaLv+HKxVZ0CyPltZ57f0GehzHwosxWKL5ACuDvy24k8N9JXoQJYNDmbCMDRCEDOb4dx2PpxtuaCSnKCooD7iQDcGvntgZ7f4/yQAjNdnxCsxhRgvOwo+iakwIGy4xAAd0b/Lip+SL+EFLkIAXCHK2S70CchRU7SwNMDAch+9Lejpc+nP0LK2BzAEAQge8bKOtIfIQOGIQDZczb9EDLiKEWgnRCA7ML/XkFhRhYgKz/rgwBkx8n0QciYwxCA7OA4LsiaegQgO/rS/yBj9kQAssn/27ra+FBV7IEAZEMP+h44QC0CkA270ffAAbZHAPiOADhHynTgZwYHaEIAsuFd+h44wBoEIBvW0/fAAd528aLa5LGlw5V/dqqP7ft3An0PHOBlBKByDm/fo2/o8IPD/+YsP3CJ5QhAeZ3eDmE4PnT4Y2Sd6WPgMEsRgNKd3u7pf092esApPuAXixGA5I5v4fwk2ZSAE3zAP1bW1dW96OK24G08cH7bvnuebCD9CDzlblcvrI3jzm9beNnBCn3oQ+Axt7t6Ya4/B3Azzg+eY+cDLkQA4o/+Q1WMoP+A5/xa+X8TAhDP+e26ptF3wHPWBo6fPu1qBGDHKe1L/wHPmaXR/10EID7D6DvgOW/Irnf9Il0VADbxBN+5UKP/+whAMvam/4DHzJfzz/PhQl0VgHb0IfCUV2XjfLlYVwVgC/0IPOQj2TCN/m8jAKXxOn0JPGScnH+xTxfsqgAspS+BZ0yW8//Ot4t2VQDupz+BR0yS88/08cJdFYC5sk30K/Ag5z9Lzj/L1y/gpACoQW0H1V/Rv8BhGmUD1Ff/6POXcHk14NVB4WkqANcwpz9Qzr/I9y9S4+IuJTU1NZ+WDQ0N/VUskO1AnwMHeE42UY5/T5LKLvqa0/sBqKFtHfVw5gMgY2xL7/GFLpnM+YkAEkQAW1EkcJiKPwU8Igzp8ojsF7J5cvzNpb6Zk77mgwCEImCbgV4ku0DWjb4JFcCW7j4pmy+7U07/SjnfHAEoQQCaCYHtEDxIdrSsPigc/tkp4CAQiMaGoLBRx7qgcFyXhfevhPn9qkru3oMAlEEAAHyFSUAAQAAAAAEAAAQAABAAAEAAAAABAIAUaEMTpMtbjZ8eb753yuL7QdeewQpaHz4PDwKl6/xfUtEgq83g4+dIBL7vc/uNf/QNe9rzYNn6Of1qX/Tt+nkQCEZk5PxGJ8+dv7OKxaGt0P9fR3dCAHxjTEaf2yib6HnbzQhH/61MkAjsQ5dCAHwJ/3uo6J/BR38sG6nwf73Ho3+divNa+FMXehYC4AujMvrcS+T8iz1vu58HX1ztuVG2jG6FAPjC6Aw+807ZbM9z/1NVDGzhTwvn9KvdSLdCAHwI//cNCnsXpJ33f1ujf5PHzr+jita23P4HPQsB8IW0J/9sv/rhPuf9IbYDVK9W/vZPuhUCQP7fMhfL+Z/0PPT/iooprfzZdvR5mm6FAPgQ/h+qIs3bVXfI+a/PQdNNDVp/dmGB8n9OkEYAvCDNyb9VsrG+N5hG//oi34P8HwHwYvTfLsXw3/L+ERr938lB09ltv209D07+jwB4ge1cvDt5f6zR/zQVA7bxktUK/1fTtRAAH0hr9M9F3i/nb6ui2DHbjP4IgBfhv51nOIK8PxYTZD2LvIb8HwHwghNku5L3Rx79bZXk5UVeZg81LaBrIQA+kMbsfy7y/pBrZR2LvGap8v+1dC0EwPXwv52KIeT9kUd/e1birAgvJf9HALzAnL8DeX8k57fbfcVu+5H/Vwj2BPQz/LetxZYp0ijG+jBNcDlvHik7MsLrNskW0rUQANfD/11UnFjhj9k5tGLsGRQWIjkpABr9LVWaGfHlLP8lBfCCYbIdHbkWOwr7GofbapKse8TXkv8jAF4wxqFrsf0AnHxqTqN/NxWTY1RBABAA58N/u5d9jCOXM1vOP9/h5pouax/xtTaX8RQ9DAFwnRGOtOkTsktcbSSN/oerOCNGFZb/IgBeMNqBa7C833YB/shR59962y8O3P5DAJwP/23rqr7k/UUZk6CdyP8RAOcZSd5fdPS3h6NmxKzWqPB/Jd0LAXCdrGf/nc77Q2zWvxujPwKQt/DfTq7Zn7x/m6O/nYw0KUFV8n8EgNHf87w/CEP/tjHrsPwXAXB+9LdZ7VEZXoLr9/tt9D8yYRstU/6/hl6GALjMAUHxXWwqxaKEYXWazm99LOnxZOT/FYbFQKWzKsxT+0R8va0TaFeGz10f5v0fO94+ts7/kIR1yf8RALeRA76n4vgYKYNt4nFBGT76bH32y46P/rbDz7SE1Vn+SwqQu/kCO+K6HBuF/lTOf5cHX/kKWW3Cuo8q//+QXkMEkCcGyb5chrz/shRGb5vctP35uwSFZ/E3xKy/V1DY5Tcp5P9EALmj1LUCaeb9t8gelN0hWy6H7h6zvm30sVMJn0/+jwDkKvw3ZxjqQ94vZz9VxTnN/sme3jsvRn07EWl4iULH8l8EIFecLOvset4v57W7FLNa+FP7GG8zs8TLeFApxyd0GQQgT5TysFAqeX/IRbJeLfx7pBn58GTfUldFkv8jALkK/zuHEYDTeb+ct6uKK1v405ZwPiAKx5ThUhAABCBX2DkBbRPWTfN+/zWtpCmLFZJHPX5snxKv4WV91gq6DAKQJ053Oe9vFrqPLcOIXGqkwuiPAOQq/Lf7/sc5nvcb2zqdJ45TvokAIADwGXY7LO4DV6k+56/R384yGNDKn+1pvMdivN2/SrgU28/gProMApAnkuwVkFreL+e3uYlZ23Jo5eRxNhoxsVie8HJu12eto8sgAHkJ/20XnCNjVvtJys/52+O6PcsVksuBbROPJEuUPwgKawcAAcgNI4Nop95u5RHZ5WldnEb/3SJ8XuycXCJwr4qpMarYbcaxqtdIl0EA8kScZ//XykZp9N+c4vVdK+u4jb+/LWtI8sZy5ilhJFBsHsN2/Bmi1/+Z7oIA5Cn831fFQTGqnCnnfzXF0f9Qm2soNvqHIX2QUARsbsE2S7U7DJ//bs/KrpLtp9f9jR6TDSwHdmP0nyHnvydF5996Ok+x9KTkW3LhQz02zzBBn2tHp9uDRmv17x/QRRCAPBP12X/L+69M+dpsU5Iok5NlvScf7imwga7hDjVNTU3uXVRNjdeNqvDf9sBbEjHvr0859Lf9CJ+T9Sjy0hVy2N64SPlw0deYA6gMUe/9p5r3h0yK4PwGG3JUAQhA+Ud/a9Mo5wSmmveHo79t7DE54st5JBcBgAT0D4qff5dF3m/Ybb8oG3vEWf4LCADECP+zuN9vo//hlnJEfPmSuJuAAgJA+N8Y7KDiNNfy/ma3/aJC/o8AQAIGB4VttJ3J+5tFJXG26XqAnxIBgPiMdi3v1+hvOf+MGFX+I3uUnxIBgHjhv91fH+pS3h9yaVB8UrI5Dyv/38QvigBAPPrJOriS94ejv93vj7s0l/wfAYAEmINvaeHfp2aU9wdh6B93M1Lu/yMAEBc5+fNB4TbbS+E/2W20HweFFW+po9HfIpK4ZxHY0txl/JrVA2sBKjMf0EGCkNlqNzm/Cfti2SExq85V/j8at6gMrAWonmgg66WuZyVwfsJ/UgDwHY3+tsPPtITVEQAEADzH9virTVDvRYX/L9F8CAD4O/rvpWIioz8gANWJHcu9EwIACED1jf5HB4VTiJLA8l8EADx2fvsdZ5fwFk9xIg8CAP5ip/rWl1Cfx38RAPB09O8UxDuFh/wfEIAcYXvudy2hPst/EQDwmHNLrL9Q+f9GmhEBAP/C/+4qupf4NuT/CAB4Ss8yvAf5PwIAnvJJifVtpyKW/yIA4CmNJdZfoPx/C82IAICHyHlfV/FCCW8xn1ZEAMBvbkhY7y3ZPJoPAQC/uUnWkKDexez+CwiA/2mA3cO3w0jXxKj2S9W7ldYDBCAfIvCsCjv7b0mRl9qIb08OjqfVwGBT0BwRrgo8RXaOrI9sd9lHsmdk94Yj/2u0VDY46WsuXhQAkAIAAAIAAAgAACAAAIAAAAACAAAIAAAgAACAAAAAAgAACAAAIAAAgAAAAAIAAAgAAAIAAAgAACAAAIAAAAACAAAIAAAgAACAAAAAAgAACAAAIAAAgAAAAAIAAAgAACAAAIAAAAACAACu8F8BBgDlSreLhu1kMQAAAABJRU5ErkJggg==</script>
There's a few solutions
Make sure transparent area actually has color in.
In other words, if all the pixels in the top left of the image above are RED with 0 alpha then when the pixels get filtered they'll be blending (1,0,0,0) transparent red instead of (0,0,0,0) transparent black. Unfortunately there's no easy way to do this in most drawing programs.
There's a plugin for Photoshop that lets you do it called SuperPNG It lets you create a 4th channel for the alpha instead of using photoshop's transparency. That lets you set the alpha separate from the image.
In your case you'd end up with an image with layers like this
Now there are no bad colors to blend with.
Switch to pre-multiplied alpha
In this case before calling gl.texImage2D to upload the image call
gl.pixelStorei(UNPACK_PREMULTIPLY_ALPHA_WEBGL, true);
before calling gl.texImage2D. That tells WebGL to multiply the colors by their alpha when the image is loaded. You then use blending with
gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);
Turn off filtering in GL
gl.texParameter(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameter(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
Assuming your source image doesn't have any bad colors this means GL won't making new bad colors as it filters but of course it also means if you scale or rotate the image you'll get aliasing.
Create your own mips
Most apps use gl.genereateMipmap to generate mips but you can generate them yourself offline and upload them yourself. That's not a perfect solution either but it does let you use `gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_NEAREST);
Combinations of the above
Have you tried disabling the alpha on the WebGL context?
var gl = this.canvas.getContext('webgl', {antialias: false, alpha: false })
|| this.canvas.getContext('experimental-webgl', {antialias: false, alpha: false });
I intend to create a simple photo editor in JS. My main question is, is it possible to create filters that render in real-time? For example, adjusting brightness and saturation. All I need is a 2D image where I can apply filters using the GPU.
All the tutorials I've read are very complex and don't really explain what the API mean. Please point me in the right direction. Thanks.
I was going to write a tutorial and post it on my blog but I don't know when I'll have time to finish so here's what I have Here's a more detailed set of posts on my blog.
WebGL is actually a rasterization library. I takes in attributes (streams of data), uniforms (variables) and expects you to provide "clip space" coordinates in 2d and color data for pixels.
Here's a simple example of 2d in WebGL (some details left out)
// Get A WebGL context
var gl = canvas.getContext("experimental-webgl");
// setup GLSL program
vertexShader = createShaderFromScriptElement(gl, "2d-vertex-shader");
fragmentShader = createShaderFromScriptElement(gl, "2d-fragment-shader");
program = createProgram(gl, vertexShader, fragmentShader);
gl.useProgram(program);
// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");
// Create a buffer and put a single clipspace rectangle in
// it (2 triangles)
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
-1.0, -1.0,
1.0, -1.0,
-1.0, 1.0,
-1.0, 1.0,
1.0, -1.0,
1.0, 1.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
// draw
gl.drawArrays(gl.TRIANGLES, 0, 6);
Here's the 2 shaders
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
void main() {
gl_Position = vec4(a_position, 0, 1);
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
void main() {
gl_FragColor = vec4(0,1,0,1); // green
}
</script>
This will draw a green rectangle the entire size of the canvas.
In WebGL it's your responsibility to provide a vertex shader that provides clipspace coordinates. Clipspace coordinates always go from -1 to +1 regardless of the size of the canvas. If you want 3d it's up to you to supply shaders that convert from 3d to 2d because WebGL is only a rasterization API
In one simple example, if you want to work in pixels you could pass in a rectangle that uses pixels instead of clip space coordinates and convert to clip space in the shader
For example:
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
uniform vec2 u_resolution;
void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace, 0, 1);
}
</script>
Now we can draw rectangles by changing the data we supply
// set the resolution
var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
gl.uniform2f(resolutionLocation, canvas.width, canvas.height);
// setup a rectangle from 10,20 to 80,30 in pixels
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
10, 20,
80, 20,
10, 30,
10, 30,
80, 20,
80, 30]), gl.STATIC_DRAW);
You'll notice WebGL considers the bottom right corner to be 0,0. To get it to be the more traditional top right corner used for 2d graphics we just flip the y coordinate.
gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
You want to manipulate images you need to pass in textures. In the same way the size of the canvas is represented by clipspace coordinates textures are are referenced by texture coordinates that go from 0 to 1.
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
attribute vec2 a_texCoord;
uniform vec2 u_resolution;
varying vec2 v_texCoord;
void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace, 0, 1);
// pass the texCoord to the fragment shader
// The GPU will interpolate this value between points.
v_texCoord = a_texCoord;
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision float mediump;
// our texture
uniform sampler2D u_image;
// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;
void main() {
gl_FragColor = texture2D(u_image, v_texCoord);
}
</script>
To draw an image requires loading the image and since that happen asynchronously we need to change our code a little. Take all the code we had and put it in a function called "render"
var image = new Image();
image.src = "http://someimage/on/our/server"; // MUST BE SAME DOMAIN!!!
image.onload = function() {
render();
}
function render() {
...
// all the code we had before except gl.draw
// look up where the vertex data needs to go.
var texCoordLocation = gl.getAttribLocation(program, "a_texCoord");
// provide texture coordinates for the rectangle.
var texCoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
1.0, 1.0,
0.0, 1.0,
0.0, 0.0,
1.0, 1.0,
0.0, 0.0,
1.0, 0.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(texCoordLocation);
gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0);
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.draw(...)
If you want to do image processing you just change your shader. Example, Swap red and blue
void main() {
gl_FragColor = texture2D(u_image, v_texCoord).bgra;
}
Or blend with the pixels next to it.
uniform vec2 u_textureSize;
void main() {
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
gl_FragColor = (texture2D(u_image, v_texCoord) +
texture2D(u_image, v_texCoord + vec2(onePixel.x, 0.0)) +
texture2D(u_image, v_texCoord + vec2(-onePixel.x, 0.0))) / 3.0;
}
And we have to pass in the size of the texture
var textureSizeLocation = gl.getUniformLocation(program, "u_textureSize");
...
gl.uniform2f(textureSizeLocation, image.width, image.height);
Etc... Click the last link below for a convolution sample.
Here are working versions with a slightly different progression
Draw Rect in Clip Space
Draw Rect in Pixels
Draw Rect with origin at top left
Draw a bunch of rects in different colors
Draw an image
Draw an image red and blue swapped
Draw an image with left and right pixels averaged
Draw an image with a 3x3 convolution
Draw an image with multiple effects
You can make a custom pixel shader for each operation you're intending to use. Just learn some GLSL and follow the "Learning WebGL" tutorials to get a grasp of basic WebGL.
You can render your image with the shader modifying the parameters you can include to control the different visual styles and then when the user clicks "ok" you can read back the pixels to store it as your current image.
Just remember to avoid cross domain images, because that will disable the reading back of pixels.
Also, check the quick reference card (PDF) for quick info on shader operations.
Just try glfx ( http://evanw.github.com/glfx.js/ )
I think it is exactly what you need.
You can use set of predefined shaders or easily add yours ;)
enjoy! It is very easy with glfx!
<script src="glfx.js"></script>
<script>
window.onload = function() {
// try to create a WebGL canvas (will fail if WebGL isn't supported)
try {
var canvas = fx.canvas();
} catch (e) {
alert(e);
return;
}
// convert the image to a texture
var image = document.getElementById('image');
var texture = canvas.texture(image);
// apply the ink filter
canvas.draw(texture).ink(0.25).update();
// replace the image with the canvas
image.parentNode.insertBefore(canvas, image);
image.parentNode.removeChild(image);
};
</script>
<img id="image" src="image.jpg">