Develop Reference

Recuperating pixel data from fragment shader results in unexpected behavior

I'm currently trying to use a fragment shader to transport some particle position data (and in the future, modify it). I have no problem sending the data to the shader using a sampler2D texture, but when I try to recuperate the data, My 20 particles suddenly have the wrong positions. I've printed as many outputs as possible and have minimized the code as much as I can yet still fail to understand where I'm wrong.
a reproducible minimized version is available on the p5js website here
Here is my sketch.js :
let theShader;
let shaderTexture;
let NUM_PARTICLES = 20;
let particleTexture;
let particles = [];
function preload(){
theShader = loadShader('basic.vert', 'basic.frag');
}
function setup() {
createCanvas(400, 400, WEBGL);
// Initiate Particles
for(let i = 0; i < NUM_PARTICLES;i++){
particles.push(createVector(i*20,i*20,0));
}
// Initialize Shader
shaderTexture = createGraphics(NUM_PARTICLES, 1, WEBGL);
shaderTexture.noStroke();
// Create Particle Texture
particleTexture = createImage(NUM_PARTICLES, 1);
// Fill Particle Texture
particleTexture.loadPixels();
for (let i = 0; i < NUM_PARTICLES; i++) {
particleTexture.pixels[i*4+0] = map(particles[i].x,0,width,0,255); // R
particleTexture.pixels[i*4+1] = map(particles[i].y,0,height,0,255); // G
particleTexture.pixels[i*4+2] = 0; // B
particleTexture.pixels[i*4+3] = 255; // A
}
particleTexture.updatePixels();
}
function draw() {
translate(-width/2, -height/2);
background(255);
// Display Particles Before Modification
for(let i = 0; i < NUM_PARTICLES;i++){
circle(particles[i].x,particles[i].y,10); // draw circle at particle location
}
// Apply Texture
shaderTexture.shader(theShader); // set shader
theShader.setUniform('text', particleTexture); // send particleTexture to shader
shaderTexture.rect(0,0,NUM_PARTICLES,1); // set rect to recieve shader out
// Print Shader Output
for(let i = 0; i < NUM_PARTICLES;i++){
let newPos = shaderTexture.get(i, 0);
print(newPos);
}
// Update and Display Particles
for(let i = 0; i < NUM_PARTICLES;i++){
let newPos = shaderTexture.get(i, 0);
particles[i].x = map(newPos[0],0,255,0,width);
particles[i].y = map(newPos[1],0,255,0,height);
fill(255,0,0);
circle(particles[i].x,particles[i].y,10);
}
noLoop();
}
and here is my fragment shader which should not be modifying anything :
#ifdef GL_ES
precision highp float;
#endif
uniform sampler2D text;
void main() {
vec2 particle = texture2D(text, vec2(gl_FragCoord.x, gl_FragCoord.y)).xy;
gl_FragColor = vec4(particle.x,particle.y,0.0,1.0); // R,G,B,A
}
also my vertex shader which is default :
#ifdef GL_ES
precision highp float;
#endif
attribute vec3 aPosition;
void main() {
vec4 positionVec4 = vec4(aPosition, 1.0);
positionVec4.xy = positionVec4.xy * 2.0 - 1.0;
gl_Position = positionVec4;
}

When looking up a texture with texture2D the texture coordinates must be specified in range [0.0, 1.0]. (0, 0) is the bottom left and (1, 1) is the top right. However gl_FragCoord.xy contains window coordinates, with top left (0.5, 0.5) and top right (width-0.5, height-0.5).
Hence you need to divide gl_FragCoord by the size of the viewport. Since you draw on a rectangle with the size NUM_PARTICLESx1 (width == NUM_PARTICLES, height = 1), you must divide gl_FragCoord.x by NUM_PARTICLES:
vec2 particle = texture2D(text, vec2(gl_FragCoord.x, gl_FragCoord.y)).xy;
vec2 particle = texture2D(text, vec2(gl_FragCoord.x/20.0, 0.0)).xy;
Alternatively you can add a size uniform to the fragment shader and divide gl_FragCoord.xy by size:
#ifdef GL_ES
precision highp float;
#endif
uniform sampler2D text;
uniform vec2 size;
void main() {
vec2 particle = texture2D(text, gl_FragCoord.xy / size).xy;
gl_FragColor = vec4(particle.x,particle.y,0.0,1.0); // R,G,B,A
}
Set the value of the size uniform by [NUM_PARTICLES, 1]:
// Apply Texture
shaderTexture.shader(theShader); // set shader
theShader.setUniform('text', particleTexture); // send particleTexture to shader
theShader.setUniform('size', [NUM_PARTICLES, 1]);
shaderTexture.rect(0,0,NUM_PARTICLES,1); // set rect to recieve shader out
If you are using OpenGL ES Shading Language 3.00 you have the option to use texelFetch to lookup the texture with integral pixel coordinates or get the size of the texture using textureSize.
Unfortunately p5.js (createCanvas()) doesn't seem to provide a WebGL 2.0 context, so this is not an option.

Draw particles in a single quad so they can interact with each other, how to pass particle positions to shaders?

I can render a single 2d particle using sdf like this:
void renderShape(out vec3 col, vec2 p) {
vec2 translation = vec2(0.0, 0.5);
// some math
col = // some color.
}
translation variable controls where to render this particle.
Now I calculate particle positions on the cpu, and need to pass these positions to the shaders and render each particle on different positions. How do I do that?
Currently only data I pass is the vertices of a quad like this:
let positions = [
-1, 1,
-1, -1,
1, -1,
-1, 1,
1,-1,
1, 1
];
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
// on render
gl.bindVertexArray(vao);
gl.drawArrays(gl.TRIANGLES, 0, 6);
vertex shader:
#version 300 es
precision mediump float;
in vec2 a_position;
uniform vec2 u_resolution;
void main() {
gl_Position = vec4(a_position, 0, 1);
}
I want particles to combine with each other (using union operation) so I can't render them individually like with gl.POINTS

Unable to load texture in WebGl

I am able to load RGB colours but not textures. If it could be some settings problem please prompt me.
This is the screenshot of chrome://flags
The HTML code is given :
<!DOCTYPE html>
<meta charset="UTF-8">
<html>
<head>
<title>WebGL Cube with Texture</title>
<script type="x-shader/x-vertex" id="vshader">
attribute vec3 coords;
attribute vec2 texCoords;
uniform vec3 normal;
uniform mat4 modelview;
uniform mat4 projection;
uniform mat3 normalMatrix;
varying vec3 vNormal;
varying vec2 vTexCoords;
void main() {
vec4 coords = vec4(coords,1.0);
vec4 transformedVertex = modelview * coords;
vNormal = normalMatrix * normal;
vTexCoords = texCoords;
gl_Position = projection * transformedVertex;
}
</script>
<script type="x-shader/x-fragment" id="fshader">
precision mediump float;
uniform bool textured;
uniform sampler2D sampler;
varying vec3 vNormal;
varying vec2 vTexCoords;
uniform vec4 color;
void main() {
if (textured) {
vec4 color = texture2D(sampler, vTexCoords);
vec3 unitNormal = normalize(vNormal);
float multiplier = abs(unitNormal.z);
gl_FragColor = vec4( multiplier*color.r, multiplier*color.g, multiplier*color.b, color.a );
}
else {
gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0); // use basic white when texture's not there.
}
}
</script>
<script type="text/javascript" src="gl-matrix-min.js"></script>
<script type="text/javascript" src="simple-rotator.js"></script>
<script type="text/javascript">
"use strict";
var gl; // The webgl context.
var aCoords; // Location of the coords attribute variable in the shader program.
var aCoordsBuffer; // Buffer to hold coords.
var aTexCoords; // Location of the texCoords attribute variable in the shader program.
var aTexCoordsBuffer; // Buffer to hold texCoords.
var uProjection; // Location of the projection uniform matrix in the shader program.
var uModelview; // Location of the modelview unifirm matrix in the shader program.
var uNormal; // Location of the normal uniform in the shader program.
var uColor; // Location of the color uniform in the shader program, used only for axes.
var uTextured; // Location of the textured uniform in the shader program.
var uSampler; // Location of the sampler in the shader program.
var uNormalMatrix; // Location of the normalMatrix uniform matrix in the shader program.
var projection = mat4.create(); // projection matrix
var modelview = mat4.create(); // modelview matrix
var normalMatrix = mat3.create(); // matrix, derived from modelview matrix, for transforming normal vectors
var rotator; // A SimpleRotator object to enable rotation by mouse dragging.
var textureID = null; // Texture object, to be created after image has loaded.
/* Draws a colored cube, along with a set of coordinate axes.
* (Note that the use of the above drawPrimitive function is not an efficient
* way to draw with WebGL. Here, the geometry is so simple that it doesn't matter.)
*/
function draw() {
gl.clearColor(0,0,0,1);
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
if (document.getElementById("persproj").checked) {
mat4.perspective(projection, Math.PI/4, 1, 2, 10);
}
else {
mat4.ortho(projection,-2.5, 2.5, -2.5, 2.5, 2, 10);
}
gl.uniformMatrix4fv(uProjection, false, projection );
var modelview = rotator.getViewMatrix();
var saveModelview = mat4.clone(modelview);
if (textureID) {
gl.uniform1i( uTextured, 1 ); // Tell shader to use texture and lighting.
gl.bindTexture(gl.TEXTURE_2D, textureID); // Which texture should be used.
gl.uniform1i(uSampler, 0); // Set sampler in shadre to use texture unit zero.
}
else {
gl.uniform1i( uTextured, 0 ); // Cube will appear in plain white.
}
drawFace(modelview) // front face of the cube
mat4.rotateY(modelview,modelview,Math.PI/2); //right face
drawFace(modelview) // front face
mat4.rotateY(modelview,modelview,Math.PI/2); //back face
drawFace(modelview) // front face
mat4.rotateY(modelview,modelview,Math.PI/2); //left face
drawFace(modelview) // front face
modelview = mat4.clone(saveModelview);
mat4.rotateX(modelview,modelview,Math.PI/2);
drawFace(modelview) // top face
mat4.rotateX(modelview,modelview,Math.PI);
drawFace(modelview) // bottom face
}
/**
* Draws the front face of the cube, subject to a modelview transform.
*/
function drawFace(modelview) {
gl.uniformMatrix4fv(uModelview, false, modelview );
mat3.normalFromMat4(normalMatrix, modelview);
gl.uniformMatrix3fv(uNormalMatrix, false, normalMatrix);
gl.uniform3f(uNormal, 0, 0, 1);
gl.drawArrays(gl.TRIANGLE_FAN, 0, 4); // front face
}
/**
* Loads data for the front face of the cube into VBOs.
*/
function createFace() {
var vertices = [ -1,-1,1, 1,-1,1, 1,1,1, -1,1,1 ];
var texCoords = [ 0,0, 2,0, 2,2, 0,2 ];
gl.enableVertexAttribArray(aCoords);
gl.bindBuffer(gl.ARRAY_BUFFER,aCoordsBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);
gl.vertexAttribPointer(aCoords, 3, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(aTexCoords);
gl.bindBuffer(gl.ARRAY_BUFFER,aTexCoordsBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(texCoords), gl.STATIC_DRAW);
gl.vertexAttribPointer(aTexCoords, 2, gl.FLOAT, false, 0, 0);
}
/**
* Load an image from the URL "textures/bridk001.jpg". The image is loade
* asynchronously. When the
*/
function loadTexture() {
var img = new Image();
img.onload = function() {
var id = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D,id);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);
gl.generateMipmap(gl.TEXTURE_2D);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.REPEAT);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.REPEAT);
textureID = id;
draw();
}
img.src = "./skin.jpg";
}
/* Creates a program for use in the WebGL context gl, and returns the
* identifier for that program. If an error occurs while compiling or
* linking the program, an exception of type String is thrown. The error
* string contains the compilation or linking error. If no error occurs,
* the program identifier is the return value of the function.
*/
function createProgram(gl, vertexShaderSource, fragmentShaderSource) {
var vsh = gl.createShader( gl.VERTEX_SHADER );
gl.shaderSource(vsh,vertexShaderSource);
gl.compileShader(vsh);
if ( ! gl.getShaderParameter(vsh, gl.COMPILE_STATUS) ) {
throw "Error in vertex shader: " + gl.getShaderInfoLog(vsh);
}
var fsh = gl.createShader( gl.FRAGMENT_SHADER );
gl.shaderSource(fsh, fragmentShaderSource);
gl.compileShader(fsh);
if ( ! gl.getShaderParameter(fsh, gl.COMPILE_STATUS) ) {
throw "Error in fragment shader: " + gl.getShaderInfoLog(fsh);
}
var prog = gl.createProgram();
gl.attachShader(prog,vsh);
gl.attachShader(prog, fsh);
gl.linkProgram(prog);
if ( ! gl.getProgramParameter( prog, gl.LINK_STATUS) ) {
throw "Link error in program: " + gl.getProgramInfoLog(prog);
}
return prog;
}
/* Gets the text content of an HTML element. This is used
* to get the shader source from the script elements that contain
* it. The parameter should be the id of the script element.
*/
function getTextContent( elementID ) {
var element = document.getElementById(elementID);
var fsource = "";
var node = element.firstChild;
var str = "";
while (node) {
if (node.nodeType == 3) // this is a text node
str += node.textContent;
node = node.nextSibling;
}
return str;
}
/**
* Initializes the WebGL program including the relevant global variables
* and the WebGL state. Creates a SimpleView3D object for viewing the
* cube and installs a mouse handler that lets the user rotate the cube.
*/
function init() {
try {
var canvas = document.getElementById("glcanvas");
gl = canvas.getContext("webgl");
if ( ! gl ) {
gl = canvas.getContext("experimental-webgl");
}
if ( ! gl ) {
throw "Could not create WebGL context.";
}
var vertexShaderSource = getTextContent("vshader");
var fragmentShaderSource = getTextContent("fshader");
var prog = createProgram(gl,vertexShaderSource,fragmentShaderSource);
gl.useProgram(prog);
aCoords = gl.getAttribLocation(prog, "coords");
aTexCoords = gl.getAttribLocation(prog, "texCoords");
uModelview = gl.getUniformLocation(prog, "modelview");
uProjection = gl.getUniformLocation(prog, "projection");
uSampler = gl.getUniformLocation(prog, "sampler");
uNormal = gl.getUniformLocation(prog, "normal");
uColor = gl.getUniformLocation(prog, "color");
uTextured = gl.getUniformLocation(prog, "textured");
uNormalMatrix = gl.getUniformLocation(prog, "normalMatrix");
aCoordsBuffer = gl.createBuffer();
aTexCoordsBuffer = gl.createBuffer();
gl.enable(gl.DEPTH_TEST);
gl.enable(gl.CULL_FACE); // no need to draw back faces
document.getElementById("persproj").checked = true;
rotator = new SimpleRotator(canvas,draw);
rotator.setView( [2,2,5], [0,1,0], 6 );
}
catch (e) {
document.getElementById("message").innerHTML =
"Could not initialize WebGL: " + e;
return;
}
createFace();
loadTexture();
draw();
}
</script>
</head>
<body onload="init()" style="background-color:#DDD">
<h2>A Cube With a Brick Texture</h2>
<p id=message>Drag the mouse on the canvas to rotate the view.</p>
<p>
<input type="radio" name="projectionType" id="persproj" value="perspective" onchange="draw()">
<label for="persproj">Perspective projection</label>
<input type="radio" name="projectionType" id="orthproj" value="orthogonal" onchange="draw()" style="margin-left:1cm">
<label for="orthproj">Orthogonal projection</label>
<button onclick="rotator.setView( [2,2,5], [0,1,0], 6 ); draw()" style="margin-left:1cm">Reset View</button>
</p>
<noscript><hr><h3>This page requires Javascript and a web browser that supports WebGL</h3><hr></noscript>
<div>
<canvas width=600 height=600 id="glcanvas" style="background-color:red"></canvas>
</div>
</body>
</html>
All i get as an output is
The other functions are loading fine. The file paths are correct.

The issue is you need to run a simple web server for WebGL dev. It should take you about 2 minutes to get setup
See this

Try defining the minification and magnification parameters for the texture object.
eg:
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
Use the appropriate value for min and mag filter, based on your project requirement.

What is causing invalid operation on my draw arrays call?

Using WebGL I am trying to draw a simple triangle from scratch.
I have experience writing openGL applications in C++, and have looked at the webGL reference card to translate my code.
However, I am having difficulty debugging the application.
The particular error message I am getting is:
GL ERROR :GL_INVALID_OPERATION : glDrawArrays: attempt to access out of range vertices in attribute 0
The entire code is here: https://github.com/gheshu/webGL_experiments
The vertex data is laid out as 3 vectors of 3 floats.
Three attributes exist: position, normal, and color, and should be bound on indices 0, 1, 2.
some important snippets:
mesh class:
class Mesh{
constructor(){
this.num_vertices = 0;
this.vbo = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, this.vbo);
gl.enableVertexAttribArray(0);
gl.vertexAttribPointer(0, 3, gl.FLOAT, false, 4*3*3, 0);
gl.enableVertexAttribArray(1);
gl.vertexAttribPointer(1, 3, gl.FLOAT, false, 4*3*3, 4*3);
gl.enableVertexAttribArray(2);
gl.vertexAttribPointer(2, 3, gl.FLOAT, false, 4*3*3, 4*3*2);
}
upload(buffer){
console.log(buffer);
gl.bindBuffer(gl.ARRAY_BUFFER, this.vbo);
gl.bufferData(gl.ARRAY_BUFFER, buffer, gl.STATIC_DRAW);
this.num_vertices = buffer.length / 9;
}
draw(){
gl.bindBuffer(gl.ARRAY_BUFFER, this.vbo);
gl.drawArrays(gl.TRIANGLES, 0, this.num_vertices);
}
destroy(){
gl.deleteBuffer(this.vbo);
}
}
program class:
class GLProgram{
constructor(vertShader, fragShader){
this.prog = gl.createProgram();
gl.attachShader(this.prog, vertShader.id);
gl.attachShader(this.prog, fragShader.id);
gl.bindAttribLocation(this.prog, 0, "position");
gl.bindAttribLocation(this.prog, 1, "normal");
gl.bindAttribLocation(this.prog, 2, "color");
gl.linkProgram(this.prog);
var log = gl.getProgramInfoLog(this.prog);
if(log.length){
console.log();
}
gl.detachShader(this.prog, vertShader.id);
vertShader.destroy();
gl.detachShader(this.prog, fragShader.id);
fragShader.destroy();
}
bind(){
gl.useProgram(this.prog);
}
destroy(){
gl.deleteProgram(this.prog);
}
}
vertex shader:
attribute vec3 position;
attribute vec3 normal;
attribute vec3 color;
varying vec3 vColor;
void main(){
gl_Position = vec4(position, 1.0);
vColor = color;
}
fragment shader:
precision mediump float;
varying vec3 vColor;
void main(){
gl_FragColor = vec4(vColor, 1.0);
}
I would greatly appreciate any help or tips you may have in fixing this issue.

At the bottom of your draw.js file, you destroy mesh and prog:
mesh.destroy();
prog.destroy();
In JavaScript window.requestAnimationFrame(onFrame); will actually invoke onFrame after those destroy methods are called. So by the time onFrame is executed both mesh and prog don't exist. I would recommend reading more about requestAnimationFrame so you can destroy these later (i.e. after your animation loop has stopped running).
You can verify the behavior just by removing those destroy lines and it will render fine.

2D Image Processing With WebGL

I intend to create a simple photo editor in JS. My main question is, is it possible to create filters that render in real-time? For example, adjusting brightness and saturation. All I need is a 2D image where I can apply filters using the GPU.
All the tutorials I've read are very complex and don't really explain what the API mean. Please point me in the right direction. Thanks.

I was going to write a tutorial and post it on my blog but I don't know when I'll have time to finish so here's what I have Here's a more detailed set of posts on my blog.
WebGL is actually a rasterization library. I takes in attributes (streams of data), uniforms (variables) and expects you to provide "clip space" coordinates in 2d and color data for pixels.
Here's a simple example of 2d in WebGL (some details left out)
// Get A WebGL context
var gl = canvas.getContext("experimental-webgl");
// setup GLSL program
vertexShader = createShaderFromScriptElement(gl, "2d-vertex-shader");
fragmentShader = createShaderFromScriptElement(gl, "2d-fragment-shader");
program = createProgram(gl, vertexShader, fragmentShader);
gl.useProgram(program);
// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");
// Create a buffer and put a single clipspace rectangle in
// it (2 triangles)
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
-1.0, -1.0,
1.0, -1.0,
-1.0, 1.0,
-1.0, 1.0,
1.0, -1.0,
1.0, 1.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
// draw
gl.drawArrays(gl.TRIANGLES, 0, 6);
Here's the 2 shaders
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
void main() {
gl_Position = vec4(a_position, 0, 1);
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
void main() {
gl_FragColor = vec4(0,1,0,1); // green
}
</script>
This will draw a green rectangle the entire size of the canvas.
In WebGL it's your responsibility to provide a vertex shader that provides clipspace coordinates. Clipspace coordinates always go from -1 to +1 regardless of the size of the canvas. If you want 3d it's up to you to supply shaders that convert from 3d to 2d because WebGL is only a rasterization API
In one simple example, if you want to work in pixels you could pass in a rectangle that uses pixels instead of clip space coordinates and convert to clip space in the shader
For example:
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
uniform vec2 u_resolution;
void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace, 0, 1);
}
</script>
Now we can draw rectangles by changing the data we supply
// set the resolution
var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
gl.uniform2f(resolutionLocation, canvas.width, canvas.height);
// setup a rectangle from 10,20 to 80,30 in pixels
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
10, 20,
80, 20,
10, 30,
10, 30,
80, 20,
80, 30]), gl.STATIC_DRAW);
You'll notice WebGL considers the bottom right corner to be 0,0. To get it to be the more traditional top right corner used for 2d graphics we just flip the y coordinate.
gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
You want to manipulate images you need to pass in textures. In the same way the size of the canvas is represented by clipspace coordinates textures are are referenced by texture coordinates that go from 0 to 1.
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
attribute vec2 a_texCoord;
uniform vec2 u_resolution;
varying vec2 v_texCoord;
void main() {
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace, 0, 1);
// pass the texCoord to the fragment shader
// The GPU will interpolate this value between points.
v_texCoord = a_texCoord;
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision float mediump;
// our texture
uniform sampler2D u_image;
// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;
void main() {
gl_FragColor = texture2D(u_image, v_texCoord);
}
</script>
To draw an image requires loading the image and since that happen asynchronously we need to change our code a little. Take all the code we had and put it in a function called "render"
var image = new Image();
image.src = "http://someimage/on/our/server"; // MUST BE SAME DOMAIN!!!
image.onload = function() {
render();
}
function render() {
...
// all the code we had before except gl.draw
// look up where the vertex data needs to go.
var texCoordLocation = gl.getAttribLocation(program, "a_texCoord");
// provide texture coordinates for the rectangle.
var texCoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
1.0, 1.0,
0.0, 1.0,
0.0, 0.0,
1.0, 1.0,
0.0, 0.0,
1.0, 0.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(texCoordLocation);
gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0);
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.draw(...)
If you want to do image processing you just change your shader. Example, Swap red and blue
void main() {
gl_FragColor = texture2D(u_image, v_texCoord).bgra;
}
Or blend with the pixels next to it.
uniform vec2 u_textureSize;
void main() {
vec2 onePixel = vec2(1.0, 1.0) / u_textureSize;
gl_FragColor = (texture2D(u_image, v_texCoord) +
texture2D(u_image, v_texCoord + vec2(onePixel.x, 0.0)) +
texture2D(u_image, v_texCoord + vec2(-onePixel.x, 0.0))) / 3.0;
}
And we have to pass in the size of the texture
var textureSizeLocation = gl.getUniformLocation(program, "u_textureSize");
...
gl.uniform2f(textureSizeLocation, image.width, image.height);
Etc... Click the last link below for a convolution sample.
Here are working versions with a slightly different progression
Draw Rect in Clip Space
Draw Rect in Pixels
Draw Rect with origin at top left
Draw a bunch of rects in different colors
Draw an image
Draw an image red and blue swapped
Draw an image with left and right pixels averaged
Draw an image with a 3x3 convolution
Draw an image with multiple effects

You can make a custom pixel shader for each operation you're intending to use. Just learn some GLSL and follow the "Learning WebGL" tutorials to get a grasp of basic WebGL.
You can render your image with the shader modifying the parameters you can include to control the different visual styles and then when the user clicks "ok" you can read back the pixels to store it as your current image.
Just remember to avoid cross domain images, because that will disable the reading back of pixels.
Also, check the quick reference card (PDF) for quick info on shader operations.

Just try glfx ( http://evanw.github.com/glfx.js/ )
I think it is exactly what you need.
You can use set of predefined shaders or easily add yours ;)
enjoy! It is very easy with glfx!
<script src="glfx.js"></script>
<script>
window.onload = function() {
// try to create a WebGL canvas (will fail if WebGL isn't supported)
try {
var canvas = fx.canvas();
} catch (e) {
alert(e);
return;
}
// convert the image to a texture
var image = document.getElementById('image');
var texture = canvas.texture(image);
// apply the ink filter
canvas.draw(texture).ink(0.25).update();
// replace the image with the canvas
image.parentNode.insertBefore(canvas, image);
image.parentNode.removeChild(image);
};
</script>
<img id="image" src="image.jpg">