#### Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

# Squeezy sponge - deforming a cube in the shader

edited August 2015 Posts: 2,020

I've been experimenting with deforming shapes in the shader.  Drag on the left and right sides of the screen to stretch and twist the left and right sides of this cube. It's like squeezing out a sponge. It's done by uploading two modelMatrices to the shader, one for each side of the sponge, and then interpolating between the positions and normals generated by the matrices using mix and smoothstep. ``````--# Main
-- Cube Deform

local tArray = {{x=0,y=0, mm = matrix(), id = 0},{x=0,y=0, mm = matrix(),id=0}} --coords and matrices for 2 bones/ touch points

function setup()
m = mesh()
m:setColors(color(255))
m.vertices, m.normals, m.colors, m.texCoords = cube(5, 4)
m.shader.lightColor = color(223, 216, 145, 255)
cam = {pos = vec3(5,-10,10)}
centre = vec2(WIDTH, HEIGHT)/2
print("Drag on the left and right sides of the screen to squeeze out the sponge!")

end

function draw()
background(40, 40, 50)
perspective()
camera(cam.pos.x, cam.pos.y, cam.pos.z,0,0,0)

for i,v in ipairs(tArray) do
if v.id == 0 then
v.x = v.x *  0.97 --damping/ return sponge to form
v.y = v.y *  0.97
end
pushMatrix()
translate(v.x*0.02,v.y*0.02,0)
rotate(v.x, 0,1,0)
rotate(-v.y, 1,0,0)
v.mm = modelMatrix()
popMatrix()
end

--nb draw at the identity matrix
m:draw()
end

local sc = 0.02
function touched(t)
if t.state == BEGAN then
if t.x<centre.x then --bone 1 on the left
tArray.id = t.id
else
tArray.id = t.id -- bone 2 on the right
end
else
for i,v in ipairs(tArray) do
if t.id == v.id then  --find out which side this touch originated on
v.y = v.y + t.deltaY * 0.3
v.x = v.x + t.deltaX * 0.3
if t.state == ENDED then v.id = 0 end
--[[
if i==1 then --only right-side touches influence lighting
m.shader.light = vec4((t.x - centre.x)*sc, (t.y-centre.y)*sc, 7, 1)
end
]]
end
end
end

end

Deform3={
vert = [[
const float size = 2.5;
uniform mat4 modelViewProjection;
uniform mat4 modelMatrix1; //bone1
uniform mat4 modelMatrix2; //bone2

attribute vec4 position;
attribute vec4 color;
attribute vec3 normal;
attribute vec2 texCoord;

varying lowp vec4 vNormal;
varying lowp vec4 vPosition;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
float bl = smoothstep(-size, size, position.x);
mat4 mm = modelMatrix1 * (1.-bl) + modelMatrix2 * bl;
vPosition = mm * position;
vNormal = normalize(mm * vec4( normal, 0.0 ));

vColor = color;
vTexCoord = texCoord;
gl_Position = modelViewProjection * vPosition;
}

]],

frag = [[

precision highp float;

uniform lowp sampler2D texture;
uniform float ambient; // --strength of ambient light 0-1
uniform vec4 light; //--directional light direction (x,y,z,0)
uniform vec4 lightColor; //--directional light colour
uniform vec4 eye; // -- position of camera (x,y,z,1)

varying lowp vec4 vNormal;
varying lowp vec4 vPosition;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
lowp vec4 pixel= texture2D( texture, vec2( fract(vTexCoord.x), fract(vTexCoord.y) ) ) * vColor;
lowp vec4 ambientLight = pixel * ambient;
lowp vec4 norm = normalize(vNormal);
lowp vec4 lightDirection = normalize(light - vPosition * light.w);
lowp vec4 diffuse = pixel * lightColor * max( 0.0, dot( norm, lightDirection ));

//specular blinn-phong
vec4 cameraDirection = normalize( eye - vPosition );
vec4 halfAngle = normalize( cameraDirection + lightDirection );
float spec = pow( max( 0.0, dot( norm, halfAngle)), 8. );//last number is specularPower, higher number = smaller highlight
lowp vec4 specular = lightColor  * spec * 64. ;// add optional shininess at end here

vec4 totalColor = ambientLight + diffuse + specular;
totalColor.a=vColor.a;
gl_FragColor=totalColor;
}

]]}

--# Cube
function cube(s, depth)
-- all the unique vertices that make up a cube
local vertices = {
vec3(-0.5, -0.5,  0.5), -- Left  bottom front
vec3( 0.5, -0.5,  0.5), -- Right bottom front
vec3( 0.5,  0.5,  0.5), -- Right top    front
vec3(-0.5,  0.5,  0.5), -- Left  top    front
vec3(-0.5, -0.5, -0.5), -- Left  bottom back
vec3( 0.5, -0.5, -0.5), -- Right bottom back
vec3( 0.5,  0.5, -0.5), -- Right top    back
vec3(-0.5,  0.5, -0.5), -- Left  top    back
}

local faces = {
-- Front
{vertices, vertices, vertices, vertices},
-- Right
{vertices, vertices, vertices, vertices},
-- Back
{vertices, vertices, vertices, vertices},
-- Left
{vertices, vertices, vertices, vertices},
-- Top
{vertices, vertices, vertices, vertices},
-- Bottom
{vertices, vertices, vertices, vertices},
}

--recursive subdivide function
local origin = vertices
local tOri = vec2(0.5,0.5)
local verts, norms, cols, texC = {}, {}, {}, {}
local rnd = math.random

local function subdivide(f, depth, n, c, t)
if depth == 0 then --triangulate
local tc = {vec2(f[t.x], f[t.y])+tOri, vec2(f[t.x], f[t.y])+tOri,vec2(f[t.x], f[t.y])+tOri,vec2(f[t.x], f[t.y])+tOri}
-- local c = color(rnd(128)+128, rnd(128)+128, rnd(128)+128)
table.move({f*s,f*s,f*s, f*s,f*s,f*s}, 1, 6, #verts+1, verts) --triangulate
table.move({tc,tc,tc, tc,tc,tc}, 1, 6, #texC+1, texC)
table.move({n,n,n,n,n,n},  1, 6, #norms+1, norms)
table.move({c,c,c,c,c,c},  1, 6, #cols+1, cols)
return --end recursion
end
--else, do some more subdividing
local mid = {}
for i = 1, 4 do --4 mid points
local j = (i % 4) + 1
mid[i] = (f[i] + f[j]) * 0.5
end
local cent = (f + f + f + f) * 0.25 --centre point
for i = 1, 4 do --4 sub-faces
local j = (i % 4) + 1
subdivide ({f[j], mid[j], cent, mid[i]}, depth - 1, n, c, t)
end
end
--start subdivision

for i = 1,6 do --6 faces.
--establish norm, color, and axis of texCoords for each face
local norm = (faces[i]-faces[i]):cross(faces[i]-faces[i])
local col = color(rnd(100)+155, rnd(100)+155, rnd(100)+155) --color(rnd(255), rnd(255), rnd(255))
local plane = (faces[i]-origin)-(faces[i]-origin) --measure diagonal
local tc = {x="x",y="y"}
if plane.x==0 then tc.x="z" --if no x dimension...
elseif plane.y==0 then tc.y="z" --or no y dimension, then remap texCoord onto Z
end
subdivide(faces[i], depth, norm, col, tc)
end

print ("recursions:", depth, "verts:", #verts) -- = 4 corners ^ depth * 6 faces * 6 tri points
return verts, norms, cols, texC
end

``````
Tagged:

• Posts: 5,396

Step 1 - squeezy sponge

Step 2 - Spongebob )

• Posts: 2,020

it's quite relaxing to play with. I wonder if you could game-ify it.... A very odd variation on Geometry Wars maybe??

This is what I love about Codea, just that combo of incredible graphics power plus multitouch.... An idea pops into your head, and you never quite know what's going to pop out a few hours later.

Btw, how do you roll your own version of `modelViewProjection` in the vertex shader? Or rather, I want just `viewProjection`.

You can see in the code above, after deforming the position I have to convert it back to local coords (by multiplying by the inverse of the model matrix) to multiply it by modelViewProjection. The back conversion step would be unnecessary if I could create a `viewProjection` matrix. This nearly worked:

`projectionMatrix * viewMatrix * modelMatrix * position` but the perspective was weird, as if the object was getting bigger, rather than receding with distance, and the object was dark, as if the normals were all facing the wrong way. The projection, view, and model matrices were all just loaded in from Codea. I tried making various ones inverse, transpose, inverse():transpose() etc, but I couldn't get a combination that worked. Anyone know how to do this?

• edited August 2015 Posts: 3,295

wow! you've become our current locomotive! Thanks for that.

• Posts: 2,020

You're welcome. It's when there's something else really important that I should be doing that I most often end up procrastinating!

• Posts: 289

wonderful, sponge cube

• Posts: 5,396

@yojimbo2000 - you know, if you forget that inverse matrix stuff and just write

``````gl_Position = modelViewProjection * vPosition;
``````

you get very similar results.

Looking at the Codea draw code, and specicially, only the code that affects modelMatrix...

``````function draw()
<snip>
for i,v in ipairs(tArray) do  --two passes
<snip>
resetMatrix() --create 2 matrices
translate(v.x*0.02,v.y*0.02,0)
rotate(v.x, 0,1,0)
rotate(-v.y, 1,0,0)
v.mm = modelMatrix()
end
<snip>
m:draw()
end
``````

So the modelMatrix that is used by the shader is the modelMatrix created by the second pass through the loop. Is this intended?

Why not reset the matrix after the second pass, then modelMatrix is just the unit matrix and there is no need for any inverse transformation?

• Posts: 745

Really impressive. Reminds me of an accordion - might be nice to use the corner positions to modify a sound - a sort of squeezy theremin

• Posts: 2,020

@Ignatz this is actually a proof of concept of doing bone deformation/ matrix interpolation on the shader. So the two sides of the cube are the bones, and the vertex weight/ skin weight is set by feeding local position.x into the smoothstep curve.

The idea is that `modelMatrix` / `mmInverse` represents the "root"/ parent bone in world space, and `modelMatrix2` is a child-bone in local space. I admit that it's confusing!

if you forget that inverse matrix stuff and just write `gl_Position = modelViewProjection * vPosition;` you get very similar results.

That doesn't work because whatever transformation is applied to the left side of the cube gets doubled.

But your suggestion about resetting the matrix is a really interesting one. I didn't think of this, but I guess what you're saying is that one way of getting just the `viewProjection` matrix that I was asking for, is to make sure the `modelMatrix` is the identity matrix when you draw. So then the two matrices for the bones would both need to be in world space. That's probably an easier way of doing it. I'll post some updated code in a bit.

In the shader, I also came up with an alternate way of interpolating between the two matrices directly (instead of producing two position vectors and two normal vectors and interpolating between those vectors):

`mat4 mm = modelMatrix * (1.-bl) + modelMatrix2 * bl;`

You then calculate a single position and normal with the interpolated matrix.

@West all mods are welcome! It's like a Rorschach. I saw a sponge, @Ignatz saw SpongeBob, you saw an accordion.

• Posts: 2,020

OK, I've changed the code above to incorporate @Ignatz 's suggestion to draw at the identity matrix and with the 2 bone matrices now both in world space. I think it's easier to understand now.

• Posts: 5,396

Very nice..