Fragment shader won't compile with IF or mod() present.

Resolved

**NOW RESOLVED! Details: There's a cap on how big an array you can pass into a shader, and its a lot less than 224 elements regardless of what the compiler output says (I cant get above ~100). Breaching this limit makes shader compilation take an extremely long time, and also seems to be bypassed by commenting out random lines of code. I dunno.

If you are having this problem, here is the solution: Do not pass data into a shader as an array unless it's tiny. For large datasets you should instead stuff them into a texture and then pass in the texture. This is more complicated, find a tutorial or something.

/////////////////////////////////////////////////////////// My shader is not compiling. It doesn't give an error (under GLSL ES, it does give one under GLSL. See below). I have determined which line of code is stopping the compilation through the process of elimination (commenting out lines starting from the bottom and moving up until it compiles) - there are no syntax errors or other obvious problems. For reference, here is the fragment shader:

//
// Shader to render floors/ceilings.
//Currently always uses the same texture, will allow lookups in version 2
varying vec2 v_vTexcoord;
varying vec4 v_vColor;

uniform float u_floorProps[6*512]; //Big ole array
uniform float u_height;
uniform float u_width;
uniform float u_playerX;
uniform float u_playerY;
uniform float u_distPlayer;
uniform sampler2D s_floorTex;

void main()
{       
    //Okay, so we currently have two texture coords that are between 0 and 1.
    //from that we need to know what the screen coords were, so we can look things up
      
    float X = v_vTexcoord.x;
    X = floor(X * u_width);
    float Y = v_vTexcoord.y;
    Y = floor(Y * u_height);
    
    int arrayStart = int(X * 6.0);
    float floorScreenY = u_floorProps[arrayStart];
    float distWall = u_floorProps[arrayStart + 3];
    float floorXWall = u_floorProps[arrayStart + 1];
    float floorYWall = u_floorProps[arrayStart + 2];
    
    //Discard this fragment if its above our minimum Y value.
    if (Y < floorScreenY)
    {
            discard;
    }

    //Determine the texture co-ords for this fragment
    float currentDist = u_height / (2.0 * floorScreenY - u_height); //you could make a small lookup table for this instead
    float weight = (currentDist - u_distPlayer) / (distWall - u_distPlayer);

    float currentFloorX = weight * floorXWall + (1.0 - weight) * u_playerX;
    float currentFloorY = weight * floorYWall + (1.0 - weight) * u_playerY;
    
    //These texture dimensions aren't actually used.
    float floorTexWidth = u_floorProps[arrayStart + 4];
    float floorTexHeight = u_floorProps[arrayStart + 5];
      
    float floorTexX = mod(currentFloorX,1.0);
    float floorTexY = mod(currentFloorY,1.0);
    
    vec2 floorUV = vec2(floorTexX,floorTexY);
    gl_FragColor = texture2D( s_floorTex, floorUV);
}

The vertex shader is just the default passthrough, but here it is:

//
// Simple passthrough vertex shader
//
attribute vec3 in_Position;                  // (x,y,z)
//attribute vec3 in_Normal;                  // (x,y,z)     unused in this shader.
attribute vec4 in_Color;                    // (r,g,b,a)
attribute vec2 in_TextureCoord;              // (u,v)


varying vec2 v_vTexcoord;
varying vec4 v_vColor;

void main()
{
    vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
    gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
    
    v_vColor = in_Color;
    v_vTexcoord = in_TextureCoord;
}

Here is the area of code (in the draw step) where I set up my uniforms, set the shader and try to draw a quad:

if (shader_is_compiled(floorShader))
{
    shader_set(floorShader); //Gotta do this first, apaz
    
    
    //Okay, here we do all the floor shader business
    var arrayHandle = shader_get_uniform(floorShader,"u_floorProps");
    var heightHandle = shader_get_uniform(floorShader,"u_height");
    var widthHandle = shader_get_uniform(floorShader,"u_width");
    var playerXHandle = shader_get_uniform(floorShader,"u_playerX");
    var playerYHandle = shader_get_uniform(floorShader,"u_playerY");
    var distPlayerHandle = shader_get_uniform(floorShader,"u_distPlayer");
    //var floorTexHandle = shader_get_uniform(floorShader,"u_floorTex");
    
    shader_set_uniform_f_array(arrayHandle, SHADER_UNIFORMS);
    shader_set_uniform_f(heightHandle,h);
    shader_set_uniform_f(widthHandle,w);
    shader_set_uniform_f(playerXHandle,playerPos[1]);
    shader_set_uniform_f(playerYHandle,playerPos[2]);
    shader_set_uniform_f(distPlayerHandle,distPlayer);
    
    //Texture passing
    var t_handle = shader_get_sampler_index(floorShader,"s_floorTex");
    var lol_a_texture = sprite_get_texture(Wall4,0)
    texture_set_stage(t_handle,lol_a_texture); 

    draw_primitive_begin_texture(pr_trianglelist,sprite_get_texture(Wall1,0));
    //triangle 1
    draw_vertex_texture(0,0,0,0)
    draw_vertex_texture(0,screenHeight,0,1)
    draw_vertex_texture(screenWidth,0,1,0)
    
    //triangle 2
    draw_vertex_texture(0,screenHeight,0,1)
    draw_vertex_texture(screenWidth,0,1,0)
    draw_vertex_texture(screenWidth,screenHeight,1,1)

    draw_primitive_end();
    shader_reset();
}

The first issue is in this part of the fragment shader:

float floorTexX = mod(currentFloorX,1.0);
float floorTexY = mod(currentFloorY,1.0);

If the operands I pass to mod() towards the end of the fragment shader are anything but literals it wont compile. It doesn't produce any errors, the compile output gets up to:

Process Chunk: SPRT   5872
Process Chunk: BGND   28
Process Chunk: PATH   4
Process Chunk: SCPT   520
Process Chunk: GLOB   4
Process Chunk: SHDR   228

and then just hangs, forever. That's if I have my shader language set to GLSL ES, if I set it to just GLSL then the game will run but the shader will not work. The shader part of the compile output reads:

Process Chunk: SHDR   228
Invalid shader (is it marked as incompatible type for this target?) "floorShader":

And attempting to set it or pass any uniforms to it will crash the game.

The second issue is this block:

if (Y < floorScreenY)
{
discard;
}

The same thing happens as long as this if-statement exists, with the same behaviour difference between GLSL ES and regular GLSL. Replacing the conditional with (true) fixes the crash, so like the issue with mod() it seems to be an issue with performing operations on the uniforms I've passed in.

GMS Version: 1.4.1763 (r41504)(STEAM) Here are some things I have tried: -Biasing all the values that go into mod upward by a tiny amount (0.0001) to avoid any divide-by-zero cases. Although I now realise that wouldn't actually work even if that was the problem. -Googling the compiler error message, there is one relevant result that's a YYG forums post where someone is trying to use an older version of HLSL than GMS supports. Doesn't seem applicable. -Instead of using uniforms in the mod() function I've tried floating point literals both positive and negative (and zero), these work fine. -I've also replaced the mod() operands with v_vTexcoord.x and v_vTexcoord.y, values I know work in other parts of the shader. It still won't compile. -Replacing the mod(value,1.0) call with

float newValue = oldValue - floor(oldValue); 

These operations should be equivalent, and when the second operation is substituted the shader does actually compile. However, at this point the stall point becomes the line:

vec2 floorUV = vec2(floorTexX,floorTexY);

I determined this the same way as the original error, line-by-line commenting. -Using the modf() function, which is supposed to exist in both GLSL ES and GLSL, produces this error:

Reading project file....Error compiling fragment shader:
In Shader floorShader at line 52 : 'modf' : no matching overloaded function found
Compile Failed - Please check the Compile window for any additional information

To make my goal clear, I want to discard the non-fractional portion of the values currentFloorX and currentFloorY so I can use them as texture co-ordinates. There's probably something fundamentally wrong with my understanding of how shaders work, and I would be ecstatic if someone could enlighten me.

Posted Jul 18, 2017, 10:57 AM UTC · edited

14
Loading comments…