Ok,

this is where I got so far by still trying to make a sense out of all this and by ‘dissecting’ the HoloPlay.js that btw uses a lib ‘even weirder than the other’.

While doing all this I am out of my usual country in another one so I don’t have the looking glass with me I can test ‘the final result’ but I can see some things.

So first of all if I understood correctly, you render the 32 images ( or sub-images as you call them ) in a 2048x2048 Render Target that gets “split” into 4 x 8 images.

So we start with :

```
L_RenderResolution = 2048;
TilesX = 4;
TilesY = 8;
//render texture dimensions
L_RenderSizeX = L_RenderResolution / TilesX;
L_RenderSizeY = L_RenderResolution / TilesY;
```

#ifdef SHOW_TEST

```
L_RenderSizeX = wp.w / TilesX;
L_RenderSizeY = wp.h / TilesY;
```

#endif

```
the_rt = g_Neon_Device->CreateRenderTarget_Neon(Texture_RGBA, L_RenderResolution, L_RenderResolution, NULL);
```

Furthermore you create a set of ViewPorts to match those “little sub images” in a way like this :

```
// we create a set of cameras and viewports for it
int i;
float x, y;
i = 0;
for ( y = 0; y < TilesY; y++) {
for ( x = 0; x < TilesX; x++) {
//var subcamera = new THREE.PerspectiveCamera();
Subcamera[i] = glm::perspectiveRH(50.0/180.0*M_PI, 1.0, 0.1, 2000.0);
//subcamera.viewport = new THREE.Vector4(x * renderSizeX, y * renderSizeY, renderSizeX, renderSizeY);
SubcamViewPort[i].x = x * L_RenderSizeX;
SubcamViewPort[i].y = y * L_RenderSizeY;
SubcamViewPort[i].w = L_RenderSizeX;
SubcamViewPort[i].h = L_RenderSizeY;
// cameras.push(subcamera);
i++;
}
}
```

Finally you create a ‘quad’ that you’ll use for the ‘final full screen rendering’ using the ‘lenticular shader’, this is nothing special :

```
g_Neon_Device->GetVertexBuffer(&Holo_Quad, 4, COLORVERTEX_BASIC);
AddColorVertex(&Holo_Quad, -1, 1, 0, 0xffffffff, 0, 1, 0, 1, 0, 1);
AddColorVertex(&Holo_Quad, 1, 1, 0, 0xffffffff, 1, 1, 1, 1, 1, 1);
AddColorVertex(&Holo_Quad, -1, -1, 0, 0xffffffff, 0, 0, 0, 0, 0, 0);
AddColorVertex(&Holo_Quad, 1, -1, 0, 0xffffffff, 1, 0, 1, 0, 1, 0);
```

The idea is you render TilesX * TilesY images creating this “quilt” by moving the camera and the projection matrix “in some way” and here we have ‘some problems’ because I tried to use the “pseudo code” you supplied in the CPP “documentation”.

If I understand correctly you move the “LookAT” point by offsetting in X and you do also something in the Proj Matrix to compensate for the ‘skew’ ?

Anyway FIRST problems, it’s not clear what SORT of View/Proj matrices you use, I use this :

```
proj = glm::perspectiveRH(
fov,
(float)L_RenderSizeX / (float)L_RenderSizeY,
//1.0f,
0.1f, // z near
20.0f); // z far
```

Fundamentally a standard Perspective fov Right Handed matrix for ‘Proj’ and a standard :

```
M = glm::lookAtRH(
glm::vec3(eyeX, eyeY, eyeZ), // Camera is at (), in World Space
glm::vec3(centerX, centerY, centerZ), // and looks at the origin
glm::vec3(upX, upY, upZ) // Head is up (set to 0,-1,0 to look upside-down)
);
```

Which in turn I put into this bit of code that basically should be your “capture views” :

```
if (nviews == 32)
{
// ok we ARE in openGL !!
// simple case, simple screen
```

#ifndef SHOW_TEST

g_Neon_Device->SetRenderTarget(the_rt);

#endif

```
float fov = 0.244; // 14° in radians
float viewCone = 0.698; // 40° in radians
float cameraSize = 0.4; // for example
float aspectRatio = 1.6; // derived from calibration's screenW / screenH
Vector3 focalPosition;// = (0, 0, 0); // the center of the focal pane
focalPosition.zero();
float cameraDistance = -cameraSize / tan(fov / 2);
Vector3 cameraPosition = focalPosition + Vector3(0, 0, -cameraDistance);
proj = glm::perspectiveRH(
fov,
(float)L_RenderSizeX / (float)L_RenderSizeY,
//1.0f,
0.1f, // z near
20.0f); // z far
g_Neon_Device->camera(cameraPosition.x, cameraPosition.y, cameraPosition.z,
0, 0, 0,
0, 1, 0,
&view, true);
// now let's do what they do
//PRINTF("View Index %d\n", view_index);
// start at -viewCone * 0.5 and go up to viewCone * 0.5
float offsetAngle = ((float)view_index / (float)(nviews - 1) - 0.5) * viewCone;
// calculate the offset
float offset = cameraDistance * tan(offsetAngle);
// modify the view matrix (position)
view[0][3] += offset;
/*
g_Neon_Device->camera(cameraPosition.x+offset, cameraPosition.y, cameraPosition.z,
0, 0, 0,
0, 1, 0,
&view, true);
*/
// modify the projection matrix, relative to the camera size and aspect ratio
//proj[0, 2] += offset / (cameraSize * aspectRatio);
proj[0][2] += offset / (cameraSize * aspectRatio);
g_Neon_Device->setProjection(&proj);
g_Neon_Device->setCamera(&view);
g_Neon_Device->SetViewport(SubcamViewPort[view_index]);
if ( 0 == view_index ) g_Neon_Device->ClearScreen(0x00318eff); // RGBA
}
```

After all that with view_index going from 0 to 31 a set of 32 images are drawn that look “reasonably correct” each image fills a place in the quilt and is L_RenderSizeX * L_RenderSizeY big so 512 x 256.

At this point “Problem number 2” I re-created the “lenticular” shader as this :

#version 330 core

// Input vertex data, different for all executions of this shader.

layout(location = 0) in vec3 position;

layout(location = 1) in vec4 color;

layout(location = 2) in vec2 tex0;

// Output data ; will be interpolated for each fragment.

out VertexData {

vec4 Color;

vec2 TexCoord0;

vec2 TexCoord1;

};

// Values that stay constant for the whole mesh.

uniform mat4 worldview;

uniform mat4 proj;

void main(){

```
// Output position of the vertex : MV * position
vec4 modelViewPosition = worldview *vec4(position,1);
// Output in clip space
gl_Position = proj * modelViewPosition;
// Color is just passed by
Color = color;
// UV of the vertex. Just pass through, that's what they call iUv or uv
TexCoord0 = tex0;
TexCoord1 = tex0;
```

}

And …

// Giles’ fragment shader for Looking Glass

#version 330 core

// Interpolated values from the vertex shaders

in VertexData {

vec4 Color;

vec2 TexCoord0;

vec2 TexCoord1;

} inData;

// Ouput data

out vec4 color;

// Values that stay constant for the whole mesh.

uniform sampler2D quiltTexture;

uniform float pitch;

uniform float tilt;

uniform float center;

uniform float invView;

uniform float flipX;

uniform float flipY;

uniform float subp;

uniform float tilesX;

uniform float tilesY;

// “varying vec2 iUv;”+

vec2 texArr(vec3 uvz) {

float z = floor(uvz.z * tilesX * tilesY);

float x = (mod(z, tilesX) + uvz.x) / tilesX;

float y = (floor(z / tilesX) + uvz.y) / tilesY;

return vec2(x, y);

}

float Remap(float value, float from1, float to1, float from2, float to2) {

return (value - from1) / (to1 - from1) * (to2 - from2) + from2;

}

void main() {

vec4 rgb[3];

//vec3 nuv = vec3(iUv.xy, 0.0);

vec3 nuv = vec3(inData.TexCoord0, 0.0);

```
//Flip UVs if necessary
//nuv.x = (1.0 - flipX) * nuv.x + flipX * (1.0 - nuv.x);
//nuv.y = (1.0 - flipY) * nuv.y + flipY * (1.0 - nuv.y);
for (int i = 0; i < 3; i++) {
nuv.z = (inData.TexCoord0.x + float(i) * subp + inData.TexCoord0.y * tilt) * pitch - center;
//nuv.z = (inData.TexCoord0.x + float(i) * subp + inData.TexCoord0.y * tilt);
//nuv.z = (inData.TexCoord0.x + float(i)*0.0013+ inData.TexCoord0.y * 0.306);
nuv.z = mod(nuv.z + ceil(abs(nuv.z)), 1.0);
nuv.z = (1.0 - invView) * nuv.z + invView * (1.0 - nuv.z);
rgb[i] = texture2D(quiltTexture, texArr(vec3(inData.TexCoord0.x, inData.TexCoord0.y, nuv.z)));
//rgb[i] = texture2D(quiltTexture, nuv.xy);
}
//"gl_FragColor = vec4(rgb[0].r, rgb[1].g, rgb[2].b, 1);"+
color = vec4(rgb[0].r, rgb[1].g, rgb[2].b, 1);
//color = texture2D(quiltTexture,vec2(inData.TexCoord0.x,inData.TexCoord0.y) );
}
```

Various commented things was me testing stuff in pieces also that function ‘Remap’ as you can see seems to be never used.

Note that I noted your shader seems to have ‘a number of uniform NEVER used’ , that I took out , then I re-created your setShaderValues function in this way :

void LS_Display_Compositor::SetLookingGlassShaderValues(float dpi, float pitch, float slope, float screenH, float screenW, float center, int flipX, int flipY, int invView)

{

float screenInches = (float) g_Neon_Device->GetDisplayWidth() / dpi;

```
screenInches = 2560.0 / dpi;
float newPitch = pitch * screenInches;
//account for tilt in measuring pitch horizontally
newPitch *= cos(atan(1.0 / slope));
LookingGlassUniforms[LOOKING_PITCH] = newPitch;
//PRINTF("PITCH %f\n",)
float newTilt = (float) g_Neon_Device->GetDisplayWidth() / ((float) g_Neon_Device->GetDisplayHeight() * slope);
newTilt = 2560.0 / (1600.0)*slope;
if (flipX == 1)
newTilt *= -1;
LookingGlassUniforms[LOOKING_TILT] = newTilt;
//center
//I need the relationship between the amount of pixels I have moved over to the amount of lenticulars I have jumped
//ie how many pixels are there to a lenticular?
LookingGlassUniforms[LOOKING_CENTER] = center;
// should we invert ?
LookingGlassUniforms[LOOKING_INVVIEW] = invView;
//Should we flip it for peppers?
LookingGlassUniforms[LOOKING_FLIPX] = (float)flipX;
LookingGlassUniforms[LOOKING_FLIPY] = (float)flipY;
LookingGlassUniforms[LOOKING_SUBP] = 1.0 / (screenW * 3.0);
//tiles
LookingGlassUniforms[LOOKING_TILESX] = TilesX;
LookingGlassUniforms[LOOKING_TILESY] = TilesY;
g_Neon_Device->looking_glass_params = &LookingGlassUniforms[0];
```

}

Now here PROBLEM, I could not really figure out, by looking the .js code, what are the variables :

var newTilt = window.innerHeight / (window.innerWidth * slope);

window “what” ? The display itself so 2560x1600 ? The RT where the quilt is, so 2048x2048 ? Something else ?

If you render the quilt first in 2048x2048 that is “immutable” and the Looking Glass that is 2560x1600 that is “immutable” as well why you need that window and why you even care if it gets resized ?

Also the vertex shader part calls for a ‘proj’ and ‘worldview’ that fundamentally if I understood correctly you want to ‘blast it over a full quad’ so I use an ‘identity’ matrix for both i.e. 1:1 pixels mapping, I am not sure what’s the point of using that Three.Ortographic camera ( which BTW what matrix is that ? )

So in all and all I get ‘some images out’ that I am still not 100% are correct and ATM I cannot test with the device ( becuase I don’t have it here with me ) but they look like ‘could be something’ ( I will see if I can upload them later here ).

Ah yes I call that function like this :

```
SetLookingGlassShaderValues(338.0, 49.825218200683597, 5.2160325050354, 1600.0, 2560.0, -0.23396748304367066, 0, 0, 0);
```

So if you could clarify a little maybe I can get “somewhere” I still insist all this stuff needs MUCH better documentation.

Cheers.