Why matrices seems transposed and how to change it

[frozar]

This is not a bug report, sorry.

This issue maybe related to #33

First of all, thank you for WebGL foundation, 1 and 2, awesome!

I have read many times the article 2D matrices as well as matrix vs math and I think I finally figure out why there is this baffling issue around matrices.

In the 2D matrices article, you show in detail the way matrices store their component (conversion from 1D array to matrix interpretation) and you also show the matrix multiplication to use to make it works.

As I'm a math person, as you warned in matrix vs math article, the matrices handling seems confusing to me. In fact I'd rather say, it seems like everything gets transposed. And the way the multiplication between matrices is defined make it works if the 2 input matrices are transposed and store the result in the transpose way, it's coherent. (I made the computation of the projection matrix from 2D matrices article by hand to convince myself (and check the results by printing some value from m3.multiplication function).)

Then I check the way the computed matrix is provided to a shader:

gl.uniformMatrix3fv(matrixLocation, false, matrix);

The second argument of this function is the transpose option. From the MDN web docs:

transpose
A GLboolean specifying whether to transpose the matrix. Must be false.

I don't understand why the documentation tells that the option must be false!?!

Eventually if it is possible to send the computed matrix in a transpose way to the GPU (maybe manually if the WebGL function doesn't do it for us), every computation on the CPU side can be done as usual, in the math way (and avoid me some headache).

It should really reconcile the math people with matrices manipulation in the excellent WebGL foundation.

PS: I know it's certainly impossible to change the way all the articles are written and the implementation associated with them to. But I think that you can still mention this trick somewhere 😉

[greggman]

Sorry I didn't respond to this.

As the article says

OpenGL claims computer science rows are columns

So

[
   1, 2, 3, 4,
   5, 6, 7, 8,
   9,10,11,12,
  13,14,15,16,
]

Is interpreted as

[
  1, 5, 9,13,
  2, 6,10,14,
  3, 7,11,15,
  4, 8,12,16,
]

Another way to write your code would be like this

const firstColumn = 
  1,
  2,
  3,
  4,
];

const secondColumn = 
  5,
  6,
  7,
  8,
];

const thirdColumn = 
  9,
  10,
  11,
  12,
];

const forthColumn =
  13,
  14,
  15,
  16,
];

const mathMatrix = [ ...firstColumm, ...secondColumn, ...thirdColumn, ...forthColumn ];

I don't know if that helps visualize how OpenGL want's you to see it or not but above I declared four column looking things and then at the bottom I laid them left to right as columns.

If you print mathMatrix though it will print 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16 which if interpreted as a 4x4 matrix in computer science is generally called considered row 1 being 1,2,3,4 but OpenGL says think of that as a column.

Then the math works as you expect right?

As for not being able to transpose the matrices I have no idea why they chose that except that OpenGL ES 2.0 was designed for fairly low powered machines. The transpose takes extra time so in the interest of pushing devs to write optimal code maybe they required it to be false. As for why it's there at all OpenGL ES 2.0 is based on or at least inspired by OpenGL (not ES) where that parameter already existed so rather than change the API to not have the parameter they left it there but require it to be false. Similar to the border parameter on texImage2D.

[frozar]

First of all, thank you again for all the work you put in reading the WebGL fundation books, really useful. Thank to your sharing knowledge, I was able to build this tiny project to visualise a 3D mandelbrot fractal:
https://frozar.github.io/mandelbrot-viewer/

Satisfying 😄.

Back to your answer, I completely understand there's 2 way to interpret a 1D array as a 2D matrix: either the element are stored in a Row Major fashion or a Column Major fashion. The first time I've been sensitised to this fact was when I were using an old linear algebra library, LAPACK.
OpenGL uses the Column Major memory layout and math people likes to think in Row Major memory layout.

The reason why OpenGL uses the Column Major layout rather than the Row Major layout is arbitrary I think (or maybe impose by hardware and the way hardware works is arbitrary (until hardware will be AI driven designed))... But OpenGL, or at least WebGL, is able to transpose a matrix before to send it to the GPU. Even if they say that the transpose argument of gl.uniformMatrix* function must be false, it's not a requirement.

In my little fractal project, I set the transpose argument to true:

https://github.com/frozar/mandelbrot-viewer/blob/9ffad71e4212ad59df651c2137ab66c18a41cd9d/index.html#L565-L566

(the implementation is not the cleanest possible one, but still, it's shareable)

I do every matrix computation in the Row Major fashion and it works. Sure I pay an extra cost due to the transpose at each frame computation, but for me it's reasonable and now it's easier follow the computation.

This issue should be closed from my side.

[greggman]

you're using WebGL2 which allows transpose be true. In WebGL1 it would generate an error

I'm also not sure what you mean by "it's easier to follow the computation"

To put another way, from the math POV, each column is a vector. So, just like the article points out, in C++ you'd do this

struct vec4 {
  float v[4];
}

struct mat4 {
  mat4(const mat4& col0, const mat4& col1, const mat4& col2, const mat4& col3) {
    columns[0] = col0;
    columns[1] = col1;
    columns[2] = col2;
    columns[3] = col3;
  }
  vec4 columns[4];
}

and you'd could build columns

vec4 xAxisColumn = 
  xx,
  xy,
  xz,
  0,
};
vec4 yAxisColumn = {
  yx,
  yy,
  yz,
  0,
}; 
vec4 yAxisColumn = {
  zx,
  zy,
  zz,
  0,
},
vec4 translationColumn = {
  tx,
  ty,
  tz,
  1,
};

and them make a matrix from those columns

mat4 matrix = mat4(xAxisColumn, yAxisColumn, zAxisColumn, translationColumn);

and you can pass that matrix directly into GL

glUniformMatrix4fv(someMatrixLocation, GL_FALSE, &matrix[0][0]);

And AFAIK it works as expected, the mathy way, including in GLSL

attribute vec4 somePosition;
uniform mat4 someMatrix;
...
  gl_Position = someMatrix * somePosition;

Those all work exactly how math people expect them to work. No reason to set transpose to true so I'm not convinced the reason it works this way in GL is arbitrary.

The only issue come up if you write them as a 4x4 matrix

mat4 m = {
   xx, xy, xz, 0,
   yx, yy, yz, 0,
   zx, zy, zz, 0,
   tx, ty, tz, 1,
};

That looks wrong to a math person's eyes.

Further, there are hardware issues. The math person wants to be able to pull out columns. Let's say you want the z-axis

vec4 zAxis = someMat4.columns[2]

just works.

vs if things are transposed you'd have to do grab every 4th float. That would be horribly slow.

you're free to do what you want but If you're setting transpose to true you're basically going against the tide of the entire graphics industry. There's nothing wrong with that except possible that you'll get used to a non-standard way and find all other examples and people's code to be confusing.

[frozar]

I understand your answer, I think that I just get used to weird habit. For example, when I see a matrix like Rx(th) in the follow picture:

I use to implement it as:

mat4 Rx = {
   1,       0,        0, 0,
   0, cos(th), -sin(th), 0,
   0, sin(th),  cos(th), 0,
   0,       0,        0, 1,
};

And when I implement matrix multiplication, I use to implement it and think about it as shown in this picture:
https://texample.net/media/tikz/examples/PNG/matrix-multiplication.png

For me it's easier to think about implementation if 1D arrays are Row Major. I well understood that with this representation there is the downside that to retrieve retrieve a column you have to pick numbers every 4th float. And of course the cost of the transposition before to send it to the GPU.
I'm happy to know that WebGL2 allows the final transposition, it saves me.

Sad to know that generally code are implement Column Major manner. Thank you for your answer.