gluProject transforms the specified object coordinates into window coordinates using model, proj, and view. The result is stored in winX, winY, and winZ. A return value of GL_TRUE indicates success, a return value of GL_FALSE indicates failure.To compute the coordinates, let $v ~=~ ("objX", "objY", "objZ", 1.0)$ represented as a matrix with 4 rows and 1 column. Then
gluProject computes $v sup prime$ as follows:
v sup prime ~=~ P ~times~ M ~times~ v
where $P$ is the current projection matrix
proj, $M$ is the current modelview matrix model (both represented as $4 times 4$ matrices in column-major order) and '$times$' represents matrix multiplication.
The window coordinates are then computed as follows:
"winX" ~=~ "view" (0) ~+~ "view" (2) ~*~ (v sup prime (0) ~+~ 1)~/~2
"winY" ~=~ "view" (1) ~+~ "view" (3)~*~(v sup prime (1) ~+~ 1)~/~2
"winZ" ~=~ (v sup prime (2) ~+~ 1)~/~ 2