I've been struggling to get this to work for quite a while now and finally succeeded yesterday after going through a number of suggested ways of doing it.
The method below (screenToWorld) works for the case where you want to pick a point on a plane that is parallel with the x & y axis and has a z value of zero.
The input values are as follows:
- viewMatrix - The view matrix, a float matrix of 16 elements.
- projMatrix - The projection matrix, a float matrix of 16 elements.
- screenX - The x-coordinate on the screen. The value must be between 0 (left edge) and 1 (right edge)
- screenY - The y-coordinate on the screen. The value must be between 0 (bottom edge) and 1 (top edge). Please note that this is inverted compared to Androids normal coordinate system.
The output is a point (x & y) on the plane with z equal to zero.
How it works
The point on the screen is not enough to calculate an exact point in 3D space (we don't have a z-value!) so what we do is to calculate the two extremes (that the click hit either the near plane or the far plane). Each of these two calculations result in a 3d point (x, y & z) and which together make up a line (I call the positions nearPos and farPos in the code). It is then up to us to device what point on that line was clicked.
To make things as simple as possible I limit myself to only caring about clicks on the z=0 plane. To get that coordinate we have to interpolate between nearPos and farPos. I do that based on the distance between nearPos and the z=0 plane.
Source based on a stackoverflow.com post by Erhannis
public static PointF screenToWorld(float[] viewMatrix,
float[] projMatrix, float screenX, float screenY) {
float[] projMatrix, float screenX, float screenY) {
float[] nearPos = unProject(viewMatrix, projMatrix, screenX, screenY, 0);
float[] farPos = unProject(viewMatrix, projMatrix, screenX, screenY, 1);
// The click occurred in somewhere on the line between the two points
// nearPos and farPos. We want to find
// nearPos and farPos. We want to find
// where that line intersects the plane at z=0
float distance = nearPos[2] / (nearPos[2] - farPos[2]); // Distance between nearPos and z=0
float x = nearPos[0] + (farPos[0] - nearPos[0]) * distance;
float y = nearPos[1] + (farPos[1] - nearPos[0]) * distance;
return new PointF(x, y);
}
private static float[] unProject(float[] viewMatrix,
float[] projMatrix, float screenX, float screenY, float depth) {
float[] projMatrix, float screenX, float screenY, float depth) {
float[] position = {0, 0, 0, 0};
int[] viewPort = {0, 0, 1, 1};
GLU.gluUnProject(screenX, screenY, depth, viewMatrix, 0, projMatrix, 0,
viewPort, 0, position, 0);
viewPort, 0, position, 0);
position[0] /= position[3];
position[1] /= position[3];
position[2] /= position[3];
position[3] = 1;
return position;
}
Hello,
ReplyDeleteThank you for the code
How can I get view and proj Matrix
Hi Muhridin, I guess that depends on many things :). Are you using some 3D engine or library or working with a GLSurfaceView and setting everything up yourself? If it's the latter, then at some point you need to bind those matrices to the shader arguments, so then if should be readily available.
Delete