eos
0.7.1
|
3D mesh and texture extraction functionality. More...
Classes | |
struct | Mesh |
This class represents a 3D mesh consisting of vertices, vertex colour information and texture coordinates. More... | |
class | Texture |
Represents a texture for rendering. More... | |
Enumerations | |
enum | TextureInterpolation { NearestNeighbour, Bilinear, Area } |
Functions | |
void | write_obj (Mesh mesh, std::string filename) |
Writes the given Mesh to an obj file that for example can be read by MeshLab. More... | |
void | write_textured_obj (Mesh mesh, std::string filename) |
Writes an obj file of the given Mesh, including texture coordinates, and an mtl file containing a reference to the isomap. More... | |
std::pair< cv::Mat, cv::Mat > | render (Mesh mesh, cv::Mat model_view_matrix, cv::Mat projection_matrix, int viewport_width, int viewport_height, const boost::optional< Texture > &texture=boost::none, bool enable_backface_culling=false, bool enable_near_clipping=true, bool enable_far_clipping=true) |
std::pair< cv::Mat, cv::Mat > | render_affine (Mesh mesh, cv::Mat affine_camera_matrix, int viewport_width, int viewport_height, bool do_backface_culling=true) |
cv::Mat | extract_texture (Mesh mesh, cv::Mat affine_camera_matrix, cv::Mat image, cv::Mat depthbuffer, bool compute_view_angle, TextureInterpolation mapping_type, int isomap_resolution) |
cv::Mat | extract_texture (Mesh mesh, cv::Mat affine_camera_matrix, cv::Mat image, bool compute_view_angle=false, TextureInterpolation mapping_type=TextureInterpolation::NearestNeighbour, int isomap_resolution=512) |
cv::Vec2f | clip_to_screen_space (const cv::Vec2f &clip_coordinates, int screen_width, int screen_height) |
cv::Vec2f | screen_to_clip_space (const cv::Vec2f &screen_coordinates, int screen_width, int screen_height) |
cv::Vec3f | calculate_face_normal (const cv::Vec3f &v0, const cv::Vec3f &v1, const cv::Vec3f &v2) |
cv::Mat | draw_texcoords (Mesh mesh, cv::Mat image=cv::Mat()) |
unsigned int | get_max_possible_mipmaps_num (unsigned int width, unsigned int height) |
bool | is_power_of_two (int x) |
Texture | create_mipmapped_texture (cv::Mat image, unsigned int mipmapsNum=0) |
3D mesh and texture extraction functionality.
|
strong |
The interpolation types that can be used to map the texture from the original image to the isomap.
cv::Vec3f eos::render::calculate_face_normal | ( | const cv::Vec3f & | v0, |
const cv::Vec3f & | v1, | ||
const cv::Vec3f & | v2 | ||
) |
Calculates the normal of a face (or triangle), i.e. the per-face normal. Return normal will be normalised. Assumes the triangle is given in CCW order, i.e. vertices in counterclockwise order on the screen are front-facing.
[in] | v0 | First vertex. |
[in] | v1 | Second vertex. |
[in] | v2 | Third vertex. |
|
inline |
Transforms a point from clip space ([-1, 1] x [-1, 1]) to image (screen) coordinates, i.e. the window transform. Note that the y-coordinate is flipped because the image origin is top-left while in clip space top is +1 and bottom is -1. No z-division is performed. Note: It should rather be called from NDC to screen space?
Exactly conforming to the OpenGL viewport transform, except that we flip y at the end. Qt: Origin top-left. OpenGL: bottom-left. OCV: top-left.
[in] | clip_coordinates | A point in clip coordinates. |
[in] | screen_width | Width of the screen or window. |
[in] | screen_height | Height of the screen or window. |
cv::Mat eos::render::draw_texcoords | ( | Mesh | mesh, |
cv::Mat | image = cv::Mat() |
||
) |
Draws the texture coordinates (uv-coords) of the given mesh into an image by looping over the triangles and drawing each triangle's texcoords.
[in] | mesh | A mesh with texture coordinates. |
[in] | image | An optional image to draw onto. |
|
inline |
Extracts the texture of the face from the given image and stores it as isomap (a rectangular texture map). This function can be used if a depth buffer has already been computed. To just run the texture extraction, see the overload extract_texture(Mesh, cv::Mat, cv::Mat, TextureInterpolation, int).
It might be wise to remove this overload as it can get quite confusing with the zbuffer. Obviously the depthbuffer given should have been created with the same (affine or ortho) projection matrix than the texture extraction is called with.
[in] | mesh | A mesh with texture coordinates. |
[in] | affine_camera_matrix | An estimated 3x4 affine camera matrix. |
[in] | image | The image to extract the texture from. |
[in] | depthbuffer | A pre-calculated depthbuffer image. |
[in] | compute_view_angle | A flag whether the view angle of each vertex should be computed and returned. If set to true, the angle will be encoded into the alpha channel (0 meaning occluded or facing away 90°, 127 meaning facing a 45° angle and 255 meaning front-facing, and all values in between). If set to false, the alpha channel will only contain 0 for occluded vertices and 255 for visible vertices. |
[in] | mapping_type | The interpolation type to be used for the extraction. |
[in] | isomap_resolution | The resolution of the generated isomap. Defaults to 512x512. |
|
inline |
Extracts the texture of the face from the given image and stores it as isomap (a rectangular texture map).
Note/Todo: Only use TextureInterpolation::NearestNeighbour for the moment, the other methods don't have correct handling of the alpha channel (and will most likely throw an exception).
Todo: These should be renamed to extract_texture_affine? Can we combine both cases somehow? Or an overload with RenderingParameters?
For TextureInterpolation::NearestNeighbour, returns a 4-channel isomap with the visibility in the 4th channel (0=invis, 255=visible).
[in] | mesh | A mesh with texture coordinates. |
[in] | affine_camera_matrix | An estimated 3x4 affine camera matrix. |
[in] | image | The image to extract the texture from. Should be 8UC3, other types not supported yet. |
[in] | compute_view_angle | A flag whether the view angle of each vertex should be computed and returned. If set to true, the angle will be encoded into the alpha channel (0 meaning occluded or facing away 90°, 127 meaning facing a 45° angle and 255 meaning front-facing, and all values in between). If set to false, the alpha channel will only contain 0 for occluded vertices and 255 for visible vertices. |
[in] | mapping_type | The interpolation type to be used for the extraction. |
[in] | isomap_resolution | The resolution of the generated isomap. Defaults to 512x512. |
std::pair<cv::Mat, cv::Mat> eos::render::render | ( | Mesh | mesh, |
cv::Mat | model_view_matrix, | ||
cv::Mat | projection_matrix, | ||
int | viewport_width, | ||
int | viewport_height, | ||
const boost::optional< Texture > & | texture = boost::none , |
||
bool | enable_backface_culling = false , |
||
bool | enable_near_clipping = true , |
||
bool | enable_far_clipping = true |
||
) |
This file implements a software renderer conforming to OpenGL conventions. The following are implementation notes, mostly for reference, or as a reminder of what exactly is going on. Don't try to understand them :-)
The renderer was initially based on code by Wojciech Sterna (http://maxest.gct-game.net/content/vainmoinen/index.html), however, it has since then been completely rewritten. Still I'd like to thank him for making his code available and bravely answering my questions via email.
Coordinate systems: When specifying the vertices: +x = right, +y = up, we look into -z. So z = 0.5 is in front of 0.0. Z-Buffer:
Shirley: Specify n and f with negative values. which makes sense b/c the points are along the -z axis. Consequences: notably: orthogonal(2, 3): Shirley has denominator (n-f). In what space are the points in Shirley after this? OGL: We're in the orthographic viewing volume looking down -z. However, n and f are specified positive.
B/c the 3D points in front of the cam obviously still have negative z values, the z-value is negated. So: n = 0.1, f = 100; With the given OpenGL ortho matrix, it means a point on the near-plane which will have z = -0.1 will land up on z_clip (which equals z_ndc with ortho because w=1) = -1, and a point on the far plane z = -100 will have z_ndc = +1.
That's also why in the perspective case, w_clip is set to -z_eye because to project a point the formula is $x_p = (-n * x_e)/z_e$ (because our near is specified with positive values, but the near-plane is really at -n); but now we just move the minus-sign to the denominator, $x_p = (n * x_e)/-z_e$, so in the projection matrix we can use the (positive) n and f values and afterwards we divide by w = -z_e.
http://www.songho.ca/opengl/gl_projectionmatrix.html
Random notes: clip-space: after applying the projection matrix. ndc: after division by w NDC cube: the range of x-coordinate from [l, r] to [-1, 1], the y-coordinate from [b, t] to [-1, 1] and the z-coordinate from [n, f] to [-1, 1].
Note/Todo: I read that in screen space, OpenGL transform the z-values again to be between 0 and 1?
In contrast to OGL, this renderer doesn't have state, it's just a function that gets called with all necessary parameters. It's easiest for our purposes.
Here's the whole rendering pipeline: Model space -> model transforms World space -> camera (view/eye) transform View / eye / camera space ("truncated pyramid frustum". In case of ortho, it's already rectangular.) -> perspective/ortho projection Clip coords (x_c, y_c, z_c, w_c); the z-axis is flipped now. z [z=-n, z=-f] is mapped to [-1, +1] in case of ortho, but not yet in case of persp (it's also flipped though), but the not-[-1,1]-range is fine as we test against w_c. I.e. the larger the z-value, the further back we are. Do frustum culling (clipping) here. Test the clip-coords with w_c, and discard if a tri is completely outside. Of the partially visible tris, clip them against the near-plane and construct the visible part of the triangle. We only do this for the near-plane here. Clipping to the near plane must be done here because after w-division triangles crossing it would get distorted. "Then, OpenGL will reconstruct the edges of the polygon where clipping occurs." -> Then divide by the w component of the clip coordinates NDC. (now only 3D vectors: [x_ndc, y_ndc, z_ndc]). nearest points have z=-1, points on far plane have z=+1. -> window transform. (also, OGL does some more to the z-buffer?) Screen / window space Directly after window-transform (still processing triangles), do backface culling with areVerticesCCWInScreenSpace() Directly afterwards we calculate the triangle's bounding box and clip x/y (screen) against 0 and the viewport width/height. Rasterising: Clipping against the far plane here by only drawing those pixels with a z-value of <= 1.0f.
OGL: "both clipping (frustum culling) and NDC transformations are integrated into GL_PROJECTION matrix"
Note: In both the ortho and persp case, points at z=-n end up at -1, z=-f at +1. In case of persp proj., this happens only after the divide by w. Renders the given mesh onto a 2D image using 4x4 model-view and projection matrices. Conforms to OpenGL conventions.
[in] | mesh | A 3D mesh. |
[in] | model_view_matrix | A 4x4 OpenGL model-view matrix. |
[in] | projection_matrix | A 4x4 orthographic or perspective OpenGL projection matrix. |
[in] | viewport_width | Screen width. |
[in] | viewport_height | Screen height. |
[in] | texture | An optional texture map. If not given, vertex-colouring is used. |
[in] | enable_backface_culling | Whether the renderer should perform backface culling. If true, only draw triangles with vertices ordered CCW in screen-space. |
[in] | enable_near_clipping | Whether vertices should be clipped against the near plane. |
[in] | enable_far_clipping | Whether vertices should be clipped against the far plane. |
std::pair<cv::Mat, cv::Mat> eos::render::render_affine | ( | Mesh | mesh, |
cv::Mat | affine_camera_matrix, | ||
int | viewport_width, | ||
int | viewport_height, | ||
bool | do_backface_culling = true |
||
) |
Renders the mesh using the given affine camera matrix and returns the colour and depth buffer images. The camera matrix should be one estimated with fitting::estimate_affine_camera (Hartley & Zisserman algorithm).
If the given mesh is a shape-only mesh without vertex-colour information, the vertices will be rendered in grey.
#Todo: May consider an overload where we pass in an image, use that as colourbuffer and draw over it. #Todo: Add texture rendering to this. Then, create an additional function in extract_texure that is fully optimised for only the extraction.
[in] | mesh | A 3D mesh. |
[in] | affine_camera_matrix | 3x4 affine camera matrix. |
[in] | viewport_width | Screen width. |
[in] | viewport_height | Screen height. |
[in] | do_backface_culling | Whether the renderer should perform backface culling. |
|
inline |
Transforms a point from image (screen) coordinates to clip space ([-1, 1] x [-1, 1]). Note that the y-coordinate is flipped because the image origin is top-left while in clip space top is +1 and bottom is -1.
[in] | screen_coordinates | A point in screen coordinates. |
[in] | screen_width | Width of the screen or window. |
[in] | screen_height | Height of the screen or window. |
|
inline |
Writes the given Mesh to an obj file that for example can be read by MeshLab.
If the mesh contains vertex colour information, it will be written to the obj as well.
[in] | mesh | The mesh to save as obj. |
[in] | filename | Output filename (including ".obj"). |
|
inline |
Writes an obj file of the given Mesh, including texture coordinates, and an mtl file containing a reference to the isomap.
The obj will contain texture coordinates for the mesh, and the mtl file will link to a file named <filename>.isomap.png. Note that the texture (isomap) has to be saved separately.
[in] | mesh | The mesh to save as obj. |
[in] | filename | Output filename. |