Skip to content
Snippets Groups Projects
Commit 9e9cddab authored by Patrik Huber's avatar Patrik Huber
Browse files

Added render.hpp with the render function

parent 89356c24
No related branches found
No related tags found
No related merge requests found
/*
* Eos - A 3D Morphable Model fitting library written in modern C++11/14.
*
* File: include/eos/render/render.hpp
*
* Copyright 2014, 2015 Patrik Huber
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once
#ifndef RENDER_HPP_
#define RENDER_HPP_
#include "eos/render/detail/render_detail.hpp"
#include "eos/render/utils.hpp"
#include "opencv2/core/core.hpp"
#ifdef WIN32
#define BOOST_ALL_DYN_LINK // Link against the dynamic boost lib. Seems to be necessary because we use /MD, i.e. link to the dynamic CRT.
#define BOOST_ALL_NO_LIB // Don't use the automatic library linking by boost with VS2010 (#pragma ...). Instead, we specify everything in cmake.
#endif
#include "boost/optional.hpp"
#include <array>
#include <vector>
#include <memory>
namespace eos {
namespace render {
/**
* This file implements a software renderer conforming to OpenGL conventions. The
* following are implementation notes, mostly for reference, or as a reminder of
* what exactly is going on. Don't try to understand them :-)
*
* The renderer was initially based on code by Wojciech Sterna
* (http://maxest.gct-game.net/content/vainmoinen/index.html), however, it has since
* then been completely rewritten. Still I'd like to thank him for making his code
* available and bravely answering my questions via email.
*
* Coordinate systems:
* When specifying the vertices: +x = right, +y = up, we look into -z.
* So z = 0.5 is in front of 0.0.
* Z-Buffer:
*
* Shirley: Specify n and f with negative values. which makes sense b/c the points
* are along the -z axis.
* Consequences: notably: orthogonal(2, 3): Shirley has denominator (n-f).
* In what space are the points in Shirley after this?
* OGL: We're in the orthographic viewing volume looking down -z.
* However, n and f are specified positive.
* B/c the 3D points in front of the cam obviously still have negative z values, the
* z-value is negated. So: n = 0.1, f = 100; With the given OpenGL ortho matrix,
* it means a point on the near-plane which will have z = -0.1 will land up
* on z_clip (which equals z_ndc with ortho because w=1) = -1, and a point on
* the far plane z = -100 will have z_ndc = +1.
*
* That's also why in the perspective case, w_clip is set to -z_eye because
* to project a point the formula is $x_p = (-n * x_e)/z_e$ (because our near is
* specified with positive values, but the near-plane is _really_ at -n); but now we
* just move the minus-sign to the denominator, $x_p = (n * x_e)/-z_e$, so in the projection matrix we can use
* the (positive) n and f values and afterwards we divide by w = -z_e.
*
* http://www.songho.ca/opengl/gl_projectionmatrix.html
*
* Random notes:
* clip-space: after applying the projection matrix.
* ndc: after division by w
* NDC cube: the range of x-coordinate from [l, r] to [-1, 1], the y-coordinate from [b, t] to [-1, 1] and the z-coordinate from [n, f] to [-1, 1].
*
* Note/Todo: I read that in screen space, OpenGL transform the z-values again to be between 0 and 1?
*
* In contrast to OGL, this renderer doesn't have state, it's just a function that gets called with all
* necessary parameters. It's easiest for our purposes.
*
* Here's the whole rendering pipeline:
* Model space
* -> model transforms
* World space
* -> camera (view/eye) transform
* View / eye / camera space ("truncated pyramid frustum". In case of ortho, it's already rectangular.)
* -> perspective/ortho projection
* Clip coords (x_c, y_c, z_c, w_c); the z-axis is flipped now. z [z=-n, z=-f] is mapped to [-1, +1] in case of ortho, but not yet in case of persp (it's also flipped though), but the not-[-1,1]-range is fine as we test against w_c. I.e. the larger the z-value, the further back we are.
* Do frustum culling (clipping) here. Test the clip-coords with w_c, and discard if a tri is completely outside.
* Of the partially visible tris, clip them against the near-plane and construct the visible part of the triangle.
* We only do this for the near-plane here. Clipping to the near plane must be done here because after w-division triangles crossing it would get distorted.
* "Then, OpenGL will reconstruct the edges of the polygon where clipping occurs."
* -> Then divide by the w component of the clip coordinates
* NDC. (now only 3D vectors: [x_ndc, y_ndc, z_ndc]). nearest points have z=-1, points on far plane have z=+1.
* -> window transform. (also, OGL does some more to the z-buffer?)
* Screen / window space
* Directly after window-transform (still processing triangles), do backface culling with areVerticesCCWInScreenSpace()
* Directly afterwards we calculate the triangle's bounding box and clip x/y (screen) against 0 and the viewport width/height.
* Rasterising: Clipping against the far plane here by only drawing those pixels with a z-value of <= 1.0f.
*
* OGL: "both clipping (frustum culling) and NDC transformations are integrated into GL_PROJECTION matrix"
*
* Note: In both the ortho and persp case, points at z=-n end up at -1, z=-f at +1. In case of persp proj., this happens only after the divide by w.
*/
/**
* Renders the given mesh onto a 2D image using 4x4 model-view and
* projection matrices. Conforms to OpenGL conventions.
*
* @param[in] mesh A 3D mesh.
* @param[in] model_view_matrix A 4x4 OpenGL model-view matrix.
* @param[in] projection_matrix A 4x4 orthographic or perspective OpenGL projection matrix.
* @param[in] viewport_width Screen width.
* @param[in] viewport_height Screen height.
* @param[in] texture An optional texture map (TODO: Not optional yet!).
* @param[in] enable_backface_culling Whether the renderer should perform backface culling. If true, only draw triangles with vertices ordered CCW in screen-space.
* @param[in] enable_near_clipping Screen height.
* @param[in] enable_far_clipping Screen height.
* @return A pair with the colourbuffer as its first element and the depthbuffer as the second element.
*/
std::pair<cv::Mat, cv::Mat> render(Mesh mesh, cv::Mat model_view_matrix, cv::Mat projection_matrix, int viewport_width, int viewport_height, const Texture& texture, bool enable_backface_culling = false, bool enable_near_clipping = true, bool enable_far_clipping = true)
{
// Some internal documentation / old todos or notes:
// maybe change and pass depthBuffer as an optional arg (&?), because usually we never need it outside the renderer. Or maybe even a getDepthBuffer().
// modelViewMatrix goes to eye-space (camera space), projection does ortho or perspective proj.
// bool enable_texturing = false; Maybe re-add later, not sure
// take a cv::Mat texture instead and convert to Texture internally? no, we don't want to recreate mipmap levels on each render() call.
assert(mesh.vertices.size() == mesh.colors.size() || mesh.colors.empty()); // The number of vertices has to be equal for both shape and colour, or, alternatively, it has to be a shape-only model.
assert(mesh.vertices.size() == mesh.texcoords.size() || mesh.texcoords.empty()); // same for the texcoords
// another assert: If cv::Mat texture != empty, then we need texcoords?
using cv::Mat;
using std::vector;
Mat colourbuffer = Mat::zeros(viewport_height, viewport_width, CV_8UC4); // make sure it's CV_8UC4?
Mat depthbuffer = std::numeric_limits<float>::max() * Mat::ones(viewport_height, viewport_width, CV_64FC1);
// Vertex shader:
//processedVertex = shade(Vertex); // processedVertex : pos, col, tex, texweight
// Assemble the vertices, project to clip space, and store as detail::Vertex (the internal representation):
vector<detail::Vertex> clipspace_vertices;
clipspace_vertices.reserve(mesh.vertices.size());
for (int i = 0; i < mesh.vertices.size(); ++i) { // "previously": mesh.vertex
Mat clipspace_coords = projection_matrix * model_view_matrix * Mat(mesh.vertices[i]);
cv::Vec3f vertex_colour;
if (mesh.colors.empty()) {
vertex_colour = cv::Vec3f(0.5f, 0.5f, 0.5f);
}
else {
vertex_colour = mesh.colors[i];
}
clipspace_vertices.push_back(detail::Vertex(clipspace_coords, vertex_colour, mesh.texcoords[i]));
}
// All vertices are in clip-space now.
// Prepare the rasterisation stage.
// For every vertex/tri:
vector<detail::TriangleToRasterize> triangles_to_raster;
for (const auto& tri_indices : mesh.tvi) {
// Todo: Split this whole stuff up. Make a "clip" function, ... rename "processProspective..".. what is "process"... get rid of "continue;"-stuff by moving stuff inside process...
// classify vertices visibility with respect to the planes of the view frustum
// we're in clip-coords (NDC), so just check if outside [-1, 1] x ...
// Actually we're in clip-coords and it's not the same as NDC. We're only in NDC after the division by w.
// We should do the clipping in clip-coords though. See http://www.songho.ca/opengl/gl_projectionmatrix.html for more details.
// However, when comparing against w_c below, we might run into the trouble of the sign again in the affine case.
// 'w' is always positive, as it is -z_camspace, and all z_camspace are negative.
unsigned char visibility_bits[3];
for (unsigned char k = 0; k < 3; k++)
{
visibility_bits[k] = 0;
float x_cc = clipspace_vertices[tri_indices[k]].position[0];
float y_cc = clipspace_vertices[tri_indices[k]].position[1];
float z_cc = clipspace_vertices[tri_indices[k]].position[2];
float w_cc = clipspace_vertices[tri_indices[k]].position[3];
if (x_cc < -w_cc) // true if outside of view frustum. False if on or inside the plane.
visibility_bits[k] |= 1; // set bit if outside of frustum
if (x_cc > w_cc)
visibility_bits[k] |= 2;
if (y_cc < -w_cc)
visibility_bits[k] |= 4;
if (y_cc > w_cc)
visibility_bits[k] |= 8;
if (enable_near_clipping && z_cc < -w_cc) // near plane frustum clipping
visibility_bits[k] |= 16;
if (enable_far_clipping && z_cc > w_cc) // far plane frustum clipping
visibility_bits[k] |= 32;
} // if all bits are 0, then it's inside the frustum
// all vertices are not visible - reject the triangle.
if ((visibility_bits[0] & visibility_bits[1] & visibility_bits[2]) > 0)
{
continue;
}
// all vertices are visible - pass the whole triangle to the rasterizer. = All bits of all 3 triangles are 0.
if ((visibility_bits[0] | visibility_bits[1] | visibility_bits[2]) == 0)
{
boost::optional<detail::TriangleToRasterize> t = detail::process_prospective_tri(clipspace_vertices[tri_indices[0]], clipspace_vertices[tri_indices[1]], clipspace_vertices[tri_indices[2]], viewport_width, viewport_height, enable_backface_culling);
if (t) {
triangles_to_raster.push_back(*t);
}
continue;
}
// at this moment the triangle is known to be intersecting one of the view frustum's planes
std::vector<detail::Vertex> vertices;
vertices.push_back(clipspace_vertices[tri_indices[0]]);
vertices.push_back(clipspace_vertices[tri_indices[1]]);
vertices.push_back(clipspace_vertices[tri_indices[2]]);
// split the triangle if it intersects the near plane:
if (enable_near_clipping)
{
vertices = detail::clip_polygon_to_plane_in_4d(vertices, cv::Vec4f(0.0f, 0.0f, -1.0f, -1.0f)); // "Normal" (or "4D hyperplane") of the near-plane. I tested it and it works like this but I'm a little bit unsure because Songho says the normal of the near-plane is (0,0,-1,1) (maybe I have to switch around the < 0 checks in the function?)
}
// triangulation of the polygon formed of vertices array
if (vertices.size() >= 3)
{
for (unsigned char k = 0; k < vertices.size() - 2; k++)
{
boost::optional<detail::TriangleToRasterize> t = detail::process_prospective_tri(vertices[0], vertices[1 + k], vertices[2 + k], viewport_width, viewport_height, enable_backface_culling);
if (t) {
triangles_to_raster.push_back(*t);
}
}
}
}
// Fragment/pixel shader: Colour the pixel values
// for every tri:
for (const auto& tri : triangles_to_raster) {
detail::raster_triangle(tri, colourbuffer, depthbuffer, texture, enable_far_clipping);
}
return std::make_pair(colourbuffer, depthbuffer);
};
} /* namespace render */
} /* namespace eos */
#endif /* RENDER_HPP_ */
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment