return{current_mesh,rendering_params};// I think we could also work with a Mat face_instance in this function instead of a Mesh, but it would convolute the code more (i.e. more complicated to access vertices).
// Fit the occluding (away-facing) contour using the detected contour LMs:
return{current_meshs,rendering_params};// I think we could also work with a Mat face_instance in this function instead of a Mesh, but it would convolute the code more (i.e. more complicated to access vertices).
};
};
// philipp helper, todo: is the blendshape coefficient thing really solved correctly here?
assert(affine_camera_matrix.size()==landmarks.size()&&landmarks.size()==vertex_ids.size());// same number of instances (i.e. images/frames) for each of them
Matbasis_rows=morphable_model.get_shape_model().get_normalised_pca_basis(vertex_ids[i]);// In the paper, the not-normalised basis might be used? I'm not sure, check it. It's even a mess in the paper. PH 26.5.2014: I think the normalised basis is fine/better.
// Sigma(i, i) = sqrt(sigma_squared_2D), but then Omega is Sigma.t() * Sigma (squares the diagonal) - so we just assign 1/sigma_squared_2D to Omega here:
Omega.at<float>(i,i)=1.0f/sigma_squared_2D;// the higher the sigma_squared_2D, the smaller the diagonal entries of Sigma will be
}
// The landmarks in matrix notation (in homogeneous coordinates), $3N\times 1$
// The landmarks in matrix notation (in homogeneous coordinates), $3N\times 1$
Matbasis_rows=morphable_model.get_shape_model().get_normalised_pca_basis(vertex_ids[k][i]);// In the paper, the not-normalised basis might be used? I'm not sure, check it. It's even a mess in the paper. PH 26.5.2014: I think the normalised basis is fine/better.
Sigma.at<float>(Sigma_index,Sigma_index)=1.0f/std::sqrt(sigma_squared_2D);// the higher the sigma_squared_2D, the smaller the diagonal entries of Sigma will be
++Sigma_index;
}
//Mat Omega = Sigma.t() * Sigma; // just squares the diagonal
// => moved outside the loop
// The landmarks in matrix notation (in homogeneous coordinates), $3N\times 1$
//Mat y = Mat::ones(3 * num_landmarks, 1, CV_32FC1);
boolnon_singular=cv::solve(AtOmegaAReg,-A.t()*Omega.t()*b,c_s,cv::DECOMP_SVD);// DECOMP_SVD calculates the pseudo-inverse if the matrix is not invertible.
boolnon_singular=cv::solve(AtOmegaAReg,-A.t()*Omega.t()*b,c_s,cv::DECOMP_SVD);// DECOMP_SVD calculates the pseudo-inverse if the matrix is not invertible.
// Because we're using SVD, non_singular will always be true. If we were to use e.g. Cholesky, we could return an expected<T>.
// Because we're using SVD, non_singular will always be true. If we were to use e.g. Cholesky, we could return an expected<T>.
/* using RowMajorMatrixXf = Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
Mat c_s(res.rows(), res.cols(), CV_32FC1, res.data()); // Note: Could also use LDLT.
*/
returnstd::vector<float>(c_s);
returnstd::vector<float>(c_s);
};
};
/**
* Fits the shape of a Morphable Model to given 2D landmarks (i.e. estimates the maximum likelihood solution of the shape coefficients) as proposed in [1].
* It's a linear, closed-form solution fitting of the shape, with regularisation (prior towards the mean).
*
* [1] O. Aldrian & W. Smith, Inverse Rendering of Faces with a 3D Morphable Model, PAMI 2013.
*
* Note: Using less than the maximum number of coefficients to fit is not thoroughly tested yet and may contain an error.
* Note: Returns coefficients following standard normal distribution (i.e. all have similar magnitude). Why? Because we fit using the normalised basis?
* Note: The standard deviations given should be a vector, i.e. different for each landmark. This is not implemented yet.
*
* @param[in] morphable_model The Morphable Model whose shape (coefficients) are estimated.
* @param[in] affine_camera_matrix A 3x4 affine camera matrix from model to screen-space (should probably be of type CV_32FC1 as all our calculations are done with float).
* @param[in] landmarks 2D landmarks from an image to fit the model to.
* @param[in] vertex_ids The vertex ids in the model that correspond to the 2D points.
* @param[in] base_face The base or reference face from where the fitting is started. Usually this would be the models mean face, which is what will be used if the parameter is not explicitly specified.
* @param[in] lambda The regularisation parameter (weight of the prior towards the mean).
* @param[in] num_coefficients_to_fit How many shape-coefficients to fit (all others will stay 0). Should be bigger than zero, or boost::none to fit all coefficients.
* @param[in] detector_standard_deviation The standard deviation of the 2D landmarks given (e.g. of the detector used), in pixels.
* @param[in] model_standard_deviation The standard deviation of the 3D vertex points in the 3D model, projected to 2D (so the value is in pixels).
* @return The estimated shape-coefficients (alphas).
