Nsryjdtyk

1) Context

I'm using EmguCV's plugin for Unity, in order to call openCV functions from within my C# code. More precisely I need to call the SolvePnPRansac method.

Here is the documentation for the method SolvePnPRansac and the expected arguments:

http://www.emgu.com/wiki/files/3.1.0/document/html/36ff5d6b-498a-0587-5350-370bdecf64d7.htm

2) Problem:

The method call crashes the unity editor itself, leaving no exceptions, no stack trace, because the plugin crashes internally and unity doesn't seem to be able to handle it, despite my try-catch block.

from the logs that unity stores persistently, this is what I get, which clearly suggests a type mismatch:

Stacktrace:
at (wrapper managed-to-native) Emgu.CV.CvInvoke.cveSolvePnPRansac (intptr,intptr,intptr,intptr,intptr,intptr,bool,int,single,double,intptr,Emgu.CV.CvEnum.SolvePnpMethod)
at (wrapper managed-to-native) Emgu.CV.CvInvoke.cveSolvePnPRansac (intptr,intptr,intptr,intptr,intptr,intptr,bool,int,single,double,intptr,Emgu.CV.CvEnum.SolvePnpMethod)
at Emgu.CV.CvInvoke.SolvePnPRansac (Emgu.CV.IInputArray,Emgu.CV.IInputArray,Emgu.CV.IInputArray,Emgu.CV.IInputArray,Emgu.CV.IOutputArray,Emgu.CV.IOutputArray,bool,int,single,double,Emgu.CV.IOutputArray,Emgu.CV.CvEnum.SolvePnpMethod) [0x00090] in /Users/user/Documents/dgunity-v2/Assets/Emgu.CV/Emgu.CV/PInvoke/CvInvokeCalib3d.cs:659
at ObjectLocalization.LocalToCameraMatrixFromPnP ...

3) What I tried :

I can successfully run:

        CvInvoke.SolvePnP(objectPoints, imagePoints, cameraMatrix, distortionCoeffs, rvec, tvec, useIntrinsicGuess);

where

• objectPoints is of type Emgu.CV.Util.VectorOfPoint3D32F


• imagePoints is of type Emgu.CV.Util.VectorOfPointF


• cameraMatrix is 3X3 of type Emgu.CV.Matrix


• distortionCoeffs is 1X4 of type Emgu.CV.Util.VectorOfFloat


• rvec and tvec are of type Emgu.CV.Util.VectorOfFloat


• and useIntrinsicGuess is type bool

4) What doesn't work

now I try:

CvInvoke.SolvePnPRansac(ransacObjPoints, ransacImgPoints, cameraMatrix, distortionCoeffs, rvec, tvec, useIntrinsicGuess, iterationsCount, reprojectionError, confident, inliers, Emgu.CV.CvEnum.SolvePnpMethod.Iterative);

where

• iterationsCount is 100 of type int


• reprojectionError is 8f of type float


• confident is 0.99 of type double


• inliers is empty of type Emgu.CV.Util.VectorOfInt

It just crashes the unity editor.
What am I doing wrong?

Answering my own question here after countless hours of frustration : EmguCV expects arguments of type Mat. Period.

It exposes method arguments of interface types iInputArray and iOutputArray, but some of the classes implementing those interfaces can really only be handled in some combinations in some methods.

So the safe way is just to forget about other classes implementing the stated interface and just go for type Mat; so use

• objectPoints of type Emgu.CV.Util.VectorOfPoint3D32F


• imagePoints of type Emgu.CV.Util.VectorOfPointF


• cameraMatrix 3X3 of type MAT
  


• distortionCoeffs 1X 5 of type MAT
  


• rvec and tvec of type MAT
  


• and useIntrinsicGuess type bool


• iterationsCount 100 of type int


• reprojectionError 8f of type float


• confident 0.99 of type double


• inliers of type MAT
  

To get or set values in in your instances of type mat, you have to either transform them into images or matrices. I used methods CopyTo and SetTo to get values out of the Mat and into the Mat through instances of type Cv.Matrix. Be aware only the SetTo(double) worked for me. From a InputArray did not get handled properly (if you give it a MAT maybe it will, but not those implementations of iInputArray i tried using)