How do you convert a python trained model into c++

[ForInLoveAting]

How to convert the model trained in python into c++ format, and then whether the inference will affect the result of inference?Can I use Windows to do this？

[lsrock1]

My converting script is very naive. It just extracts weights from backbone and heads seperately.
I am not sure if i understood 'the inference will affect the result of inference', Converting python to cpp doesn't affect the result. Score is exactly same and faster.

[ForInLoveAting]

我的转换脚本非常幼稚。它只是从骨架和头部分别提取重量。
我不确定我是否理解“推论会影响推论的结果”，将python转换为cpp不会影响结果。分数完全相同且更快。

Because our project is very demanding, but maskrcnn_benchmark framework is based on python to write, the speed is certainly not as fast as c++.So I saw your framework, and I wanted to ask if this could be applied to reasoning.

[lsrock1]

This project is not as fast as you expect and need more optimization.
I don't recommend it, if you need hight performance, speed and memory efficient.

[ForInLoveAting]

该项目的速度不如您期望的那么快，需要更多的优化。
如果您需要更高的性能，速度和内存效率，我不建议您这样做。

Thank you very much for your reply, because our current reasoning and subsequent processing need more than two seconds, normally we need to do less than one second, so I wonder if we can use the model to c++

[lsrock1]

So, you mean that faster rcnn's process must be under 1 second per many images?

[ForInLoveAting]

So, you mean that faster rcnn's process must be under 1 second per many images?

Yes, because our project requires that we compress the speed to 1 second or less

[lsrock1]

How much images per second? It really depends on the number of images, backbone type, the number of classes and nms size.
If you can accpet the drop of accuracy, adjusting hyperparameters can make that speed.

[ForInLoveAting]

每秒多少图像？这实际上取决于图像的数量，主干类型，类的数量和nms大小。
如果您可以弥补精度下降的问题，那么调整超参数可以提高速度。

We're not doing it for video, we're doing it for photos, and we're going to be able to identify the defects in the image in less than a second and do some subsequent operations on it, which all take less than a second.We now have more than 20 classes.Accuracy is also very high

[lsrock1]

I think you need ablation study using nms. NMS is dominant factor in speed.

[ForInLoveAting]

我认为您需要使用nms进行消融研究。NMS是速度的主要因素。

Thank you very much for your answer, which is very helpful to me.

[lsrock1]

check this config option
PRE_NMS_TOP_N_TEST: 1000
POST_NMS_TOP_N_TEST: 1000
FPN_POST_NMS_TOP_N_TEST: 1000

[ForInLoveAting]

检查此配置选项
PRE_NMS_TOP_N_TEST：1000
POST_NMS_TOP_N_TEST：1000
FPN_POST_NMS_TOP_N_TEST：1000
I used the default value for this, but will this change affect the accuracy of the whole model, or some things are less box.

[lsrock1]

Decreasing these options can potentially drop the accuracy. But not always, when i tested it to half NMS candidates, it only dropped 0.01 mAP. So, you have to adjust and check mAP. It doesn't require re-training.

[ForInLoveAting]

减少这些选项可能会降低准确性。但并非总是如此，当我对一半的NMS候选人进行测试时，它仅下降了0.01 mAP。因此，您必须调整并检查mAP。它不需要重新培训。

ok thranks for your information

[shenjun1994]

Could you write a document about how to let the model from python to c++ just for inference. When I see your code, I can't understand the whool process what you played, I need some help from you.