diff --git a/benchmark/README_CPU.md b/benchmark/README_CPU.md index 6113e2e2c..0f847ce85 100644 --- a/benchmark/README_CPU.md +++ b/benchmark/README_CPU.md @@ -205,74 +205,6 @@ We tested them on single-CPU with different thread numbers. 4 | 18074 | 118696 6 | 26607 | 102044 -2. **`Anakin`** VS **`PaddlePaddle/Fluid`** -We use private dataset and different QPS index in this benchmark. -### language model in E5-2650 v4 - -- Latency (`ms`) of one batch - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 42.7418 | 1.93589 - 2 | 42.7418 | 2.49537 - 6 | 42.7734 | 3.14332 - 10 | 43.0721 | 4.55329 - 12 | 42.8501 | 5.09893 - -- Throughput (`sentence/s`) - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 23 | 504 - 2 | 46 | 762 - 6 | 134 | 1393 - 10 | 218 | 1556 - 12 | 260 | 1541 - -### Chinese_ner model in E5-2650 v4 - -- Latency (`ms`) of one batch - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 0.380475 | 0.17034 - 4 | 0.380475 | 0.171143 - 6 | 0.380475 | 0.172688 - 10 | 0.380475 | 0.173269 - 12 | 0.380475 | 0.17668 - -- Throughput (`sentence/s`) - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 7844 | 5822 - 4 | 7844 | 11377 - 6 | 7844 | 29725 - 10 | 7844 | 41238 - 12 | 7844 | 42790 - -### text_classfication model in E5-2650 v4 - -- Latency (`ms`) of one batch - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 1.48578 | 1.10088 - 4 | 1.54025 | 1.11258 - 6 | 1.68529 | 1.1257 - 10 | 1.9817 | 1.13267 - 12 | 2.21864 | 1.1429 - -- Throughput (`sentence/s`) - - ThreadNum | Fluid | Anakin - :---: | :---: | :---: | - 1 | 673 | 901 - 4 | 1289 | 1665 - 6 | 3458 | 4449 - 10 | 4875 | 6183 - 12 | 5265 | 6188 - ## How to run those Benchmark models? > 1. You can just run `sh benchmark_tensorflow.sh` and `sh benchmark_anakin.sh` diff --git a/cmake/compiler_options.cmake b/cmake/compiler_options.cmake index 6d0881545..835b7d884 100644 --- a/cmake/compiler_options.cmake +++ b/cmake/compiler_options.cmake @@ -41,6 +41,8 @@ anakin_add_compile_option(-Wshadow) anakin_add_compile_option(-fpermissive) anakin_add_compile_option(-Wsign-promo) anakin_add_compile_option(-fdiagnostics-show-option) +anakin_add_compile_option(-Wno-missing-field-initializers) +anakin_add_compile_option(-Wno-extra) if(ENABLE_NOISY_WARNINGS) anakin_add_compile_option(-Wcast-align) diff --git a/docs/Manual/Converter_ch.md b/docs/Manual/Converter_ch.md index f8573f2f8..afeb37181 100644 --- a/docs/Manual/Converter_ch.md +++ b/docs/Manual/Converter_ch.md @@ -13,8 +13,8 @@ Anakin 模型转换器输入支持 Caffe 和 Fluid 两种格式的预测模型 ## 系统要求 -- Python 2.7+ -- Protobuf 3.1+（务必注意 Python 与系统环境 Protobuf 版本一致） +- Python 2.7 +- Protobuf 3.1+（务必注意 Pip Protobuf 与系统环境 Protobuf 版本一致） - PaddlePaddle 0.12.0+ (Fluid 模式下) - flask, bson, matplotlib, scikit-image - tkinter diff --git a/docs/Manual/DesignConverter_ch.md b/docs/Manual/DesignConverter_ch.md new file mode 100644 index 000000000..7031c034e --- /dev/null +++ b/docs/Manual/DesignConverter_ch.md @@ -0,0 +1,56 @@ +# Parser的编写指南 +下文称Anakin为AK，运算操作为OP,本文参考Tensorflow的Parser编写,参考代码目录为tools/external_converter_v2/parser/tensorflow +## Parser的功能和执行流程 +功能是将其他深度学习框架(如CAFFE，FLUID，TENSORFLOW，ONNEX)的模型转换为AK的模型.对AK的作用是屏蔽不同框架间的差异，这种差异包括模型存储、OP的定义、图差异 +因此Parser的执行流程是： +1. 将源框架的模型载入Parser +2. 将原框架的图解析为AK中的OP节点和OP节点的连接关系 +3. 进行OP定义的转换和图优化 +4. 将符合AK标准的图写入protobuf +## Parser的目录结构 +Parser工具在tools/external_converter_v2/parser目录下 +Parser的目录主要包含3部分: +1. Parser的运行配置文件包括 config.py, config.yaml, converter.py, 用户只用执行converter.py，Parser就会按照config.yaml中的声明去解析模型 +2. Parser的公共定义，包括operations,pbs,proto三个目录. Parser的公共工具函数 graph*.py logger.py utils.py +3. 各个框架对应的Parser，其目录的命名方式为框架名,如caffe, tensorflow +## Parser的编写流程 +### 1、声明你的Parser +1. 在config.yaml中填写你的Parser运行的必要信息，包括ProtoPath和SavePath等.OPTIONS/Framework改为你的Parser的类型，TARGET下填写对应的参数列表 +2. 添加你的Parser目录，如tensorflow，导出你的Parser符号.注意，Parser的框架默认调用你的Parser类中的__call__方法来执行解析，这个方法需要返回填写完毕的GraphProtoIO对象 +3. 在config.py中Configuration下__init__函数中增加对你的Parser的调用，将yaml中读取的配置信息传给你的Parser，此处调用你的Parser中的__init__方法 +### 2、添加你的Parser主体 +可以参考parser_tf.py +1. 你需要在Parser主体构造时获取模型路径，input，ouput名字等解析必须的信息， +2. 在__call__中返回填写好的GraphProtoIO对象，该对象为填写protobuf的辅助工具 +3. 建议Parser的解析过程分成三部分，先将原框架的模型载入并转换为一种便于修改的中间的图形式；对中间图修改使得图满足AK的要求；将满足要求的中间图利用NodeProtoIO和GraphProtoIO这两个辅助类填入protobuf.具体细节可以参考parser_tf +### 3、读取原始模型，并将模型转换为中间类型 +可以参考parse_tf_2_med.py +1. 这一步与原始框架结合紧密，你可能需要import原始框架的工具函数来完成模型的裁剪、固定、加载等操作 +2. 大部分的框架都是使用tensor来连接OP的，但AK中是OP直接相连，这点需要注意 +3. AK的shape默认是4维的，有的参数的shape不足4维，需要Parser补全 +### 4、对中间类型的图进行优化 +可以参考med_graph.py +1. 由于AK不支持普通OP多输出的情况，需要在多输出的OP后面补上Splite类型的OP节点 +2. 对于Convlution后接Batchnorm这种可以合并又不会导致OP定义改变的情况，需要Parser在这一步做掉 +3. AK规定所有的输入类型OP的名字必须是input_x这种命名方式，其中x为从0开始的数字 +### 5、将中间类型的图以GraphProtoIO的方式保存 +可以参考parse_med_2_ak.py 和 parser_tf.py +1. 你首先需要构造Node节点，Node节点的名字是OP的名字(如conv2d_1_a_0)，Node节点中OP成员变量的名字是Node节点的类型(如Convlution) +2. Node节点需要按照输入的顺序用Node的add_in方法填写输入Node的名字，add_out方法按顺序填写输出Node的名字 +3. 通过调用GraphProtoIO的add_node方法将构造好的Node的__call__方法的返回值作为参数，将Node节点加入AK的graph中 +4. 调用GraphProtoIO的add_in_edge和add_out_edge完成AK图中OP间关系的构建. 如果Node中的in和out填写正确，你也可以通过调用GraphProtoIO的format_edge_from_nodes方法完成这个工作 +5. AK的模型需要Parser给出输出Node的名字，使用GraphProtoIO的add_out方法填写输出Node的名字 +### 6、检查模型解析的正确性 +1. 默认的config.yaml配置会在解析结束后启动一个web服务器展示解析后的AK模型图，你需要对比原框架的模型图进行验证.这里最容易出现的错误是边关系的错误，表现为图非常乱，你需要逐条边地检查错误.第二个容易出错的地方是参数漏填，需要你检查OP中的属性 +2. 将解析后的模型放入AK中执行，使用相同的输入，原框架与AK有相同的输出.若果输出不一致可以开启AK的DEBUG模式，在net.cpp中将没层的输出打印.如果AK在解析阶段陷入死循环，大概率是边的关系出错. +## 如何添加新OP +1. 需要在AK代码中加入该OP的实现，包括对应设备Saber的OP，Saber单测和Framework中的OP +2. 根据Framework的OP在ops.py中添加Parser公共的OP定义 +3. 从原框架的模型中解析出该OP的节点，并在AK的graph中填入该OP节点 +## AK模型与其他框架模型的不同之处 ++ AK模型与CAFFE的模型相似，因此与其他模型有很多不同的地方，需要Parser在解析过程中处理掉. ++ 最大的不同是与FLUID或TENSORFLOW这种OP粒度很细的框架，AK的模型中OP的粒度很粗，这是为了节省访存开销.这会导致解析这些框架的模型时存在大量的合并操作. ++ 其次是OP的行为不同,如TENSORFLOW中Pooling默认都是exclusive的，而AK中是inclusive的.TENSORFLOW的Padding如果是奇数pad则在右方和下方多pad，AK是在左方和上方多Pad ++ AK默认的布局是NCHW，如果其他框架的OP是其他形式的，需要在Parser中做weights的布局转换，并处理reshape的问题. ++ AK中有的weights是需要预先做布局转换的(如GRU，LSTM).AK中也支持同一OP的不同算法，如(GRU，Pooling). + diff --git a/docs/Manual/INSTALL_ch.md b/docs/Manual/INSTALL_ch.md index 16b13cd9e..40c95392f 100644 --- a/docs/Manual/INSTALL_ch.md +++ b/docs/Manual/INSTALL_ch.md @@ -43,7 +43,7 @@ #### Anakin - CPU ### -在编译 CPU 版本前，我们建议您升级 GCC-G++ 至 5.4.0 以上，链接的 libm.so 库版本高于 2.17，以发挥更佳性能。 +在编译 CPU 版本前，我们建议您升级 GCC-G++ 至 5.4.0，链接的 libm.so 库版本为 2.17 ~ 2.23，以发挥更佳性能。 #### Anakin - AMDGPU ### diff --git a/framework/core/net/net.cpp b/framework/core/net/net.cpp index c972c0c26..c39fb8938 100755 --- a/framework/core/net/net.cpp +++ b/framework/core/net/net.cpp @@ -393,9 +393,11 @@ void Net::prediction() { #define RECORD_INNER #if defined(RECORD_INNER) && defined(USE_X86_PLACE) record_tensor_to_file(*out,("record_"+executer.name).c_str()); - if(executer.name=="") #endif LOG(INFO) <data(); +#ifdef USE_CUDA + record_tensor_to_file(*out,("record_"+executer.name).c_str()); +#endif #ifdef USE_X86_PLACE // for (int i = 0; i < 10; ++i) { // std::cout << out->data()[i]<<" "; diff --git a/framework/graph/graph.cpp b/framework/graph/graph.cpp index f8ca3183f..8832b15ca 100644 --- a/framework/graph/graph.cpp +++ b/framework/graph/graph.cpp @@ -40,6 +40,21 @@ Status Graph::load(std::string model_path) EXCLUSIVE_LOCKS_ return ret; } +template +Status Graph::load(const char* buffer, size_t len) EXCLUSIVE_LOCKS_REQUIRED(_mut) { + std::unique_lock lock(this->_mut); + Status ret = Status::OK(); + if(len != _len || buffer != _buffer) { + this->Clean(); + ret = parser::load(this, buffer, len); + _buffer = buffer; + _len = len; + } + + return ret; +} + + template Status Graph::load(const char* model_path) { return parser::load(this, model_path); diff --git a/framework/graph/graph.h b/framework/graph/graph.h index e456a461e..5ae9dea27 100644 --- a/framework/graph/graph.h +++ b/framework/graph/graph.h @@ -71,6 +71,8 @@ class Graph : public GraphBase_mut); /// this make the graph optimized. bool _has_graph_optimized{false}; GUARDED_BY(this->_mut); + + const char* _buffer{NULL} GUARDED_BY(this->_mut); + size_t _len{0} GUARDED_BY(this->_mut); std::mutex _mut; }; diff --git a/framework/model_parser/parser/parser.cpp b/framework/model_parser/parser/parser.cpp index 540abdb4c..c0ff9ba6a 100644 --- a/framework/model_parser/parser/parser.cpp +++ b/framework/model_parser/parser/parser.cpp @@ -22,8 +22,29 @@ Status load(graph::Graph* graph, std::string& model_path) { return load(graph, model_path.c_str()); } -template -Status load(graph::Graph* graph, const char* model_path) { +Status parse_graph_proto(GraphProto& graph_proto, const char* buffer, size_t len) { + google::protobuf::io::ArrayInputStream* raw_input = new google::protobuf::io::ArrayInputStream(buffer, len); + google::protobuf::io::CodedInputStream* coded_input = new google::protobuf::io::CodedInputStream(raw_input); + coded_input->SetTotalBytesLimit(INT_MAX, 536870912); + bool success = graph_proto.ParseFromCodedStream(coded_input) && coded_input->ConsumedEntireMessage(); + if (!success) { + LOG(FATAL) << " Parsing GraphProto " << " ERROR"; + } + + delete coded_input; + delete raw_input; + return Status::OK(); +} + +Status parse_graph_proto(GraphProto& graph_proto, std::istream* instream){ + if (!graph_proto.ParseFromIstream(instream)) { + DLOG(ERROR) << "Fail to parse GraphProto."; + return Status::FAIL("Fail to parse GraphProto."); + } + return Status::OK(); +} + +Status parse_graph_proto(GraphProto& graph_proto, const char* model_path) { #if 0 std::fstream input(model_path, std::ios::in | std::ios::binary); @@ -41,7 +62,6 @@ Status load(graph::Graph* graph, const char* model_path) { } #else - GraphProto graph_proto; int file_descriptor = open(model_path, O_RDONLY); if (file_descriptor == -1) { @@ -65,7 +85,13 @@ Status load(graph::Graph* graph, const char* model_path) { delete coded_input; delete raw_input; close(file_descriptor); + return Status::OK(); #endif +} + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto) { + // fill the graph with name LOG(INFO) << "graph name: " << graph_proto.name(); graph->set_name(graph_proto.name()); @@ -158,87 +184,26 @@ Status load(graph::Graph* graph, const char* model_path) { } template -Status load(graph::Graph* graph, std::istream* instream) { - +Status load(graph::Graph* graph, const char* model_path) { GraphProto graph_proto; + parse_graph_proto(graph_proto, model_path); + return generate_graph_with_graph_proto(graph, graph_proto); +} - // parsing GraphProto from model - if (!graph_proto.ParseFromIstream(instream)) { - DLOG(ERROR) << "Fail to parse GraphProto."; - return Status::FAIL("Fail to parse GraphProto."); - } - - // fill the graph with name - LOG(INFO) << "graph name: " << graph_proto.name(); - graph->set_name(graph_proto.name()); - - // fill the graph with ins/outs - for (int i = 0; i < graph_proto.ins().size(); i++) { - LOG(INFO) << "graph in: " << graph_proto.ins()[i]; - std::string in_name(graph_proto.ins()[i]); - graph->add_in(in_name); - } - - for (int i = 0; i < graph_proto.outs().size(); i++) { - LOG(INFO) << "graph out: " << graph_proto.outs()[i]; - std::string out_name(graph_proto.outs()[i]); - graph->add_out(out_name); - } - - // fill the graph with nodes - NodeIO node_io; - - for (int i = 0; i < graph_proto.nodes().size(); i++) { - node_io >> graph_proto.nodes()[i]; - } - - node_io << *graph; - - // fill the graph with edges - auto it_in = graph_proto.edges_in().begin(); - - for (; it_in != graph_proto.edges_in().end(); ++it_in) { - DLOG(WARNING) << " Parsing in edges of node : " << it_in->first; - auto& key = it_in->first; - auto& second = it_in->second; - - for (int i = 0; i < second.val().size(); i++) { - //Tensor4dPtr tensor_p = std::make_shared>(); - graph::Edge edge(second.val()[i], key); - //edge.weight() = new Tensor4d(); - //edge.weight() = std::make_shared >(); - edge.shared() = (*graph_proto.mutable_edges_info())[edge.name()].shared(); - edge.share_from() = (*graph_proto.mutable_edges_info())[edge.name()].share_from(); - graph->add_in_arc(edge); - } - } - - auto it_out = graph_proto.edges_out().begin(); - - for (; it_out != graph_proto.edges_out().end(); ++it_out) { - auto& key = it_out->first; - auto& second = it_out->second; +template +Status load(graph::Graph* graph, std::istream* instream) { - for (int i = 0; i < second.val().size(); i++) { - //Tensor4dPtr tensor_p = std::make_shared>(); - graph::Edge edge(key, second.val()[i]); - //edge.weight() = new Tensor4d(); - //edge.weight() = std::make_shared >(); - edge.shared() = (*graph_proto.mutable_edges_info())[edge.name()].shared(); - edge.share_from() = (*graph_proto.mutable_edges_info())[edge.name()].share_from(); - graph->add_out_arc(edge); - } - } + GraphProto graph_proto; + parse_graph_proto(graph_proto, instream); + return generate_graph_with_graph_proto(graph, graph_proto);; +} - // fill the graph with info (only use the key value: is_optimized) - graph->statistics.template set_info(graph_proto.summary().is_optimized()); - graph->statistics.template set_info(graph_proto.summary().temp_mem_used()); - graph->statistics.template set_info - (graph_proto.summary().original_temp_mem_used()); - graph->statistics.template set_info(graph_proto.summary().system_mem_used()); - graph->statistics.template set_info(graph_proto.summary().model_mem_used()); +template +Status load(graph::Graph* graph, const char* buffer, size_t len) { - return Status::OK(); + GraphProto graph_proto; + parse_graph_proto(graph_proto, buffer, len); + return generate_graph_with_graph_proto(graph, graph_proto);; } template @@ -382,6 +347,25 @@ Status save(graph::Graph(graph::Graph* graph, const char* model_path); + +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); #endif #if defined(USE_X86_PLACE) || defined(BUILD_LITE) @@ -434,6 +418,23 @@ Status save(graph::Graph(graph::Graph* graph, const char* model_path); + +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); #endif #ifdef USE_ARM_PLACE @@ -453,6 +454,13 @@ Status load(graph::Graph(graph::Graph* graph, const char* model_path); + +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); #endif #ifdef ANAKIN_TYPE_FP16 @@ -472,6 +480,13 @@ Status load(graph::Graph(graph::Graph* graph, const char* model_path); + +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); #endif #ifdef ANAKIN_TYPE_INT8 @@ -490,6 +505,13 @@ Status load(graph::Graph(graph::Graph* graph, const char* model_path); + +template +Status load(graph::Graph* graph, + const char* buffer, size_t len); + +template +Status generate_graph_with_graph_proto(graph::Graph* graph, GraphProto& graph_proto); #endif #endif //USE_ARM_PLACE diff --git a/framework/model_parser/parser/parser.h b/framework/model_parser/parser/parser.h index b200bf00e..9d442b916 100644 --- a/framework/model_parser/parser/parser.h +++ b/framework/model_parser/parser/parser.h @@ -36,6 +36,9 @@ Status load(graph::Graph* graph, const char* model_path); template Status load(graph::Graph* graph, std::istream* instream); +template +Status load(graph::Graph* graph, const char* buffer, size_t len); + //! save graph to disk. use to save improved Graph. template Status save(graph::Graph* graph, std::string& model_path); diff --git a/framework/operators/fusion_ops/conv_3x3_batchnorm_scale_relu.cpp b/framework/operators/fusion_ops/conv_3x3_batchnorm_scale_relu.cpp index 0d6d6ee8a..0094f46fe 100644 --- a/framework/operators/fusion_ops/conv_3x3_batchnorm_scale_relu.cpp +++ b/framework/operators/fusion_ops/conv_3x3_batchnorm_scale_relu.cpp @@ -88,12 +88,17 @@ Status SassConvBatchnormScaleReluHelper::InitParam() { // get relu param auto alpha = GET_PARAMETER(float, relu_0_alpha); - ActivationParam> active_param(Active_relu);//, alpha); // TEMP - - - ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, - scale_param); - _param_conv_batchnorm_scale_relu = conv_act_param; + if (alpha != 0) { + ActivationParam> active_param(Active_prelu, alpha); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, + scale_param); + _param_conv_batchnorm_scale_relu = conv_act_param; + } else { + ActivationParam> active_param(Active_relu); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, + scale_param); + _param_conv_batchnorm_scale_relu = conv_act_param; + } return Status::OK(); } @@ -102,8 +107,13 @@ template Status SassConvBatchnormScaleReluHelper::Init(OpContext& ctx, const std::vector >& ins, std::vector >& outs) { - _funcs_conv_batchnorm_scale_relu.init(ins, outs, _param_conv_batchnorm_scale_relu, SPECIFY, - SABER_IMPL, ctx); + if (_param_conv_batchnorm_scale_relu.activation_param.active == Active_relu) { + _funcs_conv_batchnorm_scale_relu.init(ins, outs, _param_conv_batchnorm_scale_relu, SPECIFY, + SABER_IMPL, ctx); + } else { + _funcs_conv_batchnorm_scale_relu.init(ins, outs, _param_conv_batchnorm_scale_relu, SPECIFY, + VENDER_IMPL, ctx); + } return Status::OK(); } diff --git a/framework/operators/fusion_ops/conv_batchnorm_scale_relu.cpp b/framework/operators/fusion_ops/conv_batchnorm_scale_relu.cpp index b8a83269d..f8ffe978e 100644 --- a/framework/operators/fusion_ops/conv_batchnorm_scale_relu.cpp +++ b/framework/operators/fusion_ops/conv_batchnorm_scale_relu.cpp @@ -83,12 +83,19 @@ Status ConvBatchnormScaleReluHelper::InitParam() { // get relu param auto alpha = GET_PARAMETER(float, relu_0_alpha); - ActivationParam> active_param(Active_relu);//, alpha); // TEMP + if (alpha != 0) { + ActivationParam> active_param(Active_prelu, alpha); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, + scale_param); + _param_conv_batchnorm_scale_relu = conv_act_param; + } else { + ActivationParam> active_param(Active_relu); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, + scale_param); + _param_conv_batchnorm_scale_relu = conv_act_param; + } - ConvActiveParam> conv_act_param(_conv_param, active_param, batchnorm_param, - scale_param); - _param_conv_batchnorm_scale_relu = conv_act_param; return Status::OK(); } @@ -115,7 +122,9 @@ Status ConvBatchnormScaleReluHelper::InferShape(const template <> Status ConvBatchnormScaleReluHelper::Init(OpContext &ctx, \ const std::vector >& ins, std::vector >& outs) { - if (_param_conv_batchnorm_scale_relu.conv_param.group == ins[0]->channel() && \ + bool use_saber = true; + use_saber = use_saber && (_param_conv_batchnorm_scale_relu.activation_param.active == Active_relu); + if (use_saber && _param_conv_batchnorm_scale_relu.conv_param.group == ins[0]->channel() && \ _param_conv_batchnorm_scale_relu.conv_param.group == outs[0]->channel()) { _funcs_conv_batchnorm_scale_relu.init(ins, outs, _param_conv_batchnorm_scale_relu, SPECIFY, SABER_IMPL, ctx); diff --git a/framework/operators/fusion_ops/conv_relu.cpp b/framework/operators/fusion_ops/conv_relu.cpp index da44143f4..757d9f901 100644 --- a/framework/operators/fusion_ops/conv_relu.cpp +++ b/framework/operators/fusion_ops/conv_relu.cpp @@ -62,11 +62,16 @@ Status ConvReluHelper::InitParam() { // get relu param auto alpha = GET_PARAMETER(float, relu_0_alpha); - ActivationParam> active_param(Active_relu);//, alpha); // TEMP - + if (alpha != 0) { + ActivationParam> active_param(Active_prelu, alpha); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param); + _param_conv_relu = conv_act_param; + } else { + ActivationParam> active_param(Active_relu); // TEMP + ConvActiveParam> conv_act_param(_conv_param, active_param); + _param_conv_relu = conv_act_param; + } - ConvActiveParam> conv_act_param(_conv_param, active_param); - _param_conv_relu = conv_act_param; return Status::OK(); @@ -100,6 +105,7 @@ Status ConvReluHelper::Init(OpContext& ctx, \ use_saber = use_saber && (_param_conv_relu.conv_param.weight()->width()==3); use_saber = use_saber && (_param_conv_relu.conv_param.dilation_h == 1); use_saber = use_saber && (_param_conv_relu.conv_param.dilation_w == 1); + use_saber = use_saber && (_param_conv_relu.activation_param.active == Active_relu); if (((_param_conv_relu.conv_param.group == 1) && use_saber)|| (_param_conv_relu.conv_param.group == ins[0]->channel() && \ _param_conv_relu.conv_param.group == outs[0]->channel())) { _funcs_conv_relu.init(ins, outs, _param_conv_relu, SPECIFY, SABER_IMPL, ctx); diff --git a/framework/operators/normalize.cpp b/framework/operators/normalize.cpp index 56cfa921c..db8ba69de 100644 --- a/framework/operators/normalize.cpp +++ b/framework/operators/normalize.cpp @@ -32,12 +32,16 @@ Status NormalizeHelper::InitParam() { auto eps = GET_PARAMETER(float, eps); auto p = GET_PARAMETER(int, p); - using pblock_type = PBlock::type, Ttype>; - auto input_scale = GET_PARAMETER(pblock_type, weight_1); - - saber::NormalizeParam> normalize_param(is_across_spatial, is_shared_channel, \ - &(input_scale.d_tensor()), eps, p); - _param_normalize = normalize_param; + if (FIND_PARAMETER(weight_1)) { + using pblock_type = PBlock::type, Ttype>; + auto input_scale = GET_PARAMETER(pblock_type, weight_1); + saber::NormalizeParam> normalize_param(is_across_spatial, is_shared_channel, \ + &(input_scale.d_tensor()), eps, p); + _param_normalize = normalize_param; + } else { + saber::NormalizeParam> normalize_param(is_across_spatial, is_shared_channel, eps, p); + _param_normalize = normalize_param; + } return Status::OK(); } diff --git a/framework/operators/ops.h b/framework/operators/ops.h index 046b80b37..5bd3d20e6 100644 --- a/framework/operators/ops.h +++ b/framework/operators/ops.h @@ -54,6 +54,7 @@ #include "framework/operators/priorbox.h" #include "framework/operators/relu.h" #include "framework/operators/reshape.h" +#include "framework/operators/resize.h" #include "framework/operators/scale.h" #include "framework/operators/sequence_pool.h" #include "framework/operators/slice.h" diff --git a/framework/operators/resize.cpp b/framework/operators/resize.cpp new file mode 100644 index 000000000..773f206b3 --- /dev/null +++ b/framework/operators/resize.cpp @@ -0,0 +1,91 @@ +#include "framework/operators/resize.h" + +namespace anakin { + +namespace ops { + +#define INSTANCE_RESIZE(Ttype, Dtype, Ptype) \ +template<> \ +void Resize::operator()(OpContext& ctx, \ + const std::vector >& ins,\ + std::vector >& outs) { \ + auto* impl = \ + static_cast*>(this->_helper); \ + auto& param = impl->_param_resize; \ + impl->_funcs_resize(ins, outs, param, ctx); \ +} + + +template +Status ResizeHelper::InitParam() { + DLOG(WARNING) << "Parsing Resize op parameter."; + + // get resize param + auto width_scale = GET_PARAMETER(float, width_scale); + auto height_scale = GET_PARAMETER(float, height_scale); + + ResizeParam> resize_param(height_scale, width_scale); + _param_resize = resize_param; + return Status::OK(); +} + +template +Status ResizeHelper::Init(OpContext &ctx, const std::vector> &ins, + std::vector> &outs) { + SABER_CHECK(_funcs_resize.init(ins, outs, _param_resize, SPECIFY, SABER_IMPL, ctx)); + return Status::OK(); +} + +template +Status ResizeHelper::InferShape(const std::vector> &ins, + std::vector> &outs) { + SABER_CHECK(_funcs_resize.compute_output_shape(ins, outs, _param_resize)); + return Status::OK(); +} + +#ifdef USE_CUDA +INSTANCE_RESIZE(NV, AK_FLOAT, Precision::FP32); +template <> +Status ResizeHelper::Init(OpContext &ctx, + const std::vector >& ins, + std::vector >& outs) { + SABER_CHECK(_funcs_resize.init(ins, outs, _param_resize, SPECIFY, SABER_IMPL, ctx)); + return Status::OK(); +} +ANAKIN_REGISTER_OP_HELPER(Resize, ResizeHelper, NV, AK_FLOAT, Precision::FP32); +#endif + +#ifdef USE_X86_PLACE +INSTANCE_RESIZE(X86, AK_FLOAT, Precision::FP32); +template class ResizeHelper; +ANAKIN_REGISTER_OP_HELPER(Resize, ResizeHelper, X86, AK_FLOAT, Precision::FP32); +#endif + +#ifdef USE_ARM_PLACE +INSTANCE_RESIZE(ARM, AK_FLOAT, Precision::FP32); +template class ResizeHelper; +ANAKIN_REGISTER_OP_HELPER(Resize, ResizeHelper, ARM, AK_FLOAT, Precision::FP32); +#endif//arm + +//! register op +ANAKIN_REGISTER_OP(Resize) +.Doc("Resize operator") +#ifdef USE_CUDA +.__alias__("Resize") +#endif +#ifdef USE_ARM_PLACE +.__alias__("Resize") +#endif +#ifdef USE_X86_PLACE +.__alias__("Resize") +#endif +.num_in(1) +.num_out(1) +.Args("height_scale", " height scale for resize") +.Args("width_scale", " width scale for resize"); + +} /* namespace ops */ + +} /* namespace anakin */ + + diff --git a/framework/operators/resize.h b/framework/operators/resize.h new file mode 100644 index 000000000..4d6a5a7ef --- /dev/null +++ b/framework/operators/resize.h @@ -0,0 +1,99 @@ +/* Copyright (c) 2018 Anakin Authors, Inc. All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +#ifndef ANAKIN_OPERATOR_RESIZE_H +#define ANAKIN_OPERATOR_RESIZE_H + +#include "framework/core/base.h" +#include "framework/core/data_types.h" +#include "framework/core/operator/operator.h" +#include "utils/logger/logger.h" +#include "saber/funcs/resize.h" + +namespace anakin { + +namespace ops { + +template +class ResizeHelper; + +/// pooling op +/** + * \brief Resize implementation class + * public inherit Operator + */ +template +class Resize : public Operator { +public: + Resize() {} + + /// forward impl + virtual void operator() (OpContext &ctx, + const std::vector >& ins, + std::vector >& outs) { + LOG(ERROR) << "Not Impl Yet Operator power::type>().type_info()<<">"; + } + + friend class ResizeHelper; +}; + +/** + * \brief Resize helper class to implement Resize + * public inherit OperatorHelper + * including init resource and shape size in Resize context + */ +template +class ResizeHelper : public OperatorHelper { +public: + ResizeHelper()=default; + + ~ResizeHelper() {} + + Status InitParam() override; + + /** + * \brief initial all the resource needed by pooling + * \param ctx stand for Resize operation context + * \param ins stand for input tensor vector + * \param outs stand for output tensor vector + * \return status + */ + Status Init(OpContext &ctx, + const std::vector >& ins, + std::vector >& outs) override; + + /** + * \brief infer the shape of output and input. + * \param ins stand for input tensor vector + * \param outs stand for output tensor vector + * \return status + */ + Status InferShape(const std::vector >& ins, + std::vector >& outs) override; + +public: + ///< _param_resize stand for Resize parameter + saber::ResizeParam> _param_resize; + ///< _funcs_resize stand for Resize function + saber::Resize _funcs_resize; + +}; + +} /* namespace ops */ + +} /* namespace anakin */ + +#endif diff --git a/saber/funcs/impl/cuda/base/cuda_c/saber_activation.cu b/saber/funcs/impl/cuda/base/cuda_c/saber_activation.cu index 9f2c42b92..229e7802d 100644 --- a/saber/funcs/impl/cuda/base/cuda_c/saber_activation.cu +++ b/saber/funcs/impl/cuda/base/cuda_c/saber_activation.cu @@ -175,6 +175,34 @@ __global__ void ker_prelu_fwd(Dtype * out_data, } } +template +__global__ void ker_prelu_fwd(Dtype * out_data, + const Dtype* in_data, const int count, + const Dtype slope, bool is_channel_shared, + int in_n, int in_c, int in_h, int in_w, + int in_n_stride, int in_c_stride, int in_h_stride, int in_w_stride, + int out_n_stride, int out_c_stride, int out_h_stride, int out_w_stride) { + CUDA_KERNEL_LOOP(tid, count){ + int w = tid % in_w; + int h = (tid / (in_w)) % in_h; + int c = (tid / (in_h * in_w)) % in_c; + int n = (tid / (in_c * in_h * in_w)) % in_n; + + int in_idx = n * in_n_stride + + c * in_c_stride + + h * in_h_stride + + w * in_w_stride; + + int out_idx = n * out_n_stride + + c * out_c_stride + + h * out_h_stride + + w * out_w_stride; + + Dtype in_var = in_data[in_idx]; + out_data[out_idx] = in_var > 0 ? in_var : slope * in_var; + } +} + template <> SaberStatus SaberActivation::dispatch( \ @@ -248,12 +276,21 @@ SaberStatus SaberActivation - <<>>( - out_data, in_data, count, prelu_param.slope->data(), prelu_param.channel_shared, - in_shape[0], in_shape[1], in_shape[2], in_shape[3], - stride_in[0], stride_in[1], stride_in[2], stride_in[3], - stride_out[0], stride_out[1], stride_out[2], stride_out[3]); + if (param.prelu_param.slope == nullptr) { + ker_prelu_fwd + << < CUDA_GET_BLOCKS(count), CUDA_NUM_THREADS, 0, cuda_stream >> > ( + out_data, in_data, count, param.negative_slope, prelu_param.channel_shared, + in_shape[0], in_shape[1], in_shape[2], in_shape[3], + stride_in[0], stride_in[1], stride_in[2], stride_in[3], + stride_out[0], stride_out[1], stride_out[2], stride_out[3]); + } else { + ker_prelu_fwd + << < CUDA_GET_BLOCKS(count), CUDA_NUM_THREADS, 0, cuda_stream >> > ( + out_data, in_data, count, prelu_param.slope->data(), prelu_param.channel_shared, + in_shape[0], in_shape[1], in_shape[2], in_shape[3], + stride_in[0], stride_in[1], stride_in[2], stride_in[3], + stride_out[0], stride_out[1], stride_out[2], stride_out[3]); + } break; } diff --git a/saber/funcs/impl/cuda/base/cuda_c/saber_resize.cu b/saber/funcs/impl/cuda/base/cuda_c/saber_resize.cu index 5c2f4cbe1..6760c7f54 100644 --- a/saber/funcs/impl/cuda/base/cuda_c/saber_resize.cu +++ b/saber/funcs/impl/cuda/base/cuda_c/saber_resize.cu @@ -73,9 +73,9 @@ __global__ void resize_bilinear_2d_kernel(const int wout, const int hout, dtype br = (w > win || h > hin)? 0 : src[src_indexBR]; #else dtype tl = src[src_indexTL]; - dtype tr = w > win? 0 : src[src_indexTR];//w > win? 0 : - dtype bl = h > hin? 0 : src[src_indexBL];//h > hin? 0 : - dtype br = (w > win || h > hin)? 0 : src[src_indexBR];//(w > win || h > hin)? 0 : + dtype tr = w < win? src[src_indexTR]:0;//w > win? 0 : + dtype bl = h < hin? src[src_indexBL]:0;//h > hin? 0 : + dtype br = (w < win && h < hin) ? src[src_indexBR]: 0;//(w > win || h > hin)? 0 : #endif dst[dst_index] = static_cast(w_00 * tl + w_01 * tr + w_10 * bl + w_11 * br); src_indexBR += src_stride_c; @@ -152,19 +152,13 @@ SaberStatus SaberResizecount(height_idx + 1, dims); int dst_stride_channel = dst_real_shape.count(channel_idx + 1);//outputs[0]->count(channel_idx + 1, dims); int dst_stride_batch = dst_real_shape.count(num_idx + 1);//outputs[0]->count(num_idx + 1, dims); - const InDataType* in_data_batch = in_data; - OutDataType* out_data_batch = out_data; - for (int i = 0; i < n_out; ++i) { - resize_bilinear_2d_kernel<<>>( - w_out, h_out, n_out, c_out, - dst_stride_w, dst_stride_h, dst_stride_channel, dst_stride_batch, - w_in, h_in, - src_stride_w, src_stride_h, src_stride_channel, src_stride_batch, - 1 / param.width_scale, 1 / param.height_scale, - in_data, out_data); - in_data_batch += src_stride_batch; - out_data_batch += dst_stride_batch; - } + resize_bilinear_2d_kernel<<>>( + w_out, h_out, n_out, c_out, + dst_stride_w, dst_stride_h, dst_stride_channel, dst_stride_batch, + w_in, h_in, + src_stride_w, src_stride_h, src_stride_channel, src_stride_batch, + 1 / param.width_scale, 1 / param.height_scale, + in_data, out_data); //outputs[0]->record_event(stream); return SaberSuccess; } diff --git a/saber/funcs/impl/cuda/saber_activation.h b/saber/funcs/impl/cuda/saber_activation.h index a39004469..b799dec7b 100644 --- a/saber/funcs/impl/cuda/saber_activation.h +++ b/saber/funcs/impl/cuda/saber_activation.h @@ -65,7 +65,7 @@ class SaberActivation& inputs, std::vector& outputs, ActivationParam& param); - + OpTensor _slope; }; //template class SaberActivation; diff --git a/saber/funcs/impl/cuda/vender_conv_act.cpp b/saber/funcs/impl/cuda/vender_conv_act.cpp index 88bddf9ca..d973daef0 100644 --- a/saber/funcs/impl/cuda/vender_conv_act.cpp +++ b/saber/funcs/impl/cuda/vender_conv_act.cpp @@ -1,4 +1,5 @@ #include "saber/funcs/impl/cuda/vender_conv_act.h" +#include "saber/funcs/impl/cuda/saber_activation.h" #include "cuda_fp16.h" namespace anakin { @@ -10,6 +11,9 @@ SaberStatus VenderConv2DAct: std::vector& outputs, ConvActiveParam& param, Context& ctx) { + if (_use_saber_act) { + _saber_act.create(inputs, outputs, param.activation_param, ctx); + } if (!(&ctx == this->_ctx)) { if (_handle != NULL) { CUDNN_CHECK(cudnnDestroy(_handle)); @@ -65,7 +69,7 @@ SaberStatus VenderConv2DAct: inputs[0]->dims() - 2, pad_a, filter_stride_a, dilation_a); // set activation descriptor - if(param.has_active) { + if(param.has_active && !_use_saber_act) { cudnn::set_activation_des(&_active_descs, param.activation_param.active); } @@ -113,12 +117,11 @@ SaberStatus VenderConv2DAct: dispatch(const std::vector& inputs, std::vector& outputs, ConvActiveParam& param) { - const InDataType *in_data = (const InDataType*)inputs[0]->data(); InDataType *out_data = (InDataType*)outputs[0]->mutable_data(); const float *weight_data = (const float *) param.conv_param.weight()->data(); - if (param.has_active == false) { + if (_use_saber_act || param.has_active == false) { CUDNN_CHECK(cudnnConvolutionForward(_handle, cudnn::cudnnTypeWrapper::kOne(), _input_descs, in_data, @@ -140,6 +143,9 @@ SaberStatus VenderConv2DAct: _output_descs, out_data)); } + if (_use_saber_act) { + _saber_act.dispatch(outputs, outputs, param.activation_param); + } return SaberSuccess; } diff --git a/saber/funcs/impl/cuda/vender_conv_act.h b/saber/funcs/impl/cuda/vender_conv_act.h index ec72de353..965e6a5e5 100644 --- a/saber/funcs/impl/cuda/vender_conv_act.h +++ b/saber/funcs/impl/cuda/vender_conv_act.h @@ -17,7 +17,8 @@ #define ANAKIN_SABER_FUNCS_IMPL_CUDA_CUDNN_CONV_ACT_H #include "saber/funcs/impl/impl_conv_act.h" -#include "saber/funcs/impl/cuda/cudnn_helper.h" +#include "saber/funcs/impl/cuda/cudnn_helper.h" +#include "saber/funcs/impl/cuda/saber_activation.h" #include namespace anakin{ @@ -119,6 +120,10 @@ class VenderConv2DAct& outputs, ConvActiveParam& param, Context& ctx) { // ---- init cudnn resources ---- + if (param.activation_param.active!= Active_relu) { + _use_saber_act = true; + _saber_act.init(inputs, outputs, param.activation_param, ctx); + } _workspaceSizeInBytes = 0; _workspaceData = NULL; @@ -192,6 +197,9 @@ class VenderConv2DAct _saber_act; void *x8_data; void *y8_data; diff --git a/saber/saber_funcs_param.h b/saber/saber_funcs_param.h index c2c045cff..260caf1a8 100644 --- a/saber/saber_funcs_param.h +++ b/saber/saber_funcs_param.h @@ -1515,7 +1515,17 @@ struct NormalizeParam { eps = eps_in; CHECK_EQ(p == 2 || p == 1, true) << "only support L1 and L2 norm"; } + NormalizeParam(bool is_across_spatial, bool is_shared_channel, \ + float eps_in = 1e-6f, int pin = 2) { + across_spatial = is_across_spatial; + channel_shared = is_shared_channel; + p = pin; + has_scale = false; + scale = nullptr; + eps = eps_in; + CHECK_EQ(p == 2 || p == 1, true) << "only support L1 and L2 norm"; + } NormalizeParam(const NormalizeParam& right) { channel_shared = right.channel_shared; across_spatial = right.across_spatial; diff --git a/test/saber/cuda/test_saber_func_power.cpp b/test/saber/cuda/test_saber_func_power.cpp index 277157d73..05a9c1f1c 100644 --- a/test/saber/cuda/test_saber_func_power.cpp +++ b/test/saber/cuda/test_saber_func_power.cpp @@ -135,16 +135,16 @@ TEST(TestSaberFuncPowerNV, test_func_constructor) { PowerParam param(/*power*/1.0f, /*scale*/ float(5), /*shift*/0.0f); for (auto input_share : { - false, true + false }) { for (auto output_share : { - false, true + false }) { for (auto input_share_sub : { - false, true + false }) { for (auto output_share_sub : { - false, true + false }) { for (auto get_time : { false diff --git a/tools/external_converter_v2/config.py b/tools/external_converter_v2/config.py index 74b249674..538162457 100644 --- a/tools/external_converter_v2/config.py +++ b/tools/external_converter_v2/config.py @@ -3,6 +3,7 @@ # -*- coding: utf-8 -*- import os +import sys import subprocess from yaml import load, dump try: @@ -38,6 +39,8 @@ def __init__(self, config_file_path=ConfigFilePath): # parse TARGET info from config file. if self.framework == "CAFFE": proto_list = data['TARGET'][self.framework]['ProtoPaths'] + assert type(proto_list) == list, \ + "The ProtoPaths format maybe incorrect, please check if there is any HORIZONTAL LINE." self.__generate_pbs(proto_list) self.framework_config_dict = data['TARGET'][self.framework] elif self.framework == "PADDLE": @@ -58,9 +61,28 @@ def __init__(self, config_file_path=ConfigFilePath): except NameError: raise + def check_protobuf_version(self): + for path in sys.path: + module_path = os.path.join(path, 'google', 'protobuf', '__init__.py') + if os.path.exists(module_path): + with open(module_path) as f: + __name__ = '__main__' + exec(f.read(), locals()) + break + try: + protoc_out = subprocess.check_output(["protoc", "--version"]).split()[1] + except OSError as exc: + raise OSError('Can not find Protobuf in system environment.') + sys_versions = map(int, protoc_out.split('.')) + pip_versions = map(int, __version__.split('.')) + assert sys_versions[0] >= 3 and pip_versions[0] >= 3 , \ + "Protobuf version must be greater than 3.0. Please refer to the Anakin Docs." + assert pip_versions[1] >= sys_versions[1], \ + "ERROR: Protobuf must be the same.\nSystem Protobuf %s\nPython Protobuf %s\n" \ + % (protoc_out, __version__) + "Try to execute pip install protobuf==%s" % (protoc_out) + def generate_pbs_of_anakin(self): protoFilesStr = subprocess.check_output(["ls", "parser/proto/"]) - print(protoFilesStr) filesList = protoFilesStr.split('\n') protoFilesList = [] for file in filesList: @@ -78,6 +100,7 @@ def __generate_pbs(self, proto_list, default_save_path="parser/pbs/"): proto_list: ['/path/to/proto_0','/path/to/proto_1', ... ] default_save_path: default saved to 'parser/pbs/' """ + self.check_protobuf_version() for pFile in proto_list: subprocess.check_call(['protoc', '-I', os.path.dirname(pFile) + "/", diff --git a/tools/external_converter_v2/config.yaml b/tools/external_converter_v2/config.yaml index 2c00def1d..674774563 100644 --- a/tools/external_converter_v2/config.yaml +++ b/tools/external_converter_v2/config.yaml @@ -40,15 +40,14 @@ ##-------------------------------------------------------------- # OPTIONS: - Framework: TENSORFLOW - SavePath: /ak_model/save/path - ResultName: resnet50_v1 - + Framework: CAFFE + SavePath: ./output + ResultName: googlenet Config: LaunchBoard: ON Server: ip: 0.0.0.0 - port: 8889 + port: 8888 OptimizedGraph: enable: OFF path: /path/to/anakin_optimized/googlenet.anakin.bin.saved @@ -77,9 +76,9 @@ TARGET: ModelPath: TENSORFLOW: - ProtoPaths: /path/to/your/model/ - PrototxtPath: - ModelPath: + ProtoPaths: / + PrototxtPath: / + ModelPath: / OutPuts: ONNX: diff --git a/tools/external_converter_v2/parser/kill_caffe/.gitignore b/tools/external_converter_v2/parser/caffe/.gitignore similarity index 100% rename from tools/external_converter_v2/parser/kill_caffe/.gitignore rename to tools/external_converter_v2/parser/caffe/.gitignore diff --git a/tools/external_converter_v2/parser/kill_caffe/__init__.py b/tools/external_converter_v2/parser/caffe/__init__.py similarity index 100% rename from tools/external_converter_v2/parser/kill_caffe/__init__.py rename to tools/external_converter_v2/parser/caffe/__init__.py diff --git a/tools/external_converter_v2/parser/kill_caffe/caffe_helper.py b/tools/external_converter_v2/parser/caffe/caffe_helper.py similarity index 100% rename from tools/external_converter_v2/parser/kill_caffe/caffe_helper.py rename to tools/external_converter_v2/parser/caffe/caffe_helper.py diff --git a/tools/external_converter_v2/parser/kill_caffe/caffe_layer_param_transmit.py b/tools/external_converter_v2/parser/caffe/caffe_layer_param_transmit.py similarity index 93% rename from tools/external_converter_v2/parser/kill_caffe/caffe_layer_param_transmit.py rename to tools/external_converter_v2/parser/caffe/caffe_layer_param_transmit.py index 02043b7c7..f6a2a258e 100755 --- a/tools/external_converter_v2/parser/kill_caffe/caffe_layer_param_transmit.py +++ b/tools/external_converter_v2/parser/caffe/caffe_layer_param_transmit.py @@ -88,6 +88,17 @@ def Parser_concat(args): concat_param = layer.concat_param OpsRegister()["Concat"].axis = concat_param.axis +@ParserFeedDecorator("Resize") +def Parser_resize(args): + layer = args[1] + # parser caffe parameter + resize_param = layer.resize_param + if resize_param.HasField("out_width_scale"): + OpsRegister()["Resize"].width_scale = resize_param.out_width_scale + if resize_param.HasField("out_height_scale"): + OpsRegister()["Resize"].height_scale = resize_param.out_height_scale + + @ParserFeedDecorator("DeformConvolution") def Parser_deformable_convolution(args): @@ -721,6 +732,7 @@ def Parser_rpn_proposal_ssd(args): OpsRegister()["RPNProposalSSD"].nms_among_classes = nms_param.nms_among_classes OpsRegister()["RPNProposalSSD"].voting = list(nms_param.voting) OpsRegister()["RPNProposalSSD"].vote_iou = list(nms_param.vote_iou) + OpsRegister()["RPNProposalSSD"].nms_gpu_max_n_per_time = nms_param.nms_gpu_max_n_per_time # parsing gen_anchor_param pkg gen_anchor_param = detect_output_ssd.gen_anchor_param OpsRegister()["RPNProposalSSD"].base_size = gen_anchor_param.base_size @@ -825,6 +837,7 @@ def Parser_rcnn_net_output_with_attr(args): OpsRegister()["RCNNDetOutputWithAttr"].nms_among_classes = nms_param.nms_among_classes OpsRegister()["RCNNDetOutputWithAttr"].voting = list(nms_param.voting) OpsRegister()["RCNNDetOutputWithAttr"].vote_iou = list(nms_param.vote_iou) + OpsRegister()["RCNNDetOutputWithAttr"].nms_gpu_max_n_per_time = nms_param.nms_gpu_max_n_per_time # parsing gen_anchor_param pkg gen_anchor_param = detect_output_ssd.gen_anchor_param OpsRegister()["RCNNDetOutputWithAttr"].base_size = gen_anchor_param.base_size @@ -946,6 +959,7 @@ def Parser_rcnn_proposal(args): OpsRegister()["RCNNProposal"].nms_among_classes = nms_param.nms_among_classes OpsRegister()["RCNNProposal"].voting = list(nms_param.voting) OpsRegister()["RCNNProposal"].vote_iou = list(nms_param.vote_iou) + OpsRegister()["RCNNProposal"].nms_gpu_max_n_per_time = nms_param.nms_gpu_max_n_per_time # parsing gen_anchor_param pkg gen_anchor_param = detect_output_ssd.gen_anchor_param OpsRegister()["RCNNProposal"].base_size = gen_anchor_param.base_size @@ -1048,12 +1062,32 @@ def Parser_proposal_img_scale_to_cam_coords(args): OpsRegister()["ProposalImgScaleToCamCoords"].cords_offset_y = proposal_img_scale_to_cam_coords.cords_offset_y OpsRegister()["ProposalImgScaleToCamCoords"].bbox_size_add_one = proposal_img_scale_to_cam_coords.bbox_size_add_one OpsRegister()["ProposalImgScaleToCamCoords"].rotate_coords_by_pitch = proposal_img_scale_to_cam_coords.rotate_coords_by_pitch - OpsRegister()["ProposalImgScaleToCamCoords"].refine_coords_by_bbox = proposal_img_scale_to_cam_coords.refine_coords_by_bbox - OpsRegister()["ProposalImgScaleToCamCoords"].refine_min_dist = proposal_img_scale_to_cam_coords.refine_min_dist - OpsRegister()["ProposalImgScaleToCamCoords"].refine_dist_for_height_ratio_one = proposal_img_scale_to_cam_coords.refine_dist_for_height_ratio_one - OpsRegister()["ProposalImgScaleToCamCoords"].max_3d2d_height_ratio_for_min_dist = proposal_img_scale_to_cam_coords.max_3d2d_height_ratio_for_min_dist + #OpsRegister()["ProposalImgScaleToCamCoords"].refine_coords_by_bbox = proposal_img_scale_to_cam_coords.refine_coords_by_bbox + #OpsRegister()["ProposalImgScaleToCamCoords"].refine_min_dist = proposal_img_scale_to_cam_coords.refine_min_dist + #OpsRegister()["ProposalImgScaleToCamCoords"].refine_dist_for_height_ratio_one = proposal_img_scale_to_cam_coords.refine_dist_for_height_ratio_one + #OpsRegister()["ProposalImgScaleToCamCoords"].max_3d2d_height_ratio_for_min_dist = proposal_img_scale_to_cam_coords.max_3d2d_height_ratio_for_min_dist OpsRegister()["ProposalImgScaleToCamCoords"].with_trunc_ratio = proposal_img_scale_to_cam_coords.with_trunc_ratio + OpsRegister()["ProposalImgScaleToCamCoords"].regress_ph_rh_as_whole = proposal_img_scale_to_cam_coords.regress_ph_rh_as_whole + OpsRegister()["ProposalImgScaleToCamCoords"].real_h_means_as_whole = list(proposal_img_scale_to_cam_coords.real_h_means_as_whole) + OpsRegister()["ProposalImgScaleToCamCoords"].real_h_stds_as_whole = list(proposal_img_scale_to_cam_coords.real_h_stds_as_whole) +@ParserFeedDecorator("RoisAnchorFeature") +def Parser_rois_anchor_feature(args): + layer = args[1] + # parser caffe parameter + rois_anchor_feature_param = layer.rois_anchor_feature_param + OpsRegister()["RoisAnchorFeature"].min_anchor_size = rois_anchor_feature_param.min_anchor_size + OpsRegister()["RoisAnchorFeature"].num_anchor_scales = rois_anchor_feature_param.num_anchor_scales + OpsRegister()["RoisAnchorFeature"].anchor_scale_pow_base = rois_anchor_feature_param.anchor_scale_pow_base + OpsRegister()["RoisAnchorFeature"].anchor_wph_ratios = list(rois_anchor_feature_param.anchor_wph_ratios) + OpsRegister()["RoisAnchorFeature"].num_top_iou_anchor = rois_anchor_feature_param.num_top_iou_anchor + OpsRegister()["RoisAnchorFeature"].min_num_top_iou_anchor = rois_anchor_feature_param.min_num_top_iou_anchor + OpsRegister()["RoisAnchorFeature"].iou_thr = rois_anchor_feature_param.iou_thr + OpsRegister()["RoisAnchorFeature"].ft_ratio_h = rois_anchor_feature_param.ft_ratio_h + OpsRegister()["RoisAnchorFeature"].ft_ratio_w = rois_anchor_feature_param.ft_ratio_w + OpsRegister()["RoisAnchorFeature"].ft_log_ratio_h = rois_anchor_feature_param.ft_log_ratio_h + OpsRegister()["RoisAnchorFeature"].ft_log_ratio_w = rois_anchor_feature_param.ft_log_ratio_w + OpsRegister()["RoisAnchorFeature"].bbox_size_add_one = rois_anchor_feature_param.bbox_size_add_one @ParserFeedDecorator("Axpy") def Parser_axpy(args): @@ -1064,13 +1098,14 @@ def Parser_axpy(args): def Parser_priorbox(args): layer = args[1] prior_box_param = layer.prior_box_param - OpsRegister()["PriorBox"].min_size = list(prior_box_param.min_size) - OpsRegister()["PriorBox"].max_size = list(prior_box_param.max_size) - OpsRegister()["PriorBox"].aspect_ratio = list(prior_box_param.aspect_ratio) - OpsRegister()["PriorBox"].fixed_size = list(prior_box_param.fixed_size) - OpsRegister()["PriorBox"].fixed_ratio = list(prior_box_param.fixed_ratio) - OpsRegister()["PriorBox"].density = list(prior_box_param.density) - OpsRegister()["PriorBox"].aspect_ratio = list(prior_box_param.aspect_ratio) + if len(prior_box_param.aspect_ratio) > 0: + OpsRegister()["PriorBox"].min_size = list(prior_box_param.min_size) + OpsRegister()["PriorBox"].max_size = list(prior_box_param.max_size) + OpsRegister()["PriorBox"].aspect_ratio = list(prior_box_param.aspect_ratio) + if len(prior_box_param.density) > 0: + OpsRegister()["PriorBox"].fixed_size = list(prior_box_param.fixed_size) + OpsRegister()["PriorBox"].fixed_ratio = list(prior_box_param.fixed_ratio) + OpsRegister()["PriorBox"].density = list(prior_box_param.density) OpsRegister()["PriorBox"].is_flip = prior_box_param.flip OpsRegister()["PriorBox"].is_clip = prior_box_param.clip OpsRegister()["PriorBox"].variance = list(prior_box_param.variance) @@ -1139,6 +1174,16 @@ def Parser_relu6(args): OpsRegister()["Activation"].type = "ClippedRelu" OpsRegister()["Activation"].clip_relu_num = 6 +@ParserFeedDecorator("Interp") +def Parser_interp(args): + layer = args[1] + interp_param = layer.interp_param + OpsRegister()["Interp"].height = interp_param.height + OpsRegister()["Interp"].width = interp_param.width + OpsRegister()["Interp"].zoom_factor = interp_param.zoom_factor + OpsRegister()["Interp"].shrink_factor = interp_param.shrink_factor + OpsRegister()["Interp"].pad_beg = interp_param.pad_beg + OpsRegister()["Interp"].pad_end = interp_param.pad_end # caffe layer parameter parser map CAFFE_LAYER_PARSER = { @@ -1205,6 +1250,10 @@ def Parser_relu6(args): "DetectionOutput": OpsParam().set_parser(Parser_detectionoutput), # vis add "ArgMax": OpsParam().set_parser(Parser_argmax), "Normalize": OpsParam().set_parser(Parser_normalize), + "Resize": OpsParam().set_parser(Parser_resize), "ReLU6": OpsParam().set_parser(Parser_relu6), - "ShuffleChannel": OpsParam().set_parser(Parser_ShuffleChannel) + "Normalization": OpsParam().set_parser(Parser_normalize), + "ShuffleChannel": OpsParam().set_parser(Parser_ShuffleChannel), + "RoisAnchorFeature": OpsParam().set_parser(Parser_rois_anchor_feature), + "Interp": OpsParam().set_parser(Parser_interp) } diff --git a/tools/external_converter_v2/parser/kill_caffe/parser_caffe.py b/tools/external_converter_v2/parser/caffe/parser_caffe.py similarity index 68% rename from tools/external_converter_v2/parser/kill_caffe/parser_caffe.py rename to tools/external_converter_v2/parser/caffe/parser_caffe.py index f83ba2557..43a6d8fe9 100644 --- a/tools/external_converter_v2/parser/kill_caffe/parser_caffe.py +++ b/tools/external_converter_v2/parser/caffe/parser_caffe.py @@ -39,11 +39,151 @@ def _DetectionArch(self): self._ParserPrototxt() self._UpgradeNetAsNeeded() self._FilterNet() + self._SplitInception(False) + self._InsSplitBtwSliceConcat() + self._InsSplitBtwSliceEltwise() self._InsertSplits() self._ScatterInputLayer() # create input node #self._CreateInputNode() maybe not need + + def _SplitInception(self, is_weight): + print is_weight + net = self.net_parameter + + if is_weight: + net = self.net_param_weights + else: + print net + layers = net.layers or net.layer; + new_layers = [] + net_param = NetParameter() + blob_name_dict = {} + for idx, layer in enumerate(layers): + if layer.type != "Inception": + for b_id, blob in enumerate(layer.bottom): + if blob in blob_name_dict: + layer.bottom[b_id] = blob_name_dict[blob] + new_layers.append(layer) + if is_weight: + if (layer.type == "Slice"): + print layer + else: + for b_id, blob in enumerate(layer.bottom): + if blob in blob_name_dict: + layer.bottom[b_id] = blob_name_dict[blob] + #if is_weight: + #print layer + inception_top = layer.top + bottom_name = layer.bottom + inception_param = layer.inception_param + relu_param = inception_param.relu_param + need_relu = inception_param.need_relu + relu_at_top = inception_param.relu_at_top + columns = inception_param.inception_column + columns_top = [] + blob_id = 0 + inception = [] + blobs = layer.blobs + if len(blobs) != 0: + print "****************************************" + for b in blobs: + print "inception blob shape", b.shape + for cl_id, column in enumerate(columns): + convs = column.convolution_param + col_name = inception_top[0] + "_" + column.column_name + bottom = bottom_name[0] + for conv_id, conv in enumerate(convs): + conv_layer = net_param.layer.add() + conv_layer.type = "Convolution" + top = col_name + "_" + str(conv_id + 1) + "_conv" + conv_layer.convolution_param.CopyFrom(conv) + if conv_layer.convolution_param.HasField("kernel_size"): + conv_layer.convolution_param.pad = (conv_layer.convolution_param.kernel_size - 1) / 2 + else: + conv_layer.convolution_param.pad_h = (conv_layer.convolution_param.kernel_h - 1) / 2 + conv_layer.convolution_param.pad_w = (conv_layer.convolution_param.kernel_w - 1) / 2 + + conv_layer.top.append(top) + conv_layer.bottom.append(bottom) + conv_layer.name = top; + inception.append(conv_layer) + if len(blobs) != 0: + if conv_layer.convolution_param.bias_term: + # conv_layer.blobs.append(blobs[blob_id]) + # conv_layer.blobs.append(blobs[blob_id+1]) + blob_id += 2 + else: + # conv_layer.blobs.append(blobs[blob_id+1]) + blob_id += 1 + bottom = top + if (need_relu): + if relu_at_top: + relu_layer = net_param.layer.add() + relu_layer.type = "ReLU" + relu_layer.top.append(bottom) + relu_layer.bottom.append(bottom) + relu_layer.relu_param.CopyFrom(inception_param.relu_param) + relu_layer.name = col_name + "_" + str(conv_id + 1) + "relu"; + inception.append(relu_layer) + else: + relu_layer = net_param.layer.add() + relu_layer.type = "ReLU" + top = col_name + "_relu_" + str(conv_id) + relu_layer.name = col_name + "_" + str(conv_id + 1) + "relu"; + relu_layer.bottom.append(bottom) + relu_layer.top.append(top) + if inception_param.HasField("relu_param"): + relu_layer.relu_param.CopyFrom(inception_param.relu_param) + else: + relu_layer.relu_param = ReLUParameter() + bottom = top + inception.append(relu_layer) + + if column.HasField("pooling_param"): + pool_layer = net_param.layer.add() + pool_layer.type = "Pool" + top = col_name + "_pool" + pool_layer.name = top + pool_layer.top.append(top) + pool_layer.bottom.append(bottom) + pool_layer.pooling_param.CopyFrom(column.pooling_param) + bottom = top + inception.append(pool_layer) + columns_top.append(bottom) + if len(columns_top) > 1: + concat_layer = net_param.layer.add() + for bot in columns_top: + concat_layer.bottom.append(bot) + concat_layer.top.append(inception_top[0]) + concat_layer.type = "Concat" + concat_layer.name = top + "_concat" + inception.append(concat_layer) + bottom = inception_top[0] + else: + blob_name_dict[inception_top[0]] = bottom + + new_layers.extend(inception) + #print inception + #for com in inception: + # new_layers.append(com) + if is_weight: + if self.net_param_weights.layers: + del self.net_param_weights.layers[:] + self.net_param_weights.layers.extend(new_layers) + if self.net_param_weights.layer: + del self.net_param_weights.layer[:] + self.net_param_weights.layer.extend(new_layers) + else: + if self.net_parameter.layers: + del self.net_parameter.layers[:] + self.net_parameter.layers.extend(new_layers) + if self.net_parameter.layer: + del self.net_parameter.layer[:] + self.net_parameter.layer.extend(new_layers) + + def _ParserPrototxt(self): """ don't need to be used. @@ -51,6 +191,13 @@ def _ParserPrototxt(self): with open(self.PrototxtPath, "r") as f: text_format.Merge(f.read(), self.net_parameter) + def _ParserPrototxtWithModel(self): + """ + don't need to be used. + """ + with open(self.ModelPath, "r") as f: + self.net_parameter.MergeFromString(f.read()) + def _ParserModel(self): with open(self.ModelPath, "r") as f: self.net_param_weights.MergeFromString(f.read()) @@ -70,7 +217,7 @@ def _UpgradeNetAsNeeded(self): logger(verbose.FATAL).feed("[ Upgrade Level 1 ] Details: need to upgrade from V0 to V1 [ FAILED ]") exit() if NetNeedsDataUpgrade(self.net_parameter): - logger(verbose.INFO).feed("[ Upgrade Level 2 ] Details: need Data upgrade [ IGNORED ]") + logger(verbose.ERROR).feed("[ Upgrade Level 2 ] Details: need Data upgrade [ IGNORED ]") if NetNeedsV1ToV2Upgrade(self.net_parameter): logger(verbose.INFO).feed("[ Upgrade Level 3 ] Details: need to upgrade from V1 to V2 [ ... ]") original_param = NetParameter() @@ -81,7 +228,7 @@ def _UpgradeNetAsNeeded(self): logger(verbose.FATAL).feed("[ Upgrade Level 3 ] Details: need to upgrade from V1 to V2 [ FAILED ]") exit() if NetNeedsInputUpgrade(self.net_parameter): - logger(verbose.INFO).feed("[ Upgrade Level 4 ] Details: need Input upgrade [ ... ]") + logger(verbose.INFO).feed("[ Upgrade Level 4 ] Details: need Input upgrade [ ... ]") UpgradeNetInput(self.net_parameter) logger(verbose.WARNING).feed("[ Upgrade Level 4 ] Details: need Input upgrade [ SUC ]") if NetNeedsBatchNormUpgrade(self.net_parameter): @@ -89,6 +236,70 @@ def _UpgradeNetAsNeeded(self): UpgradeNetBatchNorm(self.net_parameter) logger(verbose.INFO).feed("[ Upgrade Level 5 ] Details: need BatchNorm upgrade [ ... ]") + def _InsSplitBtwSliceConcat(self): + ''' + Currently, the connection between Slice and Concat must be implemented via Split. + ''' + layers = self.net_parameter.layer or self.net_parameter.layers + top_blobs_of_slices = list() + btm_blobs_of_concats = list() + for layer in layers: + if layer.type == 'Slice': + top_blobs_of_slices.extend(layer.top) + elif layer.type == 'Concat': + btm_blobs_of_concats.extend(layer.bottom) + intersection_blobs = list(set(top_blobs_of_slices).intersection(set(btm_blobs_of_concats))) + new_param = NetParameter() + for layer in layers: + new_layer = new_param.layer.add() + new_layer.CopyFrom(layer) + if layer.type == 'Slice': + for top_blob in layer.top: + if top_blob in intersection_blobs: + split_param = new_param.layer.add() + split_param.bottom.append(top_blob) + split_param.top.append(top_blob) + split_param.name = 'Split_' + top_blob + split_param.type = 'Split' + if self.net_parameter.layer: + del self.net_parameter.layer[:] + self.net_parameter.layer.extend(new_param.layer) + else: + del self.net_parameter.layers[:] + self.net_parameter.layers.extend(new_param.layer) + + def _InsSplitBtwSliceEltwise(self): + ''' + Currently, the connection between Slice and Concat must be implemented via Split. + ''' + layers = self.net_parameter.layer or self.net_parameter.layers + top_blobs_of_slices = list() + btm_blobs_of_concats = list() + for layer in layers: + if layer.type == 'Slice': + top_blobs_of_slices.extend(layer.top) + elif layer.type == 'Eltwise': + btm_blobs_of_concats.extend(layer.bottom) + intersection_blobs = list(set(top_blobs_of_slices).intersection(set(btm_blobs_of_concats))) + new_param = NetParameter() + for layer in layers: + new_layer = new_param.layer.add() + new_layer.CopyFrom(layer) + if layer.type == 'Slice': + for top_blob in layer.top: + if top_blob in intersection_blobs: + split_param = new_param.layer.add() + split_param.bottom.append(top_blob) + split_param.top.append(top_blob) + split_param.name = 'Split_' + top_blob + split_param.type = 'Split' + if self.net_parameter.layer: + del self.net_parameter.layer[:] + self.net_parameter.layer.extend(new_param.layer) + else: + del self.net_parameter.layers[:] + self.net_parameter.layers.extend(new_param.layer) + def _InsertSplits(self): """ Same as caffe InsertSplits. @@ -112,7 +323,7 @@ def _InsertSplits(self): layer_idx_to_name[idx] = layer.name for j, btm in enumerate(layer.bottom): if btm not in self.blob_name_to_last_top_idx.keys(): - logger(verbose.FATAL).feed("Unknown bottom (blob: %s) in (layer: '%s')" % (btm, layer.name)) + logger(verbose.FATAL).feed("Unknown bottom (blob: %s) in (layer: '%s')" % (btm, layer.name)) exit() bottom_idx = (idx, j) top_idx = self.blob_name_to_last_top_idx[btm] @@ -250,19 +461,20 @@ def _CreateInputNode(self): if not input_dim: if len(self.net_parameter.input_shape) > 0: input_dim = map(int, self.net_parameter.input_shape[0].dim) - node_io.set_name(in_name) - node_io.add_in(in_name) - # leak out name , need to be added later. - shape = TensorShape() - shape.dim.value[:] = input_dim - shape.dim.size = len(input_dim) - node_io.add_attr("shape", shape, "shape") - op_io.set_name("Input") - op_io.set_in_num(1) - op_io.set_commutative(True) - node_io.set_op(op_io()) - self.graphIO.add_node(node_io()) - self.graphIO.add_in(in_name) + for in_name in inputs: + node_io.set_name(in_name) + node_io.add_in(in_name) + # leak out name , need to be added later. + shape = TensorShape() + shape.dim.value[:] = input_dim + shape.dim.size = len(input_dim) + node_io.add_attr("shape", shape, "shape") + op_io.set_name("Input") + op_io.set_in_num(1) + op_io.set_commutative(True) + node_io.set_op(op_io()) + self.graphIO.add_node(node_io()) + self.graphIO.add_in(in_name) else: # parser InputParameter instead. logger(verbose.INFO).feed(" Need to parse the layer of type InputParameter.") @@ -309,11 +521,12 @@ def _Parsing_new(self): blob_top_to_layer_name[top].put(tmp_rlayer.name) # set graph proto's name self.graphIO.set_name(self.net_parameter.name) - logger(verbose.INFO).feed(" [CAFFE] Archtecture Parsing ...") + logger(verbose.ERROR).feed(" [CAFFE] Archtecture Parsing ...") # parsing model - logger(verbose.INFO).feed(" [CAFFE] Model Parameter Parsing ...") + logger(verbose.ERROR).feed(" [CAFFE] Model Parameter Parsing ...") self._ParserModel() + self._SplitInception(True) model_layers = self.net_param_weights.layers or self.net_param_weights.layer # we must setting graph edge first @@ -429,11 +642,12 @@ def _Parsing(self): #blob_btm_to_layer_name[top] = tmp_rlayer.name # set graph proto's name self.graphIO.set_name(self.net_parameter.name) - logger(verbose.INFO).feed(" [CAFFE] Archtecture Parsing ...") + logger(verbose.ERROR).feed(" [CAFFE] Archtecture Parsing ...") # parsing model - logger(verbose.INFO).feed(" [CAFFE] Model Parameter Parsing ...") + logger(verbose.ERROR).feed(" [CAFFE] Model Parameter Parsing ...") self._ParserModel() + #self._SplitInception(True) model_layers = self.net_param_weights.layers or self.net_param_weights.layer for idx, rlayer in enumerate(real_layers): source_layer_name = rlayer.name diff --git a/tools/external_converter_v2/parser/kill_fluid/__init__.py b/tools/external_converter_v2/parser/fluid/__init__.py similarity index 57% rename from tools/external_converter_v2/parser/kill_fluid/__init__.py rename to tools/external_converter_v2/parser/fluid/__init__.py index 4d8ee7a14..4d195813c 100644 --- a/tools/external_converter_v2/parser/kill_fluid/__init__.py +++ b/tools/external_converter_v2/parser/fluid/__init__.py @@ -1,5 +1,4 @@ #! /usr/bin/env python -# Copyright (c) 2017, Cuichaowen. All rights reserved. # -*- coding: utf-8 -*- from parser_fluid import * diff --git a/tools/external_converter_v2/parser/fluid/fluid_debugger.py b/tools/external_converter_v2/parser/fluid/fluid_debugger.py new file mode 100644 index 000000000..9cfa4499c --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/fluid_debugger.py @@ -0,0 +1,65 @@ +from ..proto import * +from ..graph_io import * +import copy +import paddle.fluid as fluid +import numpy as np +from paddle.fluid.core import VarDesc, AttrType + + +class Fluid_debugger: + def var_names_of_fetch(self, fetch_targets): + """ + """ + var_names_list = [] + for var in fetch_targets: + var_names_list.append(var.name) + return var_names_list + def fetch_tmp_vars(self, block, fetch_targets, var_names_list=None): + """ + """ + fetch_var = block.var('fetch') + old_fetch_names = self.var_names_of_fetch(fetch_targets) + new_fetch_vars = [] + for var_name in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + i = len(new_fetch_vars) + if var_names_list is None: + var_names_list = block.vars.keys() + for var_name in var_names_list: + if '.tmp_' in var_name and var_name not in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + block.append_op( + type='fetch', + inputs={'X': [var_name]}, + outputs={'Out': [fetch_var]}, + attrs={'col': i}) + i = i + 1 + return new_fetch_vars + + def print_tmp_vars(self, block, var_names_list): + """ + Print the temporary variable for fluid. + """ + for var_name in var_names_list: + var_to_print = block.var(var_name) + out_to_print = block.create_var( + name=var_name + '.print', + dtype="float32", + persistable=True, + stop_gradient=False) + block.append_op( + type='print', + inputs={'In': var_to_print}, + attrs={ + 'first_n': -1, + 'summarize': -1, + 'message': "", + 'print_tensor_name': True, + 'print_tensor_type': True, + 'print_tensor_shape': True, + 'print_tensor_lod': True, + 'print_phase': 'FORWARD' + }, + outputs={'Out': out_to_print}) diff --git a/tools/external_converter_v2/parser/fluid/fluid_helper.py b/tools/external_converter_v2/parser/fluid/fluid_helper.py new file mode 100644 index 000000000..73ad3f931 --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/fluid_helper.py @@ -0,0 +1,617 @@ +from ..proto import * +from ..graph_io import * +import paddle.fluid as fluid +import numpy as np +from paddle.fluid.core import VarDesc, AttrType + + +def union(list_a, list_b): + ''' + ''' + return list(set(list_a).union(set(list_b))) + +def difference(list_a, list_b): + ''' + ''' + return list(set(list_a).difference(set(list_b))) + + +class Edge_for_fluid: + + def __init__(self, param, target, var): + ''' + ''' + self.param = param + self.target = target + self.var = var + + +class Fluid_edger: + + def __init__(self, param = None, target = None, var = None): + ''' + ''' + self.edges = [] + if param is not None and target is not None: + edge = Edge_for_fluid(param, target, var) + self.edges.append(edge) + + def __call__(self): + ''' + ''' + return self.all_targets() + + def add(self, param, target, var = None): + ''' + ''' + edge = Edge_for_fluid(param, target, var) + self.edges.append(edge) + + def rm_edges_by_param(self, param): + ''' + ''' + for edge in self.edges: + if edge.param == param: + edge_idx = self.edges.index(edge) + del self.edges[edge_idx] + + def rm(self, target): + ''' + ''' + res = -1 + for edge in self.edges: + if target == edge.target: + edge_idx = self.edges.index(edge) + del self.edges[edge_idx] + res = res + 1 + if res != 0: + pass + + def mv(self, old_target, new_target): + ''' + ''' + res = -1 + for edge in self.edges: + if old_target == edge.target: + edge.target = new_target + res = res + 1 + if res != 0: + pass + + def all_params(self): + ''' + ''' + params = [] + for edge in self.edges: + if edge.param not in params: + params.append(edge.param) + return params + + def all_targets(self): + ''' + ''' + targets = [] + for edge in self.edges: + targets.append(edge.target) + return targets + + def targets(self, param): + ''' + ''' + targets = [] + for edge in self.edges: + if edge.param == param: + targets.append(edge.target) + return targets + + def target(self, param, idx = 0): + ''' + ''' + return self.targets(param)[idx] + + def clear(self): + ''' + ''' + targets_list = self.all_targets() + for target in targets_list: + self.rm(target) + + def targets_with_params(self): + ''' + ''' + list_of_targets_and_params = [] + for edge in self.edges: + target_and_param = [edge.target, edge.param] + list_of_targets_and_params.append(target_and_param) + return list_of_targets_and_params + + def vars_by_target(self, target): + ''' + ''' + vars = [] + for edge in self.edges: + if edge.target == target and edge.var is not None: + vars.append(edge.var) + return vars + + def __getitem__(self, idx): + ''' + ''' + if idx < len(self.edges): + return self.edges[idx] + return None + + +class Fluid_helper: + ''' + ''' + def __init__(self, scope, block): + ''' + ''' + self.scope = scope + self.block = block + + def args_by_input_param(self, op, param_name): + ''' + ''' + if param_name in op.input_names: + return op.input(param_name) + else: + raise NameError('ERROR: param_name %s is not exists.' % (param_name)) + + def args_by_output_param(self, op, param_name): + ''' + ''' + if param_name in op.output_names: + return op.output(param_name) + else: + raise NameError('ERROR: param_name %s is not exists.' % (param_name)) + + def var_by_input_param(self, op, param_name, var_idx = 0): + ''' + ''' + var_name = self.args_by_input_param(op, param_name)[var_idx] + var = self.block.var(var_name) + return var + + def var_by_output_param(self, op, param_name, var_idx = 0): + ''' + ''' + var_name = self.args_by_output_param(op, param_name)[var_idx] + var = self.block.var(var_name) + return var + + def var_name_by_param(self, op, param_name, var_idx = 0): + ''' + ''' + if param_name not in op.input_names + op.output_names: + raise NameError('ERROR: param_name %s is not exists.' % (param_name)) + elif param_name in op.input_names: + if len(op.input(param_name)) > 0: + var_name_unicode = op.input(param_name)[var_idx] + else: + raise NameError('ERROR: param %s has not var.' % (param_name)) + elif param_name in op.output_names: + if len(op.output(param_name)) > 0: + var_name_unicode = op.output(param_name)[var_idx] + else: + raise NameError('ERROR: param %s has not var.' % (param_name)) + var = self.block.var(var_name_unicode) + var_name = var.name + return var_name + + def var_by_param(self, op, param_name, var_idx = 0): + ''' + ''' + var_name = self.var_name_by_param(op, param_name, var_idx) + var = self.block.var(var_name) + return var + + def shape_by_var_name(self, var_name, layout = 'NCHW'): + ''' + ''' + var = self.block.var(var_name) + long_tuple = var.shape + long_list = list(long_tuple) + if layout == 'NCHW': + int_list_4d = map(int, [1] * (4 - len(long_list)) + long_list) + return int_list_4d + elif layout == 'UNMODIFIED': + return long_list + else: + raise NameError('ERROR: layout %s is not implemented yet.' % (layout)) + + def np_data_by_var_name(self, var_name): + ''' + ''' + if hasattr(fluid.executor, '_fetch_var'): + numpy_array = fluid.executor._fetch_var(str(var_name), self.scope, True) + elif hasattr(fluid.executor, 'fetch_var'): + numpy_array = fluid.executor.fetch_var(var_name, self.scope, True) + else: + raise NameError('ERROR: Unknown Fluid version.') + return numpy_array + + def dtype_by_var_name(self, var_name): + ''' + ''' + var = self.block.var(var_name) + fluid_var_type = var.dtype + dtype = ANAKIN_TENSOR_DTYPE[fluid_var_type] + return dtype + + def is_persistable_param(self, op, param_name, var_idx = 0): + ''' + ''' + var = self.var_by_param(op, param_name, var_idx) + is_persistable_var = var.persistable + return is_persistable_var + + def var_shape_by_param(self, transpose, op, param_name, var_idx = 0, layout = 'NCHW'): + ''' + ''' + if transpose is True: + raise NameError('ERROR: var_shape transpose is not implemented yet.') + else: + var_name = self.var_name_by_param(op, param_name, var_idx) + shape = self.shape_by_var_name(var_name, layout) + return shape + + def data_with_shape_by_param(self, + op, + param_name, + transpose = False, + axes = None, + var_idx = 0, + is_flat_list = True, + layout = 'NCHW'): + ''' + ''' + np.set_printoptions(threshold=np.inf, suppress=True) + + var_name = self.var_name_by_param(op, param_name, var_idx) + np_array = self.np_data_by_var_name(var_name) + if transpose is True: + np_array = np.transpose(np_array, axes) + np_shape = np.shape(np_array) + if layout == 'NCHW': + np_shape = map(int, [1] * (4 - len(np_shape)) + list(np_shape)) + if is_flat_list is True: + flat_list = list(np_array.flatten()) + return [flat_list, np_shape] + else: + return [np_array, np_shape] + + def np_param(self, + op, + param_name, + transpose = False, + axes = None, + var_idx = 0): + ''' + ''' + [data, np_shape] = self.data_with_shape_by_param(op, param_name, transpose, \ + axes, var_idx, False) + return data + + def dtype_by_param(self, op, param_name, var_idx = 0): + ''' + ''' + var_name = self.var_name_by_param(op, param_name, var_idx) + dtype = self.dtype_by_var_name(var_name) + return dtype + + def is_list_type(self, op, attr_name): + ''' + ''' + if op.has_attr(attr_name): + fluid_attr_type = op.attr_type(attr_name) + if fluid_attr_type in ANAKIN_ATTR_IS_LIST.keys(): + return ANAKIN_ATTR_IS_LIST[fluid_attr_type] + else: + return False # AttrType.LONG + else: + raise NameError('ERROR: attr_name %s is not exists.' % (attr_name)) + + def dtype_of_attr(self, op, attr_name): + ''' + ''' + if op.has_attr(attr_name): + fluid_attr_type = op.attr_type(attr_name) + if fluid_attr_type in ANAKIN_ATTR_DTYPE.keys(): + return ANAKIN_ATTR_DTYPE[fluid_attr_type] + else: + return INT32 # AttrType.LONG + else: + raise NameError('ERROR: attr_name %s is not exists.' % (attr_name)) + + def attr_data_required(self, op, attr_name): + ''' + ''' + data = op.attr(attr_name) + is_list = self.is_list_type(op, attr_name) + dtype = self.dtype_of_attr(op, attr_name) + if dtype not in [INT32, FLOAT, STR]: + return data + elif dtype == INT32: + return map(int, data) if is_list else int(data) + elif dtype == FLOAT: + return map(float, data) if is_list else float(data) + elif dtype == STR: + return bytes(data) + + def attr_data(self, op, attr_name, default_value = 0, type = None): + ''' + ''' + if op.has_attr(attr_name): + return self.attr_data_required(op, attr_name) + else: + #raise NameError('ERROR: attr_name %s is not exists.' % ( attr_name ) ) + return default_value + + def param_tensor_sh(self, + op, + param_name, + transpose = False, + axes = None, + reshape = None, + var_idx = 0, + layout = 'NCHW'): + ''' + ''' + tensor = TensorProtoIO() + [flat_data, shape] = self.data_with_shape_by_param(op, param_name, transpose, \ + axes, var_idx, True, layout) + dtype = self.dtype_by_param(op, param_name, var_idx) + tensor.set_data_type(dtype) + if dtype in ANAKIN_TENSOR_DTYPESTR.keys(): + tensor.set_data(flat_data, ANAKIN_TENSOR_DTYPESTR[dtype]) + #pass #debug + else: + raise NameError('ERROR: Unknown data type (%s)' % (dtype)) + if reshape is not None: + tensor.set_shape(reshape) + else: + tensor.set_shape(shape) + return [tensor, shape] + + def param_tensor(self, + op, + param_name, + transpose = False, + axes = None, + reshape = None, + var_idx = 0, + layout = 'NCHW'): + ''' + ''' + [tensor, shape] = self.param_tensor_sh(op, param_name, transpose, axes, \ + reshape, var_idx, layout) + return tensor + + def create_tensor(self, data_list, data_shape, dtype): + ''' + ''' + tensor = TensorProtoIO() + tensor.set_data_type(dtype) + tensor.set_data(data_list, ANAKIN_TENSOR_DTYPESTR[dtype]) + tensor.set_shape(data_shape) + return tensor + + def gru_tensor_convert(self, origin_h2h, origin_i2h, origin_b, offset=[2, 1, 0]): + ''' + ''' + hidden_size = int(origin_b.size // 3) + word_size = int(origin_i2h.size // hidden_size // 3) + tar_h2h = np.array(origin_h2h.flatten().tolist()[2 * hidden_size * hidden_size:]\ + + np.array(origin_h2h.flatten().tolist()[: 2 * hidden_size * hidden_size])\ + .reshape(hidden_size, 2, hidden_size)[:, [1, 0], :].flatten().tolist())\ + .reshape(1, 1, hidden_size, 3 * hidden_size) + tar_i2h = origin_i2h.reshape(word_size, 3, hidden_size)[:, offset, :]\ + .reshape(1, 1, word_size, 3 * hidden_size) + tar_b = origin_b.reshape(3, hidden_size)[offset, :].reshape(1, 1, 1, 3 * hidden_size) + tar_i2h_h2h = np.concatenate([tar_i2h.flatten(), tar_h2h.flatten()])\ + .reshape(1, 1, 1, 3 * hidden_size * hidden_size + 3 * word_size * hidden_size) + return tar_i2h_h2h, tar_b + + def lstm_fc_tensor_merge_convert(self, + origin_hidden_size, + origin_lstm_w, + origin_lstm_b, + origin_fc_w, + origin_fc_b): + ''' + ''' + layer_size = int(origin_hidden_size // 4) + input_size = int(origin_fc_w.size // origin_hidden_size) + lstm_bias_num = int(origin_lstm_b.size // layer_size) + tar_w = np.vstack((np.hstack((origin_fc_w[:, 1 * layer_size: 2 * layer_size], + origin_fc_w[:, 2 * layer_size: 3 * layer_size], + origin_fc_w[:,: 1 * layer_size], + origin_fc_w[:, 3 * layer_size:])), + np.hstack((origin_lstm_w[:, 1 * layer_size: 2 * layer_size], + origin_lstm_w[:, 2 * layer_size: 3 * layer_size], + origin_lstm_w[:, : 1 * layer_size], + origin_lstm_w[:, 3 * layer_size:])))) + + if origin_fc_b is not None: + split_fc_bc = origin_fc_b.flatten()[: 1 * layer_size] + split_fc_bi = origin_fc_b.flatten()[1 * layer_size : 2 * layer_size] + split_fc_bf = origin_fc_b.flatten()[2 * layer_size : 3 * layer_size] + split_fc_bo = origin_fc_b.flatten()[3 * layer_size : 4 * layer_size] + else: + split_fc_bc = np.zeros(layer_size) + split_fc_bi = np.zeros(layer_size) + split_fc_bf = np.zeros(layer_size) + split_fc_bo = np.zeros(layer_size) + + split_lstm_bc = origin_lstm_b.flatten()[: 1 * layer_size] + split_lstm_bi = origin_lstm_b.flatten()[1 * layer_size: 2 * layer_size] + split_lstm_bf = origin_lstm_b.flatten()[2 * layer_size: 3 * layer_size] + split_lstm_bo = origin_lstm_b.flatten()[3 * layer_size: 4 * layer_size] + split_lstm_bc = np.add(split_lstm_bc, split_fc_bc) + split_lstm_bi = np.add(split_lstm_bi, split_fc_bi) + split_lstm_bf = np.add(split_lstm_bf, split_fc_bf) + split_lstm_bo = np.add(split_lstm_bo, split_fc_bo) + + if lstm_bias_num == 4: + tar_b = np.array(split_lstm_bi.flatten().tolist() + + split_lstm_bf.flatten().tolist() + + split_lstm_bc.flatten().tolist() + + split_lstm_bo.flatten().tolist()) + else: + split_lstm_wic = origin_lstm_b.flatten()[4 * layer_size : 5 * layer_size] + split_lstm_wfc = origin_lstm_b.flatten()[5 * layer_size : 6 * layer_size] + split_lstm_woc = origin_lstm_b.flatten()[6 * layer_size :] + tar_b = np.array(split_lstm_bi.flatten().tolist() + + split_lstm_bf.flatten().tolist() + + split_lstm_bc.flatten().tolist() + + split_lstm_bo.flatten().tolist() + + split_lstm_wic.flatten().tolist() + + split_lstm_wfc.flatten().tolist() + + split_lstm_woc.flatten().tolist()) + return tar_w.reshape(input_size + layer_size, 4 * layer_size, 1, 1),\ + tar_b.reshape(1, origin_lstm_b.size, 1, 1) + + +class Fluid_comparator: + """ + """ + def __init__(self, helper): + """ + """ + self.helper = helper + self.only_list = ['feed', 'fetch'] + + def compare_by_param(self, op_a, op_b, param): + """ + """ + is_weight_a = self.helper.is_persistable_param(op_a, param) + is_weight_b = self.helper.is_persistable_param(op_b, param) + if is_weight_a and is_weight_b: + np_a = self.helper.np_param(op_a, param) + np_b = self.helper.np_param(op_b, param) + if (np_a == np_b).all() == True: + return True + else: + return False + elif is_weight_a is is_weight_b: + return True + else: + return False + + def have_same_weights(self, op_a, op_b): + """ + """ + is_same = True + if op_a.input_names == op_b.input_names: + params = op_a.input_names + for param in params: + if self.compare_by_param(op_a, op_b, param) is False: + is_same = False + return is_same + else: + return False + + def compare_by_attr(self, op_a, op_b, attr_name): + """ + """ + data_a = self.helper.attr_data(op_a, attr_name) + data_b = self.helper.attr_data(op_b, attr_name) + return data_a == data_b + + def have_same_attrs(self, op_a, op_b): + """ + """ + is_same = True + if op_a.attr_names == op_b.attr_names: + attrs = op_a.attr_names + for attr in attrs: + if self.compare_by_attr(op_a, op_b, attr) is False: + is_same = False + return is_same + else: + return False + + def brothers(self, op_list): + """ + """ + is_same = True + if len(op_list) > 1: + idx = 0 + for op_b in op_list[1:]: + if op_b.type not in self.only_list: + idx = op_list.index(op_b) + op_a = op_list[idx - 1] + if op_a.type not in self.only_list: + same_weights = self.have_same_weights(op_a, op_b) + same_attrs = self.have_same_attrs(op_a, op_b) + if (same_weights and same_attrs) is False: + is_same = False + else: + raise NameError('ERROR: %s is in only_list.' % (op_a.type)) + else: + raise NameError('ERROR: %s is in only_list.' % (op_b.type)) + return is_same + else: + raise NameError('ERROR: Members of op_list must be greater than 2.') + + +ANAKIN_TENSOR_DTYPE = { + VarDesc.VarType.BOOL: BOOLEN, + VarDesc.VarType.INT32: INT32, + VarDesc.VarType.FP16: FLOAT16, + VarDesc.VarType.FP32: FLOAT, + VarDesc.VarType.FP64: DOUBLE, +} + +ANAKIN_TENSOR_DTYPESTR = { + STR: "string", + INT32: "int", + FLOAT: "float", + BOOLEN: "bool", +} + +ANAKIN_ATTR_DTYPE = { + AttrType.INT: INT32, + AttrType.INTS: INT32, + AttrType.FLOAT: FLOAT, + AttrType.FLOATS: FLOAT, + AttrType.STRING: STR, + AttrType.STRINGS: STR, + AttrType.BOOL: BOOLEN, + AttrType.BOOLS: BOOLEN, +} + +ANAKIN_ATTR_IS_LIST = { + AttrType.INT: False, + AttrType.INTS: True, + AttrType.FLOAT: False, + AttrType.FLOATS: True, + AttrType.STRING: False, + AttrType.STRINGS: True, + AttrType.BOOL: False, + AttrType.BOOLS: True, +} + +APPEND_BIAS_OP_TYPE = [ + 'FC', + 'mul', + 'sequence_conv', + 'conv2d', + 'conv2d_transpose', + 'depthwise_conv2d', + 'elementwise_mul', +] + +APPEND_ACT_OP_TYPE = [ + 'FC', + 'mul', + 'sequence_conv', + 'conv2d', + 'conv2d_transpose', + 'batch_norm', + 'layer_norm', + 'row_conv', + 'reshape', +] diff --git a/tools/external_converter_v2/parser/fluid/fluid_layer_param_transmit.py b/tools/external_converter_v2/parser/fluid/fluid_layer_param_transmit.py new file mode 100644 index 000000000..cfc1cd252 --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/fluid_layer_param_transmit.py @@ -0,0 +1,505 @@ +from ..operations import OpsParam, OpsRegister +from ..logger import * +from ..proto import * +from fluid_helper import * + + +def ParserFeedDecorator(OpName): + def warpper(Parser): + def warpper_args(args): + Parser(args) + OpsRegister()[OpName].feed_node_attr(args[0]) + args[2].set_name(OpName) + args[0].set_op(args[2]()) + return warpper_args + return warpper + +# common +def NotNeededInInference(args): + # args is tuple object + node_io = args[0] + layer = args[1] + +@ParserFeedDecorator("Input") +def Parser_feed(args): + private_data = args[4] + input_shape = private_data['input_shape'] + alias = private_data['alias'] + OpsRegister()["Input"].input_shape = input_shape + OpsRegister()["Input"].alias = alias + +@ParserFeedDecorator("Convolution") +def Parser_conv2d(args): + op = args[1] + helper = args[3] + private_data = args[4] + [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Filter') + OpsRegister()["Convolution"].weight_1 = weights_tensor + OpsRegister()["Convolution"].filter_num = weights_shape[0] + OpsRegister()["Convolution"].kernel_size = weights_shape[-2:] + OpsRegister()["Convolution"].strides = helper.attr_data(op, 'strides') + OpsRegister()["Convolution"].padding = helper.attr_data(op, 'paddings') + OpsRegister()["Convolution"].dilation_rate = helper.attr_data(op, 'dilations') + OpsRegister()["Convolution"].group = helper.attr_data(op, 'groups') + OpsRegister()["Convolution"].axis = 1 + if 'bias' in private_data.keys(): + OpsRegister()["Convolution"].bias_term = True + OpsRegister()["Convolution"].weight_2 = private_data['bias'] + else: + OpsRegister()["Convolution"].bias_term = False + +@ParserFeedDecorator("ReLU") +def Parser_relu(args): + OpsRegister()["ReLU"].alpha = 0.0 + +@ParserFeedDecorator("Pooling") +def Parser_pool2d(args): + op = args[1] + helper = args[3] + OpsRegister()["Pooling"].pool_size = helper.attr_data(op, 'ksize') + OpsRegister()["Pooling"].strides = helper.attr_data(op, 'strides') + OpsRegister()["Pooling"].padding = helper.attr_data(op, 'paddings') + OpsRegister()["Pooling"].global_pooling = helper.attr_data(op, 'global_pooling') + if helper.attr_data(op, 'pooling_type') == 'max': + OpsRegister()["Pooling"].method = "MAX" + elif helper.attr_data(op, 'pooling_type') in ['average', 'avg']: + OpsRegister()["Pooling"].method = "AVG" + if helper.attr_data(op, 'ceil_mode') == False: + OpsRegister()["Pooling"].cmp_out_shape_floor_as_conv = True + else: + OpsRegister()["Pooling"].cmp_out_shape_floor_as_conv = False + +@ParserFeedDecorator("Dense") +def Parser_mul(args): + op = args[1] + helper = args[3] + private_data = args[4] + weights_needs_trans = True + [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Y', weights_needs_trans) + OpsRegister()["Dense"].weight_1 = weights_tensor + OpsRegister()["Dense"].out_dim = weights_shape[2] + OpsRegister()["Dense"].axis = helper.attr_data(op, 'x_num_col_dims') + if 'bias' in private_data.keys(): + OpsRegister()["Dense"].bias_term = True + OpsRegister()["Dense"].weight_2 = private_data['bias'] + else: + OpsRegister()["Dense"].bias_term = False + +@ParserFeedDecorator("Softmax") +def Parser_softmax(args): + private_data = args[4] + if 'axis' in private_data.keys(): + axis = private_data['axis'] + else: + axis = 1 + OpsRegister()["Softmax"].axis = axis + +@ParserFeedDecorator("Activation") +def Parser_sigmoid(args): + OpsRegister()["Activation"].type = "Sigmoid" + +@ParserFeedDecorator("Axpy") +def Parser_axpy(args): + pass + +@ParserFeedDecorator("BatchNorm") +def Parser_batch_norm(args): + op = args[1] + helper = args[3] + OpsRegister()["BatchNorm"].weight_1 = helper.param_tensor(op, 'Mean') + OpsRegister()["BatchNorm"].weight_2 = helper.param_tensor(op, 'Variance') + OpsRegister()["BatchNorm"].weight_3 = helper.create_tensor([1], [1, 1, 1, 1], FLOAT) + OpsRegister()["BatchNorm"].momentum = helper.attr_data(op, 'momentum') + OpsRegister()["BatchNorm"].epsilon = helper.attr_data(op, 'epsilon') + +@ParserFeedDecorator("Scale") +def Parser_scale_disc_bn(args): + op = args[1] + helper = args[3] + mean = helper.np_param(op, 'Mean') + var = helper.np_param(op, 'Variance') + alpha = helper.np_param(op, 'Scale') + beta = helper.np_param(op, 'Bias') + eps = helper.attr_data(op, 'epsilon') + var = np.sqrt(var + eps) + np_scale = alpha / var + np_bias = beta - (alpha * mean / var) + np_scale_shape = map(int, [1] * (4 - len(np_scale.shape)) + list(np_scale.shape)) + np_bias_shape = map(int, [1] * (4 - len(np_bias.shape)) + list(np_bias.shape)) + np_scale_tensor = helper.create_tensor(list(np_scale.flatten()), np_scale_shape, FLOAT) + np_bias_tensor = helper.create_tensor(list(np_bias.flatten()), np_bias_shape, FLOAT) + OpsRegister()["Scale"].bias_term = True + OpsRegister()["Scale"].weight_1 = np_scale_tensor + OpsRegister()["Scale"].weight_2 = np_bias_tensor + OpsRegister()["Scale"].axis = 1 + OpsRegister()["Scale"].num_axes = 1 + +@ParserFeedDecorator("Scale") +def Parser_scale_of_bn(args): + op = args[1] + helper = args[3] + OpsRegister()["Scale"].weight_1 = helper.param_tensor(op, 'Scale') + OpsRegister()["Scale"].axis = 1 + OpsRegister()["Scale"].num_axes = 1 + has_bias = helper.is_persistable_param(op, 'Bias') + if has_bias is True: + OpsRegister()["Scale"].bias_term = True + OpsRegister()["Scale"].weight_2 = helper.param_tensor(op, 'Bias') + else: + OpsRegister()["Scale"].bias_term = False + +@ParserFeedDecorator("Split") +def Parser_split(args): + private_data = args[4] + split_num = private_data['split_num'] + OpsRegister()["Split"].split_num = split_num + +@ParserFeedDecorator("Reshape") +def Parser_reshape(args): + op = args[1] + helper = args[3] + private_data = args[4] + if 'new_shape' in private_data.keys(): + shape = private_data['new_shape'] + else: + shape = helper.attr_data(op, 'shape') + shape = map(int, shape + [1] * (4 - len(shape))) + OpsRegister()["Reshape"].dims = shape + +@ParserFeedDecorator("Concat") +def Parser_concat(args): + op = args[1] + helper = args[3] + OpsRegister()["Concat"].axis = helper.attr_data(op, 'axis') + +@ParserFeedDecorator("Concat") +def Parser_concat_btw_priorbox_boxcoder(args): + op = args[1] + helper = args[3] + OpsRegister()["Concat"].axis = 3 + +@ParserFeedDecorator("Permute") +def Parser_transpose(args): + op = args[1] + helper = args[3] + fluid_dims = helper.attr_data(op, 'axis') + n = 4 - len(fluid_dims) + dims = range(0, n) + tail_dims = [i + n for i in fluid_dims] + dims.extend(tail_dims) + OpsRegister()["Permute"].dims = dims + + +########## SSD Model ########## + +@ParserFeedDecorator("PriorBox") +def Parser_prior_box(args): + op = args[1] + helper = args[3] + OpsRegister()["PriorBox"].min_size = helper.attr_data(op, 'min_sizes') + OpsRegister()["PriorBox"].max_size = helper.attr_data(op, 'max_sizes') + OpsRegister()["PriorBox"].aspect_ratio = helper.attr_data(op, 'aspect_ratios') + OpsRegister()["PriorBox"].is_flip = helper.attr_data(op, 'flip') + OpsRegister()["PriorBox"].is_clip = helper.attr_data(op, 'clip') + OpsRegister()["PriorBox"].variance = helper.attr_data(op, 'variances') + OpsRegister()["PriorBox"].img_h = 0 + OpsRegister()["PriorBox"].img_w = 0 + OpsRegister()["PriorBox"].step_h = helper.attr_data(op, 'step_h') + OpsRegister()["PriorBox"].step_w = helper.attr_data(op, 'step_w') + OpsRegister()["PriorBox"].offset = helper.attr_data(op, 'offset') + OpsRegister()["PriorBox"].order = ['MIN', 'COM', 'MAX'] + +@ParserFeedDecorator("box_coder") +def Parser_box_coder(args): + pass + +@ParserFeedDecorator("DetectionOutput") +def Parser_multiclass_nms(args): + op = args[1] + helper = args[3] + private_data = args[4] + OpsRegister()["DetectionOutput"].share_location = True + OpsRegister()["DetectionOutput"].variance_encode_in_target = False + OpsRegister()["DetectionOutput"].class_num = 0 + OpsRegister()["DetectionOutput"].background_id = helper.attr_data(op, 'background_label') + OpsRegister()["DetectionOutput"].keep_top_k = helper.attr_data(op, 'keep_top_k') + OpsRegister()["DetectionOutput"].conf_thresh = helper.attr_data(op, 'score_threshold') + OpsRegister()["DetectionOutput"].nms_top_k = helper.attr_data(op, 'nms_top_k') + OpsRegister()["DetectionOutput"].nms_thresh = helper.attr_data(op, 'nms_threshold') + OpsRegister()["DetectionOutput"].nms_eta = helper.attr_data(op, 'nms_eta') + if 'code_type' in private_data.keys(): + if private_data['code_type'] == 'decode_center_size': + OpsRegister()["DetectionOutput"].code_type = "CENTER_SIZE" + else: + OpsRegister()["DetectionOutput"].code_type = "CORNER" + + +########## VIS Model ########## + +@ParserFeedDecorator("Im2Sequence") +def Parser_im2sequence(args): + op = args[1] + helper = args[3] + OpsRegister()["Im2Sequence"].paddings = helper.attr_data(op, 'paddings') + OpsRegister()["Im2Sequence"].strides = helper.attr_data(op, 'strides') + OpsRegister()["Im2Sequence"].window_size = helper.attr_data(op, 'kernels') + OpsRegister()["Im2Sequence"].dilations = helper.attr_data(op, 'dilations', [1, 1]) + +@ParserFeedDecorator("Cast") +def Parser_cast(args): + op = args[1] + helper = args[3] + OpsRegister()["Cast"].in_type = helper.attr_data(op, 'in_dtype') + OpsRegister()["Cast"].out_type = helper.attr_data(op, 'out_dtype') + +@ParserFeedDecorator("Argmax") # new256 +def Parser_top_k(args): + op = args[1] + helper = args[3] + OpsRegister()["Argmax"].out_max_val = True + OpsRegister()["Argmax"].top_k = helper.attr_data(op, 'k') + OpsRegister()["Argmax"].axis_term = False + +@ParserFeedDecorator("CtcAlign") +def Parser_ctc_align(args): + op = args[1] + helper = args[3] + OpsRegister()["CtcAlign"].merge_repeated = helper.attr_data(op, 'merge_repeated') + OpsRegister()["CtcAlign"].blank = helper.attr_data(op, 'blank') + +@ParserFeedDecorator("Eltwise") +def Parser_sum(args): + OpsRegister()["Eltwise"].type = "Add" + OpsRegister()["Eltwise"].coeff = [1.0, 1.0] + +@ParserFeedDecorator("LRN") +def Parser_lrn(args): + op = args[1] + helper = args[3] + OpsRegister()["LRN"].local_size = helper.attr_data(op, 'n') + OpsRegister()["LRN"].alpha = helper.attr_data(op, 'alpha') + OpsRegister()["LRN"].beta = helper.attr_data(op, 'beta') + OpsRegister()["LRN"].norm_region = "ACROSS_CHANNELS" + OpsRegister()["LRN"].k = helper.attr_data(op, 'k') + +@ParserFeedDecorator("Gru") +def Parser_gru(args): + op = args[1] + helper = args[3] + private_data = args[4] + OpsRegister()["Gru"].is_reverse = helper.attr_data(op, 'is_reverse') + OpsRegister()["Gru"].gate_activation = helper.attr_data(op, 'gate_activation') + '_fluid' + OpsRegister()["Gru"].activation = helper.attr_data(op, 'activation') + '_fluid' + OpsRegister()["Gru"].gru_formula = "gru_origin" + if bool(private_data) is True: + ori_bx = private_data['np_bias_x'] + ori_bh = helper.np_param(op, 'Bias') + ori_b = ori_bx + ori_bh + ori_wx = private_data['np_weight_x'] + ori_wh = helper.np_param(op, 'Weight') + new_tensors = helper.gru_tensor_convert(ori_wh, ori_wx, ori_b) + weights = [] + for tensor in new_tensors: + weights.append(helper.create_tensor(list(tensor.flatten()), \ + list(np.shape(tensor)), FLOAT)) + OpsRegister()["Gru"].weight_1 = weights[0] + OpsRegister()["Gru"].weight_2 = weights[1] + else: + OpsRegister()["Gru"].weight_1 = helper.param_tensor(op, 'Weight') + OpsRegister()["Gru"].weight_2 = helper.create_tensor([0], [-1], FLOAT) + +@ParserFeedDecorator("LSTM") +def Parser_lstm(args): + op = args[1] + helper = args[3] + private_data = args[4] + OpsRegister()["LSTM"].candidate_activation = helper.attr_data(op, 'candidate_activation') + OpsRegister()["LSTM"].cell_activation = helper.attr_data(op, 'cell_activation') + OpsRegister()["LSTM"].gate_activation = helper.attr_data(op, 'gate_activation') + OpsRegister()["LSTM"].is_reverse = helper.attr_data(op, 'is_reverse') + OpsRegister()["LSTM"].use_peepholes = helper.attr_data(op, 'use_peepholes') + OpsRegister()["LSTM"].num_direction = 1 + OpsRegister()["LSTM"].dropout_param = 1.0 + OpsRegister()["LSTM"].num_layers = 1 + OpsRegister()["LSTM"].input_activation = "null" + if bool(private_data) is True: + np_fc_bias = private_data['np_flat_fc_bias'] + np_fc_weight = private_data['np_flat_fc_weight'] + np_fc_outdim = private_data['np_fc_outdim'] + np_lstm_bias = helper.np_param(op, 'Bias') + np_lstm_weight = helper.np_param(op, 'Weight') + np_tensors = helper.lstm_fc_tensor_merge_convert(np_fc_outdim, np_lstm_weight, \ + np_lstm_bias, np_fc_weight, np_fc_bias) + np_weight = np_tensors[0] + np_bias = np_tensors[1] + np_weight_shape = map(int, [1] * (4 - len(np_weight.shape)) + list(np_weight.shape)) + np_bias_shape = map(int, [1] * (4 - len(np_bias.shape)) + list(np_bias.shape)) + np_weight_tensor = helper.create_tensor(list(np_weight.flatten()), np_weight_shape, FLOAT) + np_bias_tensor = helper.create_tensor(list(np_bias.flatten()), np_bias_shape, FLOAT) + OpsRegister()["LSTM"].weight_1 = np_weight_tensor + OpsRegister()["LSTM"].weight_2 = np_bias_tensor + else: + OpsRegister()["LSTM"].weight_1 = helper.param_tensor(op, 'Weight') + OpsRegister()["LSTM"].weight_2 = helper.create_tensor([0], [-1], FLOAT) + + +############### RNN ############### + +@ParserFeedDecorator("Embedding") +def Parser_lookup_table(args): + op = args[1] + helper = args[3] + [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'W') + OpsRegister()["Embedding"].weight_1 = weights_tensor + OpsRegister()["Embedding"].padding_idx = helper.attr_data(op, 'padding_idx') + OpsRegister()["Embedding"].word_num = weights_shape[2] + OpsRegister()["Embedding"].emb_dim = weights_shape[3] + +@ParserFeedDecorator("SequencePool") +def Parser_sequence_pool(args): + op = args[1] + helper = args[3] + OpsRegister()["SequencePool"].pooltype = helper.attr_data(op, 'pooltype') + +@ParserFeedDecorator("Activation") +def Parser_tanh(args): + OpsRegister()["Activation"].type = "TanH" + +@ParserFeedDecorator("SequenceConv") +def Parser_sequence_conv(args): + op = args[1] + helper = args[3] + private_data = args[4] + [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Filter') + OpsRegister()["SequenceConv"].weight_1 = weights_tensor + OpsRegister()["SequenceConv"].filter_num = weights_shape[0] + OpsRegister()["SequenceConv"].kernel_size = weights_shape[-2:] + OpsRegister()["SequenceConv"].padding_trainable = helper.attr_data(op, 'paddingTrainable') + OpsRegister()["SequenceConv"].context_stride = helper.attr_data(op, 'contextStride') + OpsRegister()["SequenceConv"].context_start = helper.attr_data(op, 'contextStart') + OpsRegister()["SequenceConv"].context_length = helper.attr_data(op, 'contextLength') + if 'bias' in private_data.keys(): + OpsRegister()["SequenceConv"].bias_term = True + OpsRegister()["SequenceConv"].weight_2 = private_data['bias'] + else: + OpsRegister()["SequenceConv"].bias_term = False + +@ParserFeedDecorator("CrfDecoding") +def Parser_crf_decoding(args): + op = args[1] + helper = args[3] + [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Transition') + OpsRegister()["CrfDecoding"].weight_1 = weights_tensor + +@ParserFeedDecorator("MatMul") +def Parser_matmul(args): + op = args[1] + helper = args[3] + private_data = args[4] + if 'coeff' in private_data.keys(): + coeff = private_data['coeff'] + else: + coeff = 1.0 + OpsRegister()["MatMul"].transpose_x = helper.attr_data(op, 'transpose_X') + OpsRegister()["MatMul"].transpose_y = helper.attr_data(op, 'transpose_Y') + OpsRegister()["MatMul"].coeff = coeff + +@ParserFeedDecorator("Scale") +def Parser_scale(args): + op = args[1] + helper = args[3] + scale_val = helper.attr_data(op, 'scale') + OpsRegister()["Scale"].axis = 0 + OpsRegister()["Scale"].num_axes = 0 + OpsRegister()["Scale"].bias_term = False + OpsRegister()["Scale"].weight_1 = helper.create_tensor([scale_val], [1, 1, 1, 1], FLOAT) + +@ParserFeedDecorator("LayerNorm") +def Parser_layer_norm(args): + op = args[1] + helper = args[3] + OpsRegister()["LayerNorm"].weight_1 = helper.param_tensor(op, 'Scale') + OpsRegister()["LayerNorm"].weight_2 = helper.param_tensor(op, 'Bias') + OpsRegister()["LayerNorm"].begin_norm_axis = helper.attr_data(op, 'begin_norm_axis') + OpsRegister()["LayerNorm"].eps = helper.attr_data(op, 'epsilon') + +@ParserFeedDecorator("Scale") +def Parser_dropout(args): + op = args[1] + helper = args[3] + scale_val = 1 - helper.attr_data(op, 'dropout_prob') + OpsRegister()["Scale"].axis = 0 + OpsRegister()["Scale"].num_axes = 0 + OpsRegister()["Scale"].bias_term = False + OpsRegister()["Scale"].weight_1 = helper.create_tensor([scale_val], [1, 1, 1, 1], FLOAT) + +@ParserFeedDecorator("Scale") +def Parser_elementwise_mul(args): + op = args[1] + helper = args[3] + private_data = args[4] + OpsRegister()["Scale"].weight_1 = helper.param_tensor(op, 'Y') + OpsRegister()["Scale"].axis = helper.attr_data(op, 'axis') + OpsRegister()["Scale"].num_axes = 1 + if 'bias' in private_data.keys(): + OpsRegister()["Scale"].bias_term = True + OpsRegister()["Scale"].weight_2 = private_data['bias'] + else: + OpsRegister()["Scale"].bias_term = False + +@ParserFeedDecorator("Flatten") +def Parser_flatten(args): + pass + +@ParserFeedDecorator("assign_value") +def Parser_assign_value(args): + pass + +@ParserFeedDecorator("shape") +def Parser_shape(args): + pass + +FLUID_NODE_FILLER = { + "feed":OpsParam().set_parser(Parser_feed), + "conv2d":OpsParam().set_parser(Parser_conv2d), + "elementwise_add":OpsParam().set_parser(Parser_sum), + "relu":OpsParam().set_parser(Parser_relu), + "pool2d":OpsParam().set_parser(Parser_pool2d), + "mul":OpsParam().set_parser(Parser_mul), + "softmax":OpsParam().set_parser(Parser_softmax), + "sigmoid":OpsParam().set_parser(Parser_sigmoid), + "axpy":OpsParam().set_parser(Parser_axpy), + "batch_norm":OpsParam().set_parser(Parser_batch_norm), + "disc_bn":OpsParam().set_parser(Parser_scale_disc_bn), + "scale_of_bn":OpsParam().set_parser(Parser_scale_of_bn), + "elementwise_mul":OpsParam().set_parser(Parser_elementwise_mul), + "split":OpsParam().set_parser(Parser_split), + "depthwise_conv2d":OpsParam().set_parser(Parser_conv2d), + "reshape":OpsParam().set_parser(Parser_reshape), + "concat":OpsParam().set_parser(Parser_concat), + "transpose":OpsParam().set_parser(Parser_transpose), + "prior_box":OpsParam().set_parser(Parser_prior_box), + "box_coder":OpsParam().set_parser(Parser_box_coder), + "multiclass_nms":OpsParam().set_parser(Parser_multiclass_nms), + "concat_btw_priorbox_boxcoder":OpsParam().set_parser(Parser_concat_btw_priorbox_boxcoder), + "im2sequence":OpsParam().set_parser(Parser_im2sequence), + "gru":OpsParam().set_parser(Parser_gru), + "sum":OpsParam().set_parser(Parser_sum), + "lrn":OpsParam().set_parser(Parser_lrn), + "top_k":OpsParam().set_parser(Parser_top_k), + "ctc_align":OpsParam().set_parser(Parser_ctc_align), + "cast":OpsParam().set_parser(Parser_cast), + "lookup_table":OpsParam().set_parser(Parser_lookup_table), + "sequence_pool":OpsParam().set_parser(Parser_sequence_pool), + "tanh":OpsParam().set_parser(Parser_tanh), + "sequence_conv":OpsParam().set_parser(Parser_sequence_conv), + "crf_decoding":OpsParam().set_parser(Parser_crf_decoding), + "lstm":OpsParam().set_parser(Parser_lstm), + "matmul":OpsParam().set_parser(Parser_matmul), + "layer_norm":OpsParam().set_parser(Parser_layer_norm), + "dropout":OpsParam().set_parser(Parser_dropout), + "scale":OpsParam().set_parser(Parser_scale), + "flatten":OpsParam().set_parser(Parser_flatten), + "assign_value":OpsParam().set_parser(Parser_assign_value), + "shape":OpsParam().set_parser(Parser_shape), +} diff --git a/tools/external_converter_v2/parser/fluid/parser_fluid.py b/tools/external_converter_v2/parser/fluid/parser_fluid.py new file mode 100644 index 000000000..84efb8949 --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/parser_fluid.py @@ -0,0 +1,932 @@ +import numpy as np +import paddle.fluid as fluid +import os +from ..graph_io import * +from ..logger import * +from ..proto import * +from fluid_layer_param_transmit import * + +class FluidParser: + + def __init__(self, fluid_config_dict): + # anakin graph model io + self.graphIO = None + # config info + self.ModelPath = fluid_config_dict['ModelPath'] + self.NetType = fluid_config_dict['NetType'] + self.Debug = fluid_config_dict['Debug'] + # config fluid + self.place = fluid.CPUPlace() + self.exe = fluid.Executor(self.place) + self.scope = fluid.core.Scope() + # in and out edges of node + self.ins = {} + self.outs = {} + # inplaced main node + self.inplace_nodes = {} + self.graph_ins = [] + self.graph_outs = [] + + def __call__(self): + return self._Parsing() + + def _NameNodeMid(self, op): + first_outparam = op.output_names[0] + arg_name = str(op.output(first_outparam)[0]).split('.')[0] + #new_name = op.type + '_' + bytes(op.idx) + new_name = op.type + '#' + bytes(op.idx) + '(' + arg_name + ')' + return new_name + + def _NameNodeIn(self, in_name): + new_name = 'input_' + bytes(self.graph_ins.index(in_name)) + return new_name + + def _NameNodeOut(self, out_name): + new_name = out_name + '_gout' + return new_name + + def _AddPairEdges(self, start_node_name, end_node_name, out_param, in_param): + self.outs[start_node_name].add(out_param, end_node_name) + self.ins[end_node_name].add(in_param, start_node_name) + + def _RmPairEdges(self, start_node_name, end_node_name): + self.outs[start_node_name].rm(end_node_name) + self.ins[end_node_name].rm(start_node_name) + + def _InitEdges(self, node_name): + self.ins[node_name] = Fluid_edger() + self.outs[node_name] = Fluid_edger() + + def _ClearEdges(self, node_name): + if node_name.startswith('input_') is False: + del self.ins[node_name] + if node_name.endswith('_gout') is False: + del self.outs[node_name] + + def _GetOp(self, ops, mid_node_name): + mid_op = None + for op in ops: + node_name = self._NameNodeMid(op) + if mid_node_name == node_name: + mid_op = op + return mid_op + + def _OpTypes(self, ops): + types_cache = [] + for op in ops: + if op.type not in types_cache: + types_cache.append(op.type) + return types_cache + + def _AddProtoNode(self, node_name, op_of_node, helper, private_data, op_type=None): + nodeIO = NodeProtoIO() + opIO = OpsProtoIO() + nodeIO.set_name(node_name) + if op_type is None: + op_type = op_of_node.type + FLUID_NODE_FILLER[op_type](nodeIO, op_of_node, opIO, helper, private_data) + self.graphIO.add_node(nodeIO()) + + def _RmProtoNode(self, node_name): + self.graphIO.rm_node(self.graphIO.find_node_proto(node_name)) + + def _InplaceNodes(self, position): + inplace_heads = self.inplace_nodes.keys() + inplace_mids = [] + inplace_ends = [] + for main_node_name in self.inplace_nodes.keys(): + mid_nodes_name = self.inplace_nodes[main_node_name][1: -1] + inplace_mids.extend(mid_nodes_name) + for main_node_name in self.inplace_nodes.keys(): + end_node_name = self.inplace_nodes[main_node_name][-1] + inplace_ends.append(end_node_name) + if position == 'Head': + return inplace_heads + elif position == 'Mid': + return inplace_mids + elif position == 'End': + return inplace_ends + elif position == 'All': + return inplace_heads + inplace_mids + inplace_ends + + def _EdgeInplace(self, source_ops, helper): + for source_op in source_ops: + if source_op.type not in ['feed', 'fetch']: + if len(source_op.input_arg_names) == 1 \ + and source_op.input_arg_names == source_op.output_arg_names: + source_node_name = self._NameNodeMid(source_op) + inplace_arg = source_op.input_arg_names[0] + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and inplace_arg in tmp_op.output_arg_names: + main_node_name = self._NameNodeMid(tmp_op) + if main_node_name not in self.inplace_nodes.keys(): + self.inplace_nodes[main_node_name] = [main_node_name] + self.inplace_nodes[main_node_name].append(source_node_name) + for main_node_name in self.inplace_nodes.keys(): + inplace_list = self.inplace_nodes[main_node_name] + for inplace_node in inplace_list: + idx = inplace_list.index(inplace_node) + if idx != 0: + self.ins[inplace_node] = Fluid_edger('_In', inplace_list[idx - 1]) + if idx != len(inplace_list) - 1: + self.outs[inplace_node] = Fluid_edger('_Out', inplace_list[idx + 1]) + + def _GetDebugOuts(self, source_ops, helper): + if self.Debug == 'DEBUG': + debug_fetch_list = [] + for source_op in source_ops: + if source_op.type == 'fetch': + var_name = source_op.input_arg_names[0] + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and var_name in tmp_op.input_arg_names: + if var_name not in debug_fetch_list: + debug_fetch_list.append(var_name) + elif tmp_op.type == 'gru' and var_name in tmp_op.output_arg_names: + if var_name not in debug_fetch_list: + debug_fetch_list.append(var_name) + else: + pass + return debug_fetch_list + else: + return [] + + def _ParseBase(self, source_ops, helper, sub_graph_nodes=None): + # Create the original base graph as described in fluid program. + if sub_graph_nodes is None: + sub_graph_nodes = list() + self.graphIO = GraphProtoIO() + self.graphIO.set_name('default_graph_name') + debug_fetch_list = self._GetDebugOuts(source_ops, helper) + self._EdgeInplace(source_ops, helper) + for source_op in source_ops: + if source_op.type not in ['feed', 'fetch']: + main_node_name = self._NameNodeMid(source_op) + in_edges = Fluid_edger() + out_edges = Fluid_edger() + for param in source_op.input_names: + for idx in range(0, len(helper.args_by_input_param(source_op, param))): + arg = helper.var_name_by_param(source_op, param, idx) + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and arg in tmp_op.output_arg_names: + if tmp_op.type == 'feed': + if arg not in self.graph_ins: + self.graph_ins.append(arg) + self.graphIO.add_in(self._NameNodeIn(arg)) + in_edges.add(param, self._NameNodeIn(arg), arg) + else: + tmp_node_name = self._NameNodeMid(tmp_op) + if tmp_node_name in self.inplace_nodes.keys(): + inplace_node_name = self.inplace_nodes[tmp_node_name][-1] + in_edges.add(param, inplace_node_name, arg) + elif tmp_node_name not in self._InplaceNodes('All'): + in_edges.add(param, tmp_node_name, arg) + for param in source_op.output_names: + for idx in range(0, len(helper.args_by_output_param(source_op, param))): + arg = helper.var_name_by_param(source_op, param, idx) + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and arg in tmp_op.input_arg_names: + if tmp_op.type == 'fetch': + if arg not in debug_fetch_list: + arg_node_name = self._NameNodeOut(arg) + if arg not in self.graph_outs: + self.graph_outs.append(arg) + self.graphIO.add_out_fluid(arg_node_name, \ + main_node_name) + out_edges.add(param, arg_node_name, arg) + self.ins[arg_node_name] = Fluid_edger(bytes(source_op.idx), \ + main_node_name) + else: + out_edges.add(param, self._NameNodeMid(tmp_op), arg) + self._AddProtoNode(main_node_name, source_op, helper, {}) + if main_node_name not in self._InplaceNodes('Mid'): + if main_node_name not in self._InplaceNodes('End'): + self.ins[main_node_name] = in_edges + if main_node_name not in self._InplaceNodes('Head'): + if main_node_name not in self._InplaceNodes('End'): + self.outs[main_node_name] = out_edges + else: + inplace_node_name = self.inplace_nodes[main_node_name][-1] + self.outs[inplace_node_name] = out_edges + for redundant_target in self.inplace_nodes[main_node_name][1:]: + self.outs[inplace_node_name].rm(redundant_target) + + def _PrintEdge(self, node, target, direction): + var_name = 'Unknown' + if direction == 'in': + var = self.ins[node].vars_by_target(target) + elif direction == 'out': + var = self.outs[node].vars_by_target(target) + if len(var) > 0: + var_name = var[0] + print node + ",\t" + target + ",\t" + var_name + + def _Graph(self, need_print=False): + for node in self.ins.keys(): + targets_list = self.ins[node]() + for target in targets_list: + self.graphIO.add_in_edge(target, node) + for node in self.outs.keys(): + targets_list = self.outs[node]() + for target in targets_list: + self.graphIO.add_out_edge(node, target) + if need_print is True: + self._PrintEdge(node, target, 'out') + + def _ReplaceInputs(self, source_ops, helper, reshape_dict=None, layout='NCHW'): + if reshape_dict is None: + reshape_dict = dict() + for source_op in source_ops: + if source_op.type in ['feed']: + out_edges = Fluid_edger() + for param in source_op.output_names: + private_data = {} + arg = helper.var_name_by_param(source_op, param) + input_node_name = self._NameNodeIn(arg) + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and arg in tmp_op.input_arg_names: + out_edges.add(param, self._NameNodeMid(tmp_op)) + arg_idx = source_op.output_arg_names.index(arg) + shape = helper.var_shape_by_param(False, source_op, \ + "Out", arg_idx, 'UNMODIFIED') + if shape[0] == -1: + shape[0] = 1 + if layout == 'NCHW': + shape = map(int, [1] * (4 - len(shape)) + shape) + if input_node_name in reshape_dict.keys(): + shape = reshape_dict[input_node_name] + private_data['input_shape'] = shape + private_data['alias'] = arg + self.outs[input_node_name] = out_edges + self._AddProtoNode(input_node_name, source_op, helper, private_data) + + def _InsertSplit(self, source_ops, helper): + # If a layer has two identical output tensors, add a split layer. + for node in self.outs.keys(): + if node.startswith('split#') is False: + out_edges = self.outs[node] + for param in out_edges.all_params(): + out_targets_list = out_edges.targets(param) + if len(out_targets_list) > 1: + private_data = {} + private_data['split_num'] = len(out_targets_list) + split_node_name = 'split#' + \ + bytes(out_edges.all_params().index(param)) + '#' + node + self._InitEdges(split_node_name) + for out_target in out_targets_list: + self.outs[node].rm(out_target) + self.ins[out_target].mv(node, split_node_name) + self.outs[split_node_name].add('_Out', out_target) + self._AddPairEdges(node, split_node_name, param, '_In') + self._AddProtoNode(split_node_name, None, helper, private_data, 'split') + + def _Subgraph(self, starts, ends): + """ + """ + out_idx = {} + results = union(starts, ends) + def outs(node): + """ + """ + if node in self.outs.keys(): + return self.outs[node]() + else: + return [] + def next_out(node): + """ + """ + next_out = '' + if len(outs(node)) == 0: + return -1 + elif node not in out_idx.keys(): + out_idx[node] = 0 + if out_idx[node] < len(outs(node)): + next_out = outs(node)[out_idx[node]] + out_idx[node] += 1 + return next_out + for start in starts: + cache = [start] + while len(cache) > 0: + target = next_out(cache[-1]) + while target != -1 and target not in results: + if bool(target) is True: + cache.append(target) + target = next_out(target) + else: + if cache[-1] in results: + results = union(results, cache) + break + if target in results: + cache.append(target) + results = union(results, cache) + cache.pop() + return results + + def _CropGraph(self, ins_of_subgraph, outs_of_subgraph, helper, need_io = True): + ''' + ''' + def all_nodes(): + ''' + ''' + all_nodes = [] + for main_node in self.ins.keys(): + all_nodes.extend(self.ins[main_node].all_targets()) + for main_node in self.outs.keys(): + all_nodes.extend(self.outs[main_node].all_targets()) + return list(set(all_nodes)) + stayed_nodes = self._Subgraph(ins_of_subgraph, outs_of_subgraph) + all_nodes = all_nodes() + extra_nodes = difference(all_nodes, stayed_nodes) + for node_name in extra_nodes: + self._RmProtoNode(node_name) + self._ClearEdges(node_name) + if node_name in self.graphIO.ins(): + self.graphIO.rm_in(node_name) + if node_name in self.graphIO.outs(): + self.graphIO.rm_out(node_name) + for node_name in ins_of_subgraph: + if node_name in self.ins: + self.ins[node_name].clear() + for node_name in outs_of_subgraph: + if node_name in self.outs: + self.outs[node_name].clear() + if need_io is True: + for node_name in outs_of_subgraph: + if node_name not in self.graphIO.outs(): + out_node_name = node_name + '_crop_out' + self.ins[out_node_name] = Fluid_edger('_In', node_name) + self.outs[node_name] = Fluid_edger('_Out', out_node_name) + self.graphIO.add_out_fluid(out_node_name, node_name) + for node_name in ins_of_subgraph: + if node_name not in self.graphIO.ins(): + in_node_name = node_name + '_crop_in' + private_data = {'input_shape': [-1, -1, -1, -1]} + self.ins[node_name] = Fluid_edger('_In', in_node_name) + self.outs[in_node_name] = Fluid_edger('_Out', node_name) + self._AddProtoNode(in_node_name, None, helper, private_data, 'feed') + + def _IntegrateNodes(self, main_op, main_node_name, sec_node_name, helper, private_data): + # Merge secondary nodes to the primary node and process the edges. + self._RmProtoNode(main_node_name) + self._RmProtoNode(sec_node_name) + target_nodes_names = self.outs[sec_node_name]() + for target_node_name in target_nodes_names: + self.ins[target_node_name].mv(sec_node_name, main_node_name) + self.outs[main_node_name].mv(sec_node_name, target_node_name) + self.ins[target_node_name].rm(sec_node_name) + self.outs[sec_node_name].rm(target_node_name) + self.ins[sec_node_name].rm(main_node_name) + self.outs[main_node_name].rm(sec_node_name) + self._AddProtoNode(main_node_name, main_op, helper, private_data) + + def _DealWithBias(self, source_ops, helper): + # In fluid, the bias parameter of the conv2d is split into elementwise_add. + for source_op in source_ops: + if source_op.type in APPEND_BIAS_OP_TYPE: + private_data = {} + main_node_name = self._NameNodeMid(source_op) + if main_node_name in self.outs.keys(): + tmp_nodes_names = self.outs[main_node_name]() + if len(tmp_nodes_names) == 1 and \ + tmp_nodes_names[0].startswith('elementwise_add'): + elt_node_name = tmp_nodes_names[0] + elt_op = self._GetOp(source_ops, elt_node_name) + has_weights = helper.is_persistable_param(elt_op, 'Y') + if self._NameNodeMid(elt_op) == elt_node_name and has_weights: + [elt_tensor, shape] = helper.param_tensor_sh(elt_op, 'Y') + new_shape = [1, shape[3], 1, 1] + elt_tensor.set_shape(new_shape) + private_data['bias'] = elt_tensor + self._IntegrateNodes(source_op, main_node_name, \ + elt_node_name, helper, private_data) + + def _DealWithBatchnorm(self, source_ops, helper): + # In anakin, the scale part of batchnorm layer is independent. + for source_op in source_ops: + if source_op.type == 'batch_norm': + discrete_flag = True + main_node_name = self._NameNodeMid(source_op) + input_name = self.ins[main_node_name].target('X') + has_scale = helper.is_persistable_param(source_op, 'Scale') + if input_name.startswith('elementwise_add'): + elt_op = self._GetOp(source_ops, input_name) + x_of_elt = self.ins[input_name].target('X') + has_weights = helper.is_persistable_param(elt_op, 'Y') + if (x_of_elt.startswith('conv2d') or \ + x_of_elt.startswith('depthwise_conv2d')) and has_weights: + discrete_flag = False + elif input_name.startswith('conv2d') or input_name.startswith('depthwise_conv2d'): + discrete_flag = False + if discrete_flag is True: + self._RmProtoNode(main_node_name) + self._AddProtoNode(main_node_name, source_op, helper, {}, 'disc_bn') + elif has_scale is True: + append_node_name = main_node_name + '__scale' + tmp_all_targets_params = self.outs[main_node_name].targets_with_params() + self._InitEdges(append_node_name) + for [tmp_node_name, tmp_param_name] in tmp_all_targets_params: + self.outs[append_node_name].add(tmp_param_name, tmp_node_name) + self.ins[tmp_node_name].mv(main_node_name, append_node_name) + self.outs[main_node_name].rm(tmp_node_name) + self.ins[tmp_node_name].rm(main_node_name) + self.outs[main_node_name].add('_Scale_out', append_node_name) + self.ins[append_node_name].add('_Ins', main_node_name) + self._AddProtoNode(append_node_name, source_op, helper, {}, 'scale_of_bn') + + def _DealWithAxpy(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'elementwise_mul': + mul_node_name = self._NameNodeMid(source_op) + out_targets = self.outs[mul_node_name]() + if len(out_targets) == 1 and out_targets[0].startswith('elementwise_add'): + add_node_name = out_targets[0] + self._RmProtoNode(add_node_name) + a_node_name = self.ins[mul_node_name].target('Y') + x_node_name = self.ins[mul_node_name].target('X') + y_node_name = self.ins[add_node_name].target('X') + self._ClearEdges(mul_node_name) + self.ins[add_node_name].clear() + self.outs[a_node_name].mv(mul_node_name, add_node_name) + self.outs[x_node_name].mv(mul_node_name, add_node_name) + self.ins[add_node_name].add('A', a_node_name) + self.ins[add_node_name].add('X', x_node_name) + self.ins[add_node_name].add('Y', y_node_name) + self._RmProtoNode(mul_node_name) + self._AddProtoNode(add_node_name, None, helper, {}, 'axpy') + + def _DealWithPriorBox(self, source_ops, helper): + nodes_to_del = [] + for source_op in source_ops: + if source_op.type == 'prior_box': + pb_node_name = self._NameNodeMid(source_op) + br_node_name = self.outs[pb_node_name].target('Boxes') + vr_node_name = self.outs[pb_node_name].target('Variances') + bc_node_name = self.outs[br_node_name].target('Out') + vc_node_name = self.outs[vr_node_name].target('Out') + boxcoder_node_name = self.outs[bc_node_name].target('Out') + self.outs[pb_node_name].mv(br_node_name, bc_node_name) + self.outs[pb_node_name].rm(vr_node_name) + self.ins[bc_node_name].mv(br_node_name, pb_node_name) + self.ins[boxcoder_node_name].rm(vc_node_name) + for node_name in [br_node_name, vr_node_name, vc_node_name]: + if node_name not in nodes_to_del: + nodes_to_del.append(node_name) + input_node_name = self.ins[pb_node_name].target('Input') + image_node_name = self.ins[pb_node_name].target('Image') + self.ins[pb_node_name].rm(input_node_name) + self.ins[pb_node_name].rm(image_node_name) + self.ins[pb_node_name].add('Input', input_node_name) + self.ins[pb_node_name].add('Image', image_node_name) + self._RmProtoNode(bc_node_name) + self._AddProtoNode(bc_node_name, None, helper, {}, 'concat_btw_priorbox_boxcoder') + for node_name in nodes_to_del: + self._RmProtoNode(node_name) + self._ClearEdges(node_name) + + def _DealWithDetectionOutput(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'box_coder': + bc_node_name = self._NameNodeMid(source_op) + out_targets = self.outs[bc_node_name]() + if len(out_targets) == 1 and out_targets[0].startswith('multiclass_nms'): + private_data = {} + private_data['code_type'] = helper.attr_data(source_op, 'code_type') + bc_out_arg = helper.var_name_by_param(source_op, 'OutputBox') + for tmp_op in source_ops: + if tmp_op.idx != source_op.idx and bc_out_arg in tmp_op.input_arg_names: + nms_op = tmp_op + nms_node_name = out_targets[0] + loc_node_name = self.ins[bc_node_name].target('TargetBox') + conf_node_name = self.ins[nms_node_name].target('Scores') + box_node_name = self.ins[bc_node_name].target('PriorBox') + self._ClearEdges(bc_node_name) + self.ins[nms_node_name].clear() + self.outs[loc_node_name].mv(bc_node_name, nms_node_name) + self.outs[box_node_name].mv(bc_node_name, nms_node_name) + self.ins[nms_node_name].add('mbox_loc', loc_node_name) + self.ins[nms_node_name].add('mbox_conf_flatten', conf_node_name) + self.ins[nms_node_name].add('mbox_prior_box', box_node_name) + self._RmProtoNode(bc_node_name) + self._RmProtoNode(nms_node_name) + self._AddProtoNode(nms_node_name, nms_op, helper, \ + private_data, 'multiclass_nms') + + def _DealWithMultiFC(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'sum': + sum_node_name = self._NameNodeMid(source_op) + mul_names_list = self.ins[sum_node_name].targets('X') + elt_node_name = self.outs[sum_node_name].target('Out') + if elt_node_name.startswith('elementwise_add') and len(mul_names_list) > 1: + elt_op = self._GetOp(source_ops, elt_node_name) + elt_has_weights = helper.is_persistable_param(elt_op, 'Y') + fc_flag = True + for mul_node_name in mul_names_list: + if mul_node_name.startswith('mul') is False: + fc_flags = False + if fc_flag and elt_has_weights: + private_data = {} + first_mul_name = mul_names_list[0] + first_mul_op = self._GetOp(source_ops, first_mul_name) + in_of_mul_name = self.ins[first_mul_name].target('X') + out_of_elt_name = self.outs[elt_node_name].target('Out') + self.outs[sum_node_name].mv(elt_node_name, out_of_elt_name) + self.ins[out_of_elt_name].mv(elt_node_name, sum_node_name) + self._ClearEdges(elt_node_name) + [elt_tensor, shape] = helper.param_tensor_sh(elt_op, 'Y') + new_shape = [1, shape[3], 1, 1] + elt_tensor.set_shape(new_shape) + private_data['bias'] = elt_tensor + self._RmProtoNode(elt_node_name) + self._RmProtoNode(first_mul_name) + self._AddProtoNode(first_mul_name, first_mul_op, helper, private_data) + + def _DealWithGru(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'gru': + private_data = {} + gru_flags = [False, False] + gru_node_name = self._NameNodeMid(source_op) + gru_op = self._GetOp(source_ops, gru_node_name) + input_list_of_gru = self.ins[gru_node_name].targets('Input') + if len(input_list_of_gru) == 1 and \ + input_list_of_gru[0].startswith('elementwise_add'): + elt_node_name = input_list_of_gru[0] + elt_op = self._GetOp(source_ops, elt_node_name) + has_weights = helper.is_persistable_param(elt_op, 'Y') + if has_weights is True: + private_data['np_bias_x'] = helper.np_param(elt_op, 'Y') + gru_flags[0] = True + input_list_of_elt = self.ins[elt_node_name].targets('X') + if len(input_list_of_elt) == 1 and input_list_of_elt[0].startswith('mul'): + mul_node_name = input_list_of_elt[0] + mul_op = self._GetOp(source_ops, mul_node_name) + if helper.var_name_by_param(mul_op, 'Y').startswith('fc'): + if helper.attr_data(mul_op, 'x_num_col_dims') == 1: + input_list_of_mul = self.ins[mul_node_name].targets('X') + input_name_of_mul = input_list_of_mul[0] + private_data['np_weight_x'] = helper.np_param(mul_op, 'Y') + gru_flags[1] = True + else: + raise NameError('ERROR: Axis of GRU_FC must be 1.') + if gru_flags == [True, True]: + self.outs[input_name_of_mul].mv(mul_node_name, gru_node_name) + self.ins[gru_node_name].mv(elt_node_name, input_name_of_mul) + for node_to_del_name in [mul_node_name, elt_node_name, gru_node_name]: + self._RmProtoNode(node_to_del_name) + if node_to_del_name is not gru_node_name: + self._ClearEdges(node_to_del_name) + self._AddProtoNode(gru_node_name, gru_op, helper, private_data) + + def _SearchBilstm(self, source_ops, helper): + comp = Fluid_comparator(helper) + lstm_ops = [] + for source_op in source_ops: + if source_op.type == 'lstm': + lstm_ops.append(source_op) + if len(lstm_ops) == 2: + lstm_a = lstm_ops[0] + lstm_b = lstm_ops[1] + same_bias = comp.compare_by_param(lstm_a, lstm_b, 'Bias') + same_weight = comp.compare_by_param(lstm_a, lstm_b, 'Weight') + if same_bias and same_weight: + return True + else: + return False + else: + return False + + def _DealWithLstm(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'lstm': + private_data = {} + lstm_flags = [False, False] + lstm_node_name = self._NameNodeMid(source_op) + lstm_op = self._GetOp(source_ops, lstm_node_name) + input_list_of_lstm = self.ins[lstm_node_name].targets('Input') + input_list = [] + if len(input_list_of_lstm) == 1: + in_lstm_node_name = input_list_of_lstm[0] + if input_list_of_lstm[0].split('#')[0] == 'elementwise_add': + elt_op = self._GetOp(source_ops, in_lstm_node_name) + has_weights = helper.is_persistable_param(elt_op, 'Y') + if has_weights is True: + private_data['np_flat_fc_bias'] = helper.np_param(elt_op, 'Y') + lstm_flags[0] = True + input_list = self.ins[in_lstm_node_name].targets('X') + elif input_list_of_lstm[0].split('#')[0] == 'mul': + private_data['np_flat_fc_bias'] = None + input_list = input_list_of_lstm + lstm_flags[0] = True + if lstm_flags[0] is True and len(input_list) == 1: + if input_list[0].split('#')[0] == 'mul': + mul_node_name = input_list[0] + mul_op = self._GetOp(source_ops, mul_node_name) + #if helper.var_name_by_param(mul_op, 'Y').startswith('fc'): + if helper.attr_data(mul_op, 'x_num_col_dims') == 1: + input_list_of_mul = self.ins[mul_node_name].targets('X') + input_name_of_mul = input_list_of_mul[0] + [w_np, w_sh] = helper.data_with_shape_by_param(mul_op, 'Y', \ + False, None, 0, False) + private_data['np_flat_fc_weight'] = w_np + private_data['np_fc_outdim'] = w_sh[3] + lstm_flags[1] = True + else: + raise NameError('ERROR: Axis of LSTM_FC must be 1.') + if lstm_flags == [True, True]: + self.outs[input_name_of_mul].mv(mul_node_name, lstm_node_name) + self.ins[lstm_node_name].mv(in_lstm_node_name, input_name_of_mul) + if in_lstm_node_name == mul_node_name: + nodes_to_del = [mul_node_name, lstm_node_name] + else: + nodes_to_del = [mul_node_name, in_lstm_node_name, lstm_node_name] + for node_to_del_name in nodes_to_del: + self._RmProtoNode(node_to_del_name) + if node_to_del_name is not lstm_node_name: + self._ClearEdges(node_to_del_name) + self._AddProtoNode(lstm_node_name, lstm_op, helper, private_data) + + def _DealWithCast(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'cast': + if helper.attr_data(source_op, 'out_dtype') == 5: + cast_node_name = self._NameNodeMid(source_op) + if cast_node_name in self.ins: + input_name_of_cast = self.ins[cast_node_name].target('X') + if input_name_of_cast.startswith('top_k') is False: + output_name_of_cast = self.outs[cast_node_name].target('Out') + self.outs[input_name_of_cast].mv(cast_node_name, output_name_of_cast) + self.ins[output_name_of_cast].mv(cast_node_name, input_name_of_cast) + self._RmProtoNode(cast_node_name) + self._ClearEdges(cast_node_name) + else: + print 'Cannot find the layer corresponding to cast.' + else: + raise NameError('The out type of cast must be float32.') + + def _DealWithArgmax(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'top_k': + private_data = {} + topk_node_name = self._NameNodeMid(source_op) + out_list = self.outs[topk_node_name].targets('Out') + index_list = self.outs[topk_node_name].targets('Indices') + if len(index_list) > 0: + if len(out_list) == 1 and index_list[0].startswith('cast'): + private_data['out_max_val'] = True + idxcast_node_name = index_list[0] + output_name_of_idxcast = self.outs[idxcast_node_name].target('Out') + if output_name_of_idxcast == out_list[0] and \ + out_list[0].startswith('concat'): + concat_node_name = out_list[0] + output_name_of_concat = self.outs[concat_node_name].target('Out') + self.outs[topk_node_name].rm(idxcast_node_name) + self.outs[topk_node_name].mv(concat_node_name, output_name_of_concat) + self.ins[output_name_of_concat].mv(concat_node_name, topk_node_name) + for node_to_del_name in [concat_node_name, idxcast_node_name]: + self._RmProtoNode(node_to_del_name) + self._ClearEdges(node_to_del_name) + elif output_name_of_idxcast != out_list[0]: + if output_name_of_idxcast.endswith('_gout') and \ + out_list[0].endswith('_gout'): + gout_node_name = out_list[0] + idx_gout_node_name = output_name_of_idxcast + self.outs[topk_node_name].rm(idxcast_node_name) + for node_to_del_name in [idx_gout_node_name, idxcast_node_name]: + self._RmProtoNode(node_to_del_name) + self._ClearEdges(node_to_del_name) + self.graphIO.rm_out(idx_gout_node_name) + elif len(out_list) == 0: + private_data['out_max_val'] = False + self._DealWithCast(source_ops, helper) + else: + raise NameError('ERROR: Unknown top_k layer.') + self._RmProtoNode(topk_node_name) + self._AddProtoNode(topk_node_name, source_op, helper, private_data) + + def _RefreshReshape(self, source_ops, helper, need_assign=False): + for source_op in source_ops: + if source_op.type == 'reshape': + reshape_node_name = self._NameNodeMid(source_op) + # Make sure this node exists in this graph. + if reshape_node_name in self.ins: + shape_inputs = self.ins[reshape_node_name].targets('Shape') + tensor_inputs = self.ins[reshape_node_name].targets('X') + if len(shape_inputs) == 1 and len(tensor_inputs) == 1: + self.ins[reshape_node_name].rm(shape_inputs[0]) + if shape_inputs[0].split('#')[0] != 'assign_value' \ + or need_assign is True: + self.ins[reshape_node_name].add('Shape', shape_inputs[0]) + else: + self._RmProtoNode(shape_inputs[0]) + self._ClearEdges(shape_inputs[0]) + + def _CutReshape(self, reshape_node_name): + branch = [] + branch.append(reshape_node_name) + shape_inputs = self.ins[reshape_node_name].targets('Shape') + tensor_input = self.ins[reshape_node_name].target('X') + tensor_output = self.outs[reshape_node_name].target('Out') + if len(shape_inputs) == 1: + branch.append(shape_inputs[0]) + if len(branch) == 2 and branch[1].split('#')[0] == 'split': + split_node_name = branch[1] + self.outs[split_node_name].rm(reshape_node_name) + self.ins[reshape_node_name].rm(split_node_name) + if len(self.outs[split_node_name].targets('_Out')) == 0: + input_of_split = self.ins[split_node_name].target('_In') + branch.append(input_of_split) + self._RmProtoNode(split_node_name) + self._ClearEdges(split_node_name) + elif len(branch) == 2 and branch[1].split('#')[0] == 'shape': + shape_node_name = branch[1] + input_of_shape = self.ins[shape_node_name].targets('Input') + assert len(input_of_shape) == 1 + self.outs[input_of_shape[0]].rm(shape_node_name) + self.ins[reshape_node_name].rm(shape_node_name) + self._RmProtoNode(shape_node_name) + self._ClearEdges(shape_node_name) + elif len(branch) == 2 and branch[1].split('#')[0] == 'assign_value': + assign_node_name = branch[1] + self.ins[reshape_node_name].rm(assign_node_name) + self._RmProtoNode(assign_node_name) + self._ClearEdges(assign_node_name) + elif len(branch) == 2 and branch[1].startswith('input'): + raise NameError('ERROR: None-split input of Softmax has not supported.') + else: + pass + self.outs[tensor_input].mv(reshape_node_name, tensor_output) + self.ins[tensor_output].mv(reshape_node_name, tensor_input) + self._RmProtoNode(reshape_node_name) + self._ClearEdges(reshape_node_name) + if len(branch) == 3 and branch[2].startswith('input'): + input_node_name = branch[2] + self._RmProtoNode(input_node_name) + self._ClearEdges(input_node_name) + + def _RefreshSplit(self, split_node_name, helper): + outputs_of_split = self.outs[split_node_name].targets('_Out') + inputs_of_split = self.ins[split_node_name].targets('_In') + assert len(inputs_of_split) == 1 + split_num = len(outputs_of_split) + if split_num == 1: + self.ins[outputs_of_split[0]].mv(split_node_name, inputs_of_split[0]) + self.outs[inputs_of_split[0]].mv(split_node_name, outputs_of_split[0]) + self._RmProtoNode(split_node_name) + self._ClearEdges(split_node_name) + else: + private_data = {'split_num': split_num} + self._RmProtoNode(split_node_name) + self._AddProtoNode(split_node_name, None, helper, private_data, 'split') + + def _DealWithSoftmax(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'softmax': + softmax_node_name = self._NameNodeMid(source_op) + outs_of_softmax = self.outs[softmax_node_name].targets('Out') + ins_of_softmax = self.ins[softmax_node_name].targets('X') + if outs_of_softmax[0].split('#')[0] == 'reshape': + if ins_of_softmax[0].split('#')[0] == 'reshape' or \ + ins_of_softmax[0].split('#')[0] == 'flatten': + private_data = {} + private_data['axis'] = 3 + self._CutReshape(outs_of_softmax[0]) + self._CutReshape(ins_of_softmax[0]) + self._RmProtoNode(softmax_node_name) + self._AddProtoNode(softmax_node_name, source_op, helper, private_data) + ins_of_softmax = self.ins[softmax_node_name].targets('X') + print 'ins_of_softmax', ins_of_softmax + assert len(ins_of_softmax) == 1 + self._RefreshSplit(ins_of_softmax[0], helper) + + def _DealWithMatmal(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'matmul': + matmul_node_name = self._NameNodeMid(source_op) + x_input_name = self.ins[matmul_node_name].target('X') + y_input_name = self.ins[matmul_node_name].target('Y') + flag = False + coeff = 1.0 + for node_name in [x_input_name, y_input_name]: + if node_name.startswith('scale') or node_name.startswith('dropout'): + op = self._GetOp(source_ops, node_name) + if node_name.startswith('scale'): + scale = helper.attr_data(op, 'scale') + elif node_name.startswith('dropout'): + scale = 1 - helper.attr_data(op, 'dropout_prob') + input_node = self.ins[node_name].target('X') + self.outs[input_node].mv(node_name, matmul_node_name) + self.ins[matmul_node_name].mv(node_name, input_node) + self._RmProtoNode(node_name) + self._ClearEdges(node_name) + coeff = coeff * scale + flag = True + if flag is True: + private_data = {} + private_data['coeff'] = coeff + self._RmProtoNode(matmul_node_name) + self._AddProtoNode(matmul_node_name, source_op, helper, private_data) + + def _DealWithDiscBatchNorm(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'batch_norm': + discrete_flag = True + bn_node_name = self._NameNodeMid(source_op) + input_name = self.ins[bn_node_name].target('X') + if input_name.startswith('elementwise_add'): + input_of_elt = self.ins[input_name].target('X') + if input_of_elt.startswith('conv2d'): + discrete_flag = False + elif input_name.startswith('conv2d'): + discrete_flag = False + if discrete_flag is True: + self._RmProtoNode(bn_node_name) + self._AddProtoNode(bn_node_name, source_op, helper, {}, 'disc_bn') + + def _DealWithSSD(self, source_ops, helper): + for source_op in source_ops: + if source_op.type == 'reshape': + rh_node_name = self._NameNodeMid(source_op) + if rh_node_name in self.ins: + private_data = dict() + input_name = self.ins[rh_node_name].target('X') + shape = helper.attr_data(source_op, 'shape') + if input_name.startswith('concat'): + private_data['new_shape'] = [0, shape[0], shape[1], 0] + else: + private_data['new_shape'] = [0, -1, 1, 1] + self._RmProtoNode(rh_node_name) + self._AddProtoNode(rh_node_name, source_op, helper, private_data, 'reshape') + for source_op in source_ops: + if source_op.type == 'softmax': + private_data = dict() + sm_node_name = self._NameNodeMid(source_op) + private_data['axis'] = 2 + self._RmProtoNode(sm_node_name) + self._AddProtoNode(sm_node_name, source_op, helper, private_data, 'softmax') + + def _NewCommonLayer(self, + source_ops, + in_target, + in_param, + out_target, + out_param, + layer_type, + private_data, + helper, + insert_mode=True): + main_layer = layer_type + '_after_' + in_target + if insert_mode is True: + if in_target in self.ins[out_target].all_targets() and \ + out_target in self.outs[in_target].all_targets(): + self.ins[out_target].mv(in_target, main_layer) + self.outs[in_target].mv(out_target, main_layer) + else: + raise NameError('ERROR: Usage of InsertCommonLayer has not supported.') + else: + self.ins[out_target].add(in_param + '_insert', main_layer) + self.outs[in_target].add(out_param + '_insert', main_layer) + self.ins[main_layer] = Fluid_edger(in_param, in_target) + self.outs[main_layer] = Fluid_edger(out_param, out_target) + self._AddProtoNode(main_layer, None, helper, private_data, layer_type) + + def _ParseNetwork(self, source_ops, helper): + self._ParseBase(source_ops, helper) + if self.NetType == "FLUIDBASE": + pass + else: + reshape_dict = {} + if self.NetType == "OCR": + reshape_dict['input_0'] = [1, 1, 48, 1500] + self._ReplaceInputs(source_ops, helper, reshape_dict) + self._InsertSplit(source_ops, helper) + self._DealWithGru(source_ops, helper) + self._DealWithLstm(source_ops, helper) + self._DealWithBias(source_ops, helper) + self._DealWithBatchnorm(source_ops, helper) + self._DealWithMultiFC(source_ops, helper) + self._DealWithArgmax(source_ops, helper) + self._DealWithAxpy(source_ops, helper) + if self.NetType == "SSD": + self._DealWithPriorBox(source_ops, helper) + self._DealWithDetectionOutput(source_ops, helper) + self._DealWithSSD(source_ops, helper) + self._DealWithSoftmax(source_ops, helper) + self._RefreshReshape(source_ops, helper) + self._Graph() + + def _Parsing(self): + with fluid.scope_guard(self.scope): + if os.path.exists(self.ModelPath + 'model') and \ + os.path.exists(self.ModelPath + 'params'): + [self.net_program, feed_target_names, fetch_targets] = \ + fluid.io.load_inference_model(self.ModelPath, self.exe, 'model', 'params') + else: + [self.net_program, feed_target_names, fetch_targets] = \ + fluid.io.load_inference_model(self.ModelPath, self.exe) + + global_block = self.net_program.global_block() + source_ops = list(global_block.ops) + helper = Fluid_helper(self.scope, global_block) + + self._ParseNetwork(source_ops, helper) + return self.graphIO diff --git a/tools/external_converter_v2/parser/fluid/tools/feed_ones.py b/tools/external_converter_v2/parser/fluid/tools/feed_ones.py new file mode 100644 index 000000000..06238f598 --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/tools/feed_ones.py @@ -0,0 +1,138 @@ +#! /usr/bin/env python +# -*- coding: utf-8 -*- + +''' +A separate Fluid test file for feeding specific data. +''' + +import sys +import numpy as np +import os +import paddle.fluid as fluid +from paddle.fluid import debugger +from paddle.fluid import core +import subprocess + +GLB_model_path = '' +GLB_arg_name = '' +GLB_batch_size = 1 + +def load_inference_model(model_path, exe): + ''' + ''' + model_abs_path = os.path.join(model_path, 'model') + param_abs_path = os.path.join(model_path, 'params') + if os.path.exists(model_abs_path) and os.path.exists(param_abs_path): + return fluid.io.load_inference_model(model_path, exe, 'model', 'params') + else: + return fluid.io.load_inference_model(model_path, exe) + +def feed_ones(block, feed_target_names, batch_size=1): + """ + """ + feed_dict = dict() + def set_batch_size(shape, batch_size): + if shape[0] == -1: + shape[0] = batch_size + return shape + def fill_ones(var_name, batch_size): + var = block.var(var_name) + np_shape = set_batch_size(list(var.shape), 1) + var_np = { + core.VarDesc.VarType.BOOL: np.bool_, + core.VarDesc.VarType.INT32: np.int32, + core.VarDesc.VarType.INT64: np.int64, + core.VarDesc.VarType.FP16: np.float16, + core.VarDesc.VarType.FP32: np.float32, + core.VarDesc.VarType.FP64: np.float64, + } + np_dtype = var_np[var.dtype] + return np.ones(np_shape, dtype=np_dtype) + for feed_target_name in feed_target_names: + feed_dict[feed_target_name] = fill_ones(feed_target_name, batch_size) + return feed_dict + +def draw(block, filename='debug'): + """ + """ + dot_path = './' + filename + '.dot' + pdf_path = './' + filename + '.pdf' + debugger.draw_block_graphviz(block, path=dot_path) + cmd = ["dot", "-Tpdf", dot_path, "-o", pdf_path] + subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + +def fetch_tmp_vars(block, fetch_targets, var_names_list=None): + """ + """ + def var_names_of_fetch(fetch_targets): + var_names_list = [] + for var in fetch_targets: + var_names_list.append(var.name) + return var_names_list + + fetch_var = block.var('fetch') + old_fetch_names = var_names_of_fetch(fetch_targets) + new_fetch_vars = [] + for var_name in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + i = len(new_fetch_vars) + if var_names_list is None: + var_names_list = block.vars.keys() + for var_name in var_names_list: + if '.tmp_' in var_name and var_name not in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + block.append_op( + type='fetch', + inputs={'X': [var_name]}, + outputs={'Out': [fetch_var]}, + attrs={'col': i}) + i = i + 1 + return new_fetch_vars + +def print_results(results, fetch_targets, need_save=True): + """ + """ + for result in results: + idx = results.index(result) + print fetch_targets[idx] + print np.array(result) + if need_save is True: + fluid_fetch_list = list(np.array(result).flatten()) + fetch_txt_fp = open('result_' + fetch_targets[idx].name + '.txt', 'w') + for num in fluid_fetch_list: + fetch_txt_fp.write(str(num) + '\n') + fetch_txt_fp.close() + +def fluid_inference_test(model_path): + """ + """ + place = fluid.CPUPlace() + exe = fluid.Executor(place) + scope = fluid.core.Scope() + with fluid.scope_guard(scope): + [net_program, + feed_target_names, + fetch_targets] = load_inference_model(model_path, exe) + global_block = net_program.global_block() + draw(global_block) + feed_list = feed_ones(global_block, feed_target_names) + fetch_targets = fetch_tmp_vars(global_block, fetch_targets, [GLB_arg_name]) + results = exe.run(program=net_program, + feed=feed_list, + fetch_list=fetch_targets, + return_numpy=False) + print_results(results, fetch_targets) + +if __name__ == "__main__": + if len(sys.argv) == 1: + raise NameError('Usage: python ./all_ones.py path/to/model arg_name batch_size') + if len(sys.argv) > 1: + GLB_model_path = sys.argv[1] + if len(sys.argv) > 2: + GLB_arg_name = sys.argv[2] + if len(sys.argv) > 3: + GLB_batch_size = sys.argv[3] + fluid_inference_test(GLB_model_path) + diff --git a/tools/external_converter_v2/parser/fluid/tools/feed_txt.py b/tools/external_converter_v2/parser/fluid/tools/feed_txt.py new file mode 100644 index 000000000..03ef81104 --- /dev/null +++ b/tools/external_converter_v2/parser/fluid/tools/feed_txt.py @@ -0,0 +1,182 @@ +#! /usr/bin/env python +# -*- coding: utf-8 -*- + +""" +A separate Fluid test file for feeding specific data. +""" + +import sys +import numpy as np +import os +import paddle.fluid as fluid +from paddle.fluid import debugger +from paddle.fluid import core +from prettytable import PrettyTable +from operator import mul +import subprocess + +GLB_model_path = '/path/to/fluid/inference/model/' +GLB_feed_example = { + 'var_name': 'data', + 'tensor_shape': [n, c, h, w], + 'txt_path': '/path/to/input/txt/', +} +GLB_feed_list = [GLB_feed_example] + +# Do not modify +GLB_arg_name = '' +GLB_batch_size = 1 + +def load_inference_model(model_path, exe): + """ + """ + model_abs_path = os.path.join(model_path, 'model') + param_abs_path = os.path.join(model_path, 'params') + if os.path.exists(model_abs_path) and os.path.exists(param_abs_path): + return fluid.io.load_inference_model(model_path, exe, 'model', 'params') + else: + return fluid.io.load_inference_model(model_path, exe) + +def print_feed_fetch(block, feed_target_names, fetch_targets): + """ + """ + tag_list = ["Index", "Name", "Shape", "Data Type", "Tensor Type"] + def add_var_table(table, var, idx): + table.add_row([idx, var.name, var.shape, str(var.dtype), str(var.type)]) + def feed_table(block, var_name_list): + table = PrettyTable(tag_list) + for var_name in var_name_list: + idx = var_name_list.index(var_name) + var = block.var(var_name) + add_var_table(table, var, idx) + return table + def fetch_table(var_list): + idx = 0 + table = PrettyTable(tag_list) + for var in var_list: + add_var_table(table, var, idx) + idx = idx + 1 + return table + print "\n", "=========== FEED TABLE ===========" + print feed_table(block, feed_target_names) + print "\n", "=========== FETCH TABLE ===========" + print fetch_table(fetch_targets), "\n" + +def add_feed_list(feed_list, fluid_feed_dict=None): + """ + """ + if fluid_feed_dict is None: + fluid_feed_dict = dict() + def numpy_from_txt(txt_path, + tensor_shape=list(), + dtype=np.float32, + delimiter=None, + comments='#'): + data = np.loadtxt(txt_path, dtype, comments, delimiter) + data_size = np.size(data) + tensor_size = reduce(mul, tensor_shape) + assert (data_size == tensor_size), \ + "data size of txt (%d) must be equal to shape size (%d)." % (data_size, tensor_size) + return np.reshape(data, tensor_shape) + + def add_feed_var(input_dict, fluid_feed_dict): + var_name = input_dict['var_name'] + tensor_shape = input_dict['tensor_shape'] + txt_path = input_dict['txt_path'] + if 'data_type' in input_dict.keys(): + dtype = input_dict['data_type'] + else: + dtype = np.float32 + if 'delimiter' in input_dict.keys(): + delim = input_dict['delimiter'] + else: + delim = None + fluid_feed_dict[var_name] = numpy_from_txt(txt_path, tensor_shape, dtype, delim) + return fluid_feed_dict + for input_dict in feed_list: + add_feed_var(input_dict, fluid_feed_dict) + return fluid_feed_dict + +def draw(block, filename='debug'): + """ + """ + dot_path = './' + filename + '.dot' + pdf_path = './' + filename + '.pdf' + debugger.draw_block_graphviz(block, path=dot_path) + cmd = ["dot", "-Tpdf", dot_path, "-o", pdf_path] + subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + +def fetch_tmp_vars(block, fetch_targets, var_names_list=None): + """ + """ + def var_names_of_fetch(fetch_targets): + var_names_list = [] + for var in fetch_targets: + var_names_list.append(var.name) + return var_names_list + fetch_var = block.var('fetch') + old_fetch_names = var_names_of_fetch(fetch_targets) + new_fetch_vars = [] + for var_name in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + i = len(new_fetch_vars) + if var_names_list is None: + var_names_list = block.vars.keys() + for var_name in var_names_list: + if '.tmp_' in var_name and var_name not in old_fetch_names: + var = block.var(var_name) + new_fetch_vars.append(var) + block.append_op( + type='fetch', + inputs={'X': [var_name]}, + outputs={'Out': [fetch_var]}, + attrs={'col': i}) + i = i + 1 + return new_fetch_vars + +def print_results(results, fetch_targets, need_save=True): + """ + """ + for result in results: + idx = results.index(result) + print fetch_targets[idx] + print np.array(result) + if need_save is True: + fluid_fetch_list = list(np.array(result).flatten()) + fetch_txt_fp = open('result_' + fetch_targets[idx].name + '.txt', 'w') + for num in fluid_fetch_list: + fetch_txt_fp.write(str(num) + '\n') + fetch_txt_fp.close() + +def fluid_inference_test(model_path, feed_list): + """ + """ + place = fluid.CPUPlace() + exe = fluid.Executor(place) + scope = fluid.core.Scope() + with fluid.scope_guard(scope): + [net_program, + feed_target_names, + fetch_targets] = load_inference_model(model_path, exe) + global_block = net_program.global_block() + print_feed_fetch(global_block, feed_target_names, fetch_targets) + draw(global_block) + feed_list = add_feed_list(feed_list) + fetch_targets = fetch_tmp_vars(global_block, fetch_targets, [GLB_arg_name]) + results = exe.run(program=net_program, + feed=feed_list, + fetch_list=fetch_targets, + return_numpy=False) + print_results(results, fetch_targets) + +if __name__ == "__main__": + if len(sys.argv) == 1 and GLB_model_path == '': + raise NameError('Usage: python ./all_ones.py path/to/model arg_name batch_size') + if len(sys.argv) > 1 and GLB_model_path == '': + GLB_model_path = sys.argv[1] + if len(sys.argv) > 2: + GLB_arg_name = sys.argv[2] + if len(sys.argv) > 3: + GLB_batch_size = sys.argv[3] + fluid_inference_test(GLB_model_path, GLB_feed_list) diff --git a/tools/external_converter_v2/parser/graph.py b/tools/external_converter_v2/parser/graph.py index 030817195..9d94a6900 100644 --- a/tools/external_converter_v2/parser/graph.py +++ b/tools/external_converter_v2/parser/graph.py @@ -20,19 +20,19 @@ def __init__(self, config): """ self.save_file_path = config.SavePath + config.ResultName + ".anakin.bin" if config.framework == 'CAFFE': - from kill_caffe import CaffeParser + from caffe import CaffeParser self.parser = CaffeParser(config.framework_config_dict) elif config.framework == 'PADDLE': pass elif config.framework == 'LEGO': pass elif config.framework == 'TENSORFLOW': - from kill_tf import TFParser + from tensorflow import TFParser self.parser=TFParser(config.framework_config_dict) elif config.framework == 'MXNET': pass elif config.framework == 'FLUID': - from kill_fluid import FluidParser + from fluid import FluidParser self.parser = FluidParser(config.framework_config_dict) else: raise NameError('ERROR: GrapProtoIO not support %s model.' % (config.framework)) @@ -105,11 +105,17 @@ def serialization(self): self.graph_io.serialization(self.save_file_path) def info_table(self): + """ + print input table. + """ tables = list() in_table = PrettyTable(["Input Name", "Shape", "Alias", "Data Type"]) out_table = PrettyTable(["Output Name"]) def ins_attr(): + """ + get inputs attr. + """ ins = list() for graph_in in self.ins(): attr = dict() diff --git a/tools/external_converter_v2/parser/graph_io.py b/tools/external_converter_v2/parser/graph_io.py index a471afa39..ba9acc70c 100644 --- a/tools/external_converter_v2/parser/graph_io.py +++ b/tools/external_converter_v2/parser/graph_io.py @@ -336,26 +336,25 @@ def rm_out(self, node_name): self.graph_proto.outs[:] = graph_outs def format_edge_from_nodes(self): - in_set=set() + in_set = set() out_set = set() for node in self.graph_proto.nodes: - name=node.name + name = node.name for node_name in node.ins: - self.add_in_edge(node_name,name) - in_set.add((node_name,name)) + self.add_in_edge(node_name, name) + in_set.add((node_name, name)) for node_name in node.outs: - self.add_out_edge(name,node_name) - out_set.add((name,node_name)) - ab_set=in_set-out_set - ba_set=out_set-in_set + self.add_out_edge(name, node_name) + out_set.add((name, node_name)) + ab_set = in_set - out_set + ba_set = out_set - in_set print(ab_set) print('------') print(ba_set) - assert len(ab_set)==0 and len(ba_set)==0,'in edge must equal with out edge' + assert len(ab_set) == 0 and len(ba_set) == 0, 'in edge must equal with out edge' def __call__(self): - return self.graph_proto diff --git a/tools/external_converter_v2/parser/kill_fluid/fluid_debugger.py b/tools/external_converter_v2/parser/kill_fluid/fluid_debugger.py deleted file mode 100644 index d07e97799..000000000 --- a/tools/external_converter_v2/parser/kill_fluid/fluid_debugger.py +++ /dev/null @@ -1,60 +0,0 @@ -from ..proto import * -from ..graph_io import * -import copy -import paddle.fluid as fluid -import numpy as np -from paddle.fluid.core import VarDesc, AttrType - - -class Fluid_debugger: - - def var_names_of_fetch(self, fetch_targets): - var_names_list = [] - for var in fetch_targets: - var_names_list.append(var.name) - return var_names_list - - def fetch_tmp_vars(self, block, fetch_targets, var_names_list = None): - fetch_var = block.var('fetch') - old_fetch_names = self.var_names_of_fetch(fetch_targets) - new_fetch_vars = [] - for var_name in old_fetch_names: - var = block.var(var_name) - new_fetch_vars.append(var) - i = len(new_fetch_vars) - if var_names_list is None: - var_names_list = block.vars.keys() - for var_name in var_names_list: - if '.tmp_' in var_name and var_name not in old_fetch_names: - var = block.var(var_name) - new_fetch_vars.append(var) - block.append_op( - type='fetch', - inputs={'X': [var_name]}, - outputs={'Out': [fetch_var]}, - attrs={'col': i}) - i = i + 1 - return new_fetch_vars - - def print_tmp_vars(self, block, var_names_list = []): - for var_name in var_names_list: - var_to_print = block.var(var_name) - out_to_print = block.create_var( - name=var_name+'.print', - dtype="float32", - persistable=True, - stop_gradient=False) - block.append_op( - type='print', - inputs={'In': var_to_print}, - attrs={ - 'first_n': -1, - 'summarize': -1, - 'message': "", - 'print_tensor_name': True, - 'print_tensor_type': True, - 'print_tensor_shape': True, - 'print_tensor_lod': True, - 'print_phase': 'FORWARD' - }, - outputs={'Out': out_to_print}) diff --git a/tools/external_converter_v2/parser/kill_fluid/fluid_helper.py b/tools/external_converter_v2/parser/kill_fluid/fluid_helper.py deleted file mode 100644 index f702a0679..000000000 --- a/tools/external_converter_v2/parser/kill_fluid/fluid_helper.py +++ /dev/null @@ -1,516 +0,0 @@ -from ..proto import * -from ..graph_io import * -import paddle.fluid as fluid -import numpy as np -from paddle.fluid.core import VarDesc, AttrType - - -def union(list_a, list_b): - return list(set(list_a).union(set(list_b))) - -def difference(list_a, list_b): - return list(set(list_a).difference(set(list_b))) - - -class Edge_for_fluid: - - def __init__(self, param, target, var): - self.param = param - self.target = target - self.var = var - - -class Fluid_edger: - - def __init__(self, param = None, target = None, var = None): - self.edges = [] - if param is not None and target is not None: - edge = Edge_for_fluid(param, target, var) - self.edges.append(edge) - - def __call__(self): - return self.all_targets() - - def add(self, param, target, var = None): - edge = Edge_for_fluid(param, target, var) - self.edges.append(edge) - - def rm_edges_by_param(self, param): - for edge in self.edges: - if edge.param == param: - edge_idx = self.edges.index(edge) - del self.edges[edge_idx] - - def rm(self, target): - res = -1 - for edge in self.edges: - if target == edge.target: - edge_idx = self.edges.index(edge) - del self.edges[edge_idx] - res = res + 1 - if res != 0: - pass - - def mv(self, old_target, new_target): - res = -1 - for edge in self.edges: - if old_target == edge.target: - edge.target = new_target - res = res + 1 - if res != 0: - pass - - def all_params(self): - params = [] - for edge in self.edges: - if edge.param not in params: - params.append(edge.param) - return params - - def all_targets(self): - targets = [] - for edge in self.edges: - targets.append(edge.target) - return targets - - def targets(self, param): - targets = [] - for edge in self.edges: - if edge.param == param: - targets.append(edge.target) - return targets - - def target(self, param, idx = 0): - return self.targets(param)[idx] - - def clear(self): - targets_list = self.all_targets() - for target in targets_list: - self.rm(target) - - def targets_with_params(self): - list_of_targets_and_params = [] - for edge in self.edges: - target_and_param = [edge.target, edge.param] - list_of_targets_and_params.append(target_and_param) - return list_of_targets_and_params - - def vars_by_target(self, target): - vars = [] - for edge in self.edges: - if edge.target == target and edge.var is not None: - vars.append(edge.var) - return vars - - def __getitem__(self, idx): - if idx < len(self.edges): - return self.edges[idx] - return None - - -class Fluid_helper: - - def __init__(self, scope, block): - self.scope = scope - self.block = block - - def args_by_input_param(self, op, param_name): - if param_name in op.input_names: - return op.input(param_name) - else: - raise NameError('ERROR: param_name %s is not exists.' % ( param_name ) ) - - def args_by_output_param(self, op, param_name): - if param_name in op.output_names: - return op.output(param_name) - else: - raise NameError('ERROR: param_name %s is not exists.' % ( param_name ) ) - - def var_by_input_param(self, op, param_name, var_idx = 0): - var_name = self.args_by_input_param(op, param_name)[var_idx] - var = self.block.var(var_name) - return var - - def var_by_output_param(self, op, param_name, var_idx = 0): - var_name = self.args_by_output_param(op, param_name)[var_idx] - var = self.block.var(var_name) - return var - - def var_name_by_param(self, op, param_name, var_idx = 0): - if param_name not in op.input_names + op.output_names: - raise NameError('ERROR: param_name %s is not exists.' % ( param_name ) ) - elif param_name in op.input_names: - if len(op.input(param_name)) > 0: - var_name_unicode = op.input(param_name)[var_idx] - else: - raise NameError('ERROR: param %s has not var.' % ( param_name ) ) - elif param_name in op.output_names: - if len(op.output(param_name)) > 0: - var_name_unicode = op.output(param_name)[var_idx] - else: - raise NameError('ERROR: param %s has not var.' % ( param_name ) ) - var = self.block.var(var_name_unicode) - var_name = var.name - return var_name - - def var_by_param(self, op, param_name, var_idx = 0): - var_name = self.var_name_by_param(op, param_name, var_idx) - var = self.block.var(var_name) - return var - - def shape_by_var_name(self, var_name, layout = 'NCHW'): - var = self.block.var(var_name) - long_tuple = var.shape - long_list = list(long_tuple) - if layout == 'NCHW': - int_list_4d = map(int, [1]*(4-len(long_list)) + long_list) - return int_list_4d - elif layout == 'UNMODIFIED': - return long_list - else: - raise NameError('ERROR: layout %s is not implemented yet.' % ( layout ) ) - - def np_data_by_var_name(self, var_name): - numpy_array = fluid.executor.fetch_var(var_name, self.scope, True) - return numpy_array - - def dtype_by_var_name(self, var_name): - var = self.block.var(var_name) - fluid_var_type = var.dtype - dtype = ANAKIN_TENSOR_DTYPE[fluid_var_type] - return dtype - - def is_persistable_param(self, op, param_name, var_idx = 0): - var = self.var_by_param(op, param_name, var_idx) - is_persistable_var = var.persistable - return is_persistable_var - - def var_shape_by_param(self, transpose, op, param_name, var_idx = 0, layout = 'NCHW'): - if transpose is True: - raise NameError('ERROR: var_shape transpose is not implemented yet.') - else: - var_name = self.var_name_by_param(op, param_name, var_idx) - shape = self.shape_by_var_name(var_name, layout) - return shape - - def data_with_shape_by_param(self, - op, - param_name, - transpose = False, - axes = None, - var_idx = 0, - is_flat_list = True, - layout = 'NCHW'): - - np.set_printoptions(threshold=np.inf, suppress=True) - - var_name = self.var_name_by_param(op, param_name, var_idx) - np_array = self.np_data_by_var_name(var_name) - if transpose is True: - np_array = np.transpose(np_array, axes) - np_shape = np.shape(np_array) - if layout == 'NCHW': - np_shape = map(int, [1]*(4-len(np_shape)) + list(np_shape)) - if is_flat_list is True: - flat_list = list(np_array.flatten()) - return [flat_list, np_shape] - else: - return [np_array, np_shape] - - def np_param(self, - op, - param_name, - transpose = False, - axes = None, - var_idx = 0): - - [data, np_shape] = self.data_with_shape_by_param(op, param_name, transpose, \ - axes, var_idx, False) - return data - - def dtype_by_param(self, op, param_name, var_idx = 0): - var_name = self.var_name_by_param(op, param_name, var_idx) - dtype = self.dtype_by_var_name(var_name) - return dtype - - def is_list_type(self, op, attr_name): - if op.has_attr(attr_name): - fluid_attr_type = op.attr_type(attr_name) - if fluid_attr_type in ANAKIN_ATTR_IS_LIST.keys(): - return ANAKIN_ATTR_IS_LIST[fluid_attr_type] - else: - return False # AttrType.LONG - else: - raise NameError('ERROR: attr_name %s is not exists.' % ( attr_name ) ) - - def dtype_of_attr(self, op, attr_name): - if op.has_attr(attr_name): - fluid_attr_type = op.attr_type(attr_name) - if fluid_attr_type in ANAKIN_ATTR_DTYPE.keys(): - return ANAKIN_ATTR_DTYPE[fluid_attr_type] - else: - return INT32 # AttrType.LONG - else: - raise NameError('ERROR: attr_name %s is not exists.' % ( attr_name ) ) - - def attr_data_required(self, op, attr_name): - data = op.attr(attr_name) - is_list = self.is_list_type(op, attr_name) - dtype = self.dtype_of_attr(op, attr_name) - if dtype not in [INT32, FLOAT, STR]: - return data - elif dtype == INT32: - return map(int, data) if is_list else int(data) - elif dtype == FLOAT: - return map(float, data) if is_list else float(data) - elif dtype == STR: - return bytes(data) - - def attr_data(self, op, attr_name, default_value = 0, type = None): - if op.has_attr(attr_name): - return self.attr_data_required(op, attr_name) - else: - #raise NameError('ERROR: attr_name %s is not exists.' % ( attr_name ) ) - return default_value - - def param_tensor_sh(self, - op, - param_name, - transpose = False, - axes = None, - reshape = None, - var_idx = 0, - layout = 'NCHW'): - - tensor = TensorProtoIO() - [flat_data, shape] = self.data_with_shape_by_param(op, param_name, transpose, \ - axes, var_idx, True, layout) - dtype = self.dtype_by_param(op, param_name, var_idx) - tensor.set_data_type(dtype) - if dtype in ANAKIN_TENSOR_DTYPESTR.keys(): - tensor.set_data(flat_data, ANAKIN_TENSOR_DTYPESTR[dtype]) - #pass #debug - else: - raise NameError('ERROR: Unknown data type (%s)' % ( dtype ) ) - if reshape is not None: - tensor.set_shape(reshape) - else: - tensor.set_shape(shape) - return [tensor, shape] - - def param_tensor(self, - op, - param_name, - transpose = False, - axes = None, - reshape = None, - var_idx = 0, - layout = 'NCHW'): - - [tensor, shape] = self.param_tensor_sh(op, param_name, transpose, axes, \ - reshape, var_idx, layout) - return tensor - - def create_tensor(self, data_list, data_shape, dtype): - tensor = TensorProtoIO() - tensor.set_data_type(dtype) - tensor.set_data(data_list, ANAKIN_TENSOR_DTYPESTR[dtype]) - tensor.set_shape(data_shape) - return tensor - - def gru_tensor_convert(self, origin_h2h, origin_i2h, origin_b, offset=[2, 1, 0]): - hidden_size = int(origin_b.size // 3) - word_size = int(origin_i2h.size // hidden_size // 3) - tar_h2h=np.array(origin_h2h.flatten().tolist()[2*hidden_size*hidden_size:]\ - +np.array(origin_h2h.flatten().tolist()[:2*hidden_size*hidden_size])\ - .reshape(hidden_size,2,hidden_size)[:,[1,0],:].flatten().tolist())\ - .reshape(1,1,hidden_size,3*hidden_size) - tar_i2h=origin_i2h.reshape(word_size,3,hidden_size)[:,offset,:]\ - .reshape(1,1,word_size,3*hidden_size) - tar_b=origin_b.reshape(3, hidden_size)[offset, :].reshape(1,1,1,3 * hidden_size) - tar_i2h_h2h=np.concatenate([tar_i2h.flatten(),tar_h2h.flatten()])\ - .reshape(1,1,1,3*hidden_size*hidden_size+3*word_size*hidden_size) - return tar_i2h_h2h, tar_b - - def lstm_fc_tensor_merge_convert(self, origin_hidden_size, origin_lstm_w, origin_lstm_b, origin_fc_w, origin_fc_b): - - layer_size = int (origin_hidden_size // 4) - input_size = int (origin_fc_w.size // origin_hidden_size) - lstm_bias_num = int (origin_lstm_b.size // layer_size) - tar_w = np.vstack((np.hstack((origin_fc_w[:, 1 * layer_size : 2 * layer_size], - origin_fc_w[:, 2 * layer_size : 3 * layer_size], - origin_fc_w[:, : 1 * layer_size], - origin_fc_w[:, 3 * layer_size :])), - np.hstack((origin_lstm_w[:, 1 * layer_size : 2 * layer_size], - origin_lstm_w[:, 2 * layer_size : 3 * layer_size], - origin_lstm_w[:, : 1 * layer_size], - origin_lstm_w[:, 3 * layer_size : ])))) - - if origin_fc_b is not None: - split_fc_bc = origin_fc_b.flatten()[: 1 * layer_size] - split_fc_bi = origin_fc_b.flatten()[1 * layer_size : 2 * layer_size] - split_fc_bf = origin_fc_b.flatten()[2 * layer_size : 3 * layer_size] - split_fc_bo = origin_fc_b.flatten()[3 * layer_size : 4 * layer_size] - else: - split_fc_bc = np.zeros(layer_size) - split_fc_bi = np.zeros(layer_size) - split_fc_bf = np.zeros(layer_size) - split_fc_bo = np.zeros(layer_size) - - split_lstm_bc = origin_lstm_b.flatten()[: 1 * layer_size] - split_lstm_bi = origin_lstm_b.flatten()[1 * layer_size: 2 * layer_size] - split_lstm_bf = origin_lstm_b.flatten()[2 * layer_size: 3 * layer_size] - split_lstm_bo = origin_lstm_b.flatten()[3 * layer_size: 4 * layer_size] - split_lstm_bc = np.add(split_lstm_bc, split_fc_bc) - split_lstm_bi = np.add(split_lstm_bi, split_fc_bi) - split_lstm_bf = np.add(split_lstm_bf, split_fc_bf) - split_lstm_bo = np.add(split_lstm_bo, split_fc_bo) - - if lstm_bias_num == 4: - tar_b = np.array(split_lstm_bi.flatten().tolist() - + split_lstm_bf.flatten().tolist() - + split_lstm_bc.flatten().tolist() - + split_lstm_bo.flatten().tolist()) - else: - split_lstm_wic = origin_lstm_b.flatten()[4 * layer_size : 5 * layer_size] - split_lstm_wfc = origin_lstm_b.flatten()[5 * layer_size : 6 * layer_size] - split_lstm_woc = origin_lstm_b.flatten()[6 * layer_size :] - tar_b = np.array(split_lstm_bi.flatten().tolist() - + split_lstm_bf.flatten().tolist() - + split_lstm_bc.flatten().tolist() - + split_lstm_bo.flatten().tolist() - + split_lstm_wic.flatten().tolist() - + split_lstm_wfc.flatten().tolist() - + split_lstm_woc.flatten().tolist()) - return tar_w.reshape(input_size+ layer_size, 4 * layer_size, 1, 1),\ - tar_b.reshape(1, origin_lstm_b.size, 1, 1) - - -class Fluid_comparator: - - def __init__(self, helper): - self.helper = helper - self.only_list = ['feed', 'fetch'] - - def compare_by_param(self, op_a, op_b, param): - is_weight_a = self.helper.is_persistable_param(op_a, param) - is_weight_b = self.helper.is_persistable_param(op_b, param) - if is_weight_a and is_weight_b: - np_a = self.helper.np_param(op_a, param) - np_b = self.helper.np_param(op_b, param) - if (np_a == np_b).all() == True: - return True - else: - return False - elif is_weight_a is is_weight_b: - return True - else: - return False - - def have_same_weights(self, op_a, op_b): - is_same = True - if op_a.input_names == op_b.input_names: - params = op_a.input_names - for param in params: - if self.compare_by_param(op_a, op_b, param) is False: - is_same = False - return is_same - else: - return False - - def compare_by_attr(self, op_a, op_b, attr_name): - data_a = self.helper.attr_data(op_a, attr_name) - data_b = self.helper.attr_data(op_b, attr_name) - return data_a == data_b - - def have_same_attrs(self, op_a, op_b): - is_same = True - if op_a.attr_names == op_b.attr_names: - attrs = op_a.attr_names - for attr in attrs: - if self.compare_by_attr(op_a, op_b, attr) is False: - is_same = False - return is_same - else: - return False - - def brothers(self, op_list): - is_same = True - if len(op_list) > 1: - idx = 0 - for op_b in op_list[1:]: - if op_b.type not in self.only_list: - idx = op_list.index(op_b) - op_a = op_list[idx - 1] - if op_a.type not in self.only_list: - same_weights = self.have_same_weights(op_a, op_b) - same_attrs = self.have_same_attrs(op_a, op_b) - if (same_weights and same_attrs) is False: - is_same = False - else: - raise NameError('ERROR: %s is in only_list.' % ( op_a.type )) - else: - raise NameError('ERROR: %s is in only_list.' % ( op_b.type )) - return is_same - else: - raise NameError('ERROR: Members of op_list must be greater than 2.') - - -ANAKIN_TENSOR_DTYPE = { - VarDesc.VarType.BOOL: BOOLEN, - VarDesc.VarType.INT32: INT32, - VarDesc.VarType.FP16: FLOAT16, - VarDesc.VarType.FP32: FLOAT, - VarDesc.VarType.FP64: DOUBLE, -} - -ANAKIN_TENSOR_DTYPESTR = { - STR: "string", - INT32: "int", - FLOAT: "float", - BOOLEN: "bool", -} - -ANAKIN_ATTR_DTYPE = { - AttrType.INT: INT32, - AttrType.INTS: INT32, - AttrType.FLOAT: FLOAT, - AttrType.FLOATS: FLOAT, - AttrType.STRING: STR, - AttrType.STRINGS: STR, - AttrType.BOOL: BOOLEN, - AttrType.BOOLS: BOOLEN, -} - -ANAKIN_ATTR_IS_LIST = { - AttrType.INT: False, - AttrType.INTS: True, - AttrType.FLOAT: False, - AttrType.FLOATS: True, - AttrType.STRING: False, - AttrType.STRINGS: True, - AttrType.BOOL: False, - AttrType.BOOLS: True, -} - -APPEND_BIAS_OP_TYPE = [ - 'FC', - 'mul', - 'sequence_conv', - 'conv2d', - 'conv2d_transpose', - 'depthwise_conv2d', - 'elementwise_mul', -] - -APPEND_ACT_OP_TYPE = [ - 'FC', - 'mul', - 'sequence_conv', - 'conv2d', - 'conv2d_transpose', - 'batch_norm', - 'layer_norm', - 'row_conv', - 'reshape', -] diff --git a/tools/external_converter_v2/parser/kill_fluid/fluid_layer_param_transmit.py b/tools/external_converter_v2/parser/kill_fluid/fluid_layer_param_transmit.py deleted file mode 100644 index 46ed42324..000000000 --- a/tools/external_converter_v2/parser/kill_fluid/fluid_layer_param_transmit.py +++ /dev/null @@ -1,490 +0,0 @@ -from ..operations import OpsParam, OpsRegister -from ..logger import * -from ..proto import * -from fluid_helper import * - - -def ParserFeedDecorator(OpName): - def warpper(Parser): - def warpper_args(args): - Parser(args) - OpsRegister()[OpName].feed_node_attr(args[0]) - args[2].set_name(OpName) - args[0].set_op(args[2]()) - return warpper_args - return warpper - -# common -def NotNeededInInference(args): - # args is tuple object - node_io = args[0] - layer = args[1] - -@ParserFeedDecorator("Input") -def Parser_feed(args): - private_data = args[4] - input_shape = private_data['input_shape'] - alias = private_data['alias'] - OpsRegister()["Input"].input_shape = input_shape - OpsRegister()["Input"].alias = alias - -@ParserFeedDecorator("Convolution") -def Parser_conv2d(args): - op = args[1] - helper = args[3] - private_data = args[4] - [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Filter') - OpsRegister()["Convolution"].weight_1 = weights_tensor - OpsRegister()["Convolution"].filter_num = weights_shape[0] - OpsRegister()["Convolution"].kernel_size = weights_shape[-2:] - OpsRegister()["Convolution"].strides = helper.attr_data(op, 'strides') - OpsRegister()["Convolution"].padding = helper.attr_data(op, 'paddings') - OpsRegister()["Convolution"].dilation_rate = helper.attr_data(op, 'dilations') - OpsRegister()["Convolution"].group = helper.attr_data(op, 'groups') - OpsRegister()["Convolution"].axis = 1 - if 'bias' in private_data.keys(): - OpsRegister()["Convolution"].bias_term = True - OpsRegister()["Convolution"].weight_2 = private_data['bias'] - else: - OpsRegister()["Convolution"].bias_term = False - -@ParserFeedDecorator("ReLU") -def Parser_relu(args): - OpsRegister()["ReLU"].alpha = 0.0 - -@ParserFeedDecorator("Pooling") -def Parser_pool2d(args): - op = args[1] - helper = args[3] - OpsRegister()["Pooling"].pool_size = helper.attr_data(op, 'ksize') - OpsRegister()["Pooling"].strides = helper.attr_data(op, 'strides') - OpsRegister()["Pooling"].padding = helper.attr_data(op, 'paddings') - OpsRegister()["Pooling"].global_pooling = helper.attr_data(op, 'global_pooling') - if helper.attr_data(op, 'pooling_type') == 'max': - OpsRegister()["Pooling"].method = "MAX" - elif helper.attr_data(op, 'pooling_type') in ['average', 'avg']: - OpsRegister()["Pooling"].method = "AVG" - if helper.attr_data(op, 'ceil_mode') == False: - OpsRegister()["Pooling"].cmp_out_shape_floor_as_conv = True - else: - OpsRegister()["Pooling"].cmp_out_shape_floor_as_conv = False - -@ParserFeedDecorator("Dense") -def Parser_mul(args): - op = args[1] - helper = args[3] - private_data = args[4] - weights_needs_trans = True - [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Y', weights_needs_trans) - OpsRegister()["Dense"].weight_1 = weights_tensor - OpsRegister()["Dense"].out_dim = weights_shape[2] - OpsRegister()["Dense"].axis = helper.attr_data(op, 'x_num_col_dims') - if 'bias' in private_data.keys(): - OpsRegister()["Dense"].bias_term = True - OpsRegister()["Dense"].weight_2 = private_data['bias'] - else: - OpsRegister()["Dense"].bias_term = False - -@ParserFeedDecorator("Softmax") -def Parser_softmax(args): - private_data = args[4] - if 'axis' in private_data.keys(): - axis = private_data['axis'] - else: - axis = 1 - OpsRegister()["Softmax"].axis = axis - -@ParserFeedDecorator("Activation") -def Parser_sigmoid(args): - OpsRegister()["Activation"].type = "Sigmoid" - -@ParserFeedDecorator("Axpy") -def Parser_axpy(args): - pass - -@ParserFeedDecorator("BatchNorm") -def Parser_batch_norm(args): - op = args[1] - helper = args[3] - OpsRegister()["BatchNorm"].weight_1 = helper.param_tensor(op, 'Mean') - OpsRegister()["BatchNorm"].weight_2 = helper.param_tensor(op, 'Variance') - OpsRegister()["BatchNorm"].weight_3 = helper.create_tensor([1], [1, 1, 1, 1], FLOAT) - OpsRegister()["BatchNorm"].momentum = helper.attr_data(op, 'momentum') - OpsRegister()["BatchNorm"].epsilon = helper.attr_data(op, 'epsilon') - -@ParserFeedDecorator("Scale") -def Parser_scale_disc_bn(args): - op = args[1] - helper = args[3] - mean = helper.np_param(op, 'Mean') - var = helper.np_param(op, 'Variance') - alpha = helper.np_param(op, 'Scale') - beta = helper.np_param(op, 'Bias') - eps = helper.attr_data(op, 'epsilon') - var = np.sqrt(var + eps) - np_scale = alpha / var - np_bias = beta - (alpha * mean / var) - np_scale_shape = map(int, [1]*(4-len(np_scale.shape)) + list(np_scale.shape)) - np_bias_shape = map(int, [1]*(4-len(np_bias.shape)) + list(np_bias.shape)) - np_scale_tensor = helper.create_tensor(list(np_scale.flatten()), np_scale_shape, FLOAT) - np_bias_tensor = helper.create_tensor(list(np_bias.flatten()), np_bias_shape, FLOAT) - OpsRegister()["Scale"].bias_term = True - OpsRegister()["Scale"].weight_1 = np_scale_tensor - OpsRegister()["Scale"].weight_2 = np_bias_tensor - OpsRegister()["Scale"].axis = 1 - OpsRegister()["Scale"].num_axes = 1 - -@ParserFeedDecorator("Scale") -def Parser_scale_of_bn(args): - op = args[1] - helper = args[3] - OpsRegister()["Scale"].weight_1 = helper.param_tensor(op, 'Scale') - OpsRegister()["Scale"].axis = 1 - OpsRegister()["Scale"].num_axes = 1 - has_bias = helper.is_persistable_param(op, 'Bias') - if has_bias is True: - OpsRegister()["Scale"].bias_term = True - OpsRegister()["Scale"].weight_2 = helper.param_tensor(op, 'Bias') - else: - OpsRegister()["Scale"].bias_term = False - -@ParserFeedDecorator("Split") -def Parser_split(args): - private_data = args[4] - split_num = private_data['split_num'] - OpsRegister()["Split"].split_num = split_num - -@ParserFeedDecorator("Reshape") # NMT -def Parser_reshape(args): - op = args[1] - helper = args[3] - private_data = args[4] - if 'new_shape' in private_data.keys(): - shape = private_data['new_shape'] - else: - shape = helper.attr_data(op, 'shape') - shape = map(int, shape + [1] * (4 - len(shape))) - OpsRegister()["Reshape"].dims = shape - -@ParserFeedDecorator("Concat") -def Parser_concat(args): - op = args[1] - helper = args[3] - OpsRegister()["Concat"].axis = helper.attr_data(op, 'axis') - -@ParserFeedDecorator("Concat") -def Parser_concat_btw_priorbox_boxcoder(args): - op = args[1] - helper = args[3] - OpsRegister()["Concat"].axis = 3 - -@ParserFeedDecorator("Permute") -def Parser_transpose(args): - op = args[1] - helper = args[3] - fluid_dims = helper.attr_data(op, 'axis') - n = 4 - len(fluid_dims) - dims = range(0, n) - tail_dims = [i + n for i in fluid_dims] - dims.extend(tail_dims) - OpsRegister()["Permute"].dims = dims - - -########## SSD Model ########## - -@ParserFeedDecorator("PriorBox") -def Parser_prior_box(args): - op = args[1] - helper = args[3] - OpsRegister()["PriorBox"].min_size = helper.attr_data(op, 'min_sizes') - OpsRegister()["PriorBox"].max_size = helper.attr_data(op, 'max_sizes') - OpsRegister()["PriorBox"].aspect_ratio = helper.attr_data(op, 'aspect_ratios') - OpsRegister()["PriorBox"].is_flip = helper.attr_data(op, 'flip') - OpsRegister()["PriorBox"].is_clip = helper.attr_data(op, 'clip') - OpsRegister()["PriorBox"].variance = helper.attr_data(op, 'variances') - OpsRegister()["PriorBox"].img_h = 0 - OpsRegister()["PriorBox"].img_w = 0 - OpsRegister()["PriorBox"].step_h = helper.attr_data(op, 'step_h') - OpsRegister()["PriorBox"].step_w = helper.attr_data(op, 'step_w') - OpsRegister()["PriorBox"].offset = helper.attr_data(op, 'offset') - OpsRegister()["PriorBox"].order = ['MIN', 'COM', 'MAX'] - -@ParserFeedDecorator("box_coder") -def Parser_box_coder(args): - pass - -@ParserFeedDecorator("DetectionOutput") -def Parser_multiclass_nms(args): - op = args[1] - helper = args[3] - private_data = args[4] - OpsRegister()["DetectionOutput"].share_location = True - OpsRegister()["DetectionOutput"].variance_encode_in_target = False - OpsRegister()["DetectionOutput"].class_num = 0 - OpsRegister()["DetectionOutput"].background_id = helper.attr_data(op, 'background_label') - OpsRegister()["DetectionOutput"].keep_top_k = helper.attr_data(op, 'keep_top_k') - OpsRegister()["DetectionOutput"].conf_thresh = helper.attr_data(op, 'score_threshold') - OpsRegister()["DetectionOutput"].nms_top_k = helper.attr_data(op, 'nms_top_k') - OpsRegister()["DetectionOutput"].nms_thresh = helper.attr_data(op, 'nms_threshold') - OpsRegister()["DetectionOutput"].nms_eta = helper.attr_data(op, 'nms_eta') - if 'code_type' in private_data.keys(): - if private_data['code_type'] == 'decode_center_size': - OpsRegister()["DetectionOutput"].code_type = "CENTER_SIZE" - else: - OpsRegister()["DetectionOutput"].code_type = "CORNER" - - -########## VIS Model ########## - -@ParserFeedDecorator("Im2Sequence") -def Parser_im2sequence(args): - op = args[1] - helper = args[3] - OpsRegister()["Im2Sequence"].paddings = helper.attr_data(op, 'paddings') - OpsRegister()["Im2Sequence"].strides = helper.attr_data(op, 'strides') - OpsRegister()["Im2Sequence"].window_size = helper.attr_data(op, 'kernels') - OpsRegister()["Im2Sequence"].dilations = helper.attr_data(op, 'dilations', [1, 1]) - -@ParserFeedDecorator("Cast") -def Parser_cast(args): - op = args[1] - helper = args[3] - OpsRegister()["Cast"].in_type = helper.attr_data(op, 'in_dtype') - OpsRegister()["Cast"].out_type = helper.attr_data(op, 'out_dtype') - -@ParserFeedDecorator("Argmax") # new256 -def Parser_top_k(args): - op = args[1] - helper = args[3] - OpsRegister()["Argmax"].out_max_val = True - OpsRegister()["Argmax"].top_k = helper.attr_data(op, 'k') - OpsRegister()["Argmax"].axis_term = False - -@ParserFeedDecorator("CtcAlign") -def Parser_ctc_align(args): - op = args[1] - helper = args[3] - OpsRegister()["CtcAlign"].merge_repeated = helper.attr_data(op, 'merge_repeated') - OpsRegister()["CtcAlign"].blank = helper.attr_data(op, 'blank') - -@ParserFeedDecorator("Eltwise") -def Parser_sum(args): - OpsRegister()["Eltwise"].type = "Add" - OpsRegister()["Eltwise"].coeff = [1.0, 1.0] - -@ParserFeedDecorator("LRN") -def Parser_lrn(args): - op = args[1] - helper = args[3] - OpsRegister()["LRN"].local_size = helper.attr_data(op, 'n') - OpsRegister()["LRN"].alpha = helper.attr_data(op, 'alpha') - OpsRegister()["LRN"].beta = helper.attr_data(op, 'beta') - OpsRegister()["LRN"].norm_region = "ACROSS_CHANNELS" - OpsRegister()["LRN"].k = helper.attr_data(op, 'k') - -@ParserFeedDecorator("Gru") -def Parser_gru(args): - op = args[1] - helper = args[3] - private_data = args[4] - OpsRegister()["Gru"].is_reverse = helper.attr_data(op, 'is_reverse') - OpsRegister()["Gru"].gate_activation = helper.attr_data(op, 'gate_activation') + '_fluid' - OpsRegister()["Gru"].activation = helper.attr_data(op, 'activation') + '_fluid' - OpsRegister()["Gru"].gru_formula = "gru_origin" - if bool(private_data) is True: - ori_bx = private_data['np_bias_x'] - ori_bh = helper.np_param(op, 'Bias') - ori_b = ori_bx + ori_bh - ori_wx = private_data['np_weight_x'] - ori_wh = helper.np_param(op, 'Weight') - new_tensors = helper.gru_tensor_convert(ori_wh, ori_wx, ori_b) - weights = [] - for tensor in new_tensors: - weights.append(helper.create_tensor(list(tensor.flatten()), list(np.shape(tensor)), FLOAT)) - OpsRegister()["Gru"].weight_1 = weights[0] - OpsRegister()["Gru"].weight_2 = weights[1] - else: - OpsRegister()["Gru"].weight_1 = helper.param_tensor(op, 'Weight') - OpsRegister()["Gru"].weight_2 = helper.create_tensor([0], [-1], FLOAT) - -@ParserFeedDecorator("LSTM") -def Parser_lstm(args): - op = args[1] - helper = args[3] - private_data = args[4] - OpsRegister()["LSTM"].candidate_activation = helper.attr_data(op, 'candidate_activation') - OpsRegister()["LSTM"].cell_activation = helper.attr_data(op, 'cell_activation') - OpsRegister()["LSTM"].gate_activation = helper.attr_data(op, 'gate_activation') - OpsRegister()["LSTM"].is_reverse = helper.attr_data(op, 'is_reverse') - OpsRegister()["LSTM"].use_peepholes = helper.attr_data(op, 'use_peepholes') - OpsRegister()["LSTM"].num_direction = 1 - OpsRegister()["LSTM"].dropout_param = 1.0 - OpsRegister()["LSTM"].num_layers = 1 - OpsRegister()["LSTM"].input_activation = "null" - if bool(private_data) is True: - np_fc_bias = private_data['np_flat_fc_bias'] - np_fc_weight = private_data['np_flat_fc_weight'] - np_fc_outdim = private_data['np_fc_outdim'] - np_lstm_bias = helper.np_param(op, 'Bias') - np_lstm_weight = helper.np_param(op, 'Weight') - np_tensors = helper.lstm_fc_tensor_merge_convert(np_fc_outdim, np_lstm_weight, \ - np_lstm_bias, np_fc_weight, np_fc_bias) - np_weight = np_tensors[0] - np_bias = np_tensors[1] - np_weight_shape = map(int, [1]*(4-len(np_weight.shape)) + list(np_weight.shape)) - np_bias_shape = map(int, [1]*(4-len(np_bias.shape)) + list(np_bias.shape)) - np_weight_tensor = helper.create_tensor(list(np_weight.flatten()), np_weight_shape, FLOAT) - np_bias_tensor = helper.create_tensor(list(np_bias.flatten()), np_bias_shape, FLOAT) - OpsRegister()["LSTM"].weight_1 = np_weight_tensor - OpsRegister()["LSTM"].weight_2 = np_bias_tensor - else: - OpsRegister()["LSTM"].weight_1 = helper.param_tensor(op, 'Weight') - OpsRegister()["LSTM"].weight_2 = helper.create_tensor([0], [-1], FLOAT) - - -############### RNN ############### - -@ParserFeedDecorator("Embedding") -def Parser_lookup_table(args): - op = args[1] - helper = args[3] - [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'W') - OpsRegister()["Embedding"].weight_1 = weights_tensor - OpsRegister()["Embedding"].padding_idx = helper.attr_data(op, 'padding_idx') - OpsRegister()["Embedding"].word_num = weights_shape[2] - OpsRegister()["Embedding"].emb_dim = weights_shape[3] - -@ParserFeedDecorator("SequencePool") -def Parser_sequence_pool(args): - op = args[1] - helper = args[3] - OpsRegister()["SequencePool"].pooltype = helper.attr_data(op, 'pooltype') - -@ParserFeedDecorator("Activation") -def Parser_tanh(args): - OpsRegister()["Activation"].type = "TanH" - -@ParserFeedDecorator("SequenceConv") -def Parser_sequence_conv(args): - op = args[1] - helper = args[3] - private_data = args[4] - [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Filter') - OpsRegister()["SequenceConv"].weight_1 = weights_tensor - OpsRegister()["SequenceConv"].filter_num = weights_shape[0] - OpsRegister()["SequenceConv"].kernel_size = weights_shape[-2:] - OpsRegister()["SequenceConv"].padding_trainable = helper.attr_data(op, 'paddingTrainable') - OpsRegister()["SequenceConv"].context_stride = helper.attr_data(op, 'contextStride') - OpsRegister()["SequenceConv"].context_start = helper.attr_data(op, 'contextStart') - OpsRegister()["SequenceConv"].context_length = helper.attr_data(op, 'contextLength') - if 'bias' in private_data.keys(): - OpsRegister()["SequenceConv"].bias_term = True - OpsRegister()["SequenceConv"].weight_2 = private_data['bias'] - else: - OpsRegister()["SequenceConv"].bias_term = False - -@ParserFeedDecorator("CrfDecoding") -def Parser_crf_decoding(args): - op = args[1] - helper = args[3] - [weights_tensor, weights_shape] = helper.param_tensor_sh(op, 'Transition') - OpsRegister()["CrfDecoding"].weight_1 = weights_tensor - -@ParserFeedDecorator("MatMul") -def Parser_matmul(args): - op = args[1] - helper = args[3] - private_data = args[4] - if 'coeff' in private_data.keys(): - coeff = private_data['coeff'] - else: - coeff = 1.0 - OpsRegister()["MatMul"].transpose_x = helper.attr_data(op, 'transpose_X') - OpsRegister()["MatMul"].transpose_y = helper.attr_data(op, 'transpose_Y') - OpsRegister()["MatMul"].coeff = coeff - -@ParserFeedDecorator("Scale") -def Parser_scale(args): - op = args[1] - helper = args[3] - scale_val = helper.attr_data(op, 'scale') - OpsRegister()["Scale"].axis = 0 - OpsRegister()["Scale"].num_axes = 0 - OpsRegister()["Scale"].bias_term = False - OpsRegister()["Scale"].weight_1 = helper.create_tensor([scale_val], [1, 1, 1, 1], FLOAT) - -@ParserFeedDecorator("LayerNorm") -def Parser_layer_norm(args): - op = args[1] - helper = args[3] - OpsRegister()["LayerNorm"].weight_1 = helper.param_tensor(op, 'Scale') - OpsRegister()["LayerNorm"].weight_2 = helper.param_tensor(op, 'Bias') - OpsRegister()["LayerNorm"].begin_norm_axis = helper.attr_data(op, 'begin_norm_axis') - OpsRegister()["LayerNorm"].eps = helper.attr_data(op, 'epsilon') - -@ParserFeedDecorator("Scale") -def Parser_dropout(args): - op = args[1] - helper = args[3] - scale_val = 1 - helper.attr_data(op, 'dropout_prob') - OpsRegister()["Scale"].axis = 0 - OpsRegister()["Scale"].num_axes = 0 - OpsRegister()["Scale"].bias_term = False - OpsRegister()["Scale"].weight_1 = helper.create_tensor([scale_val], [1, 1, 1, 1], FLOAT) - -@ParserFeedDecorator("Scale") -def Parser_elementwise_mul(args): - op = args[1] - helper = args[3] - private_data = args[4] - OpsRegister()["Scale"].weight_1 = helper.param_tensor(op, 'Y') - OpsRegister()["Scale"].axis = helper.attr_data(op, 'axis') - OpsRegister()["Scale"].num_axes = 1 - if 'bias' in private_data.keys(): - OpsRegister()["Scale"].bias_term = True - OpsRegister()["Scale"].weight_2 = private_data['bias'] - else: - OpsRegister()["Scale"].bias_term = False - - -FLUID_NODE_FILLER = { - "feed":OpsParam().set_parser(Parser_feed), - "conv2d":OpsParam().set_parser(Parser_conv2d), - "elementwise_add":OpsParam().set_parser(Parser_sum), - "relu":OpsParam().set_parser(Parser_relu), - "pool2d":OpsParam().set_parser(Parser_pool2d), - "mul":OpsParam().set_parser(Parser_mul), - "softmax":OpsParam().set_parser(Parser_softmax), - "sigmoid":OpsParam().set_parser(Parser_sigmoid), - "axpy":OpsParam().set_parser(Parser_axpy), - "batch_norm":OpsParam().set_parser(Parser_batch_norm), - "disc_bn":OpsParam().set_parser(Parser_scale_disc_bn), - "scale_of_bn":OpsParam().set_parser(Parser_scale_of_bn), - "elementwise_mul":OpsParam().set_parser(Parser_elementwise_mul), - "split":OpsParam().set_parser(Parser_split), - "depthwise_conv2d":OpsParam().set_parser(Parser_conv2d), - "reshape":OpsParam().set_parser(Parser_reshape), - "concat":OpsParam().set_parser(Parser_concat), - "transpose":OpsParam().set_parser(Parser_transpose), - "prior_box":OpsParam().set_parser(Parser_prior_box), - "box_coder":OpsParam().set_parser(Parser_box_coder), - "multiclass_nms":OpsParam().set_parser(Parser_multiclass_nms), - "concat_btw_priorbox_boxcoder":OpsParam().set_parser(Parser_concat_btw_priorbox_boxcoder), - "im2sequence":OpsParam().set_parser(Parser_im2sequence), - "gru":OpsParam().set_parser(Parser_gru), - "sum":OpsParam().set_parser(Parser_sum), - "lrn":OpsParam().set_parser(Parser_lrn), - "top_k":OpsParam().set_parser(Parser_top_k), - "ctc_align":OpsParam().set_parser(Parser_ctc_align), - "cast":OpsParam().set_parser(Parser_cast), - "lookup_table":OpsParam().set_parser(Parser_lookup_table), - "sequence_pool":OpsParam().set_parser(Parser_sequence_pool), - "tanh":OpsParam().set_parser(Parser_tanh), - "sequence_conv":OpsParam().set_parser(Parser_sequence_conv), - "crf_decoding":OpsParam().set_parser(Parser_crf_decoding), - "lstm":OpsParam().set_parser(Parser_lstm), - "matmul":OpsParam().set_parser(Parser_matmul), - "layer_norm":OpsParam().set_parser(Parser_layer_norm), - "dropout":OpsParam().set_parser(Parser_dropout), - "scale":OpsParam().set_parser(Parser_scale), -} diff --git a/tools/external_converter_v2/parser/kill_fluid/parser_fluid.py b/tools/external_converter_v2/parser/kill_fluid/parser_fluid.py deleted file mode 100644 index 088b0d84c..000000000 --- a/tools/external_converter_v2/parser/kill_fluid/parser_fluid.py +++ /dev/null @@ -1,868 +0,0 @@ -import numpy as np -import paddle.fluid as fluid -import os -from ..graph_io import * -from ..logger import * -from ..proto import * -from fluid_layer_param_transmit import * - -class FluidParser: - - def __init__(self, fluid_config_dict): - # anakin graph model io - self.graphIO = None - # config info - self.ModelPath = fluid_config_dict['ModelPath'] - self.NetType = fluid_config_dict['NetType'] - self.Debug = fluid_config_dict['Debug'] - # config fluid - self.place = fluid.CPUPlace() - self.exe = fluid.Executor(self.place) - self.scope = fluid.core.Scope() - # in and out edges of node - self.ins = {} - self.outs = {} - # inplaced main node - self.inplace_nodes = {} - self.graph_ins = [] - self.graph_outs = [] - - def __call__(self): - return self._Parsing() - - def _NameNodeMid(self, op): - first_outparam = op.output_names[0] - arg_name = str(op.output(first_outparam)[0]).split('.')[0] - #new_name = op.type + '_' + bytes(op.idx) - new_name = op.type + '#' + bytes(op.idx) + '(' + arg_name + ')' - return new_name - - def _NameNodeIn(self, in_name): - new_name = 'input_' + bytes(self.graph_ins.index(in_name)) - return new_name - - def _NameNodeOut(self, out_name): - new_name = out_name + '_gout' - return new_name - - def _AddPairEdges(self, start_node_name, end_node_name, out_param, in_param): - self.outs[start_node_name].add(out_param, end_node_name) - self.ins[end_node_name].add(in_param, start_node_name) - - def _RmPairEdges(self, start_node_name, end_node_name): - self.outs[start_node_name].rm(end_node_name) - self.ins[end_node_name].rm(start_node_name) - - def _InitEdges(self, node_name): - self.ins[node_name] = Fluid_edger() - self.outs[node_name] = Fluid_edger() - - def _ClearEdges(self, node_name): - if node_name.startswith('input_') is False: - del self.ins[node_name] - if node_name.endswith('_gout') is False: - del self.outs[node_name] - - def _GetOp(self, ops, mid_node_name): - mid_op = None - for op in ops: - node_name = self._NameNodeMid(op) - if mid_node_name == node_name: - mid_op = op - return mid_op - - def _OpTypes(self, ops): - types_cache = [] - for op in ops: - if op.type not in types_cache: - types_cache.append(op.type) - return types_cache - - def _AddProtoNode(self, node_name, op_of_node, helper, private_data, op_type = None): - nodeIO = NodeProtoIO() - opIO = OpsProtoIO() - nodeIO.set_name(node_name) - if op_type is None: - op_type = op_of_node.type - FLUID_NODE_FILLER[op_type](nodeIO, op_of_node, opIO, helper, private_data) - self.graphIO.add_node(nodeIO()) - - def _RmProtoNode(self, node_name): - self.graphIO.rm_node(self.graphIO.find_node_proto(node_name)) - - def _InplaceNodes(self, position): - inplace_heads = self.inplace_nodes.keys() - inplace_mids = [] - inplace_ends = [] - for main_node_name in self.inplace_nodes.keys(): - mid_nodes_name = self.inplace_nodes[main_node_name][1: -1] - inplace_mids.extend(mid_nodes_name) - for main_node_name in self.inplace_nodes.keys(): - end_node_name = self.inplace_nodes[main_node_name][-1] - inplace_ends.append(end_node_name) - if position == 'Head': - return inplace_heads - elif position == 'Mid': - return inplace_mids - elif position == 'End': - return inplace_ends - elif position == 'All': - return inplace_heads + inplace_mids + inplace_ends - - def _EdgeInplace(self, source_ops, helper): - for source_op in source_ops: - if source_op.type not in ['feed', 'fetch']: - if len(source_op.input_arg_names) == 1 \ - and source_op.input_arg_names == source_op.output_arg_names: - source_node_name = self._NameNodeMid(source_op) - inplace_arg = source_op.input_arg_names[0] - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and inplace_arg in tmp_op.output_arg_names: - main_node_name = self._NameNodeMid(tmp_op) - if main_node_name not in self.inplace_nodes.keys(): - self.inplace_nodes[main_node_name] = [main_node_name] - self.inplace_nodes[main_node_name].append(source_node_name) - for main_node_name in self.inplace_nodes.keys(): - inplace_list = self.inplace_nodes[main_node_name] - for inplace_node in inplace_list: - idx = inplace_list.index(inplace_node) - if idx != 0: - self.ins[inplace_node] = Fluid_edger('_In', inplace_list[idx - 1]) - if idx != len(inplace_list) - 1: - self.outs[inplace_node] = Fluid_edger('_Out', inplace_list[idx + 1]) - - def _GetDebugOuts(self, source_ops, helper): - if self.Debug == 'DEBUG': - debug_fetch_list = [] - for source_op in source_ops: - if source_op.type == 'fetch': - var_name = source_op.input_arg_names[0] - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and var_name in tmp_op.input_arg_names: - if var_name not in debug_fetch_list: - debug_fetch_list.append(var_name) - elif tmp_op.type == 'gru' and var_name in tmp_op.output_arg_names: - if var_name not in debug_fetch_list: - debug_fetch_list.append(var_name) - else: - pass - return debug_fetch_list - else: - return [] - - def _ParseBase(self, source_ops, helper, sub_graph_nodes = []): - # Create the original base graph as described in fluid program. - self.graphIO = GraphProtoIO() - self.graphIO.set_name('default_graph_name') - debug_fetch_list = self._GetDebugOuts(source_ops, helper) - self._EdgeInplace(source_ops, helper) - for source_op in source_ops: - if source_op.type not in ['feed', 'fetch']: - main_node_name = self._NameNodeMid(source_op) - in_edges = Fluid_edger() - out_edges = Fluid_edger() - for param in source_op.input_names: - for idx in range(0, len(helper.args_by_input_param(source_op, param))): - arg = helper.var_name_by_param(source_op, param, idx) - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and arg in tmp_op.output_arg_names: - if tmp_op.type == 'feed': - if arg not in self.graph_ins: - self.graph_ins.append(arg) - self.graphIO.add_in(self._NameNodeIn(arg)) - in_edges.add(param, self._NameNodeIn(arg), arg) - else: - tmp_node_name = self._NameNodeMid(tmp_op) - if tmp_node_name in self.inplace_nodes.keys(): - inplace_node_name = self.inplace_nodes[tmp_node_name][-1] - in_edges.add(param, inplace_node_name, arg) - elif tmp_node_name not in self._InplaceNodes('All'): - in_edges.add(param, tmp_node_name, arg) - for param in source_op.output_names: - for idx in range(0, len(helper.args_by_output_param(source_op, param))): - arg = helper.var_name_by_param(source_op, param, idx) - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and arg in tmp_op.input_arg_names: - if tmp_op.type == 'fetch': - if arg not in debug_fetch_list: - arg_node_name = self._NameNodeOut(arg) - if arg not in self.graph_outs: - self.graph_outs.append(arg) - self.graphIO.add_out_fluid(arg_node_name, main_node_name) - out_edges.add(param, arg_node_name, arg) - self.ins[arg_node_name] = Fluid_edger(bytes(source_op.idx), \ - main_node_name) - else: - out_edges.add(param, self._NameNodeMid(tmp_op), arg) - self._AddProtoNode(main_node_name, source_op, helper, {}) - if main_node_name not in self._InplaceNodes('Mid'): - if main_node_name not in self._InplaceNodes('End'): - self.ins[main_node_name] = in_edges - if main_node_name not in self._InplaceNodes('Head'): - if main_node_name not in self._InplaceNodes('End'): - self.outs[main_node_name] = out_edges - else: - inplace_node_name = self.inplace_nodes[main_node_name][-1] - self.outs[inplace_node_name] = out_edges - for redundant_target in self.inplace_nodes[main_node_name][1:]: - self.outs[inplace_node_name].rm(redundant_target) - - def _PrintEdge(self, node, target, direction): - var_name = 'Unknown' - if direction == 'in': - var = self.ins[node].vars_by_target(target) - elif direction == 'out': - var = self.outs[node].vars_by_target(target) - if len(var) > 0: - var_name = var[0] - print node + ",\t" + target + ",\t" + var_name - - def _Graph(self, need_print = False): - for node in self.ins.keys(): - targets_list = self.ins[node]() - for target in targets_list: - self.graphIO.add_in_edge(target, node) - for node in self.outs.keys(): - targets_list = self.outs[node]() - for target in targets_list: - self.graphIO.add_out_edge(node, target) - if need_print is True: - self._PrintEdge(node, target, 'out') - - def _ReplaceInputs(self, source_ops, helper, reshape_dict = {}, layout = 'NCHW'): - for source_op in source_ops: - if source_op.type in ['feed']: - out_edges = Fluid_edger() - for param in source_op.output_names: - private_data = {} - arg = helper.var_name_by_param(source_op, param) - input_node_name = self._NameNodeIn(arg) - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and arg in tmp_op.input_arg_names: - out_edges.add(param, self._NameNodeMid(tmp_op)) - arg_idx = source_op.output_arg_names.index(arg) - shape = helper.var_shape_by_param(False, source_op, "Out", arg_idx, 'UNMODIFIED') - if shape[0] == -1: - shape[0] = 1 - if layout == 'NCHW': - shape = map(int, [1] * (4 - len(shape)) + shape) - if input_node_name in reshape_dict.keys(): - shape = reshape_dict[input_node_name] - private_data['input_shape'] = shape - private_data['alias'] = arg - self.outs[input_node_name] = out_edges - self._AddProtoNode(input_node_name, source_op, helper, private_data) - - def _InsertSplit(self, source_ops, helper): - # If a layer has two identical output tensors, add a split layer. - for node in self.outs.keys(): - if node.startswith('split#') is False: - out_edges = self.outs[node] - for param in out_edges.all_params(): - out_targets_list = out_edges.targets(param) - if len(out_targets_list) > 1: - private_data = {} - private_data['split_num'] = len(out_targets_list) - split_node_name = 'split#' + bytes(out_edges.all_params().index(param)) + '#' + node - self._InitEdges(split_node_name) - for out_target in out_targets_list: - self.outs[node].rm(out_target) - self.ins[out_target].mv(node, split_node_name) - self.outs[split_node_name].add('_Out', out_target) - self._AddPairEdges(node, split_node_name, param, '_In') - self._AddProtoNode(split_node_name, None, helper, private_data, 'split') - - def _Subgraph(self, starts, ends): - out_idx = {} - results = union(starts, ends) - def outs(node): - if node in self.outs.keys(): - return self.outs[node]() - else: - return [] - def next_out(node): - next_out = '' - if len(outs(node)) == 0: - return -1 - elif node not in out_idx.keys(): - out_idx[node] = 0 - if out_idx[node] < len(outs(node)): - next_out = outs(node)[out_idx[node]] - out_idx[node] += 1 - return next_out - for start in starts: - cache = [start] - while len(cache) > 0: - target = next_out(cache[-1]) - while target != -1 and target not in results: - if bool(target) is True: - cache.append(target) - target = next_out(target) - else: - if cache[-1] in results: - results = union(results, cache) - break - if target in results: - cache.append(target) - results = union(results, cache) - cache.pop() - return results - - def _CropGraph(self, ins_of_subgraph, outs_of_subgraph, helper, need_io = True): - def all_nodes(): - all_nodes = [] - for main_node in self.ins.keys(): - all_nodes.extend(self.ins[main_node].all_targets()) - for main_node in self.outs.keys(): - all_nodes.extend(self.outs[main_node].all_targets()) - return list(set(all_nodes)) - stayed_nodes = self._Subgraph(ins_of_subgraph, outs_of_subgraph) - all_nodes = all_nodes() - extra_nodes = difference(all_nodes, stayed_nodes) - for node_name in extra_nodes: - self._RmProtoNode(node_name) - self._ClearEdges(node_name) - if node_name in self.graphIO.ins(): - self.graphIO.rm_in(node_name) - if node_name in self.graphIO.outs(): - self.graphIO.rm_out(node_name) - for node_name in ins_of_subgraph: - if node_name in self.ins: - self.ins[node_name].clear() - for node_name in outs_of_subgraph: - if node_name in self.outs: - self.outs[node_name].clear() - if need_io is True: - for node_name in outs_of_subgraph: - if node_name not in self.graphIO.outs(): - out_node_name = node_name + '_crop_out' - self.ins[out_node_name] = Fluid_edger('_In', node_name) - self.outs[node_name] = Fluid_edger('_Out', out_node_name) - self.graphIO.add_out_fluid(out_node_name, node_name) - for node_name in ins_of_subgraph: - if node_name not in self.graphIO.ins(): - in_node_name = node_name + '_crop_in' - private_data = {'input_shape': [-1, -1, -1, -1]} - self.ins[node_name] = Fluid_edger('_In', in_node_name) - self.outs[in_node_name] = Fluid_edger('_Out', node_name) - self._AddProtoNode(in_node_name, None, helper, private_data, 'feed') - - def _IntegrateNodes(self, main_op, main_node_name, sec_node_name, helper, private_data): - # Merge secondary nodes to the primary node and process the edges. - self._RmProtoNode(main_node_name) - self._RmProtoNode(sec_node_name) - target_nodes_names = self.outs[sec_node_name]() - for target_node_name in target_nodes_names: - self.ins[target_node_name].mv(sec_node_name, main_node_name) - self.outs[main_node_name].mv(sec_node_name, target_node_name) - self.ins[target_node_name].rm(sec_node_name) - self.outs[sec_node_name].rm(target_node_name) - self.ins[sec_node_name].rm(main_node_name) - self.outs[main_node_name].rm(sec_node_name) - self._AddProtoNode(main_node_name, main_op, helper, private_data) - - def _DealWithBias(self, source_ops, helper): - # In fluid, the bias parameter of the conv2d is split into elementwise_add. - for source_op in source_ops: - if source_op.type in APPEND_BIAS_OP_TYPE: - private_data = {} - main_node_name = self._NameNodeMid(source_op) - if main_node_name in self.outs.keys(): - tmp_nodes_names = self.outs[main_node_name]() - if len(tmp_nodes_names) == 1 and tmp_nodes_names[0].startswith('elementwise_add'): - elt_node_name = tmp_nodes_names[0] - elt_op = self._GetOp(source_ops, elt_node_name) - has_weights = helper.is_persistable_param(elt_op, 'Y') - if self._NameNodeMid(elt_op) == elt_node_name and has_weights: - [elt_tensor, shape] = helper.param_tensor_sh(elt_op, 'Y') - new_shape = [1, shape[3], 1, 1] - elt_tensor.set_shape(new_shape) - private_data['bias'] = elt_tensor - self._IntegrateNodes(source_op, main_node_name, elt_node_name, helper, private_data) - - def _DealWithBatchnorm(self, source_ops, helper): - # In anakin, the scale part of batchnorm layer is independent. - for source_op in source_ops: - if source_op.type == 'batch_norm': - discrete_flag = True - main_node_name = self._NameNodeMid(source_op) - input_name = self.ins[main_node_name].target('X') - has_scale = helper.is_persistable_param(source_op, 'Scale') - if input_name.startswith('elementwise_add'): - elt_op = self._GetOp(source_ops, input_name) - x_of_elt = self.ins[input_name].target('X') - has_weights = helper.is_persistable_param(elt_op, 'Y') - if (x_of_elt.startswith('conv2d') or x_of_elt.startswith('depthwise_conv2d')) and has_weights: - discrete_flag = False - elif input_name.startswith('conv2d') or input_name.startswith('depthwise_conv2d'): - discrete_flag = False - if discrete_flag is True: - self._RmProtoNode(main_node_name) - self._AddProtoNode(main_node_name, source_op, helper, {}, 'disc_bn') - elif has_scale is True: - append_node_name = main_node_name + '__scale' - tmp_all_targets_params = self.outs[main_node_name].targets_with_params() - self._InitEdges(append_node_name) - for [tmp_node_name, tmp_param_name] in tmp_all_targets_params: - self.outs[append_node_name].add(tmp_param_name, tmp_node_name) - self.ins[tmp_node_name].mv(main_node_name, append_node_name) - self.outs[main_node_name].rm(tmp_node_name) - self.ins[tmp_node_name].rm(main_node_name) - self.outs[main_node_name].add('_Scale_out', append_node_name) - self.ins[append_node_name].add('_Ins', main_node_name) - self._AddProtoNode(append_node_name, source_op, helper, {}, 'scale_of_bn') - - def _DealWithAxpy(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'elementwise_mul': - mul_node_name = self._NameNodeMid(source_op) - out_targets = self.outs[mul_node_name]() - if len(out_targets) == 1 and out_targets[0].startswith('elementwise_add'): - add_node_name = out_targets[0] - self._RmProtoNode(add_node_name) - a_node_name = self.ins[mul_node_name].target('Y') - x_node_name = self.ins[mul_node_name].target('X') - y_node_name = self.ins[add_node_name].target('X') - self._ClearEdges(mul_node_name) - self.ins[add_node_name].clear() - self.outs[a_node_name].mv(mul_node_name, add_node_name) - self.outs[x_node_name].mv(mul_node_name, add_node_name) - self.ins[add_node_name].add('A', a_node_name) - self.ins[add_node_name].add('X', x_node_name) - self.ins[add_node_name].add('Y', y_node_name) - self._RmProtoNode(mul_node_name) - self._AddProtoNode(add_node_name, None, helper, {}, 'axpy') - - def _DealWithPriorBox(self, source_ops, helper): - nodes_to_del = [] - for source_op in source_ops: - if source_op.type == 'prior_box': - pb_node_name = self._NameNodeMid(source_op) - br_node_name = self.outs[pb_node_name].target('Boxes') - vr_node_name = self.outs[pb_node_name].target('Variances') - bc_node_name = self.outs[br_node_name].target('Out') - vc_node_name = self.outs[vr_node_name].target('Out') - boxcoder_node_name = self.outs[bc_node_name].target('Out') - self.outs[pb_node_name].mv(br_node_name, bc_node_name) - self.outs[pb_node_name].rm(vr_node_name) - self.ins[bc_node_name].mv(br_node_name, pb_node_name) - self.ins[boxcoder_node_name].rm(vc_node_name) - for node_name in [br_node_name, vr_node_name, vc_node_name]: - if node_name not in nodes_to_del: - nodes_to_del.append(node_name) - input_node_name = self.ins[pb_node_name].target('Input') - image_node_name = self.ins[pb_node_name].target('Image') - self.ins[pb_node_name].rm(input_node_name) - self.ins[pb_node_name].rm(image_node_name) - self.ins[pb_node_name].add('Input', input_node_name) - self.ins[pb_node_name].add('Image', image_node_name) - self._RmProtoNode(bc_node_name) - self._AddProtoNode(bc_node_name, None, helper, {}, 'concat_btw_priorbox_boxcoder') - for node_name in nodes_to_del: - self._RmProtoNode(node_name) - self._ClearEdges(node_name) - - def _DealWithDetectionOutput(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'box_coder': - bc_node_name = self._NameNodeMid(source_op) - out_targets = self.outs[bc_node_name]() - if len(out_targets) == 1 and out_targets[0].startswith('multiclass_nms'): - private_data = {} - private_data['code_type'] = helper.attr_data(source_op, 'code_type') - bc_out_arg = helper.var_name_by_param(source_op, 'OutputBox') - for tmp_op in source_ops: - if tmp_op.idx != source_op.idx and bc_out_arg in tmp_op.input_arg_names: - nms_op = tmp_op - nms_node_name = out_targets[0] - loc_node_name = self.ins[bc_node_name].target('TargetBox') - conf_node_name = self.ins[nms_node_name].target('Scores') - box_node_name = self.ins[bc_node_name].target('PriorBox') - self._ClearEdges(bc_node_name) - self.ins[nms_node_name].clear() - self.outs[loc_node_name].mv(bc_node_name, nms_node_name) - self.outs[box_node_name].mv(bc_node_name, nms_node_name) - self.ins[nms_node_name].add('mbox_loc', loc_node_name) - self.ins[nms_node_name].add('mbox_conf_flatten', conf_node_name) - self.ins[nms_node_name].add('mbox_prior_box', box_node_name) - self._RmProtoNode(bc_node_name) - self._RmProtoNode(nms_node_name) - self._AddProtoNode(nms_node_name, nms_op, helper, private_data, 'multiclass_nms') - - def _DealWithMultiFC(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'sum': - sum_node_name = self._NameNodeMid(source_op) - mul_names_list = self.ins[sum_node_name].targets('X') - elt_node_name = self.outs[sum_node_name].target('Out') - if elt_node_name.startswith('elementwise_add') and len(mul_names_list) > 1: - elt_op = self._GetOp(source_ops, elt_node_name) - elt_has_weights = helper.is_persistable_param(elt_op, 'Y') - fc_flag = True - for mul_node_name in mul_names_list: - if mul_node_name.startswith('mul') is False: - fc_flags = False - if fc_flag and elt_has_weights: - private_data = {} - first_mul_name = mul_names_list[0] - first_mul_op = self._GetOp(source_ops, first_mul_name) - in_of_mul_name = self.ins[first_mul_name].target('X') - out_of_elt_name = self.outs[elt_node_name].target('Out') - self.outs[sum_node_name].mv(elt_node_name, out_of_elt_name) - self.ins[out_of_elt_name].mv(elt_node_name, sum_node_name) - self._ClearEdges(elt_node_name) - [elt_tensor, shape] = helper.param_tensor_sh(elt_op, 'Y') - new_shape = [1, shape[3], 1, 1] - elt_tensor.set_shape(new_shape) - private_data['bias'] = elt_tensor - self._RmProtoNode(elt_node_name) - self._RmProtoNode(first_mul_name) - self._AddProtoNode(first_mul_name, first_mul_op, helper, private_data) - - def _DealWithGru(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'gru': - private_data = {} - gru_flags = [False, False] - gru_node_name = self._NameNodeMid(source_op) - gru_op = self._GetOp(source_ops, gru_node_name) - input_list_of_gru = self.ins[gru_node_name].targets('Input') - if len(input_list_of_gru) == 1 and input_list_of_gru[0].startswith('elementwise_add'): - elt_node_name = input_list_of_gru[0] - elt_op = self._GetOp(source_ops, elt_node_name) - has_weights = helper.is_persistable_param(elt_op, 'Y') - if has_weights is True: - private_data['np_bias_x'] = helper.np_param(elt_op, 'Y') - gru_flags[0] = True - input_list_of_elt = self.ins[elt_node_name].targets('X') - if len(input_list_of_elt) == 1 and input_list_of_elt[0].startswith('mul'): - mul_node_name = input_list_of_elt[0] - mul_op = self._GetOp(source_ops, mul_node_name) - if helper.var_name_by_param(mul_op, 'Y').startswith('fc'): - if helper.attr_data(mul_op, 'x_num_col_dims') == 1: - input_list_of_mul = self.ins[mul_node_name].targets('X') - input_name_of_mul = input_list_of_mul[0] - private_data['np_weight_x'] = helper.np_param(mul_op, 'Y') - gru_flags[1] = True - else: - raise NameError('ERROR: Axis of GRU_FC must be 1.') - if gru_flags == [True, True]: - self.outs[input_name_of_mul].mv(mul_node_name, gru_node_name) - self.ins[gru_node_name].mv(elt_node_name, input_name_of_mul) - for node_to_del_name in [mul_node_name, elt_node_name, gru_node_name]: - self._RmProtoNode(node_to_del_name) - if node_to_del_name is not gru_node_name: - self._ClearEdges(node_to_del_name) - self._AddProtoNode(gru_node_name, gru_op, helper, private_data) - - def _SearchBilstm(self, source_ops, helper): - comp = Fluid_comparator(helper) - lstm_ops = [] - for source_op in source_ops: - if source_op.type == 'lstm': - lstm_ops.append(source_op) - if len(lstm_ops) == 2: - lstm_a = lstm_ops[0] - lstm_b = lstm_ops[1] - same_bias = comp.compare_by_param(lstm_a, lstm_b, 'Bias') - same_weight = comp.compare_by_param(lstm_a, lstm_b, 'Weight') - if same_bias and same_weight: - return True - else: - return False - else: - return False - - def _DealWithLstm(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'lstm': - private_data = {} - lstm_flags = [False, False] - lstm_node_name = self._NameNodeMid(source_op) - lstm_op = self._GetOp(source_ops, lstm_node_name) - input_list_of_lstm = self.ins[lstm_node_name].targets('Input') - input_list = [] - if len(input_list_of_lstm) == 1: - in_lstm_node_name = input_list_of_lstm[0] - if input_list_of_lstm[0].split('#')[0] == 'elementwise_add': - elt_op = self._GetOp(source_ops, in_lstm_node_name) - has_weights = helper.is_persistable_param(elt_op, 'Y') - if has_weights is True: - private_data['np_flat_fc_bias'] = helper.np_param(elt_op, 'Y') - lstm_flags[0] = True - input_list = self.ins[in_lstm_node_name].targets('X') - elif input_list_of_lstm[0].split('#')[0] == 'mul': - private_data['np_flat_fc_bias'] = None - input_list = input_list_of_lstm - lstm_flags[0] = True - if lstm_flags[0] is True and len(input_list) == 1: - if input_list[0].split('#')[0] == 'mul': - mul_node_name = input_list[0] - mul_op = self._GetOp(source_ops, mul_node_name) - #if helper.var_name_by_param(mul_op, 'Y').startswith('fc'): - if helper.attr_data(mul_op, 'x_num_col_dims') == 1: - input_list_of_mul = self.ins[mul_node_name].targets('X') - input_name_of_mul = input_list_of_mul[0] - [w_np, w_sh] = helper.data_with_shape_by_param(mul_op, 'Y', \ - False, None, 0, False) - private_data['np_flat_fc_weight'] = w_np - private_data['np_fc_outdim'] = w_sh[3] - lstm_flags[1] = True - else: - raise NameError('ERROR: Axis of LSTM_FC must be 1.') - if lstm_flags == [True, True]: - self.outs[input_name_of_mul].mv(mul_node_name, lstm_node_name) - self.ins[lstm_node_name].mv(in_lstm_node_name, input_name_of_mul) - if in_lstm_node_name == mul_node_name: - nodes_to_del = [mul_node_name, lstm_node_name] - else: - nodes_to_del = [mul_node_name, in_lstm_node_name, lstm_node_name] - for node_to_del_name in nodes_to_del: - self._RmProtoNode(node_to_del_name) - if node_to_del_name is not lstm_node_name: - self._ClearEdges(node_to_del_name) - self._AddProtoNode(lstm_node_name, lstm_op, helper, private_data) - - def _DealWithCast(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'cast': - if helper.attr_data(source_op, 'out_dtype') == 5: - cast_node_name = self._NameNodeMid(source_op) - if cast_node_name in self.ins: - input_name_of_cast = self.ins[cast_node_name].target('X') - if input_name_of_cast.startswith('top_k') is False: - output_name_of_cast = self.outs[cast_node_name].target('Out') - self.outs[input_name_of_cast].mv(cast_node_name, output_name_of_cast) - self.ins[output_name_of_cast].mv(cast_node_name, input_name_of_cast) - self._RmProtoNode(cast_node_name) - self._ClearEdges(cast_node_name) - else: - print 'Cannot find the layer corresponding to cast.' - else: - raise NameError('The out type of cast must be float32.') - - def _DealWithArgmax(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'top_k': - private_data = {} - topk_node_name = self._NameNodeMid(source_op) - out_list = self.outs[topk_node_name].targets('Out') - index_list = self.outs[topk_node_name].targets('Indices') - if len(index_list) > 0: - if len(out_list) == 1 and index_list[0].startswith('cast'): - private_data['out_max_val'] = True - idxcast_node_name = index_list[0] - output_name_of_idxcast = self.outs[idxcast_node_name].target('Out') - if output_name_of_idxcast == out_list[0] and out_list[0].startswith('concat'): - concat_node_name = out_list[0] - output_name_of_concat = self.outs[concat_node_name].target('Out') - self.outs[topk_node_name].rm(idxcast_node_name) - self.outs[topk_node_name].mv(concat_node_name, output_name_of_concat) - self.ins[output_name_of_concat].mv(concat_node_name, topk_node_name) - for node_to_del_name in [concat_node_name, idxcast_node_name]: - self._RmProtoNode(node_to_del_name) - self._ClearEdges(node_to_del_name) - elif output_name_of_idxcast != out_list[0]: - if output_name_of_idxcast.endswith('_gout') and \ - out_list[0].endswith('_gout'): - gout_node_name = out_list[0] - idx_gout_node_name = output_name_of_idxcast - self.outs[topk_node_name].rm(idxcast_node_name) - for node_to_del_name in [idx_gout_node_name, idxcast_node_name]: - self._RmProtoNode(node_to_del_name) - self._ClearEdges(node_to_del_name) - self.graphIO.rm_out(idx_gout_node_name) - elif len(out_list) == 0: - private_data['out_max_val'] = False - self._DealWithCast(source_ops, helper) - else: - raise NameError('ERROR: Unknown top_k layer.') - self._RmProtoNode(topk_node_name) - self._AddProtoNode(topk_node_name, source_op, helper, private_data) - - def _DealWithReshape(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'reshape': - reshape_node_name = self._NameNodeMid(source_op) - if reshape_node_name in self.ins: - shape_inputs = self.ins[reshape_node_name].targets('Shape') - tensor_inputs = self.ins[reshape_node_name].targets('X') - if len(shape_inputs) == 1 and len(tensor_inputs) == 1: - self.ins[reshape_node_name].rm(shape_inputs[0]) - self.ins[reshape_node_name].add('Shape', shape_inputs[0]) - else: - pass - - def _DealWithSoftmax(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'softmax': - softmax_node_name = self._NameNodeMid(source_op) - outs_of_softmax = self.outs[softmax_node_name].targets('Out') - ins_of_softmax = self.ins[softmax_node_name].targets('X') - def prune(reshape_node_name): - branch = [] - branch.append(reshape_node_name) - shape_inputs = self.ins[reshape_node_name].targets('Shape') - tensor_input = self.ins[reshape_node_name].target('X') - tensor_output = self.outs[reshape_node_name].target('Out') - if len(shape_inputs) == 1: - branch.append(shape_inputs[0]) - if len(branch) == 2 and branch[1].split('#')[0] == 'split': - split_node_name = branch[1] - self.outs[split_node_name].rm(reshape_node_name) - self.ins[reshape_node_name].rm(split_node_name) - if len(self.outs[split_node_name].targets('_Out')) == 0: - input_of_split = self.ins[split_node_name].target('_In') - branch.append(input_of_split) - self._RmProtoNode(split_node_name) - self._ClearEdges(split_node_name) - elif len(branch) == 2 and branch[1].startswith('input'): - raise NameError('ERROR: None-split input of Softmax has not supported.') - self.outs[tensor_input].mv(reshape_node_name, tensor_output) - self.ins[tensor_output].mv(reshape_node_name, tensor_input) - self._RmProtoNode(reshape_node_name) - self._ClearEdges(reshape_node_name) - if len(branch) == 3 and branch[2].startswith('input'): - input_node_name = branch[2] - self._RmProtoNode(input_node_name) - self._ClearEdges(input_node_name) - if outs_of_softmax[0].split('#')[0] == 'reshape' and \ - ins_of_softmax[0].split('#')[0] == 'reshape': - private_data = {} - private_data['axis'] = 3 - prune(outs_of_softmax[0]) - prune(ins_of_softmax[0]) - self._RmProtoNode(softmax_node_name) - self._AddProtoNode(softmax_node_name, source_op, helper, private_data) - - def _DealWithMatmal(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'matmul': - matmul_node_name = self._NameNodeMid(source_op) - x_input_name = self.ins[matmul_node_name].target('X') - y_input_name = self.ins[matmul_node_name].target('Y') - flag = False - coeff = 1.0 - for node_name in [x_input_name, y_input_name]: - if node_name.startswith('scale') or node_name.startswith('dropout'): - op = self._GetOp(source_ops, node_name) - if node_name.startswith('scale'): - scale = helper.attr_data(op, 'scale') - elif node_name.startswith('dropout'): - scale = 1 - helper.attr_data(op, 'dropout_prob') - input_node = self.ins[node_name].target('X') - self.outs[input_node].mv(node_name, matmul_node_name) - self.ins[matmul_node_name].mv(node_name, input_node) - self._RmProtoNode(node_name) - self._ClearEdges(node_name) - coeff = coeff * scale - flag = True - if flag is True: - private_data = {} - private_data['coeff'] = coeff - self._RmProtoNode(matmul_node_name) - self._AddProtoNode(matmul_node_name, source_op, helper, private_data) - - def _DealWithDiscBatchNorm(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'batch_norm': - discrete_flag = True - bn_node_name = self._NameNodeMid(source_op) - input_name = self.ins[bn_node_name].target('X') - if input_name.startswith('elementwise_add'): - input_of_elt = self.ins[input_name].target('X') - if input_of_elt.startswith('conv2d'): - discrete_flag = False - elif input_name.startswith('conv2d'): - discrete_flag = False - if discrete_flag is True: - self._RmProtoNode(bn_node_name) - self._AddProtoNode(bn_node_name, source_op, helper, {}, 'disc_bn') - - def _DealWithSSD(self, source_ops, helper): - for source_op in source_ops: - if source_op.type == 'reshape': - rh_node_name = self._NameNodeMid(source_op) - if rh_node_name in self.ins: - private_data = dict() - input_name = self.ins[rh_node_name].target('X') - shape = helper.attr_data(source_op, 'shape') - if input_name.startswith('concat'): - private_data['new_shape'] = [0, shape[0], shape[1], 0] - else: - private_data['new_shape'] = [0, -1, 1, 1] - self._RmProtoNode(rh_node_name) - self._AddProtoNode(rh_node_name, source_op, helper, private_data, 'reshape') - for source_op in source_ops: - if source_op.type == 'softmax': - private_data = dict() - sm_node_name = self._NameNodeMid(source_op) - private_data['axis'] = 2 - self._RmProtoNode(sm_node_name) - self._AddProtoNode(sm_node_name, source_op, helper, private_data, 'softmax') - - def _NewCommonLayer(self, - source_ops, - in_target, - in_param, - out_target, - out_param, - layer_type, - private_data, - helper, - insert_mode = True): - main_layer = layer_type + '_after_' + in_target - if insert_mode is True: - if in_target in self.ins[out_target].all_targets() and \ - out_target in self.outs[in_target].all_targets(): - self.ins[out_target].mv(in_target, main_layer) - self.outs[in_target].mv(out_target, main_layer) - else: - raise NameError('ERROR: Usage of InsertCommonLayer has not supported.') - else: - self.ins[out_target].add(in_param + '_insert', main_layer) - self.outs[in_target].add(out_param + '_insert', main_layer) - self.ins[main_layer] = Fluid_edger(in_param, in_target) - self.outs[main_layer] = Fluid_edger(out_param, out_target) - self._AddProtoNode(main_layer, None, helper, private_data, layer_type) - - def _ParseNetwork(self, source_ops, helper): - self._ParseBase(source_ops, helper) - if self.NetType == "FLUIDBASE": - pass - else: - reshape_dict = {} - if self.NetType == "OCR": - reshape_dict['input_0'] = [1, 1, 48, 1500] - self._ReplaceInputs(source_ops, helper, reshape_dict) - self._InsertSplit(source_ops, helper) - self._DealWithGru(source_ops, helper) - self._DealWithLstm(source_ops, helper) - self._DealWithBias(source_ops, helper) - self._DealWithBatchnorm(source_ops, helper) - self._DealWithMultiFC(source_ops, helper) - self._DealWithArgmax(source_ops, helper) - self._DealWithAxpy(source_ops, helper) - if self.NetType == "SSD": - self._DealWithPriorBox(source_ops, helper) - self._DealWithDetectionOutput(source_ops, helper) - self._DealWithSSD(source_ops, helper) - self._Graph() - - - - def _Parsing(self): - with fluid.scope_guard(self.scope): - if os.path.exists(self.ModelPath + 'model') and os.path.exists(self.ModelPath + 'params'): - [self.net_program, feed_target_names, fetch_targets] = \ - fluid.io.load_inference_model(self.ModelPath, self.exe, 'model', 'params') - else: - [self.net_program, feed_target_names, fetch_targets] = \ - fluid.io.load_inference_model(self.ModelPath, self.exe) - - global_block = self.net_program.global_block() - source_ops = list(global_block.ops) - helper = Fluid_helper(self.scope, global_block) - - self._ParseNetwork(source_ops, helper) - return self.graphIO diff --git a/tools/external_converter_v2/parser/kill_tf/auto_debug.py b/tools/external_converter_v2/parser/kill_tf/auto_debug.py deleted file mode 100644 index ffa2c16a5..000000000 --- a/tools/external_converter_v2/parser/kill_tf/auto_debug.py +++ /dev/null @@ -1,124 +0,0 @@ -import tensorflow as tf -import numpy as np -import os -import tempfile -from parse_tf_2_med import ParseTF2Med -from med_graph import MedGraphUtil -from tf_trans_util import load_graph -from tf_util import TfUtil -class AutoDebug: - - def __init__(self,tf_model_path,ak_output_dir,workspace=None): - self.ak_perfix='record_' - self.tf_model_path=tf_model_path - self.ak_output_dir=os.path.dirname(ak_output_dir) - med_graph = ParseTF2Med(self.tf_model_path).parse() - MedGraphUtil.solve(med_graph) - self.med_graph = med_graph - # if workspace is None: - # self.workspace=tempfile.mkdtemp() - # else: - # self.workspace = workspace - # print('workspace is ', self.workspace) - - - def _debug_graph(self,graph): - for i in graph.values(): - print(i) - exit() - - def _convert_name_tf2ak(self,tf_name): - ak_name = tf_name[:] - for index, x in enumerate(tf_name): - if x == '/': - ak_name = ak_name[:index] + '_' + ak_name[index + 1:] - return self.ak_perfix + ak_name - - def _find_ak_output(self): - result = {} - for file_name in os.listdir(self.ak_output_dir): - real_path=os.path.join(self.ak_output_dir,file_name) - if not os.path.isdir(real_path) and file_name.startswith(self.ak_perfix) and not file_name.startswith(self.ak_perfix+'_'): - result[file_name]=[float(i.split(' ')[1]) for i in open(real_path,'r').readlines()] - return result - - def _creat_and_clear_workspace(self): - if os.path.exists(self.workspace): - assert os.path.isdir(self.workspace) - ls = os.listdir(self.workspace) - for i in ls: - c_path = os.path.join(self.workspace, i) - os.remove(c_path) - else: - os.mkdir(self.workspace) - - def _prepare_data(self,med_graph): - input_cnt=0 - inputs={} - result = {} - np.random.seed(1234567) - for i in med_graph.values(): - if i['ak_type'] == 'Input': - shape=i['output'][0]['shape'] - size=1 - for k in shape: - size*=k - tensor=np.random.randn(size).reshape(shape) - inputs['input_'+str(input_cnt)]=tensor - result[i['name']+':0']=tensor - input_cnt+=1 - - for i in inputs.keys(): - import struct - with open(os.path.join(self.ak_output_dir,i), "wb") as file: - x_value_new = inputs[i].transpose((0, 3, 1, 2)) - for value in x_value_new.flatten(): - file.write(struct.pack('f', value)) - - return result - - - def run_ak(self): - self._prepare_data(self.med_graph) - - def run(self): - inputs = self._prepare_data(self.med_graph) - akopname2ak_out=self._find_ak_output() - for i in akopname2ak_out: - print(i) - compare_dic={} - for node in self.med_graph.values(): - print(node['name'], node['ak_type']) - if node['ak_type'] is not None and node['type'] is not None: - - name=node['name'] - if node.get('fusion_out_name') is not None: - name=node['fusion_out_name'] - ak_name=self._convert_name_tf2ak(name) - print('key ',ak_name) - if akopname2ak_out.get(ak_name) is not None: - compare_dic[name+':0']=akopname2ak_out[ak_name] - outlist=compare_dic.keys() - out=TfUtil.tf_run_model(self.tf_model_path,inputs,outlist) - for index,name in enumerate(outlist): - correct=out[index].transpose() - ak_result=akopname2ak_out[name] - assert len() - for i in range(len(ak_result)): - if abs() - # tensor_shape = {} - # graph={} - # for node in tf_ops: - # for out in node.outputs: - # tensor_shape[out.name] = out.get_shape().as_list() - # input_names = [i.name for i in node.inputs] - # output_names = [i.name for i in node.outputs] - # graph[node.name]={'name':node.name,'input':input_names,'output':output_names} - # self._fix_self_output(graph,tensor_shape) - # self._debug_graph(graph) -ak_work_space='/home/ljj/docker_mount_dev2/anakin2_developing_parser/build/' - -debug=AutoDebug('./resnet_model/frozen_resnet_v1_50.pb',ak_work_space) -debug.run_ak() -debug.run() - diff --git a/tools/external_converter_v2/parser/kill_tf/freeze_graph.py b/tools/external_converter_v2/parser/kill_tf/freeze_graph.py deleted file mode 100644 index 73a430866..000000000 --- a/tools/external_converter_v2/parser/kill_tf/freeze_graph.py +++ /dev/null @@ -1,404 +0,0 @@ -# Copyright 2015 The TensorFlow Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# ============================================================================== -r"""Converts checkpoint variables into Const ops in a standalone GraphDef file. - -This script is designed to take a GraphDef proto, a SaverDef proto, and a set of -variable values stored in a checkpoint file, and output a GraphDef with all of -the variable ops converted into const ops containing the values of the -variables. - -It's useful to do this when we need to load a single file in C++, especially in -environments like mobile or embedded where we may not have access to the -RestoreTensor ops and file loading calls that they rely on. - -An example of command-line usage is: -bazel build tensorflow/python/tools:freeze_graph && \ -bazel-bin/tensorflow/python/tools/freeze_graph \ ---input_graph=some_graph_def.pb \ ---input_checkpoint=model.ckpt-8361242 \ ---output_graph=/tmp/frozen_graph.pb --output_node_names=softmax - -You can also look at freeze_graph_test.py for an example of how to use it. - -""" -from __future__ import absolute_import -from __future__ import division -from __future__ import print_function - -import argparse -import re -import sys - -from google.protobuf import text_format - -from tensorflow.core.framework import graph_pb2 -from tensorflow.core.protobuf import saver_pb2 -from tensorflow.core.protobuf.meta_graph_pb2 import MetaGraphDef -from tensorflow.python import pywrap_tensorflow -from tensorflow.python.client import session -from tensorflow.python.framework import graph_util -from tensorflow.python.framework import importer -from tensorflow.python.platform import app -from tensorflow.python.platform import gfile -from tensorflow.python.saved_model import loader -from tensorflow.python.saved_model import tag_constants -from tensorflow.python.tools import saved_model_utils -from tensorflow.python.training import saver as saver_lib - - -def freeze_graph_with_def_protos(input_graph_def, - input_saver_def, - input_checkpoint, - output_node_names, - restore_op_name, - filename_tensor_name, - output_graph, - clear_devices, - initializer_nodes, - variable_names_whitelist="", - variable_names_blacklist="", - input_meta_graph_def=None, - input_saved_model_dir=None, - saved_model_tags=None, - checkpoint_version=saver_pb2.SaverDef.V2): - """Converts all variables in a graph and checkpoint into constants.""" - del restore_op_name, filename_tensor_name # Unused by updated loading code. - - if not output_node_names: - print("You need to supply the name of a node to --output_node_names.") - return -1 - - # Remove all the explicit device specifications for this node. This helps to - # make the graph more portable. - if clear_devices: - if input_meta_graph_def: - for node in input_meta_graph_def.graph_def.node: - node.device = "" - elif input_graph_def: - for node in input_graph_def.node: - node.device = "" - - if input_graph_def: - _ = importer.import_graph_def(input_graph_def, name="") - with session.Session() as sess: - if input_saver_def: - saver = saver_lib.Saver( - saver_def=input_saver_def, write_version=checkpoint_version) - saver.restore(sess, input_checkpoint) - elif input_meta_graph_def: - restorer = saver_lib.import_meta_graph( - input_meta_graph_def, clear_devices=True) - restorer.restore(sess, input_checkpoint) - if initializer_nodes: - sess.run(initializer_nodes.replace(" ", "").split(",")) - elif input_saved_model_dir: - if saved_model_tags is None: - saved_model_tags = [] - loader.load(sess, saved_model_tags, input_saved_model_dir) - else: - var_list = {} - reader = pywrap_tensorflow.NewCheckpointReader(input_checkpoint) - var_to_shape_map = reader.get_variable_to_shape_map() - - # List of all partition variables. Because the condition is heuristic - # based, the list could include false positives. - all_parition_variable_names = [ - tensor.name.split(":")[0] - for op in sess.graph.get_operations() - for tensor in op.values() - if re.search(r"/part_\d+/", tensor.name) - ] - has_partition_var = False - - for key in var_to_shape_map: - try: - tensor = sess.graph.get_tensor_by_name(key + ":0") - if any(key in name for name in all_parition_variable_names): - has_partition_var = True - except KeyError: - # This tensor doesn't exist in the graph (for example it's - # 'global_step' or a similar housekeeping element) so skip it. - continue - var_list[key] = tensor - - try: - saver = saver_lib.Saver( - var_list=var_list, write_version=checkpoint_version) - except TypeError as e: - # `var_list` is required to be a map of variable names to Variable - # tensors. Partition variables are Identity tensors that cannot be - # handled by Saver. - if has_partition_var: - print("Models containing partition variables cannot be converted " - "from checkpoint files. Please pass in a SavedModel using " - "the flag --input_saved_model_dir.") - return -1 - else: - raise e - - saver.restore(sess, input_checkpoint) - if initializer_nodes: - sess.run(initializer_nodes.replace(" ", "").split(",")) - - variable_names_whitelist = ( - variable_names_whitelist.replace(" ", "").split(",") - if variable_names_whitelist else None) - variable_names_blacklist = ( - variable_names_blacklist.replace(" ", "").split(",") - if variable_names_blacklist else None) - - if input_meta_graph_def: - output_graph_def = graph_util.convert_variables_to_constants( - sess, - input_meta_graph_def.graph_def, - output_node_names.replace(" ", "").split(","), - variable_names_whitelist=variable_names_whitelist, - variable_names_blacklist=variable_names_blacklist) - else: - output_graph_def = graph_util.convert_variables_to_constants( - sess, - input_graph_def, - output_node_names.replace(" ", "").split(","), - variable_names_whitelist=variable_names_whitelist, - variable_names_blacklist=variable_names_blacklist) - - # Write GraphDef to file if output path has been given. - if output_graph: - with gfile.GFile(output_graph, "wb") as f: - f.write(output_graph_def.SerializeToString()) - - return output_graph_def - - -def _parse_input_graph_proto(input_graph, input_binary): - """Parser input tensorflow graph into GraphDef proto.""" - if not gfile.Exists(input_graph): - print("Input graph file '" + input_graph + "' does not exist!") - return -1 - input_graph_def = graph_pb2.GraphDef() - mode = "rb" if input_binary else "r" - with gfile.FastGFile(input_graph, mode) as f: - if input_binary: - input_graph_def.ParseFromString(f.read()) - else: - text_format.Merge(f.read(), input_graph_def) - return input_graph_def - - -def _parse_input_meta_graph_proto(input_graph, input_binary): - """Parser input tensorflow graph into MetaGraphDef proto.""" - if not gfile.Exists(input_graph): - print("Input meta graph file '" + input_graph + "' does not exist!") - return -1 - input_meta_graph_def = MetaGraphDef() - mode = "rb" if input_binary else "r" - with gfile.FastGFile(input_graph, mode) as f: - if input_binary: - input_meta_graph_def.ParseFromString(f.read()) - else: - text_format.Merge(f.read(), input_meta_graph_def) - print("Loaded meta graph file '" + input_graph) - return input_meta_graph_def - - -def _parse_input_saver_proto(input_saver, input_binary): - """Parser input tensorflow Saver into SaverDef proto.""" - if not gfile.Exists(input_saver): - print("Input saver file '" + input_saver + "' does not exist!") - return -1 - mode = "rb" if input_binary else "r" - with gfile.FastGFile(input_saver, mode) as f: - saver_def = saver_pb2.SaverDef() - if input_binary: - saver_def.ParseFromString(f.read()) - else: - text_format.Merge(f.read(), saver_def) - return saver_def - - -def freeze_graph(input_graph, - input_saver, - input_binary, - input_checkpoint, - output_node_names, - restore_op_name, - filename_tensor_name, - output_graph, - clear_devices, - initializer_nodes, - variable_names_whitelist="", - variable_names_blacklist="", - input_meta_graph=None, - input_saved_model_dir=None, - saved_model_tags=tag_constants.SERVING, - checkpoint_version=saver_pb2.SaverDef.V2): - """Converts all variables in a graph and checkpoint into constants.""" - input_graph_def = None - if input_saved_model_dir: - input_graph_def = saved_model_utils.get_meta_graph_def( - input_saved_model_dir, saved_model_tags).graph_def - elif input_graph: - input_graph_def = _parse_input_graph_proto(input_graph, input_binary) - input_meta_graph_def = None - if input_meta_graph: - input_meta_graph_def = _parse_input_meta_graph_proto( - input_meta_graph, input_binary) - input_saver_def = None - if input_saver: - input_saver_def = _parse_input_saver_proto(input_saver, input_binary) - freeze_graph_with_def_protos( - input_graph_def, - input_saver_def, - input_checkpoint, - output_node_names, - restore_op_name, - filename_tensor_name, - output_graph, - clear_devices, - initializer_nodes, - variable_names_whitelist, - variable_names_blacklist, - input_meta_graph_def, - input_saved_model_dir, - saved_model_tags.replace(" ", "").split(","), - checkpoint_version=checkpoint_version) - - -def main(unused_args, flags): - if flags.checkpoint_version == 1: - checkpoint_version = saver_pb2.SaverDef.V1 - elif flags.checkpoint_version == 2: - checkpoint_version = saver_pb2.SaverDef.V2 - else: - print("Invalid checkpoint version (must be '1' or '2'): %d" % - flags.checkpoint_version) - return -1 - freeze_graph(flags.input_graph, flags.input_saver, flags.input_binary, - flags.input_checkpoint, flags.output_node_names, - flags.restore_op_name, flags.filename_tensor_name, - flags.output_graph, flags.clear_devices, flags.initializer_nodes, - flags.variable_names_whitelist, flags.variable_names_blacklist, - flags.input_meta_graph, flags.input_saved_model_dir, - flags.saved_model_tags, checkpoint_version) - -def run_main(): - parser = argparse.ArgumentParser() - parser.register("type", "bool", lambda v: v.lower() == "true") - parser.add_argument( - "--input_graph", - type=str, - default="", - help="TensorFlow \'GraphDef\' file to load.") - parser.add_argument( - "--input_saver", - type=str, - default="", - help="TensorFlow saver file to load.") - parser.add_argument( - "--input_checkpoint", - type=str, - default="", - help="TensorFlow variables file to load.") - parser.add_argument( - "--checkpoint_version", - type=int, - default=2, - help="Tensorflow variable file format") - parser.add_argument( - "--output_graph", - type=str, - default="", - help="Output \'GraphDef\' file name.") - parser.add_argument( - "--input_binary", - nargs="?", - const=True, - type="bool", - default=False, - help="Whether the input files are in binary format.") - parser.add_argument( - "--output_node_names", - type=str, - default="", - help="The name of the output nodes, comma separated.") - parser.add_argument( - "--restore_op_name", - type=str, - default="save/restore_all", - help="""\ - The name of the master restore operator. Deprecated, unused by updated \ - loading code. - """) - parser.add_argument( - "--filename_tensor_name", - type=str, - default="save/Const:0", - help="""\ - The name of the tensor holding the save path. Deprecated, unused by \ - updated loading code. - """) - parser.add_argument( - "--clear_devices", - nargs="?", - const=True, - type="bool", - default=True, - help="Whether to remove device specifications.") - parser.add_argument( - "--initializer_nodes", - type=str, - default="", - help="Comma separated list of initializer nodes to run before freezing.") - parser.add_argument( - "--variable_names_whitelist", - type=str, - default="", - help="""\ - Comma separated list of variables to convert to constants. If specified, \ - only those variables will be converted to constants.\ - """) - parser.add_argument( - "--variable_names_blacklist", - type=str, - default="", - help="""\ - Comma separated list of variables to skip converting to constants.\ - """) - parser.add_argument( - "--input_meta_graph", - type=str, - default="", - help="TensorFlow \'MetaGraphDef\' file to load.") - parser.add_argument( - "--input_saved_model_dir", - type=str, - default="", - help="Path to the dir with TensorFlow \'SavedModel\' file and variables.") - parser.add_argument( - "--saved_model_tags", - type=str, - default="serve", - help="""\ - Group of tag(s) of the MetaGraphDef to load, in string format,\ - separated by \',\'. For tag-set contains multiple tags, all tags \ - must be passed in.\ - """) - flags, unparsed = parser.parse_known_args() - - my_main = lambda unused_args: main(unused_args, flags) - app.run(main=my_main, argv=[sys.argv[0]] + unparsed) - -if __name__ == '__main__': - run_main() diff --git a/tools/external_converter_v2/parser/kill_tf/parser_tf.py b/tools/external_converter_v2/parser/kill_tf/parser_tf.py deleted file mode 100644 index a62684340..000000000 --- a/tools/external_converter_v2/parser/kill_tf/parser_tf.py +++ /dev/null @@ -1,57 +0,0 @@ -import numpy as np -import os -from ..graph_io import * -from ..logger import * -from ..proto import * -import tensorflow as tf -from parse_tf_2_med import ParseTF2Med -from parse_med_2_ak import MedTransAK -from med_graph import MedGraphUtil,MedNodeUtil - - -class TFParser: - - def __init__(self, fluid_config_dict): - # anakin graph model io - # config info - self.ProtoPaths = fluid_config_dict['ProtoPaths'] - - self.OutPuts = fluid_config_dict['OutPuts'] - if self.OutPuts is not None: - self.OutPuts=[i for i in fluid_config_dict['OutPuts'].split(',')] - - self.med_trans_tool = MedTransAK() - self.input_count=0 - - def __call__(self): - med_graph = self._conver_tf_2_med() - if self.OutPuts is None: - self.OutPuts=MedGraphUtil.search_output_list(med_graph) - - MedGraphUtil.solve(med_graph) - anakin_graph = self._conver_med_2_anakin(med_graph) - return anakin_graph - - def _conver_tf_2_med(self): - parser = ParseTF2Med(self.ProtoPaths) - return parser.parse() - - def _add_protonode(self, ak_graph,med_node): - ak_type = med_node['ak_type'] - if ak_type is None: - return - nodeIO = NodeProtoIO() - nodeIO.set_name(med_node['name']) - self.med_trans_tool.map_med_2_ak(nodeIO,med_node) - ak_graph.add_node(nodeIO()) - if nodeIO().Op.name=='Input': - ak_graph.add_in(nodeIO().name) - - def _conver_med_2_anakin(self, med_graph): - anakin_graph = GraphProtoIO() - for node in med_graph.values(): - self._add_protonode(anakin_graph,node) - anakin_graph.format_edge_from_nodes() - for out_node_name in self.OutPuts: - anakin_graph.add_out('output_'+out_node_name,out_node_name) - return anakin_graph diff --git a/tools/external_converter_v2/parser/kill_tf/run_pb.py b/tools/external_converter_v2/parser/kill_tf/run_pb.py deleted file mode 100644 index 2b6d3a254..000000000 --- a/tools/external_converter_v2/parser/kill_tf/run_pb.py +++ /dev/null @@ -1,74 +0,0 @@ -import numpy as np -from tf_util import TfUtil - -def convert_name_tf2ak(tf_name,perfix='record_'): - ak_name=tf_name[:] - for index,x in enumerate(tf_name): - if x=='/': - ak_name=ak_name[:index]+'_'+ak_name[index+1:] - return perfix+ak_name -ak_work_space='/home/ljj/docker_mount_dev2/anakin2_developing_parser/build/' -# ak_work_space='/your/anakin/workspace' -output_compare_op=None - - -# graph_path='./vgg_model/frozen_vgg_16_i.pb' -# input_name='graph/input:0' -# output_op='graph/vgg_16/fc8/BiasAdd' -# output_compare_op='graph/vgg_16/fc8/convolution' - -graph_path='./resnet_model/frozen_resnet_v1_50.pb' -input_name='graph/input:0' -output_op='graph/resnet_v1_50/predictions/Reshape_1' - -# graph_path='./mobilnetv2/frozen_mobilnet_v2.pb' -# input_name='graph/input:0' -# output_op='graph/MobilenetV2/Predictions/Reshape_1' - -# graph_path='./inception_model/frozen_inception_v2.pb' -# input_name='graph/input:0' -# output_op='graph/InceptionV2/Predictions/Reshape_1' - - -is_compared=True -output_name=output_op+':0' -if output_compare_op is None: - compare_path=ak_work_space+convert_name_tf2ak(output_op) -else: - compare_path=ak_work_space+convert_name_tf2ak(output_compare_op) - -np.random.seed(1234567) -x_location=ak_work_space+'input_x' -x_value=np.random.randn(1,224,224,3) -outs=TfUtil.tf_run_model(graph_path,{input_name:x_value},[output_name]) - -import struct -with open(x_location, "wb") as file: - x_value_new=x_value.transpose((0,3,1,2)) - for value in x_value_new.flatten(): - file.write(struct.pack('f', value)) - -if is_compared: - out=outs[0] - if len(out.shape)==4: - out=out.transpose((0,3,1,2)) - else: - print('out shape :',out.shape) - print(out.flatten()[:10]) - try: - with open(compare_path,'r')as file: - ans=[] - for value_str in file.readlines(): - ans.append(float(value_str.split(' ')[1])) - correct=out.flatten() - assert len(correct)==len(ans) - for i in range(len(ans)): - if abs(ans[i]-correct[i])>0.0001 and abs(ans[i]-correct[i])/abs(correct[i])>0.0001: - print(i,'=',ans[i],'correct = ',correct[i]) - exit() - print('passed') - except Exception,e: - print(out) - print('can`t find file : '+compare_path) - - diff --git a/tools/external_converter_v2/parser/operations/op_io.py b/tools/external_converter_v2/parser/operations/op_io.py index 11539f38e..d7c647bb1 100644 --- a/tools/external_converter_v2/parser/operations/op_io.py +++ b/tools/external_converter_v2/parser/operations/op_io.py @@ -41,7 +41,8 @@ def __call__(self): use_soft_nms=list(), nms_among_classes=bool(), voting=list(), - vote_iou=list()) + vote_iou=list(), + nms_gpu_max_n_per_time=int()) # BBoxRegParameter bbox_reg_param = Dictionary().set_attr(bbox_mean=list(), @@ -189,11 +190,14 @@ def __call__(self): cords_offset_y=float(), bbox_size_add_one=bool(), rotate_coords_by_pitch=bool(), - refine_coords_by_bbox=bool(), - refine_min_dist=float(), - refine_dist_for_height_ratio_one=float(), - max_3d2d_height_ratio_for_min_dist=float(), - with_trunc_ratio=bool()) + #refine_coords_by_bbox=bool(), + #refine_min_dist=float(), + #refine_dist_for_height_ratio_one=float(), + #max_3d2d_height_ratio_for_min_dist=float(), + with_trunc_ratio=bool(), + regress_ph_rh_as_whole=bool(), + real_h_means_as_whole=list(), + real_h_stds_as_whole=list()) # RPNProposalSSD parameter RPNProposalSSD_param = Dictionary().set_attr(detection_output_ssd=detection_output_ssd_param(), bbox_reg=bbox_reg_param()) diff --git a/tools/external_converter_v2/parser/operations/ops.py b/tools/external_converter_v2/parser/operations/ops.py index 854b744e0..118d33909 100755 --- a/tools/external_converter_v2/parser/operations/ops.py +++ b/tools/external_converter_v2/parser/operations/ops.py @@ -355,6 +355,8 @@ begin_norm_axis=int(), eps=float()) +OpsRegister.Register("Resize").set_attr(height_scale=float(), + width_scale=float()) OpsRegister.Register("Normalize").set_attr(begin_norm_axis=int(), is_across_spatial=bool(), @@ -367,4 +369,24 @@ pad_w=list()) -OpsRegister.Register("ShuffleChannel").set_attr(group=int()) \ No newline at end of file +OpsRegister.Register("ShuffleChannel").set_attr(group=int()) + +OpsRegister.Register("RoisAnchorFeature").set_attr(min_anchor_size=float(), + num_anchor_scales=int(), + anchor_scale_pow_base=float(), + anchor_wph_ratios=list(), + num_top_iou_anchor=int(), + min_num_top_iou_anchor=int(), + iou_thr=float(), + ft_ratio_h=bool(), + ft_ratio_w=bool(), + ft_log_ratio_h=bool(), + ft_log_ratio_w=bool(), + bbox_size_add_one=bool()) + +OpsRegister.Register("Interp").set_attr(height=int(), + width=int(), + zoom_factor=int(), + shrink_factor=int(), + pad_beg=int(), + pad_end=int()) diff --git a/tools/external_converter_v2/parser/operations/ops_fluid.py b/tools/external_converter_v2/parser/operations/ops_fluid.py index a802e287c..9025a1dd9 100755 --- a/tools/external_converter_v2/parser/operations/ops_fluid.py +++ b/tools/external_converter_v2/parser/operations/ops_fluid.py @@ -27,8 +27,22 @@ OpsRegister.Register("sequence_conv").set_attr() OpsRegister.Register("stanh").set_attr() - OpsRegister.Register("matmul").set_attr() OpsRegister.Register("layer_norm").set_attr() OpsRegister.Register("dropout").set_attr() OpsRegister.Register("scale").set_attr() +OpsRegister.Register("norm").set_attr() + +OpsRegister.Register("lod_reset").set_attr() +OpsRegister.Register("fill_constant").set_attr() +OpsRegister.Register("lod_rank_table").set_attr() +OpsRegister.Register("max_sequence_len").set_attr() +OpsRegister.Register("less_than").set_attr() +OpsRegister.Register("lod_tensor_to_array").set_attr() +OpsRegister.Register("write_to_array").set_attr() +OpsRegister.Register("reorder_lod_tensor_by_rank").set_attr() +OpsRegister.Register("while").set_attr() +OpsRegister.Register("array_to_lod_tensor").set_attr() + +OpsRegister.Register("assign_value").set_attr() +OpsRegister.Register("shape").set_attr() diff --git a/tools/external_converter_v2/parser/pbs/__init__.py b/tools/external_converter_v2/parser/pbs/__init__.py index 40e5a387c..9e1c738fc 100644 --- a/tools/external_converter_v2/parser/pbs/__init__.py +++ b/tools/external_converter_v2/parser/pbs/__init__.py @@ -1,9 +1,21 @@ #! /usr/bin/env python # Copyright (c) 2017, Cuichaowen. All rights reserved. # -*- coding: utf-8 -*- +import os +from ..logger import verbose +from ..logger import shellcolors +from ..logger import with_color +from ..logger import logger -try: - from caffe_pb2 import * -except ImportError: - raise ImportError(' No module named caffe_pb2 . ') - +for module in os.listdir(os.path.dirname(__file__)): + if module == '__init__.py' or module[-3:] != '.py': + continue + m = __import__(module[:-3], locals(), globals()) + try: + attrlist = m.__all__ + except AttributeError: + attrlist = dir(m) + for attr in attrlist: + globals()[attr] = getattr(m, attr) + logger(verbose.INFO).feed("Import Module: ", module[:-3]) +del module diff --git a/tools/external_converter_v2/parser/kill_tf/__init__.py b/tools/external_converter_v2/parser/tensorflow/__init__.py similarity index 100% rename from tools/external_converter_v2/parser/kill_tf/__init__.py rename to tools/external_converter_v2/parser/tensorflow/__init__.py diff --git a/tools/external_converter_v2/parser/tensorflow/auto_debug.py b/tools/external_converter_v2/parser/tensorflow/auto_debug.py new file mode 100644 index 000000000..3865666a4 --- /dev/null +++ b/tools/external_converter_v2/parser/tensorflow/auto_debug.py @@ -0,0 +1,158 @@ +import tensorflow as tf +import numpy as np +import os +import tempfile +from parse_tf_2_med import ParseTF2Med +from med_graph import MedGraphUtil +from tf_trans_util import load_graph +from tf_util import TfUtil + + +class AutoDebug: + + def __init__(self, tf_model_path, ak_output_dir, workspace=None): + self.ak_perfix = 'record_' + self.tf_model_path = tf_model_path + self.ak_output_dir = os.path.dirname(ak_output_dir) + med_graph = ParseTF2Med(self.tf_model_path).parse() + MedGraphUtil.solve(med_graph) + self.med_graph = med_graph + # if workspace is None: + # self.workspace=tempfile.mkdtemp() + # else: + # self.workspace = workspace + # print('workspace is ', self.workspace) + + def _debug_graph(self, graph): + ''' + print debug info + :param graph: + :return: + ''' + for i in graph.values(): + print(i) + exit() + + def _convert_name_tf2ak(self, tf_name): + ''' + conver name from tf 2 ak + :param tf_name: + :return: + ''' + ak_name = tf_name[:] + for index, x in enumerate(tf_name): + if x == '/': + ak_name = ak_name[:index] + '_' + ak_name[index + 1:] + return self.ak_perfix + ak_name + + def _find_ak_output(self): + ''' + find output name in graph + :return: + ''' + result = {} + for file_name in os.listdir(self.ak_output_dir): + real_path = os.path.join(self.ak_output_dir, file_name) + if not os.path.isdir(real_path) and file_name.startswith(self.ak_perfix) and not file_name.startswith( + self.ak_perfix + '_'): + result[file_name] = [float(i.split(' ')[1]) for i in open(real_path, 'r').readlines()] + return result + + def _creat_and_clear_workspace(self): + ''' + remove temp workspace + :return: + ''' + if os.path.exists(self.workspace): + assert os.path.isdir(self.workspace) + ls = os.listdir(self.workspace) + for i in ls: + c_path = os.path.join(self.workspace, i) + os.remove(c_path) + else: + os.mkdir(self.workspace) + + def _prepare_data(self, med_graph): + ''' + gen input data + :param med_graph: + :return: + ''' + input_cnt = 0 + inputs = {} + result = {} + np.random.seed(1234567) + for i in med_graph.values(): + if i['ak_type'] == 'Input': + shape = i['output'][0]['shape'] + size = 1 + for k in shape: + size *= k + tensor = np.random.randn(size).reshape(shape) + inputs['input_' + str(input_cnt)] = tensor + result[i['name'] + ':0'] = tensor + input_cnt += 1 + + for i in inputs.keys(): + import struct + with open(os.path.join(self.ak_output_dir, i), "wb") as file: + x_value_new = inputs[i].transpose((0, 3, 1, 2)) + for value in x_value_new.flatten(): + file.write(struct.pack('f', value)) + + return result + + def run_ak(self): + ''' + run ak mode + :return: + ''' + self._prepare_data(self.med_graph) + + def run(self): + ''' + run debug mode + :return: + ''' + pass + # inputs = self._prepare_data(self.med_graph) + # akopname2ak_out=self._find_ak_output() + # for i in akopname2ak_out: + # print(i) + # compare_dic={} + # for node in self.med_graph.values(): + # print(node['name'], node['ak_type']) + # if node['ak_type'] is not None and node['type'] is not None: + # + # name=node['name'] + # if node.get('fusion_out_name') is not None: + # name=node['fusion_out_name'] + # ak_name=self._convert_name_tf2ak(name) + # print('key ',ak_name) + # if akopname2ak_out.get(ak_name) is not None: + # compare_dic[name+':0']=akopname2ak_out[ak_name] + # outlist=compare_dic.keys() + # out=TfUtil.tf_run_model(self.tf_model_path,inputs,outlist) + # for index,name in enumerate(outlist): + # correct=out[index].transpose() + # ak_result=akopname2ak_out[name] + # assert len() + # for i in range(len(ak_result)): + # if abs() + # tensor_shape = {} + # graph={} + # for node in tf_ops: + # for out in node.outputs: + # tensor_shape[out.name] = out.get_shape().as_list() + # input_names = [i.name for i in node.inputs] + # output_names = [i.name for i in node.outputs] + # graph[node.name]={'name':node.name,'input':input_names,'output':output_names} + # self._fix_self_output(graph,tensor_shape) + # self._debug_graph(graph) + + +ak_work_space = '' + +debug = AutoDebug('./resnet_model/frozen_resnet_v1_50.pb', ak_work_space) +debug.run_ak() +debug.run() diff --git a/tools/external_converter_v2/parser/kill_tf/freeze.py b/tools/external_converter_v2/parser/tensorflow/freeze.py similarity index 94% rename from tools/external_converter_v2/parser/kill_tf/freeze.py rename to tools/external_converter_v2/parser/tensorflow/freeze.py index bc5b77631..c45ccba51 100644 --- a/tools/external_converter_v2/parser/kill_tf/freeze.py +++ b/tools/external_converter_v2/parser/tensorflow/freeze.py @@ -6,7 +6,13 @@ dir = os.path.dirname(os.path.realpath(__file__)) -def freeze_graph(model_folder,output_name): +def freeze_graph(model_folder, output_name): + ''' + freeze graph from meta input + :param model_folder: + :param output_name: + :return: + ''' # We retrieve our checkpoint fullpath checkpoint = tf.train.get_checkpoint_state(model_folder) input_checkpoint = checkpoint.model_checkpoint_path @@ -59,4 +65,3 @@ def freeze_graph(model_folder,output_name): # freeze_graph('/tmp/pycharm_project_635/tensorflow/RNN/model/language_model_tf','Softmax') freeze_graph('./ease_model/', 'add_2') - diff --git a/tools/external_converter_v2/parser/tensorflow/freeze_graph.py b/tools/external_converter_v2/parser/tensorflow/freeze_graph.py new file mode 100644 index 000000000..d6e369f4d --- /dev/null +++ b/tools/external_converter_v2/parser/tensorflow/freeze_graph.py @@ -0,0 +1,416 @@ +# Copyright 2015 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +r"""Converts checkpoint variables into Const ops in a standalone GraphDef file. + +This script is designed to take a GraphDef proto, a SaverDef proto, and a set of +variable values stored in a checkpoint file, and output a GraphDef with all of +the variable ops converted into const ops containing the values of the +variables. + +It's useful to do this when we need to load a single file in C++, especially in +environments like mobile or embedded where we may not have access to the +RestoreTensor ops and file loading calls that they rely on. + +An example of command-line usage is: +bazel build tensorflow/python/tools:freeze_graph && \ +bazel-bin/tensorflow/python/tools/freeze_graph \ +--input_graph=some_graph_def.pb \ +--input_checkpoint=model.ckpt-8361242 \ +--output_graph=/tmp/frozen_graph.pb --output_node_names=softmax + +You can also look at freeze_graph_test.py for an example of how to use it. + +""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +import argparse +import re +import sys + +from google.protobuf import text_format + +from tensorflow.core.framework import graph_pb2 +from tensorflow.core.protobuf import saver_pb2 +from tensorflow.core.protobuf.meta_graph_pb2 import MetaGraphDef +from tensorflow.python import pywrap_tensorflow +from tensorflow.python.client import session +from tensorflow.python.framework import graph_util +from tensorflow.python.framework import importer +from tensorflow.python.platform import app +from tensorflow.python.platform import gfile +from tensorflow.python.saved_model import loader +from tensorflow.python.saved_model import tag_constants +from tensorflow.python.tools import saved_model_utils +from tensorflow.python.training import saver as saver_lib + + +def freeze_graph_with_def_protos(input_graph_def, + input_saver_def, + input_checkpoint, + output_node_names, + restore_op_name, + filename_tensor_name, + output_graph, + clear_devices, + initializer_nodes, + variable_names_whitelist="", + variable_names_blacklist="", + input_meta_graph_def=None, + input_saved_model_dir=None, + saved_model_tags=None, + checkpoint_version=saver_pb2.SaverDef.V2): + """Converts all variables in a graph and checkpoint into constants.""" + del restore_op_name, filename_tensor_name # Unused by updated loading code. + + if not output_node_names: + print("You need to supply the name of a node to --output_node_names.") + return -1 + + # Remove all the explicit device specifications for this node. This helps to + # make the graph more portable. + if clear_devices: + if input_meta_graph_def: + for node in input_meta_graph_def.graph_def.node: + node.device = "" + elif input_graph_def: + for node in input_graph_def.node: + node.device = "" + + if input_graph_def: + _ = importer.import_graph_def(input_graph_def, name="") + with session.Session() as sess: + if input_saver_def: + saver = saver_lib.Saver( + saver_def=input_saver_def, write_version=checkpoint_version) + saver.restore(sess, input_checkpoint) + elif input_meta_graph_def: + restorer = saver_lib.import_meta_graph( + input_meta_graph_def, clear_devices=True) + restorer.restore(sess, input_checkpoint) + if initializer_nodes: + sess.run(initializer_nodes.replace(" ", "").split(",")) + elif input_saved_model_dir: + if saved_model_tags is None: + saved_model_tags = [] + loader.load(sess, saved_model_tags, input_saved_model_dir) + else: + var_list = {} + reader = pywrap_tensorflow.NewCheckpointReader(input_checkpoint) + var_to_shape_map = reader.get_variable_to_shape_map() + + # List of all partition variables. Because the condition is heuristic + # based, the list could include false positives. + all_parition_variable_names = [ + tensor.name.split(":")[0] + for op in sess.graph.get_operations() + for tensor in op.values() + if re.search(r"/part_\d+/", tensor.name) + ] + has_partition_var = False + + for key in var_to_shape_map: + try: + tensor = sess.graph.get_tensor_by_name(key + ":0") + if any(key in name for name in all_parition_variable_names): + has_partition_var = True + except KeyError: + # This tensor doesn't exist in the graph (for example it's + # 'global_step' or a similar housekeeping element) so skip it. + continue + var_list[key] = tensor + + try: + saver = saver_lib.Saver( + var_list=var_list, write_version=checkpoint_version) + except TypeError as e: + # `var_list` is required to be a map of variable names to Variable + # tensors. Partition variables are Identity tensors that cannot be + # handled by Saver. + if has_partition_var: + print("Models containing partition variables cannot be converted " + "from checkpoint files. Please pass in a SavedModel using " + "the flag --input_saved_model_dir.") + return -1 + else: + raise e + + saver.restore(sess, input_checkpoint) + if initializer_nodes: + sess.run(initializer_nodes.replace(" ", "").split(",")) + + variable_names_whitelist = ( + variable_names_whitelist.replace(" ", "").split(",") + if variable_names_whitelist else None) + variable_names_blacklist = ( + variable_names_blacklist.replace(" ", "").split(",") + if variable_names_blacklist else None) + + if input_meta_graph_def: + output_graph_def = graph_util.convert_variables_to_constants( + sess, + input_meta_graph_def.graph_def, + output_node_names.replace(" ", "").split(","), + variable_names_whitelist=variable_names_whitelist, + variable_names_blacklist=variable_names_blacklist) + else: + output_graph_def = graph_util.convert_variables_to_constants( + sess, + input_graph_def, + output_node_names.replace(" ", "").split(","), + variable_names_whitelist=variable_names_whitelist, + variable_names_blacklist=variable_names_blacklist) + + # Write GraphDef to file if output path has been given. + if output_graph: + with gfile.GFile(output_graph, "wb") as f: + f.write(output_graph_def.SerializeToString()) + + return output_graph_def + + +def _parse_input_graph_proto(input_graph, input_binary): + """Parser input tensorflow graph into GraphDef proto.""" + if not gfile.Exists(input_graph): + print("Input graph file '" + input_graph + "' does not exist!") + return -1 + input_graph_def = graph_pb2.GraphDef() + mode = "rb" if input_binary else "r" + with gfile.FastGFile(input_graph, mode) as f: + if input_binary: + input_graph_def.ParseFromString(f.read()) + else: + text_format.Merge(f.read(), input_graph_def) + return input_graph_def + + +def _parse_input_meta_graph_proto(input_graph, input_binary): + """Parser input tensorflow graph into MetaGraphDef proto.""" + if not gfile.Exists(input_graph): + print("Input meta graph file '" + input_graph + "' does not exist!") + return -1 + input_meta_graph_def = MetaGraphDef() + mode = "rb" if input_binary else "r" + with gfile.FastGFile(input_graph, mode) as f: + if input_binary: + input_meta_graph_def.ParseFromString(f.read()) + else: + text_format.Merge(f.read(), input_meta_graph_def) + print("Loaded meta graph file '" + input_graph) + return input_meta_graph_def + + +def _parse_input_saver_proto(input_saver, input_binary): + """Parser input tensorflow Saver into SaverDef proto.""" + if not gfile.Exists(input_saver): + print("Input saver file '" + input_saver + "' does not exist!") + return -1 + mode = "rb" if input_binary else "r" + with gfile.FastGFile(input_saver, mode) as f: + saver_def = saver_pb2.SaverDef() + if input_binary: + saver_def.ParseFromString(f.read()) + else: + text_format.Merge(f.read(), saver_def) + return saver_def + + +def freeze_graph(input_graph, + input_saver, + input_binary, + input_checkpoint, + output_node_names, + restore_op_name, + filename_tensor_name, + output_graph, + clear_devices, + initializer_nodes, + variable_names_whitelist="", + variable_names_blacklist="", + input_meta_graph=None, + input_saved_model_dir=None, + saved_model_tags=tag_constants.SERVING, + checkpoint_version=saver_pb2.SaverDef.V2): + """Converts all variables in a graph and checkpoint into constants.""" + input_graph_def = None + if input_saved_model_dir: + input_graph_def = saved_model_utils.get_meta_graph_def( + input_saved_model_dir, saved_model_tags).graph_def + elif input_graph: + input_graph_def = _parse_input_graph_proto(input_graph, input_binary) + input_meta_graph_def = None + if input_meta_graph: + input_meta_graph_def = _parse_input_meta_graph_proto( + input_meta_graph, input_binary) + input_saver_def = None + if input_saver: + input_saver_def = _parse_input_saver_proto(input_saver, input_binary) + freeze_graph_with_def_protos( + input_graph_def, + input_saver_def, + input_checkpoint, + output_node_names, + restore_op_name, + filename_tensor_name, + output_graph, + clear_devices, + initializer_nodes, + variable_names_whitelist, + variable_names_blacklist, + input_meta_graph_def, + input_saved_model_dir, + saved_model_tags.replace(" ", "").split(","), + checkpoint_version=checkpoint_version) + + +def main(unused_args, flags): + ''' + main function + :param unused_args: + :param flags: + :return: + ''' + if flags.checkpoint_version == 1: + checkpoint_version = saver_pb2.SaverDef.V1 + elif flags.checkpoint_version == 2: + checkpoint_version = saver_pb2.SaverDef.V2 + else: + print("Invalid checkpoint version (must be '1' or '2'): %d" % + flags.checkpoint_version) + return -1 + freeze_graph(flags.input_graph, flags.input_saver, flags.input_binary, + flags.input_checkpoint, flags.output_node_names, + flags.restore_op_name, flags.filename_tensor_name, + flags.output_graph, flags.clear_devices, flags.initializer_nodes, + flags.variable_names_whitelist, flags.variable_names_blacklist, + flags.input_meta_graph, flags.input_saved_model_dir, + flags.saved_model_tags, checkpoint_version) + + +def run_main(): + ''' + excution entry + :return: + ''' + parser = argparse.ArgumentParser() + parser.register("type", "bool", lambda v: v.lower() == "true") + parser.add_argument( + "--input_graph", + type=str, + default="", + help="TensorFlow \'GraphDef\' file to load.") + parser.add_argument( + "--input_saver", + type=str, + default="", + help="TensorFlow saver file to load.") + parser.add_argument( + "--input_checkpoint", + type=str, + default="", + help="TensorFlow variables file to load.") + parser.add_argument( + "--checkpoint_version", + type=int, + default=2, + help="Tensorflow variable file format") + parser.add_argument( + "--output_graph", + type=str, + default="", + help="Output \'GraphDef\' file name.") + parser.add_argument( + "--input_binary", + nargs="?", + const=True, + type="bool", + default=False, + help="Whether the input files are in binary format.") + parser.add_argument( + "--output_node_names", + type=str, + default="", + help="The name of the output nodes, comma separated.") + parser.add_argument( + "--restore_op_name", + type=str, + default="save/restore_all", + help="""\ + The name of the master restore operator. Deprecated, unused by updated \ + loading code. + """) + parser.add_argument( + "--filename_tensor_name", + type=str, + default="save/Const:0", + help="""\ + The name of the tensor holding the save path. Deprecated, unused by \ + updated loading code. + """) + parser.add_argument( + "--clear_devices", + nargs="?", + const=True, + type="bool", + default=True, + help="Whether to remove device specifications.") + parser.add_argument( + "--initializer_nodes", + type=str, + default="", + help="Comma separated list of initializer nodes to run before freezing.") + parser.add_argument( + "--variable_names_whitelist", + type=str, + default="", + help="""\ + Comma separated list of variables to convert to constants. If specified, \ + only those variables will be converted to constants.\ + """) + parser.add_argument( + "--variable_names_blacklist", + type=str, + default="", + help="""\ + Comma separated list of variables to skip converting to constants.\ + """) + parser.add_argument( + "--input_meta_graph", + type=str, + default="", + help="TensorFlow \'MetaGraphDef\' file to load.") + parser.add_argument( + "--input_saved_model_dir", + type=str, + default="", + help="Path to the dir with TensorFlow \'SavedModel\' file and variables.") + parser.add_argument( + "--saved_model_tags", + type=str, + default="serve", + help="""\ + Group of tag(s) of the MetaGraphDef to load, in string format,\ + separated by \',\'. For tag-set contains multiple tags, all tags \ + must be passed in.\ + """) + flags, unparsed = parser.parse_known_args() + + my_main = lambda unused_args: main(unused_args, flags) + app.run(main=my_main, argv=[sys.argv[0]] + unparsed) + + +if __name__ == '__main__': + run_main() diff --git a/tools/external_converter_v2/parser/kill_tf/get_empt_infer_pb.py b/tools/external_converter_v2/parser/tensorflow/get_empt_infer_pb.py similarity index 76% rename from tools/external_converter_v2/parser/kill_tf/get_empt_infer_pb.py rename to tools/external_converter_v2/parser/tensorflow/get_empt_infer_pb.py index 16b9b0f06..11be45be9 100644 --- a/tools/external_converter_v2/parser/kill_tf/get_empt_infer_pb.py +++ b/tools/external_converter_v2/parser/tensorflow/get_empt_infer_pb.py @@ -4,11 +4,11 @@ from tensorflow.python.platform import gfile from google.protobuf import text_format -graph_path='./inception_model/inception_v2_inf_graph.pb' -output_node_names='InceptionV2/Predictions/Reshape_1' -output_graph='./inception_model/inception_v2_inf_graph_empty.pb' +graph_path = './inception_model/inception_v2_inf_graph.pb' +output_node_names = 'InceptionV2/Predictions/Reshape_1' +output_graph = './inception_model/inception_v2_inf_graph_empty.pb' -sess=tf.Session() +sess = tf.Session() if graph_path.endswith('.pbtxt'): input_binary = False else: @@ -29,11 +29,11 @@ tf.import_graph_def(graph_def, name='graph') output_graph_def = graph_util.convert_variables_to_constants( - sess, - graph_def, - output_node_names.split(",") # We split on comma for convenience - ) + sess, + graph_def, + output_node_names.split(",") # We split on comma for convenience +) # Finally we serialize and dump the output graph to the filesystem with tf.gfile.GFile(output_graph, "wb") as f: f.write(output_graph_def.SerializeToString()) -print("%d ops in the final graph." % len(output_graph_def.node)) \ No newline at end of file +print("%d ops in the final graph." % len(output_graph_def.node)) diff --git a/tools/external_converter_v2/parser/kill_tf/med_graph.py b/tools/external_converter_v2/parser/tensorflow/med_graph.py similarity index 58% rename from tools/external_converter_v2/parser/kill_tf/med_graph.py rename to tools/external_converter_v2/parser/tensorflow/med_graph.py index aacbc5e15..d17cc8702 100644 --- a/tools/external_converter_v2/parser/kill_tf/med_graph.py +++ b/tools/external_converter_v2/parser/tensorflow/med_graph.py @@ -1,15 +1,26 @@ import numpy as np - class MedNodeUtil: @staticmethod def new_med_node(): - return {'name':None,'ak_type': None, 'input': [], 'output': [], 'ak_attr': {},'type':None,'med_visted':False} + ''' + return instance of empty standard med graph node + :return: + ''' + return {'name': None, 'ak_type': None, 'input': [], 'output': [], 'ak_attr': {}, 'type': None, + 'med_visted': False} @staticmethod def replace_name_with_list(origin_list, name, replace_list): + ''' + replace name in input or output with replace_list + :param origin_list: + :param name: + :param replace_list: + :return: + ''' new_list = [] for index, ori_object in enumerate(origin_list): if ori_object['name'] == name: @@ -23,6 +34,12 @@ def replace_name_with_list(origin_list, name, replace_list): @staticmethod def retain_input(node, input_list): + ''' + remove node input except element in input_list + :param node: + :param input_list: + :return: + ''' new_input = [] for index, node_object in enumerate(node['input']): if node_object in input_list: @@ -31,17 +48,35 @@ def retain_input(node, input_list): node['input'] = new_input @staticmethod - def redirecto_outputs_input_to_this(node,graph,this_name,this_shape): + def redirecto_outputs_input_to_this(node, graph, this_name, this_shape): + ''' + get node_x in node`s outputs + make node_x`s inputs reference to node + :param node: + :param graph: + :param this_name: + :param this_shape: + :return: + ''' for i in node['output']: - tar_node=graph[i['name']] - tar_node['input']=MedNodeUtil.replace_name_with_list(tar_node['input'],node['name'],[{'name':this_name,'shape':this_shape}]) + tar_node = graph[i['name']] + tar_node['input'] = MedNodeUtil.replace_name_with_list(tar_node['input'], node['name'], + [{'name': this_name, 'shape': this_shape}]) + +MedGraph_Input_Cnt = 0 -MedGraph_Input_Cnt=0 class MedGraphUtil: @staticmethod def append_node(father_node, son_node, graph): + ''' + add the son_node after father_node in graph + :param father_node: + :param son_node: + :param graph: + :return: + ''' output = father_node['output'] son_shape = father_node['output'][0]['shape'] son_node['input'] = [{'name': father_node['name'], 'shape': son_shape}] @@ -54,7 +89,13 @@ def append_node(father_node, son_node, graph): graph[son_node['name']] = son_node @staticmethod - def check_one_of_input_is_const(node,graph): + def check_one_of_input_is_const(node, graph): + ''' + check one of input is const + :param node: + :param graph: + :return: + ''' for i in node['input']: if graph[i['name']]['type'] == 'Const': return True @@ -62,26 +103,45 @@ def check_one_of_input_is_const(node,graph): @staticmethod def _auto_split(med_node, med_graph): + ''' + add split to node which output size >=2 + :param med_node: + :param med_graph: + :return: + ''' output = med_node['output'] if len(output) > 1: split_node = MedNodeUtil.new_med_node() - split_node['name']=med_node['name']+'_split#' - split_node['ak_type']='Split' - split_node['ak_attr']['split_num'] =len(output) - MedGraphUtil.append_node(med_node,split_node,graph=med_graph) + split_node['name'] = med_node['name'] + '_split#' + split_node['ak_type'] = 'Split' + split_node['ak_attr']['split_num'] = len(output) + MedGraphUtil.append_node(med_node, split_node, graph=med_graph) pass @staticmethod def _auto_input_name(med_node, med_graph): - assert med_node['ak_type']=='Input' - old_name=med_node['name'] - med_node['name']='input_'+str(MedGraph_Input_Cnt) + ''' + gen input name + :param med_node: + :param med_graph: + :return: + ''' + assert med_node['ak_type'] == 'Input' + old_name = med_node['name'] + med_node['name'] = 'input_' + str(MedGraph_Input_Cnt) for i in med_node['output']: - out_node=med_graph[i['name']] - out_node['input']=MedNodeUtil.replace_name_with_list(out_node['input'],old_name,[{'name':med_node['name'],'shape':i['shape']}]) + out_node = med_graph[i['name']] + out_node['input'] = MedNodeUtil.replace_name_with_list(out_node['input'], old_name, + [{'name': med_node['name'], 'shape': i['shape']}]) @staticmethod def _fusionScale(med_node, med_graph): + ''' + fusion scale node after convolution node + :param med_node: + :param med_graph: + :return: + ''' if len(med_node['input']) == 1: input_node = med_graph[med_node['input'][0]['name']] med_ak_attr = med_node['ak_attr'] @@ -100,14 +160,22 @@ def _fusionScale(med_node, med_graph): else: input_attr['bias_weights'] = med_ak_attr['bias_weights'] med_node['ak_type'] = None - input_node['output'] = MedNodeUtil.replace_name_with_list(input_node['output'], med_node['name'], med_node['output']) - MedNodeUtil.redirecto_outputs_input_to_this(med_node,med_graph,input_node['name'],med_node['input'][0]['shape']) - input_node['fusion_out_name']=med_node['name'] + input_node['output'] = MedNodeUtil.replace_name_with_list(input_node['output'], med_node['name'], + med_node['output']) + MedNodeUtil.redirecto_outputs_input_to_this(med_node, med_graph, input_node['name'], + med_node['input'][0]['shape']) + input_node['fusion_out_name'] = med_node['name'] pass @staticmethod def _all_search_table(graph, table): + ''' + search template for dict + :param graph: + :param table: + :return: + ''' for tf_node in graph.values(): if tf_node['med_visted']: continue @@ -117,6 +185,12 @@ def _all_search_table(graph, table): @staticmethod def _all_search_fusion(graph, fusion_func): + ''' + search template for func + :param graph: + :param fusion_func: + :return: + ''' for tf_node in graph.values(): if tf_node['med_visted']: continue @@ -125,14 +199,24 @@ def _all_search_fusion(graph, fusion_func): @staticmethod def solve(med_graph): + ''' + do fusion and adjust for med graph that we can convert med graph to ak graph + :param med_graph: + :return: + ''' for node in med_graph.values(): node['med_visted'] = False MedGraphUtil._all_search_table(med_graph, {'Scale': MedGraphUtil._fusionScale}) MedGraphUtil._all_search_fusion(med_graph, MedGraphUtil._auto_split) - MedGraphUtil._all_search_table(med_graph, {'Input':MedGraphUtil._auto_input_name}) + MedGraphUtil._all_search_table(med_graph, {'Input': MedGraphUtil._auto_input_name}) @staticmethod def search_output_list(graph): + ''' + search output list in recursive method + :param graph: + :return: + ''' output_list = set() graph_cp = graph.copy() diff --git a/tools/external_converter_v2/parser/kill_tf/parse_med_2_ak.py b/tools/external_converter_v2/parser/tensorflow/parse_med_2_ak.py similarity index 69% rename from tools/external_converter_v2/parser/kill_tf/parse_med_2_ak.py rename to tools/external_converter_v2/parser/tensorflow/parse_med_2_ak.py index fa0df3c7a..2cc53107e 100644 --- a/tools/external_converter_v2/parser/kill_tf/parse_med_2_ak.py +++ b/tools/external_converter_v2/parser/tensorflow/parse_med_2_ak.py @@ -4,11 +4,21 @@ def shape_2_ak_shape(shape): + ''' + convert med shape to ak shape + :param shape: + :return: + ''' mini_shape = [i for i in shape if (i != None and i > 0)] return map(int, [1] * (4 - len(mini_shape)) + list(mini_shape)) def np_2_ak_tensor(np_tensor): + ''' + conver med tensor to ak tensor + :param np_tensor: + :return: + ''' data_type_map = { np.dtype('float32'): 'float', np.dtype('int32'): 'int', @@ -29,6 +39,12 @@ def __init__(self): self.input_count = 0 def Convolution(self, med_attr, param): + ''' + fill convlution param in ak graph + :param med_attr: + :param param: + :return: + ''' np_filters = med_attr['weights'] param.weight_1 = np_2_ak_tensor(np_filters) param.filter_num = np_filters.shape[0] @@ -48,6 +64,12 @@ def Convolution(self, med_attr, param): pass def Dense(self, med_attr, param): + ''' + fill Dense param in ak graph + :param med_attr: + :param param: + :return: + ''' param.weight_1 = np_2_ak_tensor(med_attr['weights']) param.axis = 1 if med_attr.get('bias_weights') is not None: @@ -58,18 +80,35 @@ def Dense(self, med_attr, param): pass def Relu(self, med_attr, param): + ''' + fill Relu param in ak graph + :param med_attr: + :param param: + :return: + ''' if med_attr.get('alpha') is None: param.alpha = 0.0 else: param.alpha = med_attr['type'] def Activation(self, med_attr, param): + ''' + fill Activation param in ak graph + :param med_attr: + :param param: + :return: + ''' param.type = med_attr['type'] - if med_attr['type']=='ClippedRelu': - param.clip_relu_num=med_attr['clip_relu_num'] - + if med_attr['type'] == 'ClippedRelu': + param.clip_relu_num = med_attr['clip_relu_num'] def Softmax(self, med_attr, param): + ''' + fill Softmax param in ak graph + :param med_attr: + :param param: + :return: + ''' if med_attr.get('axis') is None: param.axis = 3 else: @@ -77,17 +116,41 @@ def Softmax(self, med_attr, param): pass def Concat(self, med_attr, param): + ''' + fill Concat param in ak graph + :param med_attr: + :param param: + :return: + ''' param.axis = med_attr['axis'] def Split(self, med_attr, param): + ''' + fill Split param in ak graph + :param med_attr: + :param param: + :return: + ''' param.split_num = med_attr['split_num'] def Eltwise(self, med_attr, param): + ''' + fill Eltwise param in ak graph + :param med_attr: + :param param: + :return: + ''' assert med_attr['type'] == 'Add' param.type = med_attr['type'] param.coeff = [1.0, 1.0] def Scale(self, med_attr, param): + ''' + fill Scale param in ak graph + :param med_attr: + :param param: + :return: + ''' param.weight_1 = np_2_ak_tensor(med_attr['scale_weights']) if med_attr.get('bias_weights') is not None: param.weight_2 = np_2_ak_tensor(med_attr['bias_weights']) @@ -98,6 +161,12 @@ def Scale(self, med_attr, param): param.num_axes = 1 def Reshape(self, med_attr, param): + ''' + fill Reshape param in ak graph + :param med_attr: + :param param: + :return: + ''' shape = med_attr['shape'] if isinstance(shape, type(np.array([]))): shape = [int(i) for i in shape] @@ -105,6 +174,12 @@ def Reshape(self, med_attr, param): pass def Pooling(self, med_attr, param): + ''' + fill Pooling param in ak graph + :param med_attr: + :param param: + :return: + ''' param.method = med_attr['type'] param.pool_size = med_attr['window'] param.strides = med_attr['strides'] @@ -121,19 +196,35 @@ def Pooling(self, med_attr, param): pass - def Pad(self, med_attr, param): - param.pad_c=med_attr['pad_c'] + ''' + fill Pad param in ak graph + :param med_attr: + :param param: + :return: + ''' + param.pad_c = med_attr['pad_c'] param.pad_h = med_attr['pad_h'] param.pad_w = med_attr['pad_w'] - def Input(self, med_attr, param): + ''' + fill Input param in ak graph + :param med_attr: + :param param: + :return: + ''' param.input_shape = shape_2_ak_shape(med_attr['shape']) if med_attr.get('alias') is not None: param.alias = med_attr['alias'] def map_med_2_ak(self, ak_node, med_node): + ''' + entrance for trans med graph 2 ak graph + :param ak_node: + :param med_node: + :return: + ''' type_name = med_node['ak_type'] func = getattr(self, type_name, None) param = OpsParam() diff --git a/tools/external_converter_v2/parser/kill_tf/parse_tf_2_med.py b/tools/external_converter_v2/parser/tensorflow/parse_tf_2_med.py similarity index 73% rename from tools/external_converter_v2/parser/kill_tf/parse_tf_2_med.py rename to tools/external_converter_v2/parser/tensorflow/parse_tf_2_med.py index 14705878b..6f26d8a55 100644 --- a/tools/external_converter_v2/parser/kill_tf/parse_tf_2_med.py +++ b/tools/external_converter_v2/parser/tensorflow/parse_tf_2_med.py @@ -4,23 +4,26 @@ import logging as log import collections from tf_trans_util import * -class ParseTF2Med: - def __init__(self,tf_forzen_pb_path): - self.model_path=tf_forzen_pb_path - def _debug_nodes(self,nodes): + +class ParseTF2Med: + def __init__(self, tf_forzen_pb_path): + self.model_path = tf_forzen_pb_path + + def _debug_nodes(self, nodes): + ''' + print debug info and exit + :param nodes: + :return: + ''' for i in nodes.values(): - print(i['name'],i['input'],i['output'],i['out_shape']) + print(i['name'], i['input'], i['output'], i['out_shape']) print('debug end') exit() - - def _parse_tf_node(self,tf_graph,shape_override): - - + def _parse_tf_node(self, tf_graph, shape_override): """ - Load tensorflow graph into an onnx graph with minimal rewrites so - we can use the onnx graph as intermediate graph. + Load tensorflow graph into an raw med graph """ # ignore the following attributes @@ -37,11 +40,10 @@ def _parse_tf_node(self,tf_graph,shape_override): # find outputs ops = tf_graph.get_operations() - tensor_shape={} + tensor_shape = {} for node in ops: for out in node.outputs: - tensor_shape[out.name]=out.get_shape().as_list() - + tensor_shape[out.name] = out.get_shape().as_list() # minimal conversion of attributes for node in ops: @@ -50,7 +52,7 @@ def _parse_tf_node(self,tf_graph,shape_override): op_cnt[node.type] += 1 for a in node.node_def.attr: - a=str(a) + a = str(a) attr_cnt[a] += 1 if a == "dtype": attr[a] = map_tf_dtype(node.get_attr("dtype")) @@ -92,11 +94,17 @@ def _parse_tf_node(self,tf_graph,shape_override): log.error("pass1 convert failed for %s, ex=%s", node, ex) raise - self._fix_self_output(anakin_nodes,tensor_shape) + self._fix_self_output(anakin_nodes, tensor_shape) return anakin_nodes def _fix_self_output(self, nodes, tensor_shape_dict): + ''' + convert tensor connection to op connection + :param nodes: + :param tensor_shape_dict: + :return: + ''' out2nodename = {} for node in nodes.values(): for out_name in node['output']: @@ -114,23 +122,28 @@ def _fix_self_output(self, nodes, tensor_shape_dict): in2nodename[in_name].append(node['name']) for node in nodes.values(): - new_output=[] - new_input=[] + new_output = [] + new_input = [] for tensor_name in node['output']: if in2nodename.get(tensor_name) is not None: - new_output+=[{'name':op_name,'shape':tensor_shape_dict[tensor_name]} for op_name in in2nodename[tensor_name]] + new_output += [{'name': op_name, 'shape': tensor_shape_dict[tensor_name]} for op_name in + in2nodename[tensor_name]] for tensor_name in node['input']: if out2nodename.get(tensor_name) is not None: - new_input+=[{'name':op_name,'shape':tensor_shape_dict[tensor_name]} for op_name in out2nodename[tensor_name]] - - node['output']=new_output - node['input']=new_input - + new_input += [{'name': op_name, 'shape': tensor_shape_dict[tensor_name]} for op_name in + out2nodename[tensor_name]] + node['output'] = new_output + node['input'] = new_input - def _parse_tf_graph(self,nodes): + def _parse_tf_graph(self, nodes): + ''' + conver op in tf graph to med graph + :param nodes: + :return: + ''' def all_search(graph, table): for tf_node in graph.values(): @@ -145,19 +158,19 @@ def all_search(graph, table): 'Shape': parse_Shape }) - all_search(nodes, {'Reshape': parse_fusionReshape,}) + all_search(nodes, {'Reshape': parse_fusionReshape, }) all_search(nodes, {'MatMul': parse_MatMul, 'Conv2D': parse_Conv2D, 'DepthwiseConv2dNative': parse_Conv2D, - 'FusedBatchNorm':parse_BatchNorm, - 'Rsqrt': parse_CustmerBatchNorm,}) + 'FusedBatchNorm': parse_BatchNorm, + 'Rsqrt': parse_CustmerBatchNorm, }) - all_search(nodes, {'Add':parse_Add, + all_search(nodes, {'Add': parse_Add, 'AvgPool': parse_Pooling, 'ConcatV2': parse_Concat, 'MaxPool': parse_Pooling, - 'Mean':parse_Mean, + 'Mean': parse_Mean, 'Pad': parse_Pad, 'Relu': parse_Act, 'Relu6': parse_Act, @@ -170,12 +183,16 @@ def all_search(graph, table): return nodes def parse(self): + ''' + entrance to load tf graph and convert it to med graph + :return: + ''' tf_graph = load_graph(self.model_path) nodes = self._parse_tf_node(tf_graph, {}) - med_graph=self._parse_tf_graph(nodes) - filter_graph={i:med_graph[i] for i in med_graph.keys() if med_graph[i]['ak_type'] is not None} + med_graph = self._parse_tf_graph(nodes) + filter_graph = {i: med_graph[i] for i in med_graph.keys() if med_graph[i]['ak_type'] is not None} for node in filter_graph.values(): - node['input']=[i for i in node['input'] if filter_graph.get(i['name']) is not None] + node['input'] = [i for i in node['input'] if filter_graph.get(i['name']) is not None] node['output'] = [i for i in node['output'] if filter_graph.get(i['name']) is not None] - return filter_graph \ No newline at end of file + return filter_graph diff --git a/tools/external_converter_v2/parser/tensorflow/parser_tf.py b/tools/external_converter_v2/parser/tensorflow/parser_tf.py new file mode 100644 index 000000000..7e800f2a3 --- /dev/null +++ b/tools/external_converter_v2/parser/tensorflow/parser_tf.py @@ -0,0 +1,72 @@ +import numpy as np +import os +from ..graph_io import * +from ..logger import * +from ..proto import * +import tensorflow as tf +from parse_tf_2_med import ParseTF2Med +from parse_med_2_ak import MedTransAK +from med_graph import MedGraphUtil, MedNodeUtil + + +class TFParser: + + def __init__(self, fluid_config_dict): + # anakin graph model io + # config info + self.ProtoPaths = fluid_config_dict['ProtoPaths'] + + self.OutPuts = fluid_config_dict['OutPuts'] + if self.OutPuts is not None: + self.OutPuts = [i for i in fluid_config_dict['OutPuts'].split(',')] + + self.med_trans_tool = MedTransAK() + self.input_count = 0 + + def __call__(self): + med_graph = self._conver_tf_2_med() + if self.OutPuts is None: + self.OutPuts = MedGraphUtil.search_output_list(med_graph) + + MedGraphUtil.solve(med_graph) + anakin_graph = self._conver_med_2_anakin(med_graph) + return anakin_graph + + def _conver_tf_2_med(self): + ''' + convert tf graph to med graph + :return: + ''' + parser = ParseTF2Med(self.ProtoPaths) + return parser.parse() + + def _add_protonode(self, ak_graph, med_node): + ''' + add protobuf node in ak graph + :param ak_graph: + :param med_node: + :return: + ''' + ak_type = med_node['ak_type'] + if ak_type is None: + return + nodeIO = NodeProtoIO() + nodeIO.set_name(med_node['name']) + self.med_trans_tool.map_med_2_ak(nodeIO, med_node) + ak_graph.add_node(nodeIO()) + if nodeIO().Op.name == 'Input': + ak_graph.add_in(nodeIO().name) + + def _conver_med_2_anakin(self, med_graph): + ''' + convert med graph to ak graph + :param med_graph: + :return: + ''' + anakin_graph = GraphProtoIO() + for node in med_graph.values(): + self._add_protonode(anakin_graph, node) + anakin_graph.format_edge_from_nodes() + for out_node_name in self.OutPuts: + anakin_graph.add_out('output_' + out_node_name, out_node_name) + return anakin_graph diff --git a/tools/external_converter_v2/parser/tensorflow/run_pb.py b/tools/external_converter_v2/parser/tensorflow/run_pb.py new file mode 100644 index 000000000..5b44bc23e --- /dev/null +++ b/tools/external_converter_v2/parser/tensorflow/run_pb.py @@ -0,0 +1,80 @@ +import numpy as np +from tf_util import TfUtil + + +def convert_name_tf2ak(tf_name, perfix='record_'): + ''' + conver tf name to ak name + :param tf_name: + :param perfix: + :return: + ''' + ak_name = tf_name[:] + for index, x in enumerate(tf_name): + if x == '/': + ak_name = ak_name[:index] + '_' + ak_name[index + 1:] + return perfix + ak_name + + +# ak_work_space='/your/anakin/workspace' +output_compare_op = None + +# graph_path='./vgg_model/frozen_vgg_16_i.pb' +# input_name='graph/input:0' +# output_op='graph/vgg_16/fc8/BiasAdd' +# output_compare_op='graph/vgg_16/fc8/convolution' + +graph_path = './resnet_model/frozen_resnet_v1_50.pb' +input_name = 'graph/input:0' +output_op = 'graph/resnet_v1_50/predictions/Reshape_1' + +# graph_path='./mobilnetv2/frozen_mobilnet_v2.pb' +# input_name='graph/input:0' +# output_op='graph/MobilenetV2/Predictions/Reshape_1' + +# graph_path='./inception_model/frozen_inception_v2.pb' +# input_name='graph/input:0' +# output_op='graph/InceptionV2/Predictions/Reshape_1' + + +is_compared = True +output_name = output_op + ':0' +if output_compare_op is None: + compare_path = ak_work_space + convert_name_tf2ak(output_op) +else: + compare_path = ak_work_space + convert_name_tf2ak(output_compare_op) + +np.random.seed(1234567) +x_location = ak_work_space + 'input_x' +x_value = np.random.randn(1, 224, 224, 3) +outs = TfUtil.tf_run_model(graph_path, {input_name: x_value}, [output_name]) + +import struct + +with open(x_location, "wb") as file: + x_value_new = x_value.transpose((0, 3, 1, 2)) + for value in x_value_new.flatten(): + file.write(struct.pack('f', value)) + +if is_compared: + out = outs[0] + if len(out.shape) == 4: + out = out.transpose((0, 3, 1, 2)) + else: + print('out shape :', out.shape) + print(out.flatten()[:10]) + try: + with open(compare_path, 'r')as file: + ans = [] + for value_str in file.readlines(): + ans.append(float(value_str.split(' ')[1])) + correct = out.flatten() + assert len(correct) == len(ans) + for i in range(len(ans)): + if abs(ans[i] - correct[i]) > 0.0001 and abs(ans[i] - correct[i]) / abs(correct[i]) > 0.0001: + print(i, '=', ans[i], 'correct = ', correct[i]) + exit() + print('passed') + except Exception, e: + print(out) + print('can`t find file : ' + compare_path) diff --git a/tools/external_converter_v2/parser/kill_tf/run_pb_get_ckp.py b/tools/external_converter_v2/parser/tensorflow/run_pb_get_ckp.py similarity index 80% rename from tools/external_converter_v2/parser/kill_tf/run_pb_get_ckp.py rename to tools/external_converter_v2/parser/tensorflow/run_pb_get_ckp.py index 38ca29178..d9816c8f8 100644 --- a/tools/external_converter_v2/parser/kill_tf/run_pb_get_ckp.py +++ b/tools/external_converter_v2/parser/tensorflow/run_pb_get_ckp.py @@ -11,12 +11,12 @@ # graph_path='./ease_model/graph.pb' # graph_path='./ease_model/frozen_mnist.pb' # graph_path='./vgg_model/frozen_vgg_16_i.pb' -graph_path='./inception_model/inception_v2_inf_graph.pb' -cpkt_path='./inception_model/' +graph_path = './inception_model/inception_v2_inf_graph.pb' +cpkt_path = './inception_model/' # graph_path='./resnet_model/frozen_resnet_v1_50.pb' -sess=tf.Session() +sess = tf.Session() if graph_path.endswith('.pbtxt'): input_binary = False else: @@ -35,17 +35,17 @@ tf.train.import_meta_graph(graph_path, clear_devices=True) tf.import_graph_def(graph_def, name='graph') -sess=tf.Session(graph=graph) +sess = tf.Session(graph=graph) x = graph.get_tensor_by_name('graph/input:0') y = graph.get_tensor_by_name('graph/InceptionV2/Predictions/Reshape_1:0') for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES): - print(var.name) -init_table=tf.tables_initializer() + print(var.name) +init_table = tf.tables_initializer() sess.run(init_table) -init=tf.global_variables_initializer() +init = tf.global_variables_initializer() sess.run(init) -out=sess.run(y,{x:np.ones((1,224,224,3))}) +out = sess.run(y, {x: np.ones((1, 224, 224, 3))}) saver = tf.train.Saver() -saver.save(sess, cpkt_path+'/model.cpkt') \ No newline at end of file +saver.save(sess, cpkt_path + '/model.cpkt') diff --git a/tools/external_converter_v2/parser/kill_tf/summary.py b/tools/external_converter_v2/parser/tensorflow/summary.py similarity index 84% rename from tools/external_converter_v2/parser/kill_tf/summary.py rename to tools/external_converter_v2/parser/tensorflow/summary.py index 5edcb8358..cc1eac454 100644 --- a/tools/external_converter_v2/parser/kill_tf/summary.py +++ b/tools/external_converter_v2/parser/tensorflow/summary.py @@ -12,15 +12,21 @@ from tensorflow.python.platform import gfile - # graph_path = './resnet_v1_50_graph.pb' # graph_path = './frozen_graph.pb' # graph_path='./vgg_model/frozen_vgg_16_i.pb' # graph_path='./inception_model/frozen_inception_v2.pb' -graph_path='./resnet_model/frozen_resnet_v1_50.pb' +graph_path = './resnet_model/frozen_resnet_v1_50.pb' + + # graph_path='./mobilnetv2/frozen_mobilnet_v2.pb' def get_graph(graph_path): + ''' + get tensor board graph from pb file or meta file + :param graph_path: + :return: + ''' if graph_path.endswith('.pbtxt'): input_binary = False else: @@ -41,5 +47,5 @@ def get_graph(graph_path): return graph -graph=get_graph(graph_path) -summaryWriter = tf.summary.FileWriter('log/', graph) \ No newline at end of file +graph = get_graph(graph_path) +summaryWriter = tf.summary.FileWriter('log/', graph) diff --git a/tools/external_converter_v2/parser/kill_tf/tf_trans_util.py b/tools/external_converter_v2/parser/tensorflow/tf_trans_util.py similarity index 76% rename from tools/external_converter_v2/parser/kill_tf/tf_trans_util.py rename to tools/external_converter_v2/parser/tensorflow/tf_trans_util.py index cc8a48823..581a313bc 100644 --- a/tools/external_converter_v2/parser/kill_tf/tf_trans_util.py +++ b/tools/external_converter_v2/parser/tensorflow/tf_trans_util.py @@ -2,7 +2,7 @@ import numpy as np from tensorflow.core.framework import types_pb2, tensor_pb2 from google.protobuf import text_format -from med_graph import MedNodeUtil,MedGraphUtil +from med_graph import MedNodeUtil, MedGraphUtil TF_TO_ANAKIN_DTYPE = { types_pb2.DT_FLOAT: np.float32, @@ -62,6 +62,11 @@ def get_tf_tensor_data(tensor): def map_tf_dtype(dtype): + ''' + wraper for TF_TO_ANAKIN_DTYPE + :param dtype: + :return: + ''' return TF_TO_ANAKIN_DTYPE.get(dtype) @@ -92,9 +97,9 @@ def tf_to_anakin_tensor(tensor): if dims == [0]: dims = [1] is_raw, data = get_tf_tensor_data(tensor) - dim_size=1 + dim_size = 1 for i in dims: - dim_size*=i + dim_size *= i # print(type(data),data,tensor.dtype,is_raw) if is_raw: if len(dims) > 0: @@ -103,14 +108,19 @@ def tf_to_anakin_tensor(tensor): else: anakin_tensor = np.zeros(0) return anakin_tensor - elif dim_size >1 and len(data)==1: + elif dim_size > 1 and len(data) == 1: - return np.array([data]*dim_size).reshape(dims) + return np.array([data] * dim_size).reshape(dims) else: return data def load_graph(graph_path): + ''' + load tensorflow graph + :param graph_path: + :return: + ''' if graph_path.endswith('.pbtxt'): input_binary = False else: @@ -138,6 +148,12 @@ def load_graph(graph_path): def spatial_map(shape, perm): + ''' + convert shape in different layout + :param shape: + :param perm: + :return: + ''' # print('HI ',type(shape),shape) new_shape = shape[:] for i in perm: @@ -146,6 +162,12 @@ def spatial_map(shape, perm): def trans_move_attr(attr, format): + ''' + get h,w shape from attr + :param attr: + :param format: + :return: + ''' if attr is None: return attr if len(attr) == 2: @@ -158,24 +180,40 @@ def trans_move_attr(attr, format): raise Exception('not support format ' + format) return [h, w] -def add_special_pad(padding,tf_node,graph): + +def add_special_pad(padding, tf_node, graph): + ''' + add pad op to solve different pad method in caffe and tensorflow + :param padding: + :param tf_node: + :param graph: + :return: + ''' # print(tf_node['name']) - assert len(tf_node['input'])<=2 - now_shape=tf_node['input'][0]['shape'] - tar_shape =now_shape[:] - tar_shape[1]=now_shape[1]+padding[0]+padding[1] + assert len(tf_node['input']) <= 2 + now_shape = tf_node['input'][0]['shape'] + tar_shape = now_shape[:] + tar_shape[1] = now_shape[1] + padding[0] + padding[1] tar_shape[2] = now_shape[2] + padding[2] + padding[3] tf_name = tf_node['name'] - padding_node={'name':tf_node['name']+'_pad','ak_type':'Pad','type':None,'visted':True, - 'ak_attr':{'pad_c':[0,0],'pad_h':[padding[0],padding[1]],'pad_w':[padding[2],padding[3]]}, - 'input':[tf_node['input'][0]],'output':[{'name':tf_name,'shape':tar_shape}]} + padding_node = {'name': tf_node['name'] + '_pad', 'ak_type': 'Pad', 'type': None, 'visted': True, + 'ak_attr': {'pad_c': [0, 0], 'pad_h': [padding[0], padding[1]], 'pad_w': [padding[2], padding[3]]}, + 'input': [tf_node['input'][0]], 'output': [{'name': tf_name, 'shape': tar_shape}]} + + input_0 = graph[tf_node['input'][0]['name']] + input_0['output'] = MedNodeUtil.replace_name_with_list(input_0['output'], tf_name, + [{'name': padding_node['name'], 'shape': now_shape}]) + tf_node['input'][0] = {'name': padding_node['name'], 'shape': tar_shape} + graph[padding_node['name']] = padding_node - input_0=graph[tf_node['input'][0]['name']] - input_0['output']=MedNodeUtil.replace_name_with_list(input_0['output'],tf_name,[{'name':padding_node['name'],'shape':now_shape}]) - tf_node['input'][0]={'name':padding_node['name'],'shape':tar_shape} - graph[padding_node['name']]=padding_node def parse_Identity(tf_node, graph): + ''' + remove identity in tensorflow graph + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True outputs = tf_node['output'] input_0 = tf_node['input'][0] @@ -189,6 +227,12 @@ def parse_Identity(tf_node, graph): def parse_Shape(tf_node, graph): + ''' + parse shape for tensorflow graph + :param tf_node: + :param graph: + :return: + ''' assert len(tf_node['input']) == 1 tf_node['type'] = 'Const' tf_node['tf_attr']['value'] = tf_node['input'][0]['shape'] @@ -196,6 +240,12 @@ def parse_Shape(tf_node, graph): def parse_Squeeze(tf_node, graph): + ''' + convert squeeze to reshape + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Reshape' tf_node['ak_attr']['shape'] = tf_node['output'][0]['shape'] @@ -203,30 +253,54 @@ def parse_Squeeze(tf_node, graph): def parse_Placeholder(tf_node, graph): + ''' + conver placeholder to input op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Input' tf_node['ak_attr']['shape'] = spatial_map(tf_node['output'][0]['shape'], NHWC_TO_NCHW) def parse_Pad(tf_node, graph): + ''' + convert pad op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Pad' pad_shape_node = graph[tf_node['input'][1]['name']] assert pad_shape_node['type'] == 'Const' - pad_shape=pad_shape_node['tf_attr']['value'] - ak_attr=tf_node['ak_attr'] + pad_shape = pad_shape_node['tf_attr']['value'] + ak_attr = tf_node['ak_attr'] ak_attr['pad_c'] = pad_shape[3].flatten().tolist() ak_attr['pad_h'] = pad_shape[1].flatten().tolist() ak_attr['pad_w'] = pad_shape[2].flatten().tolist() def parse_Softmax(tf_node, graph): + ''' + convert softmax op, default axis is 3 + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Softmax' - tf_node['ak_attr']['axis']=3 + tf_node['ak_attr']['axis'] = 3 def parse_Reshape(tf_node, graph): + ''' + convert reshape op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Reshape' shape = graph[tf_node['input'][1]['name']] @@ -238,9 +312,15 @@ def parse_Reshape(tf_node, graph): def parse_Act(tf_node, graph): + ''' + convert activate op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Activation' - if tf_node['type'] == 'Relu' : + if tf_node['type'] == 'Relu': tf_node['ak_type'] = 'Relu' tf_node['ak_attr']['type'] = 'Relu' elif tf_node['type'] == 'Relu6': @@ -252,16 +332,28 @@ def parse_Act(tf_node, graph): def parse_Concat(tf_node, graph): + ''' + convert concat op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Concat' N = tf_node['tf_attr']['N'] axis_node = graph[tf_node['input'][N]['name']] assert axis_node['type'] == 'Const' - nhwc2hchw={0:0,1:2,2:3,3:1} + nhwc2hchw = {0: 0, 1: 2, 2: 3, 3: 1} tf_node['ak_attr']['axis'] = nhwc2hchw[axis_node['tf_attr']['value'][0]] def parse_Add(tf_node, graph): + ''' + convert add op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True assert len(tf_node['input']) == 2 input_0 = graph[tf_node['input'][0]['name']] @@ -278,33 +370,47 @@ def parse_Add(tf_node, graph): tf_node['ak_type'] = 'Eltwise' ak_attr['type'] = 'Add' + def parse_Mean(tf_node, graph): + ''' + convert mean op to pooling op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Pooling' ak_attr = tf_node['ak_attr'] tf_attr = tf_node['tf_attr'] - ak_attr['type']='AVG' - reduction_shape=None - keep_dims=tf_attr['keep_dims'] - - if len(tf_node['input'])>1: - reduction_shape_node=graph[tf_node['input'][1]['name']] - assert reduction_shape_node['type']=='Const' - reduction_shape=reduction_shape_node['tf_attr']['value'].flatten().tolist() + ak_attr['type'] = 'AVG' + reduction_shape = None + keep_dims = tf_attr['keep_dims'] + + if len(tf_node['input']) > 1: + reduction_shape_node = graph[tf_node['input'][1]['name']] + assert reduction_shape_node['type'] == 'Const' + reduction_shape = reduction_shape_node['tf_attr']['value'].flatten().tolist() assert reduction_shape is not None assert keep_dims is True - assert reduction_shape == [1,2] - ak_attr['strides'] = [1,1] - ak_attr['window'] = [tf_node['input'][0]['shape'][reduction_shape[0]],tf_node['input'][0]['shape'][reduction_shape[1]]] - ak_attr['padding'] = [0,0] + assert reduction_shape == [1, 2] + ak_attr['strides'] = [1, 1] + ak_attr['window'] = [tf_node['input'][0]['shape'][reduction_shape[0]], + tf_node['input'][0]['shape'][reduction_shape[1]]] + ak_attr['padding'] = [0, 0] def parse_Pooling(tf_node, graph): + ''' + convert pooling op + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Pooling' ak_attr = tf_node['ak_attr'] tf_attr = tf_node['tf_attr'] - #TODO TF default pad is exclusive method + # TODO TF default pad is exclusive method map_tf_pool = {'MaxPool': 'MAX', 'AvgPool': 'AVGEXC'} @@ -322,26 +428,38 @@ def parse_Pooling(tf_node, graph): padding = cal_padding(padding, kernel_shape, strides, dilations, data_format, tf_node['input'][0]['shape'], tf_node['output'][0]['shape']) - if padding[0]!=padding[1] or padding[2]!=padding[3]: - add_special_pad(padding,tf_node,graph) - padding=[0,0] + if padding[0] != padding[1] or padding[2] != padding[3]: + add_special_pad(padding, tf_node, graph) + padding = [0, 0] else: - padding=[padding[0],padding[2]] + padding = [padding[0], padding[2]] ak_attr['window'] = kernel_shape ak_attr['padding'] = padding ak_attr['strides'] = strides - ak_attr['cmp_out_shape_floor_as_conv']=True + ak_attr['cmp_out_shape_floor_as_conv'] = True def cal_padding(padding, kernel_shape, strides, dilations, data_format, input_shape, output_shape, spatial=2): - pads = [0] * spatial*2 + ''' + convert pad string to pad list + :param padding: + :param kernel_shape: + :param strides: + :param dilations: + :param data_format: + :param input_shape: + :param output_shape: + :param spatial: + :return: + ''' + pads = [0] * spatial * 2 if type(padding) == bytes: padding = padding.decode() if dilations is None: dilations = [1] * spatial * 2 if padding == 'SAME': - pads = [0] * spatial*2 + pads = [0] * spatial * 2 if data_format == 'NHWC': # print('in out shape = ',input_shape,output_shape) input_shape = spatial_map(input_shape, NHWC_TO_NCHW) @@ -353,7 +471,7 @@ def cal_padding(padding, kernel_shape, strides, dilations, data_format, input_sh pad = max(pad, 0) pads1 = pad // 2 pads2 = pad - pad // 2 - pads[i*2+0]=pads1 + pads[i * 2 + 0] = pads1 pads[i * 2 + 1] = pads2 return pads elif padding == 'VALID': @@ -363,6 +481,12 @@ def cal_padding(padding, kernel_shape, strides, dilations, data_format, input_sh def get_const_from_biinput(inputs, graph): + ''' + search const input of node`s input + :param inputs: + :param graph: + :return: + ''' assert len(inputs) <= 2 input_0 = graph[inputs[0]['name']] input_1 = graph[inputs[1]['name']] @@ -374,6 +498,12 @@ def get_const_from_biinput(inputs, graph): def get_bias(tf_node, graph): + ''' + try to fetch const value form node + :param tf_node: + :param graph: + :return: + ''' bias_weight = None output = tf_node['output'] output_0 = graph[output[0]['name']] @@ -383,12 +513,18 @@ def get_bias(tf_node, graph): if bias_weight is not None: output_0['visted'] = True tf_node['output'] = output_0['output'] - MedNodeUtil.redirecto_outputs_input_to_this(output_0,graph,tf_node['name'],output[0]['shape']) + MedNodeUtil.redirecto_outputs_input_to_this(output_0, graph, tf_node['name'], output[0]['shape']) return bias_weight def parse_Conv2D(tf_node, graph): + ''' + convert conv2D to convolution + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Convolution' weights_node = graph[tf_node['input'][1]['name']] @@ -407,18 +543,18 @@ def parse_Conv2D(tf_node, graph): # print('name ',tf_node['name'],input_node['name'],input_node['out_shape']) padding = cal_padding(padding, kernel_shape, strides, dilations, data_format, tf_node['input'][0]['shape'], tf_node['output'][0]['shape']) - if padding[0]!=padding[1] or padding[2]!=padding[3]: - add_special_pad(padding,tf_node,graph) - padding=[0,0] + if padding[0] != padding[1] or padding[2] != padding[3]: + add_special_pad(padding, tf_node, graph) + padding = [0, 0] else: - padding=[padding[0],padding[2]] + padding = [padding[0], padding[2]] group = 1 if tf_node['type'] == 'DepthwiseConv2dNative': weights = weights.transpose((1, 0, 2, 3)) group = weights.shape[0] - out_c=weights.shape[0]*weights.shape[1] - weights=weights.reshape(out_c,1,weights.shape[2],weights.shape[3]) + out_c = weights.shape[0] * weights.shape[1] + weights = weights.reshape(out_c, 1, weights.shape[2], weights.shape[3]) ak_attr = tf_node['ak_attr'] ak_attr['weights'] = weights @@ -431,6 +567,12 @@ def parse_Conv2D(tf_node, graph): def parse_MatMul(tf_node, graph): + ''' + convert matmul to dense + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Dense' in_name_0 = tf_node['input'][0]['name'] @@ -457,6 +599,12 @@ def parse_MatMul(tf_node, graph): def parse_fusionReshape(tf_node, graph): + ''' + convert reshape + :param tf_node: + :param graph: + :return: + ''' assert tf_node['type'] == 'Reshape' input_0 = graph[tf_node['input'][0]['name']] input_1 = graph[tf_node['input'][1]['name']] @@ -472,6 +620,7 @@ def parse_fusionReshape(tf_node, graph): def parse_CustmerBatchNorm(tf_node, graph): + '''convert user custmer batchnorm to scale''' assert tf_node['type'] == 'Rsqrt' assert len(tf_node['input']) == 1 and len(tf_node['output']) == 1 rsqrt_mul_node = graph[tf_node['output'][0]['name']] @@ -485,7 +634,7 @@ def parse_CustmerBatchNorm(tf_node, graph): var_value = var_node['tf_attr']['value'].flatten() assert rsqrt_mul_node['type'] == 'Mul' and len(rsqrt_mul_node['input']) == 2 and len(rsqrt_mul_node['output']) == 2 const_mul_node_input_1 = graph[rsqrt_mul_node['input'][1]['name']] - if MedGraphUtil.check_one_of_input_is_const(graph[rsqrt_mul_node['output'][0]['name']],graph): + if MedGraphUtil.check_one_of_input_is_const(graph[rsqrt_mul_node['output'][0]['name']], graph): mul_mul_1_node = graph[rsqrt_mul_node['output'][1]['name']] mul_mul_2_node = graph[rsqrt_mul_node['output'][0]['name']] else: @@ -503,7 +652,7 @@ def parse_CustmerBatchNorm(tf_node, graph): assert mul_add_node['type'] == 'Add' and mul2_sub_node['type'] == 'Sub' and mean_node['type'] == 'Const' beta_node = graph[mul2_sub_node['input'][0]['name']] - assert beta_node['type']=='Const' + assert beta_node['type'] == 'Const' beta_value = beta_node['tf_attr']['value'].flatten() mean_value = mean_node['tf_attr']['value'].flatten() assert mul_add_node['input'][1]['name'] == mul2_sub_node['name'] @@ -518,13 +667,14 @@ def parse_CustmerBatchNorm(tf_node, graph): var = np.sqrt(var_value + epse_value) np_scale = alpha_value / var - np_bias = beta_value -np_scale * mean_value + np_bias = beta_value - np_scale * mean_value mul_mul_1_node['output'] = mul_add_node['output'] mul_mul_1_node['type'] = 'CustomerBN' mul_mul_1_node['ak_type'] = 'Scale' MedNodeUtil.retain_input(mul_mul_1_node, [mul_mul_1_node['input'][0]]) - MedNodeUtil.redirecto_outputs_input_to_this(mul_add_node,graph,mul_mul_1_node['name'],mul_mul_1_node['output'][0]['shape']) + MedNodeUtil.redirecto_outputs_input_to_this(mul_add_node, graph, mul_mul_1_node['name'], + mul_mul_1_node['output'][0]['shape']) ak_attr = mul_mul_1_node['ak_attr'] ak_attr['scale_weights'] = np_scale.astype('float32') ak_attr['bias_weights'] = np_bias.astype('float32') @@ -532,6 +682,12 @@ def parse_CustmerBatchNorm(tf_node, graph): def parse_BatchNorm(tf_node, graph): + ''' + convert fused batchnorm to scale + :param tf_node: + :param graph: + :return: + ''' tf_node['visted'] = True tf_node['ak_type'] = 'Scale' assert len(tf_node['input']) == 5 @@ -556,4 +712,4 @@ def parse_BatchNorm(tf_node, graph): ak_attr['scale_weights'] = np_scale.astype('float32') ak_attr['bias_weights'] = np_bias.astype('float32') - MedNodeUtil.retain_input(tf_node, [tf_node['input'][0]]) \ No newline at end of file + MedNodeUtil.retain_input(tf_node, [tf_node['input'][0]]) diff --git a/tools/external_converter_v2/parser/kill_tf/tf_util.py b/tools/external_converter_v2/parser/tensorflow/tf_util.py similarity index 83% rename from tools/external_converter_v2/parser/kill_tf/tf_util.py rename to tools/external_converter_v2/parser/tensorflow/tf_util.py index 8baed082d..b8dfe78d9 100644 --- a/tools/external_converter_v2/parser/kill_tf/tf_util.py +++ b/tools/external_converter_v2/parser/tensorflow/tf_util.py @@ -3,9 +3,17 @@ from google.protobuf import text_format import numpy as np + class TfUtil: @staticmethod def tf_run_model(graph_path, inputs, output_tensor_list): + ''' + fill inputs, run tensorflow mode and fetch output in output_tensor_list + :param graph_path: + :param inputs: + :param output_tensor_list: + :return: + ''' sess = tf.Session() if graph_path.endswith('.pbtxt'): input_binary = False @@ -30,4 +38,4 @@ def tf_run_model(graph_path, inputs, output_tensor_list): output_list = [graph.get_tensor_by_name(i) for i in output_tensor_list] print(output_list) out = sess.run(output_list, inputs_dict) - return out \ No newline at end of file + return out