Skip to content

Conversation

@Xreki
Copy link
Collaborator

@Xreki Xreki commented Jan 10, 2022

当前op benchmark具备paddle静态图和动态图op性能测试的能力:静态图测试基类是PaddleAPIBenchmarkBase,测试脚本在tests_v2目录下;动态图测试基类是PaddleDynamicAPIBenchmarkBase,测试脚本在dynamic_tests_v2目录下。当前op benchmark系统存在以下问题:

  1. 静态图和动态图测试基类虽然写法差不多,但没有继承自同一个基类,存在很多重复的代码
  2. 基类的不统一,导致了op的测试代码也必须写多份,tests_v2和dynamic_tests_v2中的op测试脚本也存在很多重复

我们计划将动态图、静态图的测试脚本统一(如#1206),本PR是其中的第一步,实现动态图、静态图测试的公共基类。基于本PR,已经跑过动态图的全量OP测试,见cuda11.4_dynamic_op_benchmark_summary-2022-01-12.xlsx。静态图需要基于#1237 测试,计划先合入这个PR,在#1237 中验证静态图全量测试能力。

CI逻辑的修改:

  1. common_import.py不是op测试脚本,添加过滤功能

CI中已验证基类重构后静态图、动态图abs测试的能力:
image

image

@paddle-bot-old
Copy link

Thanks for your contribution! Please wait for the result of CI firstly.

@Xreki Xreki force-pushed the api/op_benchmark_base branch 2 times, most recently from a5516d3 to 5c42339 Compare January 11, 2022 07:21
@Xreki Xreki force-pushed the api/op_benchmark_base branch from 5c42339 to 1165dff Compare January 11, 2022 11:52
@Xreki Xreki requested a review from JamesLim-sy January 17, 2022 03:58
JamesLim-sy
JamesLim-sy previously approved these changes Jan 17, 2022
Copy link
Contributor

@JamesLim-sy JamesLim-sy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@JamesLim-sy JamesLim-sy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants