Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

svtr_tiny 性能优化 和 修改多卡启动方式为msrun #810

Merged
merged 6 commits into from
Mar 4, 2025

Conversation

iugoood
Copy link
Collaborator

@iugoood iugoood commented Feb 25, 2025

Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:

Motivation

1、GradSampler2D算子优化
2、多卡启动由mpirun修改为msrun

Test Plan

(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)

Related Issues and PRs

(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

@iugoood iugoood force-pushed the main branch 2 times, most recently from ec3670d to abe48a5 Compare February 28, 2025 08:00
@iugoood iugoood changed the title svtr_tiny 性能优化 svtr_tiny 性能优化 和 修改多卡启动方式为msrun Feb 28, 2025
@iugoood iugoood force-pushed the main branch 3 times, most recently from fb18b2a to 80667fb Compare February 28, 2025 08:32
```
**Note:** For more information about msrun configuration, please refer to [here](https://www.mindspore.cn/tutorials/experts/en/r2.3.1/parallel/msrun_launcher.html).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是不是链接mindspore 2.5.0的文档好一些

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文档上环境信息就是2.3.1,这边是考虑保持一致

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的 了解 后续v0.5.0分支适配的pr会把这里修改掉

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后续统一修改为2.5分支

```
**Note:** For more information about msrun configuration, please refer to [here](https://www.mindspore.cn/tutorials/experts/en/r2.3.1/parallel/msrun_launcher.html).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后续统一修改为2.5分支

msrun --worker_num=8 --local_worker_num=8 python tools/train.py --config configs/det/east/east_r50_icdar15.yaml

# 经验证,绑核在大部分情况下有性能加速,请配置参数并运行
export MS_ENABLE_NUMA=True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么numa和msrun的bind_core要同时加?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我这边尝试一起加性能会有提升,只加bind_core性能不明显

Copy link
Collaborator Author

@iugoood iugoood Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

最开始尝试的只设置MS_ENABLE_NUMA=True,没有加bind_core参数,性能基本没什么变化,后面参考
image这个写法两个一起尝试,性能有所提升

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

云上或者在容器跑numa绑核会有问题,这个应该只有裸机跑可以有效

@Ash-Lee233 Ash-Lee233 merged commit 21f0713 into mindspore-lab:main Mar 4, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants