You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Aug 7, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: CODE_OF_CONDUCT.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
Copy file name to clipboardExpand all lines: README.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
4
+
1
5
# ❗ANNOUNCEMENT: Security Changes❗
2
6
TorchServe now enforces token authorization enabled and model API control disabled by default. These security features are intended to address the concern of unauthorized API calls and to prevent potential malicious code from being introduced to the model server. Refer the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
Copy file name to clipboardExpand all lines: SECURITY.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
Copy file name to clipboardExpand all lines: benchmarks/README.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
4
+
1
5
# Torchserve Model Server Benchmarking
2
6
3
7
The benchmarks measure the performance of TorchServe on various models and benchmarks. It supports either a number of built-in models or a custom model passed in as a path or URL to the .mar file. It also runs various benchmarks using these models (see benchmarks section below). The benchmarks are executed in the user machine through a python3 script in case of jmeter and a shell script in case of apache benchmark. TorchServe is run on the same machine in a docker instance to avoid network latencies. The benchmark must be run from within `serve/benchmarks`
Copy file name to clipboardExpand all lines: benchmarks/add_jmeter_test.md
+9-5Lines changed: 9 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,20 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
4
+
1
5
## Adding a new test plan for torchserve
2
6
3
7
A new Jmeter test plan for torchserve benchmark can be added as follows:
4
8
5
9
* Assuming you know how to create a jmeter test plan. If not then please use this jmeter [guide](https://jmeter.apache.org/usermanual/build-test-plan.html)
6
10
* Here, we will show you how 'MMS Benchmarking Image Input Model Test Plan' plan can be added.
7
-
This test plan does following:
8
-
11
+
This test plan does following:
12
+
9
13
* Register a model - `default is resnet-18`
10
14
* Scale up to add workers for inference
11
15
* Send Inference request in a loop
12
16
* Unregister a model
13
-
17
+
14
18
(NOTE - This is an existing plan in `serve/benchmarks`)
15
19
* Open jmeter GUI
16
20
e.g. on macOS, type `jmeter` on commandline
@@ -63,7 +67,7 @@ You can create variables or use them directly in your test plan.
63
67
* input_filepath - input image file for prediction
64
68
* min_workers - minimum workers to be launch for serving inference request
65
69
66
-
NOTE -
70
+
NOTE -
67
71
68
72
* In above, screenshot, some variables/input box are partially displayed. You can view details by opening an existing test cases from serve/benchmarks/jmx for details.
69
-
* Apart from above argument, you can define custom arguments specific to you test plan if needed. Refer `benchmark.py` for details
73
+
* Apart from above argument, you can define custom arguments specific to you test plan if needed. Refer `benchmark.py` for details
Copy file name to clipboardExpand all lines: benchmarks/jmeter.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
Copy file name to clipboardExpand all lines: benchmarks/sample_report.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
Copy file name to clipboardExpand all lines: binaries/README.md
+20-16Lines changed: 20 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,8 @@
1
-
# Building TorchServe and Torch-Model-Archiver release binaries
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
4
+
5
+
# Building TorchServe and Torch-Model-Archiver release binaries
## Uploading packages to production torchserve account
159
163
160
-
As a first step binaries and docker containers need to be available in some staging environment. In that scenario the binaries can just be `wget`'d and then uploaded using the instructions below and the docker staging environment just needs a 1 line code change in https://github.com/pytorch/serve/blob/master/docker/promote-docker.sh#L8
164
+
As a first step binaries and docker containers need to be available in some staging environment. In that scenario the binaries can just be `wget`'d and then uploaded using the instructions below and the docker staging environment just needs a 1 line code change in https://github.com/pytorch/serve/blob/2a0ce756b179677f905c3216b9c8427cd530a129/docker/promote-docker.sh#L8
161
165
162
166
### pypi
163
167
Binaries should show up here: https://pypi.org/project/torchserve/
Binaries should show up here: https://hub.docker.com/r/pytorch/torchserve
184
188
185
-
Change the staging org to your personal docker or test docker account https://github.com/pytorch/serve/blob/master/docker/promote-docker.sh#L8
189
+
Change the staging org to your personal docker or test docker account https://github.com/pytorch/serve/blob/2a0ce756b179677f905c3216b9c8427cd530a129/docker/promote-docker.sh#L8
186
190
187
191
188
192
### Direct upload
@@ -197,7 +201,7 @@ For an official release our tags include `pytorch/torchserve/<version_number>-cp
197
201
## Direct upload Kserve
198
202
To build the Kserve docker image follow instructions from [kubernetes/kserve](../kubernetes/kserve/README.md)
199
203
200
-
When tagging images for an official release make sure to tag with the following format `pytorch/torchserve-kfs/<version_number>-cpu` and `pytorch/torchserve-kfs/<version_number>-gpu`.
204
+
When tagging images for an official release make sure to tag with the following format `pytorch/torchserve-kfs/<version_number>-cpu` and `pytorch/torchserve-kfs/<version_number>-gpu`.
Copy file name to clipboardExpand all lines: binaries/conda/README.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,7 @@
1
+
# ⚠️ Notice: Limited Maintenance
2
+
3
+
This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
4
+
1
5
# Building conda packages
2
6
3
7
1. To build conda packages you must first produce wheels for the project, see [this readme](../README.md) for more details on building `torchserve` and `torch-model-archiver` wheel files.
@@ -21,4 +25,3 @@ The built conda packages are available in the `output` directory
21
25
Anaconda packages are both OS specific and python version specific so copying them one by one from a test/staging environment like https://anaconda.org/pytorch/torchserve/ to an official environment like https://anaconda.org/torchserve-staging can be fiddly
22
26
23
27
Instead you can run `anaconda copy torchserve-staging/<package>/<version_number> --to-owner pytorch`
0 commit comments