You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: INSTRUCTIONS.md
+39-36Lines changed: 39 additions & 36 deletions
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,9 @@
2
2
3
3
Here we give some how-to-use instructions and tips.
4
4
5
-
## Command-Line Interface:
5
+
Notice that these are basic, simplified instructions. For more details and the complete documentation, go to: [https://spbench-doc.rtfd.io](https://spbench-doc.rtfd.io)
6
+
7
+
## Command-Line Interface:
6
8
7
9
The framework works with a command-line interface (CLI).
8
10
@@ -14,42 +16,49 @@ For each command there is also a "help" guide. You can access it using:
14
16
15
17
`./spbench [command] -h`
16
18
17
-
## Framework commands:
19
+
## Basic concepts:
20
+
21
+
-**Benchmark**: In this context, it means any application version registered in SPBench. Benchmarks can be the sequential applications, the parallel versions pre-available within SPBench, or any new custom implementations registered latter by users.
22
+
-**PPI**: It stands for Parallel Programming Interface, which means any library, framework, language, etc. that leverages parallelism.
23
+
24
+
## Main command-line commands:
25
+
26
+
The table bellow shows all commands that can be used through the CLI
27
+
| Command | Description | Subcommands |
28
+
|---|---|---|
29
+
|**install**| It installs the dependencies for the SPBench benchmarks. |`-app <some_application>`|
30
+
|**register**| It automatically sets the paths and generates new makefiles for all benchmarks. | None |
31
+
|**new**| It allows for creating a new benchmark based on a SPBench application.<br /> This command will create a fresh copy of the selected application. |`-name <new_benchmark_id>`(any name)<br /> `-app <some_SPBench_application>`<br /> `-ppi <some_ppi>` (any name)<br /> `-replicate <other_benchmark_id>`(Optional) |
32
+
|**edit**| It opens a choosen benchmark for code editing. |`-bench <benchmark_id>`<br /> `-editor <editor_name>`(Optional)(Default: nano) |
33
+
|**configure**| It allows users to add compiling dependencies for a benchmark.<br /> It opens a JSON configuration file for you to write the required dependencies. |`-bench <benchmark_id>`<br /> `-editor <editor_name>`(Optional)(Default: nano) |
34
+
|**compile**| It compiles a selected benchmark. |`-bench <benchmark_id>`|
|**list**| This command lists all SPBench benchmarks. Users' new benchmark versions are added to this list. | None |
37
+
|**delete**| It completely deletes the given benchmark. |`-bench <benchmark_id>`|
38
+
|**new_input**| It allows for adding an alias for a custom input. |`-input_id <new_input_id>`<br /> `-app <SPBench_application>`<br /> `-input <"input info/string">`|
39
+
|**list_input**| It lists all inputs registered for a given application and their respective alias.<br /> (Attention: You can only use pre-registered alias after downloading the respective inputs.<br /> See the section about inputs below). |`-app <SPBench_application>`(optional) |
40
+
18
41
19
-
-**install**: It install the dependencies for the SPBench applications and parallel implementations.
20
-
-**register**: It automatically generates makefiles for all benchmark versions available within the framework (You must install the dependencies first).
21
-
-**new**: You use it to create new version of a given application from the framework. This command will create a fresh copy of the selected application for you to work on. See the 'help' option for this command to check the required arguments.
22
-
-**edit**: You can use this comand to edit a given application version. See the 'help' option for this command to check the required arguments.
23
-
-**configure**: You can use this command to configure the compiling dependencies of an application version. It opens a JSON configuration file for you to write the required dependencies. See the 'help' option for this command to check the required arguments.
24
-
-**compile**: Use this command to compile a given version. See the 'help' option for this command to check the required arguments.
25
-
-**exec**: Use this command to run an application version. See the 'help' option for this command to check the required arguments.
26
-
-**list**: This command lists all applications versions of the framework. Your new versions will be added to this list.
27
-
-**delete**: It completely deletes a given application version.
28
-
-**new_input**: It allows you to add an alias for a custom input. See the 'help' option for this command to check the required arguments.
29
-
-**list_inputs**: List all inputs registered for a given application and their respective alias. (Attention: You can only use these alias after downloading the respective inputs. See the section about inputs below).
30
42
31
43
## How to use
32
44
33
45
In this framework you can simply run the benchmarks that are already provided or implement your own specific benchmark.
34
46
35
47
To run a given benchmark version, you can follow these steps:
36
48
- Run: `./spbench install` (you can use `-app <some application>` to install only a specific application)
37
-
- Run: `./spbench register`
38
49
- Run: `./spbench list` (this shows the list of available benchmarks)
@@ -72,7 +81,9 @@ With SPBench you can evaluate four performance metrics in different ways.
72
81
- CPU usage (%)
73
82
- Memory usage (KB)
74
83
75
-
There are four OPTIONAL arguments you can select when running the 'exec' command:
84
+
There are some OPTIONAL arguments you can select when running the 'exec' command:
85
+
86
+
## For metrics:
76
87
77
88
-`-monitor <time interval>`
78
89
@@ -94,11 +105,11 @@ There are four OPTIONAL arguments you can select when running the 'exec' command
94
105
95
106
It prints the global memory and CPU usage for the selected application.
96
107
97
-
## Other configuration arguments
108
+
## Other optional management arguments
98
109
99
110
All of these are also OPTIONAL arguments that you can use within the 'exec' command.
100
111
101
-
-`-in_memory`
112
+
-`-in-memory`
102
113
103
114
It runs the application in-memory, which means that all the input is first loaded into the memory before start processing it. The opposite is true for the writing phase, where the result stays in memory and is only writen to the disk after all the input is proccessed.
104
115
@@ -109,23 +120,15 @@ All of these are also OPTIONAL arguments that you can use within the 'exec' comm
109
120
-`-batch <batch_size>`
110
121
111
122
Here you can change the size of the batches, so each item can carry more data. By default, the size of each batch is 1.
112
-
113
-
-`-frequency <time_in_microseconds>`
114
123
115
-
Here you can change the frequency of itens generated by source operator. The minimum frequency is limited by the memory access speed in your system.
124
+
-`-frequency <time-delay_per_item_in_us>`
125
+
126
+
Here you can set the frequency in which the sources of the benchmarks will produce new items. To increase or decrese the frequency, you can decrease or increase the time-delay between items generation. You must insert a time-delay in microseconds (e.g. 500000 = 500 ms).
116
127
117
128
-`-d`
118
129
119
130
This a specific argument for bzip2. You can use it to run this application in decompress mode. Observe that for de decompression mode it is required a compressed file as input (e.g. my_compressed_file.tar.bz2).
120
131
121
-
## Global attributes
122
-
123
-
SPBench implements some global variables that can be used and modified by users at execution time in custom implementations. Some useful are:
124
-
125
-
- nthreads
126
-
- batch_size
127
-
- items_reading_frequency
128
-
129
132
## Important
130
133
131
134
You must run `source setup_vars.sh` to load the libraries.
SPBench is a new Framework for Benchmarking Stream Processing Applications.
6
-
It allows users to easily use parallel libraries to explore stream parallelism with different applications.
7
-
SPBench manages the application execution and performance evaluation aspects, which allows the user to focus exclusively on the parallelism aspects.
5
+
[SPBench](https://spbench-doc.readthedocs.io/en/latest/) (Stream Processing Benchmark) is a framework for benchmarking C++ stream processing applications.
6
+
The main goal of SPBench is to enable users to easily create custom benchmarks from real-world stream processing applications and evaluate multiple PPIs.
8
7
9
-
The applications set comprises the following stream processing applications and will be more in the future:
8
+
**The SPbench documentation is available at https://spbench-doc.rtfd.io**
9
+
10
+
## How to cite
11
+
12
+
- GARCIA, A. M.; SCHEPKE, C. ; GRIEBLER, D. J. ; FERNANDES, L. G. . SPBench: A Framework for Creating Benchmarks of Stream Processing Applications. COMPUTING, v. 1, p. 1, 2021, doi: 10.1007/s00607-021-01025-6. \[[link](https://doi.org/10.1007/s00607-021-01025-6)\]
13
+
14
+
### Other papers using SPBench
15
+
16
+
- A. M. Garcia, D. Griebler, C. Schepke and L. G. L. Fernandes, Evaluating Micro-batch and Data Frequency for Stream Processing Applications on Multi-cores. In: 2022 30th Euromicro International Conference on Parallel, Distributed and NetworkBased Processing (PDP), 2022, Valladolid. 2022 30th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), 2022. p. 10-17, doi: 10.1109/PDP55904.2022.00011. \[[link](https://doi.org/10.1109/PDP55904.2022.00011)\]
17
+
18
+
- A. M. Garcia, D. Griebler, L. G. L. Fernandes and C. Schepke, "Introducing a Stream Processing Framework for Assessing Parallel Programming Interfaces", 2021 29th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), 2021, pp. 84-88, doi: 10.1109/PDP52278.2021.00021. \[[link](https://doi.org/10.1109/PDP52278.2021.00021)\]
19
+
20
+
## The SPBench is intended for three main audiences:
21
+
- Users who want to run performance tests with the SPBench benchmarks. It implements different real-world stream processing applications using different parallel programming interfaces (PPIs) and parallelism patterns, and makes available the main metrics used in this domain. The framework also enables users to create custom benchmarks with new PPIs.
22
+
- Researchers and developers who want to test and evaluate new technologies and solutions. The SPBench benchmarks are also highly parameterizable and its API allows for easy and fast code reuse across all applications.
23
+
- Students and teachers who want to learn/teach stream parallelism. The benchmarks implemented with the SPBench API abstract the low-level code and expose to users only the stream core of each application (in a few lines of code). Therefore, it allows users to easily identify each individual operator and data dependencies, to understand what the stream flow looks like. Then users can build parallelism on top of this highly simplified code.
24
+
25
+
### The SPBench suite comprises the following stream processing applications and will be more in the future:
10
26
- Ferret (PARSEC)
11
27
- Lane Detection
12
28
- Bzip2
13
-
- Person Recognition (Temporarily removed due to license constraint)
29
+
- Person Recognition
14
30
15
-
For each of them there are parallel versions implemented using the following parallel programming interfaces and will be more in the future:
31
+
### For each of them there are parallel versions implemented using the following parallel programming interfaces and will be more in the future:
16
32
- Intel TBB
17
33
- FastFlow
18
34
- SPar
19
35
- GrPPI (forthcoming)
20
36
- Standard C++ Threads (forthcoming)
21
37
22
-
## How to cite
38
+
# SPBench Framework
39
+
40
+
One of the aspects that most differentiates SPBench from related solutions is how users interact with it.
41
+
The kernel of SPBench is its API that allows users to access simplified versions of the applications included in our suite. Based on the sequential versions, users can implement parallelism, create new custom benchmarks with different parallelism strategies, or even explore new parallel PPIs.
42
+
Users can also configure and modify parameters, such as build dependencies and metrics, through a command-line interface (CLI).
43
+
This CLI accesses a database containing all benchmarks from the SPBench suite.
44
+
Other secondary parameters can be tuned via the CLI with simple commands.
45
+
This allows users to entirely focus on writing and tuning the parallelism rather than spending time with the non-relevant low-level aspects of each application. The figure bellow illustrates the SPBench framework.
23
46
24
-
- A. M. Garcia, D. Griebler, L. G. L. Fernandes and C. Schepke, "Introducing a Stream Processing Framework for Assessing Parallel Programming Interfaces," 2021 29th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), 2021, pp. 84-88, doi: 10.1109/PDP52278.2021.00021.
0 commit comments