7cef624e6e
Benchmarks: micro benchmarks - add nvbandwidth benchmark ( #669 )
...
**Description**
Add nvbandwidth benchmark.
---------
Co-authored-by: hongtaozhang <hongtaozhang@microsoft.com>
2024-11-22 04:50:47 +00:00
c8c52eb212
Benchmarks: micro benchmarks - add nvbandwidth build ( #665 )
...
**Description**
Add nvbandwidth build to repo
---------
Co-authored-by: hongtaozhang <hongtaozhang@microsoft.com>
2024-11-21 12:01:53 +00:00
54eeac25d1
Docs - Update CODEOWNERS ( #670 )
...
Update CODEOWNERS for docs.
2024-11-21 05:37:07 +00:00
9c35e80a6f
Benchmarks: micro benchmarks - add general CPU bandwidth and latency benchmark ( #662 )
...
**Description**
Add micro benchmark to measure general CPU bandwidth and latency without 'mlc'.
Test output:
```
{
"cpu-memory-bw-latency/return_code": 0,
"cpu-memory-bw-latency/mem_bandwidth_matrix_numa_0_1_bw": 5388.75021,
"cpu-memory-bw-latency/mem_bandwidth_matrix_numa_0_1_lat": 0.185571786,
"cpu-memory-bw-latency/mem_bandwidth_matrix_numa_1_0_bw": 4634.82028,
"cpu-memory-bw-latency/mem_bandwidth_matrix_numa_1_0_lat": 0.215758096,
}
```
---------
Co-authored-by: hongtaozhang <hongtaozhang@microsoft.com>
2024-11-20 22:46:44 +00:00
a8a7bed2b6
Dependency - Bump onnxruntime-gpu version from 1.10.0 to 1.12.0 ( #663 )
...
**Description**
Bump onnxruntime-gpu from 1.10.0 to 1.12.0.
---------
Co-authored-by: hongtaozhang <hongtaozhang@microsoft.com>
2024-11-15 08:57:06 +08:00
83ee4ebacd
Bump webpack from 5.76.1 to 5.96.1 in /website ( #661 )
...
Bumps [webpack](https://github.com/webpack/webpack ) from 5.76.1 to 5.96.1.
- [Release notes](https://github.com/webpack/webpack/releases )
- [Commits](webpack/webpack@v5.76.1...v5.96.1)
---
updated-dependencies:
- dependency-name: webpack
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-11-07 15:21:28 -08:00
c9b2b45565
Bump cookie and express in /website ( #655 )
...
Bumps [cookie](https://github.com/jshttp/cookie ) and [express](https://github.com/expressjs/express ). These dependencies needed to be updated together.
Updates `cookie` from 0.6.0 to 0.7.1
- [Release notes](https://github.com/jshttp/cookie/releases )
- [Commits](jshttp/cookie@v0.6.0...v0.7.1)
Updates `express` from 4.21.0 to 4.21.1
- [Release notes](https://github.com/expressjs/express/releases )
- [Changelog](https://github.com/expressjs/express/blob/4.21.1/History.md )
- [Commits](expressjs/express@4.21.0...4.21.1)
---
updated-dependencies:
- dependency-name: cookie
dependency-type: indirect
- dependency-name: express
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-11-07 11:52:24 +00:00
479491279e
Dockerfile - Add support for arm64 build ( #660 )
...
Add support for arm64 build:
- Updated dockerfile for arm64 build
- extend cpu stream compilation for neoverse
- handle onnxruntime-gpu installation
- third party builds filtering based on arch
- disable cuda decode perf build for non x86
2024-11-06 23:16:12 +00:00
59d36f7ff0
Bug Fix - Fix numa error on grace cpu in gpu-copy ( #658 )
...
The current GPU Copy BW Performance fails on Nvidia Grace systems. This
is due to the memory only numa node and thus the numa_run_on_node fails
for such nodes and halts completely.
This fix checks for the presence of assigned CPU cores for the numa
node, on checking if it has no cpu cores assigned, it skips that
specific node during the args creation and continues.
2024-11-05 23:10:51 +00:00
61770b8908
CI/CD - Update Image Build Pipeline ( #659 )
...
**Description**
Update image build.
**Major Revision**
* Remove ROCm 6.0 image due to outdated packages
* Remove build tag for ROCm
* Preserve build cache for 30 days
2024-11-02 04:50:50 +00:00
949f9cb406
Release - SuperBench v0.11.0 ( #654 )
...
**Description**
Cherry pick bug fixes from v0.11.0 to main
**Major Revision**
* #645
* #648
* #646
* #647
* #651
* #652
* #650
---------
Co-authored-by: hongtaozhang <hongtaozhang@microsoft.com>
Co-authored-by: Yifan Xiong <yifan.xiong@microsoft.com>
2024-10-10 09:59:47 +08:00
9f3231e99f
Bump serve-static and express in /website ( #643 )
...
Bumps [serve-static](https://github.com/expressjs/serve-static ) and [express](https://github.com/expressjs/express ). These dependencies needed to be updated together.
Updates `serve-static` from 1.15.0 to 1.16.2
- [Release notes](https://github.com/expressjs/serve-static/releases )
- [Changelog](https://github.com/expressjs/serve-static/blob/v1.16.2/HISTORY.md )
- [Commits](expressjs/serve-static@v1.15.0...v1.16.2)
Updates `express` from 4.19.2 to 4.21.0
- [Release notes](https://github.com/expressjs/express/releases )
- [Changelog](https://github.com/expressjs/express/blob/4.21.0/History.md )
- [Commits](expressjs/express@4.19.2...4.21.0)
---
updated-dependencies:
- dependency-name: serve-static
dependency-type: indirect
- dependency-name: express
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-09-20 07:46:20 +08:00
96cc4d9397
Bug: Executor - Fix executor for Benchmark Execution Without Explicit Framework Field ( #636 )
...
**Description**
Fix executor for Benchmark Execution Without Explicit Framework Field
2024-08-20 16:52:20 -07:00
7af75df392
Bug Fix: Data Diagnosis - Fix bug of failure test and warning of pandas in data diagnosis ( #638 )
...
**Description**
Fix bug of failure test and warning of pandas in data diagnosis.
**Major Revision**
- fix warning of pandas in replace and fillna due to type downcast
- fix bug of failure check function only check one matched metric rather
than all matched metrics
- fix bug when converting regex into str of metrics when there're more
than one match group
2024-08-16 09:04:24 +08:00
46a5792915
Bug Fix - Update Docker Exec Command for Persistent HPCX Environment ( #635 )
...
Add 10-hpcx.sh to /etc/profile.d
Update the Docker exec command to ensure a persistent HPCX environment.
2024-08-13 16:35:01 +00:00
9de841bc95
Use `types-setuptools` as `types-pkg_resources` is Yanked ( #637 )
...
* https://pypi.org/project/types-pkg-resources/
* Use types-setuptools instead
2024-08-08 22:30:37 +08:00
2101e933cc
CI/CD - Fix MSCCL build error in CUDA12.4 docker build pipeline ( #633 )
...
**Description**
Fix MSCCL build error in CUDA12.4 docker build pipeline due to OOM
issue.
2024-07-28 23:43:06 +00:00
e304cf1572
Benchmarks: Micro benchmarks - add support for NVIDIA L4/L40/L40s GPUs in gemm-flops ( #634 )
...
**Description**
Add support GPU ARCH 8.9 for NVIDIA L4/L40/L40s GPUs in gemm-flops.
2024-07-26 02:42:17 +00:00
4e27142a59
Bump express from 4.18.2 to 4.19.2 in /website ( #618 )
...
Bumps [express](https://github.com/expressjs/express ) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases )
- [Changelog](https://github.com/expressjs/express/blob/master/History.md )
- [Commits](expressjs/express@4.18.2...4.19.2)
---
updated-dependencies:
- dependency-name: express
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-07-26 09:12:11 +08:00
b4945fb29c
Bump ws from 6.2.2 to 6.2.3 in /website ( #629 )
...
Bumps [ws](https://github.com/websockets/ws ) from 6.2.2 to 6.2.3.
- [Release notes](https://github.com/websockets/ws/releases )
- [Commits](websockets/ws@6.2.2...6.2.3)
---
updated-dependencies:
- dependency-name: ws
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-07-25 15:59:48 +08:00
a4c87da0ac
Docs - fix typos ( #628 )
...
Docs - fix typos
2024-07-25 03:49:19 +00:00
4102302a96
Bump ip from 1.1.5 to 1.1.9 in /website ( #610 )
...
Bumps [ip](https://github.com/indutny/node-ip ) from 1.1.5 to 1.1.9.
- [Commits](indutny/node-ip@v1.1.5...v1.1.9)
---
updated-dependencies:
- dependency-name: ip
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-07-25 02:15:52 +00:00
6e556d76e8
Bump follow-redirects from 1.14.8 to 1.15.6 in /website ( #613 )
...
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects ) from 1.14.8 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases )
- [Commits](follow-redirects/follow-redirects@v1.14.8...v1.15.6)
---
updated-dependencies:
- dependency-name: follow-redirects
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-07-24 23:59:58 +00:00
1362732c79
Docs - Add BibTeX in README and repo ( #632 )
...
Add BibTeX for citation in README and repo.
2024-07-23 18:31:21 -07:00
9a3ce39d5a
Update omegaconf version to 2.3.0 ( #631 )
...
Update `omegaconf` version to
[2.3.0](https://pypi.org/project/omegaconf/2.3.0/ ) as omegaconf 2.0.6
has a non-standard dependency specifier PyYAML>=5.1.*. pip 24.1 will
enforce this behaviour change.
Discussion can be found at https://github.com/pypa/pip/issues/12063 .
2024-07-23 14:46:28 -07:00
7435f10a22
Dockerfile - Add CUDA 12.4 dockerfile ( #619 )
...
**Description**
Add CUDA 12.4 dockerfile.
**Major Revision**
- upgrade nvidia docker into 23.04
**Minor Revision**
- upgrade hpcx into 2.18
2024-04-22 06:36:19 +00:00
dc3846cbd4
Dockerfile - Upgrade mlc to v3.11 ( #620 )
...
**Description**
Upgrade mlc to v3.11.
2024-04-18 10:59:36 +08:00
cc89ee591c
Benchmarks: Revise Code - Add hipblasLt tuning to dist-inference cpp implementation ( #616 )
...
**Description**
Adds hipblasLt tuning to dist-inference cpp implementation.
2024-04-02 09:56:33 +08:00
eeaa9b1ac9
Bug Fix - Bug fix for cuda 12.2 dockerfile LD_LIBRARY_PATH issue ( #614 )
...
**Description**
Cuda 12.2 image will report undfined symbol error due to incomplete
LD_LIBRARY_PATH:
### How to reproduce:
1. Deploy sb with cuda12.2 image
```
sb deploy -f local.ini -i superbench/superbench:v0.10.0-cuda12.2
```
2. Enter to the container
```
sudo docker exec -it sb-workspace bash
```
3. Execute `mpirun`:
```
root@sb-container:~# mpirun
mpirun: symbol lookup error: mpirun: undefined symbol: opal_libevent2022_event_base_loop
```
### Fix to fix
* Append hpcx_load into /etc/bash.bashrc for updaing env LD_LIBRARY_PATH in each time
---------
2024-03-21 15:05:55 +00:00
2c88db907f
Release - SuperBench v0.10.0 ( #607 )
...
**Description**
Cherry-pick bug fixes from v0.10.0 to main.
**Major Revisions**
* Benchmarks: Microbenchmark - Support different hipblasLt data types in dist_inference #590
* Benchmarks: Microbenchmark - Support in-place for NCCL/RCCL benchmark #591
* Bug Fix - Fix NUMA Domains Swap Issue in NDv4 Topology File #592
* Benchmarks: Microbenchmark - Add data type option for NCCL and RCCL tests #595
* Benchmarks: Bug Fix - Make metrics of dist-inference-cpp aligned with PyTorch version #596
* CI/CD - Add ndv5 topo file #597
* Benchmarks: Microbenchmark - Improve AMD GPU P2P performance with fine-grained GPU memory #593
* Benchmarks: Build Pipeline - fix nccl and nccl test version to 2.18.3 to resolve hang issue in cuda12.2 docker #599
* Dockerfile - Bug fix for rocm docker build and deploy #598
* Benchmarks: Microbenchmark - Adapt to hipblasLt data type changes #603
* Benchmarks: Micro benchmarks - Update hipblaslt metric unit to tflops #604
* Monitor - Upgrade pyrsmi to amdsmi python library. #601
* Benchmarks: Micro benchmarks - add fp8 and initialization for hipblaslt benchmark #605
* Dockerfile - Add rocm6.0 dockerfile #602
* Bug Fix - Bug fix for latest megatron-lm benchmark #600
* Docs - Upgrade version and release note #606
Co-authored-by: Ziyue Yang <ziyyang@microsoft.com>
Co-authored-by: Yang Wang <yangwang1@microsoft.com>
Co-authored-by: Yuting Jiang <yutingjiang@microsoft.com>
Co-authored-by: guoshzhao <guzhao@microsoft.com>
2024-01-08 05:40:52 +00:00
2c2096ed83
Benchmark: Revision - Fix -O2 option passing in gpu_copy ROCm build ( #589 )
...
**Description**
`add_compile_options` will not work for ROCm build, change it to setting
`CMAKE_CXX_FLAGS`.
2023-12-11 04:34:51 +00:00
719a427fe7
Benchmarks: Microbenchmark - Add distributed inference benchmark cpp implementation ( #586 )
...
**Description**
Add distributed inference benchmark cpp implementation.
2023-12-11 06:53:51 +08:00
1f5031bd74
Dockerfile - Upgrade to rocm5.7 dockerfile ( #587 )
...
**Description**
upgrade to rocm5.7 dockerfile.
---------
Co-authored-by: yukirora <yuting.jiang@microsoft.com>
2023-12-09 17:41:12 +00:00
4fa60be7cd
Benchmarks: Micro benchmark - Add one-to-all, all-to-one, all-to-all support to gpu_copy_bw_performance ( #588 )
...
**Description**
Add one-to-all, all-to-one, all-to-all support to
gpu_copy_bw_performance, and fix performance bug in gpu_copy
2023-12-08 23:22:38 +08:00
6ef3a0110f
Benchmarks: Add MSCCL Support for Nvidia GPU ( #584 )
...
**Description**
Add MSCCL support for Nvidia GPU
2023-12-07 19:57:28 +08:00
dd5a6329ed
Benchmarks: Add benchmark: Megatron-LM/Megatron-Deepspeed GPT pretrain benchmark ( #582 )
...
**Description**
Megatron-LM/Megatron-Deepspeed GPT pretrain benchmark
2023-12-07 09:37:09 +08:00
254ea7feba
Benchmarks: Micro benchmark - Add graph mode in NCCL/RCCL benchmarks for latency metrics ( #583 )
...
**Description**
Revise NCCL/RCCL benchmarks to graph mode add latency metrics.
2023-12-05 16:48:13 +08:00
9ae8c67093
Benchmarks: micro benchmark - Support cpu-gpu and gpu-cpu in ib-validation ( #581 )
...
**Description**
Benchmarks: micro benchmark - Support cpu-gpu and gpu-cpu in
ib-validation
**Major Revision**
- Support cpu-gpu and gpu-cpu in ib-validation
**Minor Revision**
- support multi msg size, multi direction, multi ib commands in
ib-validation
2023-12-04 22:20:46 +08:00
028819b388
Monitor - Add support for AMD GPU. ( #580 )
...
**Description**
Add AMD support in monitor.
**Major Revision**
- Add library pyrsmi to collect metrics.
- Currently can get device_utilization, device_power, device_used_memory
and device_total_memory.
2023-11-27 18:45:56 +08:00
1ad1c21c38
Dockerfile - Upgrade Docker image to CUDA 12.2 ( #577 )
...
Upgrade Docker image to CUDA 12.2 for H100:
* upgrade base image to 23.10
* fix onnxruntime version in python3.10
* fix compilation errors
2023-11-22 13:48:18 +00:00
2235e084ab
Benchmarks: Micro benchmark - add initialization options for rocm gemm flops ( #578 )
...
**Description**
add initialization options for rocm gemm flops.
2023-11-22 12:52:22 +00:00
79089b6517
Benchmarks: Micro benchmark - Add hipBLASLt function benchmark ( #576 )
...
**Description**
hipblaslt function benchmark and rebase cublaslt function benchmark.
2023-11-22 19:48:10 +08:00
9f4880cb8e
Analyzer - Generate baseline given results from multiple nodes. ( #575 )
...
**Description**
Generate baseline given results from multiple nodes.
**Major Revision**
- Add sub command `sb result generate-baseline`
- Add UT and docs
---------
Co-authored-by: 454314380 <454314380@qq.com>
Co-authored-by: Yuting Jiang <yutingjiang@microsoft.com>
2023-11-22 14:42:32 +08:00
f53d941a22
Benchmarks: micro benchmarks - add int8 support for cublaslt function ( #574 )
...
**Description**
add int8 support for cublaslt function.
2023-11-20 11:21:20 +08:00
c7800bb8e0
Bug Fix - remove cp ptx file command in gpu burn test ( #567 )
...
**Description**
remove cp ptx file in gpu burn test since the command is run inside
self.args.bin_dir dir.
d246bab430/superbench/benchmarks/micro_benchmarks/micro_base.py (L183)
2023-11-14 03:52:56 +00:00
ce3737f98b
Bump @babel/traverse from 7.14.5 to 7.23.2 in /website ( #566 )
...
Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse ) from 7.14.5 to 7.23.2.
- [Release notes](https://github.com/babel/babel/releases )
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md )
- [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-traverse )
---
updated-dependencies:
- dependency-name: "@babel/traverse"
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-07 10:36:42 +08:00
07477c3bae
Bump postcss from 8.3.5 to 8.4.31 in /website ( #564 )
...
Bumps [postcss](https://github.com/postcss/postcss ) from 8.3.5 to 8.4.31.
- [Release notes](https://github.com/postcss/postcss/releases )
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md )
- [Commits](postcss/postcss@8.3.5...8.4.31)
---
updated-dependencies:
- dependency-name: postcss
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-05 11:35:49 +00:00
d246bab430
Dockerfile - update mlc version into 3.10 for cuda and rocm dockerfiles ( #562 )
...
**Description**
Update mlc version into 3.10 for cuda and rocm dockerfiles to be
consistent with cuda12 dockerfile
Co-authored-by: yukirora <yuting.jiang@microsoft.com>
2023-10-23 11:21:17 +08:00
27a10811af
Benchmarks: micro benchmark - source code for evaluating NVDEC decoding performance ( #560 )
...
**Description**
source code for evaluating NVDEC decoding performance.
---------
Co-authored-by: yukirora <yuting.jiang@microsoft.com>
2023-08-22 10:56:33 +00:00
6c0205cece
Benchmarks: micro benchmarks - add source code for DirectXRenderPerf ( #549 )
...
**Description**
add source code for DirectXRenderPerf.
---------
Co-authored-by: yukirora <yuting.jiang@microsoft.com>
2023-08-18 05:17:04 +00:00
Hongtao Zhang
Yifan Xiong
dependabot[bot]
pdr
Yuting Jiang
Yang Wang
omahs
Ziyue Yang
guoshzhao