зеркало из https://github.com/microsoft/LightGBM.git
193 строки
7.0 KiB
ReStructuredText
193 строки
7.0 KiB
ReStructuredText
LightGBM GPU Tutorial
|
||
=====================
|
||
|
||
The purpose of this document is to give you a quick step-by-step tutorial on GPU training.
|
||
|
||
We will use the GPU instance on `Microsoft Azure cloud computing platform`_ for demonstration,
|
||
but you can use any machine with modern AMD or NVIDIA GPUs.
|
||
|
||
GPU Setup
|
||
---------
|
||
|
||
You need to launch a ``NV`` type instance on Azure (available in East US, North Central US, South Central US, West Europe and Southeast Asia zones)
|
||
and select Ubuntu 16.04 LTS as the operating system.
|
||
|
||
For testing, the smallest ``NV6`` type virtual machine is sufficient, which includes 1/2 M60 GPU, with 8 GB memory, 180 GB/s memory bandwidth and 4,825 GFLOPS peak computation power.
|
||
Don't use the ``NC`` type instance as the GPUs (K80) are based on an older architecture (Kepler).
|
||
|
||
First we need to install minimal NVIDIA drivers and OpenCL development environment:
|
||
|
||
::
|
||
|
||
sudo apt-get update
|
||
sudo apt-get install --no-install-recommends nvidia-375
|
||
sudo apt-get install --no-install-recommends nvidia-opencl-icd-375 nvidia-opencl-dev opencl-headers
|
||
|
||
After installing the drivers you need to restart the server.
|
||
|
||
::
|
||
|
||
sudo init 6
|
||
|
||
After about 30 seconds, the server should be up again.
|
||
|
||
If you are using an AMD GPU, you should download and install the `AMDGPU-Pro`_ driver and also install packages ``ocl-icd-libopencl1`` and ``ocl-icd-opencl-dev``.
|
||
|
||
Build LightGBM
|
||
--------------
|
||
|
||
Now install necessary building tools and dependencies:
|
||
|
||
::
|
||
|
||
sudo apt-get install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev
|
||
|
||
The ``NV6`` GPU instance has a 320 GB ultra-fast SSD mounted at ``/mnt``.
|
||
Let's use it as our workspace (skip this if you are using your own machine):
|
||
|
||
::
|
||
|
||
sudo mkdir -p /mnt/workspace
|
||
sudo chown $(whoami):$(whoami) /mnt/workspace
|
||
cd /mnt/workspace
|
||
|
||
Now we are ready to checkout LightGBM and compile it with GPU support:
|
||
|
||
::
|
||
|
||
git clone --recursive https://github.com/microsoft/LightGBM
|
||
cd LightGBM
|
||
cmake -B build -S . -DUSE_GPU=1
|
||
# if you have installed NVIDIA CUDA to a customized location, you should specify paths to OpenCL headers and library like the following:
|
||
# cmake -B build -S . -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/
|
||
cmake --build build -j$(nproc)
|
||
|
||
You will see two binaries are generated, ``lightgbm`` and ``lib_lightgbm.so``.
|
||
|
||
If you are building on macOS, you probably need to remove macro ``BOOST_COMPUTE_USE_OFFLINE_CACHE`` in ``src/treelearner/gpu_tree_learner.h`` to avoid a known crash bug in Boost.Compute.
|
||
|
||
Install Python Interface (optional)
|
||
-----------------------------------
|
||
|
||
If you want to use the Python interface of LightGBM, you can install it now (along with some necessary Python-package dependencies):
|
||
|
||
::
|
||
|
||
sudo apt-get -y install python-pip
|
||
sudo -H pip install setuptools numpy scipy scikit-learn -U
|
||
sudo sh ./build-python.sh install --precompile
|
||
|
||
You need to set an additional parameter ``"device" : "gpu"`` (along with your other options like ``learning_rate``, ``num_leaves``, etc) to use GPU in Python.
|
||
|
||
You can read our `Python-package Examples`_ for more information on how to use the Python interface.
|
||
|
||
Dataset Preparation
|
||
-------------------
|
||
|
||
Using the following commands to prepare the Higgs dataset:
|
||
|
||
::
|
||
|
||
git clone https://github.com/guolinke/boosting_tree_benchmarks.git
|
||
cd boosting_tree_benchmarks/data
|
||
wget "https://archive.ics.uci.edu/ml/machine-learning-databases/00280/HIGGS.csv.gz"
|
||
gunzip HIGGS.csv.gz
|
||
python higgs2libsvm.py
|
||
cd ../..
|
||
ln -s boosting_tree_benchmarks/data/higgs.train
|
||
ln -s boosting_tree_benchmarks/data/higgs.test
|
||
|
||
Now we create a configuration file for LightGBM by running the following commands (please copy the entire block and run it as a whole):
|
||
|
||
::
|
||
|
||
cat > lightgbm_gpu.conf <<EOF
|
||
max_bin = 63
|
||
num_leaves = 255
|
||
num_iterations = 50
|
||
learning_rate = 0.1
|
||
tree_learner = serial
|
||
task = train
|
||
is_training_metric = false
|
||
min_data_in_leaf = 1
|
||
min_sum_hessian_in_leaf = 100
|
||
ndcg_eval_at = 1,3,5,10
|
||
device = gpu
|
||
gpu_platform_id = 0
|
||
gpu_device_id = 0
|
||
EOF
|
||
echo "num_threads=$(nproc)" >> lightgbm_gpu.conf
|
||
|
||
GPU is enabled in the configuration file we just created by setting ``device=gpu``.
|
||
In this configuration we use the first GPU installed on the system (``gpu_platform_id=0`` and ``gpu_device_id=0``). If ``gpu_platform_id`` or ``gpu_device_id`` is not set, the default platform and GPU will be selected.
|
||
You might have multiple platforms (AMD/Intel/NVIDIA) or GPUs. You can use the `clinfo`_ utility to identify the GPUs on each platform. On Ubuntu, you can install ``clinfo`` by executing ``sudo apt-get install clinfo``. If you have a discrete GPU by AMD/NVIDIA and an integrated GPU by Intel, make sure to select the correct ``gpu_platform_id`` to use the discrete GPU.
|
||
|
||
Run Your First Learning Task on GPU
|
||
-----------------------------------
|
||
|
||
Now we are ready to start GPU training!
|
||
|
||
First we want to verify the GPU works correctly.
|
||
Run the following command to train on GPU, and take a note of the AUC after 50 iterations:
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train valid=higgs.test objective=binary metric=auc
|
||
|
||
Now train the same dataset on CPU using the following command. You should observe a similar AUC:
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train valid=higgs.test objective=binary metric=auc device=cpu
|
||
|
||
Now we can make a speed test on GPU without calculating AUC after each iteration.
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train objective=binary metric=auc
|
||
|
||
Speed test on CPU:
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train objective=binary metric=auc device=cpu
|
||
|
||
You should observe over three times speedup on this GPU.
|
||
|
||
The GPU acceleration can be used on other tasks/metrics (regression, multi-class classification, ranking, etc) as well.
|
||
For example, we can train the Higgs dataset on GPU as a regression task:
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train objective=regression_l2 metric=l2
|
||
|
||
Also, you can compare the training speed with CPU:
|
||
|
||
::
|
||
|
||
./lightgbm config=lightgbm_gpu.conf data=higgs.train objective=regression_l2 metric=l2 device=cpu
|
||
|
||
Further Reading
|
||
---------------
|
||
|
||
- `GPU Tuning Guide and Performance Comparison <./GPU-Performance.rst>`__
|
||
|
||
- `GPU SDK Correspondence and Device Targeting Table <./GPU-Targets.rst>`__
|
||
|
||
Reference
|
||
---------
|
||
|
||
Please kindly cite the following article in your publications if you find the GPU acceleration useful:
|
||
|
||
Huan Zhang, Si Si and Cho-Jui Hsieh. "`GPU Acceleration for Large-scale Tree Boosting`_." SysML Conference, 2018.
|
||
|
||
.. _Microsoft Azure cloud computing platform: https://azure.microsoft.com/
|
||
|
||
.. _AMDGPU-Pro: https://www.amd.com/en/support
|
||
|
||
.. _Python-package Examples: https://github.com/microsoft/LightGBM/tree/master/examples/python-guide
|
||
|
||
.. _GPU Acceleration for Large-scale Tree Boosting: https://arxiv.org/abs/1706.08359
|
||
|
||
.. _clinfo: https://github.com/Oblomov/clinfo
|