* new cuda framework
* add histogram construction kernel
* before removing multi-gpu
* new cuda framework
* tree learner cuda kernels
* single tree framework ready
* single tree training framework
* remove comments
* boosting with cuda
* optimize for best split find
* data split
* move boosting into cuda
* parallel synchronize best split point
* merge split data kernels
* before code refactor
* use tasks instead of features as units for split finding
* refactor cuda best split finder
* fix configuration error with small leaves in data split
* skip histogram construction of too small leaf
* skip split finding of invalid leaves
stop when no leaf to split
* support row wise with CUDA
* copy data for split by column
* copy data from host to CPU by column for data partition
* add synchronize best splits for one leaf from multiple blocks
* partition dense row data
* fix sync best split from task blocks
* add support for sparse row wise for CUDA
* remove useless code
* add l2 regression objective
* sparse multi value bin enabled for CUDA
* fix cuda ranking objective
* support for number of items <= 2048 per query
* speedup histogram construction by interleaving global memory access
* split optimization
* add cuda tree predictor
* remove comma
* refactor objective and score updater
* before use struct
* use structure for split information
* use structure for leaf splits
* return CUDASplitInfo directly after finding best split
* split with CUDATree directly
* use cuda row data in cuda histogram constructor
* clean src/treelearner/cuda
* gather shared cuda device functions
* put shared CUDA functions into header file
* change smaller leaf from <= back to < for consistent result with CPU
* add tree predictor
* remove useless cuda_tree_predictor
* predict on CUDA with pipeline
* add global sort algorithms
* add global argsort for queries with many items in ranking tasks
* remove limitation of maximum number of items per query in ranking
* add cuda metrics
* fix CUDA AUC
* remove debug code
* add regression metrics
* remove useless file
* don't use mask in shuffle reduce
* add more regression objectives
* fix cuda mape loss
add cuda xentropy loss
* use template for different versions of BitonicArgSortDevice
* add multiclass metrics
* add ndcg metric
* fix cross entropy objectives and metrics
* fix cross entropy and ndcg metrics
* add support for customized objective in CUDA
* complete multiclass ova for CUDA
* separate cuda tree learner
* use shuffle based prefix sum
* clean up cuda_algorithms.hpp
* add copy subset on CUDA
* add bagging for CUDA
* clean up code
* copy gradients from host to device
* support bagging without using subset
* add support of bagging with subset for CUDAColumnData
* add support of bagging with subset for dense CUDARowData
* refactor copy sparse subrow
* use copy subset for column subset
* add reset train data and reset config for CUDA tree learner
add deconstructors for cuda tree learner
* add USE_CUDA ifdef to cuda tree learner files
* check that dataset doesn't contain CUDA tree learner
* remove printf debug information
* use full new cuda tree learner only when using single GPU
* disable all CUDA code when using CPU version
* recover main.cpp
* add cpp files for multi value bins
* update LightGBM.vcxproj
* update LightGBM.vcxproj
fix lint errors
* fix lint errors
* fix lint errors
* update Makevars
fix lint errors
* fix the case with 0 feature and 0 bin
fix split finding for invalid leaves
create cuda column data when loaded from bin file
* fix lint errors
hide GetRowWiseData when cuda is not used
* recover default device type to cpu
* fix na_as_missing case
fix cuda feature meta information
* fix UpdateDataIndexToLeafIndexKernel
* create CUDA trees when needed in CUDADataPartition::UpdateTrainScore
* add refit by tree for cuda tree learner
* fix test_refit in test_engine.py
* create set of large bin partitions in CUDARowData
* add histogram construction for columns with a large number of bins
* add find best split for categorical features on CUDA
* add bitvectors for categorical split
* cuda data partition split for categorical features
* fix split tree with categorical feature
* fix categorical feature splits
* refactor cuda_data_partition.cu with multi-level templates
* refactor CUDABestSplitFinder by grouping task information into struct
* pre-allocate space for vector split_find_tasks_ in CUDABestSplitFinder
* fix misuse of reference
* remove useless changes
* add support for path smoothing
* virtual destructor for LightGBM::Tree
* fix overlapped cat threshold in best split infos
* reset histogram pointers in data partition and spllit finder in ResetConfig
* comment useless parameter
* fix reverse case when na is missing and default bin is zero
* fix mfb_is_na and mfb_is_zero and is_single_feature_column
* remove debug log
* fix cat_l2 when one-hot
fix gradient copy when data subset is used
* switch shared histogram size according to CUDA version
* gpu_use_dp=true when cuda test
* revert modification in config.h
* fix setting of gpu_use_dp=true in .ci/test.sh
* fix linter errors
* fix linter error
remove useless change
* recover main.cpp
* separate cuda_exp and cuda
* fix ci bash scripts
add description for cuda_exp
* add USE_CUDA_EXP flag
* switch off USE_CUDA_EXP
* revert changes in python-packages
* more careful separation for USE_CUDA_EXP
* fix CUDARowData::DivideCUDAFeatureGroups
fix set fields for cuda metadata
* revert config.h
* fix test settings for cuda experimental version
* skip some tests due to unsupported features or differences in implementation details for CUDA Experimental version
* fix lint issue by adding a blank line
* fix lint errors by resorting imports
* fix lint errors by resorting imports
* fix lint errors by resorting imports
* merge cuda.yml and cuda_exp.yml
* update python version in cuda.yml
* remove cuda_exp.yml
* remove unrelated changes
* fix compilation warnings
fix cuda exp ci task name
* recover task
* use multi-level template in histogram construction
check split only in debug mode
* ignore NVCC related lines in parameter_generator.py
* update job name for CUDA tests
* apply review suggestions
* Update .github/workflows/cuda.yml
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* Update .github/workflows/cuda.yml
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* update header
* remove useless TODOs
* remove [TODO(shiyu1994): constrain the split with min_data_in_group] and record in #5062
* #include <LightGBM/utils/log.h> for USE_CUDA_EXP only
* fix include order
* fix include order
* remove extra space
* address review comments
* add warning when cuda_exp is used together with deterministic
* add comment about gpu_use_dp in .ci/test.sh
* revert changing order of included headers
Co-authored-by: Yu Shi <shiyu1994@qq.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* cmake: use object library to avoid duplicate compilation.
* debug: verbose make log for building r package.
* Include /usr/local/include for AppleClang.
* Revert "debug: verbose make log for building r package."
* update cmake comment and fix indentation
* debug cmake USE_DEBUG.
* Revert "debug cmake USt E_DEBUG."
* Add -fPIC for building shared library.
* Always set -fPIC for non MSVC compiler.
* debug: print exception in setup.py
* debug: print cmake output for vs build.
* debug: set opencl related target_xxx on lightgbm_objs.
* Define compile definitions, link libraries on lightgbm_objs.
* Add PUBLIC to target_link_libraries to expose library dependency.
* Use target_link_libraries on object library.
This should propagate usage requirements.
* Fix CUDA linking.
Linking object library (lightgbm_objs) to object library (histograms)
does not linked objects.
* Use PUBLIC link for lightgbm lib.
* Set cuda related properties on final targets.
* Remove debugging changes.
Revert "debug: print exception in setup.py"
Revert "debug: print cmake output for vs build."
etc.
* Remove -D_lightgbm_EXPORTS.
* Revert to add -fPIC only for NOT USE_DEBUG.
* Enable PIC for shared lib.
* Fix enable PIC.
* Use -fPIC for shared lib.
* testlightgbm depends only on object files.
* tweak build for R.
* Try to remove OpenMP related include dir settings.
* link with openmp for capi object library.
* Use PUBLIC for _lightgbm target_link_libraries.
* Try removing exports definition.
* fix typo
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* fix typo
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* Add some comments for cmake code.
* Try to fix cmake warnings CUDA.
* revert accidentally commited R-package path change.
* Try to fix cmake CUDA warnings, set for _lightgbm target.
* Try to fix cmake CUDA warnings, set for lightgbm target.
* empty commit to trigger ci
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* run cpp tests at CI
* Update docs/Installation-Guide.rst
Co-authored-by: James Lamb <jaylamb20@gmail.com>
Co-authored-by: James Lamb <jaylamb20@gmail.com>
* Add Eigen library.
* Working for simple test.
* Apply changes to config params.
* Handle nan data.
* Update docs.
* Add test.
* Only load raw data if boosting=gbdt_linear
* Remove unneeded code.
* Minor updates.
* Update to work with sk-learn interface.
* Update to work with chunked datasets.
* Throw error if we try to create a Booster with an already-constructed dataset having incompatible parameters.
* Save raw data in binary dataset file.
* Update docs and fix parameter checking.
* Fix dataset loading.
* Add test for regularization.
* Fix bugs when saving and loading tree.
* Add test for load/save linear model.
* Remove unneeded code.
* Fix case where not enough leaf data for linear model.
* Simplify code.
* Speed up code.
* Speed up code.
* Simplify code.
* Speed up code.
* Fix bugs.
* Working version.
* Store feature data column-wise (not fully working yet).
* Fix bugs.
* Speed up.
* Speed up.
* Remove unneeded code.
* Small speedup.
* Speed up.
* Minor updates.
* Remove unneeded code.
* Fix bug.
* Fix bug.
* Speed up.
* Speed up.
* Simplify code.
* Remove unneeded code.
* Fix bug, add more tests.
* Fix bug and add test.
* Only store numerical features
* Fix bug and speed up using templates.
* Speed up prediction.
* Fix bug with regularisation
* Visual studio files.
* Working version
* Only check nans if necessary
* Store coeff matrix as an array.
* Align cache lines
* Align cache lines
* Preallocation coefficient calculation matrices
* Small speedups
* Small speedup
* Reverse cache alignment changes
* Change to dynamic schedule
* Update docs.
* Refactor so that linear tree learner is not a separate class.
* Add refit capability.
* Speed up
* Small speedups.
* Speed up add prediction to score.
* Fix bug
* Fix bug and speed up.
* Speed up dataload.
* Speed up dataload
* Use vectors instead of pointers
* Fix bug
* Add OMP exception handling.
* Change return type of LGBM_BoosterGetLinear to bool
* Change return type of LGBM_BoosterGetLinear back to int, only parameter type needed to change
* Remove unused internal_parent_ property of tree
* Remove unused parameter to CreateTreeLearner
* Remove reference to LinearTreeLearner
* Minor style issues
* Remove unneeded check
* Reverse temporary testing change
* Fix Visual Studio project files
* Restore LightGBM.vcxproj.filters
* Speed up
* Speed up
* Simplify code
* Update docs
* Simplify code
* Initialise storage space for max num threads
* Move Eigen to include directory and delete unused files
* Remove old files.
* Fix so it compiles with mingw
* Fix gpu tree learner
* Change AddPredictionToScore back to const
* Fix python lint error
* Fix C++ lint errors
* Change eigen to a submodule
* Update comment
* Add the eigen folder
* Try to fix build issues with eigen
* Remove eigen files
* Add eigen as submodule
* Fix include paths
* Exclude eigen files from Python linter
* Ignore eigen folders for pydocstyle
* Fix C++ linting errors
* Fix docs
* Fix docs
* Exclude eigen directories from doxygen
* Update manifest to include eigen
* Update build_r to include eigen files
* Fix compiler warnings
* Store raw feature data as float
* Use float for calculating linear coefficients
* Remove eigen directory from GLOB
* Don't compile linear model code when building R package
* Fix doxygen issue
* Fix lint issue
* Fix lint issue
* Remove uneeded code
* Restore delected lines
* Restore delected lines
* Change return type of has_raw to bool
* Update docs
* Rename some variables and functions for readability
* Make tree_learner parameter const in AddScore
* Fix style issues
* Pass vectors as const reference when setting tree properties
* Make temporary storage of serial_tree_learner mutable so we can make the object's methods const
* Remove get_raw_size, use num_numeric_features instead
* Fix typo
* Make contains_nan_ and any_nan_ properties immutable again
* Remove data_has_nan_ property of tree
* Remove temporary test code
* Make linear_tree a dataset param
* Fix lint error
* Make LinearTreeLearner a separate class
* Fix lint errors
* Fix lint error
* Add linear_tree_learner.o
* Simulate omp_get_max_threads if openmp is not available
* Update PushOneData to also store raw data.
* Cast size to int
* Fix bug in ReshapeRaw
* Speed up code with multithreading
* Use OMP_NUM_THREADS
* Speed up with multithreading
* Update to use ArrayToString
* Fix tests
* Fix test
* Fix bug introduced in merge
* Minor updates
* Update docs
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* Initial CUDA work
* redirect log to python console (#3090)
* redir log to python console
* fix pylint
* Apply suggestions from code review
* Update basic.py
* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* Update c_api.h
* Apply suggestions from code review
* Apply suggestions from code review
* super-minor: better wording
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: StrikerRUS <nekit94-12@hotmail.com>
* re-order includes (fixes#3132) (#3133)
* Revert "re-order includes (fixes#3132) (#3133)" (#3153)
This reverts commit 656d2676c2.
* Missing change from previous rebase
* Minor cleanup and removal of development scripts.
* Only set gpu_use_dp on by default for CUDA. Other minor change.
* Fix python lint indentation problem.
* More python lint issues.
* Big lint cleanup - more to come.
* Another large lint cleanup - more to come.
* Even more lint cleanup.
* Minor cleanup so less differences in code.
* Revert is_use_subset changes
* Another rebase from master to fix recent conflicts.
* More lint.
* Simple code cleanup - add & remove blank lines, revert unneccessary format changes, remove added dead code.
* Removed parameters added for CUDA and various bug fix.
* Yet more lint and unneccessary changes.
* Revert another change.
* Removal of unneccessary code.
* temporary appveyor.yml for building and testing
* Remove return value in ReSize
* Removal of unused variables.
* Code cleanup from reviewers suggestions.
* Removal of FIXME comments and unused defines.
* More reviewers comments cleanup.
* More reviewers comments cleanup.
* More reviewers comments cleanup.
* Fix config variables.
* Attempt to fix check-docs failure
* Update Paramster.rst for num_gpu
* Removing test appveyor.yml
* Add CUDA_RESOLVE_DEVICE_SYMBOLS to libraries to fix linking issue.
* Fixed handling of data elements less than 2K.
* More reviewers comments cleanup.
* Removal of TODO and fix printing of int64_t
* Add cuda change for CI testing and remove cuda from device_type in python.
* Missed one change form previous check-in
* Removal AdditionConfig and fix settings.
* Limit number of GPUs to one for now in CUDA.
* Update Parameters.rst for previous check-in
* Whitespace removal.
* Cleanup unused code.
* Changed uint/ushort/ulong to unsigned int/short/long to help Windows based CUDA compiler work.
* Lint change from previous check-in.
* Changes based on reviewers comments.
* More reviewer comment changes.
* Adding warning for is_sparse. Revert tmp_subset code. Only return FeatureGroupData if not is_multi_val_
* Fix so that CUDA code will compile even if you enable the SCORE_T_USE_DOUBLE define.
* Reviewer comment cleanup.
* Replace warning with Log message. Removal of some of the USE_CUDA. Fix typo and removal of pragma once.
* Remove PRINT debug for CUDA code.
* Allow to use of multiple GPUs for CUDA.
* More multi-GPUs enablement for CUDA.
* More code cleanup based on reviews comments.
* Update docs with latest config changes.
Co-authored-by: Gordon Fossum <fossum@us.ibm.com>
Co-authored-by: ChipKerchner <ckerchne@linux.vnet.ibm.com>
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: StrikerRUS <nekit94-12@hotmail.com>
Co-authored-by: James Lamb <jaylamb20@gmail.com>
* [R-package] started cutting over from custom R-to-C interface to R.h
* replaced LGBM_SE with SEXP
* fixed error about ocnflicting definitions of length
* got linking working
* more stuff
* eliminated R CMD CHECK note about printing
* switched from hard-coded include dir to the one from FindLibR.cmake
* cleaned up formatting in FindLibR.cmake
* commented-out everything in CI that does not touch R
* more changes
* trying to get better logs
* tried ignoring
* added error message to confirm a suspicion
* still trying to find R during R CMD CHECK
* restore full CI
* fixed comment
* Update R-package/src/cmake/modules/FindLibR.cmake
* changed strategy for finding LIBR_HOME on Windows
* Removed 32-bit Windows stuff in FindLibR.cmake
* Update R-package/src/cmake/modules/FindLibR.cmake
* Update CMakeLists.txt
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update CMakeLists.txt
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* removed some duplication in cmake scripts
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* added LIBR_CORE_LIBRARY back
* small fixes to CMakeLists
* simplified FindLibR.cmake
* some fixes for windows
* Apply suggestions from code review
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* allowed for directly passing LIBR_EXECUTABLE to FindLibR.cmake
* reorganized FindLibR.cmake to catch more cases
* clean up inconsistencies in R calls in FindLibR.cmake
* Update R-package/src/cmake/modules/FindLibR.cmake
Co-Authored-By: Nikita Titov <nekit94-08@mail.ru>
* removed unnecessary log messages
* removed unnecessary unset() call
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
* [R-package] Introduced define LGB_BUILD_R to control compilation of lib_lightgbm for the R package
* fixed comment
* switched to cmake options
* fixed typo
* various minor style, docs and cpplint improvements
* fixed typo in warning
* fix recently added cpplint errors
* move note for params upper in description for consistency
* updated installation guide
* updated Python installation guide
* added note about opencl path to Windows section
* added space before path in message
* minor correction for option description in Python installation guide
shiyu1994
Nikita Titov
Chen Yufei
James Lamb
Robin Dong
Philip Hyunsu Cho
Belinda Trotta
TP Boudreau
Chip Kerchner
Darby Payne
Guolin Ke