Граф коммитов

495 Коммитов

Автор SHA1 Сообщение Дата
Nikita Titov d6ebd063ff
[ci][python] run isort in CI linting job (#3990)
* run isort in CI linting job

* workaround conda compatibility issues
2021-02-16 20:09:13 +03:00
Zhuyi Xue 4ae59494ab
[ci][python] apply isort to python-package/lightgbm/compat.py #3958 (#3968) 2021-02-16 15:05:33 +03:00
Zhuyi Xue 6110bd1585
[ci][python] apply isort to python-package/lightgbm/engine.py #3958 (#3970) 2021-02-16 15:04:21 +03:00
Zhuyi Xue af0c226057
[ci][python] apply isort to python-package/lightgbm/basic.py #3958 (#3967) 2021-02-16 05:22:12 +03:00
Zhuyi Xue 9b64b9c91b
[ci][python] apply isort to python-package/lightgbm/__init__.py #3958 (#3966) 2021-02-16 04:03:00 +03:00
Zhuyi Xue acb677416a
[ci][python] apply isort to python-package/lightgbm/sklearn.py #3958 (#3973) 2021-02-16 03:37:16 +03:00
Zhuyi Xue 332b0db5ef
[ci][python] apply isort to python-package/setup.py #3958 (#3974) 2021-02-15 23:44:21 +03:00
Zhuyi Xue e9ea85bd06
[ci][python] apply isort to python-package/lightgbm/plotting.py #3958 (#3972) 2021-02-15 23:41:21 +03:00
Zhuyi Xue 8d5c0343c2
[ci][python] apply isort to python-package/lightgbm/libpath.py #3958 (#3971) 2021-02-15 20:54:52 +03:00
Frank Fineis d74b1be904
reuse len(parts) as n_parts (#3985) 2021-02-15 09:04:09 -06:00
Zhuyi Xue 3b5470015b
[ci][python] apply isort to python-package/lightgbm/dask.py #3958 (#3969) 2021-02-15 00:52:31 -06:00
Tara Jawahar eda1effb52
[python-package] fix some warnings from mypy (#3891)
* minor mypy type errors fixed

* fix some warnings from mypy

* fix 3 mypy warnings

* selectively ignored some mypy errors

* minor mypy type errors fixed

* minor mypy type errors fixed

* minor mypy type errors fixed

* added import

* Update python-package/lightgbm/callback.py

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: James Lamb <jaylamb20@gmail.com>

Co-authored-by: James Lamb <jaylamb20@gmail.com>
2021-02-14 21:20:00 -06:00
Nikita Titov 15916a9504
[docs][python] fix shape description of returned result for predict_proba (#3933)
* Update dask.py

* Update sklearn.py
2021-02-10 03:15:29 +03:00
James Lamb 06ed4337e0
[dask] [docs] Fix inaccuracies in API docs for Dask module (fixes #3871) (#3930)
* got fit() working

* add predict()

* predict_proba()

* remove custom objective docs

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix capitalization

* Update tests/python_package_test/test_dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-02-09 15:28:10 -06:00
James Lamb 37485fff5d
[dask] Add support for 'pred_leaf' in Dask estimators (fixes #3792) (#3919)
* fix tests

* fix tests

* fix test comments

* simplify tests

* Apply suggestions from code review
2021-02-07 13:17:28 -06:00
James Lamb fc6b71e08e
[dask] Support Dask dataframes with 'category' columns (fixes #3861) (#3908)
* add support for pandas categorical columns

* remove commented code

* quotes

* syntax error

* fix shape for ranker test

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_dask.py

* trying

* fix tests

* remove unnecessary debugging stuff

* skip accuracy checks on categorical

* use category columns as categorical features

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-02-07 01:19:49 +03:00
James Lamb fc2da2ec84
[dask] remove extra 'client' kwarg in DaskLGBMRegressor (#3906) 2021-02-03 16:32:23 -06:00
Nikita Titov b1e000c045
[dask] remove unused private _client attribute (#3904)
* Update test_dask.py

* Update dask.py

* Update .vsts-ci.yml

* Revert "Update .vsts-ci.yml"

This reverts commit 98422be5b5.
2021-02-03 10:44:08 -06:00
Nikita Titov 6373356735
[docs][dask] Add type of client_ property to docs (#3902)
* Update dask.py

* Update dask.py
2021-02-03 09:46:16 -06:00
James Lamb c3ac77b570
[dask] remove 'client' kwarg from fit() and predict() (fixes #3808) (#3883)
* starting on Dask client

* more docs stuff

* fix pickling

* just copy docstrings

* fit docs

* switch test order

* linting

* use client kwarg

* remove inner set_params()

* add type hints

* fix type hints

* remove commented code

* reorder

* fix tests, add client_ property

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix tests

* linting

* simplify

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-02-02 23:48:59 -06:00
Nikita Titov 763b5f3cdc
[CUDA][docs] explicitly require CUDA 9.0 or higher (#3885) 2021-02-01 14:09:53 +03:00
Nikita Titov e0887ae0c7
[docs] improve wordings and code style (#3887) 2021-02-01 02:28:01 +03:00
Nikita Titov be2fe2f554
[python] Start supporting Python 3.9 (#3693)
* start supporting Python 3.9

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update .appveyor.yml

* Update .vsts-ci.yml

* Update .appveyor.yml

* fixed conflicts

* continue
2021-01-31 18:18:06 +03:00
Nikita Titov 8040ef94bb
[docs] document CUDA version support (#3428)
* document CUDA version support

* address review comments

* collapse CUDA section in the guide

* remove Clang support from CUDA docs as we have never tested it
2021-01-31 17:08:06 +03:00
James Lamb ea8e47ea24
[dask] Add type hints in Dask package (#3866)
* add type hints in dask module

* starting on asserts

* remove unused code

* add hints for dtypes

* replace accidentally-removed docstrings

* revert unrelated change

* Update python-package/lightgbm/dask.py

* empty commit

* fix hints on group

* capitalize array

* hide hints in signatures

* empty commit

* sphinx version

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix hint for MatrixLike

* Update python-package/lightgbm/dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update docstring

* empty commit

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-29 11:59:21 -06:00
Nikita Titov 8ef874bd48
fix docs for machine_list_filename param (#3863) 2021-01-27 22:20:13 -06:00
James Lamb d4658fbb6f
[dask] add tests on warnings, fix incorrect variable in log (#3865)
* [dask] add tests on warnings, fix incorrect variable in log

* Update tests/python_package_test/test_dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-26 22:10:50 -06:00
Nikita Titov 59153b2875
[python] simplify param aliases handling (#3864)
* Update sklearn.py

* Update dask.py
2021-01-27 02:31:43 +03:00
Nikita Titov 066720eff6
[python] use better names for imported classes from extra libraries (#3862) 2021-01-27 00:36:13 +03:00
Nikita Titov 5312b955fe
[dask] fix Dask docstrings and mimic sklearn wrapper importing way (#3855)
* fix Dask docstrings and mimic sklearn importing way

* Update .vsts-ci.yml

* revert CI checks

* use import aliases for Dask classes

* check Dask is installed in _predict() func

* fix lint issues introduced during resolving merge conflicts

* Update dask.py
2021-01-26 22:44:40 +03:00
James Lamb 56b99d4ce3
[dask] rename _LGBMModel to _DaskLGBMModel (#3859) 2021-01-26 11:10:21 -06:00
James Lamb 9f70e9685d
[python-package] respect parameter aliases for network params (#3813)
* [dask] allow parameter aliases for tree_learner and local_listen_port (fixes #3671)

* num_thread too

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* empty commit

* add _choose_param_value

* revert param order change

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* just import deepcopy

* remove machines aliases

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-26 10:00:26 -06:00
Frank Fineis 113da3af60
[dask] [python] Store co-local data parts as dicts instead of lists (#3853)
* store data parts in dict instead of list

* simplify weight/group parts dict assignment
2021-01-26 00:05:58 +03:00
James Lamb 5cdaf1bd1e
[dask] warn if attempting to use tree_learner other than data parallel (#3848)
* [dask] warn if attempting to use tree_learn other than data parallel

* add param
2021-01-25 15:29:27 +03:00
Nikita Titov 36322ceeae
[dask][docs] initial setup for Dask docs (#3822)
* initial Dask docs

* fix MRO

* address review comments
2021-01-24 20:58:52 -06:00
Nikita Titov 98a85a83ce
[dask] Drop aliases of core network parameters (#3843)
* Update dask.py

* Update basic.py

* hotfix pop
2021-01-24 23:40:34 +03:00
Nikita Titov b7ccdaf066
[python] Allow to register custom logger in Python-package (#3820)
* centralize Python-package logging in one place

* continue

* fix test name

* removed unused import

* enhance test

* fix lint

* hotfix test

* workaround for GPU test

* remove custom logger from Dask-package

* replace one log func with flags by multiple funcs
2021-01-24 23:37:45 +03:00
James Lamb b8fc476e9c
[dask] [python-package] use keyword args for internal function calls (#3755)
* [dask] use keyword args for internal function calls

* add missing comma

* Update python-package/lightgbm/dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* revert whitespace changes

* test style

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-22 20:50:29 +03:00
Nikita Titov 6bb6164e3c
Move compute and eigen libraries to external_libs folder (#3809)
* move all submodules to external_libs folder

* Update .Rbuildignore

* Update MANIFEST.in

* Update .appveyor.yml

* Update CMakeLists.txt

* Update build_r.R

* Update test.sh

* Update setup.py

* Update CMakeLists.txt

* Update test.sh

* Update setup.py

* Update conf.py

* Update MANIFEST.in

* Update LightGBM.vcxproj

* continue

* test

* test

* Update setup.py

* hotfix

* revert CI tests
2021-01-22 17:45:43 +03:00
James Lamb d9a96c90cb
[dask] Support pred_contrib in Dask predict() methods (fixes #3713) (#3774)
* adding pred_contrib support

* add tests

* linting

* remove raw_score

* add pred kwargs

* faster tests

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* changes to tests

* Update tests/python_package_test/test_dask.py

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-22 00:33:10 -06:00
Frank Fineis 3c7e7e0b7e
[python-package] [dask] Add DaskLGBMRanker (#3708)
* ranker support wip

* fix ranker tests

* fix _make_ranking rnd gen bug, add sleep to help w stoch binding port failed exceptions

* add wait_for_workers to prevent Binding port exception

* another attempt to stabilize test_dask.py

* requested changes: docstrings, dask_ml, tuples for list_of_parts

* fix lint bug, add group param to test_ranker_local_predict

* decorator to skip tests with errors on fixture teardown

* remove gpu ranker tests, reduce make_ranking data complexity

* another attempt to
silence client, decorator does not silence fixture errors

* address requested changes on 1/20/20

* skip test_dask for all GPU tasks

* address changes requested on 1/21/21

* issubclass instead of __qualname__

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* parity in group docstr with sklearn

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* _make_ranking docstr cleanup

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-21 23:17:49 -06:00
James Lamb d107872a53
[dask] allow parameter aliases for local_listen_port, num_threads, tree_learner (fixes #3671) (#3789)
* [dask] allow parameter aliases for tree_learner and local_listen_port (fixes #3671)

* num_thread too

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* empty commit

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-20 11:21:36 -06:00
James Lamb 4007b34fa8
[dask] fix Dask import order (#3788) 2021-01-19 12:05:56 -06:00
Nikita Titov 787780851f
[docs] fix current RTD failures (#3787)
* fix docs

* Update basic.py

* Update engine.py
2021-01-19 17:26:45 +03:00
James Lamb 0e5eb9e372
[docs] expand documentation on 'group' for ranking task (#3772)
* [python-package] expand documentation on 'group' for ranking task

* add R package

* update Query Data section

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix typo in group example

* regenerate parameters

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* regenerate R docs

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-18 11:58:04 -06:00
James Lamb f6d2dce4ee
[dask] [python-package] Search for available ports when setting up network (fixes #3753) (#3766)
* starting work

* fixed port-binding issue on localhost

* minor cleanup

* updates

* getting closer

* definitely working for LocalCluster

* it works, it works

* docs

* add tests

* removing testing-only files

* linting

* Apply suggestions from code review

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* remove duplicated code

* remove unnecessary listen()

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-15 10:59:45 -06:00
Ray Bell 78d31d9ae3
[docs][python] add conda-forge install instructions (#3544)
* DOC: add conda-forge install instructions

* DOC: add conda-forge instructions

* DOC: fix hyperlink

* DOC: point to installation guide

* add detailed

* Update python-package/README.rst

Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update python-package/README.rst

Co-authored-by: James Lamb <jaylamb20@gmail.com>

* rm characters

* add pip install

* add :

* Update python-package/README.rst

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/README.rst

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* remove pip from header

* channel

Co-authored-by: James Lamb <jaylamb20@gmail.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
2021-01-11 20:01:25 +03:00
Nikita Titov 28e9875e18
remove line in MANIFEST for nonexistent file (#3711) 2021-01-03 17:31:50 -06:00
James Lamb 6cb968af2e
[python-package] remove unused Eigen files, compile with EIGEN_MPL2_ONLY (fixes #3684) (#3685)
* [python-package] remove unused Eigen files (fixes #3684)

* more changes

* add EIGEN_MPL2_ONLY in VS solution file

* fix VS project

* remove EIGEN_MPL2_ONLY define in linear_tree_learner

Co-authored-by: Nikita Titov <nekit94-12@hotmail.com>
2020-12-30 02:16:34 +03:00
Belinda Trotta fcfd4132e6
Trees with linear models at leaves (#3299)
* Add Eigen library.

* Working for simple test.

* Apply changes to config params.

* Handle nan data.

* Update docs.

* Add test.

* Only load raw data if boosting=gbdt_linear

* Remove unneeded code.

* Minor updates.

* Update to work with sk-learn interface.

* Update to work with chunked datasets.

* Throw error if we try to create a Booster with an already-constructed dataset having incompatible parameters.

* Save raw data in binary dataset file.

* Update docs and fix parameter checking.

* Fix dataset loading.

* Add test for regularization.

* Fix bugs when saving and loading tree.

* Add test for load/save linear model.

* Remove unneeded code.

* Fix case where not enough leaf data for linear model.

* Simplify code.

* Speed up code.

* Speed up code.

* Simplify code.

* Speed up code.

* Fix bugs.

* Working version.

* Store feature data column-wise (not fully working yet).

* Fix bugs.

* Speed up.

* Speed up.

* Remove unneeded code.

* Small speedup.

* Speed up.

* Minor updates.

* Remove unneeded code.

* Fix bug.

* Fix bug.

* Speed up.

* Speed up.

* Simplify code.

* Remove unneeded code.

* Fix bug, add more tests.

* Fix bug and add test.

* Only store numerical features

* Fix bug and speed up using templates.

* Speed up prediction.

* Fix bug with regularisation

* Visual studio files.

* Working version

* Only check nans if necessary

* Store coeff matrix as an array.

* Align cache lines

* Align cache lines

* Preallocation coefficient calculation matrices

* Small speedups

* Small speedup

* Reverse cache alignment changes

* Change to dynamic schedule

* Update docs.

* Refactor so that linear tree learner is not a separate class.

* Add refit capability.

* Speed up

* Small speedups.

* Speed up add prediction to score.

* Fix bug

* Fix bug and speed up.

* Speed up dataload.

* Speed up dataload

* Use vectors instead of pointers

* Fix bug

* Add OMP exception handling.

* Change return type of LGBM_BoosterGetLinear to bool

* Change return type of LGBM_BoosterGetLinear back to int, only parameter type needed to change

* Remove unused internal_parent_ property of tree

* Remove unused parameter to CreateTreeLearner

* Remove reference to LinearTreeLearner

* Minor style issues

* Remove unneeded check

* Reverse temporary testing change

* Fix Visual Studio project files

* Restore LightGBM.vcxproj.filters

* Speed up

* Speed up

* Simplify code

* Update docs

* Simplify code

* Initialise storage space for max num threads

* Move Eigen to include directory and delete unused files

* Remove old files.

* Fix so it compiles with mingw

* Fix gpu tree learner

* Change AddPredictionToScore back to const

* Fix python lint error

* Fix C++ lint errors

* Change eigen to a submodule

* Update comment

* Add the eigen folder

* Try to fix build issues with eigen

* Remove eigen files

* Add eigen as submodule

* Fix include paths

* Exclude eigen files from Python linter

* Ignore eigen folders for pydocstyle

* Fix C++ linting errors

* Fix docs

* Fix docs

* Exclude eigen directories from doxygen

* Update manifest to include eigen

* Update build_r to include eigen files

* Fix compiler warnings

* Store raw feature data as float

* Use float for calculating linear coefficients

* Remove eigen directory from GLOB

* Don't compile linear model code when building R package

* Fix doxygen issue

* Fix lint issue

* Fix lint issue

* Remove uneeded code

* Restore delected lines

* Restore delected lines

* Change return type of has_raw to bool

* Update docs

* Rename some variables and functions for readability

* Make tree_learner parameter const in AddScore

* Fix style issues

* Pass vectors as const reference when setting tree properties

* Make temporary storage of serial_tree_learner mutable so we can make the object's methods const

* Remove get_raw_size, use num_numeric_features instead

* Fix typo

* Make contains_nan_ and any_nan_ properties immutable again

* Remove data_has_nan_ property of tree

* Remove temporary test code

* Make linear_tree a dataset param

* Fix lint error

* Make LinearTreeLearner a separate class

* Fix lint errors

* Fix lint error

* Add linear_tree_learner.o

* Simulate omp_get_max_threads if openmp is not available

* Update PushOneData to also store raw data.

* Cast size to int

* Fix bug in ReshapeRaw

* Speed up code with multithreading

* Use OMP_NUM_THREADS

* Speed up with multithreading

* Update to use ArrayToString

* Fix tests

* Fix test

* Fix bug introduced in merge

* Minor updates

* Update docs
2020-12-24 14:01:23 +08:00