dotnet
/
spark
зеркало из https://github.com/dotnet/spark.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
85 коммитов 2 Ветки 21 Тег 57 MiB
Граф коммитов

85 Коммитов

Автор SHA1 Сообщение Дата
Ed Elliott 4e66536107 Implement SparkSession.Range (#225) 2019-08-30 22:33:12 -07:00
Andrew Fogarty a7146cb7ce Expose DataFrameReader.Load() and DataFrameReader.Load(String path)(#228) 2019-08-30 22:14:47 -07:00
Eric Erhardt 72fdf4834d Add an example of using .NET Core 3.0 hardware intrinsics in a VectorUdf. (#211) 
When Tpch.csproj is build for netcoreapp3.0, it will use .NET Core 3.0's Avx APIs to do 4 double computations at once.
 2019-08-16 18:01:14 -07:00
Steve Suh 2deb2ee2f3 Package Worker zip fix for Mac OSX (#209) 2019-08-14 17:52:14 -07:00
Ed Elliott de1bbc02d9 Implement IsIn on Column (#171) 2019-08-01 14:55:16 -07:00
Terry Kim e2c1ae13b5
0.4.0 Release (#189) 2019-07-29 10:04:07 -07:00
Terry Kim b0d6f33599
Generate Microsoft.Spark.Worker-<version>.zip for Linux. 2019-07-26 15:53:00 -07:00
Terry Kim 139666e56f
Expose a way to override assembly search paths used by AssemblyLoader. (#187) 2019-07-26 15:49:32 -07:00
Steve Suh b043695db1 SerDe referenced assemblies (#180) 2019-07-26 09:27:32 -07:00
Terry Kim 972825c6c4
Resolve signer issue by moving DotnetRunner to different package name. (#186) 2019-07-25 12:11:59 -07:00
Andrew Fogarty b4830136d3 Clean up formatting in benchmark application (#182) 2019-07-24 16:25:38 -07:00
Steve Suh 0fcedf244e Support for SparkSession.conf (#184) 2019-07-24 16:06:14 -07:00
Eric Erhardt 36f7628fad Update Apache.Arrow reference to v0.14. (#167) 2019-07-23 10:41:03 -07:00
Terry Kim a568af5d6f
Compatibility check for Microsoft.Spark.dll in the worker. (#170) 2019-07-15 17:38:06 -07:00
Steve Suh eb9cffb8c9 Load Assembly referenced from within a lambda closure (#135) 2019-07-09 17:43:27 -07:00
Eric Erhardt 815c2a9a3e Implement Grouped Map UDFs (#143) 2019-07-09 15:47:23 -07:00
Zhiyuan Wang 73158a3757 Support Trigger in DataStreamWriter (#153) 2019-07-09 03:56:15 -07:00
Terry Kim cd91a1360e
Update developer guide to support new patch version of Spark (#109) 2019-06-29 10:34:52 +02:00
Terry Kim 574be7bdca
Add a hook to launch a debugger from the worker process. (#150) 2019-06-29 09:09:35 +02:00
Terry Kim 512589e2c5
Add NuGet badge to README.md (#152) 2019-06-29 08:16:59 +02:00
Terry Kim 27c7bb1862
[DOC] Update Databricks instruction (#146) 2019-06-14 22:26:11 -07:00
Terry Kim c14b0a10c9
Remove net461 specific code in E2E tests. (#144) 2019-06-14 19:30:10 -07:00
Steve Suh f1c5b86d84 Update the logic for Microsoft.Spark.Worker path discovery (#134) 2019-06-13 16:04:53 -07:00
Eric Erhardt c3a65a24ee Implement VectorUdf and use it in Queries 1 and 8 of TPCH benchmarks. (#127) 2019-06-06 21:08:53 -07:00
Terry Kim 68ed1dcdf2
Produce Microsoft.Spark.Worker binaries for MacOS (#133) 2019-06-06 08:23:58 -07:00
Terry Kim a0f2620880
v0.3.0 release prep (#128) 2019-06-05 07:44:14 -07:00
Terry Kim eb26baa462
Support for Spark 2.4.3. (#108) 2019-06-03 11:54:29 -07:00
Eric Erhardt aa3eb1381d
Fix VS 2017 build (#125) 
The VS 2017 F# compiler doesn't like the new AssemblyInformationAttribute added by dotnet/arcade.

Also, rename the signing leg of the build.
 2019-05-30 15:16:23 -05:00
Terry Kim a5900532cf
Fix a bug that produces an incorrect JSON for UDF return types. (#114) 2019-05-30 11:47:20 -07:00
Eric Erhardt fd5366f1b0
Use arcade's signing tool to sign all artifacts. (#124) 
* Use arcade's signing tool to sign all artifacts.

Fix #119

* Change Package Worker to be inside of MSBuild, so we can get the Version property correctly.

* Collapse packaging the worker into the same build step as signing.
 2019-05-30 11:51:05 -05:00
Eric Erhardt 0a58c1e7ed
Enable arcade support (#113) 
* Copy eng\common from Arcade repo.

* Enable Spark to build using dotnet/arcade

* Update coverlet version to work around bug in older version.

* Fix up test directory lookup for new artifacts location.

* Don't use dummy versions for local dev builds.

* Fix up official build for new output artifacts location

* Rearrange official build

- Set official build id parameter
- Build nuget package on Build leg
- Publish the worker using MSBuild instead of cmd

* Ensure only our .nupkg gets signed and copied, and the .nupkg is placed directly in the Packages folder.

Clean up official build definition.
 2019-05-24 08:59:00 -05:00
Adam Sitnik 3c986e10bd Improve Pickling performance (#111) 2019-05-21 13:02:07 -07:00
Terry Kim 20177c69cc
Create CODE_OF_CONDUCT.md. (#104) 2019-05-13 07:35:47 -07:00
Steve Suh 468bfb47a9 [Docs] Databricks App Submission (#100) 2019-05-10 09:11:26 -07:00
Terry Kim 95d86e788e
Simplify configuration service. (#92) 2019-05-07 10:20:01 -07:00
Gerhard Brueckl 02ce929cf2 Update install-worker.sh (#96) 
Examples should match the actual script/file name
 2019-05-06 18:03:06 -07:00
Steve Suh 5819d46cd7 [Doc] Separate out and link Get Started Instructions (#89) 
* Moving Getting Started instructions and fixing a few bugs with Databricks instructions
 2019-05-06 10:31:25 -07:00
Terry Kim 098b00fe0b
Fix F# project (Microsoft.Spark.FSharp.Examples) build failure on Linux (#93) 2019-05-06 10:23:46 -07:00
Rahul Potharaju 49b8f2bb1f
Databricks App Submission (#94) 
* Adding other ways of doing Databricks job submission
 2019-05-05 21:53:43 -07:00
Terry Kim ab9980cdcb
Fix the spark-submit usage in README.md. (#90) 2019-05-03 09:56:37 -07:00
Terry Kim fdb988a432 Update README.md / Release Notes for v0.2.0 (#88) 
* update readme.md

* Update release-0.2.md
 2019-05-02 02:40:09 -07:00
Luis Quintanilla 901b5aada3 Getting started ubuntu instructions (#50) 2019-05-01 23:51:56 -07:00
Terry Kim 5813c8adf2
Add author info and update repository info for Nuget package (#87) 2019-05-01 22:35:53 -07:00
Terry Kim ea7f660587
Update PackageReleaseNotes for Nuget package (#86) 2019-05-01 20:37:24 -07:00
Steve Suh efb290f358 Automate packaging release artifacts (#82) 2019-05-01 19:14:18 -07:00
Terry Kim a658ebe7f8
Revert README.md on dotnet path (#85) 2019-05-01 13:01:02 -07:00
Terry Kim 8d88b240e8
Remove formatPath() that adds ".\" or "./" to the executable. (#81) 2019-05-01 07:39:46 -07:00
Rahul Potharaju 859cd61d67
Worker installation script changes to deal with versioning (#80) 
* Changes to deal with versioning & added GitHub downloads
 2019-04-30 21:37:04 -07:00
Andrew Fogarty 93e70cbc60 Fix URL to pipeline (#79) 2019-04-30 14:26:02 -07:00
Steve Suh 330894075b Relax spark version check (#74) 2019-04-30 14:24:26 -07:00