dotnet
/
spark
зеркало из https://github.com/dotnet/spark.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
372 коммитов 2 Ветки 21 Тег 56 MiB
Граф коммитов

372 Коммитов

Автор SHA1 Сообщение Дата
Terry Kim a568af5d6f
Compatibility check for Microsoft.Spark.dll in the worker. (#170) 2019-07-15 17:38:06 -07:00
Steve Suh eb9cffb8c9 Load Assembly referenced from within a lambda closure (#135) 2019-07-09 17:43:27 -07:00
Eric Erhardt 815c2a9a3e Implement Grouped Map UDFs (#143) 2019-07-09 15:47:23 -07:00
Zhiyuan Wang 73158a3757 Support Trigger in DataStreamWriter (#153) 2019-07-09 03:56:15 -07:00
Terry Kim cd91a1360e
Update developer guide to support new patch version of Spark (#109) 2019-06-29 10:34:52 +02:00
Terry Kim 574be7bdca
Add a hook to launch a debugger from the worker process. (#150) 2019-06-29 09:09:35 +02:00
Terry Kim 512589e2c5
Add NuGet badge to README.md (#152) 2019-06-29 08:16:59 +02:00
Terry Kim 27c7bb1862
[DOC] Update Databricks instruction (#146) 2019-06-14 22:26:11 -07:00
Terry Kim c14b0a10c9
Remove net461 specific code in E2E tests. (#144) 2019-06-14 19:30:10 -07:00
Steve Suh f1c5b86d84 Update the logic for Microsoft.Spark.Worker path discovery (#134) 2019-06-13 16:04:53 -07:00
Eric Erhardt c3a65a24ee Implement VectorUdf and use it in Queries 1 and 8 of TPCH benchmarks. (#127) 2019-06-06 21:08:53 -07:00
Terry Kim 68ed1dcdf2
Produce Microsoft.Spark.Worker binaries for MacOS (#133) 2019-06-06 08:23:58 -07:00
Terry Kim a0f2620880
v0.3.0 release prep (#128) 2019-06-05 07:44:14 -07:00
Terry Kim eb26baa462
Support for Spark 2.4.3. (#108) 2019-06-03 11:54:29 -07:00
Eric Erhardt aa3eb1381d
Fix VS 2017 build (#125) 
The VS 2017 F# compiler doesn't like the new AssemblyInformationAttribute added by dotnet/arcade.

Also, rename the signing leg of the build.
 2019-05-30 15:16:23 -05:00
Terry Kim a5900532cf
Fix a bug that produces an incorrect JSON for UDF return types. (#114) 2019-05-30 11:47:20 -07:00
Eric Erhardt fd5366f1b0
Use arcade's signing tool to sign all artifacts. (#124) 
* Use arcade's signing tool to sign all artifacts.

Fix #119

* Change Package Worker to be inside of MSBuild, so we can get the Version property correctly.

* Collapse packaging the worker into the same build step as signing.
 2019-05-30 11:51:05 -05:00
Eric Erhardt 0a58c1e7ed
Enable arcade support (#113) 
* Copy eng\common from Arcade repo.

* Enable Spark to build using dotnet/arcade

* Update coverlet version to work around bug in older version.

* Fix up test directory lookup for new artifacts location.

* Don't use dummy versions for local dev builds.

* Fix up official build for new output artifacts location

* Rearrange official build

- Set official build id parameter
- Build nuget package on Build leg
- Publish the worker using MSBuild instead of cmd

* Ensure only our .nupkg gets signed and copied, and the .nupkg is placed directly in the Packages folder.

Clean up official build definition.
 2019-05-24 08:59:00 -05:00
Adam Sitnik 3c986e10bd Improve Pickling performance (#111) 2019-05-21 13:02:07 -07:00
Terry Kim 20177c69cc
Create CODE_OF_CONDUCT.md. (#104) 2019-05-13 07:35:47 -07:00
Steve Suh 468bfb47a9 [Docs] Databricks App Submission (#100) 2019-05-10 09:11:26 -07:00
Terry Kim 95d86e788e
Simplify configuration service. (#92) 2019-05-07 10:20:01 -07:00
Gerhard Brueckl 02ce929cf2 Update install-worker.sh (#96) 
Examples should match the actual script/file name
 2019-05-06 18:03:06 -07:00
Steve Suh 5819d46cd7 [Doc] Separate out and link Get Started Instructions (#89) 
* Moving Getting Started instructions and fixing a few bugs with Databricks instructions
 2019-05-06 10:31:25 -07:00
Terry Kim 098b00fe0b
Fix F# project (Microsoft.Spark.FSharp.Examples) build failure on Linux (#93) 2019-05-06 10:23:46 -07:00
Rahul Potharaju 49b8f2bb1f
Databricks App Submission (#94) 
* Adding other ways of doing Databricks job submission
 2019-05-05 21:53:43 -07:00
Terry Kim ab9980cdcb
Fix the spark-submit usage in README.md. (#90) 2019-05-03 09:56:37 -07:00
Terry Kim fdb988a432 Update README.md / Release Notes for v0.2.0 (#88) 
* update readme.md

* Update release-0.2.md
 2019-05-02 02:40:09 -07:00
Luis Quintanilla 901b5aada3 Getting started ubuntu instructions (#50) 2019-05-01 23:51:56 -07:00
Terry Kim 5813c8adf2
Add author info and update repository info for Nuget package (#87) 2019-05-01 22:35:53 -07:00
Terry Kim ea7f660587
Update PackageReleaseNotes for Nuget package (#86) 2019-05-01 20:37:24 -07:00
Steve Suh efb290f358 Automate packaging release artifacts (#82) 2019-05-01 19:14:18 -07:00
Terry Kim a658ebe7f8
Revert README.md on dotnet path (#85) 2019-05-01 13:01:02 -07:00
Terry Kim 8d88b240e8
Remove formatPath() that adds ".\" or "./" to the executable. (#81) 2019-05-01 07:39:46 -07:00
Rahul Potharaju 859cd61d67
Worker installation script changes to deal with versioning (#80) 
* Changes to deal with versioning & added GitHub downloads
 2019-04-30 21:37:04 -07:00
Andrew Fogarty 93e70cbc60 Fix URL to pipeline (#79) 2019-04-30 14:26:02 -07:00
Steve Suh 330894075b Relax spark version check (#74) 2019-04-30 14:24:26 -07:00
Terry Kim c578085707
Getting ready for releasing v0.2.0 (#70) 2019-04-30 12:40:43 -07:00
Terry Kim b131c7ff12
Sign snupkg packages during build process (#76) 2019-04-30 12:39:45 -07:00
Terry Kim dae0639df4
Update build pipeline to use .NET Core 2.2.105 (#71) 2019-04-30 10:41:12 -07:00
Rahul Potharaju 8e604ea9d8
Updating Readme to reflect the use of dotnet.exe (#73) 
* Updating readme to reflect the use of dotnet.exe in spark-submit.
 2019-04-30 09:51:14 -07:00
Steve Suh db16caddbb [Doc] Update deployment instructions to use nuget and prepacked Worker. (#66) 2019-04-29 19:50:26 -07:00
Terry Kim 4ac86e243f
Add LICENSE file to Nuget package (#69) 2019-04-29 15:11:07 -07:00
laserljy 0d73b0ed88 Add instructions for generating TPCH dataset. (#52) 2019-04-29 14:53:20 -07:00
Wentao Wu 8501594a60 Create ConvertTpchCsvToParquetApp.scala (#62) 2019-04-29 13:08:27 -07:00
Terry Kim 3389e5e542
Remove version depedency from code in E2E test (#68) 2019-04-29 12:51:37 -07:00
Eric Erhardt 3776346c8b Hide the 3rd party notice in the solution explorer (#67) 
Follow up to #54
 2019-04-29 12:30:18 -07:00
Eric Erhardt 1bca687932 Remove the duplicated Pyrolite code now that the official library is strong named signed. (#65) 2019-04-29 12:29:46 -07:00
Alessio Franceschelli 649c1928cd Migrated nuspec to csproj and improved packaging (#54) 2019-04-29 11:37:08 -07:00
Ed Elliott af05ef25d4 Port of example python/sql/datasource.py into c# (#58) 2019-04-28 18:56:28 -07:00