Xiaoyun Zhang
3659a485a0
[GenAI] Introduce CausalLMPipelineChatClient for MEAI.IChatClient ( #7270 )
...
* leverage MEAI abstraction
* Update src/Microsoft.ML.GenAI.LLaMA/Llama3CausalLMChatClient.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Update src/Microsoft.ML.GenAI.LLaMA/Llama3CausalLMChatClient.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Update src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMChatClient.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* fix comments
* Update Microsoft.ML.GenAI.Core.csproj
---------
Co-authored-by: Stephen Toub <stoub@microsoft.com>
2024-11-05 09:27:05 -08:00
Michael Sharp
5b4981a134
Update To MacOS 13 ( #7285 )
...
* mac 13 testing
* build fixes
2024-11-04 17:41:25 -07:00
Tarek Mahmoud Sayed
5c503195e8
Add Timeout to Regex used in the tokenizers ( #7284 )
...
* Add Timeout to Regex used in the tokenizers
* Address the feedback
2024-11-04 08:06:54 -08:00
Tarek Mahmoud Sayed
7cce7535b7
Add the components governance file `cgmanifest.json` for tokenizer's vocab files ( #7283 )
...
* Add the governance file cgmanifest.json for tokenizer's vocab files
* Address the feedback
* apply more schema requirements on the doc
2024-11-01 15:20:59 -07:00
Tarek Mahmoud Sayed
a9b4212eb3
Address the feedback regarding Bert tokenizer ( #7280 )
...
* Address the feedback regarding Bert tokenizer
* Small fix
2024-10-26 16:07:20 -07:00
Michael Sharp
a7a6d88b05
Adds in a way to add settings for the MLContext. ( #7273 )
...
* api, no tests
* updates from pr
* fixed rebase errors and pr comments
* updates based on ONNX team
2024-10-22 16:27:04 -06:00
Michael Sharp
869dc9fcfa
fixing osx ci ( #7279 )
2024-10-22 14:46:20 -06:00
Tarek Mahmoud Sayed
81122c4c48
Introducing WordPiece and Bert tokenizers ( #7275 )
...
* Introducing WordPiece and Bert tokenizers
* Fix corner case in WordPiece
2024-10-22 12:33:58 -07:00
Michael Sharp
32bac5e395
fixing apple silicon official build ( #7278 )
2024-10-22 10:55:12 -06:00
Eui Jung
f385b06aa0
Fixes #7271 AOT for ML.Tokenizers ( #7272 )
...
* AOT for ML.Tokenizers
* Forgot to add ModelSourceGenerationContext
* Update src/Microsoft.ML.Tokenizers/Model/ModelSourceGenerationContext.cs
Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
---------
Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
2024-10-17 14:16:22 -07:00
Tarek Mahmoud Sayed
823fc175f4
Misc Changes ( #7264 )
...
* Add o1 model support
* Replace Usage of tuples with Range in EncodedToken and Remove TorchSharp Range/Index implementation
* Rename SentencePieceBpeTokenizer to allow adding more models to it in the future.
* Make Tokenizer.Decode returns non-nullable string
* Make BPE tokenizer support added tokens
* add net9 package source to the nuget.config file
* Rename TiktokenPreTokenizer to RegexPreTokenizer
2024-10-11 16:06:22 -07:00
Michael Sharp
e7943427f4
Load onnx model from Stream working ( #7254 )
2024-10-09 11:07:20 -06:00
Aleksei Smirnov
1843bcdfa6
Fix dataframe incorrectly parse CSV when renameDuplicatedColumns is true ( #7242 )
...
* Fix dataframe incorrectly parse CSV when renameDuplicatedColumns is true
* Update System.Numerics.Tensors dependency
2024-10-09 09:46:43 -06:00
Eric StJohn
9baf26b58d
Update wording ( #7253 )
2024-10-07 14:33:42 -07:00
Weihan Li
f7d4184f27
docs: update nuget package badge ( #7236 )
2024-10-07 13:41:06 -06:00
dotnet-maestro[bot]
4919b43668
[main] Update dependencies from dotnet/arcade ( #7235 )
...
* Update dependencies from https://github.com/dotnet/arcade build 20240909.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 10.0.0-beta.24430.1 -> To Version 10.0.0-beta.24459.1
* Update dependencies from https://github.com/dotnet/arcade build 20240917.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 10.0.0-beta.24459.1 -> To Version 10.0.0-beta.24467.1
* Update dependencies from https://github.com/dotnet/arcade build 20240926.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 10.0.0-beta.24467.1 -> To Version 10.0.0-beta.24476.2
* Update dependencies from https://github.com/dotnet/arcade build 20241004.4
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 10.0.0-beta.24476.2 -> To Version 10.0.0-beta.24504.4
---------
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2024-10-07 13:40:50 -06:00
Stephen Toub
3f042f632b
Update tiktoken regexes ( #7255 )
2024-10-07 13:39:15 -06:00
Xiaoyun Zhang
62c4a13f4d
[GenAI] Support Llama 3.2 1B and 3B model ( #7245 )
...
* update
* enable llama3_2
* fix tests
2024-10-06 00:16:36 -07:00
Tarek Mahmoud Sayed
1e914273d3
Move the Tokenizer's data into separate packages. ( #7248 )
...
* Move the Tokenizer's data into separate packages.
* Address the feedback
* More feedback addressing
* More feedback addressing
* Trimming/AoT support
* Make data types internal
2024-10-04 14:47:37 -07:00
Eric StJohn
189ba24641
Enable SDL tools ( #7247 )
2024-10-01 10:38:15 -07:00
Eric StJohn
eba09d977c
Add Service Tree ID for .NET Libraries ( #7252 )
2024-10-01 10:37:34 -07:00
Xiaoyun Zhang
be1e428d41
[GenAI] pack GenAI core package ( #7246 )
...
* update
* enable llama3_2
* fix tests
* pack GenAI core
2024-09-27 13:10:30 -07:00
Xiaoyun Zhang
817a77f89a
[GenAI] Add Mistral 7B Instruction V0.3 ( #7231 )
...
* add mistral and tests
* add test and sample
* add tool call support
* update autogen to v 0.1.0
* update autogen to 0.1.0
* remove tests on non-x64 machien
* add file header
* update
* update
* update ml tokenizer test version
* fix build error
* remove .receive.txt
* Update docs/samples/Microsoft.ML.GenAI.Samples/Mistral/Mistral_7B_Instruct.cs
Co-authored-by: Weihan Li <7604648+WeihanLi@users.noreply.github.com>
* update
* set t to 0
* fix test
* Update Microsoft.ML.GenAI.Mistral.csproj
---------
Co-authored-by: Weihan Li <7604648+WeihanLi@users.noreply.github.com>
2024-09-25 20:15:59 -07:00
Xiaoyun Zhang
e14f22fe0d
set <IsPackable>true</IsPackable> to GenAI.Llama and Phi ( #7237 )
2024-09-16 11:53:03 -07:00
Tarek Mahmoud Sayed
87a41fae4a
Fix decoding special tokens in SentencePiece tokenizer ( #7233 )
...
* Fix decoding special tokens in SentencePiece tokenizer
* Apply the change to the other Decode method overload
2024-09-09 15:02:38 -07:00
dotnet-maestro[bot]
4e364e4bda
[main] Update dependencies from dotnet/arcade ( #7218 )
...
* Update dependencies from https://github.com/dotnet/arcade build 20240808.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24368.9 -> To Version 9.0.0-beta.24408.2
* Update dependencies from https://github.com/dotnet/arcade build 20240816.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24408.2 -> To Version 9.0.0-beta.24416.2
* Update dependencies from https://github.com/dotnet/arcade build 20240821.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24416.2 -> To Version 9.0.0-beta.24421.2
* Update dependencies from https://github.com/dotnet/arcade build 20240823.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24421.2 -> To Version 9.0.0-beta.24423.2
* Update dependencies from https://github.com/dotnet/arcade build 20240830.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24423.2 -> To Version 10.0.0-beta.24430.1
---------
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
2024-09-02 15:03:10 -06:00
Xiaoyun Zhang
7c937bf81a
[GenAI] Add generateEmbedding API to CausalLMPipeline ( #7227 )
...
* add embedding
* add frompretrain api to phi3 model
* fix bug
* Update CausalLMPipeline.cs
2024-08-30 10:30:46 -07:00
Xiaoyun Zhang
1d1cc997d5
Directly refer sql data client 4.8.6 package in GenAI tests to fix security vulnerable package ( #7228 )
...
* update approvaltest to 7.0
* Update Versions.props
* add direct reference to sql client in GenAI test packages
2024-08-30 10:30:32 -07:00
MohammadReza SH
1612edc770
Refactor Namespace and Seald Classes in Microsoft.ML.AutoML.SourceGenerator Project ( #7223 )
...
* Make all classes in Microsoft.ML.AutoML.SourceGenerator project `sealed`
* Refactor namespace from Microsoft.ML.ModelBuilder.SweepableEstimator.CodeGenerator to Microsoft.ML.AutoML.SourceGenerator
2024-08-29 14:09:44 -07:00
Xiaoyun Zhang
70e5ab1e27
[GenAI] Add LLaMA support ( #7220 )
...
* add llama
* add test for tokenizer
* make llama 3.1 working
* update
* add shape test for 70b and 405b
* clean up
* add tests
* update
* fix error
* calculate rotary embedding in model layer
* remove rotary_emb from attention
* update feed
* update .csproj
* Update NuGet.config
* fix test
* pass device
* fix test
* update constructor
* disable 405b test
* update
* disable 70b test
* use windows only fact
* revert change
* rename test to LLaMA3_1
2024-08-28 15:35:31 -07:00
Michael Sharp
fa8c822045
Update dependency versions. ( #7216 )
...
* version updates and fixes
* fixed torch sharp test nuget
* skipping test again
* fixing typo
* consolidating extensions versioning
* fixed remote executor app
2024-08-07 18:12:44 -06:00
Xiaoyun Zhang
5f5a199a5a
[GenAI] Add readme to Microsoft.ML.GenAI.Phi ( #7206 )
...
* add readme
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
* Update src/Microsoft.ML.GenAI.Phi/README.md
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
---------
Co-authored-by: Luis Quintanilla <46974588+luisquintanilla@users.noreply.github.com>
2024-08-05 13:36:00 -07:00
Xiaoyun Zhang
9ffb3a3175
increase cancelling time ( #7207 )
2024-08-02 12:44:00 -06:00
Xiaoyun Zhang
6d1f7e2bf8
Add Microsoft.ML.GenAI.Phi, test package and sample project. ( #7184 )
...
* add genai.phi and tests
* formatter
* refactor Phi3Tokenizer
* update
* add configuration for phi-series
* add semantic kernel and autogen intergration
* update
* add Microsoft.ML.GenAI.Sample
* use tokenzier model from testTokenizer package
* use defaults
* add quantize linear
* use version string
* remove special token from CreatePhi2 API
* set up quantize sample
* initialize linear with zeros
* update sample
* add 6.0 to targetframework
* fix tests
* update
* remove Phi3Tokenizer and use LlamaTokenizer instead
* revert change in tokenizer package
* run test on x64
* fix tests
* check in approved file
* run test in net6.0
* use meta device
* copy approval tests to output folder
* set up approval test file location
* fix comment
* rename to AddGenAITextGeneration and AddGenAIChatCompletion
* Update job-template.yml
* add mit license
* add reference
* bump code coverage version
* add <PreserveCompilationContext>true</PreserveCompilationContext>
* add runtime package
* remove flag
* add flag
* fix build error
* update
* update
2024-07-30 15:47:31 -07:00
Tarek Mahmoud Sayed
f72c9d22fc
Reduce Tiktoken Creation Memory Allocation ( #7202 )
2024-07-28 15:55:03 -07:00
dotnet-maestro[bot]
34eb579d4b
[main] Update dependencies from dotnet/arcade ( #7165 )
...
* Update dependencies from https://github.com/dotnet/arcade build 20240531.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24281.1
* Update dependencies from https://github.com/dotnet/arcade build 20240606.4
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24306.4
* Update dependencies from https://github.com/dotnet/arcade build 20240614.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24314.1
* Update dependencies from https://github.com/dotnet/arcade build 20240621.4
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24321.4
* Update dependencies from https://github.com/dotnet/arcade build 20240627.1
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24327.1
* Update dependencies from https://github.com/dotnet/arcade build 20240702.2
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24352.2
* Update dependencies from https://github.com/dotnet/arcade build 20240710.4
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24360.4
* Update dependencies from https://github.com/dotnet/arcade build 20240718.9
Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
From Version 9.0.0-beta.24272.5 -> To Version 9.0.0-beta.24368.9
* fixed async void tests
* fix bpe test
* fixed auto ml test
* skipping test
---------
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
2024-07-27 18:33:59 -06:00
Erik
b4a4b10954
create unique temporary directories to prevent permission issues ( #7173 )
...
* minor fix to prevent multiple mldotnet directories
* forgot missing increment
2024-07-26 21:04:17 -06:00
Michael Sharp
ba94d7abff
Add package readmes ( #7200 )
...
* minor formatting issues, use preview 5, and package readmes
* fixing async void test
* test fixes
* added in ReadExactly extension to stream
* fixes from PR comments
* added types to fast tree
* fixed overflow
* fixed package md file
2024-07-26 21:03:35 -06:00
Joshua Zierhut
56673e57e4
Allow developers to supply their own function to infer column data types from data while loading CSVs ( #7142 )
...
* Allow developers to supply their own GuessType function
* Add a test for using a custom GuessType function.
* Fix typo in string resource identifier
* Convert 0-based line number to 1-based line number in error message.
* Add test that FormatException is thrown when one row in a data frame has less columns than the others
* Revert "Fix typo in string resource identifier"
This reverts commit 5a2ee30a63
.
# Conflicts:
# src/Microsoft.Data.Analysis/DataFrame.IO.cs
* Add a column filled with null to the test data frame to improve test coverage.
2024-07-26 17:39:20 -06:00
Jo Shields
a38b8ccfe4
Add a stub packageSourceMapping ( #7171 )
...
Without packageSourceMapping, you may receive NuGet warning NU1507. Building with WarnAsError will elevate this to a fatal error.
```
C:\d\source-indexer\bin\repo\machinelearning\test\Microsoft.ML.CpuMath.UnitTests\Microsoft.ML.CpuMath.UnitTests.csproj : error NU1507: Warning As Error: There are 9 package sources defined in your configuration. When using central package management, please map your package sources with package source mapping (https://aka.ms/nuget-package-source-mapping ) or specify a single package source. The following sources are defined: dotnet-public, dotnet-tools, dotnet-libraries, dotnet-eng, vs-buildservices, dotnet5-roslyn, mlnet-daily, mlnet-assets, dotnet8 [C:\d\source-indexer\bin\repo\machinelearning\Microsoft.ML.sln]
```
Using * as a pattern means any repo can supply any package, the pattern can be further refined later.
2024-07-26 17:28:12 -06:00
Stephen Toub
5b920f9601
Make most Tokenizer abstract methods virtual ( #7198 )
...
* Make most Tokenizer abstract methods virtual
All of the functionality in all of the methods can be implemented in terms of just a single Decode and EncodeToTokens set of methods. Only those two need to be abstract; everything else that was abstract can instead be virtual and implemented in terms of those.
* Address feedback and clean up a few things
2024-07-25 22:53:25 -04:00
Aleksei Smirnov
0c2e82e7d4
Use new System.Numerics.Tensors library for DataFrame arithmetic operations (.net8) ( #7179 )
...
* Add dependency to the latest System.Numerics.Tensors library
* Initial implementation of NumericArithmetic for net8 and TensorPrimitives
* Use CompositeArithmetic
* Refcatoring, using .net8 features to improve performance and code readability
* Remove needless changes
* Fix build
* Update System.Numerics.Tensors lib
2024-07-24 13:11:33 -06:00
Andrew Kane
f51eaa2769
Fix crash in Microsoft.ML.Recommender with validation set ( #7196 )
2024-07-24 13:01:44 -06:00
Xiaoyun Zhang
05bb0e350b
update torchsharp and helix image ( #7188 )
...
* update torchsharp and helix image
* fix build error
* Update .vsts-dotnet-ci.yml
* use centos
* Update Versions.props
2024-07-17 12:46:59 -07:00
Tarek Mahmoud Sayed
579fe03ab7
Tokenizer APIs Update ( #7190 )
...
* ToknizersAPIsUpdate
* Address the feedback
2024-07-15 07:48:58 -07:00
Jo Shields
f5abe6a086
Publish source index directly from repo ( #7189 )
...
* Use technique from aspnetcore to publish Source Index w/o jobs.yml
2024-07-10 15:00:33 -04:00
Xiaoyun Zhang
77e90243e2
Add GenAI core package ( #7177 )
...
* add GenAI.Core project
* fix format
* move causalml generate method to causalmlpipelineextension
* fix build error from api breakchange in torchsharp
* update package reference
* fix build error
* Update job-template.yml
* Update Microsoft.ML.TorchSharp.Tests.csproj
* Update Microsoft.ML.TorchSharp.Tests.csproj
* revert change for update torchsharp runtime
* use explicit torchsharp version
* fix comments
2024-06-26 09:43:31 -07:00
Aleksei Smirnov
8e3f72d023
Add targeting .Net 8.0 for DataFrame package ( #7168 )
2024-06-13 12:14:02 -07:00
Xiaoyun Zhang
83b4f180d3
add document for GenAI ( #7170 )
2024-06-13 11:35:12 -07:00
Aleksei Smirnov
0c51cc6b90
Implement DataFrameColumn Apply and DropNulls methods ( #7123 )
...
* Refactor DropNulls method
* Refactor ArrowStringDataFrameColumn Clone()
* Refactor StringDataFrameColumn Clone()
* Refactor VBufferDataFrameColumn Clone and FillNulls methods
* Add Apply method
* Add DropNulls method to columns
* Add comments
* Fix warnings in tests
* Fix url to gpt2 ext vocabulary
2024-06-11 09:10:46 -07:00