Граф коммитов

27 Коммитов

Автор SHA1 Сообщение Дата
dotnet-maestro[bot] e7db44ebcb
Bump to dotnet/installer/main@632ddca 8.0.100-preview.3.23128.1 (#7836)
Changes: e3ab0b5...632ddca
Changes: ff7c19f...2bdc3cb

Updates:

* Microsoft.Dotnet.Sdk.Internal: from 8.0.100-preview.2.23123.10 to 8.0.100-preview.3.23128.1
* Microsoft.NET.ILLink.Tasks: from 8.0.0-preview.2.23123.4 to 8.0.0-preview.2.23127.4
* Microsoft.NETCore.App.Ref: from 8.0.0-preview.2.23123.4 to 8.0.0-preview.2.23127.4

~~ Other changes ~~

* Keep `$(DotNetMonoManifestVersionBand)` on preview.2

* Keep `$(DotNetEmscriptenManifestVersionBand)` on preview.2

* Changes for the linker's new dependency: `Microsoft.DotNet.Cecil`

* Update `BuildReleaseArm64XFormsDotNet.apkdesc`

    apkdiff: File 'assemblies/System.IO.Compression.dll' has changed by -875 bytes (-5.47 %). This exceeds the threshold of 5.00 %.

* Update to MSBuild.StructuredLogger 2.1.787

    System.NotSupportedException : Unsupported log file format. Latest supported version is 15, the log file has version 16.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2023-03-03 21:36:21 -06:00
Marek Habersack b61f677dcd
[NuGet] Add back the `dotnet7` feed (#7534)
Without it, it's impossible to install MAUI:

	dotnet msbuild Xamarin.Android.sln -t:InstallMaui -p:MauiVersion=7.0.49 -p:MauiVersionBand=7.0.100 -p:Configuration=Release
	build-tools/scripts/maui.proj : error NU1101: Unable to find package Microsoft.NET.Sdk.Maui.Manifest-7.0.100.
2022-11-07 13:16:38 -05:00
Jonathan Peppers 5b7c5bffda
Bump to dotnet/installer/main@d5520378 8.0.200-alpha.1.22511.1 (#7451)
Changes: 11f6b8f712...d552037815
Changes: 5f9bfd94d9...493ce626f3
Changes: 6d10e4c8bc...b8d49801fe

Context: https://github.com/dotnet/runtime/issues/77273
Context: e46523032d
Context: https://github.com/dotnet/runtime/issues/77385

Build and run under .NET 8!

.NET 8 is used by default for the default build.

Running unit tests also requires installing .NET 6 and .NET 7 as well.

Configure dotnet6 + dotnet7 + dotnet8 NuGet feeds.

Run `darc update-dependencies --id 152596` to update the
dotnet/installer version in `eng/Version.Details.xml`.
(ID values come from [Maestro][0].)

Update `AutoImport.props` to only evaluate default android items
in .NET 8.

Update `$(DotNetTargetFrameworkVersion)` to 8.0, and
`$(DotNetStableTargetFramework)` to `net7.0`.  (Technically .NET 7
isn't stable *yet*, but it will be soon enough…)

Update `*.apkdesc` files so that unit tests pass.

Update `PackagingTest.CheckIncludedAssemblies()` test to remove
assemblies which are no longer included in .NET 8-based apps.

Update `use-dot-net.yaml` to take a new `quality` parameter, and
install .NET Core 3.1 quality=GA, not 3.1.417 specifically.
Avoids the error:

	dotnet-install: Failed to locate the latest version in the channel '3.1.417' with 'preview' quality for 'dotnet-sdk', os: 'win', architecture: 'x64'.

Update `src/Mono.Android.Export` to pass
`MethodAttributes.Static | MethodAttributes.Public` to the
`DynamicMethod` constructor, not just `MethodAttributes.Static`.
This avoids a `NotSupportedException` under .NET 8;
see also dotnet/runtime@e4652303 and dotnet/runtime#77273:

	System.NotSupportedException: Wrong MethodAttributes or CallingConventions for DynamicMethod. Only public, static, standard supported
	   at System.Reflection.Emit.DynamicMethod.CheckConsistency(MethodAttributes attributes, CallingConventions callingConvention)
	   at System.Reflection.Emit.DynamicMethod.Init(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] signature, Type owner, Module m, Boolean skipVisibility, Boolean transparentMethod)
	   at System.Reflection.Emit.DynamicMethod..ctor(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] parameterTypes, Module m, Boolean skipVisibility)
	   at Java.Interop.DynamicCallbackCodeGenerator.GenerateNativeCallbackDelegate()
	   at Java.Interop.DynamicCallbackCodeGenerator.GetCallback()
	   at Java.Interop.DynamicCallbackCodeGenerator.Create(MethodInfo method)
	   at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, Span`1 args, BindingFlags invokeAttr)

Update `XASdkTests.DotNetPublish()` for .NET 8.  With 745214deb0
we asserted that builds using the *non-* latest target framework
would use a `Mono.Android.dll` reference assembly *from the build*,
not from an existing published/NuGet-provided `Microsoft.Android.Ref`
package.  This started failing because
`$(TargetFramework)`=net7.0-android is no longer the latest framework
and thus *should* be using published packages, not in-tree bits:

	Build should be using C:\a_work\1\s\xamarin-android\bin\Release\dotnet\packs\Microsoft.Android.Ref.33\34.0.0-ci.pr.gh7451.52\ref\net7.0\Mono.Android.dll
	Expected: True
	But was:  False

Update the test to instead make this assertion only for the latest
target framework (`net8.0-android`), skipping .NET 7.

Remove an assertion of no build warnings under `net7.0`; this isn't
currently possible, due to dotnet/runtime#77385.

Update the `Microsoft.NETCore.App.Runtime.AOT.Cross` SDK package names
when `$(AotAssemblies)`=True; the names do not (yet?) include the
.NET version in the package names.

When installing multiple .NET SDKs into a single folder with the
`dotnet-install.{.sh,ps1}` script -- needed to run the unit tests --
you can hit an issue where `dotnet` no longer runs:

	dotnet --info
	Exited with code: 137

The problem is that the `dotnet` binary can get overwritten by an
older .NET, and be completely broken.

The solution is:

 1. Install the newest .NET first, followed by any older versions

 2. Use `dotnet-install.ps1 -SkipNonVersionedFiles` or
    `dotnet-install.sh --skip-non-versioned-files` so the `dotnet`
    binary isn't overwritten

This results in the newest .NET SDK, with side-by-side older .NET SDKs
installed as well.

Rename `tests/api-compatibility/acceptable-breakages-vReference-net7.0.txt`
to use `-net8.0`, as the default `$(TargetFramework)` for
`Mono.Android.dll` changed.

[0]: https://maestro-prod.westus2.cloudapp.azure.com/3074/https:%2F%2Fgithub.com%2Fdotnet%2Finstaller/latest/graph
2022-10-26 15:17:44 -04:00
dotnet-maestro[bot] 2320e20406
Bump to dotnet/installer/release/7.0.1xx@11f6b8f 7.0.100-rc.2.22459.2 (#7367)
Changes: dd355bb...11f6b8f

Updates:

* Microsoft.Dotnet.Sdk.Internal: from 7.0.100-rc.2.22457.6 to 7.0.100-rc.2.22459.2

~~ Other Changes ~~

* Enable `$(DisableTransitiveFrameworkReferenceDownloads)` in `Directory.Build.targets`
  This transitive package restores for packs like `Microsoft.AspNetCore.App.Ref`.
  Otherwise we would need the 6.0.10 feed for ASP.NET Core packages.

* Update the dotnet/runtime feed to `darc-pub-dotnet-runtime-8c6bcad`.
  This contains 6.0.10 runtime packages

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2022-09-16 08:28:00 -05:00
Jonathan Peppers 5e0b3f48cc
Bump to dotnet/installer/main@dd355bb 7.0.100-rc.2.22457.6 (#7360)
Changes: 330dee39e5...dd355bb777
Changes: 313671b195...5f9bfd94d9
Changes: ef077d0b58...6d10e4c8bc
Changes: 216093204c...8d8d2122b1

Updates:

* Microsoft.Dotnet.Sdk.Internal: from 7.0.100-rc.2.22454.1 to 7.0.100-rc.2.22457.6
* Microsoft.NET.ILLink.Tasks: from 7.0.100-1.22423.4 to 7.0.100-1.22452.1
* Microsoft.NETCore.App.Ref: from 7.0.0-rc.1.22422.12 to 7.0.0-rc.2.22451.11
* Microsoft.NET.Workload.Emscripten.Manifest-7.0.100: from 7.0.0-rc.1.22411.1 to 7.0.0-rc.2.22430.5

This is bringing required changes from 2bf5153 to `main`.

~~ Other Changes ~~

* `darc update-dependencies --id 148018` to manually update the
  dotnet/installer version
* Add feeds in `NuGet.config` from our .NET 6 release branch
* Install both net6 & net7 workload manifests for Mono and Emscripten
* Extend both net6 & net7 Mono workloads from the `android` workload
* Support for `$(AotAssemblies)` property (directly uses Mono workload
  pack names)
* Update `.apkdesc` files
2022-09-08 16:15:08 -05:00
dotnet-maestro[bot] 745214deb0
Bump to dotnet/installer/main@c7afae69 7.0.100-preview.4.22174.1 (#6598)
Context: https://github.com/dotnet/runtime/issues/56989
Context: https://github.com/dotnet/runtime/issues/68734
Context: https://github.com/dotnet/runtime/issues/68914
Context: https://github.com/dotnet/runtime/pull/68701

Changes: 04e40fa9...c7afae69

	% git diff --shortstat 04e40fa9...c7afae69
	 98 files changed, 1788 insertions(+), 1191 deletions(-)

Changes: a21b9a2d...c5d40c9e

	% git diff --shortstat a21b9a2d...c5d40c9e
	 28347 files changed, 1609359 insertions(+), 1066473 deletions(-)

Changes: 01c4f590...04c49c9d

	% git diff --shortstat 01c4f590...04c49c9d
	 577 files changed, 28039 insertions(+), 10835 deletions(-)

Updates to build with the .NET 7 SDK and use the runtime specified by
the SDK.  We no longer use different SDK & runtime versions.

This produces a 7.0.100 Android workload.

After this is merged we should be able to enable Maestro to consume
future .NET 7 builds from dotnet/installer/main.

~~ Known Issues ~~

AOT+LLVM crashes on startup:

  * https://github.com/dotnet/runtime/issues/68914

Xamarin.Build.Download hits a breaking change with `ZipEntry`:

  * https://github.com/dotnet/runtime/issues/68734
  * https://github.com/xamarin/XamarinComponents/pull/1368

illink outputs different MVIDs per architecture:

  * https://github.com/dotnet/linker/issues/2203
  * https://github.com/dotnet/runtime/issues/67660

Size of `libmonosgen-2.0.so` regressed:

  * https://github.com/dotnet/runtime/issues/68330
  * https://github.com/dotnet/runtime/pull/68354

Newer .NET 7 builds crash on startup:

  * https://github.com/dotnet/runtime/pull/68701
  * This is worked around by *disabling* lazy loading of AOT'd
    assemblies 6dc426f1.
  * TODO: re-enable once we get a fixed .NET 7 runtime.

TODO: We can't yet push to the `dotnet7` feed. Working on this.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Co-authored-by: Peter Collins <pecolli@microsoft.com>
2022-05-05 17:04:44 -04:00
Jonathan Peppers 5f0a7da3d2
Bump to dotnet/installer/release/6.0.3xx@04e40fa 6.0.300-rtm.22214.7 (#6934)
Porting to `main` from the `release/6.0.3xx` branch.

* Bump to dotnet/installer/release/6.0.3xx@04e40fa 6.0.300-rtm.22214.7 (#6930)

Changes: 89d0c23...04e40fa

Updates: Microsoft.Dotnet.Sdk.Internal: from 6.0.300-preview.22173.2 to 6.0.300-rtm.22214.7

* Bump to dotnet/runtime/release/6.0@a21b9a2 6.0.5 (#6927)

Changes: 4050c12...a21b9a2

Updates: Microsoft.NETCore.App.Ref: from 6.0.4 to 6.0.5

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2022-04-18 08:39:59 -05:00
dotnet-maestro[bot] ceb207096f
Bump to dotnet/runtime/release/6.0@4050c12 6.0.4 (#6824)
Changes: 10b1751...4050c12

Updates:
* Microsoft.NETCore.App.Ref: from 6.0.4 to 6.0.4
* Microsoft.NET.Workload.Emscripten.Manifest-6.0.300: from 6.0.2 to 6.0.4

All dependencies should now be on the 6.0.300 version band. This unblocks
dotnet/maui so all packs can have matching version bands.

Start tracking Microsoft.NET.Workload.Emscripten.Manifest-6.0.300 instead of 6.0.100.

Create a `$(MicrosoftNETWorkloadEmscriptenPackageVersion)` property so less code
changes are required as version bands change.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2022-03-11 14:33:02 -06:00
dotnet-maestro[bot] 9f54e8aef3
Bump to dotnet/runtime/release/6.0@10b1751 6.0.4 (#6811)
Changes: 3ba4b3f...10b1751

Updates: Microsoft.NETCore.App.Ref: from 6.0.3 to 6.0.4
2022-03-10 11:42:35 -06:00
dotnet-maestro[bot] 45b04da98a
Bump to dotnet/installer/release/6.0.3xx@c5b0a45 6.0.300-preview.22151.9 (#6804)
Changes: 2ea9c6a...c5b0a45

Updates: Microsoft.Dotnet.Sdk.Internal: from 6.0.300-preview.22128.2 to 6.0.300-preview.22151.9
2022-03-02 21:34:47 -06:00
dotnet-maestro[bot] 00bb2a1c10
Bump to dotnet/runtime/release/6.0@3ba4b3f 6.0.3 (#6788)
Changes: 6dd808f...3ba4b3f

Updates: Microsoft.NETCore.App.Ref: from 6.0.3 to 6.0.3
2022-02-24 11:49:10 -06:00
dotnet-maestro[bot] 1d3d1b7fdf
Bump to dotnet/runtime/release/6.0@6dd808f 6.0.3 (#6749)
Changes: 3da69c2...6dd808f

Updates: Microsoft.NETCore.App.Ref: from 6.0.2-mauipre.1.22102.15 to 6.0.3

* Linker changes reduced app size of a Xamarin.Forms test app:

    --"PackageSize": 8582910
    ++"PackageSize": 7938591

* Start tracking dotnet/emsdk as a dependency of dotnet/runtime.

Otherwise we get errors such as:

    error MSB4242: SDK Resolver Failure: "The SDK resolver "Microsoft.DotNet.MSBuildWorkloadSdkResolver" failed while attempting to resolve the SDK "Microsoft.NET.Sdk". Exception: "Microsoft.NET.Sdk.WorkloadManifestReader.WorkloadManifestCompositionException: Workload manifest dependency 'Microsoft.NET.Workload.Emscripten' version '6.0.1' is lower than version '6.0.2' required by manifest 'microsoft.net.workload.mono.toolchain'

If we install the emscripten manifest that dotnet/runtime expects, this problem goes away.

A similar change will likely be needed upstream in dotnet/maui.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2022-02-17 10:28:24 -06:00
dotnet-maestro[bot] 63a9aca486
Bump to dotnet/installer/release/6.0.1xx@457210a 6.0.101-servicing.21562.12 (#6484)
Changes: 9a65036...457210a

* Updates: Microsoft.Dotnet.Sdk.Internal: from 6.0.100-rtm.21552.8 to 6.0.101-servicing.21562.12
* Hardcode $(DotNetPreviewVersionBand) to 6.0.100 

The various workload packs from dotnet/runtime don't have new branding yet, so we need to stick to 6.0.100 for now.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2021-11-15 09:43:22 -06:00
dotnet-maestro[bot] 8c300edfdf
Bump to dotnet/installer/release/6.0.1xx@9e8b04b 6.0.100-rtm.21527.11 (#6441)
Changes: fd022a3...9e8b04b

Updates: Microsoft.Dotnet.Sdk.Internal: from 6.0.100-rtm.21526.10 to 6.0.100-rtm.21527.11
2021-10-28 13:13:19 -05:00
dotnet-maestro[bot] 5c060143fb
Bump to dotnet/installer/release/6.0.1xx@fd022a3 6.0.100-rtm.21526.10 (#6439)
Changes: c020bef...fd022a3

Updates: Microsoft.Dotnet.Sdk.Internal: from 6.0.100-rtm.21525.3 to 6.0.100-rtm.21526.10
2021-10-27 17:50:15 -05:00
dotnet-maestro[bot] b1180c8edb
Bump to dotnet/installer/release/6.0.1xx@626c388 6.0.100-rtm.21524.1 (#6415)
Changes: 1d2eee3...626c388
Changes: 6f41165...4822e3c

* Updates Microsoft.Dotnet.Sdk.Internal: from 6.0.100-rtm.21521.5 to 6.0.100-rtm.21524.1

* [tests] set $(DisableImplicitFSharpCoreReference)

Our F# tests currently fail with:

    UnnamedProject.fsproj : error NU1102: Unable to find package FSharp.Core with version (>= 6.0.1)
    UnnamedProject.fsproj : error NU1102:   - Found 46 version(s) in dotnet-public [ Nearest version: 6.0.0 ]

`FSharp.Core` 6.0.1 isn't released yet, but this test is actually
trying to use:

    public static Package FSharp_Core_Latest = new Package {
        Id = "FSharp.Core",
        Version = "4.7.1",
        TargetFramework = "netstandard2.0",
        References = {
            new BuildItem.Reference ("mscorlib"),
            new BuildItem.Reference ("FSharp.Core") {
                MetadataValues = "HintPath=..\\packages\\FSharp.Core.4.7.1\\lib\\netstandard2.0\\FSharp.Core.dll"
            },
        }
    };

We should just set `$(DisableImplicitFSharpCoreReference)` for .NET 6,
and use `FSharp.Core` 4.7.1 as the tests setup to do.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
2021-10-25 16:20:15 -05:00
dotnet-maestro[bot] 82fed82977
Bump to dotnet/installer/release/6.0.1xx@1d2eee3 6.0.100-rtm.21521.5 (#6410)
Changes: 4cba599...1d2eee3
Changes: d51cc20...d0662ed
Changes: 92ff024...6f41165

Updates:

* Microsoft.Dotnet.Sdk.Internal: from 6.0.100-rtm.21518.6 to 6.0.100-rtm.21521.5
* Microsoft.NET.ILLink.Tasks: from 6.0.100-1.21473.1 to 6.0.100-1.21519.4
* Microsoft.NETCore.App.Ref: from 6.0.0-rtm.21517.2 to 6.0.0

Now that dotnet/runtime is "locking down" for release, the runtime
version is 6.0.0 (and not preview):

    <MicrosoftNETCoreAppRefPackageVersion>6.0.0</MicrosoftNETCoreAppRefPackageVersion>

Maestro will now update us with a "new feed" each time a new build is available:

    <!--Begin: Package sources managed by Dependency Flow automation. Do not edit the sources below.-->
    <!--  Begin: Package sources from dotnet-runtime -->
    <add key="darc-pub-dotnet-runtime-6f41165" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-6f411658/nuget/v3/index.json" />
    <!--  End: Package sources from dotnet-runtime -->
    <!--End: Package sources managed by Dependency Flow automation. Do not edit the sources above.-->

So in order for this to work, anywhere we have hardcoded the `dotnet6`
feed, I reworked this to instead use our `NuGet.config`. This enables
additional feeds from Maestro to be picked up automatically.

Since the runtime is always 6.0.0, we have to be careful and make sure
that any NuGet caches are cleared between CI runs. `xaprepare` now
deletes any cached `microsoft.netcore.app.runtime.mono.android*`
packages.

Lastly, we found a `GenerateNuGetConfig` target with a hardcoded
`dotnet6` feed that is no longer needed since runtime packs resolve
properly from workloads now. We probably could have removed this in
1bd24acc.

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Co-authored-by: Peter Collins <pecolli@microsoft.com>
2021-10-21 21:21:27 -05:00
Peter Collins 952e67be87
[build] Use dotnet/arcade dependency management tooling (#5731)
Context: ea609b8e03/Documentation/Darc.md

The dotnet core engineering group has some dependency management
tooling known as `darc` that we'd like to adopt.  Integrating this
tooling into the build system will make it easier to stay up to date
with the latest .NET 6 SDK changes.

Many dotnet repos use a [publishing workflow][0] that will push build
artifact data to a central location known as the
"Build Asset Registry".  This data includes a "[Channel][1]"
association, which is the key to dependency updating.  Local updates
and automatic update "Subscriptions" compare the version files in a
given repo against the product versions available in the channel that
you are interested in.

We hope to be able to publish Xamarin SDK build information to this
central registry in the near future, so that other products can take
a dependency on us as needed (dotnet/maui for instance).

The `darc` tool looks for four different files in a repo when adding
a dependency or when checking for an update:

  * `eng/Version.Details.xml`
  * `eng/Versions.props`
  * `global.json`
  * `nuget.config`

Both of the `Version*` files present in the `eng` folder are updated
when a new dependency is available.

To work with `darc` locally you will need to [install][2] the `darc`
global tool, join the `arcade-contrib` GitHub team, and configure your
auth settings.

To add a new dependency, use the [`darc add-dependency`][3] command:

	darc add-dependency -n Microsoft.NetCore.App.Runtime.android-arm64 -t product -v 6.0.0-preview.2.21154.6 -r https://github.com/dotnet/runtime

To update all dependencies, use the [`darc update-dependencies`][4]
command:

	darc update-dependencies --channel ".NET 6"

To configure automatic updates, use the [`darc add-subscription`][5]
command to enroll a target repo/branch into updates from a particular
channel:

	darc add-subscription --channel ".NET 6" --source-repo https://github.com/dotnet/installer --target-repo https://github.com/xamarin/xamarin-android --target-branch main --update-frequency everyWeek --standard-automerge

Once a subscription is configured, [a pull request][6] will be
created automatically by the dotnet Maestro bot when dependency
updates are available.

~~~

This PR also contains a bump to .NET 6.0.100-preview.3.21168.18.

Changes: 19e22a76...823c1dfce

The .NET 6 `.apkdesc` files have been updated accordingly.  It seems
that `System.Private.CoreLib.dll` grew in size, however reductions in
native libraries/etc. results in an overall smaller package sizes.

Simple XA:

	-"PackageSize": 2889606
	+"PackageSize": 2877318

XF/XA:

	-"PackageSize": 8746124
	+"PackageSize": 8733836

[0]: 681511f2f6/Documentation/CorePackages/Publishing.md (what-is-v3-publishing-how-is-it-different-from-v2)
[1]: https://github.com/dotnet/arcade/blob/main/Documentation/BranchesChannelsAndSubscriptions.md#branches-channels-and-subscriptions-explained
[2]: ea609b8e03/Documentation/Darc.md (setting-up-your-darc-client)
[3]: ea609b8e03/Documentation/Darc.md (add-dependency)
[4]: ea609b8e03/Documentation/Darc.md (update-dependencies)
[5]: ea609b8e03/Documentation/Darc.md (add-subscription)
[6]: https://github.com/xamarin/xamarin-android/pull/5744
2021-03-22 15:57:32 -04:00
Jonathan Pryor 95a3a2b9e0
Bump to xamarin/Java.Interop/main@b46598a2; packageSources (#5608)
Context: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
Context: https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/
Context: https://devdiv.visualstudio.com/DevDiv/_wiki/wikis/DevDiv.wiki/12676/ncident-help-for-Substitution-attack-risk-from-multiple-package-feeds

Changes: ee7b6bbe38...b46598a254

  * xamarin/Java.Interop@b46598a2: Bump to xamarin/xamarin-android-tools/main@479931ce; packageSources (#796)

There is a Package Substitution Attack inherent in NuGet, whereby
if multiple package sources provide packages with the same name,
it is *indeterminate* which package source will provide the package.

For example, consider the [`XliffTasks` package][0], currently
provided from the [`dotnet-eng`][1] feed, and *not* present in the
NuGet.org feed.  If a "hostile attacker" submits an `XliffTasks`
package to NuGet.org, then we don't know, and cannot control, whether
the build will use the "hostile" `XliffTasks` package from NuGet.org
or the "desired" package from `dotnet-eng`.

There are two ways to prevent this attack:

 1. Use `//packageSources/clear` and have *only one*
    `//packageSources/add` entry in `NuGet.config`

 2. Use `//packageSources/clear` and *fully trust* every
    `//packageSources/add` entry in `NuGet.config`.
    `NuGet.org` *cannot* be a trusted source, nor can any feed
    location which allows "anyone" to add new packages, nor can
    a feed which itself contains [upstream sources][2].

As the `XliffTasks` package is *not* in `NuGet.org`, option (1)
isn't an option.  Go with option (2), using the existing
`dotnet-eng` source and the new *trusted* [`dotnet-public`][3]
package source.

Update `azure-pipelines.yaml` to call [`NuGetAuthenticate@0`][4]
with `forceReinstallCredentialProvider: true`.  This is needed so
that our commercial builds are able to authenticate with VSTS to
access internal package feeds.

[0]: https://github.com/dotnet/xliff-tasks
[1]: https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=dotnet-eng
[2]: https://docs.microsoft.com/en-us/azure/devops/artifacts/concepts/upstream-sources?view=azure-devops
[3]: https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=dotnet-public
[4]: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/package/nuget-authenticate?view=azure-devops
2021-02-10 16:40:02 -05:00
Marek Habersack 493a2048f7
[xaprepare] Use global NuGet package cache instead of in-tree one (#5491)
Our `NuGet.config` specified a local path to store NuGet packages
(commit 3a4acf5dcb) because some of our projects need to access
various files from the packages, and they need to know where to find
the unpacked content.

However, having the `packages/` directory be a subdirectory within the
`xamarin-android` checkout means that every time the repo is cleaned
e.g. with `git clean -xdf`, the `packages/` directory is removed,
necessitating that the NuGet packages be re-downloaded on the next
`make prepare` invocation.

Fix this conundrum by partially reverting 3a4acf5dcb, removing the
`globalPackagesFolder` setting within `NuGet.config`.

`make prepare`/`xaprepare` will create the file
`bin/Build$(Configuration)/Configuration.Generated.props` and add an
`$(XAPackagesDir)` MSBuild property which contains the detected NuGet
global packages folder location.  `Configuration.props` imports the
new `Configuration.Generated.props` file, making `$(XAPackagesDir)`
available to other `.targets` files should they need to access files
located within NuGet packages.

The `$(XAPackagesDir)` variable will contain the first valid path from:

  * [The `NUGET_PACKAGES` environment variable][0]
  * The parent directory of the parent directory of the
    `$(PkgXamarin_LibZipSharp)` MSBuild property, as set by the
    [`@(PackageReference.GeneratePathProperty)` metadata][1].

The local NuGet global-packages folder was also needed by some YAML
scripts and `.vscode` files to launch `nunit3-console.exe`.
Replace these `nunit3-console.exe` invocations with calls to the new
`build-tools/scripts/nunit3-console` and
`build-tools/scripts/nunit3-console.cmd` scripts, which will correctly
determine the NuGet packages folder and execute
`nunit3-console.exe` from the `nunit.consolerunner` package.

Both scripts determine the NuGet global-packages folder by using the
first valid path from:

  * The `NUGET_PACKAGES` environment variable.

  * If `Nuget.exe` was downloaded by a previous `make prepare`
    invocation, then by running:

        mono .nuget/NuGet.exe locals -list global-packages

  * The `$HOME/.nuget/packages` directory (Unix) or
    the `%userprofile%\.nuget\packages` directory (Windows).

This removes the need to hardcode package version into our azure
pipeline as well as VSCode configuration.  However, both scripts still
have the `nunit.consolerunner` package version embedded in them.
This is done so that we don't need to generate them in scenarios where
`xaprepare` is undesirable to run.

[0]: https://docs.microsoft.com/en-us/nuget/consume-packages/managing-the-global-packages-and-cache-folders
[1]: https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#generatepathproperty
2021-01-26 14:49:01 -05:00
Jonathan Peppers 6f761ea854
Bump to .NET 6.0.100-alpha.1.20562.2 (#5269)
Changes: 53e0c8c7...efac3f25

.NET 6.0.100-alpha.1.20562.2 includes support for
`$(AfterMicrosoftNETSdkTargets)`, and so we now have a slightly
different import ordering:

 1. `AutoImport.props`
 2. The user's `.csproj` file
 3. Some early parts of `Microsoft.NET.Sdk` / `Sdk.targets`
 4. `Microsoft.Android.Sdk.targets`
 5. The rest of `Microsoft.NET.Sdk` / `Sdk.targets`
 6. `Microsoft.Android.Sdk.After.targets`

Anything that needs to happen *after* the `Microsoft.NET.Sdk` targets
can go in `After.targets`.

This allows us to set up some defaults that we lost in 91498d55 when
implementing Android as a .NET "workload":

  * `$(GenerateDependencyFile)` defaults to `false`
  * `$(ProduceReferenceAssembly)` defaults to `false` for application
    projects.

Using `$(SelfContained)` also triggers new behavior in .NET 6 that we
need to turn off:

  * `$(GenerateRuntimeConfigurationFiles)` to `false` or we get a
    `*.runtimeconfig.dev.json` file generated for Android applications.
  * `$(CopyLocalLockFileAssemblies)` to `false` - otherwise every
    `.dll` file is copied to `$(OutputPath)`, and we don't need this
    behavior.  This appears to be new behavior in .NET 6 when
    `$(SelfContained)` is `true`.
  * `$(ComputeNETCoreBuildOutputFiles)` to `false` or we get a
    `*.exe` .NET app host in the output directory.

I also adjusted the naming for `windows-x64` to `win-x64`, as
described in 011a7713.
2020-11-13 10:10:08 -05:00
Radek Doulik f68f32a4d3
Initial NET5/ILLink support (#4669)
Implements https://github.com/xamarin/xamarin-android/issues/4707

With .NET5 we are switching from XA linker implemented as msbuild task to ILLink tool with [custom steps](https://github.com/mono/linker/blob/master/docs/custom-steps.md) provided in an assembly.

The new assembly with our custom steps is named `Microsoft.Android.Sdk.ILLink` and the source code is located in `src/Microsoft.Android.Sdk.ILLink`. It is used during build by `ILLink` tool.

The initial support is already able to link and run simple XA and XA/XF samples.

Future work
* replace reflection usage in SetupStep https://github.com/xamarin/xamarin-android/issues/4709
* audit reflection usage in our code https://github.com/xamarin/xamarin-android/issues/4708
* add more customs steps - we are still missing MonoDroidMarkStep

Notes
* the Profile API is not available in the linker [public API](https://github.com/mono/linker/tree/master/src/linker/ref). we work around it by adding `src/Microsoft.Android.Sdk.ILLink/Profile.cs` with missing pieces.
* `TypeDefinition.GetMethods` extension method from `Mono.Tuner.MethodBodyRocks` is not public. It is very simple so we are inlining it.
* the ILLink is now enabled in `Release` configuration defaults.
* tests were updated to reference more packages, to not miss second level dependencies.

Initial results:

apkdiff output summary for HelloAndroid sample, comparing `Debug` and `Release` apk's
```
  -       7,300 Davik executables -2.34% (of 311,436)
  -  14,733,944 Assemblies -31.85% (of 46,265,064)
  -   2,492,452 Shared libraries -11.43% (of 21,803,164)
  -  15,528,456 Package size difference -28.70% (of 54,114,462)
```

Context:
* `_RunILLink` target https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.ILLink.targets
* linker documentation https://github.com/mono/linker/tree/master/docs

Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Co-authored-by: Jonathan Pryor <jonpryor@vt.edu>
2020-06-04 14:03:46 +02:00
Jonathan Peppers 2f677efa0e
[Xamarin.Android.Build.Tasks] add $(_AndroidAllowDeltaInstall) support (#4643)
Changes: 5849ea1...e795bce

  * xamarin/monodroid@e795bcef3: Bump to xamarin/androidtools/master@739afe5 (#1095)
  * xamarin/monodroid@97451721c: [tools/msbuild] add $(_AndroidAllowDeltaInstall) (#1089)

Add ***Experimental*** "`.apk` delta installation" support.

*What do we want?!*  Faster `.apk` installs!  (See also PR #4690.)

The question, as ever, is *how* to reduce the time of `.apk` installs,
and what the tradeoffs are.  Fast dev, which removes assemblies from
the `.apk`, is one way to do that; PR #4690 is "fastdev v2", which does
more there.

But for everything else in the `.apk` -- Android Assets, Resources, and
more -- how do we make deploying it faster?

Here is one way.

Android Studio contains an [`installer`][0] utility, built for each
supported Android target platform, which is installed onto the target
device as `/data/local/tmp/.studio/bin/installer`.  `installer` works
in terms of "protocol buffers", and can *in place update* an `.apk`
file installed on the Android target via a "delta" between the
currently installed `.apk` and the "to be installed" `.apk`.
`installer` deals with *just* `.apk` contents, and thus doesn't
special-case Android Assets, Resources, or IL assemblies.

As the `installer` utility is Apache 2 licensed, Xamarin.Android can
redistribute and use the `installer` utility as well.

To use `installer` to deploy `.apk` updates:

 1. Use a Xamarin.Android install which contains `installer`
    integration, e.g. via `make make prepare-external-git-dependencies`
    or via a master AzDO-hosted build.

 2. Install an app:

        msbuild /t:Install samples/HelloWorld/HelloWorld.csproj

 3. Make a change, e.g. by editing
    `samples/HelloWorld/Resources/layout/Main.axml`.

 4. Install the app, enabling use of this new feature:

         msbuild /t:Install samples/HelloWorld/HelloWorld.csproj /p:_AndroidAllowDeltaInstall=True 

     This triggers the following in the code:

     a. Create the `.apk` to deploy, "as normal".

     b. Check to see if `installer` is present on the Android target.
        If not, deploy to `/data/local/tmp/.studio/bin/installer`.         
        This happens optimistically, by trying to start up
	`installer`, looking for certain error codes that indicate the
	installer isn't present, then installing `installer` if needed.

     c. Invoke the target's `installer` to obtain the contents of the
        *currently installed* `.apk` file.

     d. Compute a diff between (4.c) and the `.apk` in (4.a).

     e. Deploy the diff to the Android target.

     f. Invoke `installer` to apply the diff (4.e) to the installed
        `.apk`, updating its contents to that of (4.a).

     g. If this process fails for any reason, a "normal" `adb install`
        is performed.

This new workflow is only performed when:

  * The `$(_AndroidAllowDeltaInstall)` MSBuild Property is True
  * The App is *already deployed* to the Android target.
  * The Android target is running API-24 (Android v7.0) or later.

Some initial Install timing results for step (4), against a Pixel 3a.
Timings are total times, including the build, of which the deploy part
takes on the order of 1.5 - 2.5 sec or so:

| Test                                          |    Normal (s) |     Delta (s) |
|:----------------------------------------------|--------------:|--------------:|
| 18MB APK, fast deploy off, updated 1 assembly |           6.3 |           5.6 |
| 9MB APK, fast deploy on, updated 1 resource   |           5.6 |           5.3 |

[0]: https://android.googlesource.com/platform/tools/base/+/studio-master-dev/deploy/installer
2020-05-21 19:06:50 -04:00
Peter Collins 3a4acf5dcb [build] Use an in tree `globalPackagesFolder` (#4105)
`@(PackageReference)` packages install/restore into a default global
folder of `%userprofile%\.nuget\packages` (Windows) or
`$HOME/.nuget/packages` (Linux, macOS).

In order to avoid bloating the home directories on our machines used
locally and in CI, we'll reconfigure the `globalPackagesFolder` path
to point to a `packages` folder nested under `xamarin-android`.

This also allows us to continue to use "known" paths to certain tools
that are used by our tests, such as `nunit3-console.exe`.

Each test in `Xamarin.Android.Build.Tests` copies and uses the
`NuGet.config` file from the xamarin-android repo root.  Now that a
`globalPackagesFolder` value is specified there, NuGet restore
behavior for all tests has changed.

A new `globalPackagesFolder` default has now been set for all tests
so that a common folder can be used rather than a `packages` folder
nested under each test directory.  Additionally, some tests have been
updated to no longer run in parallel due to restore issues.

Co-Authored-By: Jonathan Peppers <jonathan.peppers@microsoft.com>
2020-01-16 16:25:54 -05:00
Gustavo Varo 07e7477126 Adding new api break workflow that uses Microsoft.DotNet.ApiCompat.dll (#3884)
Fixes: https://github.com/xamarin/xamarin-android/issues/1118

Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60069
Context: https://github.com/xamarin/xamarin-android/pull/930

Our current API check workflow is not able to detect some API breakage
or is flagging non-breaking API changes as breaking.

Examples of breakages not previously reported:

  * Changing the values of `[Register]` attributes may be a breaking
    change but is not flagged by `mono-html-html`.

  * Adding `abstract` methods to an `abstract` class

Examples of acceptable changes which elicit warnings:

  * Changing a base class in a compatible manner.

Instead of using `mono-api-info` + `mono-api-html`, use
[Microsoft.DotNet.ApiCompat][0] to perform API compatibility
validation, which is what the .NET and Azure teams use.

Microsoft.DotNet.ApiCompat reports `[Register]` changes:

	CannotChangeAttribute : Attribute 'Android.Runtime.RegisterAttribute' on 'Android.Database.Sqlite.SQLiteDatabase.Path.get()' changed from '[RegisterAttribute("getPath", "()Ljava/lang/String;", "")]' in the contract to '[RegisterAttribute("getPath", "()Ljava/lang/String;", "GetGetPathHandler")]' in the implementation.

and thus would have caught the `Build.SERIAL` breakage in #930.

To override the warnings produced by Microsoft.DotNet.ApiCompat,
the message can be copied into an appropriate
`tests/api-compatibility/acceptable-breakages-*.txt` file.

See `tests/api-compatibility/README.md` for details.

[0]: https://github.com/dotnet/arcade/tree/master/src/Microsoft.DotNet.ApiCompat
2019-12-07 23:45:13 -05:00
Brendan Zagaeski 0342fe5698 [Xamarin.Android.Build.Tasks] Move XA4214 warning text into .resx file (#3900)
Context: https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1009374/

This is a first step toward localizing the MSBuild error and warning
messages produced by `Xamarin.Android.Build.Tasks.dll`.

We will be following the [.NET Resource Localization pattern][0] and
generating satellite assemblies using [`.resx` files][1], in particular
`src/Xamarin.Android.Build.Tasks/Properties/Resources.resx`.

`Resources.resx` is an XML file, and will contain `/root/data`
elements in which `//data/@name` will start with the Xamarin.Android
error or warning code, and `//data/value` will be the error or
warning message:

        <root>
          <data name="XA4214" xml:space="preserve">
            <value>The managed type `{0}` exists in multiple assemblies: {1}. Please refactor the managed type names in these assemblies so that they are not identical.</value>
          </data>
        </root>

An optional `//data/comment` element may be provided to describe the
meaning within the `//data/value` element to translators:

        <data name="XA4214" xml:space="preserve">
          <value>The managed type `{0}` exists in multiple assemblies: {1}. Please refactor the managed type names in these assemblies so that they are not identical.</value>
          <comment>
            {0} - The managed type name
            {1} - Comma-separated list of all the assemblies where the managed type exists
          </comment>
        </data>

During the build, `Resources.resx` will be translated into a
`Resources.Designer.cs` file:

        namespace Xamarin.Android.Tasks.Properties {
          internal partial class Resources {
            internal static string XA4214 {
              get => ...
            }
          }
        }

The `Resources` members should be used to obtain all strings for use
in `LogCodedError()` and `LogCodedWarning()` calls:

        Log.LogCodedWarning ("XA4214", Properties.Resources.XA4214, kvp.Key, string.Join (", ", kvp.Value));

When an MSBuild error or warning code is used with more than one
output string, then a semantically meaningful suffix should be used
to distinguish between the two:

        <data name="XA4214_Result" xml:space="preserve">
          <value>References to the type `{0}` will refer to `{0}, {1}`.</value>
        </data>

Note that this infrastructure does not interoperate with C#6 string
interpolation.  Any error or warning messages currently using C#6
string interpolation will need to use .NET 1.0-style format strings.

Our translation team doesn't work directly with `.resx` files.
Instead, the translation team works with [XLIFF files][2].
`Resources.resx` is converted into a set of
`src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.*.xlf`
files via `XliffTasks.targets` from the [dotnet/xliff-tasks][3] repo.
The `Resources.*.xlf` files should be automatically updated whenever
`Resources.resx` is updated.


Other:

  * This approach leaves the error code `XA4214` as a string literal
    for now.  This differs from what dotnet/sdk and microsoft/msbuild
    do; they instead include the message code as part of the string
    resource in the `.resx` file.  That might sometimes provide useful
    additional context for the translation team, but it also requires
    using a different set of logging methods from
    `Microsoft.Build.Utilities.TaskLoggingHelper`.

  * Fix the Test MSBuild Azure Pipelines build

    Specify the `feedsToUse` and `nugetConfigPath` inputs for the
    [`NuGetCommand@2`][6] Azure Pipelines task so that the NuGet
    restore step will be able to restore XliffTasks successfully from
    the dotnet-eng Azure DevOps NuGet package feed.

    This resolves the following error:

        The nuget command failed with exit code(1) and error(Errors in packages.config projects
            Unable to find version '1.0.0-beta.19252.1' of package 'XliffTasks'.
              C:\Users\dlab14\.nuget\packages\: Package 'XliffTasks.1.0.0-beta.19252.1' is not found on source 'C:\Users\dlab14\.nuget\packages\'.
              https://api.nuget.org/v3/index.json: Package 'XliffTasks.1.0.0-beta.19252.1' is not found on source 'https://api.nuget.org/v3/index.json'.)

TODO:

  * When `Xamarin.Android.Build.Tasks.csproj` is converted into a
    [short-form project][4], add a dependency on dotnet/arcade and
    switch to using the [`GenerateResxSource` mechanism][5] instead
    of using `%(EmbeddedResource.Generator)`=ResXFileCodeGenerator
    and set `$(UsingToolXliff)`=True.  This would match dotnet/sdk.

[0]: https://docs.microsoft.com/dotnet/framework/resources/index
[1]: https://docs.microsoft.com/dotnet/framework/resources/creating-resource-files-for-desktop-apps#resources-in-resx-files
[2]: http://docs.oasis-open.org/xliff/v1.2/os/xliff-core.html
[3]: https://github.com/dotnet/xliff-tasks
[4]: https://docs.microsoft.com/visualstudio/msbuild/how-to-use-project-sdk
[5]: e67d9f0980/Documentation/ArcadeSdk.md (generateresxsource-bool)
[6]: https://docs.microsoft.com/azure/devops/pipelines/tasks/package/nuget
2019-12-07 13:58:58 -05:00
Jonathan Peppers 04f76f864f [build] use a repository-wide NuGet.config (#2903)
Context: https://docs.microsoft.com/en-us/nuget/consume-packages/configuring-nuget-behavior
Context: http://build.devdiv.io/2500191
Context: https://github.com/xamarin/xamarin-android/pull/2859#issuecomment-476691256
Context: https://docs.microsoft.com/en-us/visualstudio/msbuild/exec-task

We are randomly getting failures during Windows builds such as:

	NuGet.targets(114,5): Unable to load the service index for source https://someurl.visualstudio.com/_packaging/Dev/nuget/v3/index.json.
	    Response status code does not indicate success: 401 (Unauthorized).

It appears that some of our build machines on Azure DevOps have a
global (or user-level) `NuGet.Config` file that points to a NuGet
feed we don't have access to.

The way we can workaround this, is to provide our own `NuGet.config`
file that *only* uses the official nuget.org feed.

  * We can put a top-level `NuGet.config` next to our SLN files.
  * We also need to use this file during MSBuild tests.

I also made sure any calls to `nuget restore` are giving us the
highest log information via `-Verbosity Detailed`.  This will tell us
what `NuGet.config` files and feeds are used.

Also in `PrepareWindows.targets`, any `<Exec/>` calls that run
`NuGet.exe` need `IgnoreStandardErrorWarningFormat` set.  Otherwise
MSBuild treats certain output as errors.

Finally, fix the
`AndroidUpdateResourcesTest.CheckXmlResourcesFilesAreProcessed()`
test so that `b.Build (proj, doNotCleanupOnUpdate: true)` is used.
Otherwise, `$(ProjectLockFile)` is deleted on the second build, which
[causes the test to fail][0]:

	Target _GenerateJavaStubs should have been skipped
	Expected: True
	But was:  False

[0]: https://jenkins.mono-project.com/job/xamarin-android-pr-pipeline-release/386/testReport/junit/Xamarin.Android.Build.Tests.AndroidUpdateResourcesTest/CheckXmlResourcesFilesAreProcessed/_False____Release/
2019-04-01 16:07:22 -04:00