Merged PR 611691: Guardian - Add documentation + sample project

- Create initial onboarding documentation for Guardian
- Create a sample project that incorporates the Guardian SDK.

Related work items: #1829804
This commit is contained in:
Pasindu Gunasekara 2021-05-05 19:57:09 +00:00
Родитель 26714f9dfe
Коммит a8410ec4f4
27 изменённых файлов: 809 добавлений и 19 удалений

Просмотреть файл

@ -0,0 +1,8 @@
# Microsoft Guardian Integration with BuildXL Documentation
The Guardian SDK for BuildXL allows for integration of the Microsoft Guardian tool into builds. To learn more about Guardian, see aka.ms/msguardian.
## Links
- [Onboarding Guide](onboarding.md)
- [SDK Location](../../../../Public/Sdk/Public/Tools/Guardian/Tool.Guardian.dsc)
- [Example Project](../../../../Examples/MsGuardian/README.md)

Просмотреть файл

@ -0,0 +1,8 @@
# Generating Baselines and Suppression files for BuildXL builds
The `guardian baseline` and `guardian suppress` commands can be used to baseline and suppress guardian violations. These must be run manually and committed to the repository.
1. Run BuildXL to get a failing build with Guardian violations.
2. Navigate to the output directory for the failing Guardian call (there will be a unique one for each call to Guardian.runGuardian()). The directory should be `Out/Objects/.../<unique directory>/guardianOut`.
3. Run the baseline or suppress command as described in the Guardian wiki with the addition of the following argument: `--settings-file .\buildxl.gdnsettings`
* Example: `guardian baseline --settings-file .\buildxl.gdnsettings -f C:\repo\root\.config\guardian\buildxl_baseline`
4. Make sure to declare the new baseline or suppression files in the GuardianArguments structure.

Просмотреть файл

@ -0,0 +1,54 @@
# Onboarding Guardian
This Guide contains information on how to integrate Guardian into a build using the Guardian SDK.
Currently the following tools are officially supported by the Guardian SDK:
- Credscan
- Roslyn Analyzers
- ESLint
- BinSkim
- Bandit
- SpotBugs
- PSScriptAnalyzers
## Prerequisites
Guardian offers two different installation methods, visit aka.ms/msguardian and pick one that works best for your use case. The sample project uses the standalone install method with NuGet. Note that with the integrated install method, Guardian will automatically check for updates and write new versions of Guardian or Guardian tools under that directory. Additionally, for integrated install, provide the Guardian root directory as a partial seal directory to indicate to the Guardian SDK where to find the binaries: See the Guardian.findGuardianExecutable() function to see how this is implemented.
## Steps
### Initialize Guardian for Repository
1. Initialize Guardian at the root of the repository by running `guardian init`
* This step only needs to be done once.
2. Commit the .gdn directory.
* The following files should be committed:
* .gdn/.gdnsettings
* .gdn/.gitignore
### Generate Guardian Configuration Files
Visit the [Guardian Tools](https://dev.azure.com/securitytools/SecurityIntegration/_wiki/wikis/Guardian/1425/Guardian-Tools) section in the Guardian wiki. Pick the tools that best fit your repository.
Next, see the sample Guardian project for examples of configuration files (under `.config/guardian`) that can be used with BuildXL. All configurations for a single Guardian call should go under one .gdnconfig file. Tool configurations can be generated by running the [guardian configure](https://dev.azure.com/securitytools/SecurityIntegration/_wiki/wikis/Guardian/1395/configure) command. Generate configuration files for each tool, and combine them into a single file as follows.
1. Create a new file under `{RepositoryRoot}/.config/guardian/{configName}.gdnconfig`
2. Inside this new file, use the following template and copy the tool configs that should be used.
```
{
"fileVersion": "1.0",
"tools": [
{
Generated Tool config here
},
{
Generated Tool config here
},
}
```
## Add calls to the Guardian SDK
1. Import Guardian with `import * as Guardian from "Sdk.Guardian"`
2. Create [Guardian.GuardianArguments](../../../../Public/Sdk/Public/Tools/Guardian/Tool.Guardian.dsc) to be passed to the Guardian tool.
* Populate this structure with the configuration file created above.
* Ensure that files that are being scanned by Guardian as per the configuration file are properly declared under the `Guardian.GuardianArguments.filesToBeScanned` array to avoid any DFAs.
* Some tools, such as CredScan, do not provide fine grained control over which files will be scanned. Instead they will recursively read all source files from the working directory.
3. Call `Guardian.runGuardian(args)` to run Guardian with the configuration files created above.
## Best Practices
* Try to minimize concurrent calls into Guardian wherever possible. Guardian does not officially support concurrent executions of the Guardian tool to avoid any bad interactions between tools.

Просмотреть файл

@ -0,0 +1,64 @@
{
"fileVersion": "1.4",
"tools": [
{
"fileVersion": "1.4",
"tool": {
"name": "CredScan",
"version": "2.1.17"
},
"arguments": {
"TargetDirectory": "$(WorkingDirectory)",
"OutputType": "pre",
"SuppressAsError": true
},
"outputExtension": "xml",
"successfulExitCodes": [
0,
2,
4,
6
],
"errorExitCodes": {
"1": "Partial scan completed with warnings.",
"3": "Partial scan completed with credential matches and warnings.",
"5": "Partial scan completed with application warnings and credential matches",
"7": "Partial scan completed with application warnings, suppressed warnings, and credential matches",
"-1000": "Argument Exception.",
"-1100": "Invalid configuration.",
"-1500": "Configuration Exception.",
"-1600": "IO Exception.",
"-9000": "Unexpected Exception."
}
},
{
"fileVersion": "1.0.0",
"tool": {
"name": "AntiMalware",
"version": "latest"
},
"arguments": {
"Function": "analyze",
"Command": "scan",
"ScanType": 3,
"ScanDirectoryOrFile": "$(WorkingDirectory)",
"DisableRemediation": true,
"BootSectorScan": false,
"EnableServices": false,
"CollectLogsOnError": false,
"ForceSignatureUpdate": false,
"SignatureUpdateFailureLoggerLevel": "Warning",
"SignatureFreshness": 48,
"OutdatedSignatureLoggerLevel": "Error",
"LoggerLevel": "Standard"
},
"outputExtension": "sarif",
"successfulExitCodes": [
0
],
"errorExitCodes": {
"2": "Malware is found and not remediated or additional user action is required to complete remediation or there is error in scanning."
}
}
]
},

Просмотреть файл

@ -0,0 +1,51 @@
{
"version": "1.0.0",
"baselines": {
"default": {
"name": "default",
"createdDate": "2021-05-04 21:51:08Z",
"lastUpdatedDate": "2021-05-04 22:22:32Z"
}
},
"results": {
"2b2812e231de8e904d7935fdcf7ba663028306eaa53a6d2643214975e1dff8f3": {
"signature": "2b2812e231de8e904d7935fdcf7ba663028306eaa53a6d2643214975e1dff8f3",
"target": "src",
"memberOf": [
"default"
],
"tool": "antimalware",
"ruleId": "NoThreatsFound",
"justification": null,
"createdDate": "2021-05-04 21:51:08Z",
"expirationDate": null,
"type": null
},
"11fd9a27477a11e2f0ac87c054490cf31cd896caf050a037a04a916808938ccd": {
"signature": "11fd9a27477a11e2f0ac87c054490cf31cd896caf050a037a04a916808938ccd",
"target": "src/Program.cs",
"memberOf": [
"default"
],
"tool": "credscan",
"ruleId": "CSCAN-GENERAL0030",
"justification": null,
"createdDate": "2021-05-04 21:51:08Z",
"expirationDate": null,
"type": null
},
"3937794e0d2e84a0f2bda9ebc6bc670e660ffaa6bc11d4ac2b1c73100d2d124f": {
"signature": "3937794e0d2e84a0f2bda9ebc6bc670e660ffaa6bc11d4ac2b1c73100d2d124f",
"target": "src/Program.cs",
"memberOf": [
"default"
],
"tool": "credscan",
"ruleId": "CSCAN-GENERAL0030",
"justification": null,
"createdDate": "2021-05-04 22:22:32Z",
"expirationDate": null,
"type": null
}
}
}

Просмотреть файл

@ -0,0 +1,7 @@
{
"files": { },
"folders": { },
"overwriteLogs": true,
"telemetryFlushTimeout": 10,
"variables": { }
}

10
Examples/MsGuardian/.gdn/.gitignore поставляемый Normal file
Просмотреть файл

@ -0,0 +1,10 @@
## Ignore Guardian internal files
.r/
rc/
rs/
i/
p/
c/
## Ignore Guardian Local settings
LocalSettings.gdn.json

1
Examples/MsGuardian/.gitignore поставляемый Normal file
Просмотреть файл

@ -0,0 +1 @@
/Out

Просмотреть файл

@ -0,0 +1,18 @@
# MS Guardian Sample
## Introduction
This sample demonstrates how to use the Guardian SDK with BuildXL. See the Guardian documentation in the BuildXL wiki for more details on the Guardian SDK, or aka.ms/msguardian for more details on the Guardian tool.
## Instructions
1. Set the `BUILDXL_BIN` environment variable to the BuildXL deployment folder.
2. Copy the latest versions of the following directories from the BuildXL source code to the sdk directory on this sample: `BuildXLRepoRoot/Public/Sdk/Public/Tools/Guardian` and `BuildXLRepoRoot/Public/Sdk/Public/Json`.
2. Open src/Program.cs and add a CredScan violation inside the Main function. This can be as simple as adding two string variables named Username and Password and setting them to any string.
3. In powershell, run `.\build.ps1`.
4. This build will fail as it has a CredScan violation. To baseline the CredScan violation, uncomment the baselineFiles line within the GuardianArguments structure in [GuardianSample.dsc](src/GuardianSample.dsc).
5. Now create a baseline file. Look at the error that is generated by BuildXL.
* From the error message in the console, look at the last line pointing to the stdout file.
* Example: `C:\path\to\BuildXL.Internal\Examples\MsGuardian\Out\Objects\1\e\98hj2aj7t0oy2cpptnyy4lvq\guardian.cmd.std_2\out.txt`
* Take root of the `guardian.cmd.std_2` directory and append `guardianOut\buildxl.gdnsettings`
* Example: `C:\path\to\BuildXL.Internal\Examples\MsGuardian\Out\Objects\1\e\98hj2aj7t0oy2cpptnyy4lvq` -> `C:\path\to\BuildXL.Internal\Examples\MsGuardian\Out\Objects\1\e\98hj2aj7t0oy2cpptnyy4lvq\guardianOut\buildxl.gdnsettings`
* Then run the following command to create the baseline (ensure that a proper absolute paths are provided as Guardian will not accept relative paths)
* `guardian baseline --settings-file C:\path\to\BuildXL.Internal\Examples\MsGuardian\Out\Objects\1\e\98hj2aj7t0oy2cpptnyy4lvq\guardianOut\buildxl.gdnsettings -f C:\path\to\BuildXL.Internal\Examples\MsGuardian\.config\guardian\buildxl_baseline`
6. Finally, run `.\build.ps1` again to get a passing build.

Просмотреть файл

@ -0,0 +1,7 @@
if (-not (Test-Path env:BUILDXL_BIN))
{
Write-Output "BUILDXL_BIN environment variable must be set to BuildXL deployment folder."
exit
}
& $Env:BUILDXL_BIN/bxl /c:config.dsc

Просмотреть файл

@ -0,0 +1,34 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
config({
modules: [
d`src`,
d`sdk`
].mapMany(dir => [...globR(dir, "module.config.dsc")]),
mounts: [
{
name: a`Deployment`,
isReadable: true,
isWritable: true,
isScrubbable: true,
path: p`Out/deployment`,
trackSourceFileChanges: true
},
],
resolvers: [
{
kind: "Nuget",
configuration: {
toolUrl: "https://dist.nuget.org/win-x86-commandline/v4.9.4/NuGet.exe",
hash: "VSO0:17E8C8C0CDCCA3A6D1EE49836847148C4623ACEA5E6E36E10B691DA7FDC4C39200"
},
// Microsoft internal only
repositories: { "Guardian": "https://securitytools.pkgs.visualstudio.com/_packaging/Guardian/nuget/v3/index.json" },
packages: [
{ id: "Microsoft.Guardian.Cli", version: "0.74.1" },
],
}
],
});

Просмотреть файл

@ -0,0 +1,4 @@
module({
name: "Sdk.Guardian",
projects: [ p`Tool.Guardian.dsc` ]
});

Просмотреть файл

@ -0,0 +1,395 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
import {Artifact, Cmd, Transformer, Tool} from "Sdk.Transformers";
import * as Json from "Sdk.Json";
@@public
export const guardianTag = "msguardian";
const guardianInstallMutex = "BuildXL.Tools.Guardian.Install.Phase";
const guardianExecutableName : PathAtom = PathAtom.create("guardian.cmd");
const defaultGuardianToolWorkingDirectory = d`${Context.getMount("SourceRoot").path}`;
const guardianUntrackedDirectories = addIfLazy(Context.getCurrentHost().os === "win", () => [
// Accessed by the Guardian CLI
d`${Context.getMount("ProgramFilesX86").path}/dotnet`,
d`${Context.getMount("ProgramFiles").path}/dotnet`,
d`${Context.getMount("ProgramData").path}/Microsoft/NetFramework`,
// Config files accessed by nuget during Guardian install phase
d`${Context.getMount("ProgramFilesX86").path}/Nuget`,
d`${Context.getMount("ProgramFiles").path}/Nuget`,
d`${Context.getMount("LocalLow").path}/Microsoft/CryptnetUrlCache`,
]);
/**
* Tool definition for guardian
* Note: Package root is untracked because we rely on the .gdn/c directory to know when a new Guardian tool is available.
*/
function getGuardianTool(guardianRoot : StaticDirectory, guardianPaths : GuardianPaths) : Transformer.ToolDefinition {
return {
exe: findGuardianExecutable(guardianRoot),
description: "Microsoft Guardian",
dependsOnWindowsDirectories: true,
dependsOnAppDataDirectory: true,
prepareTempDirectory: true,
untrackedDirectoryScopes: [...guardianUntrackedDirectories, d`${guardianPaths.install}`],
// Untracking localRepo/.gdnhistory to ignore double writes between guardian init and guardian run
// the history in here is not important because each Guardian call gets it's own .gdnhistory file.
// globalRepo/.gdnhistory will also be read from, but it will not be written to.
untrackedFiles: [ f`${guardianPaths.localHistory}`, f`${guardianPaths.globalHistory}` ]
};
}
/**
* Find a path to the guardian.cmd script given a static directory.
*/
function findGuardianExecutable(guardianRoot : StaticDirectory) : File {
// The guardian.cmd script will be under guardianRoot/tools/guardian.cmd for the standalone nuget package, or
// guardianRoot/guardian.cmd for the Guardian integrated install method.
Contract.assert(guardianRoot.kind === "full" || guardianRoot.kind === "partial", "Guardian root must either be partially or fully sealed.");
return guardianRoot.hasFile(r`tools/${guardianExecutableName}`)
? guardianRoot.getFile(r`tools/${guardianExecutableName}`)
: guardianRoot.getFile(guardianExecutableName);
}
/**
* Schedules a guardian pip with the specified arguments.
*
* Guardian is currently only supported on Windows.
*
* Guardian Lifecycle:
* - The "guardian run" command is several different Guardian commands run at once by the Guardian process.
* - These are split up here instead of a single call because some parts are not able to execute concurrently.
*
* 1. guardian init: Initializes a local .gdn directory - can be executed concurrently with other guardian calls.
* 2. guardian install: Installs packages specified in config file - *can not* be executed concurrently.
* NOTE: acquires mutex: guardianInstallMutex
* 3. guardian run: This call includes the --no-install flag, which allows BuildXL to run it concurrently. It has the following sub-phases:
* a. guardian clear: clears results directories
* b. guardian analyze: Runs analysis tools and stores raw data in .gdn/.r directory
* c. guardian process: Processes the data collected in the previous step into .gdn/r directory
* d. guardian break: Look at processed data from previous step, return bad exit code if breaking results found and export results to file.
*/
@@public
export function runGuardian(args: GuardianArguments) : Transformer.ExecuteResult {
validateArguments(args);
const outputDirectory = Context.getNewOutputDirectory("guardianOut");
const guardianPaths = createGuardianPaths(outputDirectory, args.guardianPackageDirectory);
const guardianTool = getGuardianTool(args.guardianToolRootDirectory, guardianPaths);
let guardianDependencies : Transformer.InputArtifact[] = [args.guardianToolRootDirectory, f`${guardianPaths.globalGuardianRepo}`, args.guardianConfigFile];
// 0. Create a Guardian settings
const genericSettingsFile = generateGenericGuardianSettingsFile(guardianPaths);
const installSettingsFile = generateGuardianInstallSettingsFile(guardianPaths);
// 1. Initialize Guardian for this Guardian run
// - Settings files from previous step not necessary here, can be run concurrently with the WriteFile operation.
const initializeResult = initializeGuardian(guardianTool, guardianPaths, guardianDependencies);
// Steps below this depend on the results of step 0 and step 1
guardianDependencies = guardianDependencies.concat([
f`${guardianPaths.globalSettings}`,
initializeResult.getOutputDirectory(d`${guardianPaths.localGuardianRepo}`)
]);
// 2. Run Guardian Install phase
const installResult = runGuardianInstall(args, guardianTool, installSettingsFile, guardianDependencies, guardianPaths);
// 3. Guardian run to run static analysis tools specified in config file, break build if any breaking changes are found, and export results.
guardianDependencies = guardianDependencies.concat([installResult.getOutputFile(guardianPaths.installLog.path), genericSettingsFile]);
const guardianResult = runGuardianInternal(args, guardianTool, genericSettingsFile, guardianDependencies, guardianPaths);
return guardianResult;
}
/**
* Validates that all required arguments are provided, and no conflicting arguments are set.
*/
function validateArguments(args: GuardianArguments) : void {
Contract.requires(args !== undefined, "Guardian arguments cannot be undefined.");
Contract.requires(args.guardianToolRootDirectory !== undefined, "Guardian root must be set.");
Contract.requires(args.guardianConfigFile !== undefined, "Guardian config file must be set.");
Contract.requires(args.guardianResultFile !== undefined, "Guardian output file must be set.");
Contract.requires(args.guardianPackageDirectory !== undefined, "Guardian tool package install directory must be set.");
Contract.requires(args.filesToBeScanned !== undefined, "Files to be scanned by Guardian must be set.");
if (args.fast && args.baselineFiles) {
Contract.fail("The --fast argument is incompatible with the output baseline file argument, as this will require a full run of guardian break to generate all the results.");
}
if (args.noBaseline && args.baselineFiles) {
Contract.fail("noBaseline and baselineFile cannot be specified together.");
}
if (args.noSuppressions && (args.suppressionFiles || args.suppressionSets)) {
Contract.fail("noSuppressions and suppressionFiles/suppressionSets cannot be specified together.");
}
}
/**
* Generates a set of paths that are produced/consumed by Guardian based on a generated output directory
* and user specified package install directory.
*/
function createGuardianPaths(outputDirectory : Directory, packageDirectory : Directory) : GuardianPaths {
return {
globalGuardianRepo: p`${Context.getMount("SourceRoot").path}/.gdn`,
localGuardianRepo: p`${outputDirectory}/.gdn`,
config: p`${outputDirectory}/.gdn/c`,
rawResults: p`${outputDirectory}/.r`,
results: p`${outputDirectory}/r`,
convertedResults: p`${outputDirectory}/rc`,
install: p`${packageDirectory}`,
localSettings: p`${outputDirectory}/buildxl.gdnsettings`,
localInstallSettings: p`${outputDirectory}/buildxl_install.gdnsettings`,
globalSettings: p`${Context.getMount("SourceRoot").path}/.gdn/.gdnsettings`,
localHistory: p`${outputDirectory}/.gdn/internal.gdnhistory`,
globalHistory: p`${Context.getMount("SourceRoot").path}/.gdn/internal.gdnhistory`,
installLog: f`${outputDirectory}/install`,
};
}
/**
* Generates a Guardian configuration using specified paths.
*/
function generateGenericGuardianSettingsFile(guardianPaths : GuardianPaths) : File {
const settings = {
folders: {
GuardianRepo: guardianPaths.localGuardianRepo,
Config: guardianPaths.config,
RawResults: guardianPaths.rawResults,
Results: guardianPaths.results,
ConvertedResults: guardianPaths.convertedResults,
Install: guardianPaths.install
}
};
return generateGuardianSettingsFile(guardianPaths.localSettings, settings, "Generate generic Guardian settings file");
}
/**
* Generates a config file for Guardian install step. This requires a separate settings file without config, raw results, results, or converted results directories set
* so that it uses the local .gdn repo.
*/
function generateGuardianInstallSettingsFile(guardianPaths : GuardianPaths) : File {
const settings = {
folders: {
GuardianRepo: guardianPaths.localGuardianRepo,
Install: guardianPaths.install
}
};
return generateGuardianSettingsFile(guardianPaths.localInstallSettings, settings, "Generate Guardian install settings file");
}
/**
* Call Json SDK to write settings file.
*/
function generateGuardianSettingsFile(outputPath : Path, settings : Object, description : string) : File {
const options : Json.AdditionalJsonOptions = {
pathRenderingOption: Context.getCurrentHost().os !== "win" ? "escapedBackSlashes" : "forwardSlashes"
};
return Json.write(outputPath, settings, "\"", [guardianTag], "Generate Guardian install settings file", options);
}
/**
* Initialize Guardian under a new output directory for this run. This will allow BuildXL to isolate each Guardian call into it's own repository.
*/
function initializeGuardian(guardianTool : Transformer.ToolDefinition, guardianPaths : GuardianPaths, guardianDependencies : Transformer.InputArtifact[] ) : Transformer.ExecuteResult {
return Transformer.execute({
tool: guardianTool,
tags: [ guardianTag ],
arguments: [Cmd.argument("init"), Cmd.argument("--force")],
workingDirectory: d`${guardianPaths.localGuardianRepo.parent}`,
dependencies: guardianDependencies,
outputs: [ d`${guardianPaths.localGuardianRepo}` ],
description: "Guardian Initialize"
});
}
/**
* Runs the "guardian install" command for the tools specified by the user
* Packages will be installed under guardianPaths.install
* This step requires a mutex as multiple Guardian calls may try to install at the same time to the same directory.
*/
function runGuardianInstall(args : GuardianArguments, guardianTool : Transformer.ToolDefinition, settingsFile : File, guardianDependencies : Transformer.InputArtifact[], guardianPaths : GuardianPaths) : Transformer.ExecuteResult {
const arguments : Argument[] = [
Cmd.argument("install"),
Cmd.option("--settings-file ", settingsFile.path),
Cmd.option("--config ", args.guardianConfigFile.path),
Cmd.option("--logger-filepath ", guardianPaths.installLog.path)
];
guardianDependencies = guardianDependencies.push(settingsFile);
return Transformer.execute({
tool: guardianTool,
tags: [ guardianTag ],
arguments: arguments,
workingDirectory: d`${guardianPaths.localGuardianRepo.parent}`,
dependencies: guardianDependencies,
outputs: [ guardianPaths.installLog ],
acquireMutexes: [ guardianInstallMutex ],
description: "Guardian Install"
});
}
/**
* Performs a Guardian run operation. Uses the --no-install argument to skip installing, since this was already done in a previous call.
* See block comment on the runGuardian function to see which steps this call will perform.
*/
function runGuardianInternal(args : GuardianArguments, guardianTool : Transformer.ToolDefinition, settingsFile : File, guardianDependencies : Transformer.InputArtifact[], guardianPaths : GuardianPaths) : Transformer.ExecuteResult {
const arguments: Argument[] = [
Cmd.argument("run"),
Cmd.argument("--no-install"),
Cmd.option("--settings-file ", settingsFile.path),
Cmd.option("--config ", args.guardianConfigFile.path),
Cmd.option("--export-breaking-results-to-file ", args.guardianResultFile.path),
Cmd.option("--logger-filepath ", args.loggerPath && args.loggerPath.path),
Cmd.flag("--analyze-fast", args.fast),
Cmd.flag("--no-baseline", args.noBaseline),
Cmd.option("--baseline-file ", Cmd.join(" ", args.baselineFiles && args.baselineFiles.map(e => e.path))),
Cmd.flag("--no-suppressions", args.noSuppressions),
Cmd.option("--suppression-file ", Cmd.join(" ", args.suppressionFiles && args.suppressionFiles.map(e => e.path))),
Cmd.options("--suppression-set ", args.suppressionSets),
Cmd.flag("--no-policy", args.noPolicy),
Cmd.option("--policy ", args.policy)
];
// Dependencies
guardianDependencies = guardianDependencies.concat(args.filesToBeScanned);
if (args.baselineFiles) {
guardianDependencies = guardianDependencies.concat(args.baselineFiles);
}
if (args.suppressionFiles) {
guardianDependencies = guardianDependencies.concat(args.suppressionFiles);
}
// Outputs
// If no errors are generated, then no output file will be generated.
const maybeExportFile : Transformer.FileOrPathOutput = { existence: "optional", artifact: f`${args.guardianResultFile.path}` };
// Log file only generated if caller sets args.loggerPath
const maybeLogFile : Transformer.FileOrPathOutput = args.loggerPath
? { existence: "optional", artifact: f`${args.loggerPath.path}` }
: undefined;
let outputs : Transformer.Output[] = [
maybeExportFile,
d`${guardianPaths.rawResults}`,
d`${guardianPaths.results}`,
d`${guardianPaths.convertedResults}`
];
if (maybeLogFile) {
outputs = outputs.push(maybeLogFile);
}
// Run Guardian
const result : Transformer.ExecuteResult = Transformer.execute({
tool: guardianTool,
tags: [guardianTag],
arguments: arguments,
workingDirectory: args.guardianToolWorkingDirectory ? args.guardianToolWorkingDirectory : defaultGuardianToolWorkingDirectory,
dependencies: guardianDependencies,
outputs: outputs,
description: "Guardian Run"
});
return result;
}
/**
* Set of guardian arguments. See notes on each argument for any special considerations that need to be
* taken before using them.
*/
@@public
export interface GuardianArguments extends Transformer.RunnerArguments {
/** Root directory for Guardian install (contains guardian.cmd)
** Note: Guardian will still read/write to this directory for updates to itself
** or for tool updates, so this should be partially sealed.
**
** Under the guardian root directory the guardian tool will:
** - Get new versions of itself to ./versions/, and run the latest Guardian Cli from here
** - Get the latest versions of tools being run under ./packages/
** - Read ./.gdnversion to check the current Guardian version to check whether it should be updated
** - Potentially write temporary files to ./.tmp/ */
guardianToolRootDirectory: StaticDirectory;
/** A configuration file to run with guardian */
guardianConfigFile: File;
/** Path to export guardian result file
** Note: No export file will be generated if guardian executes without any errors. */
guardianResultFile: File;
/** Specify where Guardian tool packages will be installed.
** Note: This path can either be the .gdn/i directory, but must always be declared so that Guardian does
** try to read packages from outside of this location (due to its package cache feature). */
guardianPackageDirectory: Directory;
/** Optional Guardian tool working directory. Default: SourceRoot
** Note: the default directory "TargetDirectory" for many tools will be this working directory. */
guardianToolWorkingDirectory?: Directory;
/** Collection of files that guardian will be scanning
** Note: in many cases (such as with credscan), a user cannot specify which files specifically
** to run through the tool, so glob all source files that will be touched by the tool here */
filesToBeScanned: File[];
/** Any additional artifacts to depend on before running Guardian in addition to the ones declared in filesToBeScanned */
additionalDependencies?: Transformer.InputArtifact[];
/** An optional path to a file to redirect the guardian logger output */
loggerPath?: File;
/** Indicate whether Guardian should fail entire job after the first failure
** Note: defaults to false */
fast?: boolean;
/** Will not allow the use of any baselines including the default baseline (.gdn/.gdnbaselines)*/
noBaseline?: boolean;
/** Absolute paths to optional baseline files */
baselineFiles?: File[];
/** Will not allow any suppressions including the default suppression set in .gdn/.gdnsuppress */
noSuppressions?: boolean;
/** Absolute paths to optional suppression files */
suppressionFiles?: File[];
/** Will use specified suppression set(s) to filter breaking results. Only results in these sets will cause a break */
suppressionSets?: string[];
/** Specify policy name (default: microsoft) for break method */
policy?: string;
/** For use with policy, set minimum severity as defined by policy to cause a break. Any results below this severity will NOT cause a break. */
severity?: string;
/** Do not apply any policy */
noPolicy?: boolean;
}
/**
* Collection of Paths that are produced or consumed by Guardian.
*/
interface GuardianPaths {
/** SourceRoot/.gdn */
globalGuardianRepo: Path,
/** OutputDir/.gdn */
localGuardianRepo: Path,
/** localGuardianRepo/c */
config: Path,
/** OutputDir/.r */
rawResults: Path,
/** OutputDir/r */
results: Path,
/** OutputDir/rc */
convertedResults: Path,
/** Path to Guardian tool install directory (usually located under .gdn/i or guardianRoot/packages). */
install: Path,
/** Path to locally generated Guardian settings file to be used */
localSettings: Path,
/** Path to locally generated Guardian settings file to be used only for guardian install. */
localInstallSettings: Path,
/** Path to global settings file checked into repository */
globalSettings: Path,
/** localGuardianRepo/internal.gdnhistory */
localHistory: Path,
/** globalGuardianRepo/internal.gdnhistory */
globalHistory: Path,
/** intermediate Guardian install log file. */
installLog: File,
}

Просмотреть файл

@ -0,0 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
module({
name: "BuildXL.Tools.Guardian"
});

Просмотреть файл

@ -0,0 +1,27 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
@@public
/** Writes an object as Json */
export function write<T extends Object>(destinationFile: Path, data: T, quoteChar?: "'" | "\"", tags?: string[], description?: string, additionalOptions? : AdditionalJsonOptions): File
{
return _PreludeAmbientHack_Json.write<T>(destinationFile, data, quoteChar, tags, description, additionalOptions);
}
/** Additional options that can be specified for Json output. */
@@public
export interface AdditionalJsonOptions
{
// Optionally indicate whether paths should have any special escaping added before being written.
pathRenderingOption? : PathRenderingOption;
}
/**
* Indicate how Paths should be rendered when written to Json.
* - none: No additional transformations are performed, path separator will be based on OS (default).
* - backSlashes: Always use backs lashes as path separator (not escaped).
* - escapedBackSlashes: Always use back slashes as path separator with escape characters.
* - forwardSlashes: Always use forward slashes as path separator.
*/
@@public
export type PathRenderingOption = "none" | "backSlashes" | "escapedBackSlashes" | "forwardSlashes";

Просмотреть файл

@ -0,0 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
module({
name: "Sdk.Json"
});

Просмотреть файл

@ -0,0 +1,2 @@
# Copy SDK Files
Before running the Guardian sample, please copy the latest Guardian and Json SDK directories from BuildXL to under this sdk directory.

Просмотреть файл

@ -0,0 +1,39 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
import * as Guardian from "BuildXL.Tools.Guardian";
const guardianPkg : StaticDirectory = importFrom("Microsoft.Guardian.Cli").Contents.all;
const logDirectory = d`${Context.getMount("Deployment").path}/Guardian/Logs`;
const configDirectory = d`${Context.getMount("SourceRoot").path}/.config/guardian`;
const packageDirectory = d`${Context.getMount("Deployment").path}/Guardian/packages`;
const workingDirectory = d`${Context.getMount("SourceRoot").path}/src`;
// Log files
const guardianExportFile = f`${logDirectory.path}/export.sarif`;
const guardianStdOut = f`${logDirectory.path}/guardian.log`;
// Config files
const guardianConfig = f`${configDirectory.path}/buildxl.gdnconfig`;
// Baselines
const guardianBaselines = glob(configDirectory, "*.gdnbaselines");
// Suppressions
const guardianSuppressions = glob(configDirectory, "*.gdnsuppressions");
// Files to scan - credscan will scan everything in the working directory
const files = globR(workingDirectory, "*");
const guardianArgs : Guardian.GuardianArguments = {
guardianToolRootDirectory: guardianPkg,
guardianConfigFile: guardianConfig,
guardianResultFile: guardianExportFile,
guardianPackageDirectory: packageDirectory,
guardianToolWorkingDirectory: workingDirectory,
filesToBeScanned: files,
loggerPath: guardianStdOut,
// baselineFiles: guardianBaselines // Uncomment this line to baseline the CredScan violation for a successful build
};
const guardianRun = Guardian.runGuardian(guardianArgs);

Просмотреть файл

@ -0,0 +1,12 @@
using System;
namespace Test
{
class Program
{
static void Main(string[] args)
{
// Add CredScan violation here
}
}
}

Просмотреть файл

@ -0,0 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
module({
name: "GuardianSample"
});

Просмотреть файл

@ -0,0 +1,4 @@
module({
name: "Sdk.Guardian",
projects: [ p`Tool.Guardian.dsc` ]
});

Просмотреть файл

@ -18,6 +18,7 @@ const guardianUntrackedDirectories = addIfLazy(Context.getCurrentHost().os === "
// Config files accessed by nuget during Guardian install phase
d`${Context.getMount("ProgramFilesX86").path}/Nuget`,
d`${Context.getMount("ProgramFiles").path}/Nuget`,
d`${Context.getMount("LocalLow").path}/Microsoft/CryptnetUrlCache`,
]);
/**
@ -26,7 +27,7 @@ const guardianUntrackedDirectories = addIfLazy(Context.getCurrentHost().os === "
*/
function getGuardianTool(guardianRoot : StaticDirectory, guardianPaths : GuardianPaths) : Transformer.ToolDefinition {
return {
exe: f`${guardianRoot.path}/${guardianExecutableName}`,
exe: findGuardianExecutable(guardianRoot),
description: "Microsoft Guardian",
dependsOnWindowsDirectories: true,
dependsOnAppDataDirectory: true,
@ -39,6 +40,20 @@ function getGuardianTool(guardianRoot : StaticDirectory, guardianPaths : Guardia
};
}
/**
* Find a path to the guardian.cmd script given a static directory.
*/
function findGuardianExecutable(guardianRoot : StaticDirectory) : File {
// The guardian.cmd script will be under guardianRoot/tools/guardian.cmd for the standalone nuget package, or
// guardianRoot/guardian.cmd for the Guardian integrated install method.
Contract.assert(guardianRoot.kind === "full" || guardianRoot.kind === "partial", "Guardian root must either be partially or fully sealed.");
return guardianRoot.hasFile(r`tools/${guardianExecutableName}`)
? guardianRoot.getFile(r`tools/${guardianExecutableName}`)
: guardianRoot.getFile(guardianExecutableName);
}
/**
* Schedules a guardian pip with the specified arguments.
*

Просмотреть файл

@ -9,11 +9,7 @@ namespace Sdk.Tests {
@@Testing.unitTest()
export function runGuardian() {
addMounts();
const guardianToolDirectory = Transformer.sealPartialDirectory({
root: d`path/to/guardian`,
files: [],
});
const guardianToolDirectory = getGuardianDirectory();
const guardianTest = Guardian.runGuardian({
guardianToolRootDirectory: guardianToolDirectory,
@ -31,10 +27,7 @@ namespace Sdk.Tests {
@@Testing.unitTest()
export function runGuardianWithBadInputs() {
addMounts();
const guardianToolDirectory = Transformer.sealPartialDirectory({
root: d`path/to/guardian`,
files: [],
});
const guardianToolDirectory = getGuardianDirectory();
Testing.expectFailure(
() => Guardian.runGuardian(undefined),
@ -84,10 +77,7 @@ namespace Sdk.Tests {
@@Testing.unitTest()
export function runGuardianWithConflictingInputs() {
addMounts();
const guardianToolDirectory = Transformer.sealPartialDirectory({
root: d`path/to/guardian`,
files: [],
});
const guardianToolDirectory = getGuardianDirectory();
Testing.expectFailure(
() =>
@ -153,6 +143,13 @@ namespace Sdk.Tests {
);
}
function getGuardianDirectory() : StaticContentDirectory {
return Transformer.sealPartialDirectory({
root: d`path/to/guardian`,
files: [f`path/to/guardian/guardian.cmd`],
});
}
function addMounts() {
Testing.setMountPoint({
name: a`GuardianRoot`,
@ -226,5 +223,14 @@ namespace Sdk.Tests {
isScrubbable: false,
trackSourceFileChanges: true,
});
Testing.setMountPoint({
name: a`LocalLow`,
path: p`LocalLow`,
isReadable: true,
isWritable: true,
isSystem: true,
isScrubbable: false,
trackSourceFileChanges: true,
});
}
}

Просмотреть файл

@ -1,4 +1,5 @@
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: []});
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: [f`./path/to/guardian/guardian.cmd`]});
Transformer.writeFile(
f`./obj_test/k/w/6k5ykzrovlnle3hdbgx4lv7q/guardianOut/buildxl.gdnsettings`,
{
@ -63,6 +64,7 @@ Transformer.execute({
p`./path/to/ProgramData/Microsoft/NetFramework`,
p`./path/to/ProgramFilesX86/Nuget`,
p`./path/to/ProgramFiles/Nuget`,
p`./LocalLow/Microsoft/CryptnetUrlCache`,
p`./path/to/guardian/packages`,
p`\${Context.getMount('Windows').path}`,
p`\${Context.getMount('InternetCache').path}`,
@ -137,6 +139,7 @@ Transformer.execute({
p`./path/to/ProgramData/Microsoft/NetFramework`,
p`./path/to/ProgramFilesX86/Nuget`,
p`./path/to/ProgramFiles/Nuget`,
p`./LocalLow/Microsoft/CryptnetUrlCache`,
p`./path/to/guardian/packages`,
p`\${Context.getMount('Windows').path}`,
p`\${Context.getMount('InternetCache').path}`,
@ -233,6 +236,7 @@ Transformer.execute({
p`./path/to/ProgramData/Microsoft/NetFramework`,
p`./path/to/ProgramFilesX86/Nuget`,
p`./path/to/ProgramFiles/Nuget`,
p`./LocalLow/Microsoft/CryptnetUrlCache`,
p`./path/to/guardian/packages`,
p`\${Context.getMount('Windows').path}`,
p`\${Context.getMount('InternetCache').path}`,

Просмотреть файл

@ -1 +1,2 @@
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: []});
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: [f`./path/to/guardian/guardian.cmd`]});

Просмотреть файл

@ -1 +1,2 @@
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: []});
Transformer.sealPartialDirectory({root: d`./path/to/guardian`, files: [f`./path/to/guardian/guardian.cmd`]});

Просмотреть файл

@ -6,8 +6,8 @@ namespace StandardSdk.Json {
export const hashingTest = BuildXLSdk.sdkTest({
testFiles: globR(d`.`, "Test.*.dsc"),
sdkFolders: [
d`${Context.getMount("SdkRoot").path}/Json`,
d`${Context.getMount("SdkRoot").path}/Json`
],
// autoFixLkgs: true, // Uncomment this line to have all lkgs automatically updated.
autoFixLkgs: true, // Uncomment this line to have all lkgs automatically updated.
});
}