Switch to LTRData.DiscUtils fork (#145)
* Switch nuget package to LTRData.DiscUtils * Add regression test for corrupt WIM * Implement DMG Extractor + Add DMG Test file --------- Co-authored-by: Erik White <erik.white@griffeye.com> Co-authored-by: Gabe Stocco <98900+gfs@users.noreply.github.com>
This commit is contained in:
Erik White
GitHub
Родитель
ac94200eeb
Коммит
38ad5d6472
|
|
@ -1,26 +0,0 @@
|
|||
name: Publish Code Coverage
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
jobs:
|
||||
build:
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Setup .NET Core
|
||||
uses: actions/setup-dotnet@v1
|
||||
with:
|
||||
dotnet-version: 3.1.301
|
||||
- name: Install dependencies
|
||||
run: dotnet restore
|
||||
- name: Test
|
||||
run: dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./coverage/lcov.info
|
||||
- name: Coveralls
|
||||
uses: coverallsapp/github-action@master
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
name: Publish Blazor
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
jobs:
|
||||
build:
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Setup .NET Core
|
||||
uses: actions/setup-dotnet@v1
|
||||
with:
|
||||
dotnet-version: 8.0.x
|
||||
# Include wasm-tools to optimize build
|
||||
- name: wasm-tools
|
||||
run: dotner workload install wasm-tools
|
||||
- name: Install dependencies
|
||||
run: dotnet restore
|
||||
- name: Build
|
||||
run: dotnet publish RecursiveExtractor.Blazor --configuration Release --no-restore -o blazorOut
|
||||
- name: GitHub Pages action
|
||||
uses: peaceiris/actions-gh-pages@v3
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
publish_branch: gh-pages
|
||||
publish_dir: blazorOut/wwwroot
|
||||
|
|
@ -13,7 +13,7 @@ resources:
|
|||
- repository: templates
|
||||
type: git
|
||||
name: SecurityEngineering/OSS-Tools-Pipeline-Templates
|
||||
ref: refs/tags/v1.1.0
|
||||
ref: refs/tags/v1.1.1
|
||||
|
||||
variables:
|
||||
BuildConfiguration: 'Release'
|
||||
|
|
@ -27,13 +27,13 @@ stages:
|
|||
parameters:
|
||||
jobName: 'lib_dotnet_test_windows'
|
||||
dotnetVersions: ['6.0.x','7.0.x','8.0.x']
|
||||
vmImage: 'oss-tools-win2022_1es-managed'
|
||||
vmImage: 'win2022-image-base'
|
||||
projectPath: 'RecursiveExtractor.Tests/RecursiveExtractor.Tests.csproj'
|
||||
- template: dotnet-test-job.yml@templates
|
||||
parameters:
|
||||
jobName: 'cli_dotnet_test_windows'
|
||||
dotnetVersions: ['6.0.x','7.0.x','8.0.x']
|
||||
vmImage: 'oss-tools-win2022_1es-managed'
|
||||
vmImage: 'win2022-image-base'
|
||||
projectPath: 'RecursiveExtractor.Cli.Tests/RecursiveExtractor.Cli.Tests.csproj'
|
||||
|
||||
- stage: SDL
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ resources:
|
|||
- repository: templates
|
||||
type: git
|
||||
name: SecurityEngineering/OSS-Tools-Pipeline-Templates
|
||||
ref: refs/tags/v1.1.0
|
||||
ref: refs/tags/v1.1.1
|
||||
|
||||
variables:
|
||||
BuildConfiguration: 'Release'
|
||||
|
|
@ -32,13 +32,13 @@ stages:
|
|||
parameters:
|
||||
jobName: 'lib_dotnet_test_windows'
|
||||
dotnetVersions: ['6.0.x','7.0.x','8.0.x']
|
||||
vmImage: 'oss-tools-win2022_1es-managed'
|
||||
vmImage: 'win2022-image-base'
|
||||
projectPath: 'RecursiveExtractor.Tests/RecursiveExtractor.Tests.csproj'
|
||||
- template: dotnet-test-job.yml@templates
|
||||
parameters:
|
||||
jobName: 'cli_dotnet_test_windows'
|
||||
dotnetVersions: ['6.0.x','7.0.x','8.0.x']
|
||||
vmImage: 'oss-tools-win2022_1es-managed'
|
||||
vmImage: 'win2022-image-base'
|
||||
projectPath: 'RecursiveExtractor.Cli.Tests/RecursiveExtractor.Cli.Tests.csproj'
|
||||
|
||||
- stage: SDL
|
||||
|
|
@ -87,7 +87,7 @@ stages:
|
|||
displayName: Code Sign, Generate Hashes, Publish Public Releases
|
||||
pool:
|
||||
name: 'OSS-Tools-1ESPool'
|
||||
vmImage: 'oss-tools-win2022_1es-managed'
|
||||
vmImage: 'win2022-image-base'
|
||||
steps:
|
||||
- task: UseDotNet@2 # For ESRP. Do not use variable.
|
||||
inputs:
|
||||
|
|
|
|||
32
README.md
32
README.md
|
|
@ -7,17 +7,18 @@ Recursive Extractor is a Cross-Platform [.NET Standard 2.0 Library](#library) an
|
|||
| | | |
|
||||
|-|-|-|
|
||||
| 7zip+ | ar | bzip2 |
|
||||
| deb | gzip | iso |
|
||||
| rar^ | tar | vhd |
|
||||
| vhdx | vmdk | wim* |
|
||||
| xzip | zip+ | |
|
||||
| deb | dmg** | gzip |
|
||||
| iso | rar^ | tar |
|
||||
| vhd | vhdx | vmdk |
|
||||
| wim* | xzip | zip+ |
|
||||
|
||||
<details>
|
||||
<summary>Details</summary>
|
||||
<br/>
|
||||
* Windows only<br/>
|
||||
+ Encryption Supported<br/>
|
||||
^ Rar version 4 Encryption supported<br/>
|
||||
^ Encryption supported for Rar version 4 only<br/>
|
||||
** Limited support. Unencrypted HFS+ volumes with certain compression schemes.
|
||||
</details>
|
||||
|
||||
# Variants
|
||||
|
|
@ -25,9 +26,9 @@ Recursive Extractor is a Cross-Platform [.NET Standard 2.0 Library](#library) an
|
|||
## Command Line
|
||||
### Installing
|
||||
1. Ensure you have the latest [.NET SDK](https://dotnet.microsoft.com/download).
|
||||
2. run `dotnet tool install -g Microsoft.CST.RecursiveExtractor.Cli`
|
||||
2. Run `dotnet tool install -g Microsoft.CST.RecursiveExtractor.Cli`
|
||||
|
||||
This adds `RecursiveExtractor` to your path so you can run it directly from the shell.
|
||||
This adds `RecursiveExtractor` to your path so you can run it directly from your shell.
|
||||
|
||||
### Running
|
||||
Basic usage is: `RecursiveExtractor --input archive.ext --output outputDirectory`
|
||||
|
|
@ -57,7 +58,7 @@ Run `RecursiveExtractor --help` for more details.
|
|||
</details>
|
||||
|
||||
## .NET Standard Library
|
||||
Recursive Extractor is available on NuGet as [Microsoft.CST.RecursiveExtractor](https://www.nuget.org/packages/Microsoft.CST.RecursiveExtractor/). Recursive Extractor targets netstandard2.0+ and the latest .NET, currently .NET 6.0 and .NET 7.0.
|
||||
Recursive Extractor is available on NuGet as [Microsoft.CST.RecursiveExtractor](https://www.nuget.org/packages/Microsoft.CST.RecursiveExtractor/). Recursive Extractor targets netstandard2.0+ and the latest .NET, currently .NET 6.0, .NET 7.0 and .NET 8.0.
|
||||
|
||||
### Usage
|
||||
|
||||
|
|
@ -77,8 +78,7 @@ foreach(var file in extractor.Extract(path))
|
|||
<details>
|
||||
<summary>Extracting to Disk</summary>
|
||||
<br/>
|
||||
This code adapted from the Cli extracts the contents of given archive located at `options.Input`
|
||||
to a directory located at `options.Output`, including extracting failed archives as themselves.
|
||||
This code adapted from the Cli extracts the contents of given archive located at `options.Input` to a directory located at `options.Output`, including extracting failed archives as themselves.
|
||||
|
||||
```csharp
|
||||
using Microsoft.CST.RecursiveExtractor;
|
||||
|
|
@ -166,12 +166,12 @@ catch(OverflowException)
|
|||
RecursiveExtractor protects against [ZipSlip](https://snyk.io/research/zip-slip-vulnerability), [Quines, and Zip Bombs](https://en.wikipedia.org/wiki/Zip_bomb).
|
||||
Calls to Extract will throw an `OverflowException` when a Quine or Zip bomb is detected and a `TimeOutException` if `EnableTiming` is set and the specified time period has elapsed before completion.
|
||||
|
||||
Otherwise, invalid files found while crawling will emit a logger message and be skipped. RecursiveExtractor uses NLog for logging.
|
||||
Otherwise, invalid files found while crawling will emit a logger message and be skipped. You can also enable `ExtractSelfOnFail` to return the original archive file on an extraction failure.
|
||||
|
||||
## Notes on Enumeration
|
||||
|
||||
### Multiple Enumeration
|
||||
You should not iterate the Enumeration returned from the `Extract` and `ExtractAsync` interfaces multiple times, if you need to do so, convert the Enumeration to the collection of your choice first.
|
||||
You should not iterate the Enumeration returned from the `Extract` and `ExtractAsync` interfaces multiple times, if you need to do so, convert the Enumeration to an in memory collection first.
|
||||
|
||||
### Parallel Enumeration
|
||||
If you want to enumerate the output with parallelization you should use a batching mechanism, for example:
|
||||
|
|
@ -208,7 +208,7 @@ while (moreAvailable)
|
|||
```
|
||||
|
||||
### Disposing During Enumeration
|
||||
If you are working with a very large archive or in particularly constrained environment you can reduce memory/file handle usage for the Content streams in each FileEntry by disposing as you iterate.
|
||||
If you are working with a very large archive or in particularly constrained environment you can reduce memory and file handle usage for the Content streams in each FileEntry by disposing as you iterate.
|
||||
|
||||
```csharp
|
||||
var results = extractor.Extract(path);
|
||||
|
|
@ -217,7 +217,7 @@ foreach(var file in results)
|
|||
using var theStream = file.Content;
|
||||
// Do something with the stream.
|
||||
_ = theStream.ReadByte();
|
||||
// The stream is disposed here from the using statement
|
||||
// The stream is disposed here by the using statement
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -229,11 +229,11 @@ If you are having trouble parsing a specific archive of one of the supported for
|
|||
|
||||
# Dependencies
|
||||
|
||||
Recursive Extractor uses a number of libraries to parse archives.
|
||||
Recursive Extractor aims to provide a unified interface to extract arbitrary archives and relies on a number of libraries to parse the archives.
|
||||
|
||||
* [SharpZipLib](https://github.com/icsharpcode/SharpZipLib)
|
||||
* [SharpCompress](https://github.com/adamhathcock/sharpcompress)
|
||||
* [DiscUtils](https://github.com/discutils/discutils)
|
||||
* [LTRData/DiscUtils](https://github.com/LTRData/discutils)
|
||||
|
||||
# Contributing
|
||||
|
||||
|
|
|
|||
|
|
@ -7,8 +7,8 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="8.0.0" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="8.0.0" PrivateAssets="all" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="8.0.1" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="8.0.1" PrivateAssets="all" />
|
||||
<PackageReference Include="System.Net.Http.Json" Version="8.0.0" />
|
||||
<PackageReference Include="Tewr.Blazor.FileReader" Version="3.3.2.23201" />
|
||||
</ItemGroup>
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@
|
|||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
|
||||
<PackageReference Include="MSTest.TestAdapter" Version="3.1.1" />
|
||||
<PackageReference Include="MSTest.TestFramework" Version="3.1.1" />
|
||||
<PackageReference Include="MSTest.TestAdapter" Version="3.2.0" />
|
||||
<PackageReference Include="MSTest.TestFramework" Version="3.2.0" />
|
||||
<PackageReference Include="coverlet.collector" Version="6.0.0">
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
|
|
|
|||
|
|
@ -42,7 +42,8 @@ namespace RecursiveExtractor.Tests.ExtractorTests
|
|||
new object[] { "EmptyFile.txt", 1 },
|
||||
new object[] { "TestDataArchivesNested.Zip", 54 },
|
||||
new object[] { "UdfTest.iso", 3 },
|
||||
new object[] { "UdfTestWithMultiSystem.iso", 3 }
|
||||
new object[] { "UdfTestWithMultiSystem.iso", 3 },
|
||||
new object[] { "HfsSampleUDCO.dmg", 2 }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -73,7 +74,8 @@ namespace RecursiveExtractor.Tests.ExtractorTests
|
|||
new object[] { "TestData.wim", 3 },
|
||||
new object[] { "EmptyFile.txt", 1 },
|
||||
new object[] { "TestDataArchivesNested.Zip", 14 },
|
||||
new object[] { "UdfTestWithMultiSystem.iso", 3 }
|
||||
new object[] { "UdfTestWithMultiSystem.iso", 3 },
|
||||
new object[] { "HfsSampleUDCO.dmg", 2 }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ public class MiniMagicTests : BaseExtractorTestClass
|
|||
[DataRow("TestData.vhdx", ArchiveFileType.VHDX)]
|
||||
[DataRow("TestData.wim", ArchiveFileType.WIM)]
|
||||
[DataRow("Empty.vmdk", ArchiveFileType.VMDK)]
|
||||
[DataRow("HfsSampleUDCO.dmg", ArchiveFileType.DMG)]
|
||||
[DataRow("EmptyFile.txt", ArchiveFileType.UNKNOWN)]
|
||||
public void TestMiniMagic(string fileName, ArchiveFileType expectedArchiveFileType)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -49,6 +49,7 @@ public class MiscTests
|
|||
[DataRow("TestDataCorrupt.tar", true, 1, 1)]
|
||||
[DataRow("TestDataCorrupt.tar.zip", false, 0, 2)]
|
||||
[DataRow("TestDataCorrupt.tar.zip", true, 0, 2)]
|
||||
[DataRow("TestDataCorruptWim.zip", true, 0, 0)]
|
||||
public void ExtractCorruptArchive(string fileName, bool requireTopLevelToBeArchive, int expectedNumFailures, int expectedNumFiles)
|
||||
{
|
||||
var extractor = new Extractor();
|
||||
|
|
|
|||
|
|
@ -9,12 +9,12 @@
|
|||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
|
||||
<PackageReference Include="DiscUtils.Btrfs" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.HfsPlus" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.SquashFs" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Xfs" Version="0.16.13" />
|
||||
<PackageReference Include="MSTest.TestAdapter" Version="3.1.1" />
|
||||
<PackageReference Include="MSTest.TestFramework" Version="3.1.1" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Btrfs" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.HfsPlus" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.SquashFs" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Xfs" Version="1.0.36" />
|
||||
<PackageReference Include="MSTest.TestAdapter" Version="3.2.0" />
|
||||
<PackageReference Include="MSTest.TestFramework" Version="3.2.0" />
|
||||
<PackageReference Include="NLog.Extensions.Logging" Version="5.3.8" />
|
||||
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
|
||||
</ItemGroup>
|
||||
|
|
@ -110,6 +110,12 @@
|
|||
<None Update="TestData\TestDataArchives\100trees.7z">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\DmgSampleFolderCompressed.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\DmgSampleFolderReadOnly.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\Empty.vmdk">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
|
|
@ -122,9 +128,24 @@
|
|||
<None Update="TestData\TestDataArchives\EncryptedWithPlainNames.rar4">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\HfsSample.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\HfsSampleUDCO.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\HfsSampleUlmo.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\Lorem.txt">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\Shared.dmg">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\SharedDmg.zip">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\sysvbanner_1.0-17fakesync1_amd64.deb">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
|
|
@ -176,6 +197,9 @@
|
|||
<None Update="TestData\TestDataArchives\TestDataCorrupt.tar.zip">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\TestDataCorruptWim.zip">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
<None Update="TestData\TestDataArchives\TestDataEncrypted.7z">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
|
|
|
|||
Двоичный файл не отображается.
Двоичные данные
RecursiveExtractor.Tests/TestData/TestDataArchives/TestDataCorruptWim.zip
Normal file
Двоичные данные
RecursiveExtractor.Tests/TestData/TestDataArchives/TestDataCorruptWim.zip
Normal file
Двоичный файл не отображается.
|
|
@ -1,7 +1,7 @@
|
|||
|
||||
Microsoft Visual Studio Solution File, Format Version 12.00
|
||||
# Visual Studio Version 16
|
||||
VisualStudioVersion = 16.0.30309.148
|
||||
# Visual Studio Version 17
|
||||
VisualStudioVersion = 17.8.34408.163
|
||||
MinimumVisualStudioVersion = 10.0.40219.1
|
||||
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "RecursiveExtractor", "RecursiveExtractor\RecursiveExtractor.csproj", "{A7F7492B-60E0-468C-B267-BA60EC131E86}"
|
||||
EndProject
|
||||
|
|
@ -15,9 +15,9 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
|
|||
EndProject
|
||||
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "RecursiveExtractor.Blazor", "RecursiveExtractor.Blazor\RecursiveExtractor.Blazor.csproj", "{18D0803C-052E-4338-9162-F2DB8F8E51E2}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RecursiveExtractor.Cli", "RecursiveExtractor.Cli\RecursiveExtractor.Cli.csproj", "{443B4E50-9AAF-436E-B3DF-644F782AF9B6}"
|
||||
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "RecursiveExtractor.Cli", "RecursiveExtractor.Cli\RecursiveExtractor.Cli.csproj", "{443B4E50-9AAF-436E-B3DF-644F782AF9B6}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "RecursiveExtractor.Cli.Tests", "RecursiveExtractor.Cli.Tests\RecursiveExtractor.Cli.Tests.csproj", "{F37B314B-F641-4336-BCD6-BC5B85BEC5DB}"
|
||||
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "RecursiveExtractor.Cli.Tests", "RecursiveExtractor.Cli.Tests\RecursiveExtractor.Cli.Tests.csproj", "{F37B314B-F641-4336-BCD6-BC5B85BEC5DB}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
|
|
|
|||
|
|
@ -61,6 +61,7 @@ namespace Microsoft.CST.RecursiveExtractor
|
|||
SetExtractor(ArchiveFileType.VMDK, new VmdkExtractor(this));
|
||||
SetExtractor(ArchiveFileType.XZ, new XzExtractor(this));
|
||||
SetExtractor(ArchiveFileType.ZIP, new ZipExtractor(this));
|
||||
SetExtractor(ArchiveFileType.DMG, new DmgExtractor(this));
|
||||
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
|
||||
{
|
||||
SetExtractor(ArchiveFileType.WIM, new WimExtractor(this));
|
||||
|
|
|
|||
|
|
@ -1,10 +1,9 @@
|
|||
using DiscUtils;
|
||||
using System;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Collections.Generic;
|
||||
using System.Collections.ObjectModel;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Microsoft.CST.RecursiveExtractor.Extractors
|
||||
{
|
||||
|
|
@ -28,7 +27,7 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
/// <returns></returns>
|
||||
public static async IAsyncEnumerable<FileEntry> DumpLogicalVolumeAsync(LogicalVolumeInfo volume, string parentPath, ExtractorOptions options, ResourceGovernor governor, Extractor Context, FileEntry? parent = null, bool topLevel = true)
|
||||
{
|
||||
DiscUtils.FileSystemInfo[]? fsInfos = null;
|
||||
ReadOnlyCollection<DiscUtils.FileSystemInfo>? fsInfos = null;
|
||||
try
|
||||
{
|
||||
fsInfos = FileSystemManager.DetectFileSystems(volume);
|
||||
|
|
@ -38,7 +37,7 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
Logger.Debug("Failed to get file systems from logical volume {0} Image {1} ({2}:{3})", volume.Identity, parentPath, e.GetType(), e.Message);
|
||||
}
|
||||
|
||||
foreach (var fsInfo in fsInfos ?? Array.Empty<DiscUtils.FileSystemInfo>())
|
||||
foreach (var fsInfo in fsInfos ?? Enumerable.Empty<DiscUtils.FileSystemInfo>())
|
||||
{
|
||||
using var fs = fsInfo.Open(volume);
|
||||
var diskFiles = fs.GetFiles(fs.Root.FullName, "*.*", SearchOption.AllDirectories).ToList();
|
||||
|
|
@ -90,7 +89,7 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
/// <returns>An enumerable of the contained File Entries.</returns>
|
||||
public static IEnumerable<FileEntry> DumpLogicalVolume(LogicalVolumeInfo volume, string parentPath, ExtractorOptions options, ResourceGovernor governor, Extractor Context, FileEntry? parent = null, bool topLevel = true)
|
||||
{
|
||||
DiscUtils.FileSystemInfo[]? fsInfos = null;
|
||||
ReadOnlyCollection<DiscUtils.FileSystemInfo>? fsInfos = null;
|
||||
try
|
||||
{
|
||||
fsInfos = FileSystemManager.DetectFileSystems(volume);
|
||||
|
|
@ -100,7 +99,7 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
Logger.Debug("Failed to get file systems from logical volume {0} Image {1} ({2}:{3})", volume.Identity, parentPath, e.GetType(), e.Message);
|
||||
}
|
||||
|
||||
foreach (var fsInfo in fsInfos ?? Array.Empty<DiscUtils.FileSystemInfo>())
|
||||
foreach (var fsInfo in fsInfos ?? Enumerable.Empty<DiscUtils.FileSystemInfo>())
|
||||
{
|
||||
using var fs = fsInfo.Open(volume);
|
||||
var diskFiles = fs.GetFiles(fs.Root.FullName, "*.*", SearchOption.AllDirectories).ToList();
|
||||
|
|
|
|||
|
|
@ -0,0 +1,107 @@
|
|||
using DiscUtils;
|
||||
using DiscUtils.Dmg;
|
||||
using DiscUtils.Streams;
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
|
||||
namespace Microsoft.CST.RecursiveExtractor.Extractors
|
||||
{
|
||||
/// <summary>
|
||||
/// The DMG image extractor implementation.
|
||||
/// </summary>
|
||||
public class DmgExtractor : AsyncExtractorInterface
|
||||
{
|
||||
/// <summary>
|
||||
/// The constructor takes the Extractor context for recursion.
|
||||
/// </summary>
|
||||
/// <param name="context">The Extractor context.</param>
|
||||
public DmgExtractor(Extractor context)
|
||||
{
|
||||
Context = context;
|
||||
}
|
||||
private readonly NLog.Logger Logger = NLog.LogManager.GetCurrentClassLogger();
|
||||
|
||||
internal Extractor Context { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Extracts a DMG file
|
||||
/// </summary>
|
||||
///<inheritdoc />
|
||||
public async IAsyncEnumerable<FileEntry> ExtractAsync(FileEntry fileEntry, ExtractorOptions options, ResourceGovernor governor, bool topLevel = true)
|
||||
{
|
||||
LogicalVolumeInfo[]? logicalVolumes = null;
|
||||
Disk? disk = null;
|
||||
|
||||
try
|
||||
{
|
||||
disk = new Disk(fileEntry.Content, Ownership.None);
|
||||
var manager = new VolumeManager(disk);
|
||||
logicalVolumes = manager.GetLogicalVolumes();
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
Logger.Debug("Error reading {0} disk at {1} ({2}:{3})", fileEntry.ArchiveType, fileEntry.FullPath, e.GetType(), e.Message);
|
||||
}
|
||||
if (logicalVolumes != null)
|
||||
{
|
||||
foreach (var volume in logicalVolumes)
|
||||
{
|
||||
await foreach (var entry in DiscCommon.DumpLogicalVolumeAsync(volume, fileEntry.FullPath, options, governor, Context, fileEntry, topLevel))
|
||||
{
|
||||
yield return entry;
|
||||
}
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
if (options.ExtractSelfOnFail)
|
||||
{
|
||||
fileEntry.EntryStatus = FileEntryStatus.FailedArchive;
|
||||
yield return fileEntry;
|
||||
}
|
||||
}
|
||||
disk?.Dispose();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Extracts a DMG file
|
||||
/// </summary>
|
||||
///<inheritdoc />
|
||||
public IEnumerable<FileEntry> Extract(FileEntry fileEntry, ExtractorOptions options, ResourceGovernor governor, bool topLevel = true)
|
||||
{
|
||||
LogicalVolumeInfo[]? logicalVolumes = null;
|
||||
Disk? disk = null;
|
||||
|
||||
try
|
||||
{
|
||||
disk = new Disk(fileEntry.Content, Ownership.None);
|
||||
var manager = new VolumeManager(disk);
|
||||
logicalVolumes = manager.GetLogicalVolumes();
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
Logger.Debug("Error reading {0} disk at {1} ({2}:{3})", fileEntry.ArchiveType, fileEntry.FullPath, e.GetType(), e.Message);
|
||||
}
|
||||
|
||||
if (logicalVolumes != null)
|
||||
{
|
||||
foreach (var volume in logicalVolumes)
|
||||
{
|
||||
foreach (var entry in DiscCommon.DumpLogicalVolume(volume, fileEntry.FullPath, options, governor, Context, fileEntry, topLevel))
|
||||
{
|
||||
yield return entry;
|
||||
}
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
if (options.ExtractSelfOnFail)
|
||||
{
|
||||
fileEntry.EntryStatus = FileEntryStatus.FailedArchive;
|
||||
yield return fileEntry;
|
||||
}
|
||||
}
|
||||
disk?.Dispose();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,9 +1,6 @@
|
|||
using DiscUtils;
|
||||
using System;
|
||||
using System.Collections.Concurrent;
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
|
||||
namespace Microsoft.CST.RecursiveExtractor.Extractors
|
||||
{
|
||||
|
|
@ -101,14 +98,19 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
Logger.Debug(e, "Failed to init WIM image.");
|
||||
Logger.Debug(e, "Failed to init WIM image from {0}.", fileEntry.FullPath);
|
||||
}
|
||||
if (baseFile != null)
|
||||
{
|
||||
for (var i = 0; i < baseFile.ImageCount; i++)
|
||||
{
|
||||
var image = baseFile.GetImage(i);
|
||||
foreach (var file in image.GetFiles(image.Root.FullName, "*.*", SearchOption.AllDirectories))
|
||||
if (!TryGetImage(baseFile, i, out var image))
|
||||
{
|
||||
Logger.Debug("Error reading image {0} from WIM {1}. Potentially malformed?", i, fileEntry.FullPath);
|
||||
continue;
|
||||
}
|
||||
|
||||
foreach (var file in image!.GetFiles(image.Root.FullName, "*.*", SearchOption.AllDirectories))
|
||||
{
|
||||
Stream? stream = null;
|
||||
try
|
||||
|
|
@ -119,7 +121,7 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
Logger.Debug("Error reading {0} from WIM {1} ({2}:{3})", file, image.FriendlyName, e.GetType(), e.Message);
|
||||
Logger.Debug("Error reading {0} from WIM image {1} in {2} ({3}:{4})", file, i, fileEntry.FullPath, e.GetType(), e.Message);
|
||||
}
|
||||
if (stream != null)
|
||||
{
|
||||
|
|
@ -144,12 +146,29 @@ namespace Microsoft.CST.RecursiveExtractor.Extractors
|
|||
}
|
||||
else
|
||||
{
|
||||
fileEntry.EntryStatus = FileEntryStatus.FailedArchive;
|
||||
if (options.ExtractSelfOnFail)
|
||||
{
|
||||
fileEntry.EntryStatus = FileEntryStatus.FailedArchive;
|
||||
yield return fileEntry;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private bool TryGetImage(DiscUtils.Wim.WimFile wimFile, int index, out DiscUtils.Wim.WimFileSystem? image)
|
||||
{
|
||||
image = null;
|
||||
|
||||
try
|
||||
{
|
||||
image = wimFile.GetImage(index);
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
// Image may be corrupt or invalid
|
||||
Logger.Debug(e, "Failed to retrieve WIM image with index {index}.", index);
|
||||
}
|
||||
|
||||
return image is not null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -81,6 +81,10 @@ namespace Microsoft.CST.RecursiveExtractor
|
|||
/// </summary>
|
||||
VMDK,
|
||||
/// <summary>
|
||||
/// A DMG disc image. <see cref="Extractors.DmgExtractor"/>
|
||||
/// </summary>
|
||||
DMG,
|
||||
/// <summary>
|
||||
/// Unused.
|
||||
/// </summary>
|
||||
INVALID
|
||||
|
|
@ -117,6 +121,22 @@ namespace Microsoft.CST.RecursiveExtractor
|
|||
}
|
||||
var initialPosition = fileStream.Position;
|
||||
var buffer = new byte[9];
|
||||
// DMG format uses the magic value 'koly' at the start of the 512 byte footer at the end of the file
|
||||
// Due to compression used, needs to be first or can be misidentified as other formats
|
||||
// https://newosxbook.com/DMG.html
|
||||
if (fileStream.Length > 512)
|
||||
{
|
||||
var dmgFooterMagic = new byte[] { 0x6b, 0x6f, 0x6c, 0x79 };
|
||||
fileStream.Position = fileStream.Length - 0x200; // Footer position
|
||||
fileStream.Read(buffer, 0, 4);
|
||||
fileStream.Position = initialPosition;
|
||||
|
||||
if (dmgFooterMagic.SequenceEqual(buffer[0..4]))
|
||||
{
|
||||
return ArchiveFileType.DMG;
|
||||
}
|
||||
}
|
||||
|
||||
if (fileStream.Length >= 9)
|
||||
{
|
||||
fileStream.Position = 0;
|
||||
|
|
|
|||
|
|
@ -25,24 +25,25 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="DiscUtils.Btrfs" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Core" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Ext" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Fat" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.HfsPlus" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Iso9660" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Ntfs" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Udf" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Vhd" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Vhdx" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Vmdk" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Wim" Version="0.16.13" />
|
||||
<PackageReference Include="DiscUtils.Xfs" Version="0.16.13" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Btrfs" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Core" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Dmg" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Ext" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Fat" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.HfsPlus" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Iso9660" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Ntfs" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Udf" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Vhd" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Vhdx" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Vmdk" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Wim" Version="1.0.36" />
|
||||
<PackageReference Include="LTRData.DiscUtils.Xfs" Version="1.0.36" />
|
||||
<PackageReference Include="Glob" Version="1.1.9" />
|
||||
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" />
|
||||
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All" />
|
||||
<PackageReference Include="NLog" Version="5.2.8" />
|
||||
<PackageReference Include="SharpCompress" Version="0.35.0" />
|
||||
<PackageReference Include="SharpCompress" Version="0.36.0" />
|
||||
<PackageReference Include="SharpZipLib" Version="1.4.2" />
|
||||
<PackageReference Include="System.Linq.Async" Version="6.0.1" />
|
||||
</ItemGroup>
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче