Added TenGB option + download links
This commit is contained in:
Родитель
4e59d50295
Коммит
414bf569e1
1
Enums.cs
1
Enums.cs
|
@ -14,6 +14,7 @@ namespace BenchmarkLogGenerator
|
|||
public enum BenchmarkDataSize
|
||||
{
|
||||
OneGB,
|
||||
TenGB,
|
||||
OneTB,
|
||||
HundredTB
|
||||
}
|
||||
|
|
10
Program.cs
10
Program.cs
|
@ -106,6 +106,8 @@ namespace BenchmarkLogGenerator
|
|||
{
|
||||
case BenchmarkDataSize.OneGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.TenGB:
|
||||
return 2;
|
||||
case BenchmarkDataSize.OneTB:
|
||||
return 6;
|
||||
case BenchmarkDataSize.HundredTB:
|
||||
|
@ -122,6 +124,8 @@ namespace BenchmarkLogGenerator
|
|||
{
|
||||
case BenchmarkDataSize.OneGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.TenGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.OneTB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.HundredTB:
|
||||
|
@ -155,6 +159,8 @@ namespace BenchmarkLogGenerator
|
|||
{
|
||||
case BenchmarkDataSize.OneGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.TenGB:
|
||||
return 5;
|
||||
case BenchmarkDataSize.OneTB:
|
||||
return 100;
|
||||
case BenchmarkDataSize.HundredTB:
|
||||
|
@ -170,6 +176,8 @@ namespace BenchmarkLogGenerator
|
|||
{
|
||||
case BenchmarkDataSize.OneGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.TenGB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.OneTB:
|
||||
return 1;
|
||||
case BenchmarkDataSize.HundredTB:
|
||||
|
@ -190,7 +198,7 @@ namespace BenchmarkLogGenerator
|
|||
esb.AppendLine("-localPath: The root folder");
|
||||
esb.AppendLine("-azureStorageAccountConnections: A comma separated list of Azure storage account connections (can be single connection), containers will be created automaticly using the following template: logsBenchmark-{size}-p{partition}");
|
||||
esb.AppendLine("-eventHubConnection: The connection string for Azure EventHub");
|
||||
esb.AppendLine("-size: The output size, possible values are OneGB, OneTB, HundredTB. Default is OneGB");
|
||||
esb.AppendLine("-size: The output size, possible values are OneGB, TenGB, OneTB, HundredTB. Default is OneGB");
|
||||
esb.AppendLine("-partition: The applicable partition, between -1 to 9, where -1 means single partition. Only relevant for HundredTB size. Default is -1");
|
||||
esb.AppendLine();
|
||||
CommandLineArgsParser.PrintUsage(esb);
|
||||
|
|
15
README.md
15
README.md
|
@ -4,7 +4,7 @@ This tool supports following configurable command line parameters –
|
|||
1. **-output**: This is the location where output should be written to.
|
||||
Supported values are: LocalDisk, AzureStorage, EventHub
|
||||
2. **-size**: The data size to be generated.
|
||||
Supported values are: OneGB, OneTB, HundredTB
|
||||
Supported values are: OneGB, TenGB, OneTB, HundredTB
|
||||
Default value is OneGB
|
||||
3. **-partition**: The value for data partition, it could be between -1 to 9, where -1 means single partition. Default value is -1. It’s only relevant for HundredTB data size.
|
||||
|
||||
|
@ -18,15 +18,24 @@ Default value is OneGB
|
|||
`BenchmarkLogGenerator -output:EventHub -eventHubConnection:Endpoint=sb://EHNAMESPACE.servicebus.windows.net/;EntityPath=EHNAME;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=KEYVALUE -size:OneGB"`
|
||||
|
||||
**Note** – Data size is restricted to 1 GB for event hub.
|
||||
**Note** – We host the generated files with default seed value at https://logsbenchmark00.blob.core.windows.net/logsbenchmark-onegb?sp=rl&st=2022-08-18T00:00:00Z&se=2030-01-01T00:00:00Z&spr=https&sv=2021-06-08&sr=c&sig=5pjOow5An3%2BTs5mZ%2FyosJBPtDvV7%2FXfDO8pLEeeylVc%3D. Please use Azure Storage Explorer and attach a blob container specifying this URL or leverage AzCopy to download it.
|
||||
|
||||
2. **OneTB size**
|
||||
2. **TenGB size**
|
||||
|
||||
`BenchmarkLogGenerator.exe -output:AzureStorage -size:TenGB -cc:BLOB STORAGE CONN STR`
|
||||
|
||||
`BenchmarkLogGenerator -output:LocalDisk -size:TenGB -localPath:"C:\DATA"`
|
||||
|
||||
**Note** – We host the generated files with default seed value at https://logsbenchmark00.blob.core.windows.net/logsbenchmark-tengb?sp=rl&st=2022-08-18T00:00:00Z&se=2030-01-01T00:00:00Z&spr=https&sv=2021-06-08&sr=c&sig=AcZvWrUj9EHWoV6%2BIKeo3dC12f06iq%2Fo42IRI6h4t8o%3D. Please use Azure Storage Explorer and attach a blob container specifying this URL or leverage AzCopy to download it.
|
||||
|
||||
3. **OneTB size**
|
||||
|
||||
`BenchmarkLogGenerator.exe -output:AzureStorage -size:OneTB -cc:BLOB STORAGE CONN STR`
|
||||
`BenchmarkLogGenerator -output:LocalDisk -size:OneTB -localPath:"C:\DATA"`
|
||||
|
||||
You can also use below mentioned Azure batch templates to generate 1 TB data, 3 Standard_D32_v3 VMs would be enough to generate 1 TB .
|
||||
|
||||
3. **HundredTB size**
|
||||
4. **HundredTB size**
|
||||
|
||||
Use Azure Batch compute to generate 100 TBs of data. Tool has Azure batch templates for generating required batch pools and jobs.
|
||||
Follow these steps to generate 100TBs of data -
|
||||
|
|
Загрузка…
Ссылка в новой задаче