microsoft
/
Microsoft-Performance-Tools-Linux-Android
зеркало из https://github.com/microsoft/Microsoft-Performance-Tools-Linux-Android.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Initial Perfetto plugin checkin (#21) 

This PR adds a plugin for displaying generic events from Perfetto traces. Added a new "PerfettoCds" project to the solution. Keeping this in a separate "perfetto" feature branch while I work on it.

Data is gathered from the Perfetto trace through trace_processor_shell.exe. Trace_processor_shell.exe opens an interface that allows for making SQL queries over HTTP/RPC to localhost. All the trace data is retrieved with SQL queries. Data is serialized over the HTTP interface with Protobuf objects. The original TraceProcessor protobuf object (trace_processor.proto) and the C# conversions are included in the project.

PerfettoSourceParser will start the trace_processor_shell.exe process. SQL Queries will be made over HTTP and the protobuf output will be converted to objects of type PerfettoSqlEvent. One query will be made per SQL table. Each query will produce events of the same type and will be processed by their own individual source cooker. For a complete generic event, we need data from 5 tables: slice, args, thread_track, thread, and process. A composite cooker will then take all the data gathered from each individual source cooker and join them to create the final generic event.

For example, for the slice table, the process goes like this:

Perform a SQL query into the Perfetto trace through trace_processor_shell.exe ("select * from slice"). Returns a QueryResult object
Convert QueryResult object into individual PerfettoSliceEvents. One event for each row in the SQL table
PerfettoSliceCooker will process/store all the PerfettoSliceEvents.
Once all the other tables have also finished, PerfettoGenericEventCooker will gather all the events from each cooker and do a join on them to create complete PerfettoGenericEvent objects.
Create WPA table of PerfettoGenericEvents
This commit is contained in:
Kyle Storck 2021-07-08 14:27:27 -07:00 коммит произвёл GitHub
Родитель 3083df85d0
Коммит 396aa28646
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
34 изменённых файлов: 10968 добавлений и 1 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

18
Microsoft-Perf-Tools-Linux.sln
Просмотреть файл

@ -49,6 +49,12 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "LTTngDataExtUnitTest", "LTT
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "PerfUnitTest", "PerfUnitTest\PerfUnitTest.csproj", "{159F637D-6AB1-4E7F-878E-E2C46CF2D920}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "PerfettoCds", "PerfettoCds\PerfettoCds.csproj", "{83418D84-CACE-40E8-A0C3-8EEAF7E71969}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "PerfettoProcessor", "PerfettoProcessor\PerfettoProcessor.csproj", "{90FE5422-631F-4B5A-8EF7-88FE542739DF}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "PerfettoUnitTest", "PerfettoUnitTest\PerfettoUnitTest.csproj", "{CF89184C-8CE8-4656-9C5C-3F6C547EF2A3}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@ -123,6 +129,18 @@ Global
{159F637D-6AB1-4E7F-878E-E2C46CF2D920}.Debug|Any CPU.Build.0 = Debug|Any CPU
{159F637D-6AB1-4E7F-878E-E2C46CF2D920}.Release|Any CPU.ActiveCfg = Release|Any CPU
{159F637D-6AB1-4E7F-878E-E2C46CF2D920}.Release|Any CPU.Build.0 = Release|Any CPU
{83418D84-CACE-40E8-A0C3-8EEAF7E71969}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{83418D84-CACE-40E8-A0C3-8EEAF7E71969}.Debug|Any CPU.Build.0 = Debug|Any CPU
{83418D84-CACE-40E8-A0C3-8EEAF7E71969}.Release|Any CPU.ActiveCfg = Release|Any CPU
{83418D84-CACE-40E8-A0C3-8EEAF7E71969}.Release|Any CPU.Build.0 = Release|Any CPU
{90FE5422-631F-4B5A-8EF7-88FE542739DF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{90FE5422-631F-4B5A-8EF7-88FE542739DF}.Debug|Any CPU.Build.0 = Debug|Any CPU
{90FE5422-631F-4B5A-8EF7-88FE542739DF}.Release|Any CPU.ActiveCfg = Release|Any CPU
{90FE5422-631F-4B5A-8EF7-88FE542739DF}.Release|Any CPU.Build.0 = Release|Any CPU
{CF89184C-8CE8-4656-9C5C-3F6C547EF2A3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CF89184C-8CE8-4656-9C5C-3F6C547EF2A3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CF89184C-8CE8-4656-9C5C-3F6C547EF2A3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CF89184C-8CE8-4656-9C5C-3F6C547EF2A3}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE

227
NOTICE.md
Просмотреть файл

@ -17,4 +17,229 @@ Redistribution and use in source and binary forms, with or without modification,
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
## Google Protobuf
**Source**: https://github.com/protocolbuffers/protobuf
**Nuget**: https://www.nuget.org/packages/google.protobuf
Copyright 2008 Google Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of Google Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Code generated by the Protocol Buffer compiler is owned by the owner
of the input file used when generating it. This code is not
standalone and requires a support library to be linked with it. This
support library is itself covered by the above license.
## Google Perfetto
**Source** : https://github.com/google/perfetto
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright (c) 2017, The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

21
PerfettoCds/PerfettoCds.csproj Normal file
Просмотреть файл

@ -0,0 +1,21 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Google.Protobuf" Version="3.17.3">
<GeneratePathProperty>true</GeneratePathProperty>
</PackageReference>
<PackageReference Include="Microsoft.Performance.SDK" Version="0.109.11-preview-g61bc0d83f6" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\PerfettoProcessor\PerfettoProcessor.csproj" />
</ItemGroup>
<Target Name="CopyRulesToTarget" AfterTargets="Build">
<Copy SourceFiles="$(PkgGoogle_Protobuf)\lib\netstandard2.0\Google.Protobuf.dll" DestinationFolder="$(TargetDir)" />
</Target>
</Project>

123
PerfettoCds/Pipeline/CompositeDataCookers/PerfettoGenericEventCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,123 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Processing;
using PerfettoCds.Pipeline.DataOutput;
using PerfettoProcessor;
namespace PerfettoCds.Pipeline.DataCookers
{
/// <summary>
/// Pulls data from all the individual SQL tables and joins them to create a Generic Peretto event
/// </summary>
public sealed class PerfettoGenericEventCooker : CookedDataReflector, ICompositeDataCookerDescriptor
{
public static readonly DataCookerPath DataCookerPath = PerfettoPluginConstants.GenericEventCookerPath;
public string Description => "Generic Event composite cooker";
public DataCookerPath Path => DataCookerPath;
// Declare all of the cookers that are used by this CompositeCooker.
public IReadOnlyCollection<DataCookerPath> RequiredDataCookers => new[]
{
PerfettoPluginConstants.SliceCookerPath,
PerfettoPluginConstants.ArgCookerPath,
PerfettoPluginConstants.ThreadTrackCookerPath,
PerfettoPluginConstants.ThreadCookerPath,
PerfettoPluginConstants.ProcessCookerPath
};
[DataOutput]
public ProcessedEventData<PerfettoGenericEvent> GenericEvents { get; }
/// <summary>
/// The highest number of fields found in any single event.
/// </summary>
[DataOutput]
public int MaximumEventFieldCount { get; private set; }
public PerfettoGenericEventCooker() : base(PerfettoPluginConstants.GenericEventCookerPath)
{
this.GenericEvents =
new ProcessedEventData<PerfettoGenericEvent>();
}
public void OnDataAvailable(IDataExtensionRetrieval requiredData)
{
// Gather the data from all the SQL tables
var sliceData = requiredData.QueryOutput<ProcessedEventData<PerfettoSliceEvent>>(new DataOutputPath(PerfettoPluginConstants.SliceCookerPath, nameof(PerfettoSliceCooker.SliceEvents)));
var argData = requiredData.QueryOutput<ProcessedEventData<PerfettoArgEvent>>(new DataOutputPath(PerfettoPluginConstants.ArgCookerPath, nameof(PerfettoArgCooker.ArgEvents)));
var threadTrackData = requiredData.QueryOutput<ProcessedEventData<PerfettoThreadTrackEvent>>(new DataOutputPath(PerfettoPluginConstants.ThreadTrackCookerPath, nameof(PerfettoThreadTrackCooker.ThreadTrackEvents)));
var threadData = requiredData.QueryOutput<ProcessedEventData<PerfettoThreadEvent>>(new DataOutputPath(PerfettoPluginConstants.ThreadCookerPath, nameof(PerfettoThreadCooker.ThreadEvents)));
var processData = requiredData.QueryOutput<ProcessedEventData<PerfettoProcessEvent>>(new DataOutputPath(PerfettoPluginConstants.ProcessCookerPath, nameof(PerfettoProcessCooker.ProcessEvents)));
// Join them all together
// Slice data contains event name and a few more fields
// Arg data contains the debug annotations
// ThreadTrack data allows us to get to the thread
// Thread data gives us the thread name+ID and gets us the process
// Process data gives us the process name+ID
var joined = from slice in sliceData
join arg in argData on slice.ArgSetId equals arg.ArgSetId into args
join threadTrack in threadTrackData on slice.TrackId equals threadTrack.Id
join thread in threadData on threadTrack.Utid equals thread.Utid
join process in processData on thread.Upid equals process.Upid
select new { slice, args, threadTrack, thread, process };
// Create events out of the joined results
foreach (var result in joined)
{
MaximumEventFieldCount = Math.Max(MaximumEventFieldCount, result.args.Count());
List<string> argKeys = new List<string>();
List<string> values = new List<string>();
// Each event has multiple of these "debug annotations". They get stored in lists
foreach (var arg in result.args)
{
argKeys.Add(arg.ArgKey);
switch (arg.ValueType)
{
case "string":
values.Add(arg.StringValue);
break;
case "bool":
case "int":
values.Add(arg.IntValue.ToString());
break;
case "uint":
case "pointer":
values.Add(((uint)arg.IntValue).ToString());
break;
case "real":
values.Add(arg.RealValue.ToString());
break;
default:
throw new Exception("Unexpected Perfetto value type");
}
}
PerfettoGenericEvent ev = new PerfettoGenericEvent
(
result.slice.Name,
result.slice.Type,
new TimestampDelta(result.slice.Duration),
new Timestamp(result.slice.Timestamp),
result.slice.Category,
result.slice.ArgSetId,
values,
argKeys,
string.Format($"{result.process.Name} {result.process.Pid}"),
string.Format($"{result.thread.Name} {result.thread.Tid}")
);
this.GenericEvents.AddEvent(ev);
}
this.GenericEvents.FinalizeData();
}
}
}

57
PerfettoCds/Pipeline/DataOutput/PerfettoGenericEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,57 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using System.Collections.Generic;
namespace PerfettoCds.Pipeline.DataOutput
{
/// <summary>
/// A generic app/component event that contains event name, event metadata, and thread+process
/// info.
/// </summary>
public readonly struct PerfettoGenericEvent
{
// From Slice table
public string EventName { get; }
public string Type { get; }
public TimestampDelta Duration { get; }
public Timestamp Timestamp { get; }
public string Category { get; }
// Key between slice and args table
public long ArgSetId { get; }
// From Args table. The debug annotations for an event. Variable number per event
public List<string> Values { get; }
public List<string> ArgKeys { get; }
// From Process table
public string Process { get; }
// From Thread table
public string Thread { get; }
public PerfettoGenericEvent(string eventName,
string type,
TimestampDelta duration,
Timestamp timestamp,
string category,
long argSetId,
List<string> values,
List<string> argKeys,
string process,
string thread)
{
this.EventName = eventName;
this.Type = type;
this.Duration = duration;
this.Timestamp = timestamp;
this.Category = category;
this.ArgSetId = argSetId;
this.Values = values;
this.ArgKeys = argKeys;
this.Process = process;
this.Thread = thread;
}
}
}

36
PerfettoCds/Pipeline/Events/PerfettoSqlEventKeyed.cs Normal file
Просмотреть файл

@ -0,0 +1,36 @@
using Microsoft.Performance.SDK.Extensibility;
using PerfettoProcessor;
using System;
using System.Collections.Generic;
using System.Text;
namespace PerfettoCds.Pipeline.Events
{
/// <summary>
/// This class serves as a holder for PerfettoSqlEvents as they pass through cookers. It stores
/// the PerfettoSqlEvent and the key that identifies its type and which cookers process it
/// </summary>
public class PerfettoSqlEventKeyed : IKeyedDataType<string>
{
public readonly string Key;
// The SQL event being passed on
public PerfettoSqlEvent SqlEvent;
public PerfettoSqlEventKeyed(string key, PerfettoSqlEvent sqlEvent)
{
this.Key = key;
this.SqlEvent = sqlEvent;
}
public string GetKey()
{
return Key;
}
public int CompareTo(string other)
{
return this.Key.CompareTo(other);
}
}
}

31
PerfettoCds/Pipeline/PerfettoDataProcessor.cs Normal file
Просмотреть файл

@ -0,0 +1,31 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.SourceParsing;
using Microsoft.Performance.SDK.Processing;
using System;
using System.Collections.Generic;
using PerfettoCds.Pipeline.Events;
namespace PerfettoCds
{
/// <summary>
/// This class only delegates work off to the source parser, so there's no logic inside of it.
/// <para/>
/// Since our table has required data cookers, the SDK takes care of making sure it
/// gets built.
/// </summary>
public class PerfettoDataProcessor : CustomDataProcessorBaseWithSourceParser<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
internal PerfettoDataProcessor(
ISourceParser<PerfettoSqlEventKeyed, PerfettoSourceParser, string> sourceParser,
ProcessorOptions options,
IApplicationEnvironment applicationEnvironment,
IProcessorEnvironment processorEnvironment,
IReadOnlyDictionary<TableDescriptor, Action<ITableBuilder, IDataExtensionRetrieval>> allTablesMapping,
IEnumerable<TableDescriptor> metadataTables)
: base(sourceParser, options, applicationEnvironment, processorEnvironment, allTablesMapping, metadataTables)
{
}
}
}

134
PerfettoCds/Pipeline/PerfettoDataSource.cs Normal file
Просмотреть файл

@ -0,0 +1,134 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK.Processing;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
namespace PerfettoCds
{
[CustomDataSource("9fc8515e-9206-4690-b14a-3e7b54745c5f", "Perfetto DataSource", "Processes Perfetto trace files")]
[FileDataSource(".perfetto-trace", "Perfetto trace files")]
public sealed class PerfettoDataSource : CustomDataSourceBase
{
private IApplicationEnvironment applicationEnvironment;
protected override ICustomDataProcessor CreateProcessorCore(IEnumerable<IDataSource> dataSources, IProcessorEnvironment processorEnvironment, ProcessorOptions options)
{
var filePath = dataSources.First().Uri.LocalPath;
var parser = new PerfettoSourceParser(filePath);
return new PerfettoDataProcessor(parser,
options,
this.applicationEnvironment,
processorEnvironment,
this.AllTables,
this.MetadataTables);
}
public override CustomDataSourceInfo GetAboutInfo()
{
return new CustomDataSourceInfo()
{
ProjectInfo = new ProjectInfo() { Uri = "https://aka.ms/linuxperftools" },
CopyrightNotice = "Copyright (C) " + DateTime.UtcNow.Year,
AdditionalInformation = new[]
{
"Built using Google Perfetto 4\n" +
"Copyright (C) 2020 The Android Open Source Project\n" +
"\n" +
"Licensed under the Apache License, Version 2.0 (the \"License\");\n" +
"you may not use this file except in compliance with the License.\n" +
"You may obtain a copy of the License at\n" +
"\n" +
"http://www.apache.org/licenses/LICENSE-2.0\n" +
"\n" +
"Unless required by applicable law or agreed to in writing, software\n" +
"distributed under the License is distributed on an \"AS IS\" BASIS,\n" +
"WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n" +
"See the License for the specific language governing permissions and\n" +
"limitations under the License.\n"
},
};
}
protected override bool IsDataSourceSupportedCore(IDataSource dataSource)
{
if (dataSource.IsDirectory())
{
return false;
}
var ext = Path.GetExtension(dataSource.Uri.LocalPath);
return dataSource.IsFile() && StringComparer.OrdinalIgnoreCase.Equals(".perfetto-trace", ext);
}
protected override void SetApplicationEnvironmentCore(IApplicationEnvironment applicationEnvironment)
{
this.applicationEnvironment = applicationEnvironment;
}
}
[CustomDataSource("99e4223a-6211-4ce7-a0da-917a893797f2", "Perfetto DataSource", "Processes Perfetto trace files")]
[FileDataSource(".pftrace", "Perfetto trace files")]
public sealed class PfDataSource : CustomDataSourceBase
{
private IApplicationEnvironment applicationEnvironment;
protected override ICustomDataProcessor CreateProcessorCore(IEnumerable<IDataSource> dataSources, IProcessorEnvironment processorEnvironment, ProcessorOptions options)
{
var filePath = dataSources.First().Uri.LocalPath;
var parser = new PerfettoSourceParser(filePath);
return new PerfettoDataProcessor(parser,
options,
this.applicationEnvironment,
processorEnvironment,
this.AllTables,
this.MetadataTables);
}
public override CustomDataSourceInfo GetAboutInfo()
{
return new CustomDataSourceInfo()
{
ProjectInfo = new ProjectInfo() { Uri = "https://aka.ms/linuxperftools" },
CopyrightNotice = "Copyright (C) " + DateTime.UtcNow.Year,
AdditionalInformation = new[]
{
"Built using Google Perfetto 4\n" +
"Copyright (C) 2020 The Android Open Source Project\n" +
"\n" +
"Licensed under the Apache License, Version 2.0 (the \"License\");\n" +
"you may not use this file except in compliance with the License.\n" +
"You may obtain a copy of the License at\n" +
"\n" +
"http://www.apache.org/licenses/LICENSE-2.0\n" +
"\n" +
"Unless required by applicable law or agreed to in writing, software\n" +
"distributed under the License is distributed on an \"AS IS\" BASIS,\n" +
"WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n" +
"See the License for the specific language governing permissions and\n" +
"limitations under the License.\n"
},
};
}
protected override bool IsDataSourceSupportedCore(IDataSource dataSource)
{
if (dataSource.IsDirectory())
{
return false;
}
var ext = Path.GetExtension(dataSource.Uri.LocalPath);
return dataSource.IsFile() && StringComparer.OrdinalIgnoreCase.Equals(".pftrace", ext);
}
protected override void SetApplicationEnvironmentCore(IApplicationEnvironment applicationEnvironment)
{
this.applicationEnvironment = applicationEnvironment;
}
}
}

53
PerfettoCds/Pipeline/PerfettoPluginConstants.cs Normal file
Просмотреть файл

@ -0,0 +1,53 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK.Extensibility;
using PerfettoProcessor;
namespace PerfettoCds
{
public static class PerfettoPluginConstants
{
public const string TraceProcessorShellFileName = @"trace_processor_shell.exe";
// ID for source parser
public const string ParserId = "PerfettoSourceParser";
// ID for source data cookers
public const string SliceCookerId = "PerfettoSliceCooker";
public const string ArgCookerId = "PerfettoArgCooker";
public const string ThreadCookerId = "PerfettoThreadCooker";
public const string ThreadTrackCookerId = "PerfettoThreadCookerId";
public const string ProcessCookerId = "PerfettoProcessCooker";
// ID for composite data cookers
public const string GenericEventCookerId = "PerfettoGenericEventCooker";
// Events for source cookers
public const string SliceEvent = PerfettoSliceEvent.Key;
public const string ArgEvent = PerfettoArgEvent.Key;
public const string ThreadTrackEvent = PerfettoThreadTrackEvent.Key;
public const string ThreadEvent = PerfettoThreadEvent.Key;
public const string ProcessEvent = PerfettoProcessEvent.Key;
// Output events for composite cookers
public const string GenericEvent = "PerfettoGenericEvent";
// Path from source parser to example data cooker. This is the path
// that is used to programatically access the data cooker's data outputs,
// and can be created by external binaries by just knowing the
// parser and cooker IDs defined above
public static readonly DataCookerPath SliceCookerPath =
new DataCookerPath(PerfettoPluginConstants.ParserId, PerfettoPluginConstants.SliceCookerId);
public static readonly DataCookerPath ArgCookerPath =
new DataCookerPath(PerfettoPluginConstants.ParserId, PerfettoPluginConstants.ArgCookerId);
public static readonly DataCookerPath ThreadTrackCookerPath =
new DataCookerPath(PerfettoPluginConstants.ParserId, PerfettoPluginConstants.ThreadTrackCookerId);
public static readonly DataCookerPath ThreadCookerPath =
new DataCookerPath(PerfettoPluginConstants.ParserId, PerfettoPluginConstants.ThreadCookerId);
public static readonly DataCookerPath ProcessCookerPath =
new DataCookerPath(PerfettoPluginConstants.ParserId, PerfettoPluginConstants.ProcessCookerId);
public static readonly DataCookerPath GenericEventCookerPath =
new DataCookerPath(PerfettoPluginConstants.GenericEventCookerId);
}
}

149
PerfettoCds/Pipeline/PerfettoSourceParser.cs Normal file
Просмотреть файл

@ -0,0 +1,149 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK.Extensibility.SourceParsing;
using Microsoft.Performance.SDK.Processing;
using System;
using System.Collections.Generic;
using System.Threading;
using Microsoft.Performance.SDK;
using PerfettoProcessor;
using PerfettoCds.Pipeline.Events;
using System.IO;
using System.Reflection;
namespace PerfettoCds
{
public sealed class PerfettoSourceParser : ISourceParser<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public string Id => PerfettoPluginConstants.ParserId;
public Type DataElementType => typeof(PerfettoSqlEvent);
public Type DataContextType => typeof(PerfettoSourceParser);
public Type DataKeyType => typeof(string);
public int MaxSourceParseCount => 1;
// Information about this data source the SDK requires for building tables
private DataSourceInfo dataSourceInfo { get; set; }
public DataSourceInfo DataSourceInfo => this.dataSourceInfo;
// For UI progress reporting
private IProgress<int> Progress;
private double CurrentProgress;
/// <summary>
/// Increase the progress percentage by a fixed percent
/// </summary>
/// <param name="percent">Percent to increase the loading progress (0-100) </param>
private void IncreaseProgress(double percent)
{
if (Progress != null)
{
CurrentProgress += percent;
Progress.Report((int)CurrentProgress);
}
}
// Perfetto trace file (.perfetto-trace) being processed
private readonly string filePath;
public PerfettoSourceParser(string filePath)
{
this.filePath = filePath;
}
public void PrepareForProcessing(bool allEventsConsumed, IReadOnlyCollection<string> requestedDataKeys)
{
// No preperation needed
}
public void ProcessSource(ISourceDataProcessor<PerfettoSqlEventKeyed, PerfettoSourceParser, string> dataProcessor,
ILogger logger,
IProgress<int> progress,
CancellationToken cancellationToken)
{
this.Progress = progress;
PerfettoTraceProcessor traceProc = new PerfettoTraceProcessor();
Timestamp? traceStartTime = null;
Timestamp? traceEndTime = null;
try
{
// Start the progress counter to indicate something is happening because
// OpenTraceProcessor could take a few seconds
IncreaseProgress(1);
// Shell .exe should be located in same directory as this assembly.
var shellDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var shellPath = Path.Combine(shellDir, PerfettoPluginConstants.TraceProcessorShellFileName);
// Start the Perfetto trace processor shell with the trace file
traceProc.OpenTraceProcessor(shellPath, filePath);
// Use this callback to receive events parsed and turn them in events with keys
// that get used by data cookers
void EventCallback(PerfettoSqlEvent ev)
{
if (ev.GetType() == typeof(PerfettoSliceEvent))
{
// We get the timestamps used for displaying these events from the slice event
if (traceStartTime == null)
{
traceStartTime = new Timestamp(((PerfettoSliceEvent)ev).Timestamp);
}
traceEndTime = new Timestamp(((PerfettoSliceEvent)ev).Timestamp);
}
PerfettoSqlEventKeyed newEvent = new PerfettoSqlEventKeyed(ev.GetEventKey(), ev);
// Store the event
var result = dataProcessor.ProcessDataElement(newEvent, this, cancellationToken);
}
// Perform queries for the events we need
List<PerfettoSqlEvent> eventsToQuery = new List<PerfettoSqlEvent>
{
new PerfettoSliceEvent(),
new PerfettoArgEvent(),
new PerfettoThreadTrackEvent(),
new PerfettoThreadEvent(),
new PerfettoProcessEvent()
};
// Increment progress for each table queried.
double queryProgressIncrease = 99.0 / eventsToQuery.Count;
foreach (var query in eventsToQuery)
{
logger.Verbose($"Querying for {query.GetEventKey()} using SQL query: {query.GetSqlQuery()}");
// Run the query and process the events.
var dateTimeQueryStarted = DateTime.UtcNow;
traceProc.QueryTraceForEvents(query.GetSqlQuery(), query.GetEventKey(), EventCallback);
var dateTimeQueryFinished = DateTime.UtcNow;
logger.Verbose($"Query for {query.GetEventKey()} completed in {(dateTimeQueryFinished - dateTimeQueryStarted).TotalSeconds}s at {dateTimeQueryFinished} UTC");
IncreaseProgress(queryProgressIncrease);
}
// Done with the SQL trace processor
traceProc.CloseTraceConnection();
if (traceStartTime.HasValue)
{
// Use DateTime.Now as the wall clock time. This doesn't matter for displaying events on a relative timescale
// TODO Actual wall clock time needs to be gathered from SQL somehow
this.dataSourceInfo = new DataSourceInfo(traceStartTime.Value.ToNanoseconds, traceEndTime.Value.ToNanoseconds, DateTime.Now.ToUniversalTime());
}
}
catch (Exception e)
{
logger.Error($"Error while processing Perfetto trace: {e.Message}");
traceProc.CloseTraceConnection();
}
}
}
}

51
PerfettoCds/Pipeline/SourceDataCookers/PerfettoArgCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,51 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Extensibility.DataCooking.SourceDataCooking;
using Microsoft.Performance.SDK.Processing;
using System.Collections.Generic;
using System.Threading;
using PerfettoCds.Pipeline.Events;
using PerfettoProcessor;
namespace PerfettoCds
{
/// <summary>
/// Cooks the data from the Args table in Perfetto traces
/// </summary>
public sealed class PerfettoArgCooker : BaseSourceDataCooker<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public override string Description => "Processes events from the args Perfetto SQL table";
//
// The data this cooker outputs. Tables or other cookers can query for this data
// via the SDK runtime
//
[DataOutput]
public ProcessedEventData<PerfettoArgEvent> ArgEvents { get; }
// Instructs runtime to only send events with the given keys this data cooker
public override ReadOnlyHashSet<string> DataKeys =>
new ReadOnlyHashSet<string>(new HashSet<string> { PerfettoPluginConstants.ArgEvent });
public PerfettoArgCooker() : base(PerfettoPluginConstants.ArgCookerPath)
{
this.ArgEvents = new ProcessedEventData<PerfettoArgEvent>();
}
public override DataProcessingResult CookDataElement(PerfettoSqlEventKeyed perfettoEvent, PerfettoSourceParser context, CancellationToken cancellationToken)
{
this.ArgEvents.AddEvent((PerfettoArgEvent)perfettoEvent.SqlEvent);
return DataProcessingResult.Processed;
}
public override void EndDataCooking(CancellationToken cancellationToken)
{
base.EndDataCooking(cancellationToken);
this.ArgEvents.FinalizeData();
}
}
}

52
PerfettoCds/Pipeline/SourceDataCookers/PerfettoProcessCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,52 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Extensibility.DataCooking.SourceDataCooking;
using Microsoft.Performance.SDK.Processing;
using System.Collections.Generic;
using System.Threading;
using PerfettoCds.Pipeline.Events;
using PerfettoProcessor;
namespace PerfettoCds
{
/// <summary>
/// Cooks the data from the Process table in Perfetto traces
/// </summary>
public sealed class PerfettoProcessCooker : BaseSourceDataCooker<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public override string Description => "Processes events from the process Perfetto SQL table";
//
// The data this cooker outputs. Tables or other cookers can query for this data
// via the SDK runtime
//
[DataOutput]
public ProcessedEventData<PerfettoProcessEvent> ProcessEvents { get; }
// Instructs runtime to only send events with the given keys this data cooker
public override ReadOnlyHashSet<string> DataKeys =>
new ReadOnlyHashSet<string>(new HashSet<string> { PerfettoPluginConstants.ProcessEvent });
public PerfettoProcessCooker() : base(PerfettoPluginConstants.ProcessCookerPath)
{
this.ProcessEvents = new ProcessedEventData<PerfettoProcessEvent>();
}
public override DataProcessingResult CookDataElement(PerfettoSqlEventKeyed perfettoEvent, PerfettoSourceParser context, CancellationToken cancellationToken)
{
this.ProcessEvents.AddEvent((PerfettoProcessEvent)perfettoEvent.SqlEvent);
return DataProcessingResult.Processed;
}
public override void EndDataCooking(CancellationToken cancellationToken)
{
base.EndDataCooking(cancellationToken);
this.ProcessEvents.FinalizeData();
}
}
}

52
PerfettoCds/Pipeline/SourceDataCookers/PerfettoSliceCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,52 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Extensibility.DataCooking.SourceDataCooking;
using Microsoft.Performance.SDK.Processing;
using System.Collections.Generic;
using System.Threading;
using PerfettoCds.Pipeline.Events;
using PerfettoProcessor;
namespace PerfettoCds
{
/// <summary>
/// Cooks the data from the Slice table in Perfetto traces
/// </summary>
public sealed class PerfettoSliceCooker : BaseSourceDataCooker<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public override string Description => "Processes events from the slice Perfetto SQL table";
//
// The data this cooker outputs. Tables or other cookers can query for this data
// via the SDK runtime
//
[DataOutput]
public ProcessedEventData<PerfettoSliceEvent> SliceEvents { get; }
// Instructs runtime to only send events with the given keys this data cooker
public override ReadOnlyHashSet<string> DataKeys =>
new ReadOnlyHashSet<string>(new HashSet<string> { PerfettoPluginConstants.SliceEvent });
public PerfettoSliceCooker() : base(PerfettoPluginConstants.SliceCookerPath)
{
this.SliceEvents = new ProcessedEventData<PerfettoSliceEvent>();
}
public override DataProcessingResult CookDataElement(PerfettoSqlEventKeyed perfettoEvent, PerfettoSourceParser context, CancellationToken cancellationToken)
{
this.SliceEvents.AddEvent((PerfettoSliceEvent)perfettoEvent.SqlEvent);
return DataProcessingResult.Processed;
}
public override void EndDataCooking(CancellationToken cancellationToken)
{
base.EndDataCooking(cancellationToken);
this.SliceEvents.FinalizeData();
}
}
}

52
PerfettoCds/Pipeline/SourceDataCookers/PerfettoThreadCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,52 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Extensibility.DataCooking.SourceDataCooking;
using Microsoft.Performance.SDK.Processing;
using System.Collections.Generic;
using System.Threading;
using PerfettoCds.Pipeline.Events;
using PerfettoProcessor;
namespace PerfettoCds
{
/// <summary>
/// Cooks the data from the Thread table in Perfetto traces
/// </summary>
public sealed class PerfettoThreadCooker : BaseSourceDataCooker<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public override string Description => "Processes events from the thread Perfetto SQL table";
//
// The data this cooker outputs. Tables or other cookers can query for this data
// via the SDK runtime
//
[DataOutput]
public ProcessedEventData<PerfettoThreadEvent> ThreadEvents { get; }
// Instructs runtime to only send events with the given keys this data cooker
public override ReadOnlyHashSet<string> DataKeys =>
new ReadOnlyHashSet<string>(new HashSet<string> { PerfettoPluginConstants.ThreadEvent });
public PerfettoThreadCooker() : base(PerfettoPluginConstants.ThreadCookerPath)
{
this.ThreadEvents = new ProcessedEventData<PerfettoThreadEvent>();
}
public override DataProcessingResult CookDataElement(PerfettoSqlEventKeyed perfettoEvent, PerfettoSourceParser context, CancellationToken cancellationToken)
{
this.ThreadEvents.AddEvent((PerfettoThreadEvent)perfettoEvent.SqlEvent);
return DataProcessingResult.Processed;
}
public override void EndDataCooking(CancellationToken cancellationToken)
{
base.EndDataCooking(cancellationToken);
this.ThreadEvents.FinalizeData();
}
}
}

53
PerfettoCds/Pipeline/SourceDataCookers/PerfettoThreadTrackerCooker.cs Normal file
Просмотреть файл

@ -0,0 +1,53 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK;
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Extensibility.DataCooking;
using Microsoft.Performance.SDK.Extensibility.DataCooking.SourceDataCooking;
using Microsoft.Performance.SDK.Processing;
using System.Collections.Generic;
using System.Threading;
using PerfettoCds.Pipeline.Events;
using PerfettoProcessor;
namespace PerfettoCds
{
/// <summary>
/// Cooks the data from the ThreadTrack table in Perfetto traces
/// </summary>
public sealed class PerfettoThreadTrackCooker : BaseSourceDataCooker<PerfettoSqlEventKeyed, PerfettoSourceParser, string>
{
public override string Description => "Processes events from the thread_tracks Perfetto SQL table";
//
// The data this cooker outputs. Tables or other cookers can query for this data
// via the SDK runtime
//
[DataOutput]
public ProcessedEventData<PerfettoThreadTrackEvent> ThreadTrackEvents { get; }
// Instructs runtime to only send events with the given keys this data cooker
public override ReadOnlyHashSet<string> DataKeys =>
new ReadOnlyHashSet<string>(new HashSet<string> { PerfettoPluginConstants.ThreadTrackEvent });
public PerfettoThreadTrackCooker() : base(PerfettoPluginConstants.ThreadTrackCookerPath)
{
this.ThreadTrackEvents =
new ProcessedEventData<PerfettoThreadTrackEvent>();
}
public override DataProcessingResult CookDataElement(PerfettoSqlEventKeyed perfettoEvent, PerfettoSourceParser context, CancellationToken cancellationToken)
{
this.ThreadTrackEvents.AddEvent((PerfettoThreadTrackEvent)perfettoEvent.SqlEvent);
return DataProcessingResult.Processed;
}
public override void EndDataCooking(CancellationToken cancellationToken)
{
base.EndDataCooking(cancellationToken);
this.ThreadTrackEvents.FinalizeData();
}
}
}

221
PerfettoCds/Pipeline/Tables/PerfettoGenericEventTable.cs Normal file
Просмотреть файл

@ -0,0 +1,221 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Processing;
using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using System.Diagnostics.CodeAnalysis;
using PerfettoCds.Pipeline.DataOutput;
using Microsoft.Performance.SDK;
using PerfettoCds.Pipeline.DataCookers;
namespace PerfettoCds.Pipeline.Tables
{
[Table]
public class PerfettoGenericEventTable
{
// Set some sort of max to prevent ridiculous field counts
public const int AbsoluteMaxFields = 20;
public static TableDescriptor TableDescriptor => new TableDescriptor(
Guid.Parse("{37cedaaa-5679-4366-b627-9b638aaefeb3}"),
"Perfetto Generic Events",
"All app/component events in the Perfetto trace",
"Perfetto",
requiredDataCookers: new List<DataCookerPath> { PerfettoPluginConstants.GenericEventCookerPath }
);
private static readonly ColumnConfiguration ProcessNameColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{b690f27e-7938-4e86-94ef-d048cbc476cc}"), "Process", "Name of the process"),
new UIHints { Width = 210 });
private static readonly ColumnConfiguration ThreadNameColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{dd1cf3f6-1cab-4012-bbdf-e99e920c4112}"), "Thread", "Name of the thread"),
new UIHints { Width = 210 });
private static readonly ColumnConfiguration EventNameColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{d3bc5189-c9d1-4c14-9ce2-7bb4dc4d5ee7}"), "Name", "Name of the Perfetto event"),
new UIHints { Width = 210 });
private static readonly ColumnConfiguration TimestampColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{d458382b-1320-45c6-ba86-885da9dae71d}"), "Timestamp", "Android timestamp for the event"),
new UIHints { Width = 120 });
private static readonly ColumnConfiguration DurationColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{14f4862d-5851-460d-a04b-62e4b62b6d6c}"), "Duration", "Duration of the event"),
new UIHints { Width = 70 });
private static readonly ColumnConfiguration CategoryColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{1aa73a71-1548-44fd-9bcd-854bca78ce2e}"), "Category", "StackID of the event"),
new UIHints { Width = 70 });
private static readonly ColumnConfiguration TypeColumn = new ColumnConfiguration(
new ColumnMetadata(new Guid("{01d2b15f-b0fc-4444-a240-0a96f62c2c50}"), "Type", "Type of the event"),
new UIHints { Width = 70 });
public static void BuildTable(ITableBuilder tableBuilder, IDataExtensionRetrieval tableData)
{
// We dynamically adjust the column headers
// This is the max number of fields we can expect for this table
int maxFieldCount = Math.Min(AbsoluteMaxFields, tableData.QueryOutput<int>(
new DataOutputPath(PerfettoPluginConstants.GenericEventCookerPath, nameof(PerfettoGenericEventCooker.MaximumEventFieldCount))));
// Get data from the cooker
var events = tableData.QueryOutput<ProcessedEventData<PerfettoGenericEvent>>(
new DataOutputPath(PerfettoPluginConstants.GenericEventCookerPath, nameof(PerfettoGenericEventCooker.GenericEvents)));
// Start construction of the column order. Pivot on process and thread
List<ColumnConfiguration> allColumns = new List<ColumnConfiguration>()
{
ProcessNameColumn,
ThreadNameColumn,
TableConfiguration.PivotColumn, // Columns before this get pivotted on
EventNameColumn,
DurationColumn,
CategoryColumn,
TypeColumn
};
var tableGenerator = tableBuilder.SetRowCount((int)events.Count);
var genericEventProjection = new GenericEventProjection(events);
// Add all the data projections
var processNameColumn = new BaseDataColumn<string>(
ProcessNameColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Process));
tableGenerator.AddColumn(processNameColumn);
var threadNameColumn = new BaseDataColumn<string>(
ThreadNameColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Thread));
tableGenerator.AddColumn(threadNameColumn);
var eventNameColumn = new BaseDataColumn<string>(
EventNameColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.EventName));
tableGenerator.AddColumn(eventNameColumn);
var timestampColumn = new BaseDataColumn<Timestamp>(
TimestampColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Timestamp));
tableGenerator.AddColumn(timestampColumn);
var durationColumn = new BaseDataColumn<TimestampDelta>(
DurationColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Duration));
tableGenerator.AddColumn(durationColumn);
var categoryColumn = new BaseDataColumn<string>(
CategoryColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Category));
tableGenerator.AddColumn(categoryColumn);
var typeColumn = new BaseDataColumn<string>(
TypeColumn,
genericEventProjection.Compose((genericEvent) => genericEvent.Type));
tableGenerator.AddColumn(typeColumn);
// Add the field columns, with column names depending on the given event
for (int index = 0; index < maxFieldCount; index++)
{
var colIndex = index; // This seems unncessary but causes odd runtime behavior if not done this way. Compiler is confused perhaps because w/o this func will index=genericEvent.FieldNames.Count every time. index is passed as ref but colIndex as value into func
string fieldName = "Field " + (colIndex + 1);
var genericEventFieldNameProjection = genericEventProjection.Compose((genericEvent) => colIndex < genericEvent.ArgKeys.Count ? genericEvent.ArgKeys[colIndex] : string.Empty);
// generate a column configuration
var fieldColumnConfiguration = new ColumnConfiguration(
new ColumnMetadata(GenerateGuidFromName(fieldName), fieldName, genericEventFieldNameProjection, fieldName),
new UIHints
{
IsVisible = true,
Width = 150,
TextAlignment = TextAlignment.Left,
});
// Add this column to the column order
allColumns.Add(fieldColumnConfiguration);
var genericEventFieldAsStringProjection = genericEventProjection.Compose((genericEvent) => colIndex < genericEvent.Values.Count ? genericEvent.Values[colIndex] : string.Empty);
tableGenerator.AddColumn(fieldColumnConfiguration, genericEventFieldAsStringProjection);
}
// Finish the column order with the timestamp columned being graphed
allColumns.Add(TableConfiguration.GraphColumn); // Columns after this get graphed
allColumns.Add(TimestampColumn);
var tableConfig = new TableConfiguration("Perfetto Trace Events")
{
Columns = allColumns,
Layout = TableLayoutStyle.GraphAndTable
};
tableConfig.AddColumnRole(ColumnRole.StartTime, TimestampColumn.Metadata.Guid);
tableConfig.AddColumnRole(ColumnRole.Duration, DurationColumn.Metadata.Guid);
tableBuilder.AddTableConfiguration(tableConfig).SetDefaultTableConfiguration(tableConfig);
}
[SuppressMessage("Microsoft.Security.Cryptography", "CA5354:SHA1CannotBeUsed", Justification = "Not a security related usage - just generating probabilistically unique id to identify a column from its name.")]
private static Guid GenerateGuidFromName(string name)
{
// The algorithm below is following the guidance of http://www.ietf.org/rfc/rfc4122.txt
// Create a blob containing a 16 byte number representing the namespace
// followed by the unicode bytes in the name.
var bytes = new byte[name.Length * 2 + 16];
uint namespace1 = 0x482C2DB2;
uint namespace2 = 0xC39047c8;
uint namespace3 = 0x87F81A15;
uint namespace4 = 0xBFC130FB;
// Write the bytes most-significant byte first.
for (int i = 3; 0 <= i; --i)
{
bytes[i] = (byte)namespace1;
namespace1 >>= 8;
bytes[i + 4] = (byte)namespace2;
namespace2 >>= 8;
bytes[i + 8] = (byte)namespace3;
namespace3 >>= 8;
bytes[i + 12] = (byte)namespace4;
namespace4 >>= 8;
}
// Write out the name, most significant byte first
for (int i = 0; i < name.Length; i++)
{
bytes[2 * i + 16 + 1] = (byte)name[i];
bytes[2 * i + 16] = (byte)(name[i] >> 8);
}
// Compute the Sha1 hash
var sha1 = SHA1.Create();
byte[] hash = sha1.ComputeHash(bytes);
// Create a GUID out of the first 16 bytes of the hash (SHA-1 create a 20 byte hash)
int a = (((((hash[3] << 8) + hash[2]) << 8) + hash[1]) << 8) + hash[0];
short b = (short)((hash[5] << 8) + hash[4]);
short c = (short)((hash[7] << 8) + hash[6]);
c = (short)((c & 0x0FFF) | 0x5000); // Set high 4 bits of octet 7 to 5, as per RFC 4122
Guid guid = new Guid(a, b, c, hash[8], hash[9], hash[10], hash[11], hash[12], hash[13], hash[14], hash[15]);
return guid;
}
public struct GenericEventProjection
: IProjection<int, PerfettoGenericEvent>
{
private readonly ProcessedEventData<PerfettoGenericEvent> genericEvents;
public GenericEventProjection(ProcessedEventData<PerfettoGenericEvent> genericEvents)
{
this.genericEvents = genericEvents;
}
public Type SourceType => typeof(int);
public Type ResultType => typeof(PerfettoGenericEvent);
public PerfettoGenericEvent this[int value] => this.genericEvents[(uint)value];
}
}
}

17
PerfettoCds/ReadMe.md Normal file
Просмотреть файл

@ -0,0 +1,17 @@
# WPA Perfetto Plugin
The PerfettoCds project adds a pipeline for displaying data from Perfetto traces using Windows Performance Toolkit SDK. The PerfettoProcessor project contains the logic for processing Perfetto traces.
Data is gathered from the Perfetto trace through trace_processor_shell.exe. Trace_processor_shell.exe opens an interface that allows for making SQL queries over HTTP/RPC to localhost. All the trace data is retrieved with SQL queries. Data is serialized over the HTTP interface with Protobuf objects. The original TraceProcessor protobuf object (trace_processor.proto) and the C# conversions are included in the project.
PerfettoSourceParser will start the trace_processor_shell.exe process. SQL Queries will be made over HTTP and the protobuf output will be converted to objects of type PerfettoSqlEvent. One query will be made per SQL table. Each query will produce events of the same type and will be processed by their own individual source cooker. For a complete generic event, we need data from 5 tables: slice, args, thread_track, thread, and process. A composite cooker will then take all the data gathered from each individual source cooker and join them to create the final generic event.
For example, for the slice table, the process goes like this:
1. Perform a SQL query into the Perfetto trace through trace_processor_shell.exe ("select * from slice"). This returns a QueryResult protobuf object. The QueryResult object is parsed and objects of type PerfettoSliceEvent are returned. One event for each row in the SQL table.
2. PerfettoSliceCooker will process/store all the PerfettoSliceEvents.
3. Once all the other tables have also finished, PerfettoGenericEventCooker will gather all the events from each cooker and do a join on them to create complete PerfettoGenericEvent objects.
4. Create WPA table of PerfettoGenericEvents
## Trace Processor Shell
Trace_processor_shell.exe is built from the [Perfetto GitHub repo](https://github.com/google/perfetto). To build this for Windows, follow the instructions from their [documentation](https://perfetto.dev/docs/contributing/build-instructions#building-on-windows)

88
PerfettoProcessor/Events/PerfettoArgEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,88 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using Perfetto.Protos;
namespace PerfettoProcessor
{
public class PerfettoArgEvent : PerfettoSqlEvent
{
public const string Key = "PerfettoArgEvent";
public static string SqlQuery = "select arg_set_id, flat_key, key, int_value, string_value, real_value, value_type from args order by arg_set_id";
public long ArgSetId { get; set; }
public string Flatkey { get; set; }
public string ArgKey { get; set; }
public long IntValue { get; set; }
public string StringValue { get; set; }
public double RealValue { get; set; }
public string ValueType { get; set; }
public override string GetSqlQuery()
{
return SqlQuery;
}
public override string GetEventKey()
{
return Key;
}
public override void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters)
{
var col = colName.ToLower();
switch (cellType)
{
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellInvalid:
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellNull:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellVarint:
var longVal = batch.VarintCells[counters.IntCounter++];
switch (col)
{
case "arg_set_id":
ArgSetId = longVal;
break;
case "int_value":
IntValue = longVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellFloat64:
var floatVal = batch.Float64Cells[counters.FloatCounter++];
if (col == "real_value")
{
RealValue = floatVal;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellString:
var strVal = stringCells[counters.StringCounter++];
switch (col)
{
case "flat_key":
Flatkey = strVal;
break;
case "key":
ArgKey = strVal;
break;
case "string_value":
StringValue = strVal;
break;
case "value_type":
ValueType = strVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellBlob:
break;
default:
throw new Exception("Unexpected CellType");
}
}
}
}

107
PerfettoProcessor/Events/PerfettoProcessEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,107 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using Perfetto.Protos;
namespace PerfettoProcessor
{
public class PerfettoProcessEvent : PerfettoSqlEvent
{
public const string Key = "PerfettoProcessEvent";
public static string SqlQuery = "select upid, id, type, pid, name, start_ts, end_ts, parent_upid, uid, android_appid, cmdline, arg_set_id from process";
public long Upid { get; set; }
public long Id { get; set; }
public string Type { get; set; }
public long Pid { get; set; }
public string Name { get; set; }
public long StartTimestamp { get; set; }
public long EndTimestamp{ get; set; }
public long ParentUpid { get; set; }
public long Uid { get; set; }
public long AndroidAppId { get; set; }
public string CmdLine { get; set; }
public long ArgSetId { get; set; }
public override string GetSqlQuery()
{
return SqlQuery;
}
public override string GetEventKey()
{
return Key;
}
public override void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters)
{
var col = colName.ToLower();
switch (cellType)
{
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellInvalid:
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellNull:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellVarint:
var longVal = batch.VarintCells[counters.IntCounter++];
switch (col)
{
case "upid":
Upid = longVal;
break;
case "id":
Id = longVal;
break;
case "pid":
Pid = longVal;
break;
case "uid":
Uid = longVal;
break;
case "parent_upid":
ParentUpid = longVal;
break;
case "android_appid":
AndroidAppId = longVal;
break;
case "arg_set_id":
ArgSetId = longVal;
break;
case "start_ts":
StartTimestamp = longVal;
break;
case "end_ts":
EndTimestamp = longVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellFloat64:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellString:
var strVal = stringCells[counters.StringCounter++];
switch (col)
{
case "type":
Type = strVal;
break;
case "cmdline":
CmdLine = strVal;
break;
case "name":
Name = strVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellBlob:
break;
default:
throw new Exception("Unexpected CellType");
}
}
}
}

86
PerfettoProcessor/Events/PerfettoSliceEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,86 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using Perfetto.Protos;
namespace PerfettoProcessor
{
public class PerfettoSliceEvent : PerfettoSqlEvent
{
public const string Key = "PerfettoSliceEvent";
public static string SqlQuery = "select ts, dur, arg_set_id, track_id, name, type, category from slice order by ts";
public string Name { get; set; }
public string Type { get; set; }
public long Duration { get; set; }
public long ArgSetId { get; set; }
public long Timestamp { get; set; }
public string Category { get; set; }
public long TrackId { get; set; }
public override string GetSqlQuery()
{
return SqlQuery;
}
public override string GetEventKey()
{
return Key;
}
public override void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters)
{
var col = colName.ToLower();
switch (cellType)
{
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellInvalid:
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellNull:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellVarint:
var longVal = batch.VarintCells[counters.IntCounter++];
switch (col)
{
case "ts":
Timestamp = longVal;
break;
case "dur":
Duration = longVal;
break;
case "arg_set_id":
ArgSetId = longVal;
break;
case "track_id":
TrackId = longVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellFloat64:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellString:
var strVal = stringCells[counters.StringCounter++];
switch (col)
{
case "name":
Name = strVal;
break;
case "type":
Type = strVal;
break;
case "category":
Category = strVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellBlob:
break;
default:
throw new Exception("Unexpected CellType");
}
}
}
}

58
PerfettoProcessor/Events/PerfettoSqlEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,58 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Perfetto.Protos;
namespace PerfettoProcessor
{
/// <summary>
/// Keep track of where we are in the varint, varfloat, and varstring arrays in a Perfetto batch result
/// </summary>
public class CellCounters
{
public int IntCounter { get; set; }
public int StringCounter { get; set; }
public int FloatCounter { get; set; }
}
/// <summary>
/// Base class for a row in a Perfetto SQL table
/// </summary>
public abstract class PerfettoSqlEvent
{
/// <summary>
/// This method is responsible for processing an individual cell in a SQL query, which is stored in the QueryResult
/// protobuf object (QueryResult.Batch.Cells)
///
/// CellType defines what type this cell is (int, string, float, ...) and the batch object contains
/// separate data arrays for each of those types (varintcells[], varfloatcells[]). The counters object
/// maintains the indices into each of these data arrays.
///
/// String cells need to be preprocessed by us because they come as a single string delimited by null characater.
///
/// For more information, see comments for QueryResult and CellsBatch inside TraceProcessor.cs
///
/// </summary>
/// <param name="colName">String name of the column</param>
/// <param name="cellType">Type of cell</param>
/// <param name="batch">Batch object that contains the data arrays</param>
/// <param name="stringCells">All the string cells already split into an array</param>
/// <param name="counters">Indexes into the data arrays</param>
public abstract void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters);
/// <summary>
/// Returns the SQL query used to query this objects information
/// </summary>
/// <returns></returns>
public abstract string GetSqlQuery();
/// <summary>
/// String representation of this event type
/// </summary>
/// <returns></returns>
public abstract string GetEventKey();
}
}

94
PerfettoProcessor/Events/PerfettoThreadEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,94 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using Perfetto.Protos;
namespace PerfettoProcessor
{
public class PerfettoThreadEvent : PerfettoSqlEvent
{
public const string Key = "PerfettoThreadEvent";
public static string SqlQuery = "select utid, id, type, tid, name, start_ts, end_ts, upid, is_main_thread from thread";
public long Utid { get; set; }
public long Id { get; set; }
public string Type { get; set; }
public long Tid { get; set; }
public string Name{ get; set; }
public long StartTimestamp { get; set; }
public long EndTimestamp { get; set; }
public long Upid { get; set; }
public long IsMainThread{ get; set; }
public override string GetSqlQuery()
{
return SqlQuery;
}
public override string GetEventKey()
{
return Key;
}
public override void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters)
{
var col = colName.ToLower();
switch (cellType)
{
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellInvalid:
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellNull:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellVarint:
var longVal = batch.VarintCells[counters.IntCounter++];
switch (col)
{
case "utid":
Utid = longVal;
break;
case "id":
Id = longVal;
break;
case "upid":
Upid = longVal;
break;
case "tid":
Tid = longVal;
break;
case "is_main_thread":
IsMainThread = longVal;
break;
case "start_ts":
StartTimestamp = longVal;
break;
case "end_ts":
EndTimestamp = longVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellFloat64:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellString:
var strVal = stringCells[counters.StringCounter++];
switch (col)
{
case "type":
Type = strVal;
break;
case "name":
Name = strVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellBlob:
break;
default:
throw new Exception("Unexpected CellType");
}
}
}
}

83
PerfettoProcessor/Events/PerfettoThreadTrackEvent.cs Normal file
Просмотреть файл

@ -0,0 +1,83 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System;
using Perfetto.Protos;
namespace PerfettoProcessor
{
public class PerfettoThreadTrackEvent : PerfettoSqlEvent
{
public const string Key = "PerfettoThreadTrackEvent";
public static string SqlQuery = "select id, type, name, source_arg_set_id, utid from thread_track";
public long ArgSetId { get; set; }
public long Id { get; set; }
public string Type { get; set; }
public string Name { get; set; }
public long SourceArgSetId { get; set; }
public long Utid { get; set; }
public override string GetSqlQuery()
{
return SqlQuery;
}
public override string GetEventKey()
{
return Key;
}
public override void ProcessCell(string colName,
QueryResult.Types.CellsBatch.Types.CellType cellType,
QueryResult.Types.CellsBatch batch,
string[] stringCells,
CellCounters counters)
{
var col = colName.ToLower();
switch (cellType)
{
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellInvalid:
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellNull:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellVarint:
var longVal = batch.VarintCells[counters.IntCounter++];
switch (col)
{
case "arg_set_id":
ArgSetId = longVal;
break;
case "id":
Id = longVal;
break;
case "utid":
Utid = longVal;
break;
case "source_arg_set_id":
SourceArgSetId = longVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellFloat64:
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellString:
var strVal = stringCells[counters.StringCounter++];
switch (col)
{
case "type":
Type = strVal;
break;
case "name":
Name = strVal;
break;
}
break;
case Perfetto.Protos.QueryResult.Types.CellsBatch.Types.CellType.CellBlob:
break;
default:
throw new Exception("Unexpected CellType");
}
}
}
}

13
PerfettoProcessor/PerfettoProcessor.csproj Normal file
Просмотреть файл

@ -0,0 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Google.Protobuf" Version="3.17.3">
<GeneratePathProperty>true</GeneratePathProperty>
</PackageReference>
</ItemGroup>
</Project>

242
PerfettoProcessor/PerfettoTraceProcessor.cs Normal file
Просмотреть файл

@ -0,0 +1,242 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using Google.Protobuf;
using Perfetto.Protos;
using System;
using System.Diagnostics;
using System.Net.Http;
namespace PerfettoProcessor
{
/// <summary>
/// Responsible for managing trace_processor_shell.exe. The shell executable loads the Perfetto trace and allows for SQL querying
/// through HTTP on localhost. HTTP communication happens with protobuf objects/buffers.
/// </summary>
public class PerfettoTraceProcessor
{
private Process ShellProcess;
// Default starting port
private int HttpPort = 22222;
// Highest ports can go
private const int PortMax = 65535;
// HTTP request denied takes about 3 seconds so time out after about 2 minutes
private const int MaxRetryLimit = 40;
/// <summary>
/// Check if another instance of trace_processor_shell is running on this port
/// </summary>
/// <param name="port"></param>
/// <returns></returns>
private bool IsPortAvailable(int port)
{
try
{
using (var client = new HttpClient())
{
var response = client.GetStringAsync($"http://localhost:{HttpPort}/status").Result;
return !(response.Contains("Perfetto"));
}
}
catch (Exception)
{
// Exception here is the connection refused message, meaning the RPC server has not been initialized
// Which means this port is not being used by another trace_processor_shell
return true;
}
}
/// <summary>
/// Initializes trace_processor_shell.exe in HTTP/RPC mode with the trace file
/// </summary>
/// <param name="shellPath">Full path to trace_processor_shell.exe</param>
/// <param name="tracePath">Full path to the Perfetto trace file</param>
public void OpenTraceProcessor(string shellPath, string tracePath)
{
using (var client = new HttpClient())
{
// Make sure another instance of trace_processor_shell isn't already running
while (!IsPortAvailable(HttpPort) && HttpPort < PortMax)
{
HttpPort++;
}
ShellProcess = Process.Start(shellPath, $"-D --http-port {HttpPort} -i \"{tracePath}\"");
}
if (ShellProcess.HasExited)
{
throw new Exception("Problem starting trace_processor_shell.exe");
}
}
/// <summary>
/// Initializes trace_processor_shell.exe in HTTP/RPC mode without the trace file
/// </summary>
/// <param name="shellPath">Full path to trace_processor_shell.exe</param>
public void OpenTraceProcessor(string shellPath)
{
using (var client = new HttpClient())
{
// Make sure another instance of trace_processor_shell isn't already running
while (!IsPortAvailable(HttpPort) && HttpPort < PortMax)
{
HttpPort++;
}
ShellProcess = Process.Start(shellPath, $"-D --http-port {HttpPort}");
}
if (ShellProcess.HasExited)
{
throw new Exception("Problem starting trace_processor_shell.exe");
}
}
/// <summary>
/// Check if the trace_processor_shell has a loaded trace. Assumes the shell executable is already running. Will throw otherwise.
/// </summary>
/// <returns></returns>
private bool CheckIfTraceIsLoaded()
{
if (ShellProcess == null || ShellProcess.HasExited)
{
throw new Exception("The trace_process_shell is not running");
}
try
{
using (var client = new HttpClient())
{
Perfetto.Protos.StatusResult statusResult = new StatusResult();
var response = client.GetAsync($"http://localhost:{HttpPort}/status").GetAwaiter().GetResult();
var byteArray = response.Content.ReadAsByteArrayAsync().Result;
statusResult = StatusResult.Parser.ParseFrom(byteArray);
// TODO could also check that the trace name is the same
return statusResult.HasLoadedTraceName;
}
}
catch (Exception)
{
// Exception here is the connection refused message, meaning the RPC server has not been initialized
return false;
}
}
/// <summary>
/// Perform a SQL query against trace_processor_shell to gather Perfetto trace data.
/// </summary>
/// <param name="sqlQuery">The query to perform against the loaded trace in trace_processor_shell</param>
/// <returns></returns>
public QueryResult QueryTrace(string sqlQuery)
{
// Make sure ShellProcess is running
if (ShellProcess == null || ShellProcess.HasExited)
{
throw new Exception("The trace_process_shell is not running");
}
int cnt = 0;
// Check if the trace is loaded
// We know the shell is running, so the trace could still be in the loading process. Give it a little while to finish loading
// before we error out.
while (!CheckIfTraceIsLoaded())
{
if (cnt++ > MaxRetryLimit)
{
throw new Exception("Unable to query Perfetto trace because trace_processor_shell.exe does not appear to have loaded it");
}
}
QueryResult qr = null;
using (var client = new HttpClient())
{
// Query with protobuf over RPC using /query endpoint
Perfetto.Protos.RawQueryArgs queryArgs = new Perfetto.Protos.RawQueryArgs();
queryArgs.SqlQuery = sqlQuery;
HttpContent sc = new ByteArrayContent(queryArgs.ToByteArray());
var response = client.PostAsync($"http://localhost:{HttpPort}/query", sc).GetAwaiter().GetResult();
var byteArray = response.Content.ReadAsByteArrayAsync().Result;
qr = QueryResult.Parser.ParseFrom(byteArray);
}
return qr;
}
/// <summary>
/// Perform a SQL query against trace_processor_shell to gather Perfetto trace data. Processes the QueryResult
/// and returns PerfettoSqlObjects through a callback
/// </summary>
/// <param name="sqlQuery">The query to perform against the loaded trace in trace_processor_shell</param>
/// <param name="eventKey">The event key that corresponds to the type of PerfettoSqlEvent to process for this query</param>
/// <param name="eventCallback">Completed PerfettoSqlEvents will be sent here</param>
public void QueryTraceForEvents(string sqlQuery, string eventKey, Action<PerfettoSqlEvent> eventCallback)
{
var qr = QueryTrace(sqlQuery);
var numColumns = qr.ColumnNames.Count;
var cols = qr.ColumnNames;
var numBatches = qr.Batch.Count;
foreach (var batch in qr.Batch)
{
CellCounters cellCounters = new CellCounters();
// String cells get stored as a single string delimited by null character. Split that up ourselves
var stringCells = batch.StringCells.Split('\0');
int cellCount = 0;
PerfettoSqlEvent ev = null;
foreach (var cell in batch.Cells)
{
if (ev == null)
{
switch (eventKey)
{
case PerfettoSliceEvent.Key:
ev = new PerfettoSliceEvent();
break;
case PerfettoArgEvent.Key:
ev = new PerfettoArgEvent();
break;
case PerfettoThreadTrackEvent.Key:
ev = new PerfettoThreadTrackEvent();
break;
case PerfettoThreadEvent.Key:
ev = new PerfettoThreadEvent();
break;
case PerfettoProcessEvent.Key:
ev = new PerfettoProcessEvent();
break;
default:
throw new Exception("Invalid event type");
}
}
var colIndex = cellCount % numColumns;
var colName = cols[colIndex].ToLower();
// The event itself is responsible for figuring out how to process and store cell contents
ev.ProcessCell(colName, cell, batch, stringCells, cellCounters);
// If we've reached the end of a row, we've finished an event.
if (++cellCount % numColumns == 0)
{
// Report the event back
eventCallback(ev);
ev = null;
}
}
}
}
public void CloseTraceConnection()
{
ShellProcess?.Kill();
}
}
}

2799
PerfettoProcessor/Protobuf/Descriptor.cs Normal file
Просмотреть файл

Разница между файлами не показана из-за своего большого размера Загрузить разницу

8
PerfettoProcessor/Protobuf/Readme.txt Normal file
Просмотреть файл

@ -0,0 +1,8 @@
TraceProcessor.cs and Descriptor.cs are C# equivalent versions of traceprocessor.proto and descriptor.proto.
The .proto files are from the Perfetto repo (https://github.com/google/perfetto). To convert .proto to a C# class,
the protoc.exe executable can be used, which can be retrieved from the Google.Protobuf.Tools Nuget package.
The command lines to generate the C# files are something like this:
protoc.exe -I d:\perfetto_root --csharp_out=output_dir D:\perfetto_root\protos\perfetto\trace_processor\trace_processor.proto
protoc.exe -I d:\perfetto_root --csharp_out=output_dir D:\perfetto_root\protos\perfetto\common\descriptor.proto

5402
PerfettoProcessor/Protobuf/TraceProcessor.cs Normal file
Просмотреть файл

Разница между файлами не показана из-за своего большого размера Загрузить разницу

198
PerfettoProcessor/Protobuf/descriptor.proto Normal file
Просмотреть файл

@ -0,0 +1,198 @@
/*
* Copyright (C) 2018 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// This is a subset of descriptor.proto from the Protobuf library.
syntax = "proto2";
package perfetto.protos;
// The protocol compiler can output a FileDescriptorSet containing the .proto
// files it parses.
message FileDescriptorSet {
repeated FileDescriptorProto file = 1;
}
// Describes a complete .proto file.
message FileDescriptorProto {
// file name, relative to root of source tree
optional string name = 1;
// e.g. "foo", "foo.bar", etc.
optional string package = 2;
// Names of files imported by this file.
repeated string dependency = 3;
// Indexes of the public imported files in the dependency list above.
repeated int32 public_dependency = 10;
// Indexes of the weak imported files in the dependency list.
// For Google-internal migration only. Do not use.
repeated int32 weak_dependency = 11;
// All top-level definitions in this file.
repeated DescriptorProto message_type = 4;
repeated EnumDescriptorProto enum_type = 5;
repeated FieldDescriptorProto extension = 7;
reserved 6;
reserved 8;
reserved 9;
reserved 12;
}
// Describes a message type.
message DescriptorProto {
optional string name = 1;
repeated FieldDescriptorProto field = 2;
repeated FieldDescriptorProto extension = 6;
repeated DescriptorProto nested_type = 3;
repeated EnumDescriptorProto enum_type = 4;
reserved 5;
repeated OneofDescriptorProto oneof_decl = 8;
reserved 7;
// Range of reserved tag numbers. Reserved tag numbers may not be used by
// fields or extension ranges in the same message. Reserved ranges may
// not overlap.
message ReservedRange {
// Inclusive.
optional int32 start = 1;
// Exclusive.
optional int32 end = 2;
}
repeated ReservedRange reserved_range = 9;
// Reserved field names, which may not be used by fields in the same message.
// A given name may only be reserved once.
repeated string reserved_name = 10;
}
// Describes a field within a message.
message FieldDescriptorProto {
enum Type {
// 0 is reserved for errors.
// Order is weird for historical reasons.
TYPE_DOUBLE = 1;
TYPE_FLOAT = 2;
// Not ZigZag encoded. Negative numbers take 10 bytes. Use TYPE_SINT64 if
// negative values are likely.
TYPE_INT64 = 3;
TYPE_UINT64 = 4;
// Not ZigZag encoded. Negative numbers take 10 bytes. Use TYPE_SINT32 if
// negative values are likely.
TYPE_INT32 = 5;
TYPE_FIXED64 = 6;
TYPE_FIXED32 = 7;
TYPE_BOOL = 8;
TYPE_STRING = 9;
// Tag-delimited aggregate.
// Group type is deprecated and not supported in proto3. However, Proto3
// implementations should still be able to parse the group wire format and
// treat group fields as unknown fields.
TYPE_GROUP = 10;
// Length-delimited aggregate.
TYPE_MESSAGE = 11;
// New in version 2.
TYPE_BYTES = 12;
TYPE_UINT32 = 13;
TYPE_ENUM = 14;
TYPE_SFIXED32 = 15;
TYPE_SFIXED64 = 16;
// Uses ZigZag encoding.
TYPE_SINT32 = 17;
// Uses ZigZag encoding.
TYPE_SINT64 = 18;
};
enum Label {
// 0 is reserved for errors
LABEL_OPTIONAL = 1;
LABEL_REQUIRED = 2;
LABEL_REPEATED = 3;
};
optional string name = 1;
optional int32 number = 3;
optional Label label = 4;
// If type_name is set, this need not be set. If both this and type_name
// are set, this must be one of TYPE_ENUM, TYPE_MESSAGE or TYPE_GROUP.
optional Type type = 5;
// For message and enum types, this is the name of the type. If the name
// starts with a '.', it is fully-qualified. Otherwise, C++-like scoping
// rules are used to find the type (i.e. first the nested types within this
// message are searched, then within the parent, on up to the root
// namespace).
optional string type_name = 6;
// For extensions, this is the name of the type being extended. It is
// resolved in the same manner as type_name.
optional string extendee = 2;
// For numeric types, contains the original text representation of the value.
// For booleans, "true" or "false".
// For strings, contains the default text contents (not escaped in any way).
// For bytes, contains the C escaped value. All bytes >= 128 are escaped.
// TODO(kenton): Base-64 encode?
optional string default_value = 7;
// If set, gives the index of a oneof in the containing type's oneof_decl
// list. This field is a member of that oneof.
optional int32 oneof_index = 9;
reserved 10;
reserved 8;
}
// Describes a oneof.
message OneofDescriptorProto {
optional string name = 1;
optional OneofOptions options = 2;
}
// Describes an enum type.
message EnumDescriptorProto {
optional string name = 1;
repeated EnumValueDescriptorProto value = 2;
reserved 3;
reserved 4;
// Reserved enum value names, which may not be reused. A given name may only
// be reserved once.
repeated string reserved_name = 5;
}
// Describes a value within an enum.
message EnumValueDescriptorProto {
optional string name = 1;
optional int32 number = 2;
reserved 3;
}
message OneofOptions {
reserved 999;
// Clients can define custom options in extensions of this message. See above.
extensions 1000 to max;
}

304
PerfettoProcessor/Protobuf/trace_processor.proto Normal file
Просмотреть файл

@ -0,0 +1,304 @@
/*
* Copyright (C) 2018 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
syntax = "proto2";
package perfetto.protos;
import "protos/perfetto/common/descriptor.proto";
// This file defines the schema for {,un}marshalling arguments and return values
// when interfacing to the trace processor binary interface.
// The Trace Processor can be used in three modes:
// 1. Fully native from C++ or directly using trace_processor_shell.
// In this case, this file isn't really relevant because no binary
// marshalling is involved. Look at include/trace_processor/trace_processor.h
// for the public C++ API definition.
// 2. Using WASM within the HTML ui. In this case these messages are used to
// {,un}marshall calls made through the JS<>WASM interop in
// src/trace_processor/rpc/wasm_bridge.cc .
// 3. Using the HTTP+RPC interface, by running trace_processor_shell -D.
// In this case these messages are used to {,un}marshall HTTP requests and
// response made through src/trace_processor/rpc/httpd.cc .
enum TraceProcessorApiVersion {
// This variable has been introduced in v15 and is used to deal with API
// mismatches between UI and trace_processor_shell --httpd. Increment this
// every time a new feature that the UI depends on is being introduced (e.g.
// new tables, new SQL operators, metrics that are required by the UI).
// See also TraceProcessorVersion (below).
TRACE_PROCESSOR_CURRENT_API_VERSION = 1;
}
// At lowest level, the wire-format of the RPC procol is a linear sequence of
// TraceProcessorRpc messages on each side of the byte pipe
// Each message is prefixed by a tag (field = 1, type = length delimited) and a
// varint encoding its size (this is so the whole stream can also be read /
// written as if it was a repeated field of TraceProcessorRpcStream).
message TraceProcessorRpcStream {
repeated TraceProcessorRpc msg = 1;
}
message TraceProcessorRpc {
// A monotonic counter used only for debugging purposes, to detect if the
// underlying stream is missing or duping data. The counter starts at 0 on
// each side of the pipe and is incremented on each message.
// Do NOT expect that a response has the same |seq| of its corresponding
// request: some requests (e.g., a query returning many rows) can yield more
// than one response message, bringing the tx and rq seq our of sync.
optional int64 seq = 1;
// This is returned when some unrecoverable error has been detected by the
// peer. The typical case is TraceProcessor detecting that the |seq| sequence
// is broken (e.g. when having two tabs open with the same --httpd instance).
optional string fatal_error = 5;
enum TraceProcessorMethod {
TPM_UNSPECIFIED = 0;
TPM_APPEND_TRACE_DATA = 1;
TPM_FINALIZE_TRACE_DATA = 2;
TPM_QUERY_STREAMING = 3;
TPM_QUERY_RAW_DEPRECATED = 4;
TPM_COMPUTE_METRIC = 5;
TPM_GET_METRIC_DESCRIPTORS = 6;
TPM_RESTORE_INITIAL_TABLES = 7;
TPM_ENABLE_METATRACE = 8;
TPM_DISABLE_AND_READ_METATRACE = 9;
TPM_GET_STATUS = 10;
}
oneof type {
// Client -> TraceProcessor requests.
TraceProcessorMethod request = 2;
// TraceProcessor -> Client responses.
TraceProcessorMethod response = 3;
// This is sent back instead of filling |response| when the client sends a
// |request| which is not known by the TraceProcessor service. This can
// happen when the client is newer than the service.
TraceProcessorMethod invalid_request = 4;
}
// Request/Response arguments.
// Not all requests / responses require an argument.
oneof args {
// TraceProcessorMethod request args.
// For TPM_APPEND_TRACE_DATA.
bytes append_trace_data = 101;
// For TPM_QUERY_STREAMING.
QueryArgs query_args = 103;
// For TPM_QUERY_RAW_DEPRECATED.
RawQueryArgs raw_query_args = 104;
// For TPM_COMPUTE_METRIC.
ComputeMetricArgs compute_metric_args = 105;
// TraceProcessorMethod response args.
// For TPM_APPEND_TRACE_DATA.
AppendTraceDataResult append_result = 201;
// For TPM_QUERY_STREAMING.
QueryResult query_result = 203;
// For TPM_QUERY_RAW_DEPRECATED.
RawQueryResult raw_query_result = 204;
// For TPM_COMPUTE_METRIC.
ComputeMetricResult metric_result = 205;
// For TPM_GET_METRIC_DESCRIPTORS.
DescriptorSet metric_descriptors = 206;
// For TPM_DISABLE_AND_READ_METATRACE.
DisableAndReadMetatraceResult metatrace = 209;
// For TPM_GET_STATUS.
StatusResult status = 210;
}
}
message AppendTraceDataResult {
optional int64 total_bytes_parsed = 1;
optional string error = 2;
}
message QueryArgs {
optional string sql_query = 1;
// Wall time when the query was queued. Used only for query stats.
optional uint64 time_queued_ns = 2;
}
// Input for the /raw_query endpoint.
message RawQueryArgs {
optional string sql_query = 1;
// Wall time when the query was queued. Used only for query stats.
optional uint64 time_queued_ns = 2;
}
// Output for the /raw_query endpoint.
// DEPRECATED, use /query. See QueryResult below.
message RawQueryResult {
message ColumnDesc {
optional string name = 1;
enum Type {
UNKNOWN = 0;
LONG = 1;
DOUBLE = 2;
STRING = 3;
}
optional Type type = 2;
}
message ColumnValues {
// Only one of this field will be filled for each column (according to the
// corresponding descriptor) and that one will have precisely |num_records|
// elements.
repeated int64 long_values = 1;
repeated double double_values = 2;
repeated string string_values = 3;
// This will be set to true or false depending on whether the data at the
// given index is NULL.
repeated bool is_nulls = 4;
}
repeated ColumnDesc column_descriptors = 1;
optional uint64 num_records = 2;
repeated ColumnValues columns = 3;
optional string error = 4;
optional uint64 execution_time_ns = 5;
}
// Output for the /query endpoint.
// Returns a query result set, grouping cells into batches. Batching allows a
// more efficient encoding of results, at the same time allowing to return
// O(M) results in a pipelined fashion, without full-memory buffering.
// Batches are split when either a large number of cells (~thousands) is reached
// or the string/blob payload becomes too large (~hundreds of KB).
// Data is batched in cells, scanning results by row -> column. e.g. if a query
// returns 3 columns and 2 rows, the cells will be emitted in this order:
// R0C0, R0C1, R0C2, R1C0, R1C1, R1C2.
message QueryResult {
// This determines the number and names of columns.
repeated string column_names = 1;
// If non-emty the query returned an error. Note that some cells might still
// be present, if the error happened while iterating.
optional string error = 2;
// A batch contains an array of cell headers, stating the type of each cell.
// The payload of each cell is stored in the corresponding xxx_cells field
// below (unless the cell is NULL).
// So if |cells| contains: [VARINT, FLOAT64, VARINT, STRING], the results will
// be available as:
// [varint_cells[0], float64_cells[0], varint_cells[1], string_cells[0]].
message CellsBatch {
enum CellType {
CELL_INVALID = 0;
CELL_NULL = 1;
CELL_VARINT = 2;
CELL_FLOAT64 = 3;
CELL_STRING = 4;
CELL_BLOB = 5;
}
repeated CellType cells = 1 [packed = true];
repeated int64 varint_cells = 2 [packed = true];
repeated double float64_cells = 3 [packed = true];
repeated bytes blob_cells = 4;
// The string cells are concatenated in a single field. Each cell is
// NUL-terminated. This is because JS incurs into a non-negligible overhead
// when decoding strings and one decode + split('\0') is measurably faster
// than decoding N strings. See goto.google.com/postmessage-benchmark .
optional string string_cells = 5;
// If true this is the last batch for the query result.
optional bool is_last_batch = 6;
// Padding field. Used only to re-align and fill gaps in the binary format.
reserved 7;
}
repeated CellsBatch batch = 3;
}
// Input for the /status endpoint.
message StatusArgs {}
// Output for the /status endpoint.
message StatusResult {
// If present and not empty, a trace is already loaded already. This happens
// when using the HTTP+RPC mode nad passing a trace file to the shell, via
// trace_processor_shell -D trace_file.pftrace .
optional string loaded_trace_name = 1;
// Typically something like "v11.0.123", but could be just "v11" or "unknown",
// for binaries built from Bazel or other build configurations. This is for
// human presentation only, don't attempt to parse and reason on it.
optional string human_readable_version = 2;
// The API version is incremented every time a change that the UI depends
// on is introduced (e.g. adding a new table that the UI queries).
optional int32 api_version = 3;
}
// Input for the /compute_metric endpoint.
message ComputeMetricArgs {
enum ResultFormat {
BINARY_PROTOBUF = 0;
TEXTPROTO = 1;
}
repeated string metric_names = 1;
optional ResultFormat format = 2;
}
// Output for the /compute_metric endpoint.
message ComputeMetricResult {
oneof result {
// This is meant to contain a perfetto.protos.TraceMetrics. We're using
// bytes instead of the actual type because we do not want to generate
// protozero code for the metrics protos. We always encode/decode metrics
// using a reflection based mechanism that does not require the compiled C++
// code. This allows us to read in new protos at runtime.
bytes metrics = 1;
// A perfetto.protos.TraceMetrics formatted as prototext.
string metrics_as_prototext = 3;
}
optional string error = 2;
}
// Input for the /enable_metatrace endpoint.
message EnableMetatraceArgs {}
// Output for the /enable_metatrace endpoint.
message EnableMetatraceResult {}
// Input for the /disable_and_read_metatrace endpoint.
message DisableAndReadMetatraceArgs {}
// Output for the /disable_and_read_metatrace endpoint.
message DisableAndReadMetatraceResult {
// Bytes of perfetto.protos.Trace message. Stored as bytes
// to avoid adding a dependency on trace.proto.
optional bytes metatrace = 1;
optional string error = 2;
}
// Convenience wrapper for multiple descriptors, similar to FileDescriptorSet
// in descriptor.proto.
message DescriptorSet {
repeated DescriptorProto descriptors = 1;
}

104
PerfettoUnitTest/PerfettoUnitTest.cs Normal file
Просмотреть файл

@ -0,0 +1,104 @@
using Microsoft.Performance.SDK.Extensibility;
using Microsoft.Performance.SDK.Processing;
using Microsoft.Performance.Toolkit.Engine;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using PerfettoCds;
using PerfettoCds.Pipeline.DataCookers;
using PerfettoCds.Pipeline.DataOutput;
using System.IO;
namespace PerfettoUnitTest
{
[TestClass]
public class PerfettoUnitTest
{
public static bool IsTraceProcessed = false;
public static object IsTraceProcessedLock = new object();
private static RuntimeExecutionResults RuntimeExecutionResults;
private static DataCookerPath PerfettoGenericEventDataCookerPath;
private static DataCookerPath PerfettoSliceCookerPath;
private static DataCookerPath PerfettoArgCookerPath;
private static DataCookerPath PerfettoThreadTrackCookerPath;
private static DataCookerPath PerfettoThreadCookerPath;
private static DataCookerPath PerfettoProcessCookerPath;
public static void ProcessTrace()
{
lock (IsTraceProcessedLock)
{
if (!IsTraceProcessed)
{
// Input data
string perfettoTrace = @"..\..\..\..\TestData\Perfetto\sample.perfetto-trace";
var perfettoDataPath = new FileInfo(perfettoTrace);
Assert.IsTrue(perfettoDataPath.Exists);
// Approach #1 - Engine - Doesn't test tables UI but tests processing
var runtime = Engine.Create();
runtime.AddFile(perfettoDataPath.FullName);
// Enable our various types of data cookers
var perfettoSliceCooker = new PerfettoSliceCooker();
PerfettoSliceCookerPath = perfettoSliceCooker.Path;
runtime.EnableCooker(PerfettoSliceCookerPath);
var perfettoArgCooker = new PerfettoArgCooker();
PerfettoArgCookerPath = perfettoArgCooker.Path;
runtime.EnableCooker(PerfettoArgCookerPath);
var perfettoThreadTrackCooker = new PerfettoThreadTrackCooker();
PerfettoThreadTrackCookerPath = perfettoThreadTrackCooker.Path;
runtime.EnableCooker(PerfettoThreadTrackCookerPath);
var perfettoThreadCooker = new PerfettoThreadCooker();
PerfettoThreadCookerPath = perfettoThreadCooker.Path;
runtime.EnableCooker(PerfettoThreadCookerPath);
var perfettoProcessCooker = new PerfettoProcessCooker();
PerfettoProcessCookerPath = perfettoProcessCooker.Path;
runtime.EnableCooker(PerfettoProcessCookerPath);
var perfettoGenericEventDataCooker = new PerfettoGenericEventCooker();
PerfettoGenericEventDataCookerPath = perfettoGenericEventDataCooker.Path;
runtime.EnableCooker(PerfettoGenericEventDataCookerPath);
//
// Process our data.
//
RuntimeExecutionResults = runtime.Process();
IsTraceProcessed = true;
}
}
}
/// <summary>
/// PerfettoGenericEvents gather data from multiple source cookers. Valid PerfettoGenericEvents ensure
/// that the cookers below it also worked successfully.
/// </summary>
[TestMethod]
public void TestPerfettoGenericEvents()
{
ProcessTrace();
var eventData = RuntimeExecutionResults.QueryOutput<ProcessedEventData<PerfettoGenericEvent>>(
new DataOutputPath(
PerfettoGenericEventDataCookerPath,
nameof(PerfettoGenericEventCooker.GenericEvents)));
Assert.IsTrue(eventData.Count == 3);
// Ensures join with slices table worked
Assert.IsTrue(eventData[0].EventName == "name1");
// Ensures joins with thread_track, thread, and process table worked
Assert.IsTrue(eventData[1].Thread == "t1 1");
Assert.IsTrue(eventData[2].Process == " 5");
// Ensures join with args table worked
Assert.IsTrue(eventData[0].ArgKeys.Count == 1);
Assert.IsTrue(eventData[0].ArgKeys[0] == "chrome_user_event.action");
}
}
}

28
PerfettoUnitTest/PerfettoUnitTest.csproj Normal file
Просмотреть файл

@ -0,0 +1,28 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.1" />
<PackageReference Include="Microsoft.Performance.SDK" Version="0.109.11-preview-g61bc0d83f6" />
<PackageReference Include="Microsoft.Performance.SDK.Runtime" Version="0.109.11-preview-g61bc0d83f6" />
<PackageReference Include="Microsoft.Performance.Toolkit.Engine" Version="0.109.11-preview-g61bc0d83f6" />
<PackageReference Include="Moq" Version="4.16.1" />
<PackageReference Include="MSTest.TestAdapter" Version="2.2.3" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.3" />
<PackageReference Include="coverlet.collector" Version="3.0.3">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\PerfettoCds\PerfettoCds.csproj" />
<ProjectReference Include="..\PerfettoProcessor\PerfettoProcessor.csproj" />
</ItemGroup>
</Project>

Двоичные данные
TestData/Perfetto/sample.perfetto-trace Normal file
Просмотреть файл

Двоичный файл не отображается.

8
azure-pipelines.yml
Просмотреть файл

@ -59,6 +59,7 @@ jobs:
LTTngDriver\LTTngDriver.csproj
PerfCds\PerfCds.csproj
PerfDataExtensions\PerfDataExtensions.csproj
PerfettoCds\PerfettoCds.csproj
PerfUnitTest\PerfUnitTest.csproj
LinuxLogParsers\LinuxPlugins-MicrosoftPerformanceToolkSDK\Cloud-init\Cloud-Init.csproj
LinuxLogParsers\LinuxPlugins-MicrosoftPerformanceToolkSDK\DmesgIsoLog\Dmesg.csproj
@ -94,6 +95,13 @@ jobs:
SourceFolder: 'LTTngDriver/bin/$(BuildConfiguration)/netcoreapp3.1'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/Microsoft-Performance-Tools-Linux/LTTngDriver'
- task: CopyFiles@2
displayName: Copy Perfetto Build to Output Artifacts
inputs:
SourceFolder: 'PerfettoCds/bin/$(BuildConfiguration)/netstandard2.1'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)/Microsoft-Performance-Tools-Linux/PerfettoAddin'
- task: CopyFiles@2
displayName: Copy Cloud-init Build to Output Artifacts