Initial commit of dotnet-gcdump

* NOTE: use this commit as baseline for file copied from Microsoft/PerfView
This commit is contained in:
John Salem 2019-10-25 14:55:32 -07:00
Родитель f6eda5af57
Коммит bc210aa496
14 изменённых файлов: 6004 добавлений и 1 удалений

Просмотреть файл

@ -249,3 +249,29 @@ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License notice for code from Microsoft/PerfView:
-------------------------------------------------
The MIT License (MIT)
Copyright (c) .NET Foundation and Contributors
All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Просмотреть файл

@ -578,7 +578,7 @@ Header: `{ Magic; Size; 0x0101; 0x0000 }`
* WithHeap = 2,
* Triage = 3,
* Full = 4
* `uint diagnostics`: If set to 1, log to console the dump generation diagnostics
* `uint diagnostics`: The providers to turn on for the streaming session
* `0` or `1` for on or off
#### Returns (as an IPC Message Payload):

Просмотреть файл

@ -0,0 +1,77 @@
# Heap Analysis Tool (dotnet-gcdump)
The dotnet-trace tool is a cross-platform CLI tool that collects gcdumps of live .NET processes. It is built using the EventPipe technology which is a cross-platform alternative to ETW on Windows. Gcdumps are created by triggering a GC
in the target process, turning on special events, and regenerating the graph of object roots from the event stream. This allows for gcdumps to be collected while the process is running with minimal overhead. These dumps are useful for
several scenarios:
* comparing the number of objects on the heap at several points in time
* analyzing roots of objects (answering questions like, "what still has a reference to this type?")
* collecting general statistics about the counts of objects on the heap.
dotnet-gcdump can be used on Linux, Mac, and Windows with runtime versions 3.1 or newer.
## Installing dotnet-gcdump
The first step is to install the dotnet-gcdump CLI global tool.
```cmd
$ dotnet tool install --global dotnet-gcdump
You can invoke the tool using the following command: dotnet-gcdump
Tool 'dotnet-gcdump' (version '3.0.47001') was successfully installed.
```
## Using dotnet-gcdump
In order to collect gcdumps using dotnet-gcdump, you will need to:
- First, find out the process identifier (pid) of the target .NET application.
- On Windows, there are options such as using the task manager or the `tasklist` command in the cmd prompt.
- On Linux, the trivial option could be using `pidof` in the terminal window.
You may also use the command `dotnet-gcdump ps` command to find out what .NET processes are running, along with their process IDs.
- Then, run the following command:
```cmd
dotnet-gcdump collect --process-id <PID>
Writing gcdump to 'C:\git\diagnostics\src\Tools\dotnet-gcdump\20191023_042913_24060.gcdump'...
Finished writing 486435 bytes.
```
- Note that gcdumps can take several seconds depending on the size of the application
## Viewing the gcdump captured from dotnet-gcdump
On Windows, `.gcdump` files can be viewed in PerfView (https://github.com/microsoft/perfview) for analysis or in Visual Studio. There is not currently a way of opening a `.gcdump` on non-Windows platforms.
You can collect multiple `.gcdump`s and open them simultaneously in Visual Studio to get a comparison experience.
## Known Caveats
- There is no type information in the gcdump
Prior to .NET Core 3.1, there was an issue where a type cache was not cleared between gcdumps when they were invoked with EventPipe. This resulted in the events needed for determining type information not being sent for the second and subsequent gcdumps. This was fixed in .NET Core 3.1-preview2.
- COM and static types aren't in the gcdump
Prior to .NET Core 3.1-preview2, there was an issue where static and COM types weren't sent when the gcdump was invoked via EventPipe. This has been fixed in .NET Core 3.1-preview2.
## *dotnet-gcdump* help
```cmd
collect:
Collects a diagnostic trace from a currently running process
Usage:
dotnet-gcdump collect [options]
Options:
-p, --process-id <pid> The process to collect the trace from
-o, --output <gcdump-file-path> The path where collected gcdumps should be written. Defaults to '.\YYYYMMDD_HHMMSS_<pid>.gcdump'
where YYYYMMDD is Year/Month/Day and HHMMSS is Hour/Minute/Second. Otherwise, it is the full path
and file name of the dump.
-v, --verbose Output the log while collecting the gcdump
```

Просмотреть файл

@ -0,0 +1,106 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using Microsoft.Diagnostics.Tools.RuntimeClient;
using System;
using System.Collections.Generic;
using System.CommandLine;
using System.CommandLine.Binding;
using System.CommandLine.Rendering;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Diagnostics.Tools.GCDump
{
internal static class CollectCommandHandler
{
delegate Task<int> CollectDelegate(CancellationToken ct, IConsole console, int processId, string output, bool verbose);
/// <summary>
/// Collects a gcdump from a currently running process.
/// </summary>
/// <param name="ct">The cancellation token</param>
/// <param name="console"></param>
/// <param name="processId">The process to collect the gcdump from.</param>
/// <param name="output">The output path for the collected gcdump.</param>
/// <returns></returns>
private static async Task<int> Collect(CancellationToken ct, IConsole console, int processId, string output, bool verbose)
{
try
{
output = string.IsNullOrEmpty(output) ?
$"{DateTime.Now.ToString(@"yyyyMMdd\_hhmmss")}_{processId}.gcdump" :
output;
FileInfo outputFileInfo = new FileInfo(output);
if (outputFileInfo.Exists)
{
outputFileInfo.Delete();
}
if (string.IsNullOrEmpty(outputFileInfo.Extension) || outputFileInfo.Extension != ".gcdump")
{
outputFileInfo = new FileInfo(outputFileInfo.FullName + ".gcdump");
}
Console.Out.WriteLine($"Writing gcdump to '{outputFileInfo.FullName}'...");
var dumpTask = Task.Run(() =>
{
var memoryGraph = new Graphs.MemoryGraph(50_000);
var heapInfo = new DotNetHeapInfo();
EventPipeDotNetHeapDumper.DumpFromEventPipe(ct, processId, memoryGraph, verbose ? Console.Out : TextWriter.Null, heapInfo);
memoryGraph.AllowReading();
GCHeapDump.WriteMemoryGraph(memoryGraph, outputFileInfo.FullName, "dotnet-trace");
});
await dumpTask;
outputFileInfo.Refresh();
Console.Out.WriteLine($"\tFinished writing {outputFileInfo.Length} bytes.");
return 0;
}
catch (Exception ex)
{
Console.Error.WriteLine($"[ERROR] {ex.ToString()}");
return -1;
}
}
public static Command CollectCommand() =>
new Command(
name: "collect",
description: "Collects a diagnostic trace from a currently running process",
symbols: new Option[] {
ProcessIdOption(),
OutputPathOption(),
VerboseOption()
},
handler: HandlerDescriptor.FromDelegate((CollectDelegate)Collect).GetCommandHandler());
public static Option ProcessIdOption() =>
new Option(
aliases: new[] { "-p", "--process-id" },
description: "The process to collect the trace from",
argument: new Argument<int> { Name = "pid" },
isHidden: false);
private static Option OutputPathOption() =>
new Option(
aliases: new[] { "-o", "--output" },
description: $@"The path where collected gcdumps should be written. Defaults to '.\YYYYMMDD_HHMMSS_<pid>.gcdump' where YYYYMMDD is Year/Month/Day and HHMMSS is Hour/Minute/Second. Otherwise, it is the full path and file name of the dump.",
argument: new Argument<string>(defaultValue: "") { Name = "gcdump-file-path" },
isHidden: false);
private static Option VerboseOption() =>
new Option(
aliases: new[] { "-v", "--verbose" },
description: $"Output the log while collecting the gcdump",
argument: new Argument<bool>(defaultValue: false) { Name = "verbose" },
isHidden: false);
}
}

Просмотреть файл

@ -0,0 +1,29 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using Microsoft.Diagnostics.Tools.RuntimeClient;
using Microsoft.Internal.Common.Commands;
using System;
using System.CommandLine;
using System.Threading.Tasks;
namespace Microsoft.Diagnostics.Tools.GCDump
{
internal static class ListProcessesCommandHandler
{
public static async Task<int> GetActivePorts(IConsole console)
{
ProcessStatusCommandHandler.PrintProcessStatus(console);
await Task.FromResult(0);
return 0;
}
public static Command ProcessStatusCommand() =>
new Command(
name: "ps",
description: "Lists dotnet processes that can be attached to.",
handler: System.CommandLine.Invocation.CommandHandler.Create<IConsole>(GetActivePorts),
isHidden: false);
}
}

Просмотреть файл

@ -0,0 +1,988 @@
using Graphs;
using Microsoft.Diagnostics.Tracing;
using Microsoft.Diagnostics.Tracing.Parsers;
using Microsoft.Diagnostics.Tracing.Parsers.Clr;
using Microsoft.Diagnostics.Tracing.Parsers.Kernel;
using Microsoft.Diagnostics.Tracing.Parsers.Symbol;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Text.RegularExpressions;
using Address = System.UInt64;
/// <summary>
/// Reads a .NET Heap dump generated from ETW
/// </summary>
public class DotNetHeapDumpGraphReader
{
/// <summary>
/// A class for reading ETW events from the .NET runtime and creating a MemoryGraph from it. This only works on V4.5.1 of the runtime or later.
/// </summary>
/// <param name="log">A place to put diagnostic messages.</param>
public DotNetHeapDumpGraphReader(TextWriter log)
{
m_log = log;
}
/// <summary>
/// Read in the memory dump from javaScriptEtlName. Since there can be more than one, choose the first one
/// after double startTimeRelativeMSec. If processId is non-zero only that process is considered, otherwise it considered
/// the first heap dump regardless of process.
/// </summary>
public MemoryGraph Read(string etlFilePath, string processNameOrId = null, double startTimeRelativeMSec = 0)
{
m_etlFilePath = etlFilePath;
var ret = new MemoryGraph(10000);
Append(ret, etlFilePath, processNameOrId, startTimeRelativeMSec);
ret.AllowReading();
return ret;
}
public MemoryGraph Read(TraceEventDispatcher source, string processNameOrId = null, double startTimeRelativeMSec = 0)
{
var ret = new MemoryGraph(10000);
Append(ret, source, processNameOrId, startTimeRelativeMSec);
ret.AllowReading();
return ret;
}
public void Append(MemoryGraph memoryGraph, string etlName, string processNameOrId = null, double startTimeRelativeMSec = 0)
{
using (var source = TraceEventDispatcher.GetDispatcherFromFileName(etlName))
{
Append(memoryGraph, source, processNameOrId, startTimeRelativeMSec);
}
}
public void Append(MemoryGraph memoryGraph, TraceEventDispatcher source, string processNameOrId = null, double startTimeRelativeMSec = 0)
{
SetupCallbacks(memoryGraph, source, processNameOrId, startTimeRelativeMSec);
source.Process();
ConvertHeapDataToGraph();
}
/// <summary>
/// If set before Read or Append is called, keep track of the additional information about GC generations associated with .NET Heaps.
/// </summary>
public DotNetHeapInfo DotNetHeapInfo
{
get { return m_dotNetHeapInfo; }
set { m_dotNetHeapInfo = value; }
}
#region private
/// <summary>
/// Sets up the callbacks needed to do a heap dump (work need before processing the events()
/// </summary>
internal void SetupCallbacks(MemoryGraph memoryGraph, TraceEventDispatcher source, string processNameOrId = null, double startTimeRelativeMSec = 0)
{
m_graph = memoryGraph;
m_typeID2TypeIndex = new Dictionary<Address, NodeTypeIndex>(1000);
m_moduleID2Name = new Dictionary<Address, string>(16);
m_arrayNametoIndex = new Dictionary<string, NodeTypeIndex>(32);
m_objectToRCW = new Dictionary<Address, RCWInfo>(100);
m_nodeBlocks = new Queue<GCBulkNodeTraceData>();
m_edgeBlocks = new Queue<GCBulkEdgeTraceData>();
m_typeBlocks = new Queue<GCBulkTypeTraceData>();
m_staticVarBlocks = new Queue<GCBulkRootStaticVarTraceData>();
m_ccwBlocks = new Queue<GCBulkRootCCWTraceData>();
m_typeIntern = new Dictionary<string, NodeTypeIndex>();
m_root = new MemoryNodeBuilder(m_graph, "[.NET Roots]");
m_typeStorage = m_graph.AllocTypeNodeStorage();
// We also keep track of the loaded modules in the target process just in case it is a project N scenario.
// (Not play for play but it is small).
m_modules = new Dictionary<Address, Module>(32);
m_ignoreEvents = true;
m_ignoreUntilMSec = startTimeRelativeMSec;
m_processId = 0; // defaults to a wildcard.
if (processNameOrId != null)
{
if (!int.TryParse(processNameOrId, out m_processId))
{
m_processId = -1; // an illegal value.
m_processName = processNameOrId;
}
}
// Remember the module IDs too.
Action<ModuleLoadUnloadTraceData> moduleCallback = delegate (ModuleLoadUnloadTraceData data)
{
if (data.ProcessID != m_processId)
{
return;
}
if (!m_moduleID2Name.ContainsKey((Address)data.ModuleID))
{
m_moduleID2Name[(Address)data.ModuleID] = data.ModuleILPath;
}
m_log.WriteLine("Found Module {0} ID 0x{1:x}", data.ModuleILFileName, (Address)data.ModuleID);
};
source.Clr.AddCallbackForEvents<ModuleLoadUnloadTraceData>(moduleCallback); // Get module events for clr provider
// TODO should not be needed if we use CAPTURE_STATE when collecting.
var clrRundown = new ClrRundownTraceEventParser(source);
clrRundown.AddCallbackForEvents<ModuleLoadUnloadTraceData>(moduleCallback); // and its rundown provider.
DbgIDRSDSTraceData lastDbgData = null;
var symbolParser = new SymbolTraceEventParser(source);
symbolParser.ImageIDDbgID_RSDS += delegate (DbgIDRSDSTraceData data)
{
if (data.ProcessID != m_processId)
{
return;
}
lastDbgData = (DbgIDRSDSTraceData)data.Clone();
};
source.Kernel.ImageGroup += delegate (ImageLoadTraceData data)
{
if (m_processId == 0)
{
return;
}
if (data.ProcessID != m_processId)
{
return;
}
Module module = new Module(data.ImageBase);
module.Path = data.FileName;
module.Size = data.ImageSize;
module.BuildTime = data.BuildTime;
if (lastDbgData != null && data.TimeStampRelativeMSec == lastDbgData.TimeStampRelativeMSec)
{
module.PdbGuid = lastDbgData.GuidSig;
module.PdbAge = lastDbgData.Age;
module.PdbName = lastDbgData.PdbFileName;
}
m_modules[module.ImageBase] = module;
};
// TODO this does not work in the circular case
source.Kernel.ProcessGroup += delegate (ProcessTraceData data)
{
if (0 <= m_processId || m_processName == null)
{
return;
}
if (string.Compare(data.ProcessName, processNameOrId, StringComparison.OrdinalIgnoreCase) == 0)
{
m_log.WriteLine("Found process id {0} for process Name {1}", processNameOrId, data.ProcessName);
m_processId = data.ProcessID;
}
else
{
m_log.WriteLine("Found process {0} but does not match {1}", data.ProcessName, processNameOrId);
}
};
source.Clr.GCStart += delegate (GCStartTraceData data)
{
// If this GC is not part of a heap dump, ignore it.
// TODO FIX NOW if (data.ClientSequenceNumber == 0)
// return;
if (data.TimeStampRelativeMSec < m_ignoreUntilMSec)
{
return;
}
if (m_processId == 0)
{
m_processId = data.ProcessID;
m_log.WriteLine("Process wildcard selects process id {0}", m_processId);
}
if (data.ProcessID != m_processId)
{
m_log.WriteLine("GC Start found but Process ID {0} != {1} desired ID", data.ProcessID, m_processId);
return;
}
if (!IsProjectN && data.ProviderGuid == ClrTraceEventParser.NativeProviderGuid)
{
IsProjectN = true;
}
if (data.Depth < 2 || data.Type != GCType.NonConcurrentGC)
{
m_log.WriteLine("GC Start found but not a Foreground Gen 2 GC");
return;
}
if (data.Reason != GCReason.Induced)
{
m_log.WriteLine("GC Start not induced. Skipping.");
return;
}
if (!m_seenStart)
{
m_gcID = data.Count;
m_log.WriteLine("Found a Gen2 Induced non-background GC Start at {0:n3} msec GC Count {1}", data.TimeStampRelativeMSec, m_gcID);
m_ignoreEvents = false;
m_seenStart = true;
memoryGraph.Is64Bit = (data.PointerSize == 8);
}
};
source.Clr.GCStop += delegate (GCEndTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
if (data.Count == m_gcID)
{
m_log.WriteLine("Found a GC Stop at {0:n3} for GC {1}, ignoring events from now on.", data.TimeStampRelativeMSec, m_gcID);
m_ignoreEvents = true;
if (m_nodeBlocks.Count == 0 && m_typeBlocks.Count == 0 && m_edgeBlocks.Count == 0)
{
m_log.WriteLine("Found no node events, looking for another GC");
m_seenStart = false;
return;
}
// TODO we have to continue processing to get the module rundown events.
// If we could be sure to get these early, we could optimized this.
// source.StopProcessing();
}
else
{
m_log.WriteLine("Found a GC Stop at {0:n3} but id {1} != {2} Target ID", data.TimeStampRelativeMSec, data.Count, m_gcID);
}
};
source.Clr.TypeBulkType += delegate (GCBulkTypeTraceData data)
{
// Don't check m_ignoreEvents here, as BulkType events can be emitted by other events...such as the GC allocation event.
// This means that when setting m_processId to 0 in the command line may still lose type events.
if (data.ProcessID != m_processId)
{
return;
}
m_typeBlocks.Enqueue((GCBulkTypeTraceData)data.Clone());
};
source.Clr.GCBulkNode += delegate (GCBulkNodeTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
m_nodeBlocks.Enqueue((GCBulkNodeTraceData)data.Clone());
};
source.Clr.GCBulkEdge += delegate (GCBulkEdgeTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
m_edgeBlocks.Enqueue((GCBulkEdgeTraceData)data.Clone());
};
source.Clr.GCBulkRootEdge += delegate (GCBulkRootEdgeTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
MemoryNodeBuilder staticRoot = m_root.FindOrCreateChild("[static vars]");
for (int i = 0; i < data.Count; i++)
{
var value = data.Values(i);
var flags = value.GCRootFlag;
if ((flags & GCRootFlags.WeakRef) == 0) // ignore weak references. they are not roots.
{
GCRootKind kind = value.GCRootKind;
MemoryNodeBuilder root = m_root;
string name;
if (kind == GCRootKind.Stack)
{
name = "[local vars]";
}
else
{
root = m_root.FindOrCreateChild("[other roots]");
if ((flags & GCRootFlags.RefCounted) != 0)
{
name = "[COM/WinRT Objects]";
}
else if (kind == GCRootKind.Finalizer)
{
name = "[finalizer Handles]";
}
else if (kind == GCRootKind.Handle)
{
if (flags == GCRootFlags.Pinning)
{
name = "[pinning Handles]";
}
else
{
name = "[strong Handles]";
}
}
else
{
name = "[other Handles]";
}
// Remember the root for later processing.
if (value.RootedNodeAddress != 0)
{
Address gcRootId = value.GCRootID;
if (gcRootId != 0 && IsProjectN)
{
Module gcRootModule = GetModuleForAddress(gcRootId);
if (gcRootModule != null)
{
var staticRva = (int)(gcRootId - gcRootModule.ImageBase);
var staticTypeIdx = m_graph.CreateType(staticRva, gcRootModule, 0, " (static var)");
var staticNodeIdx = m_graph.CreateNode();
m_children.Clear();
m_children.Add(m_graph.GetNodeIndex(value.RootedNodeAddress));
m_graph.SetNode(staticNodeIdx, staticTypeIdx, 0, m_children);
staticRoot.AddChild(staticNodeIdx);
Trace.WriteLine("Got Static 0x" + gcRootId.ToString("x") + " pointing at 0x" + value.RootedNodeAddress.ToString("x") + " kind " + value.GCRootKind + " flags " + value.GCRootFlag);
continue;
}
}
Trace.WriteLine("Got GC Root 0x" + gcRootId.ToString("x") + " pointing at 0x" + value.RootedNodeAddress.ToString("x") + " kind " + value.GCRootKind + " flags " + value.GCRootFlag);
}
}
root = root.FindOrCreateChild(name);
Address objId = value.RootedNodeAddress;
root.AddChild(m_graph.GetNodeIndex(objId));
}
}
};
source.Clr.GCBulkRCW += delegate (GCBulkRCWTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
for (int i = 0; i < data.Count; i++)
{
GCBulkRCWValues comInfo = data.Values(i);
m_objectToRCW[comInfo.ObjectID] = new RCWInfo(comInfo);
}
};
source.Clr.GCBulkRootCCW += delegate (GCBulkRootCCWTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
m_ccwBlocks.Enqueue((GCBulkRootCCWTraceData)data.Clone());
};
source.Clr.GCBulkRootStaticVar += delegate (GCBulkRootStaticVarTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
m_staticVarBlocks.Enqueue((GCBulkRootStaticVarTraceData)data.Clone());
};
source.Clr.GCBulkRootConditionalWeakTableElementEdge += delegate (GCBulkRootConditionalWeakTableElementEdgeTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
var otherRoots = m_root.FindOrCreateChild("[other roots]");
var dependentHandles = otherRoots.FindOrCreateChild("[Dependent Handles]");
for (int i = 0; i < data.Count; i++)
{
var value = data.Values(i);
// TODO fix this so that they you see this as an arc from source to target.
// The target is alive only if the source ID (which is a weak handle) is alive (non-zero)
if (value.GCKeyNodeID != 0)
{
dependentHandles.AddChild(m_graph.GetNodeIndex(value.GCValueNodeID));
}
}
};
source.Clr.GCGenerationRange += delegate (GCGenerationRangeTraceData data)
{
if (m_ignoreEvents || data.ProcessID != m_processId)
{
return;
}
if (m_dotNetHeapInfo == null)
{
return;
}
// We want the 'after' ranges so we wait
if (m_nodeBlocks.Count == 0)
{
return;
}
Address start = data.RangeStart;
Address end = start + data.RangeUsedLength;
if (m_dotNetHeapInfo.Segments == null)
{
m_dotNetHeapInfo.Segments = new List<GCHeapDumpSegment>();
}
GCHeapDumpSegment segment = new GCHeapDumpSegment();
segment.Start = start;
segment.End = end;
switch (data.Generation)
{
case 0:
segment.Gen0End = end;
break;
case 1:
segment.Gen1End = end;
break;
case 2:
segment.Gen2End = end;
break;
case 3:
segment.Gen3End = end;
break;
default:
throw new Exception("Invalid generation in GCGenerationRangeTraceData");
}
m_dotNetHeapInfo.Segments.Add(segment);
};
}
/// <summary>
/// After reading the events the graph is not actually created, you need to post process the information we gathered
/// from the events. This is where that happens. Thus 'SetupCallbacks, Process(), ConvertHeapDataToGraph()' is how
/// you dump a heap.
/// </summary>
internal unsafe void ConvertHeapDataToGraph()
{
int maxNodeCount = 10_000_000;
if (m_converted)
{
return;
}
m_converted = true;
if (!m_seenStart)
{
if (m_processName != null)
{
throw new ApplicationException("ETL file did not include a Heap Dump for process " + m_processName);
}
throw new ApplicationException("ETL file did not include a Heap Dump for process ID " + m_processId);
}
if (!m_ignoreEvents)
{
throw new ApplicationException("ETL file shows the start of a heap dump but not its completion.");
}
m_log.WriteLine("Processing Heap Data, BulkTypeEventCount:{0} BulkNodeEventCount:{1} BulkEdgeEventCount:{2}",
m_typeBlocks.Count, m_nodeBlocks.Count, m_edgeBlocks.Count);
// Process the type information (we can't do it on the fly because we need the module information, which may be
// at the end of the trace.
while (m_typeBlocks.Count > 0)
{
GCBulkTypeTraceData data = m_typeBlocks.Dequeue();
for (int i = 0; i < data.Count; i++)
{
GCBulkTypeValues typeData = data.Values(i);
var typeName = typeData.TypeName;
if (IsProjectN)
{
// For project N we only log the type ID and module base address.
Debug.Assert(typeName.Length == 0);
Debug.Assert((typeData.Flags & TypeFlags.ModuleBaseAddress) != 0);
var moduleBaseAddress = typeData.TypeID - (ulong)typeData.TypeNameID; // Tricky way of getting the image base.
Debug.Assert((moduleBaseAddress & 0xFFFF) == 0); // Image loads should be on 64K boundaries.
Module module = GetModuleForImageBase(moduleBaseAddress);
if (module.Path == null)
{
m_log.WriteLine("Error: Could not find DLL name for imageBase 0x{0:x} looking up typeID 0x{1:x} with TypeNameID {2:x}",
moduleBaseAddress, typeData.TypeID, typeData.TypeNameID);
}
m_typeID2TypeIndex[typeData.TypeID] = m_graph.CreateType(typeData.TypeNameID, module);
}
else
{
if (typeName.Length == 0)
{
if ((typeData.Flags & TypeFlags.Array) != 0)
{
typeName = "ArrayType(0x" + typeData.TypeNameID.ToString("x") + ")";
}
else
{
typeName = "Type(0x" + typeData.TypeNameID.ToString("x") + ")";
}
}
// TODO FIX NOW these are kind of hacks
typeName = Regex.Replace(typeName, @"`\d+", "");
typeName = typeName.Replace("[", "<");
typeName = typeName.Replace("]", ">");
typeName = typeName.Replace("<>", "[]");
string moduleName;
if (!m_moduleID2Name.TryGetValue(typeData.ModuleID, out moduleName))
{
moduleName = "Module(0x" + typeData.ModuleID.ToString("x") + ")";
m_moduleID2Name[typeData.ModuleID] = moduleName;
}
// Is this type a an RCW? If so mark the type name that way.
if ((typeData.Flags & TypeFlags.ExternallyImplementedCOMObject) != 0)
{
typeName = "[RCW " + typeName + "]";
}
m_typeID2TypeIndex[typeData.TypeID] = CreateType(typeName, moduleName);
// Trace.WriteLine(string.Format("Type 0x{0:x} = {1}", typeData.TypeID, typeName));
}
}
}
// Process all the ccw root information (which also need the type information complete)
var ccwRoot = m_root.FindOrCreateChild("[COM/WinRT Objects]");
while (m_ccwBlocks.Count > 0)
{
GCBulkRootCCWTraceData data = m_ccwBlocks.Dequeue();
GrowableArray<NodeIndex> ccwChildren = new GrowableArray<NodeIndex>(1);
for (int i = 0; i < data.Count; i++)
{
unsafe
{
GCBulkRootCCWValues ccwInfo = data.Values(i);
// TODO Debug.Assert(ccwInfo.IUnknown != 0);
if (ccwInfo.IUnknown == 0)
{
// TODO currently there are times when a CCWs IUnknown pointer is not set (it is set lazily).
// m_log.WriteLine("Warning seen a CCW with IUnknown == 0");
continue;
}
// Create a CCW node that represents the COM object that has one child that points at the managed object.
var ccwNode = m_graph.GetNodeIndex(ccwInfo.IUnknown);
var ccwTypeIndex = GetTypeIndex(ccwInfo.TypeID, 200);
var ccwType = m_graph.GetType(ccwTypeIndex, m_typeStorage);
var typeName = "[CCW 0x" + ccwInfo.IUnknown.ToString("x") + " for type " + ccwType.Name + "]";
ccwTypeIndex = CreateType(typeName);
ccwChildren.Clear();
ccwChildren.Add(m_graph.GetNodeIndex(ccwInfo.ObjectID));
m_graph.SetNode(ccwNode, ccwTypeIndex, 200, ccwChildren);
ccwRoot.AddChild(ccwNode);
}
}
}
// Process all the static variable root information (which also need the module information complete
var staticVarsRoot = m_root.FindOrCreateChild("[static vars]");
while (m_staticVarBlocks.Count > 0)
{
GCBulkRootStaticVarTraceData data = m_staticVarBlocks.Dequeue();
for (int i = 0; i < data.Count; i++)
{
GCBulkRootStaticVarValues staticVarData = data.Values(i);
var rootToAddTo = staticVarsRoot;
if ((staticVarData.Flags & GCRootStaticVarFlags.ThreadLocal) != 0)
{
rootToAddTo = m_root.FindOrCreateChild("[thread static vars]");
}
// Get the type name.
NodeTypeIndex typeIdx;
string typeName;
if (m_typeID2TypeIndex.TryGetValue(staticVarData.TypeID, out typeIdx))
{
var type = m_graph.GetType(typeIdx, m_typeStorage);
typeName = type.Name;
}
else
{
typeName = "Type(0x" + staticVarData.TypeID.ToString("x") + ")";
}
string fullFieldName = typeName + "." + staticVarData.FieldName;
rootToAddTo = rootToAddTo.FindOrCreateChild("[static var " + fullFieldName + "]");
var nodeIdx = m_graph.GetNodeIndex(staticVarData.ObjectID);
rootToAddTo.AddChild(nodeIdx);
}
}
// var typeStorage = m_graph.AllocTypeNodeStorage();
GCBulkNodeUnsafeNodes nodeStorage = new GCBulkNodeUnsafeNodes();
// Process all the node and edge nodes we have collected.
bool doCompletionCheck = true;
for (; ; )
{
GCBulkNodeUnsafeNodes* node = GetNextNode(&nodeStorage);
if (node == null)
{
break;
}
// Get the node index
var nodeIdx = m_graph.GetNodeIndex((Address)node->Address);
var objSize = (int)node->Size;
Debug.Assert(node->Size < 0x1000000000);
var typeIdx = GetTypeIndex(node->TypeID, objSize);
// TODO FIX NOW REMOVE
// var type = m_graph.GetType(typeIdx, typeStorage);
// Trace.WriteLine(string.Format("Got Object 0x{0:x} Type {1} Size {2} #children {3} nodeIdx {4}", (Address)node->Address, type.Name, objSize, node->EdgeCount, nodeIdx));
// Process the edges (which can add children)
m_children.Clear();
for (int i = 0; i < node->EdgeCount; i++)
{
Address edge = GetNextEdge();
var childIdx = m_graph.GetNodeIndex(edge);
m_children.Add(childIdx);
// Trace.WriteLine(string.Format(" Child 0x{0:x}", edge));
}
// TODO we can use the nodes type to see if this is an RCW before doing this lookup which may be a bit more efficient.
RCWInfo info;
if (m_objectToRCW.TryGetValue((Address)node->Address, out info))
{
// Add the COM object this RCW points at as a child of this node.
m_children.Add(m_graph.GetNodeIndex(info.IUnknown));
// We add 1000 to account for the overhead of the RCW that is NOT on the GC heap.
objSize += 1000;
}
Debug.Assert(!m_graph.IsDefined(nodeIdx));
m_graph.SetNode(nodeIdx, typeIdx, objSize, m_children);
if (m_graph.NodeCount >= maxNodeCount)
{
doCompletionCheck = false;
var userMessage = string.Format("Exceeded max node count {0}", maxNodeCount);
m_log.WriteLine("[WARNING: ]", userMessage);
break;
}
}
if (doCompletionCheck && m_curEdgeBlock != null && m_curEdgeBlock.Count != m_curEdgeIdx)
{
throw new ApplicationException("Error: extra edge data. Giving up on heap dump.");
}
m_root.Build();
m_graph.RootIndex = m_root.Index;
}
/// <summary>
/// Given a module image base, return a Module instance that has all the information we have on it.
/// </summary>
private Module GetModuleForImageBase(Address moduleBaseAddress)
{
Module module;
if (!m_modules.TryGetValue(moduleBaseAddress, out module))
{
module = new Module(moduleBaseAddress);
m_modules.Add(moduleBaseAddress, module);
}
if (module.PdbName == null && module.Path != null)
{
m_log.WriteLine("No PDB information for {0} in ETL file, looking for it directly", module.Path);
if (File.Exists(module.Path))
{
using (var modulePEFile = new PEFile.PEFile(module.Path))
{
if (!modulePEFile.GetPdbSignature(out module.PdbName, out module.PdbGuid, out module.PdbAge))
{
m_log.WriteLine("Could not get PDB information for {0}", module.Path);
}
}
}
}
return module;
}
/// <summary>
/// if 'addressInModule' points inside any loaded module return that module. Otherwise return null
/// </summary>
private Module GetModuleForAddress(Address addressInModule)
{
if (m_lastModule != null && m_lastModule.ImageBase <= addressInModule && addressInModule < m_lastModule.ImageBase + (uint)m_lastModule.Size)
{
return m_lastModule;
}
foreach (Module module in m_modules.Values)
{
if (module.ImageBase <= addressInModule && addressInModule < module.ImageBase + (uint)module.Size)
{
m_lastModule = module;
return module;
}
}
return null;
}
private Module m_lastModule; // one element cache
private unsafe GCBulkNodeUnsafeNodes* GetNextNode(GCBulkNodeUnsafeNodes* buffer)
{
if (m_curNodeBlock == null || m_curNodeBlock.Count <= m_curNodeIdx)
{
m_curNodeBlock = null;
if (m_nodeBlocks.Count == 0)
{
return null;
}
var nextBlock = m_nodeBlocks.Dequeue();
if (m_curNodeBlock != null && nextBlock.Index != m_curNodeBlock.Index + 1)
{
throw new ApplicationException("Error expected Node Index " + (m_curNodeBlock.Index + 1) + " Got " + nextBlock.Index + " Giving up on heap dump.");
}
m_curNodeBlock = nextBlock;
m_curNodeIdx = 0;
}
return m_curNodeBlock.UnsafeNodes(m_curNodeIdx++, buffer);
}
private Address GetNextEdge()
{
if (m_curEdgeBlock == null || m_curEdgeBlock.Count <= m_curEdgeIdx)
{
m_curEdgeBlock = null;
if (m_edgeBlocks.Count == 0)
{
throw new ApplicationException("Error not enough edge data. Giving up on heap dump.");
}
var nextEdgeBlock = m_edgeBlocks.Dequeue();
if (m_curEdgeBlock != null && nextEdgeBlock.Index != m_curEdgeBlock.Index + 1)
{
throw new ApplicationException("Error expected Node Index " + (m_curEdgeBlock.Index + 1) + " Got " + nextEdgeBlock.Index + " Giving up on heap dump.");
}
m_curEdgeBlock = nextEdgeBlock;
m_curEdgeIdx = 0;
}
return m_curEdgeBlock.Values(m_curEdgeIdx++).Target;
}
private NodeTypeIndex GetTypeIndex(Address typeID, int objSize)
{
NodeTypeIndex ret;
if (!m_typeID2TypeIndex.TryGetValue(typeID, out ret))
{
m_log.WriteLine("Error: Did not have a type definition for typeID 0x{0:x}", typeID);
Trace.WriteLine(string.Format("Error: Did not have a type definition for typeID 0x{0:x}", typeID));
var typeName = "UNKNOWN 0x" + typeID.ToString("x");
ret = CreateType(typeName);
m_typeID2TypeIndex[typeID] = ret;
}
if (objSize > 1000)
{
var type = m_graph.GetType(ret, m_typeStorage);
var suffix = GetObjectSizeSuffix(objSize); // indicates the size range
var typeName = type.Name + suffix;
// TODO FIX NOW worry about module collision
if (!m_arrayNametoIndex.TryGetValue(typeName, out ret))
{
if (IsProjectN)
{
ret = m_graph.CreateType(type.RawTypeID, type.Module, objSize, suffix);
}
else
{
ret = CreateType(typeName, type.ModuleName);
}
m_arrayNametoIndex.Add(typeName, ret);
}
}
return ret;
}
// Returns a string suffix that discriminates interesting size ranges.
private static string GetObjectSizeSuffix(int objSize)
{
if (objSize < 1000)
{
return "";
}
string size;
if (objSize < 10000)
{
size = "1K";
}
else if (objSize < 100000)
{
size = "10K";
}
else if (objSize < 1000000)
{
size = "100K";
}
else if (objSize < 10000000)
{
size = "1M";
}
else if (objSize < 100000000)
{
size = "10M";
}
else
{
size = "100M";
}
return " (Bytes > " + size + ")";
}
private NodeTypeIndex CreateType(string typeName, string moduleName = null)
{
var fullTypeName = typeName;
if (moduleName != null)
{
fullTypeName = moduleName + "!" + typeName;
}
NodeTypeIndex ret;
if (!m_typeIntern.TryGetValue(fullTypeName, out ret))
{
ret = m_graph.CreateType(typeName, moduleName);
m_typeIntern.Add(fullTypeName, ret);
}
return ret;
}
/// <summary>
/// Converts a raw TypeID (From the ETW data), to the graph type index)
/// </summary>
private Dictionary<Address, NodeTypeIndex> m_typeID2TypeIndex;
private Dictionary<Address, string> m_moduleID2Name;
private Dictionary<string, NodeTypeIndex> m_arrayNametoIndex;
/// <summary>
/// Remembers addition information about RCWs.
/// </summary>
private class RCWInfo
{
public RCWInfo(GCBulkRCWValues data) { IUnknown = data.IUnknown; }
public Address IUnknown;
};
private Dictionary<Address, RCWInfo> m_objectToRCW;
/// <summary>
/// We gather all the BulkTypeTraceData into a list m_typeBlocks which we then process as a second pass (because we need module info which may be after the type info).
/// </summary>
private Queue<GCBulkTypeTraceData> m_typeBlocks;
/// <summary>
/// We gather all the BulkTypeTraceData into a list m_typeBlocks which we then process as a second pass (because we need module info which may be after the type info).
/// </summary>
private Queue<GCBulkRootStaticVarTraceData> m_staticVarBlocks;
/// <summary>
/// We gather all the GCBulkRootCCWTraceData into a list m_ccwBlocks which we then process as a second pass (because we need type info which may be after the ccw info).
/// </summary>
private Queue<GCBulkRootCCWTraceData> m_ccwBlocks;
/// <summary>
/// We gather all the GCBulkNodeTraceData events into a list m_nodeBlocks. m_curNodeBlock is the current block we are processing and 'm_curNodeIdx' is the node within the event
/// </summary>
private Queue<GCBulkNodeTraceData> m_nodeBlocks;
private GCBulkNodeTraceData m_curNodeBlock;
private int m_curNodeIdx;
/// <summary>
/// We gather all the GCBulkEdgeTraceData events into a list m_edgeBlocks. m_curEdgeBlock is the current block we are processing and 'm_curEdgeIdx' is the edge within the event
/// </summary>
private Queue<GCBulkEdgeTraceData> m_edgeBlocks;
private int m_curEdgeIdx;
private GCBulkEdgeTraceData m_curEdgeBlock;
/// <summary>
/// We want type indexes to be shared as much as possible, so this table remembers the ones we have already created.
/// </summary>
private Dictionary<string, NodeTypeIndex> m_typeIntern;
// scratch location for creating nodes.
private GrowableArray<NodeIndex> m_children;
// This is a 'scratch location' we use to fetch type information.
private NodeType m_typeStorage;
// m_modules is populated as types are defined, and then we look up all the necessary module info later.
private Dictionary<Address, Module> m_modules; // Currently only non-null if it is a project N heap dump
private bool IsProjectN; // only set after we see the GCStart
// Information from the constructor
private string m_etlFilePath;
private double m_ignoreUntilMSec; // ignore until we see this
private int m_processId;
private string m_processName;
private TextWriter m_log;
// State that lets up pick the particular heap dump int the ETL file and ignore the rest.
private bool m_converted;
private bool m_seenStart;
private bool m_ignoreEvents;
private int m_gcID;
// The graph we generating.
private MemoryGraph m_graph;
private MemoryNodeBuilder m_root; // Used to create pseduo-nodes for the roots of the graph.
// Heap information for .NET heaps.
private DotNetHeapInfo m_dotNetHeapInfo;
#endregion
}

Просмотреть файл

@ -0,0 +1,140 @@
using FastSerialization;
using System.Collections.Generic;
using Address = System.UInt64;
public class DotNetHeapInfo : IFastSerializable
{
/// <summary>
/// If we could not properly walk an object, this is incremented.
/// Hopefully this is zero.
/// </summary>
public int CorruptedObject { get; internal set; }
/// <summary>
/// This is the number of bytes we had to skip because of errors walking the segments.
/// </summary>
public long UndumpedSegementRegion { get; internal set; }
/// <summary>
/// This is the sum of all space in the GC segments.
/// </summary>
public long SizeOfAllSegments { get; internal set; }
/// <summary>
/// The memory regions that user objects can be allocated from
/// </summary>
public List<GCHeapDumpSegment> Segments { get; internal set; }
/// <summary>
/// Given an object, determine what GC generation it is in. Gen 3 is the large object heap
/// returns -1 if the object is not in any GC segment.
/// </summary>
public int GenerationFor(Address obj)
{
// Find the segment
if ((m_lastSegment == null) || !(m_lastSegment.Start <= obj && obj < m_lastSegment.End))
{
if (Segments == null)
{
return -1;
}
for (int i = 0; ; i++)
{
if (i >= Segments.Count)
{
return -1;
}
var segment = Segments[i];
if (segment.Start <= obj && obj < segment.End)
{
m_lastSegment = segment;
break;
}
}
}
if (obj < m_lastSegment.Gen3End)
{
return 3;
}
if (obj < m_lastSegment.Gen2End)
{
return 2;
}
if (obj < m_lastSegment.Gen1End)
{
return 1;
}
if (obj < m_lastSegment.Gen0End)
{
return 0;
}
return -1;
}
#region private
void IFastSerializable.ToStream(Serializer serializer)
{
serializer.Write(SizeOfAllSegments);
if (Segments != null)
{
serializer.Write(Segments.Count);
foreach (var segment in Segments)
{
serializer.Write(segment);
}
}
else
{
serializer.Write(0);
}
}
void IFastSerializable.FromStream(Deserializer deserializer)
{
SizeOfAllSegments = deserializer.ReadInt64();
var count = deserializer.ReadInt();
Segments = new List<GCHeapDumpSegment>(count);
for (int i = 0; i < count; i++)
{
Segments.Add((GCHeapDumpSegment)deserializer.ReadObject());
}
}
private GCHeapDumpSegment m_lastSegment; // cache for GenerationFor
#endregion
}
public class GCHeapDumpSegment : IFastSerializable
{
public Address Start { get; internal set; }
public Address End { get; internal set; }
public Address Gen0End { get; internal set; }
public Address Gen1End { get; internal set; }
public Address Gen2End { get; internal set; }
public Address Gen3End { get; internal set; }
#region private
void IFastSerializable.ToStream(Serializer serializer)
{
serializer.Write((long)Start);
serializer.Write((long)End);
serializer.Write((long)Gen0End);
serializer.Write((long)Gen1End);
serializer.Write((long)Gen2End);
serializer.Write((long)Gen3End);
}
void IFastSerializable.FromStream(Deserializer deserializer)
{
Start = (Address)deserializer.ReadInt64();
End = (Address)deserializer.ReadInt64();
Gen0End = (Address)deserializer.ReadInt64();
Gen1End = (Address)deserializer.ReadInt64();
Gen2End = (Address)deserializer.ReadInt64();
Gen3End = (Address)deserializer.ReadInt64();
}
#endregion
}

Просмотреть файл

@ -0,0 +1,227 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using Graphs;
using Microsoft.Diagnostics.Tools.RuntimeClient;
using Microsoft.Diagnostics.Tracing;
using Microsoft.Diagnostics.Tracing.Parsers;
using Microsoft.Diagnostics.Tracing.Parsers.Clr;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Microsoft.Diagnostics.Tools.GCDump
{
public static class EventPipeDotNetHeapDumper
{
/// <summary>
/// Given a factory for creating an EventPipe session with the appropriate provider and keywords turned on,
/// generate a GCHeapDump using the resulting events. The correct keywords and provider name
/// are given as input to the Func eventPipeEventSourceFactory.
/// </summary>
/// <param name="processID"></param>
/// <param name="eventPipeEventSourceFactory">A delegate for creating and stopping EventPipe sessions</param>
/// <param name="memoryGraph"></param>
/// <param name="log"></param>
/// <param name="dotNetInfo"></param>
/// <returns></returns>
public static bool DumpFromEventPipe(CancellationToken ct, int processID, MemoryGraph memoryGraph, TextWriter log, DotNetHeapInfo dotNetInfo = null)
{
var sw = Stopwatch.StartNew();
var dumper = new DotNetHeapDumpGraphReader(log)
{
DotNetHeapInfo = dotNetInfo
};
bool dumpComplete = false;
bool listening = false;
EventPipeSession gcDumpSession = null;
Task readerTask = null;
try
{
bool eventPipeDataPresent = false;
TimeSpan lastEventPipeUpdate = sw.Elapsed;
EventPipeSession typeFlushSession = null;
bool fDone = false;
var otherListening = false;
log.WriteLine("{0,5:n1}s: Creating type table flushing task", sw.Elapsed.TotalSeconds);
var typeTableFlushTask = Task.Factory.StartNew(() =>
{
typeFlushSession = new EventPipeSession(processID, new List<Provider> { new Provider("Microsoft-DotNETCore-SampleProfiler") }, false);
otherListening = true;
log.WriteLine("{0,5:n1}s: Flushing the type table", sw.Elapsed.TotalSeconds);
typeFlushSession.Source.AllEvents += Task.Run(() =>
{
if (!fDone)
{
fDone = true;
typeFlushSession.EndSession();
}
});
typeFlushSession.Source.Process();
log.WriteLine("{0,5:n1}s: Done flushing the type table", sw.Elapsed.TotalSeconds);
});
await typeTableFlushTask;
// Set up a separate thread that will listen for EventPipe events coming back telling us we succeeded.
readerTask = Task.Factory.StartNew(delegate
{
// Start the providers and trigger the GCs.
log.WriteLine("{0,5:n1}s: Requesting a .NET Heap Dump", sw.Elapsed.TotalSeconds);
gcDumpSession = new EventPipeSession(processID, new List<Provider> { new Provider("Microsoft-Windows-DotNETRuntime", (ulong)(ClrTraceEventParser.Keywords.GCHeapSnapshot)) });
int gcNum = -1;
gcDumpSession.Source.Clr.GCStart += delegate (GCStartTraceData data)
{
if (data.ProcessID != processID)
{
return;
}
eventPipeDataPresent = true;
if (gcNum < 0 && data.Depth == 2 && data.Type != GCType.BackgroundGC)
{
gcNum = data.Count;
log.WriteLine("{0,5:n1}s: .NET Dump Started...", sw.Elapsed.TotalSeconds);
}
};
gcDumpSession.Source.Clr.GCStop += delegate (GCEndTraceData data)
{
if (data.ProcessID != processID)
{
return;
}
if (data.Count == gcNum)
{
log.WriteLine("{0,5:n1}s: .NET GC Complete.", sw.Elapsed.TotalSeconds);
dumpComplete = true;
}
};
gcDumpSession.Source.Clr.GCBulkNode += delegate (GCBulkNodeTraceData data)
{
if (data.ProcessID != processID)
{
return;
}
eventPipeDataPresent = true;
if ((sw.Elapsed - lastEventPipeUpdate).TotalMilliseconds > 500)
{
log.WriteLine("{0,5:n1}s: Making GC Heap Progress...", sw.Elapsed.TotalSeconds);
}
lastEventPipeUpdate = sw.Elapsed;
};
if (memoryGraph != null)
{
dumper.SetupCallbacks(memoryGraph, gcDumpSession.Source, processID.ToString());
}
listening = true;
gcDumpSession.Source.Process();
log.WriteLine("{0,5:n1}s: EventPipe Listener dying", sw.Elapsed.TotalSeconds);
});
// Wait for thread above to start listening (should be very fast)
while (!listening)
{
readerTask.Wait(1);
}
for (; ; )
{
if (ct.IsCancellationRequested)
{
break;
}
if (readerTask.Wait(100))
{
break;
}
if (!eventPipeDataPresent && sw.Elapsed.TotalSeconds > 5) // Assume it started within 5 seconds.
{
log.WriteLine("{0,5:n1}s: Assume no .NET Heap", sw.Elapsed.TotalSeconds);
break;
}
if (sw.Elapsed.TotalSeconds > 30) // Time out after 30 seconds.
{
log.WriteLine("{0,5:n1}s: Timed out after 20 seconds", sw.Elapsed.TotalSeconds);
break;
}
if (dumpComplete)
{
break;
}
}
log.WriteLine("{0,5:n1}s: Shutting down EventPipe session", sw.Elapsed.TotalSeconds);
gcDumpSession.EndSession();
while (!readerTask.Wait(1000))
log.WriteLine("{0,5:n1}s: still reading...", sw.Elapsed.TotalSeconds);
if (eventPipeDataPresent)
{
dumper.ConvertHeapDataToGraph(); // Finish the conversion.
}
}
catch (Exception e)
{
log.WriteLine($"{sw.Elapsed.TotalSeconds:0,5:n1}s: [Error] Exception during gcdump: {e.ToString()}");
}
log.WriteLine("[{0,5:n1}s: Done Dumping .NET heap success={1}]", sw.Elapsed.TotalSeconds, dumpComplete);
return dumpComplete;
}
}
internal class EventPipeSession
{
private List<Provider> _providers;
private Stream _eventPipeStream;
private EventPipeEventSource _source;
private ulong _sessionId;
private int _pid;
public ulong SessionId => _sessionId;
public IReadOnlyList<Provider> Providers => _providers.AsReadOnly();
public EventPipeEventSource Source => _source;
public EventPipeSession(int pid, List<Provider> providers, bool requestRundown = true)
{
_pid = pid;
_providers = providers;
var config = new SessionConfigurationV2(
circularBufferSizeMB: 1024,
format: EventPipeSerializationFormat.NetTrace,
requestRundown: requestRundown,
providers
);
_eventPipeStream = EventPipeClient.CollectTracing2(pid, config, out _sessionId);
_source = new EventPipeEventSource(_eventPipeStream);
}
public void EndSession()
{
EventPipeClient.StopTracing(_pid, _sessionId);
}
}
}

Разница между файлами не показана из-за своего большого размера Загрузить разницу

Разница между файлами не показана из-за своего большого размера Загрузить разницу

Просмотреть файл

@ -0,0 +1,339 @@
using FastSerialization;
using System.Collections.Generic;
using System.Diagnostics;
using Address = System.UInt64;
// Copy of version in Microsoft/PerfView
namespace Graphs
{
public class MemoryGraph : Graph, IFastSerializable
{
public MemoryGraph(int expectedSize)
: base(expectedSize)
{
m_addressToNodeIndex = new Dictionary<Address, NodeIndex>(expectedSize);
m_nodeAddresses = new SegmentedList<Address>(SegmentSize, expectedSize);
}
public void WriteAsBinaryFile(string outputFileName)
{
Serializer serializer = new Serializer(outputFileName, this);
serializer.Close();
}
public static MemoryGraph ReadFromBinaryFile(string inputFileName)
{
Deserializer deserializer = new Deserializer(inputFileName);
deserializer.TypeResolver = typeName => System.Type.GetType(typeName); // resolve types in this assembly (and mscorlib)
deserializer.RegisterFactory(typeof(MemoryGraph), delegate () { return new MemoryGraph(1); });
deserializer.RegisterFactory(typeof(Graphs.Module), delegate () { return new Graphs.Module(0); });
return (MemoryGraph)deserializer.GetEntryObject();
}
/// <summary>
/// Indicates whether the memory addresses are 64 bit or not. Note that this is not set
/// as part of normal graph processing, it needs to be set by the caller. MemoryGraph is only
/// acting as storage.
/// </summary>
public bool Is64Bit { get; set; }
public Address GetAddress(NodeIndex nodeIndex)
{
if (nodeIndex == NodeIndex.Invalid)
{
return 0;
}
return m_nodeAddresses[(int)nodeIndex];
}
public void SetAddress(NodeIndex nodeIndex, Address nodeAddress)
{
Debug.Assert(m_nodeAddresses[(int)nodeIndex] == 0, "Calling SetAddress twice for node index " + nodeIndex);
m_nodeAddresses[(int)nodeIndex] = nodeAddress;
}
public override NodeIndex CreateNode()
{
var ret = base.CreateNode();
m_nodeAddresses.Add(0);
Debug.Assert(m_nodeAddresses.Count == m_nodes.Count);
return ret;
}
public override Node AllocNodeStorage()
{
return new MemoryNode(this);
}
public override long SizeOfGraphDescription()
{
return base.SizeOfGraphDescription() + 8 * m_nodeAddresses.Count;
}
/// <summary>
/// Returns the number of distinct references in the graph so far (the size of the interning table).
/// </summary>
public int DistinctRefCount { get { return m_addressToNodeIndex.Count; } }
#region protected
/// <summary>
/// Clear puts it back into the state that existed after the constructor returned
/// </summary>
protected override void Clear()
{
base.Clear();
m_addressToNodeIndex.Clear();
m_nodeAddresses.Count = 0;
}
public override void AllowReading()
{
m_addressToNodeIndex = null; // We are done with this, abandon it.
base.AllowReading();
}
/// <summary>
/// GetNodeIndex maps an Memory address of an object (used by CLRProfiler), to the NodeIndex we have assigned to it
/// It is essentially an interning table (we assign it new index if we have not seen it before)
/// </summary>
public NodeIndex GetNodeIndex(Address objectAddress)
{
NodeIndex nodeIndex;
if (!m_addressToNodeIndex.TryGetValue(objectAddress, out nodeIndex))
{
nodeIndex = CreateNode();
m_nodeAddresses[(int)nodeIndex] = objectAddress;
m_addressToNodeIndex.Add(objectAddress, nodeIndex);
}
Debug.Assert(m_nodeAddresses[(int)nodeIndex] == objectAddress);
return nodeIndex;
}
public bool IsInGraph(Address objectAddress)
{
return m_addressToNodeIndex.ContainsKey(objectAddress);
}
/// <summary>
/// ClrProfiler identifes nodes using the physical address in Memory. 'Graph' needs it to be an NodeIndex.
/// THis table maps the ID that CLRProfiler uses (an address), to the NodeIndex we have assigned to it.
/// It is only needed while the file is being read in.
/// </summary>
protected Dictionary<Address, NodeIndex> m_addressToNodeIndex; // This field is only used during construction
#endregion
#region private
void IFastSerializable.ToStream(Serializer serializer)
{
base.ToStream(serializer);
// Write out the Memory addresses of each object
serializer.Write(m_nodeAddresses.Count);
for (int i = 0; i < m_nodeAddresses.Count; i++)
{
serializer.Write((long)m_nodeAddresses[i]);
}
serializer.WriteTagged(Is64Bit);
}
void IFastSerializable.FromStream(Deserializer deserializer)
{
base.FromStream(deserializer);
// Read in the Memory addresses of each object
int addressCount = deserializer.ReadInt();
m_nodeAddresses = new SegmentedList<Address>(SegmentSize, addressCount);
for (int i = 0; i < addressCount; i++)
{
m_nodeAddresses.Add((Address)deserializer.ReadInt64());
}
bool is64bit = false;
deserializer.TryReadTagged(ref is64bit);
Is64Bit = is64bit;
}
// This array survives after the constructor completes
// TODO Fold this into the existing blob. Currently this dominates the Size cost of the graph!
protected SegmentedList<Address> m_nodeAddresses;
#endregion
}
/// <summary>
/// Support class for code:MemoryGraph
/// </summary>
public class MemoryNode : Node
{
public Address Address { get { return m_memoryGraph.GetAddress(Index); } }
#region private
internal MemoryNode(MemoryGraph graph)
: base(graph)
{
m_memoryGraph = graph;
}
public override void WriteXml(System.IO.TextWriter writer, bool includeChildren = true, string prefix = "", NodeType typeStorage = null, string additinalAttribs = "")
{
Address end = Address + (uint)Size;
// base.WriteXml(writer, prefix, storage, typeStorage, additinalAttribs + " Address=\"0x" + Address.ToString("x") + "\"");
base.WriteXml(writer, includeChildren, prefix, typeStorage,
additinalAttribs + " Address=\"0x" + Address.ToString("x") + "\""
+ " End=\"0x" + end.ToString("x") + "\"");
}
private MemoryGraph m_memoryGraph;
#endregion
}
/// <summary>
/// MemoryNodeBuilder is helper class for building a MemoryNode graph. Unlike
/// MemoryNode you don't have to know the complete set of children at the time
/// you create the node. Instead you can keep adding children to it incrementally
/// and when you are done you call Build() which finalizes it (and all its children)
/// </summary>
public class MemoryNodeBuilder
{
public MemoryNodeBuilder(MemoryGraph graph, string typeName, string moduleName = null, NodeIndex nodeIndex = NodeIndex.Invalid)
{
Debug.Assert(typeName != null);
m_graph = graph;
TypeName = typeName;
Index = nodeIndex;
if (Index == NodeIndex.Invalid)
{
Index = m_graph.CreateNode();
}
Debug.Assert(m_graph.m_nodes[(int)Index] == m_graph.m_undefinedObjDef, "SetNode cannot be called on the nodeIndex passed");
ModuleName = moduleName;
m_mutableChildren = new List<MemoryNodeBuilder>();
m_typeIndex = NodeTypeIndex.Invalid;
}
public string TypeName { get; private set; }
public string ModuleName { get; private set; }
public int Size { get; set; }
public NodeIndex Index { get; private set; }
/// <summary>
/// Looks for a child with the type 'childTypeName' and returns it. If it is not
/// present, it will be created. Note it will ONLY find MutableNode children
/// (not children added with AddChild(NodeIndex).
/// </summary>
public MemoryNodeBuilder FindOrCreateChild(string childTypeName, string childModuleName = null)
{
foreach (var child in m_mutableChildren)
{
if (child.TypeName == childTypeName)
{
return child;
}
}
var ret = new MemoryNodeBuilder(m_graph, childTypeName, childModuleName);
AddChild(ret);
return ret;
}
public void AddChild(MemoryNodeBuilder child)
{
m_unmutableChildren.Add(child.Index);
m_mutableChildren.Add(child);
}
public void AddChild(NodeIndex child)
{
m_unmutableChildren.Add(child);
}
/// <summary>
/// This is optional phase, if you don't do it explicitly, it gets done at Build time.
/// </summary>
public void AllocateTypeIndexes()
{
AllocateTypeIndexes(new Dictionary<string, NodeTypeIndex>());
}
public NodeIndex Build()
{
if (m_typeIndex == NodeTypeIndex.Invalid)
{
AllocateTypeIndexes();
}
if (m_mutableChildren != null)
{
Debug.Assert(m_unmutableChildren.Count >= m_mutableChildren.Count);
m_graph.SetNode(Index, m_typeIndex, Size, m_unmutableChildren);
var mutableChildren = m_mutableChildren;
m_mutableChildren = null; // Signals that I have been built
foreach (var child in mutableChildren)
{
child.Build();
}
}
return Index;
}
#region private
private void AllocateTypeIndexes(Dictionary<string, NodeTypeIndex> types)
{
if (m_mutableChildren != null)
{
Debug.Assert(m_unmutableChildren.Count >= m_mutableChildren.Count);
if (!types.TryGetValue(TypeName, out m_typeIndex))
{
m_typeIndex = m_graph.CreateType(TypeName, ModuleName);
types.Add(TypeName, m_typeIndex);
}
foreach (var child in m_mutableChildren)
{
child.AllocateTypeIndexes(types);
}
}
}
private NodeTypeIndex m_typeIndex;
private List<MemoryNodeBuilder> m_mutableChildren;
private GrowableArray<NodeIndex> m_unmutableChildren;
private MemoryGraph m_graph;
#endregion
}
}
#if false
namespace Graphs.Samples
{
class Sample
{
static void Main()
{
int expectedNumberOfNodes = 1000;
MemoryGraph memoryGraph = new MemoryGraph(expectedNumberOfNodes);
GrowableArray<NodeIndex> tempForChildren = new GrowableArray<NodeIndex>();
// We can make a new Node index
NodeIndex newNodeIdx = memoryGraph.CreateNode();
NodeIndex childIdx = memoryGraph.CreateNode();
//
NodeTypeIndex newNodeType = memoryGraph.CreateType("MyChild");
memoryGraph.SetNode(childIdx, newType, 100, tempForChildren);
memoryGraph.AllowReading();
// Serialize to a file
memoryGraph.WriteAsBinaryFile("file.gcHeap");
// Can unserialize easily.
// var readBackIn = MemoryGraph.ReadFromBinaryFile("file.gcHeap");
}
}
}
#endif

Просмотреть файл

@ -0,0 +1,46 @@
# DotNetHeapDump
The following code files were copied in entirety with minimal changes from the Microsoft/Perfview repository on github: https://github.com/Microsoft/PerfView.
This was done simply because refactoring the TraceEvent library to include these classes proved to be too disruptive. Diamond dependency refactoring and
mismatched target frameworks made the refactoring too disruptive to the TraceEvent library.
This code should be treated as read-only. Any changes that _do_ need to be made should be mirrored to Microsoft/PerfView _and_ documented here.
## Files:
* DotNetHeapInfo.cs (https://github.com/microsoft/perfview/blob/76dc28af873e27aa8c4f9ce8efa0971a2c738165/src/HeapDumpCommon/DotNetHeapInfo.cs)
* GCHeapDump.cs (https://github.com/microsoft/perfview/blob/76dc28af873e27aa8c4f9ce8efa0971a2c738165/src/HeapDump/GCHeapDump.cs)
* DotNetHeapDumpGraphReader.cs (https://github.com/microsoft/perfview/blob/76dc28af873e27aa8c4f9ce8efa0971a2c738165/src/EtwHeapDump/DotNetHeapDumpGraphReader.cs)
* MemoryGraph.cs (https://github.com/microsoft/perfview/blob/76dc28af873e27aa8c4f9ce8efa0971a2c738165/src/MemoryGraph/MemoryGraph.cs)
* Graph.cs (https://github.com/microsoft/perfview/blob/76dc28af873e27aa8c4f9ce8efa0971a2c738165/src/MemoryGraph/graph.cs)
## Changes:
There is a baseline commit that contains an exact copy of the code files. All changes in this repo will be separate commits on top of that.
## License from Microsoft/PerfView
The MIT License (MIT)
Copyright (c) .NET Foundation and Contributors
All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Просмотреть файл

@ -0,0 +1,24 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.
using System.CommandLine.Builder;
using System.CommandLine.Invocation;
using System.Threading.Tasks;
namespace Microsoft.Diagnostics.Tools.GCDump
{
class Program
{
public static Task<int> Main(string[] args)
{
var parser = new CommandLineBuilder()
.AddCommand(CollectCommandHandler.CollectCommand())
.AddCommand(ListProcessesCommandHandler.ProcessStatusCommand())
.UseDefaults()
.Build();
return parser.InvokeAsync(args);
}
}
}

Просмотреть файл

@ -0,0 +1,27 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RootNamespace>Microsoft.Diagnostics.Tools.GCDump</RootNamespace>
<ToolCommandName>dotnet-gcdump</ToolCommandName>
<Description>.NET Core Performance Trace Tool</Description>
<PackageTags>Diagnostic</PackageTags>
<PackageReleaseNotes>$(Description)</PackageReleaseNotes>
<PackagedShimOutputRootDirectory>$(OutputPath)</PackagedShimOutputRootDirectory>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.CommandLine.Experimental" Version="$(SystemCommandLineExperimentalVersion)" />
<PackageReference Include="System.CommandLine.Rendering" Version="$(SystemCommandLineRenderingVersion)" />
<PackageReference Include="Microsoft.Diagnostics.Tracing.TraceEvent" Version="$(MicrosoftDiagnosticsTracingTraceEventVersion)" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="$(MSBuildThisFileDirectory)..\..\Microsoft.Diagnostics.Tools.RuntimeClient\Microsoft.Diagnostics.Tools.RuntimeClient.csproj" />
</ItemGroup>
<ItemGroup>
<Compile Include="..\Common\Commands\ProcessStatus.cs" Link="ProcessStatus.cs" />
</ItemGroup>
</Project>