Tool and library to convert ETW logs to JSON files
Перейти к файлу
microsoft-github-policy-service[bot] 4d145bf61e
Microsoft mandatory file (#12)
Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com>
2022-09-22 20:25:06 -07:00
Deserializer Upgrade to .NET Core 3.1 2019-12-19 21:02:01 -08:00
.gitattributes Convert to new project system & add SourceLink 2018-05-12 21:37:37 -07:00
.gitignore Initial commit 2015-08-02 20:54:29 -07:00
ETW2JSON.csproj Upgrade to .NET Core 3.1 2019-12-19 21:02:01 -08:00
ETW2JSON.nuspec Add pdb as embedded portable pdb 2018-05-12 21:48:20 -07:00
ETW2JSON.sln Upgrade to .NET Core 3.0 2019-10-06 02:27:17 -07:00
Etw.cs Initial import 2015-08-02 20:57:18 -07:00
EtwJsonWriter.cs Upgrade to .NET Core 3.0 2019-10-06 02:27:17 -07:00
LICENSE Initial commit 2015-08-02 20:54:29 -07:00
Program.cs Add flush statement to flush data to output (#8) 2020-01-09 13:16:44 -08:00
README.md Upgrade to .NET Core 3.1 2019-12-19 21:02:01 -08:00
SECURITY.md Microsoft mandatory file (#12) 2022-09-22 20:25:06 -07:00

README.md

ETW2JSON

ETW2JSON is a tool that converts ETW Log Files (.ETL) to JSON using the Newtonsoft.Json library. It can be used as a stand-alone command line tool that will take as input locations of ETL files and an output path, or it can take your implementation of Newtonsoft.Json's JsonWriter class.

Library usage

ConvertToJson(JsonWriter jsonWriter, IEnumerable<string> inputFiles, Action<string> reportError)

Command-line usage

ETW2JSON C:\MyFile.etl C:\MyFile.Kernel.etl --output=C:\MyFile.json

Nuget package

This library is available on Nuget -- https://www.nuget.org/packages/ETW2JSON/1.3.10

Why JSON?

Converting ETW Log Files (.ETL) to JSON makes accessible to you a plethora of data that was previously restricted to expert ETW tools or libraries. The goal of this tool is to make ETW data more accessible to a larger developer and operations audience by converting to a human-readable format that is ubiquitous.

Motivational use-case + workflow for collecting data and using ETW2JSON

Background: You are a devops team running a cloud service on Windows that runs .NET code, and you log some of your data using .NET EventSource. Other parts of your code write JSON to disk directly. Furthermore you are also interested in seeing data from Windows and the .NET CLR interspersed with your own logging data.

Pick your ETW Collection tool: Microsoft TraceEvent, plain old logman from the Windows command line.

Now you have an ETL file, or a set of ETL files, and maybe this set of ETL files is continous, considering you are cloud service.

After your collection is done, you can use ETW2JSON to convert the ETL file to JSON as follows:

ETW2JSON myFile.etl --output=myFile.json

You can now view this data in a variety of JSON log viewers, merge it with your own non-ETW event sources, push the data to a cloud logmerge system (Kafka, ElasticSearch, etc.) or store it in your favorite JSON database.

Does it understand Kernel, .NET EventSource, XPERF, etc. events?

ETW2JSON is a library that understands Windows MOF Classes events, Windows Vista Manifest events and EventSource .NET events. It also understands events that XPERF (WPR) adds as part of its merging process (to give PDB information) for profiler tools like the Windows Performance Recorder.

Example output

This is the output of ETW2JSON for a single event record of type CLRTrace/CLR Method/MethodDCEndVerbose -- you can use your favorite JSON Viewer to view this data.

{
   "CLRTrace/CLR Method/MethodDCEndVerbose":[
      {
         "MethodIdentifier":140712944189680,
         "ModuleID":140712943752376,
         "MethodStartAddress":140712947662480,
         "MethodSize":174,
         "MethodToken":100669671,
         "MethodFlags":0,
         "MethodNameSpace":"System.Xml.Schema.SchemaCollectionCompiler",
         "Methodname":"CompileGroup",
         "MethodSig":"instance void  (class System.Xml.Schema.XmlSchemaGroup)"
      }
   ]
}

Microsoft Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.