llilc/include
Bengu Li 34df455822 Major change of runtest and checkdiff
This is a major change to runtest and checkdiff.

First, it is changed to user pre-created baseline. Instead of
huge LLVM IR baseline stored in GitHub that caused issues.

Second, summary comparison is introduced. With one line
per method, the runtest and checkdiff process is much more
swift with summary only than previous approach. No dump
runtest is also provided. Verbose dump functionality is still
there. A normalization speed improvement is expected after
this change.

Third, LLVM IR dump level is introduced into LLILCJit.
Right now it read env var directly. It will be updated
when CLR Config is supported in LLILCJit.

A GitHub wiki page is created to illustrate the usage.
https://github.com/dotnet/llilc/wiki/Dev-Env-and-Test-Harness

Below are Typical use cases:

Developer Usage
1.Update your master to latest master
2.Build your baseline JIT: BuildAll or Build
3.Create a summary baseline: RunTest -Jit baseline-jit -Result SummaryBase -Dump Summary
4.Create a verbose baseline: RunTest -Jit baseline-jit -Result VerboseBase -Dump Verbose
5.Create your own branch
6.Do the new development or fix issues
7.Build your new JIT: Build
8.RunTest -Jit new-jit
  If failed with overall pass/failure and no dump, back to step 6 for fixing.
9.RunTest -Jit new-jit -Result SummaryDiff -Dump Summary
10.CheckDiff -Base SummaryBase -Diff SummaryDiff -Summary -UseDiffTool
  If failed with summary, find out which method is a new failure, back to step 6 for fixing.
11.RunTest -Jit new-jit -Result VerboseDiff -Dump Verbose
12.CheckDiff -Base VerboseBase -Diff VerboseDiff -UseDiffTool
  If any LLVM IR diff is not benign, back to step 6 for fixing.
13.Ready for a pull request.

Lab PR Usage
RunTest
Lab PR usage is most simplified version. It only checks for overall pass/fail and does not keep any result.

Lab Nightly Usage
1.RunTest -Jit last-known-good-jit -Result SummaryBase -Dump Summary
  or a summary result of running last-know-good-jit is kept in SummaryBase
2.RunTest -Jit latest-jit -Result SummaryDiff -Dump Summary
3.CheckDiff -Base SummaryBase -Diff SummaryDiff -Summary
4.If CheckDiff result is zero, mark latest-jit as last-known-good-jit. Result SummaryDiff could be kept as SummaryBase.
5.If CheckDiff result is not zero, send out failure notice to branch owner. Branch owner should analyze the case and bring relevant developer for fixing.
2015-03-17 21:47:03 -07:00
..
Driver Initial commit 2015-02-12 14:54:02 -08:00
Jit Major change of runtest and checkdiff 2015-03-17 21:47:03 -07:00
Pal Merge pull request #263 from swaroop-sridhar/clangPal 2015-03-11 17:07:38 -07:00
Reader Make ccformat --formatall compatible with Python 3 2015-03-11 16:26:43 -07:00
clr Initial commit 2015-02-12 14:54:02 -08:00
CMakeLists.txt Initial commit 2015-02-12 14:54:02 -08:00