mozilla
/
gecko-dev
зеркало из https://github.com/mozilla/gecko-dev.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
gecko-dev/build/build-clang/r322325.patch

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1601 строка
69 KiB
Diff
Исходник Обычный вид История

Bug 1484888 - Apply upstream patch that fixes a clang crash with stack overflow on PGO + LTO on Linux. r=froydnj For some reason, clang 6 crashes with a stack overflow on PGO + LTO on Linux 64 bits. Clang 7 doesn't, but has other problems. After some bisecting, I found the following: - r322684 is the first revision that is broken on the release_60 branch. - that revision is a cherry pick of r322313 from trunk, which is similarly broken. - trunk was fixed by r322325, which, funnily enough, predates when r322313 was cherry-picked. While the change from r322325 is relatively large, mixing multiple different changes in a single commit, there also haven't been significant changes to the same file on trunk since (one macro name change, one documentation change, and a change related to debug info), which would tend to indicate the change is not going to break anything, or at least not more than upgrading to clang 7 would. The exact part that fixes the issue could probably be found in this large commit, but I didn't feel like digging into it further considering the above.
2018-08-21 03:59:00 +03:00
From e5f83314a1df7ef7843f904da01f2aa236bbe7f0 Mon Sep 17 00:00:00 2001
From: Matthias Braun <matze@braunis.de>
Date: Thu, 11 Jan 2018 22:59:33 +0000
Subject: [PATCH] PeepholeOpt cleanup/refactor; NFC
- Less unnecessary use of `auto`
- Add early `using RegSubRegPair(AndIdx) =` to avoid countless
`TargetInstrInfo::` qualifications.
- Use references instead of pointers where possible.
- Remove unused parameters.
- Rewrite the CopyRewriter class hierarchy:
- Pull out uncoalescable copy rewriting functionality into
PeepholeOptimizer class.
- Use an abstract base class to make it clear that rewriters are
independent.
- Remove unnecessary \brief in doxygen comments.
- Remove unused constructor and method from ValueTracker.
- Replace UseAdvancedTracking of ValueTracker with DisableAdvCopyOpt use.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@322325 91177308-0d34-0410-b5e6-96231b3b80d8
---
lib/CodeGen/PeepholeOptimizer.cpp | 810 ++++++++++++++----------------
1 file changed, 370 insertions(+), 440 deletions(-)
diff --git a/llvm/lib/CodeGen/PeepholeOptimizer.cpp b/llvm/lib/CodeGen/PeepholeOptimizer.cpp
index 1090550243f..1320f998555 100644
--- a/llvm/lib/CodeGen/PeepholeOptimizer.cpp
+++ b/llvm/lib/CodeGen/PeepholeOptimizer.cpp
@@ -98,6 +98,8 @@
#include <utility>
using namespace llvm;
+using RegSubRegPair = TargetInstrInfo::RegSubRegPair;
+using RegSubRegPairAndIdx = TargetInstrInfo::RegSubRegPairAndIdx;
#define DEBUG_TYPE "peephole-opt"
@@ -110,6 +112,9 @@ static cl::opt<bool>
DisablePeephole("disable-peephole", cl::Hidden, cl::init(false),
cl::desc("Disable the peephole optimizer"));
+/// Specifiy whether or not the value tracking looks through
+/// complex instructions. When this is true, the value tracker
+/// bails on everything that is not a copy or a bitcast.
static cl::opt<bool>
DisableAdvCopyOpt("disable-adv-copy-opt", cl::Hidden, cl::init(false),
cl::desc("Disable advanced copy optimization"));
@@ -132,11 +137,11 @@ static cl::opt<unsigned> MaxRecurrenceChain(
"of commuting operands"));
-STATISTIC(NumReuse, "Number of extension results reused");
-STATISTIC(NumCmps, "Number of compares eliminated");
-STATISTIC(NumImmFold, "Number of move immediate folded");
-STATISTIC(NumLoadFold, "Number of loads folded");
-STATISTIC(NumSelects, "Number of selects optimized");
+STATISTIC(NumReuse, "Number of extension results reused");
+STATISTIC(NumCmps, "Number of compares eliminated");
+STATISTIC(NumImmFold, "Number of move immediate folded");
+STATISTIC(NumLoadFold, "Number of loads folded");
+STATISTIC(NumSelects, "Number of selects optimized");
STATISTIC(NumUncoalescableCopies, "Number of uncoalescable copies optimized");
STATISTIC(NumRewrittenCopies, "Number of copies rewritten");
STATISTIC(NumNAPhysCopies, "Number of non-allocatable physical copies removed");
@@ -149,9 +154,9 @@ namespace {
class PeepholeOptimizer : public MachineFunctionPass {
const TargetInstrInfo *TII;
const TargetRegisterInfo *TRI;
- MachineRegisterInfo *MRI;
- MachineDominatorTree *DT; // Machine dominator tree
- MachineLoopInfo *MLI;
+ MachineRegisterInfo *MRI;
+ MachineDominatorTree *DT; // Machine dominator tree
+ MachineLoopInfo *MLI;
public:
static char ID; // Pass identification
@@ -173,31 +178,28 @@ namespace {
}
}
- /// \brief Track Def -> Use info used for rewriting copies.
- using RewriteMapTy =
- SmallDenseMap<TargetInstrInfo::RegSubRegPair, ValueTrackerResult>;
+ /// Track Def -> Use info used for rewriting copies.
+ using RewriteMapTy = SmallDenseMap<RegSubRegPair, ValueTrackerResult>;
- /// \brief Sequence of instructions that formulate recurrence cycle.
+ /// Sequence of instructions that formulate recurrence cycle.
using RecurrenceCycle = SmallVector<RecurrenceInstr, 4>;
private:
- bool optimizeCmpInstr(MachineInstr *MI, MachineBasicBlock *MBB);
- bool optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
+ bool optimizeCmpInstr(MachineInstr &MI);
+ bool optimizeExtInstr(MachineInstr &MI, MachineBasicBlock &MBB,
SmallPtrSetImpl<MachineInstr*> &LocalMIs);
- bool optimizeSelect(MachineInstr *MI,
+ bool optimizeSelect(MachineInstr &MI,
SmallPtrSetImpl<MachineInstr *> &LocalMIs);
- bool optimizeCondBranch(MachineInstr *MI);
- bool optimizeCoalescableCopy(MachineInstr *MI);
- bool optimizeUncoalescableCopy(MachineInstr *MI,
+ bool optimizeCondBranch(MachineInstr &MI);
+ bool optimizeCoalescableCopy(MachineInstr &MI);
+ bool optimizeUncoalescableCopy(MachineInstr &MI,
SmallPtrSetImpl<MachineInstr *> &LocalMIs);
bool optimizeRecurrence(MachineInstr &PHI);
- bool findNextSource(unsigned Reg, unsigned SubReg,
- RewriteMapTy &RewriteMap);
- bool isMoveImmediate(MachineInstr *MI,
+ bool findNextSource(RegSubRegPair RegSubReg, RewriteMapTy &RewriteMap);
+ bool isMoveImmediate(MachineInstr &MI,
SmallSet<unsigned, 4> &ImmDefRegs,
DenseMap<unsigned, MachineInstr*> &ImmDefMIs);
- bool foldImmediate(MachineInstr *MI, MachineBasicBlock *MBB,
- SmallSet<unsigned, 4> &ImmDefRegs,
+ bool foldImmediate(MachineInstr &MI, SmallSet<unsigned, 4> &ImmDefRegs,
DenseMap<unsigned, MachineInstr*> &ImmDefMIs);
/// \brief Finds recurrence cycles, but only ones that formulated around
@@ -212,11 +214,11 @@ namespace {
/// the set \p CopySrcRegs and \p CopyMIs. If this virtual register was
/// previously seen as a copy, replace the uses of this copy with the
/// previously seen copy's destination register.
- bool foldRedundantCopy(MachineInstr *MI,
+ bool foldRedundantCopy(MachineInstr &MI,
SmallSet<unsigned, 4> &CopySrcRegs,
DenseMap<unsigned, MachineInstr *> &CopyMIs);
- /// \brief Is the register \p Reg a non-allocatable physical register?
+ /// Is the register \p Reg a non-allocatable physical register?
bool isNAPhysCopy(unsigned Reg);
/// \brief If copy instruction \p MI is a non-allocatable virtual<->physical
@@ -224,11 +226,10 @@ namespace {
/// non-allocatable physical register was previously copied to a virtual
/// registered and hasn't been clobbered, the virt->phys copy can be
/// deleted.
- bool foldRedundantNAPhysCopy(
- MachineInstr *MI,
+ bool foldRedundantNAPhysCopy(MachineInstr &MI,
DenseMap<unsigned, MachineInstr *> &NAPhysToVirtMIs);
- bool isLoadFoldable(MachineInstr *MI,
+ bool isLoadFoldable(MachineInstr &MI,
SmallSet<unsigned, 16> &FoldAsLoadDefCandidates);
/// \brief Check whether \p MI is understood by the register coalescer
@@ -249,10 +250,13 @@ namespace {
(MI.isRegSequenceLike() || MI.isInsertSubregLike() ||
MI.isExtractSubregLike()));
}
+
+ MachineInstr &rewriteSource(MachineInstr &CopyLike,
+ RegSubRegPair Def, RewriteMapTy &RewriteMap);
};
- /// \brief Helper class to hold instructions that are inside recurrence
- /// cycles. The recurrence cycle is formulated around 1) a def operand and its
+ /// Helper class to hold instructions that are inside recurrence cycles.
+ /// The recurrence cycle is formulated around 1) a def operand and its
/// tied use operand, or 2) a def operand and a use operand that is commutable
/// with another use operand which is tied to the def operand. In the latter
/// case, index of the tied use operand and the commutable use operand are
@@ -273,13 +277,13 @@ namespace {
Optional<IndexPair> CommutePair;
};
- /// \brief Helper class to hold a reply for ValueTracker queries. Contains the
- /// returned sources for a given search and the instructions where the sources
- /// were tracked from.
+ /// Helper class to hold a reply for ValueTracker queries.
+ /// Contains the returned sources for a given search and the instructions
+ /// where the sources were tracked from.
class ValueTrackerResult {
private:
/// Track all sources found by one ValueTracker query.
- SmallVector<TargetInstrInfo::RegSubRegPair, 2> RegSrcs;
+ SmallVector<RegSubRegPair, 2> RegSrcs;
/// Instruction using the sources in 'RegSrcs'.
const MachineInstr *Inst = nullptr;
@@ -302,16 +306,20 @@ namespace {
}
void addSource(unsigned SrcReg, unsigned SrcSubReg) {
- RegSrcs.push_back(TargetInstrInfo::RegSubRegPair(SrcReg, SrcSubReg));
+ RegSrcs.push_back(RegSubRegPair(SrcReg, SrcSubReg));
}
void setSource(int Idx, unsigned SrcReg, unsigned SrcSubReg) {
assert(Idx < getNumSources() && "Reg pair source out of index");
- RegSrcs[Idx] = TargetInstrInfo::RegSubRegPair(SrcReg, SrcSubReg);
+ RegSrcs[Idx] = RegSubRegPair(SrcReg, SrcSubReg);
}
int getNumSources() const { return RegSrcs.size(); }
+ RegSubRegPair getSrc(int Idx) const {
+ return RegSrcs[Idx];
+ }
+
unsigned getSrcReg(int Idx) const {
assert(Idx < getNumSources() && "Reg source out of index");
return RegSrcs[Idx].Reg;
@@ -367,59 +375,41 @@ namespace {
/// The register where the value can be found.
unsigned Reg;
- /// Specifiy whether or not the value tracking looks through
- /// complex instructions. When this is false, the value tracker
- /// bails on everything that is not a copy or a bitcast.
- ///
- /// Note: This could have been implemented as a specialized version of
- /// the ValueTracker class but that would have complicated the code of
- /// the users of this class.
- bool UseAdvancedTracking;
-
/// MachineRegisterInfo used to perform tracking.
const MachineRegisterInfo &MRI;
- /// Optional TargetInstrInfo used to perform some complex
- /// tracking.
+ /// Optional TargetInstrInfo used to perform some complex tracking.
const TargetInstrInfo *TII;
- /// \brief Dispatcher to the right underlying implementation of
- /// getNextSource.
+ /// Dispatcher to the right underlying implementation of getNextSource.
ValueTrackerResult getNextSourceImpl();
- /// \brief Specialized version of getNextSource for Copy instructions.
+ /// Specialized version of getNextSource for Copy instructions.
ValueTrackerResult getNextSourceFromCopy();
- /// \brief Specialized version of getNextSource for Bitcast instructions.
+ /// Specialized version of getNextSource for Bitcast instructions.
ValueTrackerResult getNextSourceFromBitcast();
- /// \brief Specialized version of getNextSource for RegSequence
- /// instructions.
+ /// Specialized version of getNextSource for RegSequence instructions.
ValueTrackerResult getNextSourceFromRegSequence();
- /// \brief Specialized version of getNextSource for InsertSubreg
- /// instructions.
+ /// Specialized version of getNextSource for InsertSubreg instructions.
ValueTrackerResult getNextSourceFromInsertSubreg();
- /// \brief Specialized version of getNextSource for ExtractSubreg
- /// instructions.
+ /// Specialized version of getNextSource for ExtractSubreg instructions.
ValueTrackerResult getNextSourceFromExtractSubreg();
- /// \brief Specialized version of getNextSource for SubregToReg
- /// instructions.
+ /// Specialized version of getNextSource for SubregToReg instructions.
ValueTrackerResult getNextSourceFromSubregToReg();
- /// \brief Specialized version of getNextSource for PHI instructions.
+ /// Specialized version of getNextSource for PHI instructions.
ValueTrackerResult getNextSourceFromPHI();
public:
- /// \brief Create a ValueTracker instance for the value defined by \p Reg.
+ /// Create a ValueTracker instance for the value defined by \p Reg.
/// \p DefSubReg represents the sub register index the value tracker will
/// track. It does not need to match the sub register index used in the
/// definition of \p Reg.
- /// \p UseAdvancedTracking specifies whether or not the value tracker looks
- /// through complex instructions. By default (false), it handles only copy
- /// and bitcast instructions.
/// If \p Reg is a physical register, a value tracker constructed with
/// this constructor will not find any alternative source.
/// Indeed, when \p Reg is a physical register that constructor does not
@@ -427,46 +417,20 @@ namespace {
/// Use the next constructor to track a physical register.
ValueTracker(unsigned Reg, unsigned DefSubReg,
const MachineRegisterInfo &MRI,
- bool UseAdvancedTracking = false,
const TargetInstrInfo *TII = nullptr)
- : DefSubReg(DefSubReg), Reg(Reg),
- UseAdvancedTracking(UseAdvancedTracking), MRI(MRI), TII(TII) {
+ : DefSubReg(DefSubReg), Reg(Reg), MRI(MRI), TII(TII) {
if (!TargetRegisterInfo::isPhysicalRegister(Reg)) {
Def = MRI.getVRegDef(Reg);
DefIdx = MRI.def_begin(Reg).getOperandNo();
}
}
- /// \brief Create a ValueTracker instance for the value defined by
- /// the pair \p MI, \p DefIdx.
- /// Unlike the other constructor, the value tracker produced by this one
- /// may be able to find a new source when the definition is a physical
- /// register.
- /// This could be useful to rewrite target specific instructions into
- /// generic copy instructions.
- ValueTracker(const MachineInstr &MI, unsigned DefIdx, unsigned DefSubReg,
- const MachineRegisterInfo &MRI,
- bool UseAdvancedTracking = false,
- const TargetInstrInfo *TII = nullptr)
- : Def(&MI), DefIdx(DefIdx), DefSubReg(DefSubReg),
- UseAdvancedTracking(UseAdvancedTracking), MRI(MRI), TII(TII) {
- assert(DefIdx < Def->getDesc().getNumDefs() &&
- Def->getOperand(DefIdx).isReg() && "Invalid definition");
- Reg = Def->getOperand(DefIdx).getReg();
- }
-
/// \brief Following the use-def chain, get the next available source
/// for the tracked value.
/// \return A ValueTrackerResult containing a set of registers
/// and sub registers with tracked values. A ValueTrackerResult with
/// an empty set of registers means no source was found.
ValueTrackerResult getNextSource();
-
- /// \brief Get the last register where the initial value can be found.
- /// Initially this is the register of the definition.
- /// Then, after each successful call to getNextSource, this is the
- /// register of the last source.
- unsigned getReg() const { return Reg; }
};
} // end anonymous namespace
@@ -476,11 +440,11 @@ char PeepholeOptimizer::ID = 0;
char &llvm::PeepholeOptimizerID = PeepholeOptimizer::ID;
INITIALIZE_PASS_BEGIN(PeepholeOptimizer, DEBUG_TYPE,
- "Peephole Optimizations", false, false)
+ "Peephole Optimizations", false, false)
INITIALIZE_PASS_DEPENDENCY(MachineDominatorTree)
INITIALIZE_PASS_DEPENDENCY(MachineLoopInfo)
INITIALIZE_PASS_END(PeepholeOptimizer, DEBUG_TYPE,
- "Peephole Optimizations", false, false)
+ "Peephole Optimizations", false, false)
/// If instruction is a copy-like instruction, i.e. it reads a single register
/// and writes a single register and it does not modify the source, and if the
@@ -491,10 +455,10 @@ INITIALIZE_PASS_END(PeepholeOptimizer, DEBUG_TYPE,
/// the code. Since this code does not currently share EXTRACTs, just ignore all
/// debug uses.
bool PeepholeOptimizer::
-optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
+optimizeExtInstr(MachineInstr &MI, MachineBasicBlock &MBB,
SmallPtrSetImpl<MachineInstr*> &LocalMIs) {
unsigned SrcReg, DstReg, SubIdx;
- if (!TII->isCoalescableExtInstr(*MI, SrcReg, DstReg, SubIdx))
+ if (!TII->isCoalescableExtInstr(MI, SrcReg, DstReg, SubIdx))
return false;
if (TargetRegisterInfo::isPhysicalRegister(DstReg) ||
@@ -535,7 +499,7 @@ optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
bool ExtendLife = true;
for (MachineOperand &UseMO : MRI->use_nodbg_operands(SrcReg)) {
MachineInstr *UseMI = UseMO.getParent();
- if (UseMI == MI)
+ if (UseMI == &MI)
continue;
if (UseMI->isPHI()) {
@@ -568,7 +532,7 @@ optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
continue;
MachineBasicBlock *UseMBB = UseMI->getParent();
- if (UseMBB == MBB) {
+ if (UseMBB == &MBB) {
// Local uses that come after the extension.
if (!LocalMIs.count(UseMI))
Uses.push_back(&UseMO);
@@ -576,7 +540,7 @@ optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
// Non-local uses where the result of the extension is used. Always
// replace these unless it's a PHI.
Uses.push_back(&UseMO);
- } else if (Aggressive && DT->dominates(MBB, UseMBB)) {
+ } else if (Aggressive && DT->dominates(&MBB, UseMBB)) {
// We may want to extend the live range of the extension result in order
// to replace these uses.
ExtendedUses.push_back(&UseMO);
@@ -640,19 +604,18 @@ optimizeExtInstr(MachineInstr *MI, MachineBasicBlock *MBB,
/// against already sets (or could be modified to set) the same flag as the
/// compare, then we can remove the comparison and use the flag from the
/// previous instruction.
-bool PeepholeOptimizer::optimizeCmpInstr(MachineInstr *MI,
- MachineBasicBlock *MBB) {
+bool PeepholeOptimizer::optimizeCmpInstr(MachineInstr &MI) {
// If this instruction is a comparison against zero and isn't comparing a
// physical register, we can try to optimize it.
unsigned SrcReg, SrcReg2;
int CmpMask, CmpValue;
- if (!TII->analyzeCompare(*MI, SrcReg, SrcReg2, CmpMask, CmpValue) ||
+ if (!TII->analyzeCompare(MI, SrcReg, SrcReg2, CmpMask, CmpValue) ||
TargetRegisterInfo::isPhysicalRegister(SrcReg) ||
(SrcReg2 != 0 && TargetRegisterInfo::isPhysicalRegister(SrcReg2)))
return false;
// Attempt to optimize the comparison instruction.
- if (TII->optimizeCompareInstr(*MI, SrcReg, SrcReg2, CmpMask, CmpValue, MRI)) {
+ if (TII->optimizeCompareInstr(MI, SrcReg, SrcReg2, CmpMask, CmpValue, MRI)) {
++NumCmps;
return true;
}
@@ -661,27 +624,26 @@ bool PeepholeOptimizer::optimizeCmpInstr(MachineInstr *MI,
}
/// Optimize a select instruction.
-bool PeepholeOptimizer::optimizeSelect(MachineInstr *MI,
+bool PeepholeOptimizer::optimizeSelect(MachineInstr &MI,
SmallPtrSetImpl<MachineInstr *> &LocalMIs) {
unsigned TrueOp = 0;
unsigned FalseOp = 0;
bool Optimizable = false;
SmallVector<MachineOperand, 4> Cond;
- if (TII->analyzeSelect(*MI, Cond, TrueOp, FalseOp, Optimizable))
+ if (TII->analyzeSelect(MI, Cond, TrueOp, FalseOp, Optimizable))
return false;
if (!Optimizable)
return false;
- if (!TII->optimizeSelect(*MI, LocalMIs))
+ if (!TII->optimizeSelect(MI, LocalMIs))
return false;
- MI->eraseFromParent();
+ MI.eraseFromParent();
++NumSelects;
return true;
}
-/// \brief Check if a simpler conditional branch can be
-/// generated
-bool PeepholeOptimizer::optimizeCondBranch(MachineInstr *MI) {
- return TII->optimizeCondBranch(*MI);
+/// Check if a simpler conditional branch can be generated.
+bool PeepholeOptimizer::optimizeCondBranch(MachineInstr &MI) {
+ return TII->optimizeCondBranch(MI);
}
/// \brief Try to find the next source that share the same register file
@@ -695,30 +657,29 @@ bool PeepholeOptimizer::optimizeCondBranch(MachineInstr *MI) {
/// share the same register file as \p Reg and \p SubReg. The client should
/// then be capable to rewrite all intermediate PHIs to get the next source.
/// \return False if no alternative sources are available. True otherwise.
-bool PeepholeOptimizer::findNextSource(unsigned Reg, unsigned SubReg,
+bool PeepholeOptimizer::findNextSource(RegSubRegPair RegSubReg,
RewriteMapTy &RewriteMap) {
// Do not try to find a new source for a physical register.
// So far we do not have any motivating example for doing that.
// Thus, instead of maintaining untested code, we will revisit that if
// that changes at some point.
+ unsigned Reg = RegSubReg.Reg;
if (TargetRegisterInfo::isPhysicalRegister(Reg))
return false;
const TargetRegisterClass *DefRC = MRI->getRegClass(Reg);
- SmallVector<TargetInstrInfo::RegSubRegPair, 4> SrcToLook;
- TargetInstrInfo::RegSubRegPair CurSrcPair(Reg, SubReg);
+ SmallVector<RegSubRegPair, 4> SrcToLook;
+ RegSubRegPair CurSrcPair = RegSubReg;
SrcToLook.push_back(CurSrcPair);
unsigned PHICount = 0;
- while (!SrcToLook.empty() && PHICount < RewritePHILimit) {
- TargetInstrInfo::RegSubRegPair Pair = SrcToLook.pop_back_val();
+ do {
+ CurSrcPair = SrcToLook.pop_back_val();
// As explained above, do not handle physical registers
- if (TargetRegisterInfo::isPhysicalRegister(Pair.Reg))
+ if (TargetRegisterInfo::isPhysicalRegister(CurSrcPair.Reg))
return false;
- CurSrcPair = Pair;
- ValueTracker ValTracker(CurSrcPair.Reg, CurSrcPair.SubReg, *MRI,
- !DisableAdvCopyOpt, TII);
+ ValueTracker ValTracker(CurSrcPair.Reg, CurSrcPair.SubReg, *MRI, TII);
// Follow the chain of copies until we find a more suitable source, a phi
// or have to abort.
@@ -747,14 +708,17 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, unsigned SubReg,
unsigned NumSrcs = Res.getNumSources();
if (NumSrcs > 1) {
PHICount++;
+ if (PHICount >= RewritePHILimit) {
+ DEBUG(dbgs() << "findNextSource: PHI limit reached\n");
+ return false;
+ }
+
for (unsigned i = 0; i < NumSrcs; ++i)
- SrcToLook.push_back(TargetInstrInfo::RegSubRegPair(
- Res.getSrcReg(i), Res.getSrcSubReg(i)));
+ SrcToLook.push_back(Res.getSrc(i));
break;
}
- CurSrcPair.Reg = Res.getSrcReg(0);
- CurSrcPair.SubReg = Res.getSrcSubReg(0);
+ CurSrcPair = Res.getSrc(0);
// Do not extend the live-ranges of physical registers as they add
// constraints to the register allocator. Moreover, if we want to extend
// the live-range of a physical register, unlike SSA virtual register,
@@ -764,7 +728,8 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, unsigned SubReg,
// Keep following the chain if the value isn't any better yet.
const TargetRegisterClass *SrcRC = MRI->getRegClass(CurSrcPair.Reg);
- if (!TRI->shouldRewriteCopySrc(DefRC, SubReg, SrcRC, CurSrcPair.SubReg))
+ if (!TRI->shouldRewriteCopySrc(DefRC, RegSubReg.SubReg, SrcRC,
+ CurSrcPair.SubReg))
continue;
// We currently cannot deal with subreg operands on PHI instructions
@@ -775,7 +740,7 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, unsigned SubReg,
// We found a suitable source, and are done with this chain.
break;
}
- }
+ } while (!SrcToLook.empty());
// If we did not find a more suitable source, there is nothing to optimize.
return CurSrcPair.Reg != Reg;
@@ -786,54 +751,50 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, unsigned SubReg,
/// successfully traverse a PHI instruction and find suitable sources coming
/// from its edges. By inserting a new PHI, we provide a rewritten PHI def
/// suitable to be used in a new COPY instruction.
-static MachineInstr *
-insertPHI(MachineRegisterInfo *MRI, const TargetInstrInfo *TII,
- const SmallVectorImpl<TargetInstrInfo::RegSubRegPair> &SrcRegs,
- MachineInstr *OrigPHI) {
+static MachineInstr &
+insertPHI(MachineRegisterInfo &MRI, const TargetInstrInfo &TII,
+ const SmallVectorImpl<RegSubRegPair> &SrcRegs,
+ MachineInstr &OrigPHI) {
assert(!SrcRegs.empty() && "No sources to create a PHI instruction?");
- const TargetRegisterClass *NewRC = MRI->getRegClass(SrcRegs[0].Reg);
+ const TargetRegisterClass *NewRC = MRI.getRegClass(SrcRegs[0].Reg);
// NewRC is only correct if no subregisters are involved. findNextSource()
// should have rejected those cases already.
assert(SrcRegs[0].SubReg == 0 && "should not have subreg operand");
- unsigned NewVR = MRI->createVirtualRegister(NewRC);
- MachineBasicBlock *MBB = OrigPHI->getParent();
- MachineInstrBuilder MIB = BuildMI(*MBB, OrigPHI, OrigPHI->getDebugLoc(),
- TII->get(TargetOpcode::PHI), NewVR);
+ unsigned NewVR = MRI.createVirtualRegister(NewRC);
+ MachineBasicBlock *MBB = OrigPHI.getParent();
+ MachineInstrBuilder MIB = BuildMI(*MBB, &OrigPHI, OrigPHI.getDebugLoc(),
+ TII.get(TargetOpcode::PHI), NewVR);
unsigned MBBOpIdx = 2;
- for (auto RegPair : SrcRegs) {
+ for (const RegSubRegPair &RegPair : SrcRegs) {
MIB.addReg(RegPair.Reg, 0, RegPair.SubReg);
- MIB.addMBB(OrigPHI->getOperand(MBBOpIdx).getMBB());
+ MIB.addMBB(OrigPHI.getOperand(MBBOpIdx).getMBB());
// Since we're extended the lifetime of RegPair.Reg, clear the
// kill flags to account for that and make RegPair.Reg reaches
// the new PHI.
- MRI->clearKillFlags(RegPair.Reg);
+ MRI.clearKillFlags(RegPair.Reg);
MBBOpIdx += 2;
}
- return MIB;
+ return *MIB;
}
namespace {
-/// \brief Helper class to rewrite the arguments of a copy-like instruction.
-class CopyRewriter {
+/// Interface to query instructions amenable to copy rewriting.
+class Rewriter {
protected:
- /// The copy-like instruction.
MachineInstr &CopyLike;
-
- /// The index of the source being rewritten.
- unsigned CurrentSrcIdx = 0;
-
+ unsigned CurrentSrcIdx = 0; ///< The index of the source being rewritten.
public:
- CopyRewriter(MachineInstr &MI) : CopyLike(MI) {}
- virtual ~CopyRewriter() = default;
+ Rewriter(MachineInstr &CopyLike) : CopyLike(CopyLike) {}
+ virtual ~Rewriter() {}
/// \brief Get the next rewritable source (SrcReg, SrcSubReg) and
- /// the related value that it affects (TrackReg, TrackSubReg).
+ /// the related value that it affects (DstReg, DstSubReg).
/// A source is considered rewritable if its register class and the
- /// register class of the related TrackReg may not be register
+ /// register class of the related DstReg may not be register
/// coalescer friendly. In other words, given a copy-like instruction
/// not all the arguments may be returned at rewritable source, since
/// some arguments are none to be register coalescer friendly.
@@ -848,137 +809,72 @@ public:
/// the only source this instruction has:
/// (SrcReg, SrcSubReg) = (src, srcSubIdx).
/// This source defines the whole definition, i.e.,
- /// (TrackReg, TrackSubReg) = (dst, dstSubIdx).
+ /// (DstReg, DstSubReg) = (dst, dstSubIdx).
///
/// The second and subsequent calls will return false, as there is only one
/// rewritable source.
///
/// \return True if a rewritable source has been found, false otherwise.
/// The output arguments are valid if and only if true is returned.
- virtual bool getNextRewritableSource(unsigned &SrcReg, unsigned &SrcSubReg,
- unsigned &TrackReg,
- unsigned &TrackSubReg) {
- // If CurrentSrcIdx == 1, this means this function has already been called
- // once. CopyLike has one definition and one argument, thus, there is
- // nothing else to rewrite.
- if (!CopyLike.isCopy() || CurrentSrcIdx == 1)
+ virtual bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) = 0;
+
+ /// Rewrite the current source with \p NewReg and \p NewSubReg if possible.
+ /// \return True if the rewriting was possible, false otherwise.
+ virtual bool RewriteCurrentSource(unsigned NewReg, unsigned NewSubReg) = 0;
+};
+
+/// Rewriter for COPY instructions.
+class CopyRewriter : public Rewriter {
+public:
+ CopyRewriter(MachineInstr &MI) : Rewriter(MI) {
+ assert(MI.isCopy() && "Expected copy instruction");
+ }
+ virtual ~CopyRewriter() = default;
+
+ bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) override {
+ // CurrentSrcIdx > 0 means this function has already been called.
+ if (CurrentSrcIdx > 0)
return false;
// This is the first call to getNextRewritableSource.
// Move the CurrentSrcIdx to remember that we made that call.
CurrentSrcIdx = 1;
// The rewritable source is the argument.
const MachineOperand &MOSrc = CopyLike.getOperand(1);
- SrcReg = MOSrc.getReg();
- SrcSubReg = MOSrc.getSubReg();
+ Src = RegSubRegPair(MOSrc.getReg(), MOSrc.getSubReg());
// What we track are the alternative sources of the definition.
const MachineOperand &MODef = CopyLike.getOperand(0);
- TrackReg = MODef.getReg();
- TrackSubReg = MODef.getSubReg();
+ Dst = RegSubRegPair(MODef.getReg(), MODef.getSubReg());
return true;
}
- /// \brief Rewrite the current source with \p NewReg and \p NewSubReg
- /// if possible.
- /// \return True if the rewriting was possible, false otherwise.
- virtual bool RewriteCurrentSource(unsigned NewReg, unsigned NewSubReg) {
- if (!CopyLike.isCopy() || CurrentSrcIdx != 1)
+ bool RewriteCurrentSource(unsigned NewReg, unsigned NewSubReg) override {
+ if (CurrentSrcIdx != 1)
return false;
MachineOperand &MOSrc = CopyLike.getOperand(CurrentSrcIdx);
MOSrc.setReg(NewReg);
MOSrc.setSubReg(NewSubReg);
return true;
}
-
- /// \brief Given a \p Def.Reg and Def.SubReg pair, use \p RewriteMap to find
- /// the new source to use for rewrite. If \p HandleMultipleSources is true and
- /// multiple sources for a given \p Def are found along the way, we found a
- /// PHI instructions that needs to be rewritten.
- /// TODO: HandleMultipleSources should be removed once we test PHI handling
- /// with coalescable copies.
- TargetInstrInfo::RegSubRegPair
- getNewSource(MachineRegisterInfo *MRI, const TargetInstrInfo *TII,
- TargetInstrInfo::RegSubRegPair Def,
- PeepholeOptimizer::RewriteMapTy &RewriteMap,
- bool HandleMultipleSources = true) {
- TargetInstrInfo::RegSubRegPair LookupSrc(Def.Reg, Def.SubReg);
- do {
- ValueTrackerResult Res = RewriteMap.lookup(LookupSrc);
- // If there are no entries on the map, LookupSrc is the new source.
- if (!Res.isValid())
- return LookupSrc;
-
- // There's only one source for this definition, keep searching...
- unsigned NumSrcs = Res.getNumSources();
- if (NumSrcs == 1) {
- LookupSrc.Reg = Res.getSrcReg(0);
- LookupSrc.SubReg = Res.getSrcSubReg(0);
- continue;
- }
-
- // TODO: Remove once multiple srcs w/ coalescable copies are supported.
- if (!HandleMultipleSources)
- break;
-
- // Multiple sources, recurse into each source to find a new source
- // for it. Then, rewrite the PHI accordingly to its new edges.
- SmallVector<TargetInstrInfo::RegSubRegPair, 4> NewPHISrcs;
- for (unsigned i = 0; i < NumSrcs; ++i) {
- TargetInstrInfo::RegSubRegPair PHISrc(Res.getSrcReg(i),
- Res.getSrcSubReg(i));
- NewPHISrcs.push_back(
- getNewSource(MRI, TII, PHISrc, RewriteMap, HandleMultipleSources));
- }
-
- // Build the new PHI node and return its def register as the new source.
- MachineInstr *OrigPHI = const_cast<MachineInstr *>(Res.getInst());
- MachineInstr *NewPHI = insertPHI(MRI, TII, NewPHISrcs, OrigPHI);
- DEBUG(dbgs() << "-- getNewSource\n");
- DEBUG(dbgs() << " Replacing: " << *OrigPHI);
- DEBUG(dbgs() << " With: " << *NewPHI);
- const MachineOperand &MODef = NewPHI->getOperand(0);
- return TargetInstrInfo::RegSubRegPair(MODef.getReg(), MODef.getSubReg());
-
- } while (true);
-
- return TargetInstrInfo::RegSubRegPair(0, 0);
- }
-
- /// \brief Rewrite the source found through \p Def, by using the \p RewriteMap
- /// and create a new COPY instruction. More info about RewriteMap in
- /// PeepholeOptimizer::findNextSource. Right now this is only used to handle
- /// Uncoalescable copies, since they are copy like instructions that aren't
- /// recognized by the register allocator.
- virtual MachineInstr *
- RewriteSource(TargetInstrInfo::RegSubRegPair Def,
- PeepholeOptimizer::RewriteMapTy &RewriteMap) {
- return nullptr;
- }
};
/// \brief Helper class to rewrite uncoalescable copy like instructions
/// into new COPY (coalescable friendly) instructions.
-class UncoalescableRewriter : public CopyRewriter {
-protected:
- const TargetInstrInfo &TII;
- MachineRegisterInfo &MRI;
-
- /// The number of defs in the bitcast
- unsigned NumDefs;
+class UncoalescableRewriter : public Rewriter {
+ unsigned NumDefs; ///< Number of defs in the bitcast.
public:
- UncoalescableRewriter(MachineInstr &MI, const TargetInstrInfo &TII,
- MachineRegisterInfo &MRI)
- : CopyRewriter(MI), TII(TII), MRI(MRI) {
+ UncoalescableRewriter(MachineInstr &MI) : Rewriter(MI) {
NumDefs = MI.getDesc().getNumDefs();
}
- /// \brief Get the next rewritable def source (TrackReg, TrackSubReg)
+ /// \see See Rewriter::getNextRewritableSource()
/// All such sources need to be considered rewritable in order to
/// rewrite a uncoalescable copy-like instruction. This method return
/// each definition that must be checked if rewritable.
- bool getNextRewritableSource(unsigned &SrcReg, unsigned &SrcSubReg,
- unsigned &TrackReg,
- unsigned &TrackSubReg) override {
+ bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) override {
// Find the next non-dead definition and continue from there.
if (CurrentSrcIdx == NumDefs)
return false;
@@ -990,64 +886,27 @@ public:
}
// What we track are the alternative sources of the definition.
+ Src = RegSubRegPair(0, 0);
const MachineOperand &MODef = CopyLike.getOperand(CurrentSrcIdx);
- TrackReg = MODef.getReg();
- TrackSubReg = MODef.getSubReg();
+ Dst = RegSubRegPair(MODef.getReg(), MODef.getSubReg());
CurrentSrcIdx++;
return true;
}
- /// \brief Rewrite the source found through \p Def, by using the \p RewriteMap
- /// and create a new COPY instruction. More info about RewriteMap in
- /// PeepholeOptimizer::findNextSource. Right now this is only used to handle
- /// Uncoalescable copies, since they are copy like instructions that aren't
- /// recognized by the register allocator.
- MachineInstr *
- RewriteSource(TargetInstrInfo::RegSubRegPair Def,
- PeepholeOptimizer::RewriteMapTy &RewriteMap) override {
- assert(!TargetRegisterInfo::isPhysicalRegister(Def.Reg) &&
- "We do not rewrite physical registers");
-
- // Find the new source to use in the COPY rewrite.
- TargetInstrInfo::RegSubRegPair NewSrc =
- getNewSource(&MRI, &TII, Def, RewriteMap);
-
- // Insert the COPY.
- const TargetRegisterClass *DefRC = MRI.getRegClass(Def.Reg);
- unsigned NewVR = MRI.createVirtualRegister(DefRC);
-
- MachineInstr *NewCopy =
- BuildMI(*CopyLike.getParent(), &CopyLike, CopyLike.getDebugLoc(),
- TII.get(TargetOpcode::COPY), NewVR)
- .addReg(NewSrc.Reg, 0, NewSrc.SubReg);
-
- NewCopy->getOperand(0).setSubReg(Def.SubReg);
- if (Def.SubReg)
- NewCopy->getOperand(0).setIsUndef();
-
- DEBUG(dbgs() << "-- RewriteSource\n");
- DEBUG(dbgs() << " Replacing: " << CopyLike);
- DEBUG(dbgs() << " With: " << *NewCopy);
- MRI.replaceRegWith(Def.Reg, NewVR);
- MRI.clearKillFlags(NewVR);
-
- // We extended the lifetime of NewSrc.Reg, clear the kill flags to
- // account for that.
- MRI.clearKillFlags(NewSrc.Reg);
-
- return NewCopy;
+ bool RewriteCurrentSource(unsigned NewReg, unsigned NewSubReg) override {
+ return false;
}
};
-/// \brief Specialized rewriter for INSERT_SUBREG instruction.
-class InsertSubregRewriter : public CopyRewriter {
+/// Specialized rewriter for INSERT_SUBREG instruction.
+class InsertSubregRewriter : public Rewriter {
public:
- InsertSubregRewriter(MachineInstr &MI) : CopyRewriter(MI) {
+ InsertSubregRewriter(MachineInstr &MI) : Rewriter(MI) {
assert(MI.isInsertSubreg() && "Invalid instruction");
}
- /// \brief See CopyRewriter::getNextRewritableSource.
+ /// \see See Rewriter::getNextRewritableSource()
/// Here CopyLike has the following form:
/// dst = INSERT_SUBREG Src1, Src2.src2SubIdx, subIdx.
/// Src1 has the same register class has dst, hence, there is
@@ -1055,29 +914,27 @@ public:
/// Src2.src2SubIdx, may not be register coalescer friendly.
/// Therefore, the first call to this method returns:
/// (SrcReg, SrcSubReg) = (Src2, src2SubIdx).
- /// (TrackReg, TrackSubReg) = (dst, subIdx).
+ /// (DstReg, DstSubReg) = (dst, subIdx).
///
/// Subsequence calls will return false.
- bool getNextRewritableSource(unsigned &SrcReg, unsigned &SrcSubReg,
- unsigned &TrackReg,
- unsigned &TrackSubReg) override {
+ bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) override {
// If we already get the only source we can rewrite, return false.
if (CurrentSrcIdx == 2)
return false;
// We are looking at v2 = INSERT_SUBREG v0, v1, sub0.
CurrentSrcIdx = 2;
const MachineOperand &MOInsertedReg = CopyLike.getOperand(2);
- SrcReg = MOInsertedReg.getReg();
- SrcSubReg = MOInsertedReg.getSubReg();
+ Src = RegSubRegPair(MOInsertedReg.getReg(), MOInsertedReg.getSubReg());
const MachineOperand &MODef = CopyLike.getOperand(0);
// We want to track something that is compatible with the
// partial definition.
- TrackReg = MODef.getReg();
if (MODef.getSubReg())
// Bail if we have to compose sub-register indices.
return false;
- TrackSubReg = (unsigned)CopyLike.getOperand(3).getImm();
+ Dst = RegSubRegPair(MODef.getReg(),
+ (unsigned)CopyLike.getOperand(3).getImm());
return true;
}
@@ -1092,41 +949,39 @@ public:
}
};
-/// \brief Specialized rewriter for EXTRACT_SUBREG instruction.
-class ExtractSubregRewriter : public CopyRewriter {
+/// Specialized rewriter for EXTRACT_SUBREG instruction.
+class ExtractSubregRewriter : public Rewriter {
const TargetInstrInfo &TII;
public:
ExtractSubregRewriter(MachineInstr &MI, const TargetInstrInfo &TII)
- : CopyRewriter(MI), TII(TII) {
+ : Rewriter(MI), TII(TII) {
assert(MI.isExtractSubreg() && "Invalid instruction");
}
- /// \brief See CopyRewriter::getNextRewritableSource.
+ /// \see Rewriter::getNextRewritableSource()
/// Here CopyLike has the following form:
/// dst.dstSubIdx = EXTRACT_SUBREG Src, subIdx.
/// There is only one rewritable source: Src.subIdx,
/// which defines dst.dstSubIdx.
- bool getNextRewritableSource(unsigned &SrcReg, unsigned &SrcSubReg,
- unsigned &TrackReg,
- unsigned &TrackSubReg) override {
+ bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) override {
// If we already get the only source we can rewrite, return false.
if (CurrentSrcIdx == 1)
return false;
// We are looking at v1 = EXTRACT_SUBREG v0, sub0.
CurrentSrcIdx = 1;
const MachineOperand &MOExtractedReg = CopyLike.getOperand(1);
- SrcReg = MOExtractedReg.getReg();
// If we have to compose sub-register indices, bail out.
if (MOExtractedReg.getSubReg())
return false;
- SrcSubReg = CopyLike.getOperand(2).getImm();
+ Src = RegSubRegPair(MOExtractedReg.getReg(),
+ CopyLike.getOperand(2).getImm());
// We want to track something that is compatible with the definition.
const MachineOperand &MODef = CopyLike.getOperand(0);
- TrackReg = MODef.getReg();
- TrackSubReg = MODef.getSubReg();
+ Dst = RegSubRegPair(MODef.getReg(), MODef.getSubReg());
return true;
}
@@ -1156,14 +1011,14 @@ public:
}
};
-/// \brief Specialized rewriter for REG_SEQUENCE instruction.
-class RegSequenceRewriter : public CopyRewriter {
+/// Specialized rewriter for REG_SEQUENCE instruction.
+class RegSequenceRewriter : public Rewriter {
public:
- RegSequenceRewriter(MachineInstr &MI) : CopyRewriter(MI) {
+ RegSequenceRewriter(MachineInstr &MI) : Rewriter(MI) {
assert(MI.isRegSequence() && "Invalid instruction");
}
- /// \brief See CopyRewriter::getNextRewritableSource.
+ /// \see Rewriter::getNextRewritableSource()
/// Here CopyLike has the following form:
/// dst = REG_SEQUENCE Src1.src1SubIdx, subIdx1, Src2.src2SubIdx, subIdx2.
/// Each call will return a different source, walking all the available
@@ -1171,17 +1026,16 @@ public:
///
/// The first call returns:
/// (SrcReg, SrcSubReg) = (Src1, src1SubIdx).
- /// (TrackReg, TrackSubReg) = (dst, subIdx1).
+ /// (DstReg, DstSubReg) = (dst, subIdx1).
///
/// The second call returns:
/// (SrcReg, SrcSubReg) = (Src2, src2SubIdx).
- /// (TrackReg, TrackSubReg) = (dst, subIdx2).
+ /// (DstReg, DstSubReg) = (dst, subIdx2).
///
/// And so on, until all the sources have been traversed, then
/// it returns false.
- bool getNextRewritableSource(unsigned &SrcReg, unsigned &SrcSubReg,
- unsigned &TrackReg,
- unsigned &TrackSubReg) override {
+ bool getNextRewritableSource(RegSubRegPair &Src,
+ RegSubRegPair &Dst) override {
// We are looking at v0 = REG_SEQUENCE v1, sub1, v2, sub2, etc.
// If this is the first call, move to the first argument.
@@ -1194,17 +1048,17 @@ public:
return false;
}
const MachineOperand &MOInsertedReg = CopyLike.getOperand(CurrentSrcIdx);
- SrcReg = MOInsertedReg.getReg();
+ Src.Reg = MOInsertedReg.getReg();
// If we have to compose sub-register indices, bail out.
- if ((SrcSubReg = MOInsertedReg.getSubReg()))
+ if ((Src.SubReg = MOInsertedReg.getSubReg()))
return false;
// We want to track something that is compatible with the related
// partial definition.
- TrackSubReg = CopyLike.getOperand(CurrentSrcIdx + 1).getImm();
+ Dst.SubReg = CopyLike.getOperand(CurrentSrcIdx + 1).getImm();
const MachineOperand &MODef = CopyLike.getOperand(0);
- TrackReg = MODef.getReg();
+ Dst.Reg = MODef.getReg();
// If we have to compose sub-registers, bail.
return MODef.getSubReg() == 0;
}
@@ -1224,16 +1078,14 @@ public:
} // end anonymous namespace
-/// \brief Get the appropriated CopyRewriter for \p MI.
-/// \return A pointer to a dynamically allocated CopyRewriter or nullptr
-/// if no rewriter works for \p MI.
-static CopyRewriter *getCopyRewriter(MachineInstr &MI,
- const TargetInstrInfo &TII,
- MachineRegisterInfo &MRI) {
+/// Get the appropriated Rewriter for \p MI.
+/// \return A pointer to a dynamically allocated Rewriter or nullptr if no
+/// rewriter works for \p MI.
+static Rewriter *getCopyRewriter(MachineInstr &MI, const TargetInstrInfo &TII) {
// Handle uncoalescable copy-like instructions.
- if (MI.isBitcast() || (MI.isRegSequenceLike() || MI.isInsertSubregLike() ||
- MI.isExtractSubregLike()))
- return new UncoalescableRewriter(MI, TII, MRI);
+ if (MI.isBitcast() || MI.isRegSequenceLike() || MI.isInsertSubregLike() ||
+ MI.isExtractSubregLike())
+ return new UncoalescableRewriter(MI);
switch (MI.getOpcode()) {
default:
@@ -1247,53 +1099,102 @@ static CopyRewriter *getCopyRewriter(MachineInstr &MI,
case TargetOpcode::REG_SEQUENCE:
return new RegSequenceRewriter(MI);
}
- llvm_unreachable(nullptr);
}
-/// \brief Optimize generic copy instructions to avoid cross
-/// register bank copy. The optimization looks through a chain of
-/// copies and tries to find a source that has a compatible register
-/// class.
-/// Two register classes are considered to be compatible if they share
-/// the same register bank.
+/// \brief Given a \p Def.Reg and Def.SubReg pair, use \p RewriteMap to find
+/// the new source to use for rewrite. If \p HandleMultipleSources is true and
+/// multiple sources for a given \p Def are found along the way, we found a
+/// PHI instructions that needs to be rewritten.
+/// TODO: HandleMultipleSources should be removed once we test PHI handling
+/// with coalescable copies.
+static RegSubRegPair
+getNewSource(MachineRegisterInfo *MRI, const TargetInstrInfo *TII,
+ RegSubRegPair Def,
+ const PeepholeOptimizer::RewriteMapTy &RewriteMap,
+ bool HandleMultipleSources = true) {
+ RegSubRegPair LookupSrc(Def.Reg, Def.SubReg);
+ while (true) {
+ ValueTrackerResult Res = RewriteMap.lookup(LookupSrc);
+ // If there are no entries on the map, LookupSrc is the new source.
+ if (!Res.isValid())
+ return LookupSrc;
+
+ // There's only one source for this definition, keep searching...
+ unsigned NumSrcs = Res.getNumSources();
+ if (NumSrcs == 1) {
+ LookupSrc.Reg = Res.getSrcReg(0);
+ LookupSrc.SubReg = Res.getSrcSubReg(0);
+ continue;
+ }
+
+ // TODO: Remove once multiple srcs w/ coalescable copies are supported.
+ if (!HandleMultipleSources)
+ break;
+
+ // Multiple sources, recurse into each source to find a new source
+ // for it. Then, rewrite the PHI accordingly to its new edges.
+ SmallVector<RegSubRegPair, 4> NewPHISrcs;
+ for (unsigned i = 0; i < NumSrcs; ++i) {
+ RegSubRegPair PHISrc(Res.getSrcReg(i), Res.getSrcSubReg(i));
+ NewPHISrcs.push_back(
+ getNewSource(MRI, TII, PHISrc, RewriteMap, HandleMultipleSources));
+ }
+
+ // Build the new PHI node and return its def register as the new source.
+ MachineInstr &OrigPHI = const_cast<MachineInstr &>(*Res.getInst());
+ MachineInstr &NewPHI = insertPHI(*MRI, *TII, NewPHISrcs, OrigPHI);
+ DEBUG(dbgs() << "-- getNewSource\n");
+ DEBUG(dbgs() << " Replacing: " << OrigPHI);
+ DEBUG(dbgs() << " With: " << NewPHI);
+ const MachineOperand &MODef = NewPHI.getOperand(0);
+ return RegSubRegPair(MODef.getReg(), MODef.getSubReg());
+ }
+
+ return RegSubRegPair(0, 0);
+}
+
+/// Optimize generic copy instructions to avoid cross register bank copy.
+/// The optimization looks through a chain of copies and tries to find a source
+/// that has a compatible register class.
+/// Two register classes are considered to be compatible if they share the same
+/// register bank.
/// New copies issued by this optimization are register allocator
/// friendly. This optimization does not remove any copy as it may
/// overconstrain the register allocator, but replaces some operands
/// when possible.
/// \pre isCoalescableCopy(*MI) is true.
/// \return True, when \p MI has been rewritten. False otherwise.
-bool PeepholeOptimizer::optimizeCoalescableCopy(MachineInstr *MI) {
- assert(MI && isCoalescableCopy(*MI) && "Invalid argument");
- assert(MI->getDesc().getNumDefs() == 1 &&
+bool PeepholeOptimizer::optimizeCoalescableCopy(MachineInstr &MI) {
+ assert(isCoalescableCopy(MI) && "Invalid argument");
+ assert(MI.getDesc().getNumDefs() == 1 &&
"Coalescer can understand multiple defs?!");
- const MachineOperand &MODef = MI->getOperand(0);
+ const MachineOperand &MODef = MI.getOperand(0);
// Do not rewrite physical definitions.
if (TargetRegisterInfo::isPhysicalRegister(MODef.getReg()))
return false;
bool Changed = false;
// Get the right rewriter for the current copy.
- std::unique_ptr<CopyRewriter> CpyRewriter(getCopyRewriter(*MI, *TII, *MRI));
+ std::unique_ptr<Rewriter> CpyRewriter(getCopyRewriter(MI, *TII));
// If none exists, bail out.
if (!CpyRewriter)
return false;
// Rewrite each rewritable source.
- unsigned SrcReg, SrcSubReg, TrackReg, TrackSubReg;
- while (CpyRewriter->getNextRewritableSource(SrcReg, SrcSubReg, TrackReg,
- TrackSubReg)) {
+ RegSubRegPair Src;
+ RegSubRegPair TrackPair;
+ while (CpyRewriter->getNextRewritableSource(Src, TrackPair)) {
// Keep track of PHI nodes and its incoming edges when looking for sources.
RewriteMapTy RewriteMap;
// Try to find a more suitable source. If we failed to do so, or get the
// actual source, move to the next source.
- if (!findNextSource(TrackReg, TrackSubReg, RewriteMap))
+ if (!findNextSource(TrackPair, RewriteMap))
continue;
// Get the new source to rewrite. TODO: Only enable handling of multiple
// sources (PHIs) once we have a motivating example and testcases for it.
- TargetInstrInfo::RegSubRegPair TrackPair(TrackReg, TrackSubReg);
- TargetInstrInfo::RegSubRegPair NewSrc = CpyRewriter->getNewSource(
- MRI, TII, TrackPair, RewriteMap, false /* multiple sources */);
- if (SrcReg == NewSrc.Reg || NewSrc.Reg == 0)
+ RegSubRegPair NewSrc = getNewSource(MRI, TII, TrackPair, RewriteMap,
+ /*HandleMultipleSources=*/false);
+ if (Src.Reg == NewSrc.Reg || NewSrc.Reg == 0)
continue;
// Rewrite source.
@@ -1312,6 +1213,47 @@ bool PeepholeOptimizer::optimizeCoalescableCopy(MachineInstr *MI) {
return Changed;
}
+/// \brief Rewrite the source found through \p Def, by using the \p RewriteMap
+/// and create a new COPY instruction. More info about RewriteMap in
+/// PeepholeOptimizer::findNextSource. Right now this is only used to handle
+/// Uncoalescable copies, since they are copy like instructions that aren't
+/// recognized by the register allocator.
+MachineInstr &
+PeepholeOptimizer::rewriteSource(MachineInstr &CopyLike,
+ RegSubRegPair Def, RewriteMapTy &RewriteMap) {
+ assert(!TargetRegisterInfo::isPhysicalRegister(Def.Reg) &&
+ "We do not rewrite physical registers");
+
+ // Find the new source to use in the COPY rewrite.
+ RegSubRegPair NewSrc = getNewSource(MRI, TII, Def, RewriteMap);
+
+ // Insert the COPY.
+ const TargetRegisterClass *DefRC = MRI->getRegClass(Def.Reg);
+ unsigned NewVReg = MRI->createVirtualRegister(DefRC);
+
+ MachineInstr *NewCopy =
+ BuildMI(*CopyLike.getParent(), &CopyLike, CopyLike.getDebugLoc(),
+ TII->get(TargetOpcode::COPY), NewVReg)
+ .addReg(NewSrc.Reg, 0, NewSrc.SubReg);
+
+ if (Def.SubReg) {
+ NewCopy->getOperand(0).setSubReg(Def.SubReg);
+ NewCopy->getOperand(0).setIsUndef();
+ }
+
+ DEBUG(dbgs() << "-- RewriteSource\n");
+ DEBUG(dbgs() << " Replacing: " << CopyLike);
+ DEBUG(dbgs() << " With: " << *NewCopy);
+ MRI->replaceRegWith(Def.Reg, NewVReg);
+ MRI->clearKillFlags(NewVReg);
+
+ // We extended the lifetime of NewSrc.Reg, clear the kill flags to
+ // account for that.
+ MRI->clearKillFlags(NewSrc.Reg);
+
+ return *NewCopy;
+}
+
/// \brief Optimize copy-like instructions to create
/// register coalescer friendly instruction.
/// The optimization tries to kill-off the \p MI by looking
@@ -1324,48 +1266,40 @@ bool PeepholeOptimizer::optimizeCoalescableCopy(MachineInstr *MI) {
/// been removed from its parent.
/// All COPY instructions created, are inserted in \p LocalMIs.
bool PeepholeOptimizer::optimizeUncoalescableCopy(
- MachineInstr *MI, SmallPtrSetImpl<MachineInstr *> &LocalMIs) {
- assert(MI && isUncoalescableCopy(*MI) && "Invalid argument");
-
- // Check if we can rewrite all the values defined by this instruction.
- SmallVector<TargetInstrInfo::RegSubRegPair, 4> RewritePairs;
- // Get the right rewriter for the current copy.
- std::unique_ptr<CopyRewriter> CpyRewriter(getCopyRewriter(*MI, *TII, *MRI));
- // If none exists, bail out.
- if (!CpyRewriter)
- return false;
+ MachineInstr &MI, SmallPtrSetImpl<MachineInstr *> &LocalMIs) {
+ assert(isUncoalescableCopy(MI) && "Invalid argument");
+ UncoalescableRewriter CpyRewriter(MI);
// Rewrite each rewritable source by generating new COPYs. This works
// differently from optimizeCoalescableCopy since it first makes sure that all
// definitions can be rewritten.
RewriteMapTy RewriteMap;
- unsigned Reg, SubReg, CopyDefReg, CopyDefSubReg;
- while (CpyRewriter->getNextRewritableSource(Reg, SubReg, CopyDefReg,
- CopyDefSubReg)) {
+ RegSubRegPair Src;
+ RegSubRegPair Def;
+ SmallVector<RegSubRegPair, 4> RewritePairs;
+ while (CpyRewriter.getNextRewritableSource(Src, Def)) {
// If a physical register is here, this is probably for a good reason.
// Do not rewrite that.
- if (TargetRegisterInfo::isPhysicalRegister(CopyDefReg))
+ if (TargetRegisterInfo::isPhysicalRegister(Def.Reg))
return false;
// If we do not know how to rewrite this definition, there is no point
// in trying to kill this instruction.
- TargetInstrInfo::RegSubRegPair Def(CopyDefReg, CopyDefSubReg);
- if (!findNextSource(Def.Reg, Def.SubReg, RewriteMap))
+ if (!findNextSource(Def, RewriteMap))
return false;
RewritePairs.push_back(Def);
}
// The change is possible for all defs, do it.
- for (const auto &Def : RewritePairs) {
+ for (const RegSubRegPair &Def : RewritePairs) {
// Rewrite the "copy" in a way the register coalescer understands.
- MachineInstr *NewCopy = CpyRewriter->RewriteSource(Def, RewriteMap);
- assert(NewCopy && "Should be able to always generate a new copy");
- LocalMIs.insert(NewCopy);
+ MachineInstr &NewCopy = rewriteSource(MI, Def, RewriteMap);
+ LocalMIs.insert(&NewCopy);
}
// MI is now dead.
- MI->eraseFromParent();
+ MI.eraseFromParent();
++NumUncoalescableCopies;
return true;
}
@@ -1374,18 +1308,18 @@ bool PeepholeOptimizer::optimizeUncoalescableCopy(
/// We only fold loads to virtual registers and the virtual register defined
/// has a single use.
bool PeepholeOptimizer::isLoadFoldable(
- MachineInstr *MI, SmallSet<unsigned, 16> &FoldAsLoadDefCandidates) {
- if (!MI->canFoldAsLoad() || !MI->mayLoad())
+ MachineInstr &MI, SmallSet<unsigned, 16> &FoldAsLoadDefCandidates) {
+ if (!MI.canFoldAsLoad() || !MI.mayLoad())
return false;
- const MCInstrDesc &MCID = MI->getDesc();
+ const MCInstrDesc &MCID = MI.getDesc();
if (MCID.getNumDefs() != 1)
return false;
- unsigned Reg = MI->getOperand(0).getReg();
+ unsigned Reg = MI.getOperand(0).getReg();
// To reduce compilation time, we check MRI->hasOneNonDBGUse when inserting
// loads. It should be checked when processing uses of the load, since
// uses can be removed during peephole.
- if (!MI->getOperand(0).getSubReg() &&
+ if (!MI.getOperand(0).getSubReg() &&
TargetRegisterInfo::isVirtualRegister(Reg) &&
MRI->hasOneNonDBGUse(Reg)) {
FoldAsLoadDefCandidates.insert(Reg);
@@ -1395,16 +1329,16 @@ bool PeepholeOptimizer::isLoadFoldable(
}
bool PeepholeOptimizer::isMoveImmediate(
- MachineInstr *MI, SmallSet<unsigned, 4> &ImmDefRegs,
+ MachineInstr &MI, SmallSet<unsigned, 4> &ImmDefRegs,
DenseMap<unsigned, MachineInstr *> &ImmDefMIs) {
- const MCInstrDesc &MCID = MI->getDesc();
- if (!MI->isMoveImmediate())
+ const MCInstrDesc &MCID = MI.getDesc();
+ if (!MI.isMoveImmediate())
return false;
if (MCID.getNumDefs() != 1)
return false;
- unsigned Reg = MI->getOperand(0).getReg();
+ unsigned Reg = MI.getOperand(0).getReg();
if (TargetRegisterInfo::isVirtualRegister(Reg)) {
- ImmDefMIs.insert(std::make_pair(Reg, MI));
+ ImmDefMIs.insert(std::make_pair(Reg, &MI));
ImmDefRegs.insert(Reg);
return true;
}
@@ -1415,11 +1349,11 @@ bool PeepholeOptimizer::isMoveImmediate(
/// Try folding register operands that are defined by move immediate
/// instructions, i.e. a trivial constant folding optimization, if
/// and only if the def and use are in the same BB.
-bool PeepholeOptimizer::foldImmediate(
- MachineInstr *MI, MachineBasicBlock *MBB, SmallSet<unsigned, 4> &ImmDefRegs,
+bool PeepholeOptimizer::foldImmediate(MachineInstr &MI,
+ SmallSet<unsigned, 4> &ImmDefRegs,
DenseMap<unsigned, MachineInstr *> &ImmDefMIs) {
- for (unsigned i = 0, e = MI->getDesc().getNumOperands(); i != e; ++i) {
- MachineOperand &MO = MI->getOperand(i);
+ for (unsigned i = 0, e = MI.getDesc().getNumOperands(); i != e; ++i) {
+ MachineOperand &MO = MI.getOperand(i);
if (!MO.isReg() || MO.isDef())
continue;
// Ignore dead implicit defs.
@@ -1432,7 +1366,7 @@ bool PeepholeOptimizer::foldImmediate(
continue;
DenseMap<unsigned, MachineInstr*>::iterator II = ImmDefMIs.find(Reg);
assert(II != ImmDefMIs.end() && "couldn't find immediate definition");
- if (TII->FoldImmediate(*MI, *II->second, Reg, MRI)) {
+ if (TII->FoldImmediate(MI, *II->second, Reg, MRI)) {
++NumImmFold;
return true;
}
@@ -1454,28 +1388,28 @@ bool PeepholeOptimizer::foldImmediate(
// %2 = COPY %0:sub1
//
// Should replace %2 uses with %1:sub1
-bool PeepholeOptimizer::foldRedundantCopy(
- MachineInstr *MI, SmallSet<unsigned, 4> &CopySrcRegs,
+bool PeepholeOptimizer::foldRedundantCopy(MachineInstr &MI,
+ SmallSet<unsigned, 4> &CopySrcRegs,
DenseMap<unsigned, MachineInstr *> &CopyMIs) {
- assert(MI->isCopy() && "expected a COPY machine instruction");
+ assert(MI.isCopy() && "expected a COPY machine instruction");
- unsigned SrcReg = MI->getOperand(1).getReg();
+ unsigned SrcReg = MI.getOperand(1).getReg();
if (!TargetRegisterInfo::isVirtualRegister(SrcReg))
return false;
- unsigned DstReg = MI->getOperand(0).getReg();
+ unsigned DstReg = MI.getOperand(0).getReg();
if (!TargetRegisterInfo::isVirtualRegister(DstReg))
return false;
if (CopySrcRegs.insert(SrcReg).second) {
// First copy of this reg seen.
- CopyMIs.insert(std::make_pair(SrcReg, MI));
+ CopyMIs.insert(std::make_pair(SrcReg, &MI));
return false;
}
MachineInstr *PrevCopy = CopyMIs.find(SrcReg)->second;
- unsigned SrcSubReg = MI->getOperand(1).getSubReg();
+ unsigned SrcSubReg = MI.getOperand(1).getSubReg();
unsigned PrevSrcSubReg = PrevCopy->getOperand(1).getSubReg();
// Can't replace different subregister extracts.
@@ -1504,19 +1438,19 @@ bool PeepholeOptimizer::isNAPhysCopy(unsigned Reg) {
}
bool PeepholeOptimizer::foldRedundantNAPhysCopy(
- MachineInstr *MI, DenseMap<unsigned, MachineInstr *> &NAPhysToVirtMIs) {
- assert(MI->isCopy() && "expected a COPY machine instruction");
+ MachineInstr &MI, DenseMap<unsigned, MachineInstr *> &NAPhysToVirtMIs) {
+ assert(MI.isCopy() && "expected a COPY machine instruction");
if (DisableNAPhysCopyOpt)
return false;
- unsigned DstReg = MI->getOperand(0).getReg();
- unsigned SrcReg = MI->getOperand(1).getReg();
+ unsigned DstReg = MI.getOperand(0).getReg();
+ unsigned SrcReg = MI.getOperand(1).getReg();
if (isNAPhysCopy(SrcReg) && TargetRegisterInfo::isVirtualRegister(DstReg)) {
// %vreg = COPY %physreg
// Avoid using a datastructure which can track multiple live non-allocatable
// phys->virt copies since LLVM doesn't seem to do this.
- NAPhysToVirtMIs.insert({SrcReg, MI});
+ NAPhysToVirtMIs.insert({SrcReg, &MI});
return false;
}
@@ -1528,8 +1462,7 @@ bool PeepholeOptimizer::foldRedundantNAPhysCopy(
if (PrevCopy == NAPhysToVirtMIs.end()) {
// We can't remove the copy: there was an intervening clobber of the
// non-allocatable physical register after the copy to virtual.
- DEBUG(dbgs() << "NAPhysCopy: intervening clobber forbids erasing " << *MI
- << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: intervening clobber forbids erasing " << MI);
return false;
}
@@ -1537,7 +1470,7 @@ bool PeepholeOptimizer::foldRedundantNAPhysCopy(
if (PrevDstReg == SrcReg) {
// Remove the virt->phys copy: we saw the virtual register definition, and
// the non-allocatable physical register's state hasn't changed since then.
- DEBUG(dbgs() << "NAPhysCopy: erasing " << *MI << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: erasing " << MI);
++NumNAPhysCopies;
return true;
}
@@ -1546,7 +1479,7 @@ bool PeepholeOptimizer::foldRedundantNAPhysCopy(
// register get a copy of the non-allocatable physical register, and we only
// track one such copy. Avoid getting confused by this new non-allocatable
// physical register definition, and remove it from the tracked copies.
- DEBUG(dbgs() << "NAPhysCopy: missed opportunity " << *MI << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: missed opportunity " << MI);
NAPhysToVirtMIs.erase(PrevCopy);
return false;
}
@@ -1611,11 +1544,11 @@ bool PeepholeOptimizer::findTargetRecurrence(
return false;
}
-/// \brief Phi instructions will eventually be lowered to copy instructions. If
-/// phi is in a loop header, a recurrence may formulated around the source and
-/// destination of the phi. For such case commuting operands of the instructions
-/// in the recurrence may enable coalescing of the copy instruction generated
-/// from the phi. For example, if there is a recurrence of
+/// Phi instructions will eventually be lowered to copy instructions.
+/// If phi is in a loop header, a recurrence may formulated around the source
+/// and destination of the phi. For such case commuting operands of the
+/// instructions in the recurrence may enable coalescing of the copy instruction
+/// generated from the phi. For example, if there is a recurrence of
///
/// LoopHeader:
/// %1 = phi(%0, %100)
@@ -1725,27 +1658,25 @@ bool PeepholeOptimizer::runOnMachineFunction(MachineFunction &MF) {
}
if (!MI->isCopy()) {
- for (const auto &Op : MI->operands()) {
+ for (const MachineOperand &MO : MI->operands()) {
// Visit all operands: definitions can be implicit or explicit.
- if (Op.isReg()) {
- unsigned Reg = Op.getReg();
- if (Op.isDef() && isNAPhysCopy(Reg)) {
+ if (MO.isReg()) {
+ unsigned Reg = MO.getReg();
+ if (MO.isDef() && isNAPhysCopy(Reg)) {
const auto &Def = NAPhysToVirtMIs.find(Reg);
if (Def != NAPhysToVirtMIs.end()) {
// A new definition of the non-allocatable physical register
// invalidates previous copies.
- DEBUG(dbgs() << "NAPhysCopy: invalidating because of " << *MI
- << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: invalidating because of " << *MI);
NAPhysToVirtMIs.erase(Def);
}
}
- } else if (Op.isRegMask()) {
- const uint32_t *RegMask = Op.getRegMask();
+ } else if (MO.isRegMask()) {
+ const uint32_t *RegMask = MO.getRegMask();
for (auto &RegMI : NAPhysToVirtMIs) {
unsigned Def = RegMI.first;
if (MachineOperand::clobbersPhysReg(RegMask, Def)) {
- DEBUG(dbgs() << "NAPhysCopy: invalidating because of " << *MI
- << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: invalidating because of " << *MI);
NAPhysToVirtMIs.erase(Def);
}
}
@@ -1761,58 +1692,57 @@ bool PeepholeOptimizer::runOnMachineFunction(MachineFunction &MF) {
// don't know what's correct anymore.
//
// FIXME: handle explicit asm clobbers.
- DEBUG(dbgs() << "NAPhysCopy: blowing away all info due to " << *MI
- << '\n');
+ DEBUG(dbgs() << "NAPhysCopy: blowing away all info due to " << *MI);
NAPhysToVirtMIs.clear();
}
if ((isUncoalescableCopy(*MI) &&
- optimizeUncoalescableCopy(MI, LocalMIs)) ||
- (MI->isCompare() && optimizeCmpInstr(MI, &MBB)) ||
- (MI->isSelect() && optimizeSelect(MI, LocalMIs))) {
+ optimizeUncoalescableCopy(*MI, LocalMIs)) ||
+ (MI->isCompare() && optimizeCmpInstr(*MI)) ||
+ (MI->isSelect() && optimizeSelect(*MI, LocalMIs))) {
// MI is deleted.
LocalMIs.erase(MI);
Changed = true;
continue;
}
- if (MI->isConditionalBranch() && optimizeCondBranch(MI)) {
+ if (MI->isConditionalBranch() && optimizeCondBranch(*MI)) {
Changed = true;
continue;
}
- if (isCoalescableCopy(*MI) && optimizeCoalescableCopy(MI)) {
+ if (isCoalescableCopy(*MI) && optimizeCoalescableCopy(*MI)) {
// MI is just rewritten.
Changed = true;
continue;
}
if (MI->isCopy() &&
- (foldRedundantCopy(MI, CopySrcRegs, CopySrcMIs) ||
- foldRedundantNAPhysCopy(MI, NAPhysToVirtMIs))) {
+ (foldRedundantCopy(*MI, CopySrcRegs, CopySrcMIs) ||
+ foldRedundantNAPhysCopy(*MI, NAPhysToVirtMIs))) {
LocalMIs.erase(MI);
MI->eraseFromParent();
Changed = true;
continue;
}
- if (isMoveImmediate(MI, ImmDefRegs, ImmDefMIs)) {
+ if (isMoveImmediate(*MI, ImmDefRegs, ImmDefMIs)) {
SeenMoveImm = true;
} else {
- Changed |= optimizeExtInstr(MI, &MBB, LocalMIs);
+ Changed |= optimizeExtInstr(*MI, MBB, LocalMIs);
// optimizeExtInstr might have created new instructions after MI
// and before the already incremented MII. Adjust MII so that the
// next iteration sees the new instructions.
MII = MI;
++MII;
if (SeenMoveImm)
- Changed |= foldImmediate(MI, &MBB, ImmDefRegs, ImmDefMIs);
+ Changed |= foldImmediate(*MI, ImmDefRegs, ImmDefMIs);
}
// Check whether MI is a load candidate for folding into a later
// instruction. If MI is not a candidate, check whether we can fold an
// earlier load into MI.
- if (!isLoadFoldable(MI, FoldAsLoadDefCandidates) &&
+ if (!isLoadFoldable(*MI, FoldAsLoadDefCandidates) &&
!FoldAsLoadDefCandidates.empty()) {
// We visit each operand even after successfully folding a previous
@@ -1861,7 +1791,7 @@ bool PeepholeOptimizer::runOnMachineFunction(MachineFunction &MF) {
// the load candidates. Note: We might be able to fold *into* this
// instruction, so this needs to be after the folding logic.
if (MI->isLoadFoldBarrier()) {
- DEBUG(dbgs() << "Encountered load fold barrier on " << *MI << "\n");
+ DEBUG(dbgs() << "Encountered load fold barrier on " << *MI);
FoldAsLoadDefCandidates.clear();
}
}
@@ -1958,14 +1888,14 @@ ValueTrackerResult ValueTracker::getNextSourceFromRegSequence() {
// duplicate the code from the generic TII.
return ValueTrackerResult();
- SmallVector<TargetInstrInfo::RegSubRegPairAndIdx, 8> RegSeqInputRegs;
+ SmallVector<RegSubRegPairAndIdx, 8> RegSeqInputRegs;
if (!TII->getRegSequenceInputs(*Def, DefIdx, RegSeqInputRegs))
return ValueTrackerResult();
// We are looking at:
// Def = REG_SEQUENCE v0, sub0, v1, sub1, ...
// Check if one of the operand defines the subreg we are interested in.
- for (auto &RegSeqInput : RegSeqInputRegs) {
+ for (const RegSubRegPairAndIdx &RegSeqInput : RegSeqInputRegs) {
if (RegSeqInput.SubIdx == DefSubReg) {
if (RegSeqInput.SubReg)
// Bail if we have to compose sub registers.
@@ -1996,8 +1926,8 @@ ValueTrackerResult ValueTracker::getNextSourceFromInsertSubreg() {
// duplicate the code from the generic TII.
return ValueTrackerResult();
- TargetInstrInfo::RegSubRegPair BaseReg;
- TargetInstrInfo::RegSubRegPairAndIdx InsertedReg;
+ RegSubRegPair BaseReg;
+ RegSubRegPairAndIdx InsertedReg;
if (!TII->getInsertSubregInputs(*Def, DefIdx, BaseReg, InsertedReg))
return ValueTrackerResult();
@@ -2050,7 +1980,7 @@ ValueTrackerResult ValueTracker::getNextSourceFromExtractSubreg() {
// duplicate the code from the generic TII.
return ValueTrackerResult();
- TargetInstrInfo::RegSubRegPairAndIdx ExtractSubregInputReg;
+ RegSubRegPairAndIdx ExtractSubregInputReg;
if (!TII->getExtractSubregInputs(*Def, DefIdx, ExtractSubregInputReg))
return ValueTrackerResult();
@@ -2083,7 +2013,7 @@ ValueTrackerResult ValueTracker::getNextSourceFromSubregToReg() {
Def->getOperand(3).getImm());
}
-/// \brief Explore each PHI incoming operand and return its sources
+/// Explore each PHI incoming operand and return its sources.
ValueTrackerResult ValueTracker::getNextSourceFromPHI() {
assert(Def->isPHI() && "Invalid definition");
ValueTrackerResult Res;
@@ -2095,7 +2025,7 @@ ValueTrackerResult ValueTracker::getNextSourceFromPHI() {
// Return all register sources for PHI instructions.
for (unsigned i = 1, e = Def->getNumOperands(); i < e; i += 2) {
- auto &MO = Def->getOperand(i);
+ const MachineOperand &MO = Def->getOperand(i);
assert(MO.isReg() && "Invalid PHI instruction");
// We have no code to deal with undef operands. They shouldn't happen in
// normal programs anyway.
@@ -2121,7 +2051,7 @@ ValueTrackerResult ValueTracker::getNextSourceImpl() {
return getNextSourceFromBitcast();
// All the remaining cases involve "complex" instructions.
// Bail if we did not ask for the advanced tracking.
- if (!UseAdvancedTracking)
+ if (DisableAdvCopyOpt)
return ValueTrackerResult();
if (Def->isRegSequence() || Def->isRegSequenceLike())
return getNextSourceFromRegSequence();
--
2.18.0