Paul Mackerras 9fb1b36ca1 powerpc: Make sure IPI handlers see data written by IPI senders ... We have been observing hangs, both of KVM guest vcpu tasks and more generally, where a process that is woken doesn't properly wake up and continue to run, but instead sticks in TASK_WAKING state. This happens because the update of rq->wake_list in ttwu_queue_remote() is not ordered with the update of ipi_message in smp_muxed_ipi_message_pass(), and the reading of rq->wake_list in scheduler_ipi() is not ordered with the reading of ipi_message in smp_ipi_demux(). Thus it is possible for the IPI receiver not to see the updated rq->wake_list and therefore conclude that there is nothing for it to do. In order to make sure that anything done before smp_send_reschedule() is ordered before anything done in the resulting call to scheduler_ipi(), this adds barriers in smp_muxed_message_pass() and smp_ipi_demux(). The barrier in smp_muxed_message_pass() is a full barrier to ensure that there is a full ordering between the smp_send_reschedule() caller and scheduler_ipi(). In smp_ipi_demux(), we use xchg() rather than xchg_local() because xchg() includes release and acquire barriers. Using xchg() rather than xchg_local() makes sense given that ipi_message is not just accessed locally. This moves the barrier between setting the message and calling the cause_ipi() function into the individual cause_ipi implementations. Most of them -- those that used outb, out_8 or similar -- already had a full barrier because out_8 etc. include a sync before the MMIO store. This adds an explicit barrier in the two remaining cases. These changes made no measurable difference to the speed of IPIs as measured using a simple ping-pong latency test across two CPUs on different cores of a POWER7 machine. The analysis of the reason why processes were not waking up properly is due to Milton Miller. Cc: stable@vger.kernel.org # v3.0+ Reported-by: Milton Miller <miltonm@bga.com> Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>		2012-09-05 16:05:22 +10:00
..
vdso32	powerpc: Add VDSO version of getcpu	2012-07-11 14:18:40 +10:00
vdso64	powerpc: Add VDSO version of getcpu	2012-07-11 14:18:40 +10:00
.gitignore	…
Makefile	KVM: PPC: Factor out guest epapr initialization	2012-05-30 11:43:10 +02:00
align.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
asm-offsets.c	powerpc: Restore correct DSCR in context switch	2012-09-05 16:05:22 +10:00
audit.c	…
btext.c	powerpc: various straight conversions from module.h --> export.h	2011-10-31 19:30:44 -04:00
cacheinfo.c	cpu: convert 'cpu' and 'machinecheck' sysdev_class to a regular subsystem	2011-12-21 14:29:42 -08:00
cacheinfo.h	…
clock.c	powerpc: various straight conversions from module.h --> export.h	2011-10-31 19:30:44 -04:00
compat_audit.c	…
cpu_setup_6xx.S	powerpc: Fix some 6xx/7xxx CPU setup functions	2011-02-07 12:57:11 +11:00
cpu_setup_44x.S	powerpc/44x: Add support for the AMCC APM821xx SoC	2010-10-13 08:47:09 -04:00
cpu_setup_a2.S	powerpc: Enforce usage of RA 0-R31 where possible	2012-07-10 19:18:35 +10:00
cpu_setup_fsl_booke.S	KVM: PPC: e500mc support	2012-04-08 12:54:33 +03:00
cpu_setup_pa6t.S	…
cpu_setup_power7.S	powerpc, KVM: Split HVMODE_206 cpu feature bit into separate HV and architecture bits	2011-07-12 13:16:58 +03:00
cpu_setup_ppc970.S	powerpc, KVM: Split HVMODE_206 cpu feature bit into separate HV and architecture bits	2011-07-12 13:16:58 +03:00
cputable.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
crash.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
crash_dump.c	powerpc: Rename mapping based RELOCATABLE to DYNAMIC_MEMSTART for BookE	2011-12-20 10:20:19 -05:00
dbell.c	powerpc: Make sure IPI handlers see data written by IPI senders	2012-09-05 16:05:22 +10:00
dma-iommu.c	powerpc/dma-iommu: Fix IOMMU window check	2012-08-24 20:26:07 +10:00
dma-swiotlb.c	common: dma-mapping: add support for generic dma_mmap_* calls	2012-07-30 12:25:46 +02:00
dma.c	Merge branch 'for-linus-for-3.6-rc1' of git://git.linaro.org/people/mszyprowski/linux-dma-mapping	2012-07-30 10:11:31 -07:00
entry_32.S	powerpc: Set stack limit properly in crit_transfer_to_handler	2012-07-27 11:42:31 +10:00
entry_64.S	powerpc: Restore correct DSCR in context switch	2012-09-05 16:05:22 +10:00
epapr_hcalls.S	KVM: PPC: Factor out guest epapr initialization	2012-05-30 11:43:10 +02:00
epapr_paravirt.c	KVM: PPC: Factor out guest epapr initialization	2012-05-30 11:43:10 +02:00
exceptions-64e.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
exceptions-64s.S	powerpc: Give hypervisor decrementer interrupts their own handler	2012-09-05 16:05:19 +10:00
fadump.c	powerpc: Fix fallout from system.h split up	2012-04-02 14:00:03 +10:00
firmware.c	powerpc: fix implicit use of cache.h in kernel/firmware.c	2011-10-31 19:30:41 -04:00
fpu.S	powerpc: Enforce usage of R0-R31 where possible	2012-07-10 19:18:30 +10:00
fsl_booke_entry_mapping.S	powerpc/fsl-booke: Fix address issue when using relocatable kernels	2010-07-11 11:04:08 -05:00
ftrace.c	powerpc/ftrace: Trace function graph entry before updating index	2012-07-27 11:42:34 +10:00
head_8xx.S	powerpc: Call do_page_fault() with interrupts off	2012-03-09 10:55:08 +11:00
head_32.S	powerpc: Call do_page_fault() with interrupts off	2012-03-09 10:55:08 +11:00
head_40x.S	powerpc: Call do_page_fault() with interrupts off	2012-03-09 10:55:08 +11:00
head_44x.S	Merge branch 'next' of git://git.kernel.org/pub/scm/virt/kvm/kvm	2012-05-24 16:17:30 -07:00
head_64.S	powerpc: Rework lazy-interrupt handling	2012-03-09 13:25:06 +11:00
head_booke.h	KVM: PPC: booke: category E.HV (GS-mode) support	2012-04-08 12:51:19 +03:00
head_fsl_booke.S	Merge remote-tracking branch 'kumar/next' into next	2012-07-13 13:38:26 +10:00
hw_breakpoint.c	powerpc: Fix null pointer deref in perf hardware breakpoints	2012-08-24 20:26:04 +10:00
ibmebus.c	Merge branch 'for-linus' of git://git.linaro.org/people/mszyprowski/linux-dma-mapping	2012-04-04 17:13:43 -07:00
idle.c	powerpc: Remove unused cpu_idle_wait()	2012-05-08 12:35:07 +02:00
idle_6xx.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
idle_book3e.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
idle_e500.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
idle_power4.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
idle_power7.S	powerpc/powernv: Always go into nap mode when CPU is offline	2012-09-05 16:05:20 +10:00
io-workarounds.c	powerpc: io-workarounds.c was implicitly getting init_mm	2011-10-31 19:30:37 -04:00
io.c	powerpc: various straight conversions from module.h --> export.h	2011-10-31 19:30:44 -04:00
iomap.c	powerpc: switch to GENERIC_PCI_IOMAP	2011-11-28 21:13:18 +02:00
iommu.c	powerpc/iommu: Fix iommu pool initialization	2012-07-13 17:45:49 +10:00
irq.c	powerpc: Fix build of some debug irq code	2012-07-10 19:16:20 +10:00
isa-bridge.c	powerpc: Remove FW_FEATURE ISERIES from arch code	2012-03-21 11:16:11 +11:00
jump_label.c	powerpc: Fix build breakage in jump_label.c	2011-11-16 14:25:19 +11:00
kgdb.c	powerpc/kgdb: Restore current_thread_info properly	2012-08-24 20:26:06 +10:00
kprobes.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
kvm.c	KVM updates for the 3.6 merge window	2012-07-24 12:01:20 -07:00
kvm_emul.S	KVM: PPC: Not optimizing MSR_CE and MSR_ME with paravirt.	2012-05-30 11:43:11 +02:00
l2cr_6xx.S	Fix common misspellings	2011-03-31 11:26:23 -03:00
legacy_serial.c	powerpc: Fix build on some non-freescale platforms	2012-01-25 13:33:22 +11:00
lparcfg.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
machine_kexec.c	powerpc/irqdomain: Fix broken NR_IRQ references	2012-04-30 10:45:26 +10:00
machine_kexec_32.c	treewide: convert uses of ATTRIB_NORETURN to __noreturn	2012-01-12 20:13:03 -08:00
machine_kexec_64.c	treewide: convert uses of ATTRIB_NORETURN to __noreturn	2012-01-12 20:13:03 -08:00
misc.S	powerpc: Remove legacy iSeries bits from assembly files	2012-03-09 10:54:59 +11:00
misc_32.S	powerpc: Use CURRENT_THREAD_INFO instead of open coded assembly	2012-07-11 14:18:22 +10:00
misc_64.S	powerpc: Fixes for instructions not using correct register naming	2012-07-10 19:18:16 +10:00
module.c	powerpc: remove non-required uses of include <linux/module.h>	2011-10-31 19:30:44 -04:00
module_32.c	powerpc: Fix kernel panic during kernel module load	2012-06-08 19:59:08 +10:00
module_64.c	modules: make arch's use default loader hooks	2011-07-24 22:06:04 +09:30
msi.c	…
nvram_64.c	powerpc/nvram: Generalize code for OS partitions in NVRAM	2011-03-04 18:19:04 +11:00
of_platform.c	powerpc/eeh: Introduce EEH device	2012-03-09 11:39:29 +11:00
paca.c	powerpc: Remove iseries specific fields in lppaca	2012-04-30 15:37:16 +10:00
pci-common.c	PCI changes for the 3.6 merge window:	2012-07-24 16:17:07 -07:00
pci_32.c	powerpc/PCI: compute I/O space bus-to-resource offset consistently	2012-03-20 10:41:44 -07:00
pci_64.c	PCI: replace struct pci_bus secondary/subordinate with busn_res	2012-06-13 15:42:22 -06:00
pci_dn.c	powerpc/powernv: PCI support for p7IOC under OPAL v2	2011-11-25 14:53:15 +11:00
pci_of_scan.c	PCI changes for the 3.6 merge window:	2012-07-24 16:17:07 -07:00
pmc.c	bug.h: add include of it to various implicit C users	2012-02-29 17:15:08 -05:00
ppc32.h	…
ppc_ksyms.c	powerpc: Use the new generic strncpy_from_user() and strnlen_user()	2012-05-27 21:00:07 -07:00
ppc_save_regs.S	Fix common misspellings	2011-03-31 11:26:23 -03:00
proc_powerpc.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
process.c	powerpc: Fix DSCR inheritance in copy_thread()	2012-09-05 16:05:21 +10:00
prom.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
prom_init.c	powerpc: Fix Section mismatch warnings in prom_init.c	2012-06-29 14:35:36 +10:00
prom_init_check.sh	powerpc/powernv: Get kernel command line accross OPAL takeover	2011-09-20 16:09:48 +10:00
prom_parse.c	powerpc: remove non-required uses of include <linux/module.h>	2011-10-31 19:30:44 -04:00
ptrace.c	Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc	2012-05-23 09:02:42 -07:00
ptrace32.c	powerpc: Remove old powerpc specific ptrace getregs/setregs calls	2012-04-30 15:37:28 +10:00
reloc_32.S	powerpc: Process dynamic relocations for kernel	2011-12-20 10:21:08 -05:00
reloc_64.S	…
rtas-proc.c	powerpc: Move /proc/ppc64 to /proc/powerpc update	2010-01-15 13:26:17 +11:00
rtas-rtc.c	powerpc/rtas-rtc: remove sideeffects of printk_ratelimit	2011-06-29 15:30:43 +10:00
rtas.c	Disintegrate and delete asm/system.h	2012-03-28 15:58:21 -07:00
rtas_flash.c	powerpc: Lack of firmware flash support is not an error	2012-07-27 11:42:33 +10:00
rtas_pci.c	Merge branch 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci	2012-03-23 14:02:12 -07:00
rtasd.c	powerpc/pseries: Cancel RTAS event scan before firmware flash	2011-11-25 14:11:29 +11:00
setup-common.c	powerpc/watchdog: move booke watchdog param related code to setup-common.c	2012-07-11 07:44:03 -05:00
setup.h	…
setup_32.c	powerpc/watchdog: move booke watchdog param related code to setup-common.c	2012-07-11 07:44:03 -05:00
setup_64.c	Disintegrate and delete asm/system.h	2012-03-28 15:58:21 -07:00
signal.c	new helper: signal_delivered()	2012-06-01 12:58:52 -04:00
signal.h	powerpc: get rid of restore_sigmask()	2012-06-01 12:58:51 -04:00
signal_32.c	powerpc: get rid of restore_sigmask()	2012-06-01 12:58:51 -04:00
signal_64.c	powerpc: get rid of restore_sigmask()	2012-06-01 12:58:51 -04:00
smp-tbsync.c	atomic: use <linux/atomic.h>	2011-07-26 16:49:47 -07:00
smp.c	powerpc: Make sure IPI handlers see data written by IPI senders	2012-09-05 16:05:22 +10:00
softemu8xx.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
stacktrace.c	powerpc: various straight conversions from module.h --> export.h	2011-10-31 19:30:44 -04:00
suspend.c	update email address	2010-07-19 10:56:54 +02:00
swsusp.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
swsusp_32.S	Fix common misspellings	2011-03-31 11:26:23 -03:00
swsusp_64.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
swsusp_asm64.S	…
swsusp_booke.S	powerpc/fsl-booke: Add hibernation support for FSL BookE processors	2010-05-21 07:41:53 -05:00
sys_ppc32.c	Disintegrate asm/system.h for PowerPC	2012-03-28 18:30:02 +01:00
syscalls.c	powerpc: Fix personality handling in ppc64_personality()	2012-08-24 20:26:07 +10:00
sysfs.c	powerpc: Keep thread.dscr and thread.dscr_inherit in sync	2012-09-05 16:05:21 +10:00
systbl.S	…
systbl_chk.c	…
systbl_chk.sh	…
tau_6xx.c	tree-wide: fix assorted typos all over the place	2009-12-04 15:39:55 +01:00
time.c	powerpc: Give hypervisor decrementer interrupts their own handler	2012-09-05 16:05:19 +10:00
traps.c	powerpc: Keep thread.dscr and thread.dscr_inherit in sync	2012-09-05 16:05:21 +10:00
udbg.c	powerpc: Random little legacy iSeries removal tidy ups	2012-03-28 11:33:23 +11:00
udbg_16550.c	powerpc: Add early debug for WSP platforms	2011-05-06 13:32:41 +10:00
vdso.c	powerpc: Add VDSO version of getcpu	2012-07-11 14:18:40 +10:00
vecemu.c	…
vector.S	powerpc: Optimise enable_kernel_altivec	2012-04-30 15:37:17 +10:00
vio.c	Merge branch 'for-linus-for-3.6-rc1' of git://git.linaro.org/people/mszyprowski/linux-dma-mapping	2012-07-30 10:11:31 -07:00
vmlinux.lds.S	powerpc: Remove legacy iSeries bits from assembly files	2012-03-09 10:54:59 +11:00