WSL2-Linux-Kernel/drivers/gpu/drm/i915/i915_drv.c

3247 строки
90 KiB
C
Исходник Обычный вид История

/* i915_drv.c -- i830,i845,i855,i865,i915 driver -*- linux-c -*-
*/
/*
*
* Copyright 2003 Tungsten Graphics, Inc., Cedar Park, Texas.
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sub license, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice (including the
* next paragraph) shall be included in all copies or substantial portions
* of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
* OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
* IN NO EVENT SHALL TUNGSTEN GRAPHICS AND/OR ITS SUPPLIERS BE LIABLE FOR
* ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*
*/
#include <linux/acpi.h>
#include <linux/device.h>
#include <linux/oom.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/pnp.h>
#include <linux/slab.h>
#include <linux/vgaarb.h>
#include <linux/vga_switcheroo.h>
#include <linux/vt.h>
#include <acpi/video.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_irq.h>
#include <drm/drm_probe_helper.h>
#include <drm/i915_drm.h>
#include "display/intel_acpi.h"
#include "display/intel_audio.h"
#include "display/intel_bw.h"
#include "display/intel_cdclk.h"
#include "display/intel_dp.h"
#include "display/intel_fbdev.h"
#include "display/intel_gmbus.h"
#include "display/intel_hotplug.h"
#include "display/intel_overlay.h"
#include "display/intel_pipe_crc.h"
#include "display/intel_sprite.h"
#include "gem/i915_gem_context.h"
#include "gem/i915_gem_ioctls.h"
#include "gt/intel_gt.h"
drm/i915: Invert the GEM wakeref hierarchy In the current scheme, on submitting a request we take a single global GEM wakeref, which trickles down to wake up all GT power domains. This is undesirable as we would like to be able to localise our power management to the available power domains and to remove the global GEM operations from the heart of the driver. (The intent there is to push global GEM decisions to the boundary as used by the GEM user interface.) Now during request construction, each request is responsible via its logical context to acquire a wakeref on each power domain it intends to utilize. Currently, each request takes a wakeref on the engine(s) and the engines themselves take a chipset wakeref. This gives us a transition on each engine which we can extend if we want to insert more powermangement control (such as soft rc6). The global GEM operations that currently require a struct_mutex are reduced to listening to pm events from the chipset GT wakeref. As we reduce the struct_mutex requirement, these listeners should evaporate. Perhaps the biggest immediate change is that this removes the struct_mutex requirement around GT power management, allowing us greater flexibility in request construction. Another important knock-on effect, is that by tracking engine usage, we can insert a switch back to the kernel context on that engine immediately, avoiding any extra delay or inserting global synchronisation barriers. This makes tracking when an engine and its associated contexts are idle much easier -- important for when we forgo our assumed execution ordering and need idle barriers to unpin used contexts. In the process, it means we remove a large chunk of code whose only purpose was to switch back to the kernel context. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Imre Deak <imre.deak@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 23:07:17 +03:00
#include "gt/intel_gt_pm.h"
#include "gt/intel_reset.h"
drm/i915: Invert the GEM wakeref hierarchy In the current scheme, on submitting a request we take a single global GEM wakeref, which trickles down to wake up all GT power domains. This is undesirable as we would like to be able to localise our power management to the available power domains and to remove the global GEM operations from the heart of the driver. (The intent there is to push global GEM decisions to the boundary as used by the GEM user interface.) Now during request construction, each request is responsible via its logical context to acquire a wakeref on each power domain it intends to utilize. Currently, each request takes a wakeref on the engine(s) and the engines themselves take a chipset wakeref. This gives us a transition on each engine which we can extend if we want to insert more powermangement control (such as soft rc6). The global GEM operations that currently require a struct_mutex are reduced to listening to pm events from the chipset GT wakeref. As we reduce the struct_mutex requirement, these listeners should evaporate. Perhaps the biggest immediate change is that this removes the struct_mutex requirement around GT power management, allowing us greater flexibility in request construction. Another important knock-on effect, is that by tracking engine usage, we can insert a switch back to the kernel context on that engine immediately, avoiding any extra delay or inserting global synchronisation barriers. This makes tracking when an engine and its associated contexts are idle much easier -- important for when we forgo our assumed execution ordering and need idle barriers to unpin used contexts. In the process, it means we remove a large chunk of code whose only purpose was to switch back to the kernel context. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Imre Deak <imre.deak@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 23:07:17 +03:00
#include "gt/intel_workarounds.h"
#include "gt/uc/intel_uc.h"
#include "i915_debugfs.h"
#include "i915_drv.h"
#include "i915_irq.h"
drm/i915/pmu: Expose a PMU interface for perf queries From: Chris Wilson <chris@chris-wilson.co.uk> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> From: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> The first goal is to be able to measure GPU (and invidual ring) busyness without having to poll registers from userspace. (Which not only incurs holding the forcewake lock indefinitely, perturbing the system, but also runs the risk of hanging the machine.) As an alternative we can use the perf event counter interface to sample the ring registers periodically and send those results to userspace. Functionality we are exporting to userspace is via the existing perf PMU API and can be exercised via the existing tools. For example: perf stat -a -e i915/rcs0-busy/ -I 1000 Will print the render engine busynnes once per second. All the performance counters can be enumerated (perf list) and have their unit of measure correctly reported in sysfs. v1-v2 (Chris Wilson): v2: Use a common timer for the ring sampling. v3: (Tvrtko Ursulin) * Decouple uAPI from i915 engine ids. * Complete uAPI defines. * Refactor some code to helpers for clarity. * Skip sampling disabled engines. * Expose counters in sysfs. * Pass in fake regs to avoid null ptr deref in perf core. * Convert to class/instance uAPI. * Use shared driver code for rc6 residency, power and frequency. v4: (Dmitry Rogozhkin) * Register PMU with .task_ctx_nr=perf_invalid_context * Expose cpumask for the PMU with the single CPU in the mask * Properly support pmu->stop(): it should call pmu->read() * Properly support pmu->del(): it should call stop(event, PERF_EF_UPDATE) * Introduce refcounting of event subscriptions. * Make pmu.busy_stats a refcounter to avoid busy stats going away with some deleted event. * Expose cpumask for i915 PMU to avoid multiple events creation of the same type followed by counter aggregation by perf-stat. * Track CPUs getting online/offline to migrate perf context. If (likely) cpumask will initially set CPU0, CONFIG_BOOTPARAM_HOTPLUG_CPU0 will be needed to see effect of CPU status tracking. * End result is that only global events are supported and perf stat works correctly. * Deny perf driver level sampling - it is prohibited for uncore PMU. v5: (Tvrtko Ursulin) * Don't hardcode number of engine samplers. * Rewrite event ref-counting for correctness and simplicity. * Store initial counter value when starting already enabled events to correctly report values to all listeners. * Fix RC6 residency readout. * Comments, GPL header. v6: * Add missing entry to v4 changelog. * Fix accounting in CPU hotplug case by copying the approach from arch/x86/events/intel/cstate.c. (Dmitry Rogozhkin) v7: * Log failure message only on failure. * Remove CPU hotplug notification state on unregister. v8: * Fix error unwind on failed registration. * Checkpatch cleanup. v9: * Drop the energy metric, it is available via intel_rapl_perf. (Ville Syrjälä) * Use HAS_RC6(p). (Chris Wilson) * Handle unsupported non-engine events. (Dmitry Rogozhkin) * Rebase for intel_rc6_residency_ns needing caller managed runtime pm. * Drop HAS_RC6 checks from the read callback since creating those events will be rejected at init time already. * Add counter units to sysfs so perf stat output is nicer. * Cleanup the attribute tables for brevity and readability. v10: * Fixed queued accounting. v11: * Move intel_engine_lookup_user to intel_engine_cs.c * Commit update. (Joonas Lahtinen) v12: * More accurate sampling. (Chris Wilson) * Store and report frequency in MHz for better usability from perf stat. * Removed metrics: queued, interrupts, rc6 counters. * Sample engine busyness based on seqno difference only for less MMIO (and forcewake) on all platforms. (Chris Wilson) v13: * Comment spelling, use mul_u32_u32 to work around potential GCC issue and somne code alignment changes. (Chris Wilson) v14: * Rebase. v15: * Rebase for RPS refactoring. v16: * Use the dynamic slot in the CPU hotplug state machine so that we are free to setup our state as multi-instance. Previously we were re-using the CPUHP_AP_PERF_X86_UNCORE_ONLINE slot which is neither used as multi-instance, nor owned by our driver to start with. * Register the CPU hotplug handlers after the PMU, otherwise the callback will get called before the PMU is initialized which can end up in perf_pmu_migrate_context with an un-initialized base. * Added workaround for a probable bug in cpuhp core. v17: * Remove workaround for the cpuhp bug. v18: * Rebase for drm_i915_gem_engine_class getting upstream before us. v19: * Rebase. (trivial) Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171121181852.16128-2-tvrtko.ursulin@linux.intel.com
2017-11-21 21:18:45 +03:00
#include "i915_pmu.h"
drm/i915: add query uAPI There are a number of information that are readable from hardware registers and that we would like to make accessible to userspace. One particular example is the topology of the execution units (how are execution units grouped in subslices and slices and also which ones have been fused off for die recovery). At the moment the GET_PARAM ioctl covers some basic needs, but generally is only able to return a single value for each defined parameter. This is a bit problematic with topology descriptions which are array/maps of available units. This change introduces a new ioctl that can deal with requests to fill structures of potentially variable lengths. The user is expected fill a query with length fields set at 0 on the first call, the kernel then sets the length fields to the their expected values. A second call to the kernel with length fields at their expected values will trigger a copy of the data to the pointed memory locations. The scope of this uAPI is only to provide information to userspace, not to allow configuration of the device. v2: Simplify dispatcher code iteration (Tvrtko) Tweak uapi drm_i915_query_item structure (Tvrtko) v3: Rename pad fields into flags (Chris) Return error on flags field != 0 (Chris) Only copy length back to userspace in drm_i915_query_item (Chris) v4: Use array of functions instead of switch (Chris) v5: More comments in uapi (Tvrtko) Return query item errors in length field (All) v6: Tweak uapi comments style to match the coding style (Lionel) v7: Add i915_query.h (Joonas) v8: (Lionel) Change the behavior of the item iterator to report invalid queries into the query item rather than stopping the iteration. This enables userspace applications to query newer items on older kernels and only have failure on the items that are not supported. v9: Edit copyright headers (Joonas) v10: Typos & comments in uapi (Joonas) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Acked-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180306122857.27317-6-lionel.g.landwerlin@intel.com
2018-03-06 15:28:56 +03:00
#include "i915_query.h"
#include "i915_trace.h"
#include "i915_vgpu.h"
#include "intel_csr.h"
#include "intel_drv.h"
#include "intel_pm.h"
static struct drm_driver driver;
#if IS_ENABLED(CONFIG_DRM_I915_DEBUG)
static unsigned int i915_probe_fail_count;
bool __i915_inject_probe_failure(const char *func, int line)
{
if (i915_probe_fail_count >= i915_modparams.inject_load_failure)
return false;
if (++i915_probe_fail_count == i915_modparams.inject_load_failure) {
DRM_INFO("Injecting failure at checkpoint %u [%s:%d]\n",
i915_modparams.inject_load_failure, func, line);
i915_modparams.inject_load_failure = 0;
return true;
}
return false;
}
bool i915_error_injected(void)
{
return i915_probe_fail_count && !i915_modparams.inject_load_failure;
}
#endif
#define FDO_BUG_URL "https://bugs.freedesktop.org/enter_bug.cgi?product=DRI"
#define FDO_BUG_MSG "Please file a bug at " FDO_BUG_URL " against DRM/Intel " \
"providing the dmesg log by booting with drm.debug=0xf"
void
__i915_printk(struct drm_i915_private *dev_priv, const char *level,
const char *fmt, ...)
{
static bool shown_bug_once;
struct device *kdev = dev_priv->drm.dev;
bool is_error = level[1] <= KERN_ERR[1];
bool is_debug = level[1] == KERN_DEBUG[1];
struct va_format vaf;
va_list args;
if (is_debug && !(drm_debug & DRM_UT_DRIVER))
return;
va_start(args, fmt);
vaf.fmt = fmt;
vaf.va = &args;
if (is_error)
dev_printk(level, kdev, "%pV", &vaf);
else
dev_printk(level, kdev, "[" DRM_NAME ":%ps] %pV",
__builtin_return_address(0), &vaf);
va_end(args);
if (is_error && !shown_bug_once) {
/*
* Ask the user to file a bug report for the error, except
* if they may have caused the bug by fiddling with unsafe
* module parameters.
*/
if (!test_taint(TAINT_USER))
dev_notice(kdev, "%s", FDO_BUG_MSG);
shown_bug_once = true;
}
}
/* Map PCH device id to PCH type, or PCH_NONE if unknown. */
static enum intel_pch
intel_pch_type(const struct drm_i915_private *dev_priv, unsigned short id)
{
switch (id) {
case INTEL_PCH_IBX_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Ibex Peak PCH\n");
WARN_ON(!IS_GEN(dev_priv, 5));
return PCH_IBX;
case INTEL_PCH_CPT_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found CougarPoint PCH\n");
WARN_ON(!IS_GEN(dev_priv, 6) && !IS_IVYBRIDGE(dev_priv));
return PCH_CPT;
case INTEL_PCH_PPT_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found PantherPoint PCH\n");
WARN_ON(!IS_GEN(dev_priv, 6) && !IS_IVYBRIDGE(dev_priv));
/* PantherPoint is CPT compatible */
return PCH_CPT;
case INTEL_PCH_LPT_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found LynxPoint PCH\n");
WARN_ON(!IS_HASWELL(dev_priv) && !IS_BROADWELL(dev_priv));
WARN_ON(IS_HSW_ULT(dev_priv) || IS_BDW_ULT(dev_priv));
return PCH_LPT;
case INTEL_PCH_LPT_LP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found LynxPoint LP PCH\n");
WARN_ON(!IS_HASWELL(dev_priv) && !IS_BROADWELL(dev_priv));
WARN_ON(!IS_HSW_ULT(dev_priv) && !IS_BDW_ULT(dev_priv));
return PCH_LPT;
case INTEL_PCH_WPT_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found WildcatPoint PCH\n");
WARN_ON(!IS_HASWELL(dev_priv) && !IS_BROADWELL(dev_priv));
WARN_ON(IS_HSW_ULT(dev_priv) || IS_BDW_ULT(dev_priv));
/* WildcatPoint is LPT compatible */
return PCH_LPT;
case INTEL_PCH_WPT_LP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found WildcatPoint LP PCH\n");
WARN_ON(!IS_HASWELL(dev_priv) && !IS_BROADWELL(dev_priv));
WARN_ON(!IS_HSW_ULT(dev_priv) && !IS_BDW_ULT(dev_priv));
/* WildcatPoint is LPT compatible */
return PCH_LPT;
case INTEL_PCH_SPT_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found SunrisePoint PCH\n");
WARN_ON(!IS_SKYLAKE(dev_priv) && !IS_KABYLAKE(dev_priv));
return PCH_SPT;
case INTEL_PCH_SPT_LP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found SunrisePoint LP PCH\n");
WARN_ON(!IS_SKYLAKE(dev_priv) && !IS_KABYLAKE(dev_priv));
return PCH_SPT;
case INTEL_PCH_KBP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Kaby Lake PCH (KBP)\n");
WARN_ON(!IS_SKYLAKE(dev_priv) && !IS_KABYLAKE(dev_priv) &&
!IS_COFFEELAKE(dev_priv));
/* KBP is SPT compatible */
return PCH_SPT;
case INTEL_PCH_CNP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Cannon Lake PCH (CNP)\n");
WARN_ON(!IS_CANNONLAKE(dev_priv) && !IS_COFFEELAKE(dev_priv));
return PCH_CNP;
case INTEL_PCH_CNP_LP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Cannon Lake LP PCH (CNP-LP)\n");
WARN_ON(!IS_CANNONLAKE(dev_priv) && !IS_COFFEELAKE(dev_priv));
return PCH_CNP;
case INTEL_PCH_CMP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Comet Lake PCH (CMP)\n");
WARN_ON(!IS_COFFEELAKE(dev_priv));
/* CometPoint is CNP Compatible */
return PCH_CNP;
case INTEL_PCH_ICP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Ice Lake PCH\n");
WARN_ON(!IS_ICELAKE(dev_priv));
return PCH_ICP;
case INTEL_PCH_MCC_DEVICE_ID_TYPE:
case INTEL_PCH_MCC2_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Mule Creek Canyon PCH\n");
WARN_ON(!IS_ELKHARTLAKE(dev_priv));
return PCH_MCC;
case INTEL_PCH_TGP_DEVICE_ID_TYPE:
DRM_DEBUG_KMS("Found Tiger Lake LP PCH\n");
WARN_ON(!IS_TIGERLAKE(dev_priv));
return PCH_TGP;
default:
return PCH_NONE;
}
}
static bool intel_is_virt_pch(unsigned short id,
unsigned short svendor, unsigned short sdevice)
{
return (id == INTEL_PCH_P2X_DEVICE_ID_TYPE ||
id == INTEL_PCH_P3X_DEVICE_ID_TYPE ||
(id == INTEL_PCH_QEMU_DEVICE_ID_TYPE &&
svendor == PCI_SUBVENDOR_ID_REDHAT_QUMRANET &&
sdevice == PCI_SUBDEVICE_ID_QEMU));
}
static unsigned short
intel_virt_detect_pch(const struct drm_i915_private *dev_priv)
{
unsigned short id = 0;
/*
* In a virtualized passthrough environment we can be in a
* setup where the ISA bridge is not able to be passed through.
* In this case, a south bridge can be emulated and we have to
* make an educated guess as to which PCH is really there.
*/
if (IS_TIGERLAKE(dev_priv))
id = INTEL_PCH_TGP_DEVICE_ID_TYPE;
else if (IS_ELKHARTLAKE(dev_priv))
id = INTEL_PCH_MCC_DEVICE_ID_TYPE;
else if (IS_ICELAKE(dev_priv))
id = INTEL_PCH_ICP_DEVICE_ID_TYPE;
else if (IS_CANNONLAKE(dev_priv) || IS_COFFEELAKE(dev_priv))
id = INTEL_PCH_CNP_DEVICE_ID_TYPE;
else if (IS_KABYLAKE(dev_priv) || IS_SKYLAKE(dev_priv))
id = INTEL_PCH_SPT_DEVICE_ID_TYPE;
else if (IS_HSW_ULT(dev_priv) || IS_BDW_ULT(dev_priv))
id = INTEL_PCH_LPT_LP_DEVICE_ID_TYPE;
else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
id = INTEL_PCH_LPT_DEVICE_ID_TYPE;
else if (IS_GEN(dev_priv, 6) || IS_IVYBRIDGE(dev_priv))
id = INTEL_PCH_CPT_DEVICE_ID_TYPE;
else if (IS_GEN(dev_priv, 5))
id = INTEL_PCH_IBX_DEVICE_ID_TYPE;
if (id)
DRM_DEBUG_KMS("Assuming PCH ID %04x\n", id);
else
DRM_DEBUG_KMS("Assuming no PCH\n");
return id;
}
static void intel_detect_pch(struct drm_i915_private *dev_priv)
{
struct pci_dev *pch = NULL;
/*
* The reason to probe ISA bridge instead of Dev31:Fun0 is to
* make graphics device passthrough work easy for VMM, that only
* need to expose ISA bridge to let driver know the real hardware
* underneath. This is a requirement from virtualization team.
*
* In some virtualized environments (e.g. XEN), there is irrelevant
* ISA bridge in the system. To work reliably, we should scan trhough
* all the ISA bridge devices and check for the first match, instead
* of only checking the first one.
*/
while ((pch = pci_get_class(PCI_CLASS_BRIDGE_ISA << 8, pch))) {
unsigned short id;
enum intel_pch pch_type;
if (pch->vendor != PCI_VENDOR_ID_INTEL)
continue;
id = pch->device & INTEL_PCH_DEVICE_ID_MASK;
pch_type = intel_pch_type(dev_priv, id);
if (pch_type != PCH_NONE) {
dev_priv->pch_type = pch_type;
dev_priv->pch_id = id;
break;
} else if (intel_is_virt_pch(id, pch->subsystem_vendor,
pch->subsystem_device)) {
id = intel_virt_detect_pch(dev_priv);
pch_type = intel_pch_type(dev_priv, id);
/* Sanity check virtual PCH id */
if (WARN_ON(id && pch_type == PCH_NONE))
id = 0;
dev_priv->pch_type = pch_type;
dev_priv->pch_id = id;
break;
}
}
/*
* Use PCH_NOP (PCH but no South Display) for PCH platforms without
* display.
*/
if (pch && !HAS_DISPLAY(dev_priv)) {
DRM_DEBUG_KMS("Display disabled, reverting to NOP PCH\n");
dev_priv->pch_type = PCH_NOP;
dev_priv->pch_id = 0;
}
if (!pch)
DRM_DEBUG_KMS("No PCH found.\n");
pci_dev_put(pch);
}
static int i915_getparam_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
const struct sseu_dev_info *sseu = &RUNTIME_INFO(dev_priv)->sseu;
drm_i915_getparam_t *param = data;
Revert "drm/i915: Expand subslice mask" This reverts commit 1ac159e23c2c ("drm/i915: Expand subslice mask"), which kills ICL due to GEM_BUG_ON() sanity checks before CI even gets a chance to do anything. The commit exposes an issue in commit 1e40d4aea57b ("drm/i915/cnl: Implement WaProgramMgsrForCorrectSliceSpecificMmioReads"), which will also need to be addressed. There's a proposed fix [1], but considering the seeming uncertainty with the fix as well as the size of the regressing commit (in this context, the one that actually brings down ICL), this warrants a revert to get ICL working, and gives us time to get all of this right without rushing. Even if this means shooting the messenger. <3>[ 9.426327] intel_sseu_get_subslices:46 GEM_BUG_ON(slice >= sseu->max_slices) <4>[ 9.426355] ------------[ cut here ]------------ <2>[ 9.426357] kernel BUG at drivers/gpu/drm/i915/gt/intel_sseu.c:46! <4>[ 9.426371] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI <4>[ 9.426377] CPU: 1 PID: 364 Comm: systemd-udevd Not tainted 5.2.0-rc2-CI-CI_DRM_6159+ #1 <4>[ 9.426385] Hardware name: Intel Corporation Ice Lake Client Platform/IceLake U DDR4 SODIMM PD RVP TLC, BIOS ICLSFWR1.R00.3183.A00.1905020411 05/02/2019 <4>[ 9.426444] RIP: 0010:intel_sseu_get_subslices+0x8a/0xe0 [i915] <4>[ 9.426452] Code: d5 76 b7 e0 48 8b 35 9d 24 21 00 49 c7 c0 07 f0 72 a0 b9 2e 00 00 00 48 c7 c2 00 8e 6d a0 48 c7 c7 a5 14 5b a0 e8 36 3c be e0 <0f> 0b 48 c7 c1 80 d5 6f a0 ba 30 00 00 00 48 c7 c6 00 8e 6d a0 48 <4>[ 9.426468] RSP: 0018:ffffc9000037b9c8 EFLAGS: 00010282 <4>[ 9.426475] RAX: 000000000000000f RBX: 0000000000000000 RCX: 0000000000000000 <4>[ 9.426482] RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffff88849e346f98 <4>[ 9.426490] RBP: ffff88848a200000 R08: 0000000000000004 R09: ffff88849d50b000 <4>[ 9.426497] R10: 0000000000000000 R11: ffff88849e346f98 R12: ffff88848a209e78 <4>[ 9.426505] R13: 0000000003000000 R14: ffff88848a20b1a8 R15: 0000000000000000 <4>[ 9.426513] FS: 00007f73d5ae8680(0000) GS:ffff88849fc80000(0000) knlGS:0000000000000000 <4>[ 9.426521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 9.426527] CR2: 0000561417b01260 CR3: 0000000494764003 CR4: 0000000000760ee0 <4>[ 9.426535] PKRU: 55555554 <4>[ 9.426538] Call Trace: <4>[ 9.426585] wa_init_mcr+0xd5/0x110 [i915] <4>[ 9.426597] ? lock_acquire+0xa6/0x1c0 <4>[ 9.426645] icl_gt_workarounds_init+0x21/0x1a0 [i915] <4>[ 9.426694] ? i915_driver_load+0xfcf/0x18a0 [i915] <4>[ 9.426739] gt_init_workarounds+0x14c/0x230 [i915] <4>[ 9.426748] ? _raw_spin_unlock_irq+0x24/0x50 <4>[ 9.426789] intel_gt_init_workarounds+0x1b/0x30 [i915] <4>[ 9.426835] i915_driver_load+0xfd7/0x18a0 [i915] <4>[ 9.426843] ? lock_acquire+0xa6/0x1c0 <4>[ 9.426850] ? __pm_runtime_resume+0x4f/0x80 <4>[ 9.426857] ? _raw_spin_unlock_irqrestore+0x4c/0x60 <4>[ 9.426863] ? _raw_spin_unlock_irqrestore+0x4c/0x60 <4>[ 9.426870] ? lockdep_hardirqs_on+0xe3/0x1b0 <4>[ 9.426915] i915_pci_probe+0x29/0xa0 [i915] <4>[ 9.426923] pci_device_probe+0x9e/0x120 <4>[ 9.426930] really_probe+0xea/0x3c0 <4>[ 9.426936] driver_probe_device+0x10b/0x120 <4>[ 9.426942] device_driver_attach+0x4a/0x50 <4>[ 9.426948] __driver_attach+0x97/0x130 <4>[ 9.426954] ? device_driver_attach+0x50/0x50 <4>[ 9.426960] bus_for_each_dev+0x74/0xc0 <4>[ 9.426966] bus_add_driver+0x13f/0x210 <4>[ 9.426971] ? 0xffffffffa083b000 <4>[ 9.426976] driver_register+0x56/0xe0 <4>[ 9.426982] ? 0xffffffffa083b000 <4>[ 9.426987] do_one_initcall+0x58/0x300 <4>[ 9.426994] ? do_init_module+0x1d/0x1f6 <4>[ 9.427001] ? rcu_read_lock_sched_held+0x6f/0x80 <4>[ 9.427007] ? kmem_cache_alloc_trace+0x261/0x290 <4>[ 9.427014] do_init_module+0x56/0x1f6 <4>[ 9.427020] load_module+0x24d1/0x2990 <4>[ 9.427032] ? __se_sys_finit_module+0xd3/0xf0 <4>[ 9.427037] __se_sys_finit_module+0xd3/0xf0 <4>[ 9.427047] do_syscall_64+0x55/0x1c0 <4>[ 9.427053] entry_SYSCALL_64_after_hwframe+0x49/0xbe <4>[ 9.427059] RIP: 0033:0x7f73d5609839 <4>[ 9.427064] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48 <4>[ 9.427082] RSP: 002b:00007ffdf34477b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 <4>[ 9.427091] RAX: ffffffffffffffda RBX: 00005559fd5d7b40 RCX: 00007f73d5609839 <4>[ 9.427099] RDX: 0000000000000000 RSI: 00007f73d52e8145 RDI: 000000000000000f <4>[ 9.427106] RBP: 00007f73d52e8145 R08: 0000000000000000 R09: 00007ffdf34478d0 <4>[ 9.427114] R10: 000000000000000f R11: 0000000000000246 R12: 0000000000000000 <4>[ 9.427121] R13: 00005559fd5c90f0 R14: 0000000000020000 R15: 00005559fd5d7b40 <4>[ 9.427131] Modules linked in: i915(+) mei_hdcp x86_pkg_temp_thermal coretemp snd_hda_intel crct10dif_pclmul crc32_pclmul snd_hda_codec snd_hwdep e1000e snd_hda_core ghash_clmulni_intel ptp snd_pcm cdc_ether usbnet mii pps_core mei_me mei prime_numbers btusb btrtl btbcm btintel bluetooth ecdh_generic ecc <4>[ 9.427254] ---[ end trace af3eeb543bd66e66 ]--- [1] http://patchwork.freedesktop.org/patch/msgid/20190528200655.11605-1-chris@chris-wilson.co.uk References: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6159/fi-icl-u2/pstore0-1517155098_Oops_1.log References: 1e40d4aea57b ("drm/i915/cnl: Implement WaProgramMgsrForCorrectSliceSpecificMmioReads") Fixes: 1ac159e23c2c ("drm/i915: Expand subslice mask") Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Cc: Manasi Navare <manasi.d.navare@intel.com> Cc: Michel Thierry <michel.thierry@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Oscar Mateo <oscar.mateo@intel.com> Cc: Stuart Summers <stuart.summers@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Yunwei Zhang <yunwei.zhang@intel.com> Acked-by: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190529082150.31526-1-jani.nikula@intel.com
2019-05-29 11:21:50 +03:00
int value;
switch (param->param) {
case I915_PARAM_IRQ_ACTIVE:
case I915_PARAM_ALLOW_BATCHBUFFER:
case I915_PARAM_LAST_DISPATCH:
drm/i915: Drop support for I915_EXEC_CONSTANTS_* execbuf parameters. This patch makes the I915_PARAM_HAS_EXEC_CONSTANTS getparam return 0 (indicating the optional feature is not supported), and makes execbuf always return -EINVAL if the flags are used. Apparently, no userspace ever shipped which used this optional feature: I checked the git history of Mesa, xf86-video-intel, libva, and Beignet, and there were zero commits showing a use of these flags. Kernel commit 72bfa19c8deb4 apparently introduced the feature prematurely. According to Chris, the intention was to use this in cairo-drm, but "the use was broken for gen6", so I don't think it ever happened. 'relative_constants_mode' has always been tracked per-device, but this has actually been wrong ever since hardware contexts were introduced, as the INSTPM register is saved (and automatically restored) as part of the render ring context. The software per-device value could therefore get out of sync with the hardware per-context value. This meant that using them is actually unsafe: a client which tried to use them could damage the state of other clients, causing the GPU to interpret their BO offsets as absolute pointers, leading to bogus memory reads. These flags were also never ported to execlist mode, making them no-ops on Gen9+ (which requires execlists), and Gen8 in the default mode. On Gen8+, userspace can write these registers directly, achieving the same effect. On Gen6-7.5, it likely makes sense to extend the command parser to support them. I don't think anyone wants this on Gen4-5. Based on a patch by Dave Gordon. v3: Return -ENODEV for the getparam, as this is what we do for other obsolete features. Suggested by Chris Wilson. Cc: stable@vger.kernel.org Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92448 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170215093446.21291-1-kenneth@whitecape.org Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
2017-02-15 12:34:46 +03:00
case I915_PARAM_HAS_EXEC_CONSTANTS:
/* Reject all old ums/dri params. */
return -ENODEV;
case I915_PARAM_CHIPSET_ID:
value = pdev->device;
break;
case I915_PARAM_REVISION:
value = pdev->revision;
break;
case I915_PARAM_NUM_FENCES_AVAIL:
value = dev_priv->ggtt.num_fences;
break;
case I915_PARAM_HAS_OVERLAY:
value = dev_priv->overlay ? 1 : 0;
break;
case I915_PARAM_HAS_BSD:
value = !!dev_priv->engine[VCS0];
break;
case I915_PARAM_HAS_BLT:
value = !!dev_priv->engine[BCS0];
break;
case I915_PARAM_HAS_VEBOX:
value = !!dev_priv->engine[VECS0];
break;
case I915_PARAM_HAS_BSD2:
value = !!dev_priv->engine[VCS1];
break;
case I915_PARAM_HAS_LLC:
value = HAS_LLC(dev_priv);
break;
case I915_PARAM_HAS_WT:
value = HAS_WT(dev_priv);
break;
case I915_PARAM_HAS_ALIASING_PPGTT:
value = INTEL_PPGTT(dev_priv);
break;
case I915_PARAM_HAS_SEMAPHORES:
drm/i915: Use HW semaphores for inter-engine synchronisation on gen8+ Having introduced per-context seqno, we now have a means to identity progress across the system without feel of rollback as befell the global_seqno. That is we can program a MI_SEMAPHORE_WAIT operation in advance of submission safe in the knowledge that our target seqno and address is stable. However, since we are telling the GPU to busy-spin on the target address until it matches the signaling seqno, we only want to do so when we are sure that busy-spin will be completed quickly. To achieve this we only submit the request to HW once the signaler is itself executing (modulo preemption causing us to wait longer), and we only do so for default and above priority requests (so that idle priority tasks never themselves hog the GPU waiting for others). As might be reasonably expected, HW semaphores excel in inter-engine synchronisation microbenchmarks (where the 3x reduced latency / increased throughput more than offset the power cost of spinning on a second ring) and have significant improvement (can be up to ~10%, most see no change) for single clients that utilize multiple engines (typically media players and transcoders), without regressing multiple clients that can saturate the system or changing the power envelope dramatically. v3: Drop the older NEQ branch, now we pin the signaler's HWSP anyway. v4: Tell the world and include it as part of scheduler caps. Testcase: igt/gem_exec_whisper Testcase: igt/benchmarks/gem_wsim Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190301170901.8340-3-chris@chris-wilson.co.uk
2019-03-01 20:09:00 +03:00
value = !!(dev_priv->caps.scheduler & I915_SCHEDULER_CAP_SEMAPHORES);
break;
case I915_PARAM_HAS_SECURE_BATCHES:
value = capable(CAP_SYS_ADMIN);
break;
case I915_PARAM_CMD_PARSER_VERSION:
value = i915_cmd_parser_get_version(dev_priv);
break;
case I915_PARAM_SUBSLICE_TOTAL:
value = intel_sseu_subslice_total(sseu);
if (!value)
return -ENODEV;
break;
case I915_PARAM_EU_TOTAL:
value = sseu->eu_total;
if (!value)
return -ENODEV;
break;
case I915_PARAM_HAS_GPU_RESET:
value = i915_modparams.enable_hangcheck &&
intel_has_gpu_reset(dev_priv);
drm/i915: Modify error handler for per engine hang recovery This is a preparatory patch which modifies error handler to do per engine hang recovery. The actual patch which implements this sequence follows later in the series. The aim is to prepare existing recovery function to adapt to this new function where applicable (which fails at this point because core implementation is lacking) and continue recovery using legacy full gpu reset. A helper function is also added to query the availability of engine reset. A subsequent patch will add the capability to query which type of reset is present (engine -> full -> no-reset) via the get-param ioctl. It has been decided that the error events that are used to notify user of reset will only be sent in case if full chip reset. In case of just single (or multiple) engine resets, userspace won't be notified by these events. Note that this implementation of engine reset is for i915 directly submitting to the ELSP, where the driver manages the hang detection, recovery and resubmission. With GuC submission these tasks are shared between driver and firmware; i915 will still responsible for detecting a hang, and when it does it will have to request GuC to reset that Engine and remind the firmware about the outstanding submissions. This will be added in different patch. v2: rebase, advertise engine reset availability in platform definition, add note about GuC submission. v3: s/*engine_reset*/*reset_engine*/. (Chris) Handle reset as 2 level resets, by first going to engine only and fall backing to full/chip reset as needed, i.e. reset_engine will need the struct_mutex. v4: Pass the engine mask to i915_reset. (Chris) v5: Rebase, update selftests. v6: Rebase, prepare for mutex-less reset engine. v7: Pass reset_engine mask as a function parameter, and iterate over the engine mask for reset_engine. (Chris) v8: Use i915.reset >=2 in has_reset_engine; remove redundant reset logging; add a reset-engine-in-progress flag to prevent concurrent resets, and avoid dual purposing of reset-backoff. (Chris) v9: Support reset of different engines in parallel (Chris) v10: Handle reset-engine flag locking better (Chris) v11: Squash in reporting of per-engine-reset availability. Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Signed-off-by: Ian Lister <ian.lister@intel.com> Signed-off-by: Tomas Elf <tomas.elf@intel.com> Signed-off-by: Arun Siluvery <arun.siluvery@linux.intel.com> Signed-off-by: Michel Thierry <michel.thierry@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170615201828.23144-4-michel.thierry@intel.com Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170620095751.13127-5-chris@chris-wilson.co.uk
2017-06-20 12:57:46 +03:00
if (value && intel_has_reset_engine(dev_priv))
value = 2;
break;
case I915_PARAM_HAS_RESOURCE_STREAMER:
value = 0;
break;
case I915_PARAM_HAS_POOLED_EU:
value = HAS_POOLED_EU(dev_priv);
break;
case I915_PARAM_MIN_EU_IN_POOL:
value = sseu->min_eu_in_pool;
break;
case I915_PARAM_HUC_STATUS:
value = intel_huc_check_status(&dev_priv->gt.uc.huc);
if (value < 0)
return value;
break;
case I915_PARAM_MMAP_GTT_VERSION:
/* Though we've started our numbering from 1, and so class all
* earlier versions as 0, in effect their value is undefined as
* the ioctl will report EINVAL for the unknown param!
*/
value = i915_gem_mmap_gtt_version();
break;
case I915_PARAM_HAS_SCHEDULER:
value = dev_priv->caps.scheduler;
break;
drm/i915/execlists: Preemption! When we write to ELSP, it triggers a context preemption at the earliest arbitration point (3DPRIMITIVE, some PIPECONTROLs, a few other operations and the explicit MI_ARB_CHECK). If this is to the same context, it triggers a LITE_RESTORE where the RING_TAIL is merely updated (used currently to chain requests from the same context together, avoiding bubbles). However, if it is to a different context, a full context-switch is performed and it will start to execute the new context saving the image of the old for later execution. Previously we avoided preemption by only submitting a new context when the old was idle. But now we wish embrace it, and if the new request has a higher priority than the currently executing request, we write to the ELSP regardless, thus triggering preemption, but we tell the GPU to switch to our special preemption context (not the target). In the context-switch interrupt handler, we know that the previous contexts have finished execution and so can unwind all the incomplete requests and compute the new highest priority request to execute. It would be feasible to avoid the switch-to-idle intermediate by programming the ELSP with the target context. The difficulty is in tracking which request that should be whilst maintaining the dependency change, the error comes in with coalesced requests. As we only track the most recent request and its priority, we may run into the issue of being tricked in preempting a high priority request that was followed by a low priority request from the same context (e.g. for PI); worse still that earlier request may be our own dependency and the order then broken by preemption. By injecting the switch-to-idle and then recomputing the priority queue, we avoid the issue with tracking in-flight coalesced requests. Having tried the preempt-to-busy approach, and failed to find a way around the coalesced priority issue, Michal's original proposal to inject an idle context (based on handling GuC preemption) succeeds. The current heuristic for deciding when to preempt are only if the new request is of higher priority, and has the privileged priority of greater than 0. Note that the scheduler remains unfair! v2: Disable for gen8 (bdw/bsw) as we need additional w/a for GPGPU. Since, the feature is now conditional and not always available when we have a scheduler, make it known via the HAS_SCHEDULER GETPARAM (now a capability mask). v3: Stylistic tweaks. v4: Appease Joonas with a snippet of kerneldoc, only to fuel to fire of the preempt vs preempting debate. Suggested-by: Michal Winiarski <michal.winiarski@intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Michal Winiarski <michal.winiarski@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Ben Widawsky <benjamin.widawsky@intel.com> Cc: Zhenyu Wang <zhenyuw@linux.intel.com> Cc: Zhi Wang <zhi.a.wang@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171003203453.15692-8-chris@chris-wilson.co.uk
2017-10-03 23:34:52 +03:00
case I915_PARAM_MMAP_VERSION:
/* Remember to bump this if the version changes! */
case I915_PARAM_HAS_GEM:
case I915_PARAM_HAS_PAGEFLIPPING:
case I915_PARAM_HAS_EXECBUF2: /* depends on GEM */
case I915_PARAM_HAS_RELAXED_FENCING:
case I915_PARAM_HAS_COHERENT_RINGS:
case I915_PARAM_HAS_RELAXED_DELTA:
case I915_PARAM_HAS_GEN7_SOL_RESET:
case I915_PARAM_HAS_WAIT_TIMEOUT:
case I915_PARAM_HAS_PRIME_VMAP_FLUSH:
case I915_PARAM_HAS_PINNED_BATCHES:
case I915_PARAM_HAS_EXEC_NO_RELOC:
case I915_PARAM_HAS_EXEC_HANDLE_LUT:
case I915_PARAM_HAS_COHERENT_PHYS_GTT:
case I915_PARAM_HAS_EXEC_SOFTPIN:
drm/i915: Enable userspace to opt-out of implicit fencing Userspace is faced with a dilemma. The kernel requires implicit fencing to manage resource usage (we always must wait for the GPU to finish before releasing its PTE) and for third parties. However, userspace may wish to avoid this serialisation if it is either using explicit fencing between parties and wants more fine-grained access to buffers (e.g. it may partition the buffer between uses and track fences on ranges rather than the implicit fences tracking the whole object). It follows that userspace needs a mechanism to avoid the kernel's serialisation on its implicit fences before execbuf execution. The next question is whether this is an object, execbuf or context flag. Hybrid users (such as using explicit EGL_ANDROID_native_sync fencing on shared winsys buffers, but implicit fencing on internal surfaces) require a per-object level flag. Given that this flag need to be only set once for the lifetime of the object, this reduces the convenience of having an execbuf or context level flag (and avoids having multiple pieces of uABI controlling the same feature). Incorrect use of this flag will result in rendering corruption and GPU hangs - but will not result in use-after-free or similar resource tracking issues. Serious caveat: write ordering is not strictly correct after setting this flag on a render target on multiple engines. This affects all subsequent GEM operations (execbuf, set-domain, pread) and shared dma-buf operations. A fix is possible - but costly (both in terms of further ABI changes and runtime overhead). Testcase: igt/gem_exec_async Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Acked-by: Chad Versace <chadversary@chromium.org> Link: http://patchwork.freedesktop.org/patch/msgid/20170127094008.27489-1-chris@chris-wilson.co.uk
2017-01-27 12:40:07 +03:00
case I915_PARAM_HAS_EXEC_ASYNC:
case I915_PARAM_HAS_EXEC_FENCE:
case I915_PARAM_HAS_EXEC_CAPTURE:
case I915_PARAM_HAS_EXEC_BATCH_FIRST:
case I915_PARAM_HAS_EXEC_FENCE_ARRAY:
2019-05-22 00:11:34 +03:00
case I915_PARAM_HAS_EXEC_SUBMIT_FENCE:
/* For the time being all of these are always true;
* if some supported hardware does not have one of these
* features this value needs to be provided from
* INTEL_INFO(), a feature macro, or similar.
*/
value = 1;
break;
case I915_PARAM_HAS_CONTEXT_ISOLATION:
value = intel_engines_has_context_isolation(dev_priv);
break;
case I915_PARAM_SLICE_MASK:
value = sseu->slice_mask;
if (!value)
return -ENODEV;
break;
case I915_PARAM_SUBSLICE_MASK:
Revert "drm/i915: Expand subslice mask" This reverts commit 1ac159e23c2c ("drm/i915: Expand subslice mask"), which kills ICL due to GEM_BUG_ON() sanity checks before CI even gets a chance to do anything. The commit exposes an issue in commit 1e40d4aea57b ("drm/i915/cnl: Implement WaProgramMgsrForCorrectSliceSpecificMmioReads"), which will also need to be addressed. There's a proposed fix [1], but considering the seeming uncertainty with the fix as well as the size of the regressing commit (in this context, the one that actually brings down ICL), this warrants a revert to get ICL working, and gives us time to get all of this right without rushing. Even if this means shooting the messenger. <3>[ 9.426327] intel_sseu_get_subslices:46 GEM_BUG_ON(slice >= sseu->max_slices) <4>[ 9.426355] ------------[ cut here ]------------ <2>[ 9.426357] kernel BUG at drivers/gpu/drm/i915/gt/intel_sseu.c:46! <4>[ 9.426371] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI <4>[ 9.426377] CPU: 1 PID: 364 Comm: systemd-udevd Not tainted 5.2.0-rc2-CI-CI_DRM_6159+ #1 <4>[ 9.426385] Hardware name: Intel Corporation Ice Lake Client Platform/IceLake U DDR4 SODIMM PD RVP TLC, BIOS ICLSFWR1.R00.3183.A00.1905020411 05/02/2019 <4>[ 9.426444] RIP: 0010:intel_sseu_get_subslices+0x8a/0xe0 [i915] <4>[ 9.426452] Code: d5 76 b7 e0 48 8b 35 9d 24 21 00 49 c7 c0 07 f0 72 a0 b9 2e 00 00 00 48 c7 c2 00 8e 6d a0 48 c7 c7 a5 14 5b a0 e8 36 3c be e0 <0f> 0b 48 c7 c1 80 d5 6f a0 ba 30 00 00 00 48 c7 c6 00 8e 6d a0 48 <4>[ 9.426468] RSP: 0018:ffffc9000037b9c8 EFLAGS: 00010282 <4>[ 9.426475] RAX: 000000000000000f RBX: 0000000000000000 RCX: 0000000000000000 <4>[ 9.426482] RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffff88849e346f98 <4>[ 9.426490] RBP: ffff88848a200000 R08: 0000000000000004 R09: ffff88849d50b000 <4>[ 9.426497] R10: 0000000000000000 R11: ffff88849e346f98 R12: ffff88848a209e78 <4>[ 9.426505] R13: 0000000003000000 R14: ffff88848a20b1a8 R15: 0000000000000000 <4>[ 9.426513] FS: 00007f73d5ae8680(0000) GS:ffff88849fc80000(0000) knlGS:0000000000000000 <4>[ 9.426521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>[ 9.426527] CR2: 0000561417b01260 CR3: 0000000494764003 CR4: 0000000000760ee0 <4>[ 9.426535] PKRU: 55555554 <4>[ 9.426538] Call Trace: <4>[ 9.426585] wa_init_mcr+0xd5/0x110 [i915] <4>[ 9.426597] ? lock_acquire+0xa6/0x1c0 <4>[ 9.426645] icl_gt_workarounds_init+0x21/0x1a0 [i915] <4>[ 9.426694] ? i915_driver_load+0xfcf/0x18a0 [i915] <4>[ 9.426739] gt_init_workarounds+0x14c/0x230 [i915] <4>[ 9.426748] ? _raw_spin_unlock_irq+0x24/0x50 <4>[ 9.426789] intel_gt_init_workarounds+0x1b/0x30 [i915] <4>[ 9.426835] i915_driver_load+0xfd7/0x18a0 [i915] <4>[ 9.426843] ? lock_acquire+0xa6/0x1c0 <4>[ 9.426850] ? __pm_runtime_resume+0x4f/0x80 <4>[ 9.426857] ? _raw_spin_unlock_irqrestore+0x4c/0x60 <4>[ 9.426863] ? _raw_spin_unlock_irqrestore+0x4c/0x60 <4>[ 9.426870] ? lockdep_hardirqs_on+0xe3/0x1b0 <4>[ 9.426915] i915_pci_probe+0x29/0xa0 [i915] <4>[ 9.426923] pci_device_probe+0x9e/0x120 <4>[ 9.426930] really_probe+0xea/0x3c0 <4>[ 9.426936] driver_probe_device+0x10b/0x120 <4>[ 9.426942] device_driver_attach+0x4a/0x50 <4>[ 9.426948] __driver_attach+0x97/0x130 <4>[ 9.426954] ? device_driver_attach+0x50/0x50 <4>[ 9.426960] bus_for_each_dev+0x74/0xc0 <4>[ 9.426966] bus_add_driver+0x13f/0x210 <4>[ 9.426971] ? 0xffffffffa083b000 <4>[ 9.426976] driver_register+0x56/0xe0 <4>[ 9.426982] ? 0xffffffffa083b000 <4>[ 9.426987] do_one_initcall+0x58/0x300 <4>[ 9.426994] ? do_init_module+0x1d/0x1f6 <4>[ 9.427001] ? rcu_read_lock_sched_held+0x6f/0x80 <4>[ 9.427007] ? kmem_cache_alloc_trace+0x261/0x290 <4>[ 9.427014] do_init_module+0x56/0x1f6 <4>[ 9.427020] load_module+0x24d1/0x2990 <4>[ 9.427032] ? __se_sys_finit_module+0xd3/0xf0 <4>[ 9.427037] __se_sys_finit_module+0xd3/0xf0 <4>[ 9.427047] do_syscall_64+0x55/0x1c0 <4>[ 9.427053] entry_SYSCALL_64_after_hwframe+0x49/0xbe <4>[ 9.427059] RIP: 0033:0x7f73d5609839 <4>[ 9.427064] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48 <4>[ 9.427082] RSP: 002b:00007ffdf34477b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 <4>[ 9.427091] RAX: ffffffffffffffda RBX: 00005559fd5d7b40 RCX: 00007f73d5609839 <4>[ 9.427099] RDX: 0000000000000000 RSI: 00007f73d52e8145 RDI: 000000000000000f <4>[ 9.427106] RBP: 00007f73d52e8145 R08: 0000000000000000 R09: 00007ffdf34478d0 <4>[ 9.427114] R10: 000000000000000f R11: 0000000000000246 R12: 0000000000000000 <4>[ 9.427121] R13: 00005559fd5c90f0 R14: 0000000000020000 R15: 00005559fd5d7b40 <4>[ 9.427131] Modules linked in: i915(+) mei_hdcp x86_pkg_temp_thermal coretemp snd_hda_intel crct10dif_pclmul crc32_pclmul snd_hda_codec snd_hwdep e1000e snd_hda_core ghash_clmulni_intel ptp snd_pcm cdc_ether usbnet mii pps_core mei_me mei prime_numbers btusb btrtl btbcm btintel bluetooth ecdh_generic ecc <4>[ 9.427254] ---[ end trace af3eeb543bd66e66 ]--- [1] http://patchwork.freedesktop.org/patch/msgid/20190528200655.11605-1-chris@chris-wilson.co.uk References: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6159/fi-icl-u2/pstore0-1517155098_Oops_1.log References: 1e40d4aea57b ("drm/i915/cnl: Implement WaProgramMgsrForCorrectSliceSpecificMmioReads") Fixes: 1ac159e23c2c ("drm/i915: Expand subslice mask") Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Cc: Manasi Navare <manasi.d.navare@intel.com> Cc: Michel Thierry <michel.thierry@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Oscar Mateo <oscar.mateo@intel.com> Cc: Stuart Summers <stuart.summers@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Yunwei Zhang <yunwei.zhang@intel.com> Acked-by: Daniel Vetter <daniel@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190529082150.31526-1-jani.nikula@intel.com
2019-05-29 11:21:50 +03:00
value = sseu->subslice_mask[0];
if (!value)
return -ENODEV;
break;
case I915_PARAM_CS_TIMESTAMP_FREQUENCY:
value = 1000 * RUNTIME_INFO(dev_priv)->cs_timestamp_frequency_khz;
break;
case I915_PARAM_MMAP_GTT_COHERENT:
value = INTEL_INFO(dev_priv)->has_coherent_ggtt;
break;
default:
DRM_DEBUG("Unknown parameter %d\n", param->param);
return -EINVAL;
}
if (put_user(value, param->value))
return -EFAULT;
return 0;
}
static int i915_get_bridge_dev(struct drm_i915_private *dev_priv)
{
int domain = pci_domain_nr(dev_priv->drm.pdev->bus);
dev_priv->bridge_dev =
pci_get_domain_bus_and_slot(domain, 0, PCI_DEVFN(0, 0));
if (!dev_priv->bridge_dev) {
DRM_ERROR("bridge device not found\n");
return -1;
}
return 0;
}
/* Allocate space for the MCH regs if needed, return nonzero on error */
static int
intel_alloc_mchbar_resource(struct drm_i915_private *dev_priv)
{
int reg = INTEL_GEN(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
u32 temp_lo, temp_hi = 0;
u64 mchbar_addr;
int ret;
if (INTEL_GEN(dev_priv) >= 4)
pci_read_config_dword(dev_priv->bridge_dev, reg + 4, &temp_hi);
pci_read_config_dword(dev_priv->bridge_dev, reg, &temp_lo);
mchbar_addr = ((u64)temp_hi << 32) | temp_lo;
/* If ACPI doesn't have it, assume we need to allocate it ourselves */
#ifdef CONFIG_PNP
if (mchbar_addr &&
pnp_range_reserved(mchbar_addr, mchbar_addr + MCHBAR_SIZE))
return 0;
#endif
/* Get some space for it */
dev_priv->mch_res.name = "i915 MCHBAR";
dev_priv->mch_res.flags = IORESOURCE_MEM;
ret = pci_bus_alloc_resource(dev_priv->bridge_dev->bus,
&dev_priv->mch_res,
MCHBAR_SIZE, MCHBAR_SIZE,
PCIBIOS_MIN_MEM,
0, pcibios_align_resource,
dev_priv->bridge_dev);
if (ret) {
DRM_DEBUG_DRIVER("failed bus alloc: %d\n", ret);
dev_priv->mch_res.start = 0;
return ret;
}
if (INTEL_GEN(dev_priv) >= 4)
pci_write_config_dword(dev_priv->bridge_dev, reg + 4,
upper_32_bits(dev_priv->mch_res.start));
pci_write_config_dword(dev_priv->bridge_dev, reg,
lower_32_bits(dev_priv->mch_res.start));
return 0;
}
/* Setup MCHBAR if possible, return true if we should disable it again */
static void
intel_setup_mchbar(struct drm_i915_private *dev_priv)
{
int mchbar_reg = INTEL_GEN(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
u32 temp;
bool enabled;
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
return;
dev_priv->mchbar_need_disable = false;
if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
pci_read_config_dword(dev_priv->bridge_dev, DEVEN, &temp);
enabled = !!(temp & DEVEN_MCHBAR_EN);
} else {
pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
enabled = temp & 1;
}
/* If it's already enabled, don't have to do anything */
if (enabled)
return;
if (intel_alloc_mchbar_resource(dev_priv))
return;
dev_priv->mchbar_need_disable = true;
/* Space is allocated or reserved, so enable it. */
if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
temp | DEVEN_MCHBAR_EN);
} else {
pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg, temp | 1);
}
}
static void
intel_teardown_mchbar(struct drm_i915_private *dev_priv)
{
int mchbar_reg = INTEL_GEN(dev_priv) >= 4 ? MCHBAR_I965 : MCHBAR_I915;
if (dev_priv->mchbar_need_disable) {
if (IS_I915G(dev_priv) || IS_I915GM(dev_priv)) {
u32 deven_val;
pci_read_config_dword(dev_priv->bridge_dev, DEVEN,
&deven_val);
deven_val &= ~DEVEN_MCHBAR_EN;
pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
deven_val);
} else {
u32 mchbar_val;
pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg,
&mchbar_val);
mchbar_val &= ~1;
pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg,
mchbar_val);
}
}
if (dev_priv->mch_res.start)
release_resource(&dev_priv->mch_res);
}
/* true = enable decode, false = disable decoder */
static unsigned int i915_vga_set_decode(void *cookie, bool state)
{
struct drm_i915_private *dev_priv = cookie;
intel_modeset_vga_set_state(dev_priv, state);
if (state)
return VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM |
VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM;
else
return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM;
}
static int i915_resume_switcheroo(struct drm_device *dev);
static int i915_suspend_switcheroo(struct drm_device *dev, pm_message_t state);
static void i915_switcheroo_set_state(struct pci_dev *pdev, enum vga_switcheroo_state state)
{
struct drm_device *dev = pci_get_drvdata(pdev);
pm_message_t pmm = { .event = PM_EVENT_SUSPEND };
if (state == VGA_SWITCHEROO_ON) {
pr_info("switched on\n");
dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
/* i915 resume handler doesn't set to D0 */
pci_set_power_state(pdev, PCI_D0);
i915_resume_switcheroo(dev);
dev->switch_power_state = DRM_SWITCH_POWER_ON;
} else {
pr_info("switched off\n");
dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
i915_suspend_switcheroo(dev, pmm);
dev->switch_power_state = DRM_SWITCH_POWER_OFF;
}
}
static bool i915_switcheroo_can_switch(struct pci_dev *pdev)
{
struct drm_device *dev = pci_get_drvdata(pdev);
/*
* FIXME: open_count is protected by drm_global_mutex but that would lead to
* locking inversion with the driver load path. And the access here is
* completely racy anyway. So don't bother with locking for now.
*/
return dev->open_count == 0;
}
static const struct vga_switcheroo_client_ops i915_switcheroo_ops = {
.set_gpu_state = i915_switcheroo_set_state,
.reprobe = NULL,
.can_switch = i915_switcheroo_can_switch,
};
static int i915_driver_modeset_probe(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
int ret;
if (i915_inject_probe_failure())
return -ENODEV;
if (HAS_DISPLAY(dev_priv)) {
ret = drm_vblank_init(&dev_priv->drm,
INTEL_INFO(dev_priv)->num_pipes);
if (ret)
goto out;
}
intel_bios_init(dev_priv);
/* If we have > 1 VGA cards, then we need to arbitrate access
* to the common VGA resources.
*
* If we are a secondary display controller (!PCI_DISPLAY_CLASS_VGA),
* then we do not take part in VGA arbitration and the
* vga_client_register() fails with -ENODEV.
*/
ret = vga_client_register(pdev, dev_priv, NULL, i915_vga_set_decode);
if (ret && ret != -ENODEV)
goto out;
intel_register_dsm_handler();
ret = vga_switcheroo_register_client(pdev, &i915_switcheroo_ops, false);
if (ret)
goto cleanup_vga_client;
/* must happen before intel_power_domains_init_hw() on VLV/CHV */
intel_update_rawclk(dev_priv);
intel_power_domains_init_hw(dev_priv, false);
intel_csr_ucode_init(dev_priv);
ret = intel_irq_install(dev_priv);
if (ret)
goto cleanup_csr;
intel_gmbus_setup(dev_priv);
/* Important: The output setup functions called by modeset_init need
* working irqs for e.g. gmbus and dp aux transfers. */
ret = intel_modeset_init(dev);
if (ret)
goto cleanup_irq;
ret = i915_gem_init(dev_priv);
if (ret)
goto cleanup_modeset;
intel_overlay_setup(dev_priv);
if (!HAS_DISPLAY(dev_priv))
return 0;
ret = intel_fbdev_init(dev);
if (ret)
goto cleanup_gem;
/* Only enable hotplug handling once the fbdev is fully set up. */
intel_hpd_init(dev_priv);
intel_init_ipc(dev_priv);
return 0;
cleanup_gem:
i915_gem_suspend(dev_priv);
i915_gem_driver_remove(dev_priv);
i915_gem_driver_release(dev_priv);
cleanup_modeset:
intel_modeset_driver_remove(dev);
cleanup_irq:
intel_irq_uninstall(dev_priv);
intel_gmbus_teardown(dev_priv);
cleanup_csr:
intel_csr_ucode_fini(dev_priv);
intel_power_domains_driver_remove(dev_priv);
vga_switcheroo_unregister_client(pdev);
cleanup_vga_client:
vga_client_register(pdev, NULL, NULL, NULL);
out:
return ret;
}
static int i915_kick_out_firmware_fb(struct drm_i915_private *dev_priv)
{
struct apertures_struct *ap;
struct pci_dev *pdev = dev_priv->drm.pdev;
struct i915_ggtt *ggtt = &dev_priv->ggtt;
bool primary;
int ret;
ap = alloc_apertures(1);
if (!ap)
return -ENOMEM;
ap->ranges[0].base = ggtt->gmadr.start;
ap->ranges[0].size = ggtt->mappable_end;
primary =
pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW;
ret = drm_fb_helper_remove_conflicting_framebuffers(ap, "inteldrmfb", primary);
kfree(ap);
return ret;
}
static void intel_init_dpio(struct drm_i915_private *dev_priv)
{
/*
* IOSF_PORT_DPIO is used for VLV x2 PHY (DP/HDMI B and C),
* CHV x1 PHY (DP/HDMI D)
* IOSF_PORT_DPIO_2 is used for CHV x2 PHY (DP/HDMI B and C)
*/
if (IS_CHERRYVIEW(dev_priv)) {
DPIO_PHY_IOSF_PORT(DPIO_PHY0) = IOSF_PORT_DPIO_2;
DPIO_PHY_IOSF_PORT(DPIO_PHY1) = IOSF_PORT_DPIO;
} else if (IS_VALLEYVIEW(dev_priv)) {
DPIO_PHY_IOSF_PORT(DPIO_PHY0) = IOSF_PORT_DPIO;
}
}
static int i915_workqueues_init(struct drm_i915_private *dev_priv)
{
/*
* The i915 workqueue is primarily used for batched retirement of
* requests (and thus managing bo) once the task has been completed
* by the GPU. i915_retire_requests() is called directly when we
* need high-priority retirement, such as waiting for an explicit
* bo.
*
* It is also used for periodic low-priority events, such as
* idle-timers and recording error state.
*
* All tasks on the workqueue are expected to acquire the dev mutex
* so there is no point in running more than one instance of the
* workqueue at any time. Use an ordered one.
*/
dev_priv->wq = alloc_ordered_workqueue("i915", 0);
if (dev_priv->wq == NULL)
goto out_err;
dev_priv->hotplug.dp_wq = alloc_ordered_workqueue("i915-dp", 0);
if (dev_priv->hotplug.dp_wq == NULL)
goto out_free_wq;
return 0;
out_free_wq:
destroy_workqueue(dev_priv->wq);
out_err:
DRM_ERROR("Failed to allocate workqueues.\n");
return -ENOMEM;
}
static void i915_workqueues_cleanup(struct drm_i915_private *dev_priv)
{
destroy_workqueue(dev_priv->hotplug.dp_wq);
destroy_workqueue(dev_priv->wq);
}
/*
* We don't keep the workarounds for pre-production hardware, so we expect our
* driver to fail on these machines in one way or another. A little warning on
* dmesg may help both the user and the bug triagers.
*
* Our policy for removing pre-production workarounds is to keep the
* current gen workarounds as a guide to the bring-up of the next gen
* (workarounds have a habit of persisting!). Anything older than that
* should be removed along with the complications they introduce.
*/
static void intel_detect_preproduction_hw(struct drm_i915_private *dev_priv)
{
bool pre = false;
pre |= IS_HSW_EARLY_SDV(dev_priv);
pre |= IS_SKL_REVID(dev_priv, 0, SKL_REVID_F0);
pre |= IS_BXT_REVID(dev_priv, 0, BXT_REVID_B_LAST);
pre |= IS_KBL_REVID(dev_priv, 0, KBL_REVID_A0);
if (pre) {
DRM_ERROR("This is a pre-production stepping. "
"It may not be fully functional.\n");
add_taint(TAINT_MACHINE_CHECK, LOCKDEP_STILL_OK);
}
}
/**
* i915_driver_early_probe - setup state not requiring device access
* @dev_priv: device private
*
* Initialize everything that is a "SW-only" state, that is state not
* requiring accessing the device or exposing the driver via kernel internal
* or userspace interfaces. Example steps belonging here: lock initialization,
* system memory allocation, setting up device specific attributes and
* function hooks not requiring accessing the device.
*/
static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
{
int ret = 0;
if (i915_inject_probe_failure())
return -ENODEV;
drm/i915: Introduce concept of a sub-platform Concept of a sub-platform already exist in our code (like ULX and ULT platform variants and similar),implemented via the macros which check a list of device ids to determine a match. With this patch we consolidate device ids checking into a single function called during early driver load. A few low bits in the platform mask are reserved for sub-platform identification and defined as a per-platform namespace. At the same time it future proofs the platform_mask handling by preparing the code for easy extending, and tidies the very verbose WARN strings generated when IS_PLATFORM macros are embedded into a WARN type statements. v2: Fixed IS_SUBPLATFORM. Updated commit msg. v3: Chris was right, there is an ordering problem. v4: * Catch-up with new sub-platforms. * Rebase for RUNTIME_INFO. * Drop subplatform mask union tricks and convert platform_mask to an array for extensibility. v5: * Fix subplatform check. * Protect against forgetting to expand subplatform bits. * Remove platform enum tallying. * Add subplatform to error state. (Chris) * Drop macros and just use static inlines. * Remove redundant IRONLAKE_M. (Ville) v6: * Split out Ironlake change. * Optimize subplatform check. * Use __always_inline. (Lucas) * Add platform_mask comment. (Paulo) * Pass stored runtime info in error capture. (Chris) v7: * Rebased for new AML ULX device id. * Bump platform mask array size for EHL. * Stop mentioning device ids in intel_device_subplatform_init by using the trick of splitting macros i915_pciids.h. (Jani) * AML seems to be either a subplatform of KBL or CFL so express it like that. v8: * Use one device id table per subplatform. (Jani) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Suggested-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Jose Souza <jose.souza@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190327142328.31780-1-tvrtko.ursulin@linux.intel.com
2019-03-27 17:23:28 +03:00
intel_device_info_subplatform_init(dev_priv);
intel_uncore_init_early(&dev_priv->uncore, dev_priv);
spin_lock_init(&dev_priv->irq_lock);
spin_lock_init(&dev_priv->gpu_error.lock);
mutex_init(&dev_priv->backlight_lock);
mutex_init(&dev_priv->sb_lock);
drm/i915: Disable preemption and sleeping while using the punit sideband While we talk to the punit over its sideband, we need to prevent the cpu from sleeping in order to prevent a potential machine hang. Note that by itself, it appears that pm_qos_update_request (via intel_idle) doesn't provide a sufficient barrier to ensure that all core are indeed awake (out of Cstate) and that the package is awake. To do so, we need to supplement the pm_qos with a manual ping on_each_cpu. v2: Restrict the heavy-weight wakeup to just the ISOF_PORT_PUNIT, there is insufficient evidence to implicate a wider problem atm. Similarly, restrict the w/a to Valleyview, as Cherryview doesn't have an angry cadre of users. The working theory, courtesy of Ville and Hans, is the issue lies within the power delivery and so is likely to be unit and board specific and occurs when both the unit/fw require extra power at the same time as the cpu package is changing its own power state. References: https://bugzilla.kernel.org/show_bug.cgi?id=109051 References: https://bugs.freedesktop.org/show_bug.cgi?id=102657 References: https://bugzilla.kernel.org/show_bug.cgi?id=195255 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Hans de Goede <hdegoede@redhat.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190426081725.31217-1-chris@chris-wilson.co.uk
2019-04-26 11:17:18 +03:00
pm_qos_add_request(&dev_priv->sb_qos,
PM_QOS_CPU_DMA_LATENCY, PM_QOS_DEFAULT_VALUE);
mutex_init(&dev_priv->av_mutex);
mutex_init(&dev_priv->wm.wm_mutex);
mutex_init(&dev_priv->pps_mutex);
mutex_init(&dev_priv->hdcp_comp_mutex);
drm/i915: Use SSE4.1 movntdqa to accelerate reads from WC memory This patch provides the infrastructure for performing a 16-byte aligned read from WC memory using non-temporal instructions introduced with sse4.1. Using movntdqa we can bypass the CPU caches and read directly from memory and ignoring the page attributes set on the CPU PTE i.e. negating the impact of an otherwise UC access. Copying using movntdqa from WC is almost as fast as reading from WB memory, modulo the possibility of both hitting the CPU cache or leaving the data in the CPU cache for the next consumer. (The CPU cache itself my be flushed for the region of the movntdqa and on later access the movntdqa reads from a separate internal buffer for the cacheline.) The write back to the memory is however cached. This will be used in later patches to accelerate accessing WC memory. v2: Report whether the accelerated copy is successful/possible. v3: Function alignment override was only necessary when using the function target("sse4.1") - which is not necessary for emitting movntdqa from __asm__. v4: Improve notes on CPU cache behaviour vs non-temporal stores. v5: Fix byte offsets for unrolled moves. v6: Find all remaining typos of "movntqda", use kernel_fpu_begin. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Akash Goel <akash.goel@intel.com> Cc: Damien Lespiau <damien.lespiau@intel.com> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1471001999-17787-2-git-send-email-chris@chris-wilson.co.uk
2016-08-12 14:39:59 +03:00
i915_memcpy_init_early(dev_priv);
intel_runtime_pm_init_early(&dev_priv->runtime_pm);
drm/i915: Use SSE4.1 movntdqa to accelerate reads from WC memory This patch provides the infrastructure for performing a 16-byte aligned read from WC memory using non-temporal instructions introduced with sse4.1. Using movntdqa we can bypass the CPU caches and read directly from memory and ignoring the page attributes set on the CPU PTE i.e. negating the impact of an otherwise UC access. Copying using movntdqa from WC is almost as fast as reading from WB memory, modulo the possibility of both hitting the CPU cache or leaving the data in the CPU cache for the next consumer. (The CPU cache itself my be flushed for the region of the movntdqa and on later access the movntdqa reads from a separate internal buffer for the cacheline.) The write back to the memory is however cached. This will be used in later patches to accelerate accessing WC memory. v2: Report whether the accelerated copy is successful/possible. v3: Function alignment override was only necessary when using the function target("sse4.1") - which is not necessary for emitting movntdqa from __asm__. v4: Improve notes on CPU cache behaviour vs non-temporal stores. v5: Fix byte offsets for unrolled moves. v6: Find all remaining typos of "movntqda", use kernel_fpu_begin. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Akash Goel <akash.goel@intel.com> Cc: Damien Lespiau <damien.lespiau@intel.com> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1471001999-17787-2-git-send-email-chris@chris-wilson.co.uk
2016-08-12 14:39:59 +03:00
ret = i915_workqueues_init(dev_priv);
if (ret < 0)
return ret;
intel_gt_init_early(&dev_priv->gt, dev_priv);
ret = i915_gem_init_early(dev_priv);
if (ret < 0)
goto err_workqueues;
/* This must be called before any calls to HAS_PCH_* */
intel_detect_pch(dev_priv);
intel_wopcm_init_early(&dev_priv->wopcm);
intel_uc_init_early(&dev_priv->gt.uc);
intel_pm_setup(dev_priv);
intel_init_dpio(dev_priv);
ret = intel_power_domains_init(dev_priv);
if (ret < 0)
goto err_uc;
intel_irq_init(dev_priv);
intel_init_display_hooks(dev_priv);
intel_init_clock_gating_hooks(dev_priv);
intel_init_audio_hooks(dev_priv);
intel_display_crc_init(dev_priv);
intel_detect_preproduction_hw(dev_priv);
return 0;
err_uc:
intel_uc_cleanup_early(&dev_priv->gt.uc);
i915_gem_cleanup_early(dev_priv);
err_workqueues:
intel_gt_driver_late_release(&dev_priv->gt);
i915_workqueues_cleanup(dev_priv);
return ret;
}
/**
* i915_driver_late_release - cleanup the setup done in
* i915_driver_early_probe()
* @dev_priv: device private
*/
static void i915_driver_late_release(struct drm_i915_private *dev_priv)
{
intel_irq_fini(dev_priv);
intel_power_domains_cleanup(dev_priv);
intel_uc_cleanup_early(&dev_priv->gt.uc);
i915_gem_cleanup_early(dev_priv);
intel_gt_driver_late_release(&dev_priv->gt);
i915_workqueues_cleanup(dev_priv);
drm/i915: Disable preemption and sleeping while using the punit sideband While we talk to the punit over its sideband, we need to prevent the cpu from sleeping in order to prevent a potential machine hang. Note that by itself, it appears that pm_qos_update_request (via intel_idle) doesn't provide a sufficient barrier to ensure that all core are indeed awake (out of Cstate) and that the package is awake. To do so, we need to supplement the pm_qos with a manual ping on_each_cpu. v2: Restrict the heavy-weight wakeup to just the ISOF_PORT_PUNIT, there is insufficient evidence to implicate a wider problem atm. Similarly, restrict the w/a to Valleyview, as Cherryview doesn't have an angry cadre of users. The working theory, courtesy of Ville and Hans, is the issue lies within the power delivery and so is likely to be unit and board specific and occurs when both the unit/fw require extra power at the same time as the cpu package is changing its own power state. References: https://bugzilla.kernel.org/show_bug.cgi?id=109051 References: https://bugs.freedesktop.org/show_bug.cgi?id=102657 References: https://bugzilla.kernel.org/show_bug.cgi?id=195255 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Hans de Goede <hdegoede@redhat.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190426081725.31217-1-chris@chris-wilson.co.uk
2019-04-26 11:17:18 +03:00
pm_qos_remove_request(&dev_priv->sb_qos);
mutex_destroy(&dev_priv->sb_lock);
}
/**
* i915_driver_mmio_probe - setup device MMIO
* @dev_priv: device private
*
* Setup minimal device state necessary for MMIO accesses later in the
* initialization sequence. The setup here should avoid any other device-wide
* side effects or exposing the driver via kernel internal or user space
* interfaces.
*/
static int i915_driver_mmio_probe(struct drm_i915_private *dev_priv)
{
int ret;
if (i915_inject_probe_failure())
return -ENODEV;
if (i915_get_bridge_dev(dev_priv))
return -EIO;
ret = intel_uncore_init_mmio(&dev_priv->uncore);
if (ret < 0)
goto err_bridge;
/* Try to make sure MCHBAR is enabled before poking at it */
intel_setup_mchbar(dev_priv);
drm/i915/icl: Check for fused-off VDBOX and VEBOX instances In Gen11, the Video Decode engines (aka VDBOX, aka VCS, aka BSD) and the Video Enhancement engines (aka VEBOX, aka VECS) could be fused off. Also, each VDBOX and VEBOX has its own power well, which only exist if the related engine exists in the HW. Unfortunately, we have a Catch-22 situation going on: we need the blitter forcewake to read the register with the fuse info, but we cannot initialize the forcewake domains without knowin about the engines present in the HW. We workaround this problem by allowing the initialization of all forcewake domains and then pruning the fused off ones, as per the fuse information. Bspec: 20680 v2: We were shifting incorrectly for vebox disable (Vinay) v3: Assert mmio is ready and warn if we have attempted to initialize forcewake for fused-off engines (Paulo) v4: - Use INTEL_GEN in new code (Tvrtko) - Shorter local variable (Tvrtko, Michal) - Keep "if (!...) continue" style (Tvrtko) - No unnecessary BUG_ON (Tvrtko) - WARN_ON and cleanup if wrong mask (Tvrtko, Michal) - Use I915_READ_FW (Michal) - Use I915_MAX_VCS/VECS macros (Michal) v5: Rebased by Rodrigo fixing conflicts on top of: "drm/i915: Simplify intel_engines_init" v6: Fix v5. Remove info->num_rings. (by Oscar) v7: Rebase (Rodrigo). v8: - s/intel_device_info_fused_off_engines/ intel_device_info_init_mmio (Chris) - Make vdbox_disable & vebox_disable local variables (Chris) v9: - Move function declaration to intel_device_info.h (Michal) - Missing indent in bit fields definitions (Michal) - When RC6 is enabled by BIOS, the fuse register cannot be read until the blitter powerwell is awake. Shuffle where the fuse is read, prune the forcewake domains after the fact and change the commit message accordingly (Vinay, Sagar, Chris). v10: - Improved commit message (Sagar) - New line in header file (Sagar) - Specify the message in fw_domain_reset applies to ICL+ (Sagar) Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Cc: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Sagar Arun Kamble <sagar.a.kamble@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Oscar Mateo <oscar.mateo@intel.com> Reviewed-by: Sagar Arun Kamble <sagar.a.kamble@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180316121456.11577-1-mika.kuoppala@linux.intel.com [Mika: soothe checkpatch on commit msg] Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
2018-03-16 15:14:49 +03:00
intel_device_info_init_mmio(dev_priv);
intel_uncore_prune_mmio_domains(&dev_priv->uncore);
drm/i915/icl: Check for fused-off VDBOX and VEBOX instances In Gen11, the Video Decode engines (aka VDBOX, aka VCS, aka BSD) and the Video Enhancement engines (aka VEBOX, aka VECS) could be fused off. Also, each VDBOX and VEBOX has its own power well, which only exist if the related engine exists in the HW. Unfortunately, we have a Catch-22 situation going on: we need the blitter forcewake to read the register with the fuse info, but we cannot initialize the forcewake domains without knowin about the engines present in the HW. We workaround this problem by allowing the initialization of all forcewake domains and then pruning the fused off ones, as per the fuse information. Bspec: 20680 v2: We were shifting incorrectly for vebox disable (Vinay) v3: Assert mmio is ready and warn if we have attempted to initialize forcewake for fused-off engines (Paulo) v4: - Use INTEL_GEN in new code (Tvrtko) - Shorter local variable (Tvrtko, Michal) - Keep "if (!...) continue" style (Tvrtko) - No unnecessary BUG_ON (Tvrtko) - WARN_ON and cleanup if wrong mask (Tvrtko, Michal) - Use I915_READ_FW (Michal) - Use I915_MAX_VCS/VECS macros (Michal) v5: Rebased by Rodrigo fixing conflicts on top of: "drm/i915: Simplify intel_engines_init" v6: Fix v5. Remove info->num_rings. (by Oscar) v7: Rebase (Rodrigo). v8: - s/intel_device_info_fused_off_engines/ intel_device_info_init_mmio (Chris) - Make vdbox_disable & vebox_disable local variables (Chris) v9: - Move function declaration to intel_device_info.h (Michal) - Missing indent in bit fields definitions (Michal) - When RC6 is enabled by BIOS, the fuse register cannot be read until the blitter powerwell is awake. Shuffle where the fuse is read, prune the forcewake domains after the fact and change the commit message accordingly (Vinay, Sagar, Chris). v10: - Improved commit message (Sagar) - New line in header file (Sagar) - Specify the message in fw_domain_reset applies to ICL+ (Sagar) Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Cc: Vinay Belgaumkar <vinay.belgaumkar@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Sagar Arun Kamble <sagar.a.kamble@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Oscar Mateo <oscar.mateo@intel.com> Reviewed-by: Sagar Arun Kamble <sagar.a.kamble@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180316121456.11577-1-mika.kuoppala@linux.intel.com [Mika: soothe checkpatch on commit msg] Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
2018-03-16 15:14:49 +03:00
intel_uc_init_mmio(&dev_priv->gt.uc);
ret = intel_engines_init_mmio(dev_priv);
if (ret)
goto err_uncore;
i915_gem_init_mmio(dev_priv);
return 0;
err_uncore:
intel_teardown_mchbar(dev_priv);
intel_uncore_fini_mmio(&dev_priv->uncore);
err_bridge:
pci_dev_put(dev_priv->bridge_dev);
return ret;
}
/**
* i915_driver_mmio_release - cleanup the setup done in i915_driver_mmio_probe()
* @dev_priv: device private
*/
static void i915_driver_mmio_release(struct drm_i915_private *dev_priv)
{
intel_engines_cleanup(dev_priv);
intel_teardown_mchbar(dev_priv);
intel_uncore_fini_mmio(&dev_priv->uncore);
pci_dev_put(dev_priv->bridge_dev);
}
static void intel_sanitize_options(struct drm_i915_private *dev_priv)
{
intel_gvt_sanitize_options(dev_priv);
}
#define DRAM_TYPE_STR(type) [INTEL_DRAM_ ## type] = #type
static const char *intel_dram_type_str(enum intel_dram_type type)
{
static const char * const str[] = {
DRAM_TYPE_STR(UNKNOWN),
DRAM_TYPE_STR(DDR3),
DRAM_TYPE_STR(DDR4),
DRAM_TYPE_STR(LPDDR3),
DRAM_TYPE_STR(LPDDR4),
};
if (type >= ARRAY_SIZE(str))
type = INTEL_DRAM_UNKNOWN;
return str[type];
}
#undef DRAM_TYPE_STR
static int intel_dimm_num_devices(const struct dram_dimm_info *dimm)
{
return dimm->ranks * 64 / (dimm->width ?: 1);
}
/* Returns total GB for the whole DIMM */
static int skl_get_dimm_size(u16 val)
{
return val & SKL_DRAM_SIZE_MASK;
}
static int skl_get_dimm_width(u16 val)
{
if (skl_get_dimm_size(val) == 0)
return 0;
switch (val & SKL_DRAM_WIDTH_MASK) {
case SKL_DRAM_WIDTH_X8:
case SKL_DRAM_WIDTH_X16:
case SKL_DRAM_WIDTH_X32:
val = (val & SKL_DRAM_WIDTH_MASK) >> SKL_DRAM_WIDTH_SHIFT;
return 8 << val;
default:
MISSING_CASE(val);
return 0;
}
}
static int skl_get_dimm_ranks(u16 val)
{
if (skl_get_dimm_size(val) == 0)
return 0;
val = (val & SKL_DRAM_RANK_MASK) >> SKL_DRAM_RANK_SHIFT;
return val + 1;
}
/* Returns total GB for the whole DIMM */
static int cnl_get_dimm_size(u16 val)
{
return (val & CNL_DRAM_SIZE_MASK) / 2;
}
static int cnl_get_dimm_width(u16 val)
{
if (cnl_get_dimm_size(val) == 0)
return 0;
switch (val & CNL_DRAM_WIDTH_MASK) {
case CNL_DRAM_WIDTH_X8:
case CNL_DRAM_WIDTH_X16:
case CNL_DRAM_WIDTH_X32:
val = (val & CNL_DRAM_WIDTH_MASK) >> CNL_DRAM_WIDTH_SHIFT;
return 8 << val;
default:
MISSING_CASE(val);
return 0;
}
}
static int cnl_get_dimm_ranks(u16 val)
{
if (cnl_get_dimm_size(val) == 0)
return 0;
val = (val & CNL_DRAM_RANK_MASK) >> CNL_DRAM_RANK_SHIFT;
return val + 1;
}
static bool
skl_is_16gb_dimm(const struct dram_dimm_info *dimm)
{
/* Convert total GB to Gb per DRAM device */
return 8 * dimm->size / (intel_dimm_num_devices(dimm) ?: 1) == 16;
}
static void
skl_dram_get_dimm_info(struct drm_i915_private *dev_priv,
struct dram_dimm_info *dimm,
int channel, char dimm_name, u16 val)
{
if (INTEL_GEN(dev_priv) >= 10) {
dimm->size = cnl_get_dimm_size(val);
dimm->width = cnl_get_dimm_width(val);
dimm->ranks = cnl_get_dimm_ranks(val);
} else {
dimm->size = skl_get_dimm_size(val);
dimm->width = skl_get_dimm_width(val);
dimm->ranks = skl_get_dimm_ranks(val);
}
DRM_DEBUG_KMS("CH%u DIMM %c size: %u GB, width: X%u, ranks: %u, 16Gb DIMMs: %s\n",
channel, dimm_name, dimm->size, dimm->width, dimm->ranks,
yesno(skl_is_16gb_dimm(dimm)));
}
static int
skl_dram_get_channel_info(struct drm_i915_private *dev_priv,
struct dram_channel_info *ch,
int channel, u32 val)
{
skl_dram_get_dimm_info(dev_priv, &ch->dimm_l,
channel, 'L', val & 0xffff);
skl_dram_get_dimm_info(dev_priv, &ch->dimm_s,
channel, 'S', val >> 16);
if (ch->dimm_l.size == 0 && ch->dimm_s.size == 0) {
DRM_DEBUG_KMS("CH%u not populated\n", channel);
return -EINVAL;
}
if (ch->dimm_l.ranks == 2 || ch->dimm_s.ranks == 2)
ch->ranks = 2;
else if (ch->dimm_l.ranks == 1 && ch->dimm_s.ranks == 1)
ch->ranks = 2;
else
ch->ranks = 1;
ch->is_16gb_dimm =
skl_is_16gb_dimm(&ch->dimm_l) ||
skl_is_16gb_dimm(&ch->dimm_s);
DRM_DEBUG_KMS("CH%u ranks: %u, 16Gb DIMMs: %s\n",
channel, ch->ranks, yesno(ch->is_16gb_dimm));
return 0;
}
static bool
intel_is_dram_symmetric(const struct dram_channel_info *ch0,
const struct dram_channel_info *ch1)
{
return !memcmp(ch0, ch1, sizeof(*ch0)) &&
(ch0->dimm_s.size == 0 ||
!memcmp(&ch0->dimm_l, &ch0->dimm_s, sizeof(ch0->dimm_l)));
}
static int
skl_dram_get_channels_info(struct drm_i915_private *dev_priv)
{
struct dram_info *dram_info = &dev_priv->dram_info;
struct dram_channel_info ch0 = {}, ch1 = {};
u32 val;
int ret;
val = I915_READ(SKL_MAD_DIMM_CH0_0_0_0_MCHBAR_MCMAIN);
ret = skl_dram_get_channel_info(dev_priv, &ch0, 0, val);
if (ret == 0)
dram_info->num_channels++;
val = I915_READ(SKL_MAD_DIMM_CH1_0_0_0_MCHBAR_MCMAIN);
ret = skl_dram_get_channel_info(dev_priv, &ch1, 1, val);
if (ret == 0)
dram_info->num_channels++;
if (dram_info->num_channels == 0) {
DRM_INFO("Number of memory channels is zero\n");
return -EINVAL;
}
/*
* If any of the channel is single rank channel, worst case output
* will be same as if single rank memory, so consider single rank
* memory.
*/
if (ch0.ranks == 1 || ch1.ranks == 1)
dram_info->ranks = 1;
else
dram_info->ranks = max(ch0.ranks, ch1.ranks);
if (dram_info->ranks == 0) {
DRM_INFO("couldn't get memory rank information\n");
return -EINVAL;
}
dram_info->is_16gb_dimm = ch0.is_16gb_dimm || ch1.is_16gb_dimm;
dram_info->symmetric_memory = intel_is_dram_symmetric(&ch0, &ch1);
DRM_DEBUG_KMS("Memory configuration is symmetric? %s\n",
yesno(dram_info->symmetric_memory));
return 0;
}
static enum intel_dram_type
skl_get_dram_type(struct drm_i915_private *dev_priv)
{
u32 val;
val = I915_READ(SKL_MAD_INTER_CHANNEL_0_0_0_MCHBAR_MCMAIN);
switch (val & SKL_DRAM_DDR_TYPE_MASK) {
case SKL_DRAM_DDR_TYPE_DDR3:
return INTEL_DRAM_DDR3;
case SKL_DRAM_DDR_TYPE_DDR4:
return INTEL_DRAM_DDR4;
case SKL_DRAM_DDR_TYPE_LPDDR3:
return INTEL_DRAM_LPDDR3;
case SKL_DRAM_DDR_TYPE_LPDDR4:
return INTEL_DRAM_LPDDR4;
default:
MISSING_CASE(val);
return INTEL_DRAM_UNKNOWN;
}
}
static int
skl_get_dram_info(struct drm_i915_private *dev_priv)
{
struct dram_info *dram_info = &dev_priv->dram_info;
u32 mem_freq_khz, val;
int ret;
dram_info->type = skl_get_dram_type(dev_priv);
DRM_DEBUG_KMS("DRAM type: %s\n", intel_dram_type_str(dram_info->type));
ret = skl_dram_get_channels_info(dev_priv);
if (ret)
return ret;
val = I915_READ(SKL_MC_BIOS_DATA_0_0_0_MCHBAR_PCU);
mem_freq_khz = DIV_ROUND_UP((val & SKL_REQ_DATA_MASK) *
SKL_MEMORY_FREQ_MULTIPLIER_HZ, 1000);
dram_info->bandwidth_kbps = dram_info->num_channels *
mem_freq_khz * 8;
if (dram_info->bandwidth_kbps == 0) {
DRM_INFO("Couldn't get system memory bandwidth\n");
return -EINVAL;
}
dram_info->valid = true;
return 0;
}
/* Returns Gb per DRAM device */
static int bxt_get_dimm_size(u32 val)
{
switch (val & BXT_DRAM_SIZE_MASK) {
case BXT_DRAM_SIZE_4GBIT:
return 4;
case BXT_DRAM_SIZE_6GBIT:
return 6;
case BXT_DRAM_SIZE_8GBIT:
return 8;
case BXT_DRAM_SIZE_12GBIT:
return 12;
case BXT_DRAM_SIZE_16GBIT:
return 16;
default:
MISSING_CASE(val);
return 0;
}
}
static int bxt_get_dimm_width(u32 val)
{
if (!bxt_get_dimm_size(val))
return 0;
val = (val & BXT_DRAM_WIDTH_MASK) >> BXT_DRAM_WIDTH_SHIFT;
return 8 << val;
}
static int bxt_get_dimm_ranks(u32 val)
{
if (!bxt_get_dimm_size(val))
return 0;
switch (val & BXT_DRAM_RANK_MASK) {
case BXT_DRAM_RANK_SINGLE:
return 1;
case BXT_DRAM_RANK_DUAL:
return 2;
default:
MISSING_CASE(val);
return 0;
}
}
static enum intel_dram_type bxt_get_dimm_type(u32 val)
{
if (!bxt_get_dimm_size(val))
return INTEL_DRAM_UNKNOWN;
switch (val & BXT_DRAM_TYPE_MASK) {
case BXT_DRAM_TYPE_DDR3:
return INTEL_DRAM_DDR3;
case BXT_DRAM_TYPE_LPDDR3:
return INTEL_DRAM_LPDDR3;
case BXT_DRAM_TYPE_DDR4:
return INTEL_DRAM_DDR4;
case BXT_DRAM_TYPE_LPDDR4:
return INTEL_DRAM_LPDDR4;
default:
MISSING_CASE(val);
return INTEL_DRAM_UNKNOWN;
}
}
static void bxt_get_dimm_info(struct dram_dimm_info *dimm,
u32 val)
{
dimm->width = bxt_get_dimm_width(val);
dimm->ranks = bxt_get_dimm_ranks(val);
/*
* Size in register is Gb per DRAM device. Convert to total
* GB to match the way we report this for non-LP platforms.
*/
dimm->size = bxt_get_dimm_size(val) * intel_dimm_num_devices(dimm) / 8;
}
static int
bxt_get_dram_info(struct drm_i915_private *dev_priv)
{
struct dram_info *dram_info = &dev_priv->dram_info;
u32 dram_channels;
u32 mem_freq_khz, val;
u8 num_active_channels;
int i;
val = I915_READ(BXT_P_CR_MC_BIOS_REQ_0_0_0);
mem_freq_khz = DIV_ROUND_UP((val & BXT_REQ_DATA_MASK) *
BXT_MEMORY_FREQ_MULTIPLIER_HZ, 1000);
dram_channels = val & BXT_DRAM_CHANNEL_ACTIVE_MASK;
num_active_channels = hweight32(dram_channels);
/* Each active bit represents 4-byte channel */
dram_info->bandwidth_kbps = (mem_freq_khz * num_active_channels * 4);
if (dram_info->bandwidth_kbps == 0) {
DRM_INFO("Couldn't get system memory bandwidth\n");
return -EINVAL;
}
/*
* Now read each DUNIT8/9/10/11 to check the rank of each dimms.
*/
for (i = BXT_D_CR_DRP0_DUNIT_START; i <= BXT_D_CR_DRP0_DUNIT_END; i++) {
struct dram_dimm_info dimm;
enum intel_dram_type type;
val = I915_READ(BXT_D_CR_DRP0_DUNIT(i));
if (val == 0xFFFFFFFF)
continue;
dram_info->num_channels++;
bxt_get_dimm_info(&dimm, val);
type = bxt_get_dimm_type(val);
WARN_ON(type != INTEL_DRAM_UNKNOWN &&
dram_info->type != INTEL_DRAM_UNKNOWN &&
dram_info->type != type);
DRM_DEBUG_KMS("CH%u DIMM size: %u GB, width: X%u, ranks: %u, type: %s\n",
i - BXT_D_CR_DRP0_DUNIT_START,
dimm.size, dimm.width, dimm.ranks,
intel_dram_type_str(type));
/*
* If any of the channel is single rank channel,
* worst case output will be same as if single rank
* memory, so consider single rank memory.
*/
if (dram_info->ranks == 0)
dram_info->ranks = dimm.ranks;
else if (dimm.ranks == 1)
dram_info->ranks = 1;
if (type != INTEL_DRAM_UNKNOWN)
dram_info->type = type;
}
if (dram_info->type == INTEL_DRAM_UNKNOWN ||
dram_info->ranks == 0) {
DRM_INFO("couldn't get memory information\n");
return -EINVAL;
}
dram_info->valid = true;
return 0;
}
static void
intel_get_dram_info(struct drm_i915_private *dev_priv)
{
struct dram_info *dram_info = &dev_priv->dram_info;
int ret;
/*
* Assume 16Gb DIMMs are present until proven otherwise.
* This is only used for the level 0 watermark latency
* w/a which does not apply to bxt/glk.
*/
dram_info->is_16gb_dimm = !IS_GEN9_LP(dev_priv);
if (INTEL_GEN(dev_priv) < 9)
return;
if (IS_GEN9_LP(dev_priv))
ret = bxt_get_dram_info(dev_priv);
else
ret = skl_get_dram_info(dev_priv);
if (ret)
return;
DRM_DEBUG_KMS("DRAM bandwidth: %u kBps, channels: %u\n",
dram_info->bandwidth_kbps,
dram_info->num_channels);
DRM_DEBUG_KMS("DRAM ranks: %u, 16Gb DIMMs: %s\n",
dram_info->ranks, yesno(dram_info->is_16gb_dimm));
}
static u32 gen9_edram_size_mb(struct drm_i915_private *dev_priv, u32 cap)
{
const unsigned int ways[8] = { 4, 8, 12, 16, 16, 16, 16, 16 };
const unsigned int sets[4] = { 1, 1, 2, 2 };
return EDRAM_NUM_BANKS(cap) *
ways[EDRAM_WAYS_IDX(cap)] *
sets[EDRAM_SETS_IDX(cap)];
}
static void edram_detect(struct drm_i915_private *dev_priv)
{
u32 edram_cap = 0;
if (!(IS_HASWELL(dev_priv) ||
IS_BROADWELL(dev_priv) ||
INTEL_GEN(dev_priv) >= 9))
return;
edram_cap = __raw_uncore_read32(&dev_priv->uncore, HSW_EDRAM_CAP);
/* NB: We can't write IDICR yet because we don't have gt funcs set up */
if (!(edram_cap & EDRAM_ENABLED))
return;
/*
* The needed capability bits for size calculation are not there with
* pre gen9 so return 128MB always.
*/
if (INTEL_GEN(dev_priv) < 9)
dev_priv->edram_size_mb = 128;
else
dev_priv->edram_size_mb =
gen9_edram_size_mb(dev_priv, edram_cap);
DRM_INFO("Found %uMB of eDRAM\n", dev_priv->edram_size_mb);
}
/**
* i915_driver_hw_probe - setup state requiring device access
* @dev_priv: device private
*
* Setup state that requires accessing the device, but doesn't require
* exposing the driver via kernel internal or userspace interfaces.
*/
static int i915_driver_hw_probe(struct drm_i915_private *dev_priv)
{
struct pci_dev *pdev = dev_priv->drm.pdev;
int ret;
if (i915_inject_probe_failure())
return -ENODEV;
intel_device_info_runtime_init(dev_priv);
if (HAS_PPGTT(dev_priv)) {
if (intel_vgpu_active(dev_priv) &&
!intel_vgpu_has_full_ppgtt(dev_priv)) {
i915_report_error(dev_priv,
"incompatible vGPU found, support for isolated ppGTT required\n");
return -ENXIO;
}
}
if (HAS_EXECLISTS(dev_priv)) {
/*
* Older GVT emulation depends upon intercepting CSB mmio,
* which we no longer use, preferring to use the HWSP cache
* instead.
*/
if (intel_vgpu_active(dev_priv) &&
!intel_vgpu_has_hwsp_emulation(dev_priv)) {
i915_report_error(dev_priv,
"old vGPU host found, support for HWSP emulation required\n");
return -ENXIO;
}
}
intel_sanitize_options(dev_priv);
/* needs to be done before ggtt probe */
edram_detect(dev_priv);
i915_perf_init(dev_priv);
ret = i915_ggtt_probe_hw(dev_priv);
if (ret)
goto err_perf;
/*
* WARNING: Apparently we must kick fbdev drivers before vgacon,
* otherwise the vga fbdev driver falls over.
*/
ret = i915_kick_out_firmware_fb(dev_priv);
if (ret) {
DRM_ERROR("failed to remove conflicting framebuffer drivers\n");
goto err_ggtt;
}
ret = vga_remove_vgacon(pdev);
if (ret) {
DRM_ERROR("failed to remove conflicting VGA console\n");
goto err_ggtt;
}
ret = i915_ggtt_init_hw(dev_priv);
if (ret)
goto err_ggtt;
intel_gt_init_hw(dev_priv);
ret = i915_ggtt_enable_hw(dev_priv);
if (ret) {
DRM_ERROR("failed to enable GGTT\n");
goto err_ggtt;
}
pci_set_master(pdev);
/* overlay on gen2 is broken and can't address above 1G */
if (IS_GEN(dev_priv, 2)) {
ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(30));
if (ret) {
DRM_ERROR("failed to set DMA mask\n");
goto err_ggtt;
}
}
/* 965GM sometimes incorrectly writes to hardware status page (HWS)
* using 32bit addressing, overwriting memory if HWS is located
* above 4GB.
*
* The documentation also mentions an issue with undefined
* behaviour if any general state is accessed within a page above 4GB,
* which also needs to be handled carefully.
*/
if (IS_I965G(dev_priv) || IS_I965GM(dev_priv)) {
ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (ret) {
DRM_ERROR("failed to set DMA mask\n");
goto err_ggtt;
}
}
pm_qos_add_request(&dev_priv->pm_qos, PM_QOS_CPU_DMA_LATENCY,
PM_QOS_DEFAULT_VALUE);
/* BIOS often leaves RC6 enabled, but disable it for hw init */
intel_sanitize_gt_powersave(dev_priv);
2018-12-03 16:33:19 +03:00
intel_gt_init_workarounds(dev_priv);
/* On the 945G/GM, the chipset reports the MSI capability on the
* integrated graphics even though the support isn't actually there
* according to the published specs. It doesn't appear to function
* correctly in testing on 945G.
* This may be a side effect of MSI having been made available for PEG
* and the registers being closely associated.
*
* According to chipset errata, on the 965GM, MSI interrupts may
drm/i915: Disable MSI for all pre-gen5 We have pretty clear evidence that MSIs are getting lost on g4x and somehow the interrupt logic doesn't seem to recover from that state even if we try hard to clear the IIR. Disabling IER around the normal IIR clearing in the irq handler isn't sufficient to avoid this, so the problem really seems to be further up the interrupt chain. This should guarantee that there's always an edge if any IIR bits are set after the interrupt handler is done, which should normally guarantee that the CPU interrupt is generated. That approach seems to work perfectly on VLV/CHV, but apparently not on g4x. MSI is documented to be broken on 965gm at least. The chipset spec says MSI is defeatured because interrupts can be delayed or lost, which fits well with what we're seeing on g4x. Previously we've already disabled GMBUS interrupts on g4x because somehow GMBUS manages to raise legacy interrupts even when MSI is enabled. Since there's such widespread MSI breakahge all over in the pre-gen5 land let's just give up on MSI on these platforms. Seqno reporting might be negatively affected by this since the legcy interrupts aren't guaranteed to be ordered with the seqno writes, whereas MSI interrupts may be? But an occasioanlly missed seqno seems like a small price to pay for generally working interrupts. Cc: stable@vger.kernel.org Cc: Diego Viola <diego.viola@gmail.com> Tested-by: Diego Viola <diego.viola@gmail.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101261 Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170626203051.28480-1-ville.syrjala@linux.intel.com Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-06-26 23:30:51 +03:00
* be lost or delayed, and was defeatured. MSI interrupts seem to
* get lost on g4x as well, and interrupt delivery seems to stay
* properly dead afterwards. So we'll just disable them for all
* pre-gen5 chipsets.
*
* dp aux and gmbus irq on gen4 seems to be able to generate legacy
* interrupts even when in MSI mode. This results in spurious
* interrupt warnings if the legacy irq no. is shared with another
* device. The kernel then disables that interrupt source and so
* prevents the other device from working properly.
*/
drm/i915: Disable MSI for all pre-gen5 We have pretty clear evidence that MSIs are getting lost on g4x and somehow the interrupt logic doesn't seem to recover from that state even if we try hard to clear the IIR. Disabling IER around the normal IIR clearing in the irq handler isn't sufficient to avoid this, so the problem really seems to be further up the interrupt chain. This should guarantee that there's always an edge if any IIR bits are set after the interrupt handler is done, which should normally guarantee that the CPU interrupt is generated. That approach seems to work perfectly on VLV/CHV, but apparently not on g4x. MSI is documented to be broken on 965gm at least. The chipset spec says MSI is defeatured because interrupts can be delayed or lost, which fits well with what we're seeing on g4x. Previously we've already disabled GMBUS interrupts on g4x because somehow GMBUS manages to raise legacy interrupts even when MSI is enabled. Since there's such widespread MSI breakahge all over in the pre-gen5 land let's just give up on MSI on these platforms. Seqno reporting might be negatively affected by this since the legcy interrupts aren't guaranteed to be ordered with the seqno writes, whereas MSI interrupts may be? But an occasioanlly missed seqno seems like a small price to pay for generally working interrupts. Cc: stable@vger.kernel.org Cc: Diego Viola <diego.viola@gmail.com> Tested-by: Diego Viola <diego.viola@gmail.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101261 Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170626203051.28480-1-ville.syrjala@linux.intel.com Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2017-06-26 23:30:51 +03:00
if (INTEL_GEN(dev_priv) >= 5) {
if (pci_enable_msi(pdev) < 0)
DRM_DEBUG_DRIVER("can't enable MSI");
}
ret = intel_gvt_init(dev_priv);
if (ret)
drm/i915: Unwind HW init after GVT setup failure Following intel_gvt_init() failure, we missed unwinding our setup leaving pointers dangling past the module unload. For our example, the pm_qos: [ 441.057615] top: 000000006b3baf1c, n: 0000000054d8ef33, p: 0000000097cdf1a2 prev: 0000000054d8ef33, n: 0000000097cdf1a2, p: 000000006b3baf1c next: 0000000097cdf1a2, n: 000000006de8fc8b, p: 0000000081087253 [ 441.057627] WARNING: CPU: 4 PID: 9277 at lib/plist.c:42 plist_check_prev_next+0x2d/0x40 [ 441.057628] Modules linked in: i915(+) vgem snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic x86_pkg_temp_thermal intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_codec snd_hwdep snd_hda_core e1000e snd_pcm mei_me mei prime_numbers [last unloaded: i915] [ 441.057652] CPU: 4 PID: 9277 Comm: drv_selftest Tainted: G U 4.18.0-rc4-CI-CI_DRM_4464+ #1 [ 441.057653] Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3402 04/26/2017 [ 441.057656] RIP: 0010:plist_check_prev_next+0x2d/0x40 [ 441.057657] Code: 08 48 39 f0 74 2b 49 89 f0 48 8b 4f 08 50 ff 32 52 48 89 fe 41 ff 70 08 48 8b 17 48 c7 c7 d8 ae 14 82 4d 8b 08 e8 63 0e 76 ff <0f> 0b 48 83 c4 20 c3 48 39 10 75 d0 f3 c3 0f 1f 44 00 00 41 54 55 [ 441.057717] RSP: 0018:ffffc900003a3a68 EFLAGS: 00010082 [ 441.057720] RAX: 0000000000000000 RBX: ffff8802193978c0 RCX: 0000000000000002 [ 441.057721] RDX: 0000000080000002 RSI: ffffffff820c65a4 RDI: 00000000ffffffff [ 441.057722] RBP: ffff8802193978c0 R08: 0000000000000000 R09: 0000000000000001 [ 441.057724] R10: ffffc900003a3a70 R11: 0000000000000000 R12: ffffffff82243de0 [ 441.057725] R13: ffffffff82243de0 R14: ffff88021a6c78c0 R15: 0000000077359400 [ 441.057726] FS: 00007fc23a4a9980(0000) GS:ffff880236d00000(0000) knlGS:0000000000000000 [ 441.057728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 441.057729] CR2: 0000563e4503d038 CR3: 0000000138f86005 CR4: 00000000003606e0 [ 441.057730] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 441.057731] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 441.057732] Call Trace: [ 441.057736] plist_check_list+0x2e/0x40 [ 441.057738] plist_add+0x23/0x130 [ 441.057743] pm_qos_update_target+0x1bd/0x2f0 [ 441.057771] i915_driver_load+0xec4/0x1060 [i915] [ 441.057775] ? trace_hardirqs_on_caller+0xe0/0x1b0 [ 441.057800] i915_pci_probe+0x29/0x90 [i915] [ 441.057804] pci_device_probe+0xa1/0x130 [ 441.057807] driver_probe_device+0x306/0x480 [ 441.057810] __driver_attach+0xdb/0x100 [ 441.057812] ? driver_probe_device+0x480/0x480 [ 441.057813] ? driver_probe_device+0x480/0x480 [ 441.057816] bus_for_each_dev+0x74/0xc0 [ 441.057819] bus_add_driver+0x15f/0x250 [ 441.057821] ? 0xffffffffa0696000 [ 441.057823] driver_register+0x56/0xe0 [ 441.057825] ? 0xffffffffa0696000 [ 441.057827] do_one_initcall+0x58/0x370 [ 441.057830] ? do_init_module+0x1d/0x1ea [ 441.057832] ? rcu_read_lock_sched_held+0x6f/0x80 [ 441.057834] ? kmem_cache_alloc_trace+0x282/0x2e0 [ 441.057838] do_init_module+0x56/0x1ea [ 441.057841] load_module+0x2435/0x2b20 [ 441.057852] ? __se_sys_finit_module+0xd3/0xf0 [ 441.057854] __se_sys_finit_module+0xd3/0xf0 [ 441.057861] do_syscall_64+0x55/0x190 [ 441.057863] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 441.057865] RIP: 0033:0x7fc239d75839 [ 441.057866] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48 [ 441.057927] RSP: 002b:00007fffb7825d38 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 441.057930] RAX: ffffffffffffffda RBX: 0000563e45035dd0 RCX: 00007fc239d75839 [ 441.057931] RDX: 0000000000000000 RSI: 0000563e4502f8a0 RDI: 0000000000000004 [ 441.057932] RBP: 0000563e4502f8a0 R08: 0000000000000004 R09: 0000000000000000 [ 441.057933] R10: 00007fffb7825ea0 R11: 0000000000000246 R12: 0000000000000000 [ 441.057934] R13: 0000563e4502f690 R14: 0000000000000000 R15: 000000000000003f [ 441.057940] irq event stamp: 231338 [ 441.057943] hardirqs last enabled at (231337): [<ffffffff8193e3fc>] _raw_spin_unlock_irqrestore+0x4c/0x60 [ 441.057944] hardirqs last disabled at (231338): [<ffffffff8193e26d>] _raw_spin_lock_irqsave+0xd/0x50 [ 441.057947] softirqs last enabled at (231024): [<ffffffff81c0034f>] __do_softirq+0x34f/0x505 [ 441.057949] softirqs last disabled at (231005): [<ffffffff8108c7b9>] irq_exit+0xa9/0xc0 [ 441.057951] WARNING: CPU: 4 PID: 9277 at lib/plist.c:42 plist_check_prev_next+0x2d/0x40 v2: Add a load failure point to intel_gvt_init() so that we always exercise this path in future. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=107129 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180710143821.1889-1-chris@chris-wilson.co.uk
2018-07-10 17:38:21 +03:00
goto err_msi;
intel_opregion_setup(dev_priv);
/*
* Fill the dram structure to get the system raw bandwidth and
* dram info. This will be used for memory latency calculation.
*/
intel_get_dram_info(dev_priv);
drm/i915: Make sure we have enough memory bandwidth on ICL ICL has so many planes that it can easily exceed the maximum effective memory bandwidth of the system. We must therefore check that we don't exceed that limit. The algorithm is very magic number heavy and lacks sufficient explanation for now. We also have no sane way to query the memory clock and timings, so we must rely on a combination of raw readout from the memory controller and hardcoded assumptions. The memory controller values obviously change as the system jumps between the different SAGV points, so we try to stabilize it first by disabling SAGV for the duration of the readout. The utilized bandwidth is tracked via a device wide atomic private object. That is actually not robust because we can't afford to enforce strict global ordering between the pipes. Thus I think I'll need to change this to simply chop up the available bandwidth between all the active pipes. Each pipe can then do whatever it wants as long as it doesn't exceed its budget. That scheme will also require that we assume that any number of planes could be active at any time. TODO: make it robust and deal with all the open questions v2: Sleep longer after disabling SAGV v3: Poll for the dclk to get raised (seen it take 250ms!) If the system has 2133MT/s memory then we pointlessly wait one full second :( v4: Use the new pcode interface to get the qgv points rather that using hardcoded numbers v5: Move the pcode stuff into intel_bw.c (Matt) s/intel_sagv_info/intel_qgv_info/ Do the NV12/P010 as per spec for now (Matt) s/IS_ICELAKE/IS_GEN11/ v6: Ignore bandwidth limits if the pcode query fails Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> Acked-by: Clint Taylor <Clinton.A.Taylor@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190524153614.32410-1-ville.syrjala@linux.intel.com
2019-05-24 18:36:14 +03:00
intel_bw_init_hw(dev_priv);
return 0;
drm/i915: Unwind HW init after GVT setup failure Following intel_gvt_init() failure, we missed unwinding our setup leaving pointers dangling past the module unload. For our example, the pm_qos: [ 441.057615] top: 000000006b3baf1c, n: 0000000054d8ef33, p: 0000000097cdf1a2 prev: 0000000054d8ef33, n: 0000000097cdf1a2, p: 000000006b3baf1c next: 0000000097cdf1a2, n: 000000006de8fc8b, p: 0000000081087253 [ 441.057627] WARNING: CPU: 4 PID: 9277 at lib/plist.c:42 plist_check_prev_next+0x2d/0x40 [ 441.057628] Modules linked in: i915(+) vgem snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic x86_pkg_temp_thermal intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_codec snd_hwdep snd_hda_core e1000e snd_pcm mei_me mei prime_numbers [last unloaded: i915] [ 441.057652] CPU: 4 PID: 9277 Comm: drv_selftest Tainted: G U 4.18.0-rc4-CI-CI_DRM_4464+ #1 [ 441.057653] Hardware name: System manufacturer System Product Name/Z170 PRO GAMING, BIOS 3402 04/26/2017 [ 441.057656] RIP: 0010:plist_check_prev_next+0x2d/0x40 [ 441.057657] Code: 08 48 39 f0 74 2b 49 89 f0 48 8b 4f 08 50 ff 32 52 48 89 fe 41 ff 70 08 48 8b 17 48 c7 c7 d8 ae 14 82 4d 8b 08 e8 63 0e 76 ff <0f> 0b 48 83 c4 20 c3 48 39 10 75 d0 f3 c3 0f 1f 44 00 00 41 54 55 [ 441.057717] RSP: 0018:ffffc900003a3a68 EFLAGS: 00010082 [ 441.057720] RAX: 0000000000000000 RBX: ffff8802193978c0 RCX: 0000000000000002 [ 441.057721] RDX: 0000000080000002 RSI: ffffffff820c65a4 RDI: 00000000ffffffff [ 441.057722] RBP: ffff8802193978c0 R08: 0000000000000000 R09: 0000000000000001 [ 441.057724] R10: ffffc900003a3a70 R11: 0000000000000000 R12: ffffffff82243de0 [ 441.057725] R13: ffffffff82243de0 R14: ffff88021a6c78c0 R15: 0000000077359400 [ 441.057726] FS: 00007fc23a4a9980(0000) GS:ffff880236d00000(0000) knlGS:0000000000000000 [ 441.057728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 441.057729] CR2: 0000563e4503d038 CR3: 0000000138f86005 CR4: 00000000003606e0 [ 441.057730] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 441.057731] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 441.057732] Call Trace: [ 441.057736] plist_check_list+0x2e/0x40 [ 441.057738] plist_add+0x23/0x130 [ 441.057743] pm_qos_update_target+0x1bd/0x2f0 [ 441.057771] i915_driver_load+0xec4/0x1060 [i915] [ 441.057775] ? trace_hardirqs_on_caller+0xe0/0x1b0 [ 441.057800] i915_pci_probe+0x29/0x90 [i915] [ 441.057804] pci_device_probe+0xa1/0x130 [ 441.057807] driver_probe_device+0x306/0x480 [ 441.057810] __driver_attach+0xdb/0x100 [ 441.057812] ? driver_probe_device+0x480/0x480 [ 441.057813] ? driver_probe_device+0x480/0x480 [ 441.057816] bus_for_each_dev+0x74/0xc0 [ 441.057819] bus_add_driver+0x15f/0x250 [ 441.057821] ? 0xffffffffa0696000 [ 441.057823] driver_register+0x56/0xe0 [ 441.057825] ? 0xffffffffa0696000 [ 441.057827] do_one_initcall+0x58/0x370 [ 441.057830] ? do_init_module+0x1d/0x1ea [ 441.057832] ? rcu_read_lock_sched_held+0x6f/0x80 [ 441.057834] ? kmem_cache_alloc_trace+0x282/0x2e0 [ 441.057838] do_init_module+0x56/0x1ea [ 441.057841] load_module+0x2435/0x2b20 [ 441.057852] ? __se_sys_finit_module+0xd3/0xf0 [ 441.057854] __se_sys_finit_module+0xd3/0xf0 [ 441.057861] do_syscall_64+0x55/0x190 [ 441.057863] entry_SYSCALL_64_after_hwframe+0x49/0xbe [ 441.057865] RIP: 0033:0x7fc239d75839 [ 441.057866] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1f f6 2c 00 f7 d8 64 89 01 48 [ 441.057927] RSP: 002b:00007fffb7825d38 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 441.057930] RAX: ffffffffffffffda RBX: 0000563e45035dd0 RCX: 00007fc239d75839 [ 441.057931] RDX: 0000000000000000 RSI: 0000563e4502f8a0 RDI: 0000000000000004 [ 441.057932] RBP: 0000563e4502f8a0 R08: 0000000000000004 R09: 0000000000000000 [ 441.057933] R10: 00007fffb7825ea0 R11: 0000000000000246 R12: 0000000000000000 [ 441.057934] R13: 0000563e4502f690 R14: 0000000000000000 R15: 000000000000003f [ 441.057940] irq event stamp: 231338 [ 441.057943] hardirqs last enabled at (231337): [<ffffffff8193e3fc>] _raw_spin_unlock_irqrestore+0x4c/0x60 [ 441.057944] hardirqs last disabled at (231338): [<ffffffff8193e26d>] _raw_spin_lock_irqsave+0xd/0x50 [ 441.057947] softirqs last enabled at (231024): [<ffffffff81c0034f>] __do_softirq+0x34f/0x505 [ 441.057949] softirqs last disabled at (231005): [<ffffffff8108c7b9>] irq_exit+0xa9/0xc0 [ 441.057951] WARNING: CPU: 4 PID: 9277 at lib/plist.c:42 plist_check_prev_next+0x2d/0x40 v2: Add a load failure point to intel_gvt_init() so that we always exercise this path in future. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=107129 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180710143821.1889-1-chris@chris-wilson.co.uk
2018-07-10 17:38:21 +03:00
err_msi:
if (pdev->msi_enabled)
pci_disable_msi(pdev);
pm_qos_remove_request(&dev_priv->pm_qos);
err_ggtt:
i915_ggtt_driver_release(dev_priv);
err_perf:
i915_perf_fini(dev_priv);
return ret;
}
/**
* i915_driver_hw_remove - cleanup the setup done in i915_driver_hw_probe()
* @dev_priv: device private
*/
static void i915_driver_hw_remove(struct drm_i915_private *dev_priv)
{
struct pci_dev *pdev = dev_priv->drm.pdev;
i915_perf_fini(dev_priv);
if (pdev->msi_enabled)
pci_disable_msi(pdev);
pm_qos_remove_request(&dev_priv->pm_qos);
}
/**
* i915_driver_register - register the driver with the rest of the system
* @dev_priv: device private
*
* Perform any steps necessary to make the driver available via kernel
* internal or userspace interfaces.
*/
static void i915_driver_register(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
i915_gem_shrinker_register(dev_priv);
drm/i915/pmu: Expose a PMU interface for perf queries From: Chris Wilson <chris@chris-wilson.co.uk> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> From: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> The first goal is to be able to measure GPU (and invidual ring) busyness without having to poll registers from userspace. (Which not only incurs holding the forcewake lock indefinitely, perturbing the system, but also runs the risk of hanging the machine.) As an alternative we can use the perf event counter interface to sample the ring registers periodically and send those results to userspace. Functionality we are exporting to userspace is via the existing perf PMU API and can be exercised via the existing tools. For example: perf stat -a -e i915/rcs0-busy/ -I 1000 Will print the render engine busynnes once per second. All the performance counters can be enumerated (perf list) and have their unit of measure correctly reported in sysfs. v1-v2 (Chris Wilson): v2: Use a common timer for the ring sampling. v3: (Tvrtko Ursulin) * Decouple uAPI from i915 engine ids. * Complete uAPI defines. * Refactor some code to helpers for clarity. * Skip sampling disabled engines. * Expose counters in sysfs. * Pass in fake regs to avoid null ptr deref in perf core. * Convert to class/instance uAPI. * Use shared driver code for rc6 residency, power and frequency. v4: (Dmitry Rogozhkin) * Register PMU with .task_ctx_nr=perf_invalid_context * Expose cpumask for the PMU with the single CPU in the mask * Properly support pmu->stop(): it should call pmu->read() * Properly support pmu->del(): it should call stop(event, PERF_EF_UPDATE) * Introduce refcounting of event subscriptions. * Make pmu.busy_stats a refcounter to avoid busy stats going away with some deleted event. * Expose cpumask for i915 PMU to avoid multiple events creation of the same type followed by counter aggregation by perf-stat. * Track CPUs getting online/offline to migrate perf context. If (likely) cpumask will initially set CPU0, CONFIG_BOOTPARAM_HOTPLUG_CPU0 will be needed to see effect of CPU status tracking. * End result is that only global events are supported and perf stat works correctly. * Deny perf driver level sampling - it is prohibited for uncore PMU. v5: (Tvrtko Ursulin) * Don't hardcode number of engine samplers. * Rewrite event ref-counting for correctness and simplicity. * Store initial counter value when starting already enabled events to correctly report values to all listeners. * Fix RC6 residency readout. * Comments, GPL header. v6: * Add missing entry to v4 changelog. * Fix accounting in CPU hotplug case by copying the approach from arch/x86/events/intel/cstate.c. (Dmitry Rogozhkin) v7: * Log failure message only on failure. * Remove CPU hotplug notification state on unregister. v8: * Fix error unwind on failed registration. * Checkpatch cleanup. v9: * Drop the energy metric, it is available via intel_rapl_perf. (Ville Syrjälä) * Use HAS_RC6(p). (Chris Wilson) * Handle unsupported non-engine events. (Dmitry Rogozhkin) * Rebase for intel_rc6_residency_ns needing caller managed runtime pm. * Drop HAS_RC6 checks from the read callback since creating those events will be rejected at init time already. * Add counter units to sysfs so perf stat output is nicer. * Cleanup the attribute tables for brevity and readability. v10: * Fixed queued accounting. v11: * Move intel_engine_lookup_user to intel_engine_cs.c * Commit update. (Joonas Lahtinen) v12: * More accurate sampling. (Chris Wilson) * Store and report frequency in MHz for better usability from perf stat. * Removed metrics: queued, interrupts, rc6 counters. * Sample engine busyness based on seqno difference only for less MMIO (and forcewake) on all platforms. (Chris Wilson) v13: * Comment spelling, use mul_u32_u32 to work around potential GCC issue and somne code alignment changes. (Chris Wilson) v14: * Rebase. v15: * Rebase for RPS refactoring. v16: * Use the dynamic slot in the CPU hotplug state machine so that we are free to setup our state as multi-instance. Previously we were re-using the CPUHP_AP_PERF_X86_UNCORE_ONLINE slot which is neither used as multi-instance, nor owned by our driver to start with. * Register the CPU hotplug handlers after the PMU, otherwise the callback will get called before the PMU is initialized which can end up in perf_pmu_migrate_context with an un-initialized base. * Added workaround for a probable bug in cpuhp core. v17: * Remove workaround for the cpuhp bug. v18: * Rebase for drm_i915_gem_engine_class getting upstream before us. v19: * Rebase. (trivial) Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171121181852.16128-2-tvrtko.ursulin@linux.intel.com
2017-11-21 21:18:45 +03:00
i915_pmu_register(dev_priv);
/*
* Notify a valid surface after modesetting,
* when running inside a VM.
*/
if (intel_vgpu_active(dev_priv))
I915_WRITE(vgtif_reg(display_ready), VGT_DRV_DISPLAY_READY);
/* Reveal our presence to userspace */
if (drm_dev_register(dev, 0) == 0) {
i915_debugfs_register(dev_priv);
i915_setup_sysfs(dev_priv);
/* Depends on sysfs having been initialized */
i915_perf_register(dev_priv);
} else
DRM_ERROR("Failed to register driver for userspace access!\n");
if (HAS_DISPLAY(dev_priv)) {
/* Must be done after probing outputs */
intel_opregion_register(dev_priv);
acpi_video_register();
}
if (IS_GEN(dev_priv, 5))
intel_gpu_ips_init(dev_priv);
intel_audio_init(dev_priv);
/*
* Some ports require correctly set-up hpd registers for detection to
* work properly (leading to ghost connected connector status), e.g. VGA
* on gm45. Hence we can only set up the initial fbdev config after hpd
* irqs are fully enabled. We do it last so that the async config
* cannot run before the connectors are registered.
*/
intel_fbdev_initial_config_async(dev);
/*
* We need to coordinate the hotplugs with the asynchronous fbdev
* configuration, for which we use the fbdev->async_cookie.
*/
if (HAS_DISPLAY(dev_priv))
drm_kms_helper_poll_init(dev);
intel_power_domains_enable(dev_priv);
intel_runtime_pm_enable(&dev_priv->runtime_pm);
}
/**
* i915_driver_unregister - cleanup the registration done in i915_driver_regiser()
* @dev_priv: device private
*/
static void i915_driver_unregister(struct drm_i915_private *dev_priv)
{
intel_runtime_pm_disable(&dev_priv->runtime_pm);
intel_power_domains_disable(dev_priv);
intel_fbdev_unregister(dev_priv);
intel_audio_deinit(dev_priv);
/*
* After flushing the fbdev (incl. a late async config which will
* have delayed queuing of a hotplug event), then flush the hotplug
* events.
*/
drm_kms_helper_poll_fini(&dev_priv->drm);
intel_gpu_ips_teardown();
acpi_video_unregister();
intel_opregion_unregister(dev_priv);
i915_perf_unregister(dev_priv);
drm/i915/pmu: Expose a PMU interface for perf queries From: Chris Wilson <chris@chris-wilson.co.uk> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> From: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> The first goal is to be able to measure GPU (and invidual ring) busyness without having to poll registers from userspace. (Which not only incurs holding the forcewake lock indefinitely, perturbing the system, but also runs the risk of hanging the machine.) As an alternative we can use the perf event counter interface to sample the ring registers periodically and send those results to userspace. Functionality we are exporting to userspace is via the existing perf PMU API and can be exercised via the existing tools. For example: perf stat -a -e i915/rcs0-busy/ -I 1000 Will print the render engine busynnes once per second. All the performance counters can be enumerated (perf list) and have their unit of measure correctly reported in sysfs. v1-v2 (Chris Wilson): v2: Use a common timer for the ring sampling. v3: (Tvrtko Ursulin) * Decouple uAPI from i915 engine ids. * Complete uAPI defines. * Refactor some code to helpers for clarity. * Skip sampling disabled engines. * Expose counters in sysfs. * Pass in fake regs to avoid null ptr deref in perf core. * Convert to class/instance uAPI. * Use shared driver code for rc6 residency, power and frequency. v4: (Dmitry Rogozhkin) * Register PMU with .task_ctx_nr=perf_invalid_context * Expose cpumask for the PMU with the single CPU in the mask * Properly support pmu->stop(): it should call pmu->read() * Properly support pmu->del(): it should call stop(event, PERF_EF_UPDATE) * Introduce refcounting of event subscriptions. * Make pmu.busy_stats a refcounter to avoid busy stats going away with some deleted event. * Expose cpumask for i915 PMU to avoid multiple events creation of the same type followed by counter aggregation by perf-stat. * Track CPUs getting online/offline to migrate perf context. If (likely) cpumask will initially set CPU0, CONFIG_BOOTPARAM_HOTPLUG_CPU0 will be needed to see effect of CPU status tracking. * End result is that only global events are supported and perf stat works correctly. * Deny perf driver level sampling - it is prohibited for uncore PMU. v5: (Tvrtko Ursulin) * Don't hardcode number of engine samplers. * Rewrite event ref-counting for correctness and simplicity. * Store initial counter value when starting already enabled events to correctly report values to all listeners. * Fix RC6 residency readout. * Comments, GPL header. v6: * Add missing entry to v4 changelog. * Fix accounting in CPU hotplug case by copying the approach from arch/x86/events/intel/cstate.c. (Dmitry Rogozhkin) v7: * Log failure message only on failure. * Remove CPU hotplug notification state on unregister. v8: * Fix error unwind on failed registration. * Checkpatch cleanup. v9: * Drop the energy metric, it is available via intel_rapl_perf. (Ville Syrjälä) * Use HAS_RC6(p). (Chris Wilson) * Handle unsupported non-engine events. (Dmitry Rogozhkin) * Rebase for intel_rc6_residency_ns needing caller managed runtime pm. * Drop HAS_RC6 checks from the read callback since creating those events will be rejected at init time already. * Add counter units to sysfs so perf stat output is nicer. * Cleanup the attribute tables for brevity and readability. v10: * Fixed queued accounting. v11: * Move intel_engine_lookup_user to intel_engine_cs.c * Commit update. (Joonas Lahtinen) v12: * More accurate sampling. (Chris Wilson) * Store and report frequency in MHz for better usability from perf stat. * Removed metrics: queued, interrupts, rc6 counters. * Sample engine busyness based on seqno difference only for less MMIO (and forcewake) on all platforms. (Chris Wilson) v13: * Comment spelling, use mul_u32_u32 to work around potential GCC issue and somne code alignment changes. (Chris Wilson) v14: * Rebase. v15: * Rebase for RPS refactoring. v16: * Use the dynamic slot in the CPU hotplug state machine so that we are free to setup our state as multi-instance. Previously we were re-using the CPUHP_AP_PERF_X86_UNCORE_ONLINE slot which is neither used as multi-instance, nor owned by our driver to start with. * Register the CPU hotplug handlers after the PMU, otherwise the callback will get called before the PMU is initialized which can end up in perf_pmu_migrate_context with an un-initialized base. * Added workaround for a probable bug in cpuhp core. v17: * Remove workaround for the cpuhp bug. v18: * Rebase for drm_i915_gem_engine_class getting upstream before us. v19: * Rebase. (trivial) Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171121181852.16128-2-tvrtko.ursulin@linux.intel.com
2017-11-21 21:18:45 +03:00
i915_pmu_unregister(dev_priv);
i915_teardown_sysfs(dev_priv);
drm_dev_unplug(&dev_priv->drm);
i915_gem_shrinker_unregister(dev_priv);
}
static void i915_welcome_messages(struct drm_i915_private *dev_priv)
{
if (drm_debug & DRM_UT_DRIVER) {
struct drm_printer p = drm_debug_printer("i915 device info:");
drm/i915: Introduce concept of a sub-platform Concept of a sub-platform already exist in our code (like ULX and ULT platform variants and similar),implemented via the macros which check a list of device ids to determine a match. With this patch we consolidate device ids checking into a single function called during early driver load. A few low bits in the platform mask are reserved for sub-platform identification and defined as a per-platform namespace. At the same time it future proofs the platform_mask handling by preparing the code for easy extending, and tidies the very verbose WARN strings generated when IS_PLATFORM macros are embedded into a WARN type statements. v2: Fixed IS_SUBPLATFORM. Updated commit msg. v3: Chris was right, there is an ordering problem. v4: * Catch-up with new sub-platforms. * Rebase for RUNTIME_INFO. * Drop subplatform mask union tricks and convert platform_mask to an array for extensibility. v5: * Fix subplatform check. * Protect against forgetting to expand subplatform bits. * Remove platform enum tallying. * Add subplatform to error state. (Chris) * Drop macros and just use static inlines. * Remove redundant IRONLAKE_M. (Ville) v6: * Split out Ironlake change. * Optimize subplatform check. * Use __always_inline. (Lucas) * Add platform_mask comment. (Paulo) * Pass stored runtime info in error capture. (Chris) v7: * Rebased for new AML ULX device id. * Bump platform mask array size for EHL. * Stop mentioning device ids in intel_device_subplatform_init by using the trick of splitting macros i915_pciids.h. (Jani) * AML seems to be either a subplatform of KBL or CFL so express it like that. v8: * Use one device id table per subplatform. (Jani) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Suggested-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Jose Souza <jose.souza@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190327142328.31780-1-tvrtko.ursulin@linux.intel.com
2019-03-27 17:23:28 +03:00
drm_printf(&p, "pciid=0x%04x rev=0x%02x platform=%s (subplatform=0x%x) gen=%i\n",
INTEL_DEVID(dev_priv),
INTEL_REVID(dev_priv),
intel_platform_name(INTEL_INFO(dev_priv)->platform),
drm/i915: Introduce concept of a sub-platform Concept of a sub-platform already exist in our code (like ULX and ULT platform variants and similar),implemented via the macros which check a list of device ids to determine a match. With this patch we consolidate device ids checking into a single function called during early driver load. A few low bits in the platform mask are reserved for sub-platform identification and defined as a per-platform namespace. At the same time it future proofs the platform_mask handling by preparing the code for easy extending, and tidies the very verbose WARN strings generated when IS_PLATFORM macros are embedded into a WARN type statements. v2: Fixed IS_SUBPLATFORM. Updated commit msg. v3: Chris was right, there is an ordering problem. v4: * Catch-up with new sub-platforms. * Rebase for RUNTIME_INFO. * Drop subplatform mask union tricks and convert platform_mask to an array for extensibility. v5: * Fix subplatform check. * Protect against forgetting to expand subplatform bits. * Remove platform enum tallying. * Add subplatform to error state. (Chris) * Drop macros and just use static inlines. * Remove redundant IRONLAKE_M. (Ville) v6: * Split out Ironlake change. * Optimize subplatform check. * Use __always_inline. (Lucas) * Add platform_mask comment. (Paulo) * Pass stored runtime info in error capture. (Chris) v7: * Rebased for new AML ULX device id. * Bump platform mask array size for EHL. * Stop mentioning device ids in intel_device_subplatform_init by using the trick of splitting macros i915_pciids.h. (Jani) * AML seems to be either a subplatform of KBL or CFL so express it like that. v8: * Use one device id table per subplatform. (Jani) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Suggested-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Jose Souza <jose.souza@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190327142328.31780-1-tvrtko.ursulin@linux.intel.com
2019-03-27 17:23:28 +03:00
intel_subplatform(RUNTIME_INFO(dev_priv),
INTEL_INFO(dev_priv)->platform),
INTEL_GEN(dev_priv));
intel_device_info_dump_flags(INTEL_INFO(dev_priv), &p);
intel_device_info_dump_runtime(RUNTIME_INFO(dev_priv), &p);
}
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG))
DRM_INFO("DRM_I915_DEBUG enabled\n");
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
DRM_INFO("DRM_I915_DEBUG_GEM enabled\n");
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM))
DRM_INFO("DRM_I915_DEBUG_RUNTIME_PM enabled\n");
}
static struct drm_i915_private *
i915_driver_create(struct pci_dev *pdev, const struct pci_device_id *ent)
{
const struct intel_device_info *match_info =
(struct intel_device_info *)ent->driver_data;
struct intel_device_info *device_info;
struct drm_i915_private *i915;
int err;
i915 = kzalloc(sizeof(*i915), GFP_KERNEL);
if (!i915)
return ERR_PTR(-ENOMEM);
err = drm_dev_init(&i915->drm, &driver, &pdev->dev);
if (err) {
kfree(i915);
return ERR_PTR(err);
}
i915->drm.pdev = pdev;
i915->drm.dev_private = i915;
pci_set_drvdata(pdev, &i915->drm);
/* Setup the write-once "constant" device info */
device_info = mkwrite_device_info(i915);
memcpy(device_info, match_info, sizeof(*device_info));
RUNTIME_INFO(i915)->device_id = pdev->device;
BUG_ON(device_info->gen > BITS_PER_TYPE(device_info->gen_mask));
return i915;
}
static void i915_driver_destroy(struct drm_i915_private *i915)
{
struct pci_dev *pdev = i915->drm.pdev;
drm_dev_fini(&i915->drm);
kfree(i915);
/* And make sure we never chase our dangling pointer from pci_dev */
pci_set_drvdata(pdev, NULL);
}
/**
* i915_driver_probe - setup chip and create an initial config
* @pdev: PCI device
* @ent: matching PCI ID entry
*
* The driver probe routine has to do several things:
* - drive output discovery via intel_modeset_init()
* - initialize the memory manager
* - allocate initial config memory
* - setup the DRM framebuffer with the allocated memory
*/
int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
const struct intel_device_info *match_info =
(struct intel_device_info *)ent->driver_data;
struct drm_i915_private *dev_priv;
int ret;
dev_priv = i915_driver_create(pdev, ent);
if (IS_ERR(dev_priv))
return PTR_ERR(dev_priv);
/* Disable nuclear pageflip by default on pre-ILK */
if (!i915_modparams.nuclear_pageflip && match_info->gen < 5)
dev_priv->drm.driver_features &= ~DRIVER_ATOMIC;
ret = pci_enable_device(pdev);
if (ret)
goto out_fini;
ret = i915_driver_early_probe(dev_priv);
if (ret < 0)
goto out_pci_disable;
disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
i915_detect_vgpu(dev_priv);
ret = i915_driver_mmio_probe(dev_priv);
if (ret < 0)
goto out_runtime_pm_put;
ret = i915_driver_hw_probe(dev_priv);
if (ret < 0)
goto out_cleanup_mmio;
ret = i915_driver_modeset_probe(&dev_priv->drm);
if (ret < 0)
goto out_cleanup_hw;
i915_driver_register(dev_priv);
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
i915_welcome_messages(dev_priv);
return 0;
out_cleanup_hw:
i915_driver_hw_remove(dev_priv);
i915_ggtt_driver_release(dev_priv);
/* Paranoia: make sure we have disabled everything before we exit. */
intel_sanitize_gt_powersave(dev_priv);
out_cleanup_mmio:
i915_driver_mmio_release(dev_priv);
out_runtime_pm_put:
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
i915_driver_late_release(dev_priv);
out_pci_disable:
pci_disable_device(pdev);
out_fini:
i915_probe_error(dev_priv, "Device initialization failed (%d)\n", ret);
i915_driver_destroy(dev_priv);
return ret;
}
void i915_driver_remove(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
i915_driver_unregister(dev_priv);
/*
* After unregistering the device to prevent any new users, cancel
* all in-flight requests so that we can quickly unbind the active
* resources.
*/
intel_gt_set_wedged(&dev_priv->gt);
/* Flush any external code that still may be under the RCU lock */
synchronize_rcu();
i915_gem_suspend(dev_priv);
drm_atomic_helper_shutdown(dev);
intel_gvt_driver_remove(dev_priv);
intel_modeset_driver_remove(dev);
intel_bios_driver_remove(dev_priv);
vga_switcheroo_unregister_client(pdev);
vga_client_register(pdev, NULL, NULL, NULL);
drm/i915: fix pch pci device enumeration pci_get_class(class, from) drops the refcount for 'from', so the extra pci_dev_put we do on it will result in a use after free bug starting with the WARN below. Regression introduced in commit 6a9c4b35e6696a63805b6da5e4889c6986e9ee1b Author: Rui Guo <firemeteor@users.sourceforge.net> Date: Wed Jun 19 21:10:23 2013 +0800 drm/i915: Fix PCH detect with multiple ISA bridges in VM [ 164.338460] WARNING: CPU: 1 PID: 2094 at include/linux/kref.h:47 klist_next+0xae/0x110() [ 164.347731] CPU: 1 PID: 2094 Comm: modprobe Tainted: G O 3.13.0-imre+ #354 [ 164.356468] Hardware name: Intel Corp. VALLEYVIEW B0 PLATFORM/NOTEBOOK, BIOS BYTICRB1.X64.0062.R70.1310112051 10/11/2013 [ 164.368796] Call Trace: [ 164.371609] [<ffffffff816a32a6>] dump_stack+0x4e/0x7a [ 164.377447] [<ffffffff8104f75d>] warn_slowpath_common+0x7d/0xa0 [ 164.384238] [<ffffffff8104f83a>] warn_slowpath_null+0x1a/0x20 [ 164.390851] [<ffffffff8169aeae>] klist_next+0xae/0x110 [ 164.396777] [<ffffffff8130a110>] ? pci_do_find_bus+0x70/0x70 [ 164.403286] [<ffffffff813cb4a9>] bus_find_device+0x89/0xc0 [ 164.409719] [<ffffffff8130a373>] pci_get_dev_by_id+0x63/0xa0 [ 164.416238] [<ffffffff8130a4e4>] pci_get_class+0x44/0x50 [ 164.422433] [<ffffffffa034821f>] intel_dsm_detect+0x16f/0x1f0 [i915] [ 164.429801] [<ffffffffa03482ae>] intel_register_dsm_handler+0xe/0x10 [i915] [ 164.437831] [<ffffffffa02d30fe>] i915_driver_load+0xafe/0xf30 [i915] [ 164.445126] [<ffffffff8158a150>] ? intel_alloc_coherent+0x110/0x110 [ 164.452340] [<ffffffffa0148c07>] drm_dev_register+0xc7/0x150 [drm] [ 164.459462] [<ffffffffa014b23f>] drm_get_pci_dev+0x11f/0x1f0 [drm] [ 164.466554] [<ffffffff816abb81>] ? _raw_spin_unlock_irqrestore+0x51/0x70 [ 164.474287] [<ffffffffa02cf7a6>] i915_pci_probe+0x56/0x60 [i915] [ 164.481185] [<ffffffff8130a028>] pci_device_probe+0x78/0xf0 [ 164.487603] [<ffffffff813cd495>] driver_probe_device+0x155/0x350 [ 164.494505] [<ffffffff813cd74e>] __driver_attach+0x6e/0xa0 [ 164.500826] [<ffffffff813cd6e0>] ? __device_attach+0x50/0x50 [ 164.507333] [<ffffffff813cb2be>] bus_for_each_dev+0x6e/0xc0 [ 164.513752] [<ffffffff813ccefe>] driver_attach+0x1e/0x20 [ 164.519870] [<ffffffff813cc958>] bus_add_driver+0x138/0x260 [ 164.526289] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.532116] [<ffffffff813cde78>] driver_register+0x98/0xe0 [ 164.538558] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.544389] [<ffffffff813087b0>] __pci_register_driver+0x60/0x70 [ 164.551336] [<ffffffffa014b37d>] drm_pci_init+0x6d/0x120 [drm] [ 164.558040] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.563928] [<ffffffffa018806a>] i915_init+0x6a/0x6c [i915] [ 164.570363] [<ffffffff810002da>] do_one_initcall+0xaa/0x160 [ 164.576783] [<ffffffff8103b140>] ? set_memory_nx+0x40/0x50 [ 164.583100] [<ffffffff810ce7f5>] load_module+0x1fb5/0x2550 [ 164.589410] [<ffffffff810caab0>] ? store_uevent+0x40/0x40 [ 164.595628] [<ffffffff810cee7d>] SyS_init_module+0xed/0x100 [ 164.602048] [<ffffffff816b3c52>] system_call_fastpath+0x16/0x1b v2: simplify the loop further (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Cc: Jesse Barnes <jbarnes@virtuousgeek.org> Reported-by: Jesse Barnes <jbarnes@virtuousgeek.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=65652 Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=74161 Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: stable@vger.kernel.org Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2014-02-14 22:23:54 +04:00
intel_csr_ucode_fini(dev_priv);
drm/i915: fix pch pci device enumeration pci_get_class(class, from) drops the refcount for 'from', so the extra pci_dev_put we do on it will result in a use after free bug starting with the WARN below. Regression introduced in commit 6a9c4b35e6696a63805b6da5e4889c6986e9ee1b Author: Rui Guo <firemeteor@users.sourceforge.net> Date: Wed Jun 19 21:10:23 2013 +0800 drm/i915: Fix PCH detect with multiple ISA bridges in VM [ 164.338460] WARNING: CPU: 1 PID: 2094 at include/linux/kref.h:47 klist_next+0xae/0x110() [ 164.347731] CPU: 1 PID: 2094 Comm: modprobe Tainted: G O 3.13.0-imre+ #354 [ 164.356468] Hardware name: Intel Corp. VALLEYVIEW B0 PLATFORM/NOTEBOOK, BIOS BYTICRB1.X64.0062.R70.1310112051 10/11/2013 [ 164.368796] Call Trace: [ 164.371609] [<ffffffff816a32a6>] dump_stack+0x4e/0x7a [ 164.377447] [<ffffffff8104f75d>] warn_slowpath_common+0x7d/0xa0 [ 164.384238] [<ffffffff8104f83a>] warn_slowpath_null+0x1a/0x20 [ 164.390851] [<ffffffff8169aeae>] klist_next+0xae/0x110 [ 164.396777] [<ffffffff8130a110>] ? pci_do_find_bus+0x70/0x70 [ 164.403286] [<ffffffff813cb4a9>] bus_find_device+0x89/0xc0 [ 164.409719] [<ffffffff8130a373>] pci_get_dev_by_id+0x63/0xa0 [ 164.416238] [<ffffffff8130a4e4>] pci_get_class+0x44/0x50 [ 164.422433] [<ffffffffa034821f>] intel_dsm_detect+0x16f/0x1f0 [i915] [ 164.429801] [<ffffffffa03482ae>] intel_register_dsm_handler+0xe/0x10 [i915] [ 164.437831] [<ffffffffa02d30fe>] i915_driver_load+0xafe/0xf30 [i915] [ 164.445126] [<ffffffff8158a150>] ? intel_alloc_coherent+0x110/0x110 [ 164.452340] [<ffffffffa0148c07>] drm_dev_register+0xc7/0x150 [drm] [ 164.459462] [<ffffffffa014b23f>] drm_get_pci_dev+0x11f/0x1f0 [drm] [ 164.466554] [<ffffffff816abb81>] ? _raw_spin_unlock_irqrestore+0x51/0x70 [ 164.474287] [<ffffffffa02cf7a6>] i915_pci_probe+0x56/0x60 [i915] [ 164.481185] [<ffffffff8130a028>] pci_device_probe+0x78/0xf0 [ 164.487603] [<ffffffff813cd495>] driver_probe_device+0x155/0x350 [ 164.494505] [<ffffffff813cd74e>] __driver_attach+0x6e/0xa0 [ 164.500826] [<ffffffff813cd6e0>] ? __device_attach+0x50/0x50 [ 164.507333] [<ffffffff813cb2be>] bus_for_each_dev+0x6e/0xc0 [ 164.513752] [<ffffffff813ccefe>] driver_attach+0x1e/0x20 [ 164.519870] [<ffffffff813cc958>] bus_add_driver+0x138/0x260 [ 164.526289] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.532116] [<ffffffff813cde78>] driver_register+0x98/0xe0 [ 164.538558] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.544389] [<ffffffff813087b0>] __pci_register_driver+0x60/0x70 [ 164.551336] [<ffffffffa014b37d>] drm_pci_init+0x6d/0x120 [drm] [ 164.558040] [<ffffffffa0188000>] ? 0xffffffffa0187fff [ 164.563928] [<ffffffffa018806a>] i915_init+0x6a/0x6c [i915] [ 164.570363] [<ffffffff810002da>] do_one_initcall+0xaa/0x160 [ 164.576783] [<ffffffff8103b140>] ? set_memory_nx+0x40/0x50 [ 164.583100] [<ffffffff810ce7f5>] load_module+0x1fb5/0x2550 [ 164.589410] [<ffffffff810caab0>] ? store_uevent+0x40/0x40 [ 164.595628] [<ffffffff810cee7d>] SyS_init_module+0xed/0x100 [ 164.602048] [<ffffffff816b3c52>] system_call_fastpath+0x16/0x1b v2: simplify the loop further (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Cc: Jesse Barnes <jbarnes@virtuousgeek.org> Reported-by: Jesse Barnes <jbarnes@virtuousgeek.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=65652 Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=74161 Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: stable@vger.kernel.org Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2014-02-14 22:23:54 +04:00
/* Free error state after interrupts are fully disabled. */
cancel_delayed_work_sync(&dev_priv->gt.hangcheck.work);
i915_reset_error_state(dev_priv);
i915_gem_driver_remove(dev_priv);
intel_power_domains_driver_remove(dev_priv);
i915_driver_hw_remove(dev_priv);
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
}
static void i915_driver_release(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
disable_rpm_wakeref_asserts(rpm);
i915_gem_driver_release(dev_priv);
i915_ggtt_driver_release(dev_priv);
/* Paranoia: make sure we have disabled everything before we exit. */
intel_sanitize_gt_powersave(dev_priv);
i915_driver_mmio_release(dev_priv);
enable_rpm_wakeref_asserts(rpm);
intel_runtime_pm_driver_release(rpm);
i915_driver_late_release(dev_priv);
i915_driver_destroy(dev_priv);
}
static int i915_driver_open(struct drm_device *dev, struct drm_file *file)
{
struct drm_i915_private *i915 = to_i915(dev);
int ret;
ret = i915_gem_open(i915, file);
if (ret)
return ret;
return 0;
}
/**
* i915_driver_lastclose - clean up after all DRM clients have exited
* @dev: DRM device
*
* Take care of cleaning up after all DRM clients have exited. In the
* mode setting case, we want to restore the kernel's initial mode (just
* in case the last client left us in a bad state).
*
* Additionally, in the non-mode setting case, we'll tear down the GTT
* and DMA structures, since the kernel won't be using them, and clea
* up any GEM state.
*/
static void i915_driver_lastclose(struct drm_device *dev)
{
intel_fbdev_restore_mode(dev);
vga_switcheroo_process_delayed_switch();
}
static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file)
{
struct drm_i915_file_private *file_priv = file->driver_priv;
mutex_lock(&dev->struct_mutex);
i915_gem_context_close(file);
i915_gem_release(dev, file);
mutex_unlock(&dev->struct_mutex);
kfree(file_priv);
}
static void intel_suspend_encoders(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
struct intel_encoder *encoder;
drm_modeset_lock_all(dev);
for_each_intel_encoder(dev, encoder)
if (encoder->suspend)
encoder->suspend(encoder);
drm_modeset_unlock_all(dev);
}
static int vlv_resume_prepare(struct drm_i915_private *dev_priv,
bool rpm_resume);
static int vlv_suspend_complete(struct drm_i915_private *dev_priv);
drm/i915/skl: Add DC6 Trigger sequence. Add triggers for DC6 as per details provided in skl_enable_dc6 and skl_disable_dc6 implementations. Also Call POSTING_READ for every write to a register to ensure it is written to immediately v1: Remove POSTING_READ and intel_prepare_ddi calls as they've been added in previous patches. v2: 1] Remove check for backlight disabled as it should be the case by that time. 2] Mark DC5 as disabled when enabling DC6. 3] Return from DC5-disabling function early if DC5 is already be disabled which can happen due to DC6-enabling earlier. 3] Ensure CSR firmware is loaded after resume from DC6 as corresponding memory contents won't be retained after runtime-suspend. 4] Ensure that CSR isn't identified as loaded before CSR-loading program is called during runtime-resume. v3: Rebase to latest Modified as per review comments from Imre and after discussion with Art: 1] DC6 should be preferably enabled when PG2 is disabled by SW as the check for PG1 being disabled is taken of by HW to enter DC6, and disabled when PG2 is enabled respectively. This helps save more power, especially in the case when display is disabled but GT is enabled. Accordingly, replacing DC5 trigger sequence with DC6 for SKL. 2] DC6 could be enabled from intel_runtime_suspend() function, if DC5 is already enabled. 3] Move CSR-load-status setting code from intel_runtime_suspend function to a new function. v4: 1] Enable/disable DC6 only when toggling the power-well using a newly defined macro ENABLE_DC6. v5: 1] Load CSR on system resume too as firmware may be lost on system suspend preventing enabling DC5, DC6. 2] DDI buffers shouldn't be programmed during driver-load/resume as it's already done during modeset initialization then and also that the encoder list is still uninitialized by then. Therefore, call intel_prepare_ddi function right after disabling DC6 but outside skl_disable_dc6 function and not during driver-load/resume. v6: 1] Rebase to latest. 2] Move SKL_ENABLE_DC6 macro definition from intel_display.c to intel_runtime_pm.c. v7: 1) Refactored the code for removing the warning got from checkpatch. 2) After adding dmc ver 1.0 support rebased on top of nightly. (Animesh) v8: - Reverted the changes done in v7. - Removed the condition check in skl_prepare_resune(). (Animesh) Issue: VIZ-2819 Signed-off-by: A.Sunil Kamath <sunil.kamath@intel.com> Signed-off-by: Suketu Shah <suketu.j.shah@intel.com> Signed-off-by: Damien Lespiau <damien.lespiau@intel.com> Signed-off-by: Animesh Manna <animesh.manna@intel.com> Reviewed-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2015-04-16 11:52:11 +03:00
static bool suspend_to_idle(struct drm_i915_private *dev_priv)
{
#if IS_ENABLED(CONFIG_ACPI_SLEEP)
if (acpi_target_system_state() < ACPI_STATE_S3)
return true;
#endif
return false;
}
static int i915_drm_prepare(struct drm_device *dev)
{
struct drm_i915_private *i915 = to_i915(dev);
/*
* NB intel_display_suspend() may issue new requests after we've
* ostensibly marked the GPU as ready-to-sleep here. We need to
* split out that work and pull it forward so that after point,
* the GPU is not woken again.
*/
i915_gem_suspend(i915);
return 0;
}
static int i915_drm_suspend(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
pci_power_t opregion_target_state;
disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
drm/i915: allow package C8+ states on Haswell (disabled) This patch allows PC8+ states on Haswell. These states can only be reached when all the display outputs are disabled, and they allow some more power savings. The fact that the graphics device is allowing PC8+ doesn't mean that the machine will actually enter PC8+: all the other devices also need to allow PC8+. For now this option is disabled by default. You need i915.allow_pc8=1 if you want it. This patch adds a big comment inside i915_drv.h explaining how it works and how it tracks things. Read it. v2: (this is not really v2, many previous versions were already sent, but they had different names) - Use the new functions to enable/disable GTIMR and GEN6_PMIMR - Rename almost all variables and functions to names suggested by Chris - More WARNs on the IRQ handling code - Also disable PC8 when there's GPU work to do (thanks to Ben for the help on this), so apps can run caster - Enable PC8 on a delayed work function that is delayed for 5 seconds. This makes sure we only enable PC8+ if we're really idle - Make sure we're not in PC8+ when suspending v3: - WARN if IRQs are disabled on __wait_seqno - Replace some DRM_ERRORs with WARNs - Fix calls to restore GT and PM interrupts - Use intel_mark_busy instead of intel_ring_advance to disable PC8 v4: - Use the force_wake, Luke! v5: - Remove the "IIR is not zero" WARNs - Move the force_wake chunk to its own patch - Only restore what's missing from RC6, not everything Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2013-08-19 20:18:09 +04:00
/* We do a lot of poking in a lot of registers, make sure they work
* properly. */
intel_power_domains_disable(dev_priv);
drm_kms_helper_poll_disable(dev);
pci_save_state(pdev);
intel_display_suspend(dev);
intel_dp_mst_suspend(dev_priv);
intel_runtime_pm_disable_interrupts(dev_priv);
intel_hpd_cancel_work(dev_priv);
intel_suspend_encoders(dev_priv);
2014-05-02 08:02:48 +04:00
intel_suspend_hw(dev_priv);
i915_gem_suspend_gtt_mappings(dev_priv);
drm/i915: Disable GGTT PTEs on GEN6+ suspend Once the machine gets to a certain point in the suspend process, we expect the GPU to be idle. If it is not, we might corrupt memory. Empirically (with an early version of this patch) we have seen this is not the case. We cannot currently explain why the latent GPU writes occur. In the technical sense, this patch is a workaround in that we have an issue we can't explain, and the patch indirectly solves the issue. However, it's really better than a workaround because we understand why it works, and it really should be a safe thing to do in all cases. The noticeable effect other than the debug messages would be an increase in the suspend time. I have not measure how expensive it actually is. I think it would be good to spend further time to root cause why we're seeing these latent writes, but it shouldn't preclude preventing the fallout. NOTE: It should be safe (and makes some sense IMO) to also keep the VALID bit unset on resume when we clear_range(). I've opted not to do this as properly clearing those bits at some later point would be extra work. v2: Fix bugzilla link Bugzilla: http://bugs.freedesktop.org/show_bug.cgi?id=65496 Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=59321 Tested-by: Takashi Iwai <tiwai@suse.de> Tested-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Tested-By: Todd Previte <tprevite@gmail.com> Cc: stable@vger.kernel.org Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2013-10-16 20:21:30 +04:00
i915_save_state(dev_priv);
opregion_target_state = suspend_to_idle(dev_priv) ? PCI_D1 : PCI_D3cold;
intel_opregion_suspend(dev_priv, opregion_target_state);
intel_fbdev_set_suspend(dev, FBINFO_STATE_SUSPENDED, true);
dev_priv->suspend_count++;
intel_csr_ucode_suspend(dev_priv);
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
return 0;
}
static enum i915_drm_suspend_mode
get_suspend_mode(struct drm_i915_private *dev_priv, bool hibernate)
{
if (hibernate)
return I915_DRM_SUSPEND_HIBERNATE;
if (suspend_to_idle(dev_priv))
return I915_DRM_SUSPEND_IDLE;
return I915_DRM_SUSPEND_MEM;
}
static int i915_drm_suspend_late(struct drm_device *dev, bool hibernation)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
int ret;
disable_rpm_wakeref_asserts(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
i915_gem_suspend_late(dev_priv);
intel_uncore_suspend(&dev_priv->uncore);
intel_power_domains_suspend(dev_priv,
get_suspend_mode(dev_priv, hibernation));
drm/i915/skl: init/uninit display core as part of the HW power domain state We need to initialize the display core part early, before initializing the rest of the display power state. This is also described in the bspec termed "Display initialization sequence". Atm we run this sequence during driver loading after power domain HW state initialization which is too late and during runtime suspend/resume which is unneeded and can interere with DMC functionality which handles HW resources toggled by this init/uninit sequence automatically. The init sequence must be run as the first step of HW power state initialization and during system resume. The uninit sequence must be run during system suspend. To address the above move the init sequence to the initial HW power state setup and the uninit sequence to a new power domains suspend function called during system suspend. As part of the init sequence we also have to reprogram the DMC firmware as it's lost across a system suspend/resume cycle. After this change CD clock initialization during driver loading will happen only later after other dependent HW/SW parts are initialized, while during system resume it will get initialized as the last step of the init sequence. This distinction can be removed by some refactoring of platform independent parts. I left this refactoring out from this series since I didn't want to change non-SKL parts. This is a TODO for later. v2: - fix error path in i915_drm_suspend_late() - don't try to re-program the DMC firmware if it failed to load Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Patrik Jakobsson <patrik.jakobsson@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1447774433-20834-1-git-send-email-imre.deak@intel.com
2015-11-17 18:33:53 +03:00
ret = 0;
if (INTEL_GEN(dev_priv) >= 11 || IS_GEN9_LP(dev_priv))
bxt_enable_dc9(dev_priv);
else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
hsw_enable_pc8(dev_priv);
else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
ret = vlv_suspend_complete(dev_priv);
if (ret) {
DRM_ERROR("Suspend complete failed: %d\n", ret);
intel_power_domains_resume(dev_priv);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
goto out;
}
pci_disable_device(pdev);
2015-03-02 14:04:41 +03:00
/*
drm/i915: apply the PCI_D0/D3 hibernation workaround everywhere on pre GEN6 commit da2bc1b9db3351addd293e5b82757efe1f77ed1d Author: Imre Deak <imre.deak@intel.com> Date: Thu Oct 23 19:23:26 2014 +0300 drm/i915: add poweroff_late handler introduced a regression on old platforms during hibernation. A workaround was added in commit ab3be73fa7b43f4c3648ce29b5fd649ea54d3adb Author: Imre Deak <imre.deak@intel.com> Date: Mon Mar 2 13:04:41 2015 +0200 drm/i915: gen4: work around hang during hibernation using an explicit blacklist for the GENs/BIOS vendors where the issue was reported. Later there we had reports of the same failure on platforms not on this list. To my best knowledge the correct thing to do is still to put the device to PCI D3 state during hibernation, see [1] and [2] for the reasons. This also aligns with our future plans to unify more the runtime and system suspend/resume paths. Since an exact blacklist seems to be impractical (multiple GENs and BIOS vendors are affected) apply the workaround on everything pre GEN6. [1] http://lists.freedesktop.org/archives/intel-gfx/2015-February/060710.html [2] https://lkml.org/lkml/2015/6/22/274 Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=95061 Reported-by: Ilya Tumaykin <itumaykin@gmail.com> Reported-by: Dirk Griesbach <spamthis@freenet.de> Reported-by: Pavel Machek <pavel@ucw.cz> Reported-by: Mikko Rapeli <mikko.rapeli@iki.fi> Tested-by: Mikko Rapeli <mikko.rapeli@iki.fi> Reported-by: Paul Bolle <pebolle@tiscali.nl> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2015-06-30 17:06:47 +03:00
* During hibernation on some platforms the BIOS may try to access
2015-03-02 14:04:41 +03:00
* the device even though it's already in D3 and hang the machine. So
* leave the device in D0 on those platforms and hope the BIOS will
drm/i915: apply the PCI_D0/D3 hibernation workaround everywhere on pre GEN6 commit da2bc1b9db3351addd293e5b82757efe1f77ed1d Author: Imre Deak <imre.deak@intel.com> Date: Thu Oct 23 19:23:26 2014 +0300 drm/i915: add poweroff_late handler introduced a regression on old platforms during hibernation. A workaround was added in commit ab3be73fa7b43f4c3648ce29b5fd649ea54d3adb Author: Imre Deak <imre.deak@intel.com> Date: Mon Mar 2 13:04:41 2015 +0200 drm/i915: gen4: work around hang during hibernation using an explicit blacklist for the GENs/BIOS vendors where the issue was reported. Later there we had reports of the same failure on platforms not on this list. To my best knowledge the correct thing to do is still to put the device to PCI D3 state during hibernation, see [1] and [2] for the reasons. This also aligns with our future plans to unify more the runtime and system suspend/resume paths. Since an exact blacklist seems to be impractical (multiple GENs and BIOS vendors are affected) apply the workaround on everything pre GEN6. [1] http://lists.freedesktop.org/archives/intel-gfx/2015-February/060710.html [2] https://lkml.org/lkml/2015/6/22/274 Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=95061 Reported-by: Ilya Tumaykin <itumaykin@gmail.com> Reported-by: Dirk Griesbach <spamthis@freenet.de> Reported-by: Pavel Machek <pavel@ucw.cz> Reported-by: Mikko Rapeli <mikko.rapeli@iki.fi> Tested-by: Mikko Rapeli <mikko.rapeli@iki.fi> Reported-by: Paul Bolle <pebolle@tiscali.nl> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2015-06-30 17:06:47 +03:00
* power down the device properly. The issue was seen on multiple old
* GENs with different BIOS vendors, so having an explicit blacklist
* is inpractical; apply the workaround on everything pre GEN6. The
* platforms where the issue was seen:
* Lenovo Thinkpad X301, X61s, X60, T60, X41
* Fujitsu FSC S7110
* Acer Aspire 1830T
2015-03-02 14:04:41 +03:00
*/
if (!(hibernation && INTEL_GEN(dev_priv) < 6))
pci_set_power_state(pdev, PCI_D3hot);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
out:
enable_rpm_wakeref_asserts(rpm);
if (!dev_priv->uncore.user_forcewake.count)
intel_runtime_pm_driver_release(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
return ret;
}
static int i915_suspend_switcheroo(struct drm_device *dev, pm_message_t state)
{
int error;
if (!dev) {
DRM_ERROR("dev: %p\n", dev);
DRM_ERROR("DRM not initialized, aborting suspend.\n");
return -ENODEV;
}
if (WARN_ON_ONCE(state.event != PM_EVENT_SUSPEND &&
state.event != PM_EVENT_FREEZE))
return -EINVAL;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
error = i915_drm_suspend(dev);
if (error)
return error;
2015-03-02 14:04:41 +03:00
return i915_drm_suspend_late(dev, false);
}
static int i915_drm_resume(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
2016-05-06 21:35:55 +03:00
int ret;
disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
intel_sanitize_gt_powersave(dev_priv);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
i915_gem_sanitize(dev_priv);
ret = i915_ggtt_enable_hw(dev_priv);
2016-05-06 21:35:55 +03:00
if (ret)
DRM_ERROR("failed to re-enable GGTT\n");
intel_csr_ucode_resume(dev_priv);
i915_restore_state(dev_priv);
intel_pps_unlock_regs_wa(dev_priv);
intel_init_pch_refclk(dev_priv);
/*
* Interrupts have to be enabled before any batches are run. If not the
* GPU will hang. i915_gem_init_hw() will initiate batches to
* update/restore the context.
*
* drm_mode_config_reset() needs AUX interrupts.
*
* Modeset enabling in intel_modeset_init_hw() also needs working
* interrupts.
*/
intel_runtime_pm_enable_interrupts(dev_priv);
drm_mode_config_reset(dev);
i915_gem_resume(dev_priv);
intel_modeset_init_hw(dev);
intel_init_clock_gating(dev_priv);
spin_lock_irq(&dev_priv->irq_lock);
if (dev_priv->display.hpd_irq_setup)
drm/i915: Small display interrupt handlers tidy I have noticed some of our interrupt handlers use both dev and dev_priv while they could get away with only dev_priv in the huge majority of cases. Tidying that up had a cascading effect on changing functions prototypes, so relatively big churn factor, but I think it is for the better. For example even where changes cascade out of i915_irq.c, for functions prefixed with intel_, genX_ or <plat>_, it makes more sense to take dev_priv directly anyway. This allows us to eliminate local variables and intermixed usage of dev and dev_priv where only one is good enough. End result is shrinkage of both source and the resulting binary. i915.ko: - .text 000b0899 + .text 000b0619 Or if we look at the Gen8 display irq chain: -00000000000006ad t gen8_irq_handler +0000000000000663 t gen8_irq_handler -0000000000000028 T intel_opregion_asle_intr +0000000000000024 T intel_opregion_asle_intr -000000000000008c t ilk_hpd_irq_handler +000000000000007f t ilk_hpd_irq_handler -0000000000000116 T intel_check_page_flip +0000000000000112 T intel_check_page_flip -000000000000011a T intel_prepare_page_flip +0000000000000119 T intel_prepare_page_flip -0000000000000014 T intel_finish_page_flip_plane +0000000000000013 T intel_finish_page_flip_plane -0000000000000053 t hsw_pipe_crc_irq_handler +000000000000004c t hsw_pipe_crc_irq_handler -000000000000022e t cpt_irq_handler +0000000000000213 t cpt_irq_handler So small shrinkage but it is all fast paths so doesn't harm. Situation is similar in other interrupt handlers as well. v2: Tidy intel_queue_rps_boost_for_request as well. (Chris Wilson) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
2016-05-06 16:48:28 +03:00
dev_priv->display.hpd_irq_setup(dev_priv);
spin_unlock_irq(&dev_priv->irq_lock);
2014-05-02 08:02:48 +04:00
intel_dp_mst_resume(dev_priv);
intel_display_resume(dev);
drm_kms_helper_poll_enable(dev);
/*
* ... but also need to make sure that hotplug processing
* doesn't cause havoc. Like in the driver load code we don't
* bother with the tiny race here where we might lose hotplug
* notifications.
* */
intel_hpd_init(dev_priv);
intel_opregion_resume(dev_priv);
intel_fbdev_set_suspend(dev, FBINFO_STATE_RUNNING, false);
intel_power_domains_enable(dev_priv);
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
return 0;
}
static int i915_drm_resume_early(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
drm/i915: Fix system resume if PCI device remained enabled During system resume we depended on pci_enable_device() also putting the device into PCI D0 state. This won't work if the PCI device was already enabled but still in D3 state. This is because pci_enable_device() is refcounted and will not change the HW state if called with a non-zero refcount. Leaving the device in D3 will make all subsequent device accesses fail. This didn't cause a problem most of the time, since we resumed with an enable refcount of 0. But it fails at least after module reload because after that we also happen to leak a PCI device enable reference: During probing we call drm_get_pci_dev() which will enable the PCI device, but during device removal drm_put_dev() won't disable it. This is a bug of its own in DRM core, but without much harm as it only leaves the PCI device enabled. Fixing it is also a bit more involved, due to DRM mid-layering and because it affects non-i915 drivers too. The fix in this patch is valid regardless of the problem in DRM core. v2: - Add a code comment about the relation of this fix to the freeze/thaw vs. the suspend/resume phases. (Ville) - Add a code comment about the inconsistent ordering of set power state and device enable calls. (Chris) CC: Ville Syrjälä <ville.syrjala@linux.intel.com> CC: Chris Wilson <chris@chris-wilson.co.uk> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1460979954-14503-1-git-send-email-imre.deak@intel.com
2016-04-18 14:45:54 +03:00
int ret;
/*
* We have a resume ordering issue with the snd-hda driver also
* requiring our device to be power up. Due to the lack of a
* parent/child relationship we currently solve this with an early
* resume hook.
*
* FIXME: This should be solved with a special hdmi sink device or
* similar so that power domains can be employed.
*/
drm/i915: Fix system resume if PCI device remained enabled During system resume we depended on pci_enable_device() also putting the device into PCI D0 state. This won't work if the PCI device was already enabled but still in D3 state. This is because pci_enable_device() is refcounted and will not change the HW state if called with a non-zero refcount. Leaving the device in D3 will make all subsequent device accesses fail. This didn't cause a problem most of the time, since we resumed with an enable refcount of 0. But it fails at least after module reload because after that we also happen to leak a PCI device enable reference: During probing we call drm_get_pci_dev() which will enable the PCI device, but during device removal drm_put_dev() won't disable it. This is a bug of its own in DRM core, but without much harm as it only leaves the PCI device enabled. Fixing it is also a bit more involved, due to DRM mid-layering and because it affects non-i915 drivers too. The fix in this patch is valid regardless of the problem in DRM core. v2: - Add a code comment about the relation of this fix to the freeze/thaw vs. the suspend/resume phases. (Ville) - Add a code comment about the inconsistent ordering of set power state and device enable calls. (Chris) CC: Ville Syrjälä <ville.syrjala@linux.intel.com> CC: Chris Wilson <chris@chris-wilson.co.uk> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1460979954-14503-1-git-send-email-imre.deak@intel.com
2016-04-18 14:45:54 +03:00
/*
* Note that we need to set the power state explicitly, since we
* powered off the device during freeze and the PCI core won't power
* it back up for us during thaw. Powering off the device during
* freeze is not a hard requirement though, and during the
* suspend/resume phases the PCI core makes sure we get here with the
* device powered on. So in case we change our freeze logic and keep
* the device powered we can also remove the following set power state
* call.
*/
ret = pci_set_power_state(pdev, PCI_D0);
drm/i915: Fix system resume if PCI device remained enabled During system resume we depended on pci_enable_device() also putting the device into PCI D0 state. This won't work if the PCI device was already enabled but still in D3 state. This is because pci_enable_device() is refcounted and will not change the HW state if called with a non-zero refcount. Leaving the device in D3 will make all subsequent device accesses fail. This didn't cause a problem most of the time, since we resumed with an enable refcount of 0. But it fails at least after module reload because after that we also happen to leak a PCI device enable reference: During probing we call drm_get_pci_dev() which will enable the PCI device, but during device removal drm_put_dev() won't disable it. This is a bug of its own in DRM core, but without much harm as it only leaves the PCI device enabled. Fixing it is also a bit more involved, due to DRM mid-layering and because it affects non-i915 drivers too. The fix in this patch is valid regardless of the problem in DRM core. v2: - Add a code comment about the relation of this fix to the freeze/thaw vs. the suspend/resume phases. (Ville) - Add a code comment about the inconsistent ordering of set power state and device enable calls. (Chris) CC: Ville Syrjälä <ville.syrjala@linux.intel.com> CC: Chris Wilson <chris@chris-wilson.co.uk> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1460979954-14503-1-git-send-email-imre.deak@intel.com
2016-04-18 14:45:54 +03:00
if (ret) {
DRM_ERROR("failed to set PCI D0 power state (%d)\n", ret);
return ret;
drm/i915: Fix system resume if PCI device remained enabled During system resume we depended on pci_enable_device() also putting the device into PCI D0 state. This won't work if the PCI device was already enabled but still in D3 state. This is because pci_enable_device() is refcounted and will not change the HW state if called with a non-zero refcount. Leaving the device in D3 will make all subsequent device accesses fail. This didn't cause a problem most of the time, since we resumed with an enable refcount of 0. But it fails at least after module reload because after that we also happen to leak a PCI device enable reference: During probing we call drm_get_pci_dev() which will enable the PCI device, but during device removal drm_put_dev() won't disable it. This is a bug of its own in DRM core, but without much harm as it only leaves the PCI device enabled. Fixing it is also a bit more involved, due to DRM mid-layering and because it affects non-i915 drivers too. The fix in this patch is valid regardless of the problem in DRM core. v2: - Add a code comment about the relation of this fix to the freeze/thaw vs. the suspend/resume phases. (Ville) - Add a code comment about the inconsistent ordering of set power state and device enable calls. (Chris) CC: Ville Syrjälä <ville.syrjala@linux.intel.com> CC: Chris Wilson <chris@chris-wilson.co.uk> CC: stable@vger.kernel.org Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/1460979954-14503-1-git-send-email-imre.deak@intel.com
2016-04-18 14:45:54 +03:00
}
/*
* Note that pci_enable_device() first enables any parent bridge
* device and only then sets the power state for this device. The
* bridge enabling is a nop though, since bridge devices are resumed
* first. The order of enabling power and enabling the device is
* imposed by the PCI core as described above, so here we preserve the
* same order for the freeze/thaw phases.
*
* TODO: eventually we should remove pci_disable_device() /
* pci_enable_enable_device() from suspend/resume. Due to how they
* depend on the device enable refcount we can't anyway depend on them
* disabling/enabling the device.
*/
if (pci_enable_device(pdev))
return -EIO;
pci_set_master(pdev);
disable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
ret = vlv_resume_prepare(dev_priv, false);
if (ret)
DRM_ERROR("Resume prepare failed: %d, continuing anyway\n",
ret);
intel_uncore_resume_early(&dev_priv->uncore);
intel_gt_check_and_clear_faults(&dev_priv->gt);
drm/i915/icl: Enable DC9 as lowest possible state during screen-off ICL supports DC5, DC6, and DC9. Enable DC9 during screen-off, and enable DC5/6 when appropriate. v2: (James Ausmus) - Also handle ICL as GEN9_LP in i915_drm_suspend_late and i915_drm_suspend_early - Add DC9 to gen9_dc_mask for ICL - Re-order GEN checks for newest platform first - Use INTEL_GEN instead of INTEL_INFO->gen - Use INTEL_GEN >= 11 instead of IS_ICELAKE - Consolidate GEN checks v3: (James Ausmus) - Also allow DC6 for ICL (Imre, Art) - Simplify !(GEN >= 11) to GEN < 11 (Imre) v4: (James Ausmus) - Don't call intel_power_sequencer_reset after DC9 for Gen11+, as the PPS regs are Always On - Rebase against upstream changes v5: (Anusha Srivatsa) - rebased against the latest upstream changes. v6: (Anusha Srivatsa) - rebased.Use INTEL_GEN consistently. - Simplify the code (Rodrigo) v7: rebased. Change order according to platforms(Jyoti) v8: rebased. Change the check from platform specific to HAS_PCH_SPLIT(). Add comment in code to be more clear.(Rodrigo) Cc: Imre Deak <imre.deak@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Animesh Manna <animesh.manna@intel.com> Signed-off-by: James Ausmus <james.ausmus@intel.com> Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com> Tested-by: Jyoti Yadav <jyoti.r.yadav@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20181029221410.4423-1-anusha.srivatsa@intel.com
2018-10-30 01:14:10 +03:00
if (INTEL_GEN(dev_priv) >= 11 || IS_GEN9_LP(dev_priv)) {
drm/i915: Fix hibernation with ACPI S0 target state After commit dd9f31c7a3887950cbd0d49eb9d43f7a1518a356 Author: Imre Deak <imre.deak@intel.com> Date: Wed Aug 16 17:46:07 2017 +0300 drm/i915/gen9+: Set same power state before hibernation image save/restore during hibernation/suspend the power domain functionality got disabled, after which resume could leave it incorrectly disabled if the ACPI target state was S0 during suspend and i915 was not loaded by the loader kernel. This was caused by not considering if we resumed from hibernation as the condition for power domains reiniting. Fix this by simply tracking if we suspended power domains during system suspend and reinit power domains accordingly during resume. This will result in reiniting power domains always when resuming from hibernation, regardless of the platform and whether or not i915 is loaded by the loader kernel. The reason we didn't catch this earlier is that the enabled/disabled state of power domains during PMSG_FREEZE/PMSG_QUIESCE is platform and kernel config dependent: on my SKL the target state is S4 during PMSG_FREEZE and (with the driver loaded in the loader kernel) S0 during PMSG_QUIESCE. On the reporter's machine it's S0 during PMSG_FREEZE but (contrary to this) power domains are not initialized during PMSG_QUIESCE since i915 is not loaded in the loader kernel, or it's loaded but without the DMC firmware being available. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=105196 Reported-and-tested-by: amn-bas@hotmail.com Fixes: dd9f31c7a388 ("drm/i915/gen9+: Set same power state before hibernation image save/restore") Cc: amn-bas@hotmail.com Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: <stable@vger.kernel.org> Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180322143642.26883-1-imre.deak@intel.com
2018-03-22 17:36:42 +03:00
gen9_sanitize_dc_state(dev_priv);
bxt_disable_dc9(dev_priv);
} else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
hsw_disable_pc8(dev_priv);
}
intel_sanitize_gt_powersave(dev_priv);
intel_power_domains_resume(dev_priv);
intel_gt_sanitize(&dev_priv->gt, true);
enable_rpm_wakeref_asserts(&dev_priv->runtime_pm);
return ret;
}
static int i915_resume_switcheroo(struct drm_device *dev)
{
int ret;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
ret = i915_drm_resume_early(dev);
if (ret)
return ret;
return i915_drm_resume(dev);
}
static int i915_pm_prepare(struct device *kdev)
{
struct drm_device *dev = dev_get_drvdata(kdev);
if (!dev) {
dev_err(kdev, "DRM not initialized, aborting suspend.\n");
return -ENODEV;
}
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
return i915_drm_prepare(dev);
}
static int i915_pm_suspend(struct device *kdev)
{
struct drm_device *dev = dev_get_drvdata(kdev);
if (!dev) {
dev_err(kdev, "DRM not initialized, aborting suspend.\n");
return -ENODEV;
}
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
return i915_drm_suspend(dev);
}
static int i915_pm_suspend_late(struct device *kdev)
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
/*
* We have a suspend ordering issue with the snd-hda driver also
* requiring our device to be power up. Due to the lack of a
* parent/child relationship we currently solve this with an late
* suspend hook.
*
* FIXME: This should be solved with a special hdmi sink device or
* similar so that power domains can be employed.
*/
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
return i915_drm_suspend_late(dev, false);
2015-03-02 14:04:41 +03:00
}
static int i915_pm_poweroff_late(struct device *kdev)
2015-03-02 14:04:41 +03:00
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
2015-03-02 14:04:41 +03:00
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
2015-03-02 14:04:41 +03:00
return 0;
return i915_drm_suspend_late(dev, true);
}
static int i915_pm_resume_early(struct device *kdev)
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
return i915_drm_resume_early(dev);
}
static int i915_pm_resume(struct device *kdev)
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
return i915_drm_resume(dev);
}
/* freeze: before creating the hibernation_image */
static int i915_pm_freeze(struct device *kdev)
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
drm/i915: Only shrink the unbound objects during freeze At the point of creating the hibernation image, the runtime power manage core is disabled - and using the rpm functions triggers a warn. i915_gem_shrink_all() tries to unbind objects, which requires device access and so tries to how an rpm reference triggering a warning: [ 44.235420] ------------[ cut here ]------------ [ 44.235424] WARNING: CPU: 2 PID: 2199 at drivers/gpu/drm/i915/intel_runtime_pm.c:2688 intel_runtime_pm_get_if_in_use+0xe6/0xf0 [ 44.235426] WARN_ON_ONCE(ret < 0) [ 44.235445] Modules linked in: ctr ccm arc4 rt2800usb rt2x00usb rt2800lib rt2x00lib crc_ccitt mac80211 cmac cfg80211 btusb rfcomm bnep btrtl btbcm btintel bluetooth dcdbas x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_codec_generic aesni_intel snd_hda_codec_hdmi aes_x86_64 lrw gf128mul snd_hda_intel glue_helper ablk_helper cryptd snd_hda_codec hid_multitouch joydev snd_hda_core binfmt_misc i2c_hid serio_raw snd_pcm acpi_pad snd_timer snd i2c_designware_platform 8250_dw nls_iso8859_1 i2c_designware_core lpc_ich mfd_core soundcore usbhid hid psmouse ahci libahci [ 44.235447] CPU: 2 PID: 2199 Comm: kworker/u8:8 Not tainted 4.8.0-rc5+ #130 [ 44.235447] Hardware name: Dell Inc. XPS 13 9343/0310JH, BIOS A07 11/11/2015 [ 44.235450] Workqueue: events_unbound async_run_entry_fn [ 44.235453] 0000000000000000 ffff8801b2f7fb98 ffffffff81306c2f ffff8801b2f7fbe8 [ 44.235454] 0000000000000000 ffff8801b2f7fbd8 ffffffff81056c01 00000a801f50ecc0 [ 44.235456] ffff88020ce50000 ffff88020ce59b60 ffffffff81a60b5c ffffffff81414840 [ 44.235456] Call Trace: [ 44.235459] [<ffffffff81306c2f>] dump_stack+0x4d/0x6e [ 44.235461] [<ffffffff81056c01>] __warn+0xd1/0xf0 [ 44.235464] [<ffffffff81414840>] ? i915_pm_suspend_late+0x30/0x30 [ 44.235465] [<ffffffff81056c6f>] warn_slowpath_fmt+0x4f/0x60 [ 44.235468] [<ffffffff814e73ce>] ? pm_runtime_get_if_in_use+0x6e/0xa0 [ 44.235469] [<ffffffff81433526>] intel_runtime_pm_get_if_in_use+0xe6/0xf0 [ 44.235471] [<ffffffff81458a26>] i915_gem_shrink+0x306/0x360 [ 44.235473] [<ffffffff81343fd4>] ? pci_platform_power_transition+0x24/0x90 [ 44.235475] [<ffffffff81414840>] ? i915_pm_suspend_late+0x30/0x30 [ 44.235476] [<ffffffff81458dfb>] i915_gem_shrink_all+0x1b/0x30 [ 44.235478] [<ffffffff814560b3>] i915_gem_freeze_late+0x33/0x90 [ 44.235479] [<ffffffff81414877>] i915_pm_freeze_late+0x37/0x40 [ 44.235481] [<ffffffff814e9b8e>] dpm_run_callback+0x4e/0x130 [ 44.235483] [<ffffffff814ea5db>] __device_suspend_late+0xdb/0x1f0 [ 44.235484] [<ffffffff814ea70f>] async_suspend_late+0x1f/0xa0 [ 44.235486] [<ffffffff81077557>] async_run_entry_fn+0x37/0x150 [ 44.235488] [<ffffffff8106f518>] process_one_work+0x148/0x3f0 [ 44.235490] [<ffffffff8106f8eb>] worker_thread+0x12b/0x490 [ 44.235491] [<ffffffff8106f7c0>] ? process_one_work+0x3f0/0x3f0 [ 44.235492] [<ffffffff81074d09>] kthread+0xc9/0xe0 [ 44.235495] [<ffffffff816e257f>] ret_from_fork+0x1f/0x40 [ 44.235496] [<ffffffff81074c40>] ? kthread_park+0x60/0x60 [ 44.235497] ---[ end trace e438706b97c7f132 ]--- Alternatively, to actually shrink everything we have to do so slightly earlier in the hibernation process. To keep lockdep silent, we need to take struct_mutex for the shrinker even though we know that we are the only user during the freeze. Fixes: 7aab2d534e35 ("drm/i915: Shrink objects prior to hibernation") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20160921135108.29574-2-chris@chris-wilson.co.uk
2016-09-21 16:51:07 +03:00
int ret;
if (dev->switch_power_state != DRM_SWITCH_POWER_OFF) {
ret = i915_drm_suspend(dev);
if (ret)
return ret;
}
drm/i915: Only shrink the unbound objects during freeze At the point of creating the hibernation image, the runtime power manage core is disabled - and using the rpm functions triggers a warn. i915_gem_shrink_all() tries to unbind objects, which requires device access and so tries to how an rpm reference triggering a warning: [ 44.235420] ------------[ cut here ]------------ [ 44.235424] WARNING: CPU: 2 PID: 2199 at drivers/gpu/drm/i915/intel_runtime_pm.c:2688 intel_runtime_pm_get_if_in_use+0xe6/0xf0 [ 44.235426] WARN_ON_ONCE(ret < 0) [ 44.235445] Modules linked in: ctr ccm arc4 rt2800usb rt2x00usb rt2800lib rt2x00lib crc_ccitt mac80211 cmac cfg80211 btusb rfcomm bnep btrtl btbcm btintel bluetooth dcdbas x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_codec_generic aesni_intel snd_hda_codec_hdmi aes_x86_64 lrw gf128mul snd_hda_intel glue_helper ablk_helper cryptd snd_hda_codec hid_multitouch joydev snd_hda_core binfmt_misc i2c_hid serio_raw snd_pcm acpi_pad snd_timer snd i2c_designware_platform 8250_dw nls_iso8859_1 i2c_designware_core lpc_ich mfd_core soundcore usbhid hid psmouse ahci libahci [ 44.235447] CPU: 2 PID: 2199 Comm: kworker/u8:8 Not tainted 4.8.0-rc5+ #130 [ 44.235447] Hardware name: Dell Inc. XPS 13 9343/0310JH, BIOS A07 11/11/2015 [ 44.235450] Workqueue: events_unbound async_run_entry_fn [ 44.235453] 0000000000000000 ffff8801b2f7fb98 ffffffff81306c2f ffff8801b2f7fbe8 [ 44.235454] 0000000000000000 ffff8801b2f7fbd8 ffffffff81056c01 00000a801f50ecc0 [ 44.235456] ffff88020ce50000 ffff88020ce59b60 ffffffff81a60b5c ffffffff81414840 [ 44.235456] Call Trace: [ 44.235459] [<ffffffff81306c2f>] dump_stack+0x4d/0x6e [ 44.235461] [<ffffffff81056c01>] __warn+0xd1/0xf0 [ 44.235464] [<ffffffff81414840>] ? i915_pm_suspend_late+0x30/0x30 [ 44.235465] [<ffffffff81056c6f>] warn_slowpath_fmt+0x4f/0x60 [ 44.235468] [<ffffffff814e73ce>] ? pm_runtime_get_if_in_use+0x6e/0xa0 [ 44.235469] [<ffffffff81433526>] intel_runtime_pm_get_if_in_use+0xe6/0xf0 [ 44.235471] [<ffffffff81458a26>] i915_gem_shrink+0x306/0x360 [ 44.235473] [<ffffffff81343fd4>] ? pci_platform_power_transition+0x24/0x90 [ 44.235475] [<ffffffff81414840>] ? i915_pm_suspend_late+0x30/0x30 [ 44.235476] [<ffffffff81458dfb>] i915_gem_shrink_all+0x1b/0x30 [ 44.235478] [<ffffffff814560b3>] i915_gem_freeze_late+0x33/0x90 [ 44.235479] [<ffffffff81414877>] i915_pm_freeze_late+0x37/0x40 [ 44.235481] [<ffffffff814e9b8e>] dpm_run_callback+0x4e/0x130 [ 44.235483] [<ffffffff814ea5db>] __device_suspend_late+0xdb/0x1f0 [ 44.235484] [<ffffffff814ea70f>] async_suspend_late+0x1f/0xa0 [ 44.235486] [<ffffffff81077557>] async_run_entry_fn+0x37/0x150 [ 44.235488] [<ffffffff8106f518>] process_one_work+0x148/0x3f0 [ 44.235490] [<ffffffff8106f8eb>] worker_thread+0x12b/0x490 [ 44.235491] [<ffffffff8106f7c0>] ? process_one_work+0x3f0/0x3f0 [ 44.235492] [<ffffffff81074d09>] kthread+0xc9/0xe0 [ 44.235495] [<ffffffff816e257f>] ret_from_fork+0x1f/0x40 [ 44.235496] [<ffffffff81074c40>] ? kthread_park+0x60/0x60 [ 44.235497] ---[ end trace e438706b97c7f132 ]--- Alternatively, to actually shrink everything we have to do so slightly earlier in the hibernation process. To keep lockdep silent, we need to take struct_mutex for the shrinker even though we know that we are the only user during the freeze. Fixes: 7aab2d534e35 ("drm/i915: Shrink objects prior to hibernation") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20160921135108.29574-2-chris@chris-wilson.co.uk
2016-09-21 16:51:07 +03:00
ret = i915_gem_freeze(kdev_to_i915(kdev));
if (ret)
return ret;
return 0;
}
static int i915_pm_freeze_late(struct device *kdev)
{
struct drm_device *dev = &kdev_to_i915(kdev)->drm;
int ret;
if (dev->switch_power_state != DRM_SWITCH_POWER_OFF) {
ret = i915_drm_suspend_late(dev, true);
if (ret)
return ret;
}
ret = i915_gem_freeze_late(kdev_to_i915(kdev));
if (ret)
return ret;
return 0;
}
/* thaw: called after creating the hibernation image, but before turning off. */
static int i915_pm_thaw_early(struct device *kdev)
{
return i915_pm_resume_early(kdev);
}
static int i915_pm_thaw(struct device *kdev)
{
return i915_pm_resume(kdev);
}
/* restore: called after loading the hibernation image. */
static int i915_pm_restore_early(struct device *kdev)
{
return i915_pm_resume_early(kdev);
}
static int i915_pm_restore(struct device *kdev)
{
return i915_pm_resume(kdev);
}
/*
* Save all Gunit registers that may be lost after a D3 and a subsequent
* S0i[R123] transition. The list of registers needing a save/restore is
* defined in the VLV2_S0IXRegs document. This documents marks all Gunit
* registers in the following way:
* - Driver: saved/restored by the driver
* - Punit : saved/restored by the Punit firmware
* - No, w/o marking: no need to save/restore, since the register is R/O or
* used internally by the HW in a way that doesn't depend
* keeping the content across a suspend/resume.
* - Debug : used for debugging
*
* We save/restore all registers marked with 'Driver', with the following
* exceptions:
* - Registers out of use, including also registers marked with 'Debug'.
* These have no effect on the driver's operation, so we don't save/restore
* them to reduce the overhead.
* - Registers that are fully setup by an initialization function called from
* the resume path. For example many clock gating and RPS/RC6 registers.
* - Registers that provide the right functionality with their reset defaults.
*
* TODO: Except for registers that based on the above 3 criteria can be safely
* ignored, we save/restore all others, practically treating the HW context as
* a black-box for the driver. Further investigation is needed to reduce the
* saved/restored registers even further, by following the same 3 criteria.
*/
static void vlv_save_gunit_s0ix_state(struct drm_i915_private *dev_priv)
{
struct vlv_s0ix_state *s = &dev_priv->vlv_s0ix_state;
int i;
/* GAM 0x4000-0x4770 */
s->wr_watermark = I915_READ(GEN7_WR_WATERMARK);
s->gfx_prio_ctrl = I915_READ(GEN7_GFX_PRIO_CTRL);
s->arb_mode = I915_READ(ARB_MODE);
s->gfx_pend_tlb0 = I915_READ(GEN7_GFX_PEND_TLB0);
s->gfx_pend_tlb1 = I915_READ(GEN7_GFX_PEND_TLB1);
for (i = 0; i < ARRAY_SIZE(s->lra_limits); i++)
s->lra_limits[i] = I915_READ(GEN7_LRA_LIMITS(i));
s->media_max_req_count = I915_READ(GEN7_MEDIA_MAX_REQ_COUNT);
s->gfx_max_req_count = I915_READ(GEN7_GFX_MAX_REQ_COUNT);
s->render_hwsp = I915_READ(RENDER_HWS_PGA_GEN7);
s->ecochk = I915_READ(GAM_ECOCHK);
s->bsd_hwsp = I915_READ(BSD_HWS_PGA_GEN7);
s->blt_hwsp = I915_READ(BLT_HWS_PGA_GEN7);
s->tlb_rd_addr = I915_READ(GEN7_TLB_RD_ADDR);
/* MBC 0x9024-0x91D0, 0x8500 */
s->g3dctl = I915_READ(VLV_G3DCTL);
s->gsckgctl = I915_READ(VLV_GSCKGCTL);
s->mbctl = I915_READ(GEN6_MBCTL);
/* GCP 0x9400-0x9424, 0x8100-0x810C */
s->ucgctl1 = I915_READ(GEN6_UCGCTL1);
s->ucgctl3 = I915_READ(GEN6_UCGCTL3);
s->rcgctl1 = I915_READ(GEN6_RCGCTL1);
s->rcgctl2 = I915_READ(GEN6_RCGCTL2);
s->rstctl = I915_READ(GEN6_RSTCTL);
s->misccpctl = I915_READ(GEN7_MISCCPCTL);
/* GPM 0xA000-0xAA84, 0x8000-0x80FC */
s->gfxpause = I915_READ(GEN6_GFXPAUSE);
s->rpdeuhwtc = I915_READ(GEN6_RPDEUHWTC);
s->rpdeuc = I915_READ(GEN6_RPDEUC);
s->ecobus = I915_READ(ECOBUS);
s->pwrdwnupctl = I915_READ(VLV_PWRDWNUPCTL);
s->rp_down_timeout = I915_READ(GEN6_RP_DOWN_TIMEOUT);
s->rp_deucsw = I915_READ(GEN6_RPDEUCSW);
s->rcubmabdtmr = I915_READ(GEN6_RCUBMABDTMR);
s->rcedata = I915_READ(VLV_RCEDATA);
s->spare2gh = I915_READ(VLV_SPAREG2H);
/* Display CZ domain, 0x4400C-0x4402C, 0x4F000-0x4F11F */
s->gt_imr = I915_READ(GTIMR);
s->gt_ier = I915_READ(GTIER);
s->pm_imr = I915_READ(GEN6_PMIMR);
s->pm_ier = I915_READ(GEN6_PMIER);
for (i = 0; i < ARRAY_SIZE(s->gt_scratch); i++)
s->gt_scratch[i] = I915_READ(GEN7_GT_SCRATCH(i));
/* GT SA CZ domain, 0x100000-0x138124 */
s->tilectl = I915_READ(TILECTL);
s->gt_fifoctl = I915_READ(GTFIFOCTL);
s->gtlc_wake_ctrl = I915_READ(VLV_GTLC_WAKE_CTRL);
s->gtlc_survive = I915_READ(VLV_GTLC_SURVIVABILITY_REG);
s->pmwgicz = I915_READ(VLV_PMWGICZ);
/* Gunit-Display CZ domain, 0x182028-0x1821CF */
s->gu_ctl0 = I915_READ(VLV_GU_CTL0);
s->gu_ctl1 = I915_READ(VLV_GU_CTL1);
s->pcbr = I915_READ(VLV_PCBR);
s->clock_gate_dis2 = I915_READ(VLV_GUNIT_CLOCK_GATE2);
/*
* Not saving any of:
* DFT, 0x9800-0x9EC0
* SARB, 0xB000-0xB1FC
* GAC, 0x5208-0x524C, 0x14000-0x14C000
* PCI CFG
*/
}
static void vlv_restore_gunit_s0ix_state(struct drm_i915_private *dev_priv)
{
struct vlv_s0ix_state *s = &dev_priv->vlv_s0ix_state;
u32 val;
int i;
/* GAM 0x4000-0x4770 */
I915_WRITE(GEN7_WR_WATERMARK, s->wr_watermark);
I915_WRITE(GEN7_GFX_PRIO_CTRL, s->gfx_prio_ctrl);
I915_WRITE(ARB_MODE, s->arb_mode | (0xffff << 16));
I915_WRITE(GEN7_GFX_PEND_TLB0, s->gfx_pend_tlb0);
I915_WRITE(GEN7_GFX_PEND_TLB1, s->gfx_pend_tlb1);
for (i = 0; i < ARRAY_SIZE(s->lra_limits); i++)
I915_WRITE(GEN7_LRA_LIMITS(i), s->lra_limits[i]);
I915_WRITE(GEN7_MEDIA_MAX_REQ_COUNT, s->media_max_req_count);
I915_WRITE(GEN7_GFX_MAX_REQ_COUNT, s->gfx_max_req_count);
I915_WRITE(RENDER_HWS_PGA_GEN7, s->render_hwsp);
I915_WRITE(GAM_ECOCHK, s->ecochk);
I915_WRITE(BSD_HWS_PGA_GEN7, s->bsd_hwsp);
I915_WRITE(BLT_HWS_PGA_GEN7, s->blt_hwsp);
I915_WRITE(GEN7_TLB_RD_ADDR, s->tlb_rd_addr);
/* MBC 0x9024-0x91D0, 0x8500 */
I915_WRITE(VLV_G3DCTL, s->g3dctl);
I915_WRITE(VLV_GSCKGCTL, s->gsckgctl);
I915_WRITE(GEN6_MBCTL, s->mbctl);
/* GCP 0x9400-0x9424, 0x8100-0x810C */
I915_WRITE(GEN6_UCGCTL1, s->ucgctl1);
I915_WRITE(GEN6_UCGCTL3, s->ucgctl3);
I915_WRITE(GEN6_RCGCTL1, s->rcgctl1);
I915_WRITE(GEN6_RCGCTL2, s->rcgctl2);
I915_WRITE(GEN6_RSTCTL, s->rstctl);
I915_WRITE(GEN7_MISCCPCTL, s->misccpctl);
/* GPM 0xA000-0xAA84, 0x8000-0x80FC */
I915_WRITE(GEN6_GFXPAUSE, s->gfxpause);
I915_WRITE(GEN6_RPDEUHWTC, s->rpdeuhwtc);
I915_WRITE(GEN6_RPDEUC, s->rpdeuc);
I915_WRITE(ECOBUS, s->ecobus);
I915_WRITE(VLV_PWRDWNUPCTL, s->pwrdwnupctl);
I915_WRITE(GEN6_RP_DOWN_TIMEOUT,s->rp_down_timeout);
I915_WRITE(GEN6_RPDEUCSW, s->rp_deucsw);
I915_WRITE(GEN6_RCUBMABDTMR, s->rcubmabdtmr);
I915_WRITE(VLV_RCEDATA, s->rcedata);
I915_WRITE(VLV_SPAREG2H, s->spare2gh);
/* Display CZ domain, 0x4400C-0x4402C, 0x4F000-0x4F11F */
I915_WRITE(GTIMR, s->gt_imr);
I915_WRITE(GTIER, s->gt_ier);
I915_WRITE(GEN6_PMIMR, s->pm_imr);
I915_WRITE(GEN6_PMIER, s->pm_ier);
for (i = 0; i < ARRAY_SIZE(s->gt_scratch); i++)
I915_WRITE(GEN7_GT_SCRATCH(i), s->gt_scratch[i]);
/* GT SA CZ domain, 0x100000-0x138124 */
I915_WRITE(TILECTL, s->tilectl);
I915_WRITE(GTFIFOCTL, s->gt_fifoctl);
/*
* Preserve the GT allow wake and GFX force clock bit, they are not
* be restored, as they are used to control the s0ix suspend/resume
* sequence by the caller.
*/
val = I915_READ(VLV_GTLC_WAKE_CTRL);
val &= VLV_GTLC_ALLOWWAKEREQ;
val |= s->gtlc_wake_ctrl & ~VLV_GTLC_ALLOWWAKEREQ;
I915_WRITE(VLV_GTLC_WAKE_CTRL, val);
val = I915_READ(VLV_GTLC_SURVIVABILITY_REG);
val &= VLV_GFX_CLK_FORCE_ON_BIT;
val |= s->gtlc_survive & ~VLV_GFX_CLK_FORCE_ON_BIT;
I915_WRITE(VLV_GTLC_SURVIVABILITY_REG, val);
I915_WRITE(VLV_PMWGICZ, s->pmwgicz);
/* Gunit-Display CZ domain, 0x182028-0x1821CF */
I915_WRITE(VLV_GU_CTL0, s->gu_ctl0);
I915_WRITE(VLV_GU_CTL1, s->gu_ctl1);
I915_WRITE(VLV_PCBR, s->pcbr);
I915_WRITE(VLV_GUNIT_CLOCK_GATE2, s->clock_gate_dis2);
}
static int vlv_wait_for_pw_status(struct drm_i915_private *i915,
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
u32 mask, u32 val)
{
i915_reg_t reg = VLV_GTLC_PW_STATUS;
u32 reg_value;
int ret;
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
/* The HW does not like us polling for PW_STATUS frequently, so
* use the sleeping loop rather than risk the busy spin within
* intel_wait_for_register().
*
* Transitioning between RC6 states should be at most 2ms (see
* valleyview_enable_rps) so use a 3ms timeout.
*/
ret = wait_for(((reg_value =
intel_uncore_read_notrace(&i915->uncore, reg)) & mask)
== val, 3);
/* just trace the final value */
trace_i915_reg_rw(false, reg, reg_value, sizeof(reg_value), true);
return ret;
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
}
int vlv_force_gfx_clock(struct drm_i915_private *dev_priv, bool force_on)
{
u32 val;
int err;
val = I915_READ(VLV_GTLC_SURVIVABILITY_REG);
val &= ~VLV_GFX_CLK_FORCE_ON_BIT;
if (force_on)
val |= VLV_GFX_CLK_FORCE_ON_BIT;
I915_WRITE(VLV_GTLC_SURVIVABILITY_REG, val);
if (!force_on)
return 0;
err = intel_wait_for_register(&dev_priv->uncore,
VLV_GTLC_SURVIVABILITY_REG,
VLV_GFX_CLK_STATUS_BIT,
VLV_GFX_CLK_STATUS_BIT,
20);
if (err)
DRM_ERROR("timeout waiting for GFX clock force-on (%08x)\n",
I915_READ(VLV_GTLC_SURVIVABILITY_REG));
return err;
}
static int vlv_allow_gt_wake(struct drm_i915_private *dev_priv, bool allow)
{
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
u32 mask;
u32 val;
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
int err;
val = I915_READ(VLV_GTLC_WAKE_CTRL);
val &= ~VLV_GTLC_ALLOWWAKEREQ;
if (allow)
val |= VLV_GTLC_ALLOWWAKEREQ;
I915_WRITE(VLV_GTLC_WAKE_CTRL, val);
POSTING_READ(VLV_GTLC_WAKE_CTRL);
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
mask = VLV_GTLC_ALLOWWAKEACK;
val = allow ? mask : 0;
err = vlv_wait_for_pw_status(dev_priv, mask, val);
if (err)
DRM_ERROR("timeout disabling GT waking\n");
return err;
}
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
static void vlv_wait_for_gt_wells(struct drm_i915_private *dev_priv,
bool wait_for_on)
{
u32 mask;
u32 val;
mask = VLV_GTLC_PW_MEDIA_STATUS_MASK | VLV_GTLC_PW_RENDER_STATUS_MASK;
val = wait_for_on ? mask : 0;
/*
* RC6 transitioning can be delayed up to 2 msec (see
* valleyview_enable_rps), use 3 msec for safety.
*
* This can fail to turn off the rc6 if the GPU is stuck after a failed
* reset and we are trying to force the machine to sleep.
*/
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
if (vlv_wait_for_pw_status(dev_priv, mask, val))
DRM_DEBUG_DRIVER("timeout waiting for GT wells to go %s\n",
onoff(wait_for_on));
}
static void vlv_check_no_gt_access(struct drm_i915_private *dev_priv)
{
if (!(I915_READ(VLV_GTLC_PW_STATUS) & VLV_GTLC_ALLOWWAKEERR))
return;
DRM_DEBUG_DRIVER("GT register access while GT waking disabled\n");
I915_WRITE(VLV_GTLC_PW_STATUS, VLV_GTLC_ALLOWWAKEERR);
}
static int vlv_suspend_complete(struct drm_i915_private *dev_priv)
{
u32 mask;
int err;
/*
* Bspec defines the following GT well on flags as debug only, so
* don't treat them as hard failures.
*/
drm/i915: Avoid busy-spinning on VLV_GLTC_PW_STATUS mmio The busy-spin, as the first stage of intel_wait_for_register(), is currently under suspicion for causing: [ 62.034926] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 62.034928] Modules linked in: i2c_dev i915 intel_gtt drm_kms_helper prime_numbers [ 62.034932] CPU: 1 PID: 183 Comm: kworker/1:2 Not tainted 4.11.0-rc7+ #471 [ 62.034933] Hardware name: / , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015 [ 62.034934] Workqueue: pm pm_runtime_work [ 62.034936] task: ffff880275a04ec0 task.stack: ffffc900002d8000 [ 62.034936] RIP: 0010:__intel_wait_for_register_fw+0x77/0x1a0 [i915] [ 62.034937] RSP: 0018:ffffc900002dbc38 EFLAGS: 00000082 [ 62.034939] RAX: ffffc90003530094 RBX: 0000000000130094 RCX: 0000000000000001 [ 62.034940] RDX: 00000000000000a1 RSI: ffff88027fd15e58 RDI: 0000000000000000 [ 62.034941] RBP: ffffc900002dbc78 R08: 0000000000000002 R09: 0000000000000000 [ 62.034942] R10: ffffc900002dbc18 R11: ffff880276429dd0 R12: ffff8802707c0000 [ 62.034943] R13: 00000000000000a0 R14: 0000000000000000 R15: 00000000fffefc10 [ 62.034945] FS: 0000000000000000(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000 [ 62.034945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 62.034947] CR2: 00007ffd3cd98ff8 CR3: 0000000274c19000 CR4: 00000000001006e0 [ 62.034947] Call Trace: [ 62.034948] intel_wait_for_register+0x77/0x140 [i915] [ 62.034949] vlv_suspend_complete+0x23/0x5b0 [i915] [ 62.034950] intel_runtime_suspend+0x16c/0x2a0 [i915] [ 62.034950] pci_pm_runtime_suspend+0x50/0x180 [ 62.034951] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034952] __rpm_callback+0xc5/0x210 [ 62.034953] rpm_callback+0x1f/0x80 [ 62.034953] ? pci_pm_runtime_resume+0xa0/0xa0 [ 62.034954] rpm_suspend+0x118/0x580 [ 62.034955] pm_runtime_work+0x64/0x90 [ 62.034956] process_one_work+0x1bb/0x3e0 [ 62.034956] worker_thread+0x46/0x4f0 [ 62.034957] ? __schedule+0x18b/0x610 [ 62.034958] kthread+0xff/0x140 [ 62.034958] ? process_one_work+0x3e0/0x3e0 [ 62.034959] ? kthread_create_on_node+ and related hard lockups in CI for byt and bsw. Note this effectively reverts commits 41ce405e6894 and b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") v2: Convert bool allow into a u32 mask for clarity and repeat the comment on vlv rc6 timing to justify the 3ms timeout used for the wait (Ville) Fixes: 41ce405e6894 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Fixes: b27366958869 ("drm/i915: Convert wait_for(I915_READ(reg)) to intel_wait_for_register()") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100718 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Tomi Sarvela <tomi.p.sarvela@intel.com> Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170421135815.11897-1-chris@chris-wilson.co.uk Tested-by: Tomi Sarvela <tomi.p.sarvela@intel.com>
2017-04-21 16:58:15 +03:00
vlv_wait_for_gt_wells(dev_priv, false);
mask = VLV_GTLC_RENDER_CTX_EXISTS | VLV_GTLC_MEDIA_CTX_EXISTS;
WARN_ON((I915_READ(VLV_GTLC_WAKE_CTRL) & mask) != mask);
vlv_check_no_gt_access(dev_priv);
err = vlv_force_gfx_clock(dev_priv, true);
if (err)
goto err1;
err = vlv_allow_gt_wake(dev_priv, false);
if (err)
goto err2;
if (!IS_CHERRYVIEW(dev_priv))
vlv_save_gunit_s0ix_state(dev_priv);
err = vlv_force_gfx_clock(dev_priv, false);
if (err)
goto err2;
return 0;
err2:
/* For safety always re-enable waking and disable gfx clock forcing */
vlv_allow_gt_wake(dev_priv, true);
err1:
vlv_force_gfx_clock(dev_priv, false);
return err;
}
drm/i915: Sharing platform specific sequence between runtime and system suspend/ resume paths On VLV, post S0i3 during i915_drm_thaw following issue is observed during ring initialization. [ 335.604039] [drm:stop_ring] ERROR render ring :timed out trying to stop ring [ 336.607340] [drm:stop_ring] ERROR render ring :timed out trying to stop ring [ 336.607345] [drm:init_ring_common] ERROR failed to set render ring head to zero ctl 00000000 head 00000000 tail 00000000 start 00000000 [ 337.610645] [drm:stop_ring] ERROR bsd ring :timed out trying to stop ring [ 338.613952] [drm:stop_ring] ERROR bsd ring :timed out trying to stop ring [ 338.613956] [drm:init_ring_common] ERROR failed to set bsd ring head to zero ctl 00000000 head 00000000 tail 00000000 start 00000000 [ 339.617256] [drm:stop_ring] ERROR render ring :timed out trying to stop ring [ 339.617258] -----------[ cut here ]----------- [ 339.617267] WARNING: CPU: 0 PID: 6 at drivers/gpu/drm/i915/intel_ringbuffer.c:1666 intel_cleanup_ring+0xe6/0xf0() [ 339.617396] --[ end trace 5ef5ed1a3c92e2a6 ]-- [ 339.617428] [drm:__i915_drm_thaw] ERROR failed to re-initialize GPU, declaring wedged! This is happening since wake is not enabled and Gunit registers are not restored. For this system suspend/resume paths need to follow save/restore and additional platform specific setup in suspend_complete and resume_prepare. suspend_complete is shared unconditionaly for VLV, HSW, BDW. resume_prepare for HSW and BDW has pc8 disabling which is needed during thaw_early so sharing uncondtionally. For VLV and SNB runtime resume specific sequence exists. Cc: Imre Deak <imre.deak@intel.com> Cc: Paulo Zanoni <paulo.r.zanoni@intel.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: Goel, Akash <akash.goel@intel.com> Signed-off-by: Sagar Kamble <sagar.a.kamble@intel.com> Reviewed-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2014-08-13 21:37:06 +04:00
static int vlv_resume_prepare(struct drm_i915_private *dev_priv,
bool rpm_resume)
{
int err;
int ret;
/*
* If any of the steps fail just try to continue, that's the best we
* can do at this point. Return the first error code (which will also
* leave RPM permanently disabled).
*/
ret = vlv_force_gfx_clock(dev_priv, true);
if (!IS_CHERRYVIEW(dev_priv))
vlv_restore_gunit_s0ix_state(dev_priv);
err = vlv_allow_gt_wake(dev_priv, true);
if (!ret)
ret = err;
err = vlv_force_gfx_clock(dev_priv, false);
if (!ret)
ret = err;
vlv_check_no_gt_access(dev_priv);
if (rpm_resume)
intel_init_clock_gating(dev_priv);
return ret;
}
static int intel_runtime_suspend(struct device *kdev)
{
struct drm_device *dev = dev_get_drvdata(kdev);
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
int ret;
if (WARN_ON_ONCE(!(dev_priv->gt_pm.rc6.enabled && HAS_RC6(dev_priv))))
return -ENODEV;
if (WARN_ON_ONCE(!HAS_RUNTIME_PM(dev_priv)))
return -ENODEV;
DRM_DEBUG_KMS("Suspending device\n");
disable_rpm_wakeref_asserts(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
/*
* We are safe here against re-faults, since the fault handler takes
* an RPM reference.
*/
i915_gem_runtime_suspend(dev_priv);
intel_uc_runtime_suspend(&dev_priv->gt.uc);
intel_runtime_pm_disable_interrupts(dev_priv);
intel_uncore_suspend(&dev_priv->uncore);
ret = 0;
drm/i915/icl: Enable DC9 as lowest possible state during screen-off ICL supports DC5, DC6, and DC9. Enable DC9 during screen-off, and enable DC5/6 when appropriate. v2: (James Ausmus) - Also handle ICL as GEN9_LP in i915_drm_suspend_late and i915_drm_suspend_early - Add DC9 to gen9_dc_mask for ICL - Re-order GEN checks for newest platform first - Use INTEL_GEN instead of INTEL_INFO->gen - Use INTEL_GEN >= 11 instead of IS_ICELAKE - Consolidate GEN checks v3: (James Ausmus) - Also allow DC6 for ICL (Imre, Art) - Simplify !(GEN >= 11) to GEN < 11 (Imre) v4: (James Ausmus) - Don't call intel_power_sequencer_reset after DC9 for Gen11+, as the PPS regs are Always On - Rebase against upstream changes v5: (Anusha Srivatsa) - rebased against the latest upstream changes. v6: (Anusha Srivatsa) - rebased.Use INTEL_GEN consistently. - Simplify the code (Rodrigo) v7: rebased. Change order according to platforms(Jyoti) v8: rebased. Change the check from platform specific to HAS_PCH_SPLIT(). Add comment in code to be more clear.(Rodrigo) Cc: Imre Deak <imre.deak@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Animesh Manna <animesh.manna@intel.com> Signed-off-by: James Ausmus <james.ausmus@intel.com> Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com> Tested-by: Jyoti Yadav <jyoti.r.yadav@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20181029221410.4423-1-anusha.srivatsa@intel.com
2018-10-30 01:14:10 +03:00
if (INTEL_GEN(dev_priv) >= 11) {
icl_display_core_uninit(dev_priv);
bxt_enable_dc9(dev_priv);
} else if (IS_GEN9_LP(dev_priv)) {
bxt_display_core_uninit(dev_priv);
bxt_enable_dc9(dev_priv);
} else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
hsw_enable_pc8(dev_priv);
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
ret = vlv_suspend_complete(dev_priv);
}
if (ret) {
DRM_ERROR("Runtime suspend failed, disabling it (%d)\n", ret);
intel_uncore_runtime_resume(&dev_priv->uncore);
intel_runtime_pm_enable_interrupts(dev_priv);
intel_uc_runtime_resume(&dev_priv->gt.uc);
intel_gt_init_swizzling(&dev_priv->gt);
i915_gem_restore_fences(dev_priv);
enable_rpm_wakeref_asserts(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
return ret;
}
drm/i915: make PC8 be part of runtime PM suspend/resume Currently, when our driver becomes idle for i915.pc8_timeout (default: 5s) we enable PC8, so we save some power, but not everything we can. Then, while PC8 is enabled, if we stay idle for more autosuspend_delay_ms (default: 10s) we'll enter runtime PM and put the graphics device in D3 state, saving even more power. The two features are separate things with increasing levels of power savings, but if we disable PC8 we'll never get into D3. While from the modularity point of view it would be nice to keep these features as separate, we have reasons to merge them: - We are not aware of anybody wanting a "PC8 without D3" environment. - If we keep both features as separate, we'll have to to test both PC8 and PC8+D3 code paths. We're already having a major pain to make QA do automated testing of just one thing, testing both paths will cost even more. - Only Haswell+ supports PC8, so if we want to add runtime PM support to, for example, IVB, we'll have to copy some code from the PC8 feature to runtime PM, so merging both features as a single thing will make it easier for enabling runtime PM on other platforms. This patch only does the very basic steps required to have PC8 and runtime PM merged on a single feature: the next patches will take care of cleaning up everything. v2: - Rebase. v3: - Rebase. - Fully remove the deprecated i915 params since Daniel doesn't consider them as part of the ABI. v4: - Rebase. - Fix typo in the commit message. v5: - Rebase, again. - Add a huge comment explaining the different forcewake usage (Chris, Daniel). - Use open-coded forcewake functions (Daniel). Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2014-03-08 03:08:05 +04:00
enable_rpm_wakeref_asserts(rpm);
intel_runtime_pm_driver_release(rpm);
if (intel_uncore_arm_unclaimed_mmio_detection(&dev_priv->uncore))
DRM_ERROR("Unclaimed access detected prior to suspending\n");
rpm->suspended = true;
/*
* FIXME: We really should find a document that references the arguments
* used below!
*/
if (IS_BROADWELL(dev_priv)) {
/*
* On Broadwell, if we use PCI_D1 the PCH DDI ports will stop
* being detected, and the call we do at intel_runtime_resume()
* won't be able to restore them. Since PCI_D3hot matches the
* actual specification and appears to be working, use it.
*/
intel_opregion_notify_adapter(dev_priv, PCI_D3hot);
} else {
/*
* current versions of firmware which depend on this opregion
* notification have repurposed the D1 definition to mean
* "runtime suspended" vs. what you would normally expect (D3)
* to distinguish it from notifications that might be sent via
* the suspend path.
*/
intel_opregion_notify_adapter(dev_priv, PCI_D1);
}
assert_forcewakes_inactive(&dev_priv->uncore);
if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv))
drm/i915: Enable polling when we don't have hpd Unfortunately, there's two situations where we lose hpd right now: - Runtime suspend - When we've shut off all of the power wells on Valleyview/Cherryview While it would be nice if this didn't cause issues, this has the ability to get us in some awkward states where a user won't be able to get their display to turn on. For instance; if we boot a Valleyview system without any monitors connected, it won't need any of it's power wells and thus shut them off. Since this causes us to lose HPD, this means that unless the user knows how to ssh into their machine and do a manual reprobe for monitors, none of the monitors they connect after booting will actually work. Eventually we should come up with a better fix then having to enable polling for this, since this makes rpm a lot less useful, but for now the infrastructure in i915 just isn't there yet to get hpd in these situations. Changes since v1: - Add comment explaining the addition of the if (!mode_config->poll_running) in intel_hpd_init() - Remove unneeded if (!dev->mode_config.poll_enabled) in i915_hpd_poll_init_work() - Call to drm_helper_hpd_irq_event() after we disable polling - Add cancel_work_sync() call to intel_hpd_cancel_work() Changes since v2: - Apparently dev->mode_config.poll_running doesn't actually reflect whether or not a poll is currently in progress, and is actually used for dynamic module paramter enabling/disabling. So now we instead keep track of our own poll_running variable in dev_priv->hotplug - Clean i915_hpd_poll_init_work() a little bit Changes since v3: - Remove the now-redundant connector loop in intel_hpd_init(), just rely on intel_hpd_poll_enable() for setting connector->polled correctly on each connector - Get rid of poll_running - Don't assign enabled in i915_hpd_poll_init_work before we actually lock dev->mode_config.mutex - Wrap enabled assignment in i915_hpd_poll_init_work() in READ_ONCE() for doc purposes - Do the same for dev_priv->hotplug.poll_enabled with WRITE_ONCE in intel_hpd_poll_enable() - Add some comments about racing not mattering in intel_hpd_poll_enable Changes since v4: - Rename intel_hpd_poll_enable() to intel_hpd_poll_init() - Drop the bool argument from intel_hpd_poll_init() - Remove redundant calls to intel_hpd_poll_init() - Rename poll_enable_work to poll_init_work - Add some kerneldoc for intel_hpd_poll_init() - Cross-reference intel_hpd_poll_init() in intel_hpd_init() - Just copy the loop from intel_hpd_init() in intel_hpd_poll_init() Changes since v5: - Minor kerneldoc nitpicks Cc: stable@vger.kernel.org Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Lyude <cpaul@redhat.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2016-06-22 00:03:44 +03:00
intel_hpd_poll_init(dev_priv);
drm/i915: make PC8 be part of runtime PM suspend/resume Currently, when our driver becomes idle for i915.pc8_timeout (default: 5s) we enable PC8, so we save some power, but not everything we can. Then, while PC8 is enabled, if we stay idle for more autosuspend_delay_ms (default: 10s) we'll enter runtime PM and put the graphics device in D3 state, saving even more power. The two features are separate things with increasing levels of power savings, but if we disable PC8 we'll never get into D3. While from the modularity point of view it would be nice to keep these features as separate, we have reasons to merge them: - We are not aware of anybody wanting a "PC8 without D3" environment. - If we keep both features as separate, we'll have to to test both PC8 and PC8+D3 code paths. We're already having a major pain to make QA do automated testing of just one thing, testing both paths will cost even more. - Only Haswell+ supports PC8, so if we want to add runtime PM support to, for example, IVB, we'll have to copy some code from the PC8 feature to runtime PM, so merging both features as a single thing will make it easier for enabling runtime PM on other platforms. This patch only does the very basic steps required to have PC8 and runtime PM merged on a single feature: the next patches will take care of cleaning up everything. v2: - Rebase. v3: - Rebase. - Fully remove the deprecated i915 params since Daniel doesn't consider them as part of the ABI. v4: - Rebase. - Fix typo in the commit message. v5: - Rebase, again. - Add a huge comment explaining the different forcewake usage (Chris, Daniel). - Use open-coded forcewake functions (Daniel). Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2014-03-08 03:08:05 +04:00
DRM_DEBUG_KMS("Device suspended\n");
return 0;
}
static int intel_runtime_resume(struct device *kdev)
{
struct drm_device *dev = dev_get_drvdata(kdev);
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_runtime_pm *rpm = &dev_priv->runtime_pm;
int ret = 0;
if (WARN_ON_ONCE(!HAS_RUNTIME_PM(dev_priv)))
return -ENODEV;
DRM_DEBUG_KMS("Resuming device\n");
WARN_ON_ONCE(atomic_read(&rpm->wakeref_count));
disable_rpm_wakeref_asserts(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
intel_opregion_notify_adapter(dev_priv, PCI_D0);
rpm->suspended = false;
if (intel_uncore_unclaimed_mmio(&dev_priv->uncore))
DRM_DEBUG_DRIVER("Unclaimed access during suspend, bios?\n");
drm/i915/icl: Enable DC9 as lowest possible state during screen-off ICL supports DC5, DC6, and DC9. Enable DC9 during screen-off, and enable DC5/6 when appropriate. v2: (James Ausmus) - Also handle ICL as GEN9_LP in i915_drm_suspend_late and i915_drm_suspend_early - Add DC9 to gen9_dc_mask for ICL - Re-order GEN checks for newest platform first - Use INTEL_GEN instead of INTEL_INFO->gen - Use INTEL_GEN >= 11 instead of IS_ICELAKE - Consolidate GEN checks v3: (James Ausmus) - Also allow DC6 for ICL (Imre, Art) - Simplify !(GEN >= 11) to GEN < 11 (Imre) v4: (James Ausmus) - Don't call intel_power_sequencer_reset after DC9 for Gen11+, as the PPS regs are Always On - Rebase against upstream changes v5: (Anusha Srivatsa) - rebased against the latest upstream changes. v6: (Anusha Srivatsa) - rebased.Use INTEL_GEN consistently. - Simplify the code (Rodrigo) v7: rebased. Change order according to platforms(Jyoti) v8: rebased. Change the check from platform specific to HAS_PCH_SPLIT(). Add comment in code to be more clear.(Rodrigo) Cc: Imre Deak <imre.deak@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Animesh Manna <animesh.manna@intel.com> Signed-off-by: James Ausmus <james.ausmus@intel.com> Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com> Tested-by: Jyoti Yadav <jyoti.r.yadav@intel.com> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20181029221410.4423-1-anusha.srivatsa@intel.com
2018-10-30 01:14:10 +03:00
if (INTEL_GEN(dev_priv) >= 11) {
bxt_disable_dc9(dev_priv);
icl_display_core_init(dev_priv, true);
if (dev_priv->csr.dmc_payload) {
if (dev_priv->csr.allowed_dc_mask &
DC_STATE_EN_UPTO_DC6)
skl_enable_dc6(dev_priv);
else if (dev_priv->csr.allowed_dc_mask &
DC_STATE_EN_UPTO_DC5)
gen9_enable_dc5(dev_priv);
}
} else if (IS_GEN9_LP(dev_priv)) {
bxt_disable_dc9(dev_priv);
bxt_display_core_init(dev_priv, true);
if (dev_priv->csr.dmc_payload &&
(dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC5))
gen9_enable_dc5(dev_priv);
} else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
hsw_disable_pc8(dev_priv);
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
ret = vlv_resume_prepare(dev_priv, true);
}
intel_uncore_runtime_resume(&dev_priv->uncore);
intel_runtime_pm_enable_interrupts(dev_priv);
intel_uc_runtime_resume(&dev_priv->gt.uc);
/*
* No point of rolling back things in case of an error, as the best
* we can do is to hope that things will still work (and disable RPM).
*/
intel_gt_init_swizzling(&dev_priv->gt);
drm/i915: Remove overzealous fence warn on runtime suspend The goal of the WARN was to catch when we are still actively using the fence as we go into the runtime suspend. However, the reg->pin_count is too coarse as it does not distinguish between exclusive ownership of the fence register from activity. I've not improved on the WARN, nor have we captured this WARN in an exact igt, but it is showing up regularly in the wild: [ 1915.935332] WARNING: CPU: 1 PID: 10861 at drivers/gpu/drm/i915/i915_gem.c:2022 i915_gem_runtime_suspend+0x116/0x130 [i915] [ 1915.935383] WARN_ON(reg->pin_count)[ 1915.935399] Modules linked in: snd_hda_intel i915 drm_kms_helper vgem netconsole scsi_transport_iscsi fuse vfat fat x86_pkg_temp_thermal coretemp intel_cstate intel_uncore snd_hda_codec_hdmi snd_hda_codec_generic snd_hda_codec snd_hwdep snd_hda_core snd_pcm snd_timer snd mei_me mei serio_raw intel_rapl_perf intel_pch_thermal soundcore wmi acpi_pad i2c_algo_bit syscopyarea sysfillrect sysimgblt fb_sys_fops drm r8169 mii video [last unloaded: drm_kms_helper] [ 1915.935785] CPU: 1 PID: 10861 Comm: kworker/1:0 Tainted: G U W 4.9.0-rc5+ #170 [ 1915.935799] Hardware name: LENOVO 80MX/Lenovo E31-80, BIOS DCCN34WW(V2.03) 12/01/2015 [ 1915.935822] Workqueue: pm pm_runtime_work [ 1915.935845] ffffc900044fbbf0 ffffffffac3220bc ffffc900044fbc40 0000000000000000 [ 1915.935890] ffffc900044fbc30 ffffffffac059bcb 000007e6044fbc60 ffff8801626e3198 [ 1915.935937] ffff8801626e0000 0000000000000002 ffffffffc05e5d4e 0000000000000000 [ 1915.935985] Call Trace: [ 1915.936013] [<ffffffffac3220bc>] dump_stack+0x4f/0x73 [ 1915.936038] [<ffffffffac059bcb>] __warn+0xcb/0xf0 [ 1915.936060] [<ffffffffac059c4f>] warn_slowpath_fmt+0x5f/0x80 [ 1915.936158] [<ffffffffc052d916>] i915_gem_runtime_suspend+0x116/0x130 [i915] [ 1915.936251] [<ffffffffc04f1c74>] intel_runtime_suspend+0x64/0x280 [i915] [ 1915.936277] [<ffffffffac0926f1>] ? dequeue_entity+0x241/0xbc0 [ 1915.936298] [<ffffffffac36bb85>] pci_pm_runtime_suspend+0x55/0x180 [ 1915.936317] [<ffffffffac36bb30>] ? pci_pm_runtime_resume+0xa0/0xa0 [ 1915.936339] [<ffffffffac4514e2>] __rpm_callback+0x32/0x70 [ 1915.936356] [<ffffffffac451544>] rpm_callback+0x24/0x80 [ 1915.936375] [<ffffffffac36bb30>] ? pci_pm_runtime_resume+0xa0/0xa0 [ 1915.936392] [<ffffffffac45222d>] rpm_suspend+0x12d/0x680 [ 1915.936415] [<ffffffffac69f6d7>] ? _raw_spin_unlock_irq+0x17/0x30 [ 1915.936435] [<ffffffffac0810b8>] ? finish_task_switch+0x88/0x220 [ 1915.936455] [<ffffffffac4534bf>] pm_runtime_work+0x6f/0xb0 [ 1915.936477] [<ffffffffac074353>] process_one_work+0x1f3/0x4d0 [ 1915.936501] [<ffffffffac074678>] worker_thread+0x48/0x4e0 [ 1915.936523] [<ffffffffac074630>] ? process_one_work+0x4d0/0x4d0 [ 1915.936542] [<ffffffffac074630>] ? process_one_work+0x4d0/0x4d0 [ 1915.936559] [<ffffffffac07a2c9>] kthread+0xd9/0xf0 [ 1915.936580] [<ffffffffac07a1f0>] ? kthread_park+0x60/0x60 [ 1915.936600] [<ffffffffac69fe62>] ret_from_fork+0x22/0x30 In the case the register is pinned, it should be present and we will need to invalidate them to be restored upon resume as we cannot expect the owner of the pin to call get_fence prior to use after resume. Fixes: 7c108fd8feac ("drm/i915: Move fence cancellation to runtime suspend") Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98804 Reported-by: Lionel Landwerlin <lionel.g.landwerlin@linux.intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Imre Deak <imre.deak@linux.intel.com> Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: <drm-intel-fixes@lists.freedesktop.org> # v4.10-rc1+ Link: http://patchwork.freedesktop.org/patch/msgid/20170203125717.8431-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> (cherry picked from commit e0ec3ec698851a6c97a12d696407b3ff77700c23) Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2017-02-03 15:57:17 +03:00
i915_gem_restore_fences(dev_priv);
/*
* On VLV/CHV display interrupts are part of the display
* power well, so hpd is reinitialized from there. For
* everyone else do it here.
*/
if (!IS_VALLEYVIEW(dev_priv) && !IS_CHERRYVIEW(dev_priv))
intel_hpd_init(dev_priv);
2017-08-17 16:45:28 +03:00
intel_enable_ipc(dev_priv);
enable_rpm_wakeref_asserts(rpm);
drm/i915: add support for checking if we hold an RPM reference Atm, we assert that the device is not suspended until the point when the device is truly put to a suspended state. This is fine, but we can catch more problems if we check that RPM refcount is non-zero. After that one drops to zero we shouldn't access the device any more, even if the actual device suspend may be delayed. Change assert_rpm_wakelock_held() accordingly to check for a non-zero RPM refcount in addition to the current device-not-suspended check. For the new asserts to work we need to annotate every place explicitly in the code where we expect that the device is powered. The places where we only assume this, but may not hold an RPM reference: - driver load We assume the device to be powered until we enable RPM. Make this explicit by taking an RPM reference around the load function. - system and runtime sudpend/resume handlers These handlers are called when the RPM reference becomes 0 and know the exact point after which the device can get powered off. Disable the RPM-reference-held check for their duration. - the IRQ, hangcheck and RPS work handlers These handlers are flushed in the system/runtime suspend handler before the device is powered off, so it's guaranteed that they won't run while the device is powered off even though they don't hold any RPM reference. Disable the RPM-reference-held check for their duration. In all these cases we still check that the device is not suspended. These explicit annotations also have the positive side effect of documenting our assumptions better. This caught additional WARNs from the atomic modeset path, those should be fixed separately. v2: - remove the redundant HAS_RUNTIME_PM check (moved to patch 1) (Ville) v3: - use a new dedicated RPM wakelock refcount to also catch cases where our own RPM get/put functions were not called (Chris) - assert also that the new RPM wakelock refcount is 0 in the RPM suspend handler (Chris) - change the assert error message to be more meaningful (Chris) - prevent false assert errors and check that the RPM wakelock is 0 in the RPM resume handler too - prevent false assert errors in the hangcheck work too - add a device not suspended assert check to the hangcheck work v4: - rename disable/enable_rpm_asserts to disable/enable_rpm_wakeref_asserts and wakelock_count to wakeref_count - disable the wakeref asserts in the IRQ handlers and RPS work too - update/clarify commit message v5: - mark places we plan to change to use proper RPM refcounting with separate DISABLE/ENABLE_RPM_WAKEREF_ASSERTS aliases (Chris) Signed-off-by: Imre Deak <imre.deak@intel.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/1450227139-13471-1-git-send-email-imre.deak@intel.com
2015-12-16 03:52:19 +03:00
if (ret)
DRM_ERROR("Runtime resume failed, disabling it (%d)\n", ret);
else
DRM_DEBUG_KMS("Device resumed\n");
return ret;
}
const struct dev_pm_ops i915_pm_ops = {
/*
* S0ix (via system suspend) and S3 event handlers [PMSG_SUSPEND,
* PMSG_RESUME]
*/
.prepare = i915_pm_prepare,
.suspend = i915_pm_suspend,
.suspend_late = i915_pm_suspend_late,
.resume_early = i915_pm_resume_early,
.resume = i915_pm_resume,
/*
* S4 event handlers
* @freeze, @freeze_late : called (1) before creating the
* hibernation image [PMSG_FREEZE] and
* (2) after rebooting, before restoring
* the image [PMSG_QUIESCE]
* @thaw, @thaw_early : called (1) after creating the hibernation
* image, before writing it [PMSG_THAW]
* and (2) after failing to create or
* restore the image [PMSG_RECOVER]
* @poweroff, @poweroff_late: called after writing the hibernation
* image, before rebooting [PMSG_HIBERNATE]
* @restore, @restore_early : called after rebooting and restoring the
* hibernation image [PMSG_RESTORE]
*/
.freeze = i915_pm_freeze,
.freeze_late = i915_pm_freeze_late,
.thaw_early = i915_pm_thaw_early,
.thaw = i915_pm_thaw,
.poweroff = i915_pm_suspend,
2015-03-02 14:04:41 +03:00
.poweroff_late = i915_pm_poweroff_late,
.restore_early = i915_pm_restore_early,
.restore = i915_pm_restore,
/* S0ix (via runtime suspend) event handlers */
.runtime_suspend = intel_runtime_suspend,
.runtime_resume = intel_runtime_resume,
};
static const struct vm_operations_struct i915_gem_vm_ops = {
.fault = i915_gem_fault,
.open = drm_gem_vm_open,
.close = drm_gem_vm_close,
};
static const struct file_operations i915_driver_fops = {
.owner = THIS_MODULE,
.open = drm_open,
.release = drm_release,
.unlocked_ioctl = drm_ioctl,
.mmap = drm_gem_mmap,
.poll = drm_poll,
.read = drm_read,
.compat_ioctl = i915_compat_ioctl,
.llseek = noop_llseek,
};
static int
i915_gem_reject_pin_ioctl(struct drm_device *dev, void *data,
struct drm_file *file)
{
return -ENODEV;
}
static const struct drm_ioctl_desc i915_ioctls[] = {
DRM_IOCTL_DEF_DRV(I915_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_FLUSH, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_FLIP, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_BATCHBUFFER, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_IRQ_EMIT, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_IRQ_WAIT, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_GETPARAM, i915_getparam_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_SETPARAM, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_ALLOC, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_FREE, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_INIT_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_CMDBUFFER, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_DESTROY_HEAP, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_SET_VBLANK_PIPE, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GET_VBLANK_PIPE, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_VBLANK_SWAP, drm_noop, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_HWS_ADDR, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_INIT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER, i915_gem_execbuffer_ioctl, DRM_AUTH),
DRM_IOCTL_DEF_DRV(I915_GEM_EXECBUFFER2_WR, i915_gem_execbuffer2_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_PIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_UNPIN, i915_gem_reject_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SET_CACHING, i915_gem_set_caching_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_GET_CACHING, i915_gem_get_caching_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_ENTERVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_LEAVEVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
DRM_IOCTL_DEF_DRV(I915_GEM_CREATE, i915_gem_create_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_PREAD, i915_gem_pread_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_PWRITE, i915_gem_pwrite_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_MMAP, i915_gem_mmap_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_MMAP_GTT, i915_gem_mmap_gtt_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SET_TILING, i915_gem_set_tiling_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_GET_TILING, i915_gem_get_tiling_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id_ioctl, 0),
DRM_IOCTL_DEF_DRV(I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image_ioctl, DRM_MASTER),
DRM_IOCTL_DEF_DRV(I915_OVERLAY_ATTRS, intel_overlay_attrs_ioctl, DRM_MASTER),
DRM_IOCTL_DEF_DRV(I915_SET_SPRITE_COLORKEY, intel_sprite_set_colorkey_ioctl, DRM_MASTER),
DRM_IOCTL_DEF_DRV(I915_GET_SPRITE_COLORKEY, drm_noop, DRM_MASTER),
DRM_IOCTL_DEF_DRV(I915_GEM_WAIT, i915_gem_wait_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_CREATE_EXT, i915_gem_context_create_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_DESTROY, i915_gem_context_destroy_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_REG_READ, i915_reg_read_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GET_RESET_STATS, i915_gem_context_reset_stats_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_USERPTR, i915_gem_userptr_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_GETPARAM, i915_gem_context_getparam_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_SETPARAM, i915_gem_context_setparam_ioctl, DRM_RENDER_ALLOW),
drm/i915: Add i915 perf infrastructure Adds base i915 perf infrastructure for Gen performance metrics. This adds a DRM_IOCTL_I915_PERF_OPEN ioctl that takes an array of uint64 properties to configure a stream of metrics and returns a new fd usable with standard VFS system calls including read() to read typed and sized records; ioctl() to enable or disable capture and poll() to wait for data. A stream is opened something like: uint64_t properties[] = { /* Single context sampling */ DRM_I915_PERF_PROP_CTX_HANDLE, ctx_handle, /* Include OA reports in samples */ DRM_I915_PERF_PROP_SAMPLE_OA, true, /* OA unit configuration */ DRM_I915_PERF_PROP_OA_METRICS_SET, metrics_set_id, DRM_I915_PERF_PROP_OA_FORMAT, report_format, DRM_I915_PERF_PROP_OA_EXPONENT, period_exponent, }; struct drm_i915_perf_open_param parm = { .flags = I915_PERF_FLAG_FD_CLOEXEC | I915_PERF_FLAG_FD_NONBLOCK | I915_PERF_FLAG_DISABLED, .properties_ptr = (uint64_t)properties, .num_properties = sizeof(properties) / 16, }; int fd = drmIoctl(drm_fd, DRM_IOCTL_I915_PERF_OPEN, &param); Records read all start with a common { type, size } header with DRM_I915_PERF_RECORD_SAMPLE being of most interest. Sample records contain an extensible number of fields and it's the DRM_I915_PERF_PROP_SAMPLE_xyz properties given when opening that determine what's included in every sample. No specific streams are supported yet so any attempt to open a stream will return an error. v2: use i915_gem_context_get() - Chris Wilson v3: update read() interface to avoid passing state struct - Chris Wilson fix some rebase fallout, with i915-perf init/deinit v4: s/DRM_IORW/DRM_IOW/ - Emil Velikov Signed-off-by: Robert Bragg <robert@sixbynine.org> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Sourab Gupta <sourab.gupta@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: http://patchwork.freedesktop.org/patch/msgid/20161107194957.3385-2-robert@sixbynine.org
2016-11-07 22:49:47 +03:00
DRM_IOCTL_DEF_DRV(I915_PERF_OPEN, i915_perf_open_ioctl, DRM_RENDER_ALLOW),
drm/i915/perf: Implement I915_PERF_ADD/REMOVE_CONFIG interface The motivation behind this new interface is expose at runtime the creation of new OA configs which can be used as part of the i915 perf open interface. This will enable the kernel to learn new configs which may be experimental, or otherwise not part of the core set currently available through the i915 perf interface. v2: Drop DRM_ERROR for userspace errors (Matthew) Add padding to userspace structure (Matthew) s/guid/uuid/ (Matthew) v3: Use u32 instead of int to iterate through registers (Matthew) v4: Lock access to dynamic config list (Lionel) v5: by Matthew: Fix uninitialized error values Fix incorrect unwiding when opening perf stream Use kmalloc_array() to store register Use uuid_is_valid() to valid config uuids Declare ioctls as write only Check padding members are set to 0 by Lionel: Return ENOENT rather than EINVAL when trying to remove non existing config v6: by Chris: Use ref counts for OA configs Store UUID in drm_i915_perf_oa_config rather then using pointer Shuffle fields of drm_i915_perf_oa_config to avoid padding v7: by Chris Rename uapi pointers fields to end with '_ptr' v8: by Andrzej, Marek, Sebastian Update register whitelisting by Lionel Add more register names for documentation Allow configuration programming in non-paranoid mode Add support for value filter for a couple of registers already programmed in other part of the kernel v9: Documentation fix (Lionel) Allow writing WAIT_FOR_RC6_EXIT only on Gen8+ (Andrzej) v10: Perform read access_ok() on register pointers (Lionel) Signed-off-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Signed-off-by: Andrzej Datczuk <andrzej.datczuk@intel.com> Reviewed-by: Andrzej Datczuk <andrzej.datczuk@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20170803165812.2373-2-lionel.g.landwerlin@intel.com
2017-08-03 20:05:50 +03:00
DRM_IOCTL_DEF_DRV(I915_PERF_ADD_CONFIG, i915_perf_add_config_ioctl, DRM_UNLOCKED|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_PERF_REMOVE_CONFIG, i915_perf_remove_config_ioctl, DRM_UNLOCKED|DRM_RENDER_ALLOW),
drm/i915: add query uAPI There are a number of information that are readable from hardware registers and that we would like to make accessible to userspace. One particular example is the topology of the execution units (how are execution units grouped in subslices and slices and also which ones have been fused off for die recovery). At the moment the GET_PARAM ioctl covers some basic needs, but generally is only able to return a single value for each defined parameter. This is a bit problematic with topology descriptions which are array/maps of available units. This change introduces a new ioctl that can deal with requests to fill structures of potentially variable lengths. The user is expected fill a query with length fields set at 0 on the first call, the kernel then sets the length fields to the their expected values. A second call to the kernel with length fields at their expected values will trigger a copy of the data to the pointed memory locations. The scope of this uAPI is only to provide information to userspace, not to allow configuration of the device. v2: Simplify dispatcher code iteration (Tvrtko) Tweak uapi drm_i915_query_item structure (Tvrtko) v3: Rename pad fields into flags (Chris) Return error on flags field != 0 (Chris) Only copy length back to userspace in drm_i915_query_item (Chris) v4: Use array of functions instead of switch (Chris) v5: More comments in uapi (Tvrtko) Return query item errors in length field (All) v6: Tweak uapi comments style to match the coding style (Lionel) v7: Add i915_query.h (Joonas) v8: (Lionel) Change the behavior of the item iterator to report invalid queries into the query item rather than stopping the iteration. This enables userspace applications to query newer items on older kernels and only have failure on the items that are not supported. v9: Edit copyright headers (Joonas) v10: Typos & comments in uapi (Joonas) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Acked-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180306122857.27317-6-lionel.g.landwerlin@intel.com
2018-03-06 15:28:56 +03:00
DRM_IOCTL_DEF_DRV(I915_QUERY, i915_query_ioctl, DRM_UNLOCKED|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_VM_CREATE, i915_gem_vm_create_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_VM_DESTROY, i915_gem_vm_destroy_ioctl, DRM_RENDER_ALLOW),
};
static struct drm_driver driver = {
/* Don't use MTRRs here; the Xserver or userspace app should
* deal with them for Intel hardware.
*/
.driver_features =
DRIVER_GEM | DRIVER_PRIME |
DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC | DRIVER_SYNCOBJ,
.release = i915_driver_release,
.open = i915_driver_open,
.lastclose = i915_driver_lastclose,
.postclose = i915_driver_postclose,
.gem_close_object = i915_gem_close_object,
.gem_free_object_unlocked = i915_gem_free_object,
.gem_vm_ops = &i915_gem_vm_ops,
i915: add dmabuf/prime buffer sharing support. This adds handle->fd and fd->handle support to i915, this is to allow for offloading of rendering in one direction and outputs in the other. v2 from Daniel Vetter: - fixup conflicts with the prepare/finish gtt prep work. - implement ppgtt binding support. Note that we have squat i-g-t testcoverage for any of the lifetime and access rules dma_buf/prime support brings along. And there are quite a few intricate situations here. Also note that the integration with the existing code is a bit hackish, especially around get_gtt_pages and put_gtt_pages. It imo would be easier with the prep code from Chris Wilson's unbound series, but that is for 3.6. Also note that I didn't bother to put the new prepare/finish gtt hooks to good use by moving the dma_buf_map/unmap_attachment calls in there (like we've originally planned for). Last but not least this patch is only compile-tested, but I've changed very little compared to Dave Airlie's version. So there's a decent chance v2 on drm-next works as well as v1 on 3.4-rc. v3: Right when I've hit sent I've noticed that I've screwed up one obj->sg_list (for dmar support) and obj->sg_table (for prime support) disdinction. We should be able to merge these 2 paths, but that's material for another patch. v4: fix the error reporting bugs pointed out by ickle. v5: fix another error, and stop non-gtt mmaps on shared objects stop pread/pwrite on imported objects, add fake kmap Signed-off-by: Dave Airlie <airlied@redhat.com> Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2012-05-10 17:25:09 +04:00
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_prime_export = i915_gem_prime_export,
.gem_prime_import = i915_gem_prime_import,
.get_vblank_timestamp = drm_calc_vbltimestamp_from_scanoutpos,
.get_scanout_position = i915_get_crtc_scanoutpos,
.dumb_create = i915_gem_dumb_create,
.dumb_map_offset = i915_gem_mmap_gtt,
.ioctls = i915_ioctls,
.num_ioctls = ARRAY_SIZE(i915_ioctls),
.fops = &i915_driver_fops,
.name = DRIVER_NAME,
.desc = DRIVER_DESC,
.date = DRIVER_DATE,
.major = DRIVER_MAJOR,
.minor = DRIVER_MINOR,
.patchlevel = DRIVER_PATCHLEVEL,
};
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/mock_drm.c"
#endif