Version three of the ARM SMMU architecture introduces significant
changes and improvements over previous versions of the specification,
necessitating a new driver in the Linux kernel.
The main change to the programming interface is that the majority of the
configuration data has been moved from MMIO registers to in-memory data
structures, with communication between the CPU and the SMMU being
mediated via in-memory circular queues.
This patch adds an initial driver for SMMUv3 to Linux. We currently
support pinned stage-1 (DMA) and stage-2 (KVM VFIO) mappings using the
generic IO-pgtable code.
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
A number of IOMMUs found in ARM SoCs can walk architecture-compatible
page tables.
This patch adds a generic allocator for Stage-1 and Stage-2 v7/v8
long-descriptor page tables. 4k, 16k and 64k pages are supported, with
up to 4-levels of walk to cover a 48-bit address space.
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
This patch introduces a generic framework for allocating page tables for
an IOMMU. There are a number of reasons we want to do this:
- It avoids duplication of complex table management code in IOMMU
drivers that use the same page table format
- It removes any coupling with the CPU table format (and even the
architecture!)
- It defines an API for IOMMU TLB maintenance
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
In preparation for sharing the IOVA allocator, split it out under its
own Kconfig symbol.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
The rk3288 has several iommus. Each iommu belongs to a single master
device. There is one device (ISP) that has two slave iommus, but that
case is not yet supported by this driver.
At subsys init, the iommu driver registers itself as the iommu driver for
the platform bus. The master devices find their slave iommus using the
"iommus" field in their devicetree description. Since each slave iommu
belongs to exactly one master, their is no additional data needed at probe
to associate a slave with its master.
An iommu device's power domain, clock and irq are all shared with its
master device, and the master device must be careful to attach from the
iommu only after powering and clocking it (and leave it powered and
clocked before detaching). Because their is no guarantee what the status
of the iommu is at probe, and since the driver does not even know if the
device is powered, we delay requesting its irq until the master device
attaches, at which point we have a guarantee that the device is powered
and clocked and we can reset it and disable its interrupt mask.
An iommu_domain describes a virtual iova address space. Each iommu_domain
has a corresponding page table that lists the mappings from iova to
physical address.
For the rk3288 iommu, the page table has two levels:
The Level 1 "directory_table" has 1024 4-byte dte entries.
Each dte points to a level 2 "page_table".
Each level 2 page_table has 1024 4-byte pte entries.
Each pte points to a 4 KiB page of memory.
An iommu_domain is created when a dma_iommu_mapping is created via
arm_iommu_create_mapping. Master devices can then attach themselves to
this mapping (or attach the mapping to themselves?) by calling
arm_iommu_attach_device(). This in turn instructs the iommu driver to
write the page table's physical address into the slave iommu's "Directory
Table Entry" (DTE) register.
In fact multiple master devices, each with their own slave iommu device,
can all attach to the same mapping. The iommus for these devices will
share the same iommu_domain and therefore point to the same page table.
Thus, the iommu domain maintains a list of iommu devices which are
attached. This driver relies on the iommu core to ensure that all devices
have detached before destroying a domain.
v6: - add .add/remove_device() callbacks.
- parse platform_device device tree nodes for "iommus" property
- store platform device pointer as group iommudata
- Check for existence of iommu group instead of relying on a
dev_get_drvdata() to return NULL for a NULL device.
v7: - fixup some strings.
- In rk_iommu_disable_paging() # and % were reversed.
Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>
Signed-off-by: Simon Xue <xxm@rock-chips.com>
Reviewed-by: Grant Grundler <grundler@chromium.org>
Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
The OMAP IOMMU driver was originally designed as modules, and split
into a core module and a thin arch-specific module through the OMAP
arch-specific struct iommu_functions, to scale for both OMAP1 and
OMAP2+ IOMMU variants. The driver can only be built for OMAP2+
platforms currently, and also can only be built-in after the
adaptation to generic IOMMU API. The OMAP1 variant was never added
and will most probably be never added (the code for the only potential
user, its parent, DSP processor has already been cleaned up). So,
consolidate the OMAP2 specific omap-iommu2 module into the core OMAP
IOMMU driver - this eliminates the arch-specific ops structure and
simplifies the driver into a single module that only implements the
generic IOMMU API's iommu_ops.
The following are the main changes:
- omap-iommu2 module is completely eliminated, with the common
definitions moved to the internal omap-iommu.h, and the ops
implementations moved into omap-iommu.c
- OMAP arch-specific struct iommu_functions is also eliminated,
with the ops implementations directly absorbed into the calling
functions
- iotlb_alloc_cr() is no longer inlined and defined only when
PREFETCH_IOTLB is defined
- iotlb_dump_cr() is similarly defined only when CONFIG_OMAP_IOMMU_DEBUG
is defined
- Elimination of the OMAP IOMMU exported functions to register the
arch ops, omap_install_iommu_arch() & omap_uninstall_iommu_arch()
- Any stale comments about OMAP1 are also cleaned up
Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
The OMAP3 ISP driver was the only user of the OMAP IOVMM API. Now that
is has been ported to the DMA API, remove the unused virtual memory
manager.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
IOMMUs currently have no common representation to userspace, most
seem to have no representation at all aside from a few printks
on bootup. There are however features of IOMMUs that are useful
to know about. For instance the IOMMU might support superpages,
making use of processor large/huge pages more important in a device
assignment scenario. It's also useful to create cross links between
devices and IOMMU hardware units, so that users might be able to
load balance their devices to avoid thrashing a single hardware unit.
This patch adds a device create and destroy interface as well as
device linking, making it very lightweight for an IOMMU driver to add
basic support. IOMMU drivers can provide additional attributes
automatically by using an attribute_group.
The attributes exposed are expected to be relatively device specific,
the means to retrieve them certainly are, so there are currently no
common attributes for the new class created here.
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Add tracing feature to iommu to report various iommu events. Classes
iommu_group, iommu_device, and iommu_map_unmap are defined.
iommu_group class events can be enabled to trigger when devices get added
to and removed from an iommu group. Trace information includes iommu group
id and device name.
iommu:add_device_to_group
iommu:remove_device_from_group
iommu_device class events can be enabled to trigger when devices are attached
to and detached from a domain. Trace information includes device name.
iommu:attach_device_to_domain
iommu:detach_device_from_domain
iommu_map_unmap class events can be enabled to trigger when iommu map and
unmap iommu ops. Trace information includes iova, physical address (map event
only), and size.
iommu:map
iommu:unmap
Signed-off-by: Shuah Khan <shuah.kh@samsung.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
Following is a brief description of the PAMU hardware:
PAMU determines what action to take and whether to authorize the action on
the basis of the memory address, a Logical IO Device Number (LIODN), and
PAACT table (logically) indexed by LIODN and address. Hardware devices which
need to access memory must provide an LIODN in addition to the memory address.
Peripheral Access Authorization and Control Tables (PAACTs) are the primary
data structures used by PAMU. A PAACT is a table of peripheral access
authorization and control entries (PAACE).Each PAACE defines the range of
I/O bus address space that is accessible by the LIOD and the associated access
capabilities.
There are two types of PAACTs: primary PAACT (PPAACT) and secondary PAACT
(SPAACT).A given physical I/O device may be able to act as one or more
independent logical I/O devices (LIODs). Each such logical I/O device is
assigned an identifier called logical I/O device number (LIODN). A LIODN is
allocated a contiguous portion of the I/O bus address space called the DSA window
for performing DSA operations. The DSA window may optionally be divided into
multiple sub-windows, each of which may be used to map to a region in system
storage space. The first sub-window is referred to as the primary sub-window
and the remaining are called secondary sub-windows.
This patch provides the PAMU driver (fsl_pamu.c) and the corresponding IOMMU
API implementation (fsl_pamu_domain.c). The PAMU hardware driver (fsl_pamu.c)
has been derived from the work done by Ashish Kalra and Timur Tabi.
[For iommu group support]
Acked-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Timur Tabi <timur@tabi.org>
Signed-off-by: Varun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
This patch adds support for SMMUs implementing the ARM System MMU
architecture versions 1 or 2. Both arm and arm64 are supported, although
the v7s descriptor format is not used.
Cc: Rob Herring <robherring2@gmail.com>
Cc: Andreas Herrmann <andreas.herrmann@calxeda.com>
Cc: Olav Haugan <ohaugan@codeaurora.org>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Andreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
This is the Renesas IPMMU driver and IOMMU API implementation.
The IPMMU module supports the MMU function and the PMB function. The
MMU function provides address translation by pagetable compatible with
ARMv6. The PMB function provides address translation including
tile-linear translation. This patch implements the MMU function.
The iommu driver does not register a platform driver directly because:
- the register space of the MMU function and the PMB function
have a common register (used for settings flush), so they should ideally
have a way to appropriately share this register.
- the MMU function uses the IOMMU API while the PMB function does not.
- the two functions may be used independently.
Signed-off-by: Hideki EIRAKU <hdk@igel.co.jp>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
This file should not be in arch/arm. Move it to drivers/iommu
to allow making most of the header local to drivers/iommu.
This is needed as we are removing plat and mach includes
from drivers for ARM common zImage support.
Cc: Ido Yariv <ido@wizery.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: linux-media@vger.kernel.org
Acked-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>
This code was based on:
"arch/microblaze/kernel/prom_parse.c"
"arch/powerpc/kernel/prom_parse.c"
Can replace "of_parse_dma_window()" in the above. This supports
different formats flexibly. "prefix" can be configured if any. "busno"
and "index" are optionally specified. Set NULL and 0 if not used.
Signed-off-by: Hiroshi DOYU <hdoyu@nvidia.com>
Acked-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
These changes are specific to some driver that may be used by multiple
boards or socs. The most significant change in here is the move of the
samsung iommu code from a platform specific in-kernel interface to the
generic iommu subsystem.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAABAgAGBQJPuevXAAoJEIwa5zzehBx3D8YQAKfoY3TFjJ9KmJgk69/pc7cf
Of0rvX+35NezGFljIyRdspz+DsV+vhJravwdVxOagRKvVBOb9qnZIXnl3gkLnTw4
dCVMFMqCwXhBeXlXCzHSeRmt2+4/fmJnr7jr4xh9omOAJ9BZv5ftmKNW4zP5wr1L
+9HBwzkxlVisR4NCz2q66YBjsV2dXA3dv2hZxEFkUdQFYJGqZoUXLYHF9yno20i3
knKNXEyAFYFUKHiFVBQJ1tYGmZlaIjw14g+GTqzZay2Pi/HjUXfrPd0VwNkBzZf/
H1N3/cf4GJ2+K/zYqh+H/Xjf/Fjkp1dFNhlUQ7+l5Jwbu7C1B2euvwTO4OaqrfdD
7eqG3+uIKhPO2Z8ZySLFgx4ghybtwgZrAwOjsa+ymTugqPbiWYB/zZR1iWu5DMk/
TnNb6P3ciP+WMpoMFh1kXRc/eCCCHtuQ0rLRxizSX6HIpxWvjYFNLH7L3wS+KtlB
7vsS764d1JFW318bsdBi+V/LWRVXeSWWetTzdzDcM/Syz3ZqfPy7e3Ge6qx0lvYe
5ojgzKwVqpJenZdt91UC16cMXNqDTzmZObz6LOCmVm1mB5kYSgEHYxAQQvuGFjXT
28kGyBQNsBboJGaYh2O/CTsVXnHnaPXrtDDWMDacWNwwPYnnA2L8lUNfAg1DgA1j
Z6CO8Knfct01EpQLtybK
=hgkg
-----END PGP SIGNATURE-----
Merge tag 'drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
Pull arm-soc driver specific updates from Olof Johansson:
"These changes are specific to some driver that may be used by multiple
boards or socs. The most significant change in here is the move of
the samsung iommu code from a platform specific in-kernel interface to
the generic iommu subsystem."
Fix up trivial conflicts in arch/arm/mach-exynos/Kconfig
* tag 'drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (28 commits)
mmc: dt: Consolidate DT bindings
iommu/exynos: Add iommu driver for EXYNOS Platforms
ARM: davinci: optimize the DMA ISR
ARM: davinci: implement DEBUG_LL port choice
ARM: tegra: Add SMMU enabler in AHB
ARM: tegra: Add Tegra AHB driver
Input: pxa27x_keypad add choice to set direct_key_mask
Input: pxa27x_keypad direct key may be low active
Input: pxa27x_keypad bug fix for direct_key_mask
Input: pxa27x_keypad keep clock on as wakeup source
ARM: dt: tegra: pinmux changes for USB ULPI
ARM: tegra: add USB ULPI PHY reset GPIO to device tree
ARM: tegra: don't hard-code USB ULPI PHY reset_gpio
ARM: tegra: change pll_p_out4's rate to 24MHz
ARM: tegra: fix pclk rate
ARM: tegra: reparent sclk to pll_c_out1
ARM: tegra: Add pllc clock init table
ARM: dt: tegra cardhu: basic audio support
ARM: dt: tegra30.dtsi: Add audio-related nodes
ARM: tegra: add AUXDATA required for audio
...
This is the System MMU driver and IOMMU API implementation for
EXYNOS SoC platforms. EXYNOS platforms has more than 10 System
MMUs dedicated for each multimedia accelerators.
The System MMU driver is already in arc/arm/plat-s5p but it is
moved to drivers/iommu due to Ohad Ben-Cohen gathered IOMMU
drivers there.
Any device driver in EXYNOS platforms that needs to control its
System MMU must call platform_set_sysmmu() to inform System MMU
driver who will control it. platform_set_sysmmu() is defined in
<mach/sysmmu.h>
Signed-off-by: KyongHo Cho <pullip.cho@samsung.com>
Acked-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Make the file names consistent with the naming conventions of irq subsystem.
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
This patch introduces irq_remap_ops to hold implementation
specific function pointer to handle interrupt remapping. As
the first part the initialization functions for VT-d are
converted to these ops.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
The files contain code mostly relevant for the Intel
implementation of interrupt remapping. Make that visible in
the file names. Also inline intr_remapping.h into
intr_remapping.c because it is only included there and the
content is very small. So there is no reason for a seperate
header file.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Tegra 20 IOMMU H/W, GART (Graphics Address Relocation Table). This
patch implements struct iommu_ops for GART for the upper IOMMU API.
This H/W module supports only single virtual address space(domain),
and manages a single level 1-to-1 mapping H/W translation page table.
[With small fixes by Joerg Roedel]
Signed-off-by: Hiroshi DOYU <hdoyu@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Add a Kconfig option for the optional driver. Since it is
optional it can be compiled as a module and will only be
loaded when required by another driver.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (33 commits)
iommu/core: Remove global iommu_ops and register_iommu
iommu/msm: Use bus_set_iommu instead of register_iommu
iommu/omap: Use bus_set_iommu instead of register_iommu
iommu/vt-d: Use bus_set_iommu instead of register_iommu
iommu/amd: Use bus_set_iommu instead of register_iommu
iommu/core: Use bus->iommu_ops in the iommu-api
iommu/core: Convert iommu_found to iommu_present
iommu/core: Add bus_type parameter to iommu_domain_alloc
Driver core: Add iommu_ops to bus_type
iommu/core: Define iommu_ops and register_iommu only with CONFIG_IOMMU_API
iommu/amd: Fix wrong shift direction
iommu/omap: always provide iommu debug code
iommu/core: let drivers know if an iommu fault handler isn't installed
iommu/core: export iommu_set_fault_handler()
iommu/omap: Fix build error with !IOMMU_SUPPORT
iommu/omap: Migrate to the generic fault report mechanism
iommu/core: Add fault reporting mechanism
iommu/core: Use PAGE_SIZE instead of hard-coded value
iommu/core: use the existing IS_ALIGNED macro
iommu/msm: ->unmap() should return order of unmapped page
...
Fixup trivial conflicts in drivers/iommu/Makefile: "move omap iommu to
dedicated iommu folder" vs "Rename the DMAR and INTR_REMAP config
options" just happened to touch lines next to each other.
Move OMAP's iommu drivers to the dedicated iommu drivers folder.
While OMAP's iovmm (virtual memory manager) driver does not strictly
belong to the iommu drivers folder, move it there as well, because
it's by no means OMAP-specific (in concept. technically it is still
coupled with OMAP's iommu).
Eventually, iovmm will be completely replaced with the generic,
iommu-based, dma-mapping API.
Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Hiroshi DOYU <Hiroshi.DOYU@nokia.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
A few parts of the driver were missing in drivers/iommu.
Move them there to have the complete driver in that
directory.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
This should ease finding similarities with different platforms,
with the intention of solving problems once in a generic framework
which everyone can use.
Note: to move intel-iommu.c, the declaration of pci_find_upstream_pcie_bridge()
has to move from drivers/pci/pci.h to include/linux/pci.h. This is handled
in this patch, too.
As suggested, also drop DMAR's EXPERIMENTAL tag while we're at it.
Compile-tested on x86_64.
Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
This should ease finding similarities with different platforms,
with the intention of solving problems once in a generic framework
which everyone can use.
Compile-tested on x86_64.
Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
This should ease finding similarities with different platforms,
with the intention of solving problems once in a generic framework
which everyone can use.
Compile-tested for MSM8X60.
Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: David Brown <davidb@codeaurora.org>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Create a dedicated folder for iommu drivers, and move the base
iommu implementation over there.
Grouping the various iommu drivers in a single location will help
finding similar problems shared by different platforms, so they
could be solved once, in the iommu framework, instead of solved
differently (or duplicated) in each driver.
Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Will Deacon
Joerg Roedel
Robin Murphy
Daniel Kurtz
Suman Anna
Laurent Pinchart
Alex Williamson
Laurent Pinchart
Shuah Khan
Varun Sethi
Hideki EIRAKU
Tony Lindgren
Hiroshi Doyu
Linus Torvalds
KyongHo Cho
Suresh Siddha
Joerg Roedel
Ohad Ben-Cohen