Enable TI EDMA option on OMAP and TI_PRIV_EDMA
Signed-off-by: Matt Porter <mporter@ti.com>
Signed-off-by: Joel A Fernandes <joelagnel@ti.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Sekhar Nori <nsekhar@ti.com>
This patch adds Kconfig in the drivers/dma/sh. This patch also adds
a new config "SH_DMAE_BASE" and the "config SH_DMAE" depends on it.
Since some drivers (e.g. sh_mmcif.c) depends on shdma-base.c if
CONFIG_DMA_ENGINE=y, the "config SH_DMAE_BASE" is set as "bool".
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
There is a new generic API to get a DMA channel for a slave device (commit
9a6cecc8 "dmaengine: add helper function to request a slave DMA channel"). In
similar fashion to the DT case (commit aa3da644 "of: Add generic device tree
DMA helpers") we introduce helpers to the DMAC drivers which are enumerated by
ACPI.
The proposed extension provides the following API calls:
acpi_dma_controller_register(), devm_acpi_dma_controller_register()
acpi_dma_controller_free(), devm_acpi_dma_controller_free()
acpi_dma_simple_xlate()
acpi_dma_request_slave_chan_by_index()
acpi_dma_request_slave_chan_by_name()
The first two should be used, for example, at probe() and remove() of the
corresponding DMAC driver. At the register stage the DMAC driver supplies a
custom xlate() function to translate a struct dma_spec into struct dma_chan.
Accordingly to the ACPI Fixed DMA resource specification the only two pieces of
information the slave device has are the channel id and the request line (slave
id). Those two are represented by struct dma_spec. The
acpi_dma_request_slave_chan_by_index() provides access to the specifix FixedDMA
resource by its index. Whereas dma_request_slave_channel() takes a string
parameter to identify the DMA resources required by the slave device. To make a
slave device driver work with both DeviceTree and ACPI enumeration a simple
convention is established: "tx" corresponds to the index 0 and "rx" to the
index 1. In case of robust configuration the slave device driver unfortunately
needs to call acpi_dma_request_slave_chan_by_index() directly.
Additionally the patch provides "managed" version of the register/free pair
i.e. devm_acpi_dma_controller_register() and devm_acpi_dma_controller_free().
Usually, the driver uses only devm_acpi_dma_controller_register().
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The PQ Val ops work on the newer hardware so we should actually provide support
for it and remove the disabling bits.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Pull slave-dmaengine updates from Vinod Koul:
"This is fairly big pull by my standards as I had missed last merge
window. So we have the support for device tree for slave-dmaengine,
large updates to dw_dmac driver from Andy for reusing on different
architectures. Along with this we have fixes on bunch of the drivers"
Fix up trivial conflicts, usually due to #include line movement next to
each other.
* 'next' of git://git.infradead.org/users/vkoul/slave-dma: (111 commits)
Revert "ARM: SPEAr13xx: Pass DW DMAC platform data from DT"
ARM: dts: pl330: Add #dma-cells for generic dma binding support
DMA: PL330: Register the DMA controller with the generic DMA helpers
DMA: PL330: Add xlate function
DMA: PL330: Add new pl330 filter for DT case.
dma: tegra20-apb-dma: remove unnecessary assignment
edma: do not waste memory for dma_mask
dma: coh901318: set residue only if dma is in progress
dma: coh901318: avoid unbalanced locking
dmaengine.h: remove redundant else keyword
dma: of-dma: protect list write operation by spin_lock
dmaengine: ste_dma40: do not remove descriptors for cyclic transfers
dma: of-dma.c: fix memory leakage
dw_dmac: apply default dma_mask if needed
dmaengine: ioat - fix spare sparse complain
dmaengine: move drivers/of/dma.c -> drivers/dma/of-dma.c
ioatdma: fix race between updating ioat->head and IOAT_COMPLETION_PENDING
dw_dmac: add support for Lynxpoint DMA controllers
dw_dmac: return proper residue value
dw_dmac: fill individual length of descriptor
...
This driver could be used on different platforms. Thus, the HAVE_CLK dependency
is dropped away.
Signed-off-by: Heikki Krogerus <heikki.krogerus@linux.intel.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Felipe Balbi <balbi@ti.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
This config item has not carried much meaning for a while now and is
almost always enabled by default. As agreed during the Linux kernel
summit, remove it.
CC: Vinod Koul <vinod.koul@intel.com>
CC: Dan Williams <djbw@fb.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
The bestcomm dma hardware, and some of its users like the FEC ethernet
component, is used in different FreeScale parts, including non-powerpc
parts like the ColdFire MCF547x & MCF548x families. Don't keep the
driver hidden in arch/powerpc where it is inaccessible for other arches.
.c files are moved to drivers/dma/bestcomm, while .h files are moved to
include/linux/fsl/bestcomm. Makefiles, Kconfigs and #include directives
are updated for the new file locations.
Tested by recompiling for MPC5200 with all bestcomm users enabled.
Signed-off-by: Philippe De Muyter <phdm@macqel.be>
Signed-off-by: Anatolij Gustschin <agust@denx.de>
The dw_dmac driver was originally developed for avr32 to be used with the
Synopsys DesignWare AHB DMA controller. Starting from 2.6.38, access to
the device's i/o memory was done with the little-endian readl/writel
functions(1)
This broke the driver for the avr32 platform, because it needs big
(native) endian accessors. This patch makes the endianness configurable
using 'DW_DMAC_BIG_ENDIAN_IO', which will default be true for AVR32
I submitted this patch before(2) but then waited for Andy to finish other
changes to the same module(3).
(1) https://patchwork.kernel.org/patch/608211
(2) https://lkml.org/lkml/2012/8/26/148
(3) https://lkml.org/lkml/2012/9/21/173
Signed-off-by: Hein Tibosch <hein_tibosch@yahoo.es>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Hans-Christian Egtvedt <egtvedt@samfundet.no>
Cc: Ludovic Desroches <ludovic.desroches@atmel.com>
Cc: Havard Skinnemoen <havard@skinnemoen.net>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
1. virtual channel vs. physical channel
Virtual channel is managed by dmaengine
Physical channel handling resource, such as irq
Physical channel is alloced dynamically as descending priority,
freed immediately when irq done.
The availble highest priority physically channel will alwayes be alloced
Issue pending list -> alloc highest dma physically channel available -> dma done -> free physically channel
2. list: running list & pending list
submit: desc list -> pending list
issue_pending_list: if (IDLE) pending list -> running list; free pending list (RUN)
irq: free running list (IDLE)
check pendlist -> pending list -> running list; free pending list (RUN)
3. irq:
Each list generate one irq, calling callback
One list may contain several desc chain, in such case, make sure only the last desc list generate irq.
4. async
Submit will add desc chain to pending list, which can be multi-called
If multi desc chain is submitted, only the last desc would generate irq -> call back
If IDLE, issue_pending_list start pending_list, transforming pendlist to running list
If RUN, irq will start pending list
5. test
5.1 pxa3xx_nand on pxa910
5.2 insmod dmatest.ko (threads_per_chan=y)
By default drivers/dma/dmatest.c test every channel and test memcpy with 1 threads per channel
Signed-off-by: Zhangfei Gao <zhangfei.gao@marvell.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Add a DMA engine driver for the TI EDMA controller. This driver
is implemented as a wrapper around the existing DaVinci private
DMA implementation. This approach allows for incremental conversion
of each peripheral driver to the DMA engine API. The EDMA driver
supports slave transfers but does not yet support cyclic transfers.
Signed-off-by: Matt Porter <mporter@ti.com>
Tested-by: Tom Rini <trini@ti.com>
Tested-by: Sekhar Nori <nsekhar@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Pull ARM DMA engine updates from Russell King:
"This looks scary at first glance, but what it is is:
- a rework of the sa11x0 DMA engine driver merged during the previous
cycle, to extract a common set of helper functions for DMA engine
implementations.
- conversion of amba-pl08x.c to use these helper functions.
- addition of OMAP DMA engine driver (using these helper functions),
and conversion of some of the OMAP DMA users to use DMA engine.
Nothing in the helper functions is ARM specific, so I hope that other
implementations can consolidate some of their code by making use of
these helpers.
This has been sitting in linux-next most of the merge cycle, and has
been tested by several OMAP folk. I've tested it on sa11x0 platforms,
and given it my best shot on my broken platforms which have the
amba-pl08x controller.
The last point is the addition to feature-removal-schedule.txt, which
will have a merge conflict. Between myself and TI, we're planning to
remove the old TI DMA implementation next year."
Fix up trivial add/add conflicts in Documentation/feature-removal-schedule.txt
and drivers/dma/{Kconfig,Makefile}
* 'dmaengine' of git://git.linaro.org/people/rmk/linux-arm: (53 commits)
ARM: 7481/1: OMAP2+: omap2plus_defconfig: enable OMAP DMA engine
ARM: 7464/1: mmc: omap_hsmmc: ensure probe returns error if DMA channel request fails
Add feature removal of old OMAP private DMA implementation
mtd: omap2: remove private DMA API implementation
mtd: omap2: add DMA engine support
spi: omap2-mcspi: remove private DMA API implementation
spi: omap2-mcspi: add DMA engine support
ARM: omap: remove mmc platform data dma_mask and initialization
mmc: omap: remove private DMA API implementation
mmc: omap: add DMA engine support
mmc: omap_hsmmc: remove private DMA API implementation
mmc: omap_hsmmc: add DMA engine support
dmaengine: omap: add support for cyclic DMA
dmaengine: omap: add support for setting fi
dmaengine: omap: add support for returning residue in tx_state method
dmaengine: add OMAP DMA engine driver
dmaengine: sa11x0-dma: add cyclic DMA support
dmaengine: sa11x0-dma: fix DMA residue support
dmaengine: PL08x: ensure all descriptors are freed when channel is released
dmaengine: PL08x: get rid of write only pool_ctr and free_txd locking
...
Initialize the vchan struct, and use the provided spinlock rather than
our own.
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Split the virtual slave channel DMA support from the sa11x0 driver so
this code can be shared with other slave DMA engine drivers.
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Add support for two-channel dma under dmaengine
support: mmp-adma and pxa910-squ
Signed-off-by: Zhangfei Gao <zhangfei.gao@marvell.com>
Signed-off-by: Leo Yan <leoy@marvell.com>
Signed-off-by: Qiao Zhou <zhouqiao@marvell.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Add dmaengine based NVIDIA's Tegra APB DMA driver.
This driver support the slave mode of data transfer from
peripheral to memory and vice versa.
The driver supports for the cyclic and non-cyclic mode
of data transfer.
Signed-off-by: Laxman Dewangan <ldewangan@nvidia.com>
Acked-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
This can get rid of the mach-dependency.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Huang Shijie <b32955@freescale.com>
Signed-off-by: Dong Aisheng <dong.aisheng@linaro.org>
Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Wolfram Sang <w.sang@pengutronix.de>
Acked-by: Marek Vasut <marex@denx.de>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Remove SoC dependency and make it generic for every Atmel ARM AT91. That will
allow to select this driver for newer chips. Keep dependency on AT91 because of
the use of an header file located in include/mach directory.
Modify the comment to reflect this.
Signed-off-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Vinod Koul <vinod.koul@linux.intel.com>
Pull slave-dmaengine update from Vinod Koul:
"This includes the cookie cleanup by Russell, the addition of context
parameter for dmaengine APIs, more arm dmaengine driver cleanup by
moving code to dmaengine, this time for imx by Javier and pl330 by
Boojin along with the usual driver fixes."
Fix up some fairly trivial conflicts with various other cleanups.
* 'next' of git://git.infradead.org/users/vkoul/slave-dma: (67 commits)
dmaengine: imx: fix the build failure on x86_64
dmaengine: i.MX: Fix merge of cookie branch.
dmaengine: i.MX: Add support for interleaved transfers.
dmaengine: imx-dma: use 'dev_dbg' and 'dev_warn' for messages.
dmaengine: imx-dma: remove 'imx_dmav1_baseaddr' and 'dma_clk'.
dmaengine: imx-dma: remove unused arg of imxdma_sg_next.
dmaengine: imx-dma: remove internal structure.
dmaengine: imx-dma: remove 'resbytes' field of 'internal' structure.
dmaengine: imx-dma: remove 'in_use' field of 'internal' structure.
dmaengine: imx-dma: remove sg member from internal structure.
dmaengine: imx-dma: remove 'imxdma_setup_sg_hw' function.
dmaengine: imx-dma: remove 'imxdma_config_channel_hw' function.
dmaengine: imx-dma: remove 'imxdma_setup_mem2mem_hw' function.
dmaengine: imx-dma: remove dma_mode member of internal structure.
dmaengine: imx-dma: remove data member from internal structure.
dmaengine: imx-dma: merge old dma-v1.c with imx-dma.c
dmaengine: at_hdmac: add slave config operation
dmaengine: add context parameter to prep_slave_sg and prep_dma_cyclic
dmaengine/dma_slave: introduce inline wrappers
dma: imx-sdma: Treat firmware messages as warnings instead of erros
...
commit 6bd081277e "dmaengine: imx-dma: merge old dma-v1.c with
imx-dma.c" removed the dependency in config for the imx dma driver,
whereas it should depend on ARCH_MXS
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
It is mainly a simple merge changing the prefix of some
functions to fit the imx-dma namings.
As there are no users of the old dma-v1.c api we can safely
remove this file.
Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Currently there were two part of DMAC PL330 driver for
support old styled s3c-pl330 which has been merged into
drivers/dma/pl330.c driver. Actually, there is no reason
to separate them now.
Basically this patch merges arch/arm/common/pl330.c into
drivers/dma/pl330.c driver and removes useless exported
symbol, externed function and so on.
The newer pl330 driver tested on SMDKV310 and SMDK4212 boards
Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Add support for the SA-11x0 DMA driver, which replaces the private
API version in arch/arm/mach-sa1100/dma.c.
We model this as a set of virtual DMA channels, one for each request
signal, and assign the virtual DMA channel to a physical DMA channel
when there is work to be done. This allows DMA users to claim their
channels, and hold them while not in use, without affecting the
availability of the physical channels.
Another advantage over this approach, compared to the private version,
is that a channel can be reconfigured on the fly without having to
release and re-request it - which for the IrDA driver, allows us to
use DMA for SIR mode transmit without eating up three physical
channels. As IrDA is half-duplex, we actually only need one physical
channel, and this architecture allows us to achieve that.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Rongjun Ying <rongjun.ying@csr.com>
Signed-off-by: Barry Song <Baohua.Song@csr.com>
[fixed direction enums and cyclic api based on changes
already merged]
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
On October 1 in 2011,
OKI SEMICONDUCTOR Co., Ltd. changed the company name in to LAPIS Semiconductor
Co., Ltd.
Signed-off-by: Tomoya MORINAGA <tomoya.rohm@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
The individual SoC dependency in Kconfig hardly scales anymore.
Instead of having such a fine grained dependency just depend
on ARCH_MXC and risk that the uninformed user has to look in
the help text to figure out which driver is the correct one.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
This patch updates following 3 items.
1. Removes unneccessary code.
2. Add AMBA, PL330 configuration
3. Change the meaning of 'peri_id' variable
from PL330 event number to specific dma id by user.
Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The ep93xx DMA controller has 10 independent memory to peripheral (M2P)
channels, and 2 dedicated memory to memory (M2M) channels. M2M channels can
also be used by SPI and IDE to perform DMA transfers to/from their memory
mapped FIFOs.
This driver supports both M2P and M2M channels with DMA_SLAVE, DMA_CYCLIC and
DMA_MEMCPY (M2M only) capabilities.
Signed-off-by: Mika Westerberg <mika.westerberg@iki.fi>
Signed-off-by: Ryan Mallon <rmallon@gmail.com>
Acked-by: H Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Support new device OKI SEMICONDUCTOR ML7223 IOH(Input/Output Hub).
The ML7223 IOH is for MP(Media Phone) use.
The ML7223 is companion chip for Intel Atom E6xx series.
The ML7223 is completely compatible for Intel EG20T PCH.
Signed-off-by: Tomoya MORINAGA <tomoya-linux@dsn.okisemi.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
As a side effect this makes IMX_DMA selectable on i.MX21 again, because
the symbol ARCH_MX21 doesn't exist (MACH_MX21 would have been more correct).
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This driver will now be used in atleast two platforms AVR32 & ARM. And there is
no actual hardware dependency of this driver over AVR32 or ARM. So this
dependency can be removed altogether.
Also dw_dmac driver uses clk framework and must have compilation dependency on
HAVE_CLK
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This patch adds dma support for Freescale MXS-based SoC i.MX23/28,
including apbh-dma and apbx-dma.
* apbh-dma and apbx-dma are supported in the driver as two mxs-dma
instances.
* apbh-dma is different between mx23 and mx28, hardware version
register is used to differentiate.
* mxs-dma supports pio function besides data transfer. The driver
uses dma_data_direction DMA_NONE to identify the pio mode, and
steals sgl and sg_len to get pio words and numbers from clients.
* mxs dmaengine has some very specific features, like sense function
and the special NAND support (nand_lock, nand_wait4ready). These
are too specific to implemented in generic dmaengine driver.
* The driver refers to imx-sdma and only a single descriptor is
statically assigned to each channel.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx: (63 commits)
ARM: PL08x: cleanup comments
Update CONFIG_MD_RAID6_PQ to CONFIG_RAID6_PQ in drivers/dma/iop-adma.c
ARM: PL08x: fix a warning
Fix dmaengine_submit() return type
dmaengine: at_hdmac: fix race while monitoring channel status
dmaengine: at_hdmac: flags located in first descriptor
dmaengine: at_hdmac: use subsys_initcall instead of module_init
dmaengine: at_hdmac: no need set ACK in new descriptor
dmaengine: at_hdmac: trivial add precision to unmapping comment
dmaengine: at_hdmac: use dma_address to program DMA hardware
pch_dma: support new device ML7213 IOH
ARM: PL08x: prevent dma_set_runtime_config() reconfiguring memcpy channels
ARM: PL08x: allow dma_set_runtime_config() to return errors
ARM: PL08x: fix locking between prepare function and submit function
ARM: PL08x: introduce 'phychan_hold' to hold on to physical channels
ARM: PL08x: put txd's on the pending list in pl08x_tx_submit()
ARM: PL08x: rename 'desc_list' as 'pend_list'
ARM: PL08x: implement unmapping of memcpy buffers
ARM: PL08x: store prep_* flags in async_tx structure
ARM: PL08x: shrink srcbus/dstbus in txd structure
...
Support new device OKI SEMICONDUCTOR's ML7213 IOH(Input/Output Hub) which is for
IVI(In-Vehicle Infotainment) use.
The ML7213 is companion chip for Intel Atom E6xx series.
The ML7213 is completely compatible for Intel EG20T PCH.
Signed-off-by: Tomoya MORINAGA <tomoya-linux@dsn.okisemi.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
MPC8308 has pretty much the same DMA controller as MPC5121 and
this patch adds support for MPC8308 to the mpc512x_dma driver.
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
Acked-by: Wolfgang Denk <wd@denx.de>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Product codenames are OK, but once an actual product name is available,
it should be referenced as well.
http://ark.intel.com/chipset.aspx?familyID=52499
Signed-off-by: Valdis Kletnieks <valdis.kletnieks@vt.edu>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The majority of drivers in drivers/dma/ will never establish cross
channel operation chains and do not need the extra overhead in struct
dma_async_tx_descriptor. Make channel switching opt-in by default.
Cc: Anatolij Gustschin <agust@denx.de>
Cc: Ira Snyder <iws@ovro.caltech.edu>
Cc: Linus Walleij <linus.walleij@stericsson.com>
Cc: Saeed Bishara <saeed@marvell.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This driver is currently implemented as a user to the old i.MX
DMA API. This allows us to convert each user of the old API to
the dmaengine API one by one. Once this is done the old DMA
driver can be merged into the i.MX dmaengine driver.
V2: remove some debug leftovers and unused variables
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch adds support for the Freescale i.MX SDMA engine.
The SDMA engine is a scatter/gather DMA engine which is implemented
as a seperate coprocessor. SDMA needs its own firmware which is
requested using the standard request_firmware mechanism. The firmware
has different entry points for each peripheral type, so drivers
have to pass the peripheral type to the DMA engine which in turn
picks the correct firmware entry point from a table contained in
the firmware image itself.
The original Freescale code also supports support for transfering
data to the internal SRAM which needs different entry points to
the firmware. Support for this is currently not implemented. Also,
support for the ASRC (asymmetric sample rate converter) is skipped.
I took a very simple approach to implement dmaengine support. Only
a single descriptor is statically assigned to a each channel. This
means that transfers can't be queued up but only a single transfer
is in progress. This simplifies implementation a lot and is sufficient
for the usual device/memory transfers.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Reviewed-by: Linus Walleij <linus.ml.walleij@gmail.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This creates a DMAengine driver for the ARM PL080/PL081 PrimeCells
based on the implementation earlier submitted by Peter Pearse.
This is working like a charm for memcpy and slave DMA to the PL011
PrimeCell on the PB11MPCore.
This DMA controller is used in mostly unmodified form in the ARM
RealView and Versatile platforms, in the ST-Ericsson Nomadik, and
in the ST SPEAr platform.
It has been converted to use the header from the Samsung PL080
derivate instead of its own defintions. The Samsungs have a custom
driver in their mach-* folders though, atleast we can share the
register definitions.
Cc: Peter Pearse <peter.pearse@arm.com>
Cc: Ben Dooks <ben-linux@fluff.org>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Alessandro Rubini <rubini@unipv.it>
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
[GFP_KERNEL to GFP_NOWAIT in pl08x_prep_dma_memcpy]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Topcliff PCH is the platform controller hub that is going to
be used in Intel's upcoming general embedded platforms. This
adds the driver for Topcliff PCH DMA controller. The DMA
channels are strictly for device to host or host to device
transfers and cannot be used for generic memcpy.
Signed-off-by: Yong Wang <yong.y.wang@intel.com>
[kill GFP_ATOMIC, kill __raw_{read|write}l, locking fixlet]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch add DMA drivers for DMA controllers in Langwell chipset
of Intel(R) Moorestown platform and DMA controllers in Penwell of
Intel(R) Medfield platfrom
This patch adds support for Moorestown DMAC1 and DMAC2 controllers.
It also add support for Medfiled GP DMA and DMAC1 controllers.
These controllers supports memory to peripheral and peripheral to
memory transfers. It support only single block transfers.
This driver is based on Kernel DMA engine
Anyone who wishes to use this controller should use DMA engine APIs
This controller exposes DMA_SLAVE capabilities and notifies the client drivers
of DMA transaction completion
Config option required to be enabled CONFIG_INTEL_MID_DMAC=y
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Add DMA Engine API driver for the PL330 DMAC.
This driver is supposed to be reusable by various
platforms that have one or more PL330 DMACs.
Atm, DMA_SLAVE and DMA_MEMCPY capabilities have been
implemented.
Signed-off-by: Jassi Brar <jassi.brar@samsung.com>
Reviewed-by: Linus Walleij <linus.walleij@stericsson.com>
[dan.j.williams@intel.com: missing slab.h and ->device_control() fixups]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This is a straightforward driver for the ST-Ericsson DMA40 DMA
controller found in U8500, implemented akin to the existing
COH 901 318 driver.
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Acked-by: Srinidh Kasagar <srinidhi.kasagar@stericsson.com>
Cc: STEricsson_nomadik_linux@list.st.com
Cc: Alessandro Rubini <rubini@unipv.it>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Adds the support for the DMA engine withing the timberdale FPGA.
The DMA channels are strict device to host, or host to device
and can not be used for generic memcpy.
Signed-off-by: Richard Röjfors <richard.rojfors@pelagicore.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Adds initial version of MPC512x DMA driver.
Only memory to memory transfers are currenly supported.
Signed-off-by: Piotr Ziecik <kosmo@semihalf.com>
Signed-off-by: Wolfgang Denk <wd@denx.de>
Signed-off-by: Anatolij Gustschin <agust@denx.de>
Cc: John Rigby <jcrigby@gmail.com>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This adds Kconfig options for DEBUG and VERBOSE_DEBUG to the DMA
engine subsystem, I got tired of editing the Makefile manually
each time I want to debug things in here, modelled this on the
debug switches for other subsystems and works like a charm when
working on our DMA engines.
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx:
ppc440spe-adma: adds updated ppc440spe adma driver
iop-adma.c: use resource_size()
dmaengine: clarify the meaning of the DMA_CTRL_ACK flag
sh: stylistic improvements for the DMA driver
dmaengine: fix dmatest to verify minimum transfer length and test buffer size
sh: DMA driver has to specify its alignment requirements
Add COH 901 318 DMA block driver v5
This patch adds new version of the PPC440SPe ADMA driver.
Signed-off-by: Yuri Tikhonov <yur@emcraft.com>
Signed-off-by: Anatolij Gustschin <agust@denx.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch adds support for the ST-Ericsson COH 901 318 DMA block,
found in the U300 series platforms. It registers a DMA slave for
device I/O and also a memcpy slave for memcpy.
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
ioat3.2 does not support asynchronous error notifications which makes
the driver experience latencies when non-zero pq validate results are
expected. Provide a mechanism for turning off async_xor_val and
async_syndrome_val via Kconfig. This approach is generally useful for
any driver that specifies ASYNC_TX_DISABLE_CHANNEL_SWITCH and would like
to force the async_tx api to fall back to the synchronous path for
certain operations.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Add at91sam9g45 dependency to drivers/dma/Kconfig
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This restriction prevented ASYNC_TX_DMA from being enabled on platform
configurations where DMA address conversion could not be performed in
place on the stack. Since commit 04ce9ab3 ("async_xor: permit callers
to pass in a 'dma/page scribble' region") the async_tx api now either
uses a caller provided 'scribble' buffer, or performs the conversion in
place when sizeof(dma_addr_t) <= sizeof(struct page *).
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This supported all DMA channels, and it was tested in SH7722,
SH7780, SH7785 and SH7763.
This can not use with SH DMA API.
Signed-off-by: Nobuhiro Iwamatsu <iwamatsu.nobuhiro@renesas.com>
Reviewed-by: Matt Fleming <matt@console-pimps.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: Paul Mundt <lethal@linux-sh.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Channel switching is problematic for some dmaengine drivers as the
architecture precludes separating the ->prep from ->submit. In these
cases the driver can select ASYNC_TX_DISABLE_CHANNEL_SWITCH to modify
the async_tx allocator to only return channels that support all of the
required asynchronous operations.
For example MD_RAID456=y selects support for asynchronous xor, xor
validate, pq, pq validate, and memcpy. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=y any channel with all these
capabilities is marked DMA_ASYNC_TX allowing async_tx_find_channel() to
quickly locate compatible channels with the guarantee that dependency
chains will remain on one channel. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=n async_tx_find_channel() may select
channels that lead to operation chains that need to cross channel
boundaries using the async_tx channel switch capability.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This AHB DMA Controller (aka HDMA or DMAC on AT91 systems) is availlable on
at91sam9rl chip. It will be used on other products in the future.
This first release covers only the memory-to-memory tranfer type. This is the
only tranfer type supported by this chip. On other products, it will be used
also for peripheral DMA transfer (slave API support to come).
I used dmatest client without problem in different configurations to test it.
Full documentation for this controller can be found in the SAM9RL datasheet:
http://www.atmel.com/dyn/products/product_card.asp?part_id=4243
Signed-off-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
On HIGHMEM64G systems dma_addr_t is known to be larger than (void *)
which precludes async_xor from performing dma address conversions by
reusing the input parameter address list. However, other parts of the
dmaengine infrastructure do not suffer this constraint, so the
HIGHMEM64G restriction can be down-levelled.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch adds support for the integrated DMAC of the TXx9 family.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
async_xor() needs space to perform dma and page address conversions. In
most cases the code can simply reuse the struct page * array because the
size of the native pointer matches the size of a dma/page address. In
order to support archs where sizeof(dma_addr_t) is larger than
sizeof(struct page *), or to preserve the input parameters, we utilize a
memory region passed in by the caller.
Since the code is now prepared to handle the case where it cannot
perform address conversions on the stack, we no longer need the
!HIGHMEM64G dependency in drivers/dma/Kconfig.
[ Impact: don't clobber input buffers for address conversions ]
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
i.MX3x SoCs contain an Image Processing Unit, consisting of a Control
Module (CM), Display Interface (DI), Synchronous Display Controller (SDC),
Asynchronous Display Controller (ADC), Image Converter (IC), Post-Filter
(PF), Camera Sensor Interface (CSI), and an Image DMA Controller (IDMAC).
CM contains, among other blocks, an Interrupt Generator (IG) and a Clock
and Reset Control Unit (CRCU). This driver serves IDMAC and IG. They are
supported over dmaengine and irq-chip APIs respectively.
IDMAC is a specialised DMA controller, its DMA channels cannot be used for
general-purpose operations, even though it might be possible to configure
a memory-to-memory channel for memcpy operation. This driver will not work
with generic dmaengine clients, clients, wishing to use it must use
respective wrapper structures, they also must specify which channels they
require, as channels are hard-wired to specific IPU functions.
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Guennadi Liakhovetski <lg@denx.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
async_tx.ko is a consumer of dma channels. A circular dependency arises
if modules in drivers/dma rely on common code in async_tx.ko. It
prevents either module from being unloaded.
Move dma_wait_for_async_tx and async_tx_run_dependencies to dmaeninge.o
where they should have been from the beginning.
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Modify the Freescale Elo / Elo Plus DMA driver so that it can be compiled as
a module.
The primary change is to stop treating the DMA controller as a bus, and the
DMA channels as devices on the bus. This is because the Open Firmware (OF)
kernel code does not allow busses to be removed, so although we can call
of_platform_bus_probe() to probe the DMA channels, there is no
of_platform_bus_remove(). Instead, the DMA channels are manually probed,
similar to what fsl_elbc_nand.c does.
Cc: Scott Wood <scottwood@freescale.com>
Acked-by: Li Yang <leoli@freescale.com>
Signed-off-by: Timur Tabi <timur@freescale.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This adds a driver for the Synopsys DesignWare DMA controller (aka
DMACA on AVR32 systems.) This DMA controller can be found integrated
on the AT32AP7000 chip and is primarily meant for peripheral DMA
transfer, but can also be used for memory-to-memory transfers.
This patch is based on a driver from David Brownell which was based on
an older version of the DMA Engine framework. It also implements the
proposed extensions to the DMA Engine API for slave DMA operations.
The dmatest client shows no problems, but there may still be room for
improvement performance-wise. DMA slave transfer performance is
definitely "good enough"; reading 100 MiB from an SD card running at ~20
MHz yields ~7.2 MiB/s average transfer rate.
Full documentation for this controller can be found in the Synopsys
DW AHB DMAC Databook:
http://www.synopsys.com/designware/docs/iip/DW_ahb_dmac/latest/doc/dw_ahb_dmac_db.pdf
The controller has lots of implementation options, so it's usually a
good idea to check the data sheet of the chip it's intergrated on as
well. The AT32AP7000 data sheet can be found here:
http://www.atmel.com/dyn/products/datasheets.asp?family_id=682
Changes since v4:
* Use client_count instead of dma_chan_is_in_use()
* Add missing include
* Unmap buffers unless client told us not to
Changes since v3:
* Update to latest DMA engine and DMA slave APIs
* Embed the hw descriptor into the sw descriptor
* Clean up and update MODULE_DESCRIPTION, copyright date, etc.
Changes since v2:
* Dequeue all pending transfers in terminate_all()
* Rename dw_dmac.h -> dw_dmac_regs.h
* Define and use controller-specific dma_slave data
* Fix up a few outdated comments
* Define hardware registers as structs (doesn't generate better
code, unfortunately, but it looks nicer.)
* Get number of channels from platform_data instead of hardcoding it
based on CONFIG_WHATEVER_CPU.
* Give slave clients exclusive access to the channel
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>,
Signed-off-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This client tests DMA memcpy using various lengths and various offsets
into the source and destination buffers. It will initialize both
buffers with a repeatable pattern and verify that the DMA engine copies
the requested region and nothing more. It will also verify that the
bytes aren't swapped around, and that the source buffer isn't modified.
The dmatest module can be configured to test a specific device, a
specific channel. It can also test multiple channels at the same time,
and it can start multiple threads competing for the same channel.
Changes since v2:
* Support testing multiple channels at the same time
* Support testing with multiple threads competing for the same channel
* Use counting test patterns in order to catch byte ordering issues
Changes since v1:
* Remove extra dashes around "help"
* Remove "default n" from Kconfig
* Turn TEST_BUF_SIZE into a module parameter
* Return DMA_NAK instead of DMA_DUP
* Print unhandled events
* Support testing specific channels and devices
* Move to the end of the Makefile
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Haavard Skinnemoen <hskinnemoen@atmel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The XOR engine found in Marvell's SoCs and system controllers
provides XOR and DMA operation, iSCSI CRC32C calculation, memory
initialization, and memory ECC error cleanup operation support.
This driver implements the DMA engine API and supports the following
capabilities:
- memcpy
- xor
- memset
The XOR engine can be used by DMA engine clients implemented in the
kernel, one of those clients is the RAID module. In that case, I
observed 20% improvement in the raid5 write throughput, and 40%
decrease in the CPU utilization when doing array construction, those
results obtained on an 5182 running at 500Mhz.
When enabling the NET DMA client, the performance decreased, so
meanwhile it is recommended to keep this client off.
Signed-off-by: Saeed Bishara <saeed@marvell.com>
Signed-off-by: Lennert Buytenhek <buytenh@marvell.com>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The dependency is redundant since all drivers set their specific arch
dependencies. The NET_DMA option is modified to be enabled only on platforms
where it is known to have a positive effect. HAS_DMA is added as an explicit
dependency for the DMADEVICES menu.
Acked-by: Adrian Bunk <bunk@kernel.org>
Acked-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Always enabling the fsl_dma_self_test() to ensure the DMA controller
should works well after the driver probed.
Signed-off-by: Zhang Wei <wei.zhang@freescale.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The driver implements DMA engine API for Freescale MPC85xx DMA controller,
which could be used by devices in the silicon. The driver supports the
Basic mode of Freescale MPC85xx DMA controller. The MPC85xx processors
supported include MPC8540/60, MPC8555, MPC8548, MPC8641 and so on.
The MPC83xx(MPC8349, MPC8360) are also supported.
[kamalesh@linux.vnet.ibm.com: build fix]
[dan.j.williams@intel.com: merge mm fixes, rebase on async_tx-2.6.25]
Signed-off-by: Zhang Wei <wei.zhang@freescale.com>
Signed-off-by: Ebony Zhu <ebony.zhu@freescale.com>
Acked-by: Kumar Gala <galak@gate.crashing.org>
Cc: Shannon Nelson <shannon.nelson@intel.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The tx_set_src and tx_set_dest methods were originally implemented to allow
an array of addresses to be passed down from async_xor to the dmaengine
driver while minimizing stack overhead. Removing these methods allows
drivers to have all transaction parameters available at 'prep' time, saves
two function pointers in struct dma_async_tx_descriptor, and reduces the
number of indirect branches..
A consequence of moving this data to the 'prep' routine is that
multi-source routines like async_xor need temporary storage to convert an
array of linear addresses into an array of dma addresses. In order to keep
the same stack footprint of the previous implementation the input array is
reused as storage for the dma addresses. This requires that
sizeof(dma_addr_t) be less than or equal to sizeof(void *). As a
consequence CONFIG_DMADEVICES now depends on !CONFIG_HIGHMEM64G. It also
requires that drivers be able to make descriptor resources available when
the 'prep' routine is polled.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Shannon Nelson <shannon.nelson@intel.com>
This patch corrects recently changed (and now invalid) Kconfig descriptions
for the DMA engine framework:
- Non-Intel(R) hardware also has DMA engines;
- DMA is used for more than memcpy and RAID offloading.
In fact, on most platforms memcpy and RAID aren't factors, and DMA
exists so that peripherals can transfer data to/from memory while
the CPU does other work.
Signed-off-by: Haavard Skinnemoen <hskinnemoen@atmel.com>
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
No reason I can think of of making them default y Most people don't have
the hardware and with default y they just pollute lots of configs during
make oldconfig.
Signed-off-by: Andi Kleen <ak@suse.de>
Acked-by: Jeff Garzik <jeff@garzik.org>
Acked-by: "Nelson, Shannon" <shannon.nelson@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add code to connect to the DCA driver and provide cpu tags for use by
drivers that would like to use Direct Cache Access hints.
[Adrian Bunk] Several Kconfig cleanup items
[Andrew Morten, Chris Leech] Fix for using cpu_physical_id() even when
built for uni-processor
Signed-off-by: Shannon Nelson <shannon.nelson@intel.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Continuing the work started in 411f0f3edc ...
This enables code with a dma path, that compiles away, to build without
requiring additional code factoring. It also prevents code that calls
dma_alloc_coherent and dma_free_coherent from linking whereas previously
the code would hit a BUG() at run time. Finally, it allows archs that set
!HAS_DMA to delete their asm/dma-mapping.h file.
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: John W. Linville <linville@tuxdriver.com>
Cc: Kyle McMartin <kyle@parisc-linux.org>
Cc: James Bottomley <James.Bottomley@SteelEye.com>
Cc: Tejun Heo <htejun@gmail.com>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: <geert@linux-m68k.org>
Cc: <zippel@linux-m68k.org>
Cc: <spyro@f2s.com>
Cc: <ysato@users.sourceforge.jp>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The Intel(R) IOP series of i/o processors integrate an Xscale core with
raid acceleration engines. The capabilities per platform are:
iop219:
(2) copy engines
iop321:
(2) copy engines
(1) xor and block fill engine
iop33x:
(2) copy and crc32c engines
(1) xor, xor zero sum, pq, pq zero sum, and block fill engine
iop34x (iop13xx):
(2) copy, crc32c, xor, xor zero sum, and block fill engines
(1) copy, crc32c, xor, xor zero sum, pq, pq zero sum, and block fill engine
The driver supports the features of the async_tx api:
* asynchronous notification of operation completion
* implicit (interupt triggered) handling of inter-channel transaction
dependencies
The driver adapts to the platform it is running by two methods.
1/ #include <asm/arch/adma.h> which defines the hardware specific
iop_chan_* and iop_desc_* routines as a series of static inline
functions
2/ The private platform data attached to the platform_device defines the
capabilities of the channels
20070626: Callbacks are run in a tasklet. Given the recent discussion on
LKML about killing tasklets in favor of workqueues I did a quick conversion
of the driver. Raid5 resync performance dropped from 50MB/s to 30MB/s, so
the tasklet implementation remains until a generic softirq interface is
available.
Changelog:
* fixed a slot allocation bug in do_iop13xx_adma_xor that caused too few
slots to be requested eventually leading to data corruption
* enabled the slot allocation routine to attempt to free slots before
returning -ENOMEM
* switched the cleanup routine to solely use the software chain and the
status register to determine if a descriptor is complete. This is
necessary to support other IOP engines that do not have status writeback
capability
* make the driver iop generic
* modified the allocation routines to understand allocating a group of
slots for a single operation
* added a null xor initialization operation for the xor only channel on
iop3xx
* support xor operations on buffers larger than the hardware maximum
* split the do_* routines into separate prep, src/dest set, submit stages
* added async_tx support (dependent operations initiation at cleanup time)
* simplified group handling
* added interrupt support (callbacks via tasklets)
* brought the pending depth inline with ioat (i.e. 4 descriptors)
* drop dma mapping methods, suggested by Chris Leech
* don't use inline in C files, Adrian Bunk
* remove static tasklet declarations
* make iop_adma_alloc_slots easier to read and remove chances for a
corrupted descriptor chain
* fix locking bug in iop_adma_alloc_chan_resources, Benjamin Herrenschmidt
* convert capabilities over to dma_cap_mask_t
* fixup sparse warnings
* add descriptor flush before iop_chan_enable
* checkpatch.pl fixes
* gpl v2 only correction
* move set_src, set_dest, submit to async_tx methods
* move group_list and phys to async_tx
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The async_tx api provides methods for describing a chain of asynchronous
bulk memory transfers/transforms with support for inter-transactional
dependencies. It is implemented as a dmaengine client that smooths over
the details of different hardware offload engine implementations. Code
that is written to the api can optimize for asynchronous operation and the
api will fit the chain of operations to the available offload resources.
I imagine that any piece of ADMA hardware would register with the
'async_*' subsystem, and a call to async_X would be routed as
appropriate, or be run in-line. - Neil Brown
async_tx exploits the capabilities of struct dma_async_tx_descriptor to
provide an api of the following general format:
struct dma_async_tx_descriptor *
async_<operation>(..., struct dma_async_tx_descriptor *depend_tx,
dma_async_tx_callback cb_fn, void *cb_param)
{
struct dma_chan *chan = async_tx_find_channel(depend_tx, <operation>);
struct dma_device *device = chan ? chan->device : NULL;
int int_en = cb_fn ? 1 : 0;
struct dma_async_tx_descriptor *tx = device ?
device->device_prep_dma_<operation>(chan, len, int_en) : NULL;
if (tx) { /* run <operation> asynchronously */
...
tx->tx_set_dest(addr, tx, index);
...
tx->tx_set_src(addr, tx, index);
...
async_tx_submit(chan, tx, flags, depend_tx, cb_fn, cb_param);
} else { /* run <operation> synchronously */
...
<operation>
...
async_tx_sync_epilog(flags, depend_tx, cb_fn, cb_param);
}
return tx;
}
async_tx_find_channel() returns a capable channel from its pool. The
channel pool is organized as a per-cpu array of channel pointers. The
async_tx_rebalance() routine is tasked with managing these arrays. In the
uniprocessor case async_tx_rebalance() tries to spread responsibility
evenly over channels of similar capabilities. For example if there are two
copy+xor channels, one will handle copy operations and the other will
handle xor. In the SMP case async_tx_rebalance() attempts to spread the
operations evenly over the cpus, e.g. cpu0 gets copy channel0 and xor
channel0 while cpu1 gets copy channel 1 and xor channel 1. When a
dependency is specified async_tx_find_channel defaults to keeping the
operation on the same channel. A xor->copy->xor chain will stay on one
channel if it supports both operation types, otherwise the transaction will
transition between a copy and a xor resource.
Currently the raid5 implementation in the MD raid456 driver has been
converted to the async_tx api. A driver for the offload engines on the
Intel Xscale series of I/O processors, iop-adma, is provided in a later
commit. With the iop-adma driver and async_tx, raid456 is able to offload
copy, xor, and xor-zero-sum operations to hardware engines.
On iop342 tiobench showed higher throughput for sequential writes (20 - 30%
improvement) and sequential reads to a degraded array (40 - 55%
improvement). For the other cases performance was roughly equal, +/- a few
percentage points. On a x86-smp platform the performance of the async_tx
implementation (in synchronous mode) was also +/- a few percentage points
of the original implementation. According to 'top' on iop342 CPU
utilization drops from ~50% to ~15% during a 'resync' while the speed
according to /proc/mdstat doubles from ~25 MB/s to ~50 MB/s.
The tiobench command line used for testing was: tiobench --size 2048
--block 4096 --block 131072 --dir /mnt/raid --numruns 5
* iop342 had 1GB of memory available
Details:
* if CONFIG_DMA_ENGINE=n the asynchronous path is compiled away by making
async_tx_find_channel a static inline routine that always returns NULL
* when a callback is specified for a given transaction an interrupt will
fire at operation completion time and the callback will occur in a
tasklet. if the the channel does not support interrupts then a live
polling wait will be performed
* the api is written as a dmaengine client that requests all available
channels
* In support of dependencies the api implicitly schedules channel-switch
interrupts. The interrupt triggers the cleanup tasklet which causes
pending operations to be scheduled on the next channel
* Xor engines treat an xor destination address differently than a software
xor routine. To the software routine the destination address is an implied
source, whereas engines treat it as a write-only destination. This patch
modifies the xor_blocks routine to take a an explicit destination address
to mirror the hardware.
Changelog:
* fixed a leftover debug print
* don't allow callbacks in async_interrupt_cond
* fixed xor_block changes
* fixed usage of ASYNC_TX_XOR_DROP_DEST
* drop dma mapping methods, suggested by Chris Leech
* printk warning fixups from Andrew Morton
* don't use inline in C files, Adrian Bunk
* select the API when MD is enabled
* BUG_ON xor source counts <= 1
* implicitly handle hardware concerns like channel switching and
interrupts, Neil Brown
* remove the per operation type list, and distribute operation capabilities
evenly amongst the available channels
* simplify async_tx_find_channel to optimize the fast path
* introduce the channel_table_initialized flag to prevent early calls to
the api
* reorganize the code to mimic crypto
* include mm.h as not all archs include it in dma-mapping.h
* make the Kconfig options non-user visible, Adrian Bunk
* move async_tx under crypto since it is meant as 'core' functionality, and
the two may share algorithms in the future
* move large inline functions into c files
* checkpatch.pl fixes
* gpl v2 only correction
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-By: NeilBrown <neilb@suse.de>
Disable some more menus in the configuration files that are of no
interest to a s390 machine.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Attempts to allocate per-CPU DMA channels
Signed-off-by: Chris Leech <christopher.leech@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Provides an API for offloading memory copies to DMA devices
Signed-off-by: Chris Leech <christopher.leech@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Matt Porter
Shimoda, Yoshihiro
Andy Shevchenko
Dave Jiang
Heiko Carstens
Linus Torvalds
Vinod Koul
Barry Song
Heikki Krogerus
Kees Cook
Philippe De Muyter
Hein Tibosch
Zhangfei Gao
Russell King
Vinod Koul
Laxman Dewangan
Huang Shijie
Dong Aisheng
Nicolas Ferre
Javier Martin
Boojin Kim
Rongjun Ying
Tomoya MORINAGA
Sascha Hauer
Mika Westerberg
Tomoya MORINAGA
Uwe Kleine-König
Viresh Kumar
Shawn Guo
Ilya Yanok
Valdis.Kletnieks@vt.edu
Dan Williams
Linus Walleij
Yong Wang
Paul Mundt
Jassi Brar
Magnus Damm
Richard Röjfors
Piotr Ziecik
Anatolij Gustschin
Yegor Yefremov
Nobuhiro Iwamatsu
Atsushi Nemoto
Guennadi Liakhovetski
Timur Tabi
Haavard Skinnemoen
Haavard Skinnemoen
Saeed Bishara
Zhang Wei
Andi Kleen
Shannon Nelson
Martin Schwidefsky
Chris Leech