Граф коммитов

111 Коммитов

Автор SHA1 Сообщение Дата
Felipe Balbi 8437ab99a6 usb: host: ehci: remove unnecessary max_packet() macro
Now that usb_endpoint_maxp() only returns the lowest
11 bits from wMaxPacketSize, we can remove this macro
from the driver.

Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: <linux-usb@vger.kernel.org>
Signed-off-by: Felipe Balbi <felipe.balbi@linux.intel.com>
2016-11-03 10:38:24 +02:00
Felipe Balbi e3b89080f2 usb: host: ehci: make use of new usb_endpoint_maxp_mult()
We have introduced a helper to calculate multiplier
value from wMaxPacketSize. Start using it.

Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: <linux-usb@vger.kernel.org>
Signed-off-by: Felipe Balbi <felipe.balbi@linux.intel.com>
2016-11-03 10:38:20 +02:00
Alan Stern 8df0d77d8c USB: EHCI: improvements to unlink_empty_async_suspended()
unlink_empty_async_suspended() is marked __maybe_unused.  This is
because its caller, ehci_bus_suspend(), is protected by "#ifdef
CONFIG_PM".  We should use the same protection here instead of
__maybe_unused.

unlink_empty_async_suspended() gets called only when the root hub is
suspended.  It's silly for it to call start_iaa_cycle() at such a
time; the IAA mechanism doesn't work when the root hub isn't running.
It should call end_unlink_async() instead.  But even this isn't
necessary, since there already is a call to end_iaa_cycle() right
before the call to unlink_empty_async_suspended().  All we have to do
is interchange the two subroutine calls.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2016-02-03 13:14:52 -08:00
Alan Stern 87d61912c2 USB: EHCI: add a delay when unlinking an active QH
Michael Reutman reports that an AMD/ATI EHCI host controller on one of
his computers does not stop transferring data when an active bulk QH
is unlinked from the async schedule.  Apparently that host controller
fails to implement the IAA mechanism correctly when an active QH is
unlinked.  This leads to data corruption, because the controller
continues to update the QH in memory when the driver doesn't expect
it.  As a result, the next URB submitted for that QH can hang, because
the link pointers for the TD queue have been messed up.  This
misbehavior is observed quite regularly.

To be fair, the EHCI spec (section 4.8.2) says that active QHs should
not be unlinked.  It goes on to recommend a procedure that involves
waiting for the QH to go inactive before unlinking it.  In the real
world this is impractical, not least because the QH may _never_ go
inactive.  (What were they thinking?)  Sometimes we have no choice but
to unlink an active QH.

In an attempt to avoid the problems that can ensue, this patch changes
how the driver decides when the unlink is complete.  In addition to
waiting through two IAA cycles, in cases where the QH was not known to
be inactive beforehand we now wait until a 2-ms period has elapsed
with the host controller making no change to the QH data structure
(the hw_current and hw_token fields in particular).  The intuition
here is that after such a long period, the endpoint must be NAKing and
hopefully the QH has been dropped from the host controller's internal
cache.  There's no way to know if this reasoning is really valid --
the spec is no help in this regard -- but at least this approach fixes
Michael's problem.

The test for whether the QH is already known to be inactive involves
the reason for unlinking the QH originally.  If it was unlinked
because it had halted, or it stopped in response to a short read, or
it overlaid a dummy TD (a silicon bug), then it certainly is inactive.
If it was unlinked because the TD queue was empty and no TDs have been
added to the queue in the meantime, then it must be inactive.  Or if
the hardware status indicates that the QH is currently halted (even if
that wasn't the reason for unlinking it), then it is inactive.
Otherwise, if none of those checks apply, we go through the 2-ms
delay.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: Michael Reutman <mreutman@epiqsolutions.com>
Tested-by: Michael Reutman <mreutman@epiqsolutions.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2016-02-03 13:14:52 -08:00
Alan Stern f96fba0dbf USB: EHCI: improve handling of the ehci->iaa_in_progress flag
This patch improves the way ehci-hcd handles the iaa_in_progress flag.
The current code is somewhat careless in this regard:

	The flag is meaningless when the root hub isn't running, most
	particularly after the root hub has been suspended.  But in
	start_iaa_cycle(), the driver checks the flag before checking
	the root hub's state.  They should be checked in the opposite
	order.

	That routine also sets the flag too early, before it has
	definitely committed to starting an IAA cycle.

	The flag is turned off in end_unlink_async().  Upcoming
	changes will call that routine at other times, not just at the
	end of an IAA cycle.  The two actions are logically separate
	(although related), so we separate out a new routine to be
	called in place of end_unlink_async() whenever an IAA cycle
	ends: end_iaa_cycle().

	iaa_in_progress should be turned off when the root hub is
	suspended -- we certainly don't want it still to be set when
	the root hub resumes.  Therefore the call to
	end_unlink_async() in ehci_bus_suspend() should also be
	replaced with a call to end_iaa_cycle().

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2016-02-03 13:14:52 -08:00
Alan Stern fcc5184ec1 USB: EHCI: store reason for unlinking a QH
This patch replaces the "exception" bitflag in the ehci_qh structure
with a more explicit "unlink_reason" bitmask.  This is for use in the
following patch, where we will need to have a good idea of the
reason for unlinking a QH, not just "something exceptional happened".

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Tested-by: Michael Reutman <mreutman@epiqsolutions.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2016-02-03 13:14:52 -08:00
Alan Stern fc0855f274 USB: EHCI: warn on unexpectedly active QH
This patch adds a new warning message to ehci-hcd.  The warning is
triggered whenever the driver finds that the hardware has set the
Active bit in a QH at a time when the driver expects the QH to be
completely idle.  Such bugs have been observed by users in the past,
and since they can lead to serious problems (such as inability to
unlink an URB that never completes), it would be good to know about
them when they occur.

This won't fix these bugs; that's a bigger job for a later patch.  But
success isn't guaranteed, since this depends on aspects of the
hardware which are not documented in the EHCI spec or for which the
spec's recommendations are clearly unworkable.  It therefore seems
worthwhile to check for these bugs proactively.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: Michael Reutman <mreutman@epiqsolutions.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2015-12-01 10:44:19 -08:00
Oliver Neukum 1c20163d30 usb: kill DEBUG compile option
In the drivers that no longer need it, it is removed.
It is removed from the Makefile. Drivers not fully converted
to dynamic debug have it shifted down into the individual
drivers.

Signed-off-by: Oliver Neukum <oneukum@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-12-03 10:34:33 -08:00
Alan Stern d0ce5c6b92 USB: EHCI: use a bandwidth-allocation table
This patch significantly changes the scheduling code in ehci-hcd.
Instead of calculating the current bandwidth utilization by trudging
through the schedule and adding up the times used by the existing
transfers, we will now maintain a table holding the time used for each
of 64 microframes.  This will drastically speed up the bandwidth
computations.

In addition, it eliminates a theoretical bug.  An isochronous endpoint
may have bandwidth reserved even at times when it has no transfers
listed in the schedule.  The table will keep track of the reserved
bandwidth, whereas adding up entries in the schedule would miss it.

As a corollary, we can keep bandwidth reserved for endpoints even
when they aren't in active use.  Eventually the bandwidth will be
reserved when a new alternate setting is installed; for now the
endpoint's reservation takes place when its first URB is submitted.

A drawback of this approach is that transfers with an interval larger
than 64 microframes will have to be charged for bandwidth as though
the interval was 64.  In practice this shouldn't matter much;
transfers with longer intervals tend to be rather short anyway (things
like hubs or HID devices).

Another minor drawback is that we will keep track of two different
period and phase values: the actual ones and the ones used for
bandwidth allocation (which are limited to 64).  This adds only a
small amount of overhead: 3 bytes for each endpoint.

The patch also adds a new debugfs file named "bandwidth" to display
the information stored in the new table.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-10-11 16:45:43 -07:00
Alan Stern ffa0248e64 USB: EHCI: create a "periodic schedule info" struct
This patch begins the process of unifying the scheduling parameters
that ehci-hcd uses for interrupt and isochronous transfers.  It
creates an ehci_per_sched structure, which will be stored in both
ehci_qh and ehci_iso_stream structures, and will contain the common
scheduling information needed for both.

Initially we merely create the new structure and move some existing
fields into it.  Later patches will add more fields and utilize these
structures in improved scheduling algorithms.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-10-11 16:40:27 -07:00
Greg Kroah-Hartman c04ee4b113 Revert "Revert "USB: EHCI: support running URB giveback in tasklet context""
This reverts commit 3b8d7321ed, which
brings back commit 428aac8a81 as it should
be working for the 3.13-rc1 merge window now that Alan's other fixes are
here in the tree already.

Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-09-23 13:32:51 -07:00
Greg Kroah-Hartman 3b8d7321ed Revert "USB: EHCI: support running URB giveback in tasklet context"
This reverts commit 428aac8a81.

This isn't quite ready for 3.12, we need some more EHCI driver changes
that are just now showing up.  So revert this for now, and queue it up
later for 3.13.

Reported-by: Alan Stern <stern@rowland.harvard.edu>
Cc: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-09-17 09:36:10 -07:00
Xenia Ragiadakou 1512c91f1c ehci: enable debugging code when CONFIG_DYNAMIC_DEBUG is set
The debugging code for ehci is enabled to run if the DEBUG flag is defined.
This patch enables the debugging code also when the kernel is configured
with dynamic debugging on.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-08-30 12:02:07 -07:00
Xenia Ragiadakou fea26ef095 ehci: remove ehci_vdbg() verbose debugging statements
This patch removes ehci_vdbg debugging statements from EHCI host controller
driver because they produce too much information, lowering the signal to noise
ratio when debugging, and because they are not used anymore.

Signed-off-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-08-30 12:02:07 -07:00
Jack Pham 726a85caa3 usb: host: add Kconfig option for EHSET
commit 9841f37a1c ("usb: ehci: Add support for SINGLE_STEP_SET_FEATURE
test of EHSET") added additional code to the EHCI hub driver but it is
anticipated to only have a limited audience (e.g. embedded silicon
vendors and integrators). Avoid subjecting all EHCI (and in the future
maybe xHCI/OHCI, etc.) HCD users to code bloat by conditionally
compiling the EHSET-specific additions with a new Kconfig option,
CONFIG_USB_HCD_TEST_MODE.

Signed-off-by: Jack Pham <jackp@codeaurora.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-08-14 12:18:26 -07:00
Manu Gautam 9841f37a1c usb: ehci: Add support for SINGLE_STEP_SET_FEATURE test of EHSET
The USB Embedded High-speed Host Electrical Test (EHSET) defines the
SINGLE_STEP_SET_FEATURE test as follows:

1) The host enumerates the test device with VID:0x1A0A, PID:0x0108
2) The host sends the SETUP stage of a GetDescriptor(Device)
3) The device ACKs the request
4) The host issues SOFs for 15 seconds allowing the test operator to
   raise the scope trigger just above the SOF voltage level
5) The host sends the IN packet
6) The device sends data in response, triggering the scope
7) The host sends an ACK in response to the data

This patch adds additional handling to the EHCI hub driver and allows
the EHSET driver to initiate this test mode by issuing a a SetFeature
request to the root hub with a Test Selector value of 0x06. From there
it mimics ehci_urb_enqueue() but separately submits QTDs for the
SETUP and DATA/STATUS stages in order to insert a delay in between.

Signed-off-by: Manu Gautam <mgautam@codeaurora.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
[jackp@codeaurora.org: imported from commit c2084930 on codeaurora.org;
 minor cleanup and updated author email]
Signed-off-by: Jack Pham <jackp@codeaurora.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-08-12 13:13:32 -07:00
Ming Lei 428aac8a81 USB: EHCI: support running URB giveback in tasklet context
All 4 transfer types can work well on EHCI HCD after switching to run
URB giveback in tasklet context, so mark all HCD drivers to support
it.

Also we don't need to release ehci->lock during URB giveback any more.

>From below test results on 3 machines(2 ARM and one x86), time
consumed by EHCI interrupt handler droped much without performance
loss.

1 test description
1.1 mass storage performance test:
- run below command 10 times and compute the average performance

    dd if=/dev/sdN iflag=direct of=/dev/null bs=200M count=1

- two usb mass storage device:
A: sandisk extreme USB 3.0 16G(used in test case 1 & case 2)
B: kingston DataTraveler G2 4GB(only used in test case 2)

1.2 uvc function test:
- run one simple capture program in the below link

   http://kernel.ubuntu.com/~ming/up/capture.c

- capture format 640*480 and results in High Bandwidth mode on the
uvc device: Z-Star 0x0ac8/0x3450

- on T410(x86) laptop, also use guvcview to watch video capture/playback

1.3 about test2 and test4
- both two devices involved are tested concurrently by above test items

1.4 how to compute irq time(the time consumed by ehci_irq)
- use trace points of irq:irq_handler_entry and irq:irq_handler_exit

1.5 kernel
3.10.0-rc3-next-20130528

1.6 test machines
Pandaboard A1: ARM CortexA9 dural core
Arndale board: ARM CortexA15 dural core
T410: i5 CPU 2.67GHz quad core

2 test result
2.1 test case1: single mass storage device performance test
--------------------------------------------------------------------
		upstream 		| patched
		perf(MB/s)+irq time(us)	| perf(MB/s)+irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  25.280(avg:145,max:772)	| 25.540(avg:14, max:75)
Arndale board:  29.700(avg:33, max:129)	| 29.700(avg:10,  max:50)
T410: 		34.430(avg:17, max:154*)| 34.660(avg:12, max:155)
---------------------------------------------------------------------

2.2 test case2: two mass storage devices' performance test
--------------------------------------------------------------------
		upstream 			| patched
		perf(MB/s)+irq time(us)		| perf(MB/s)+irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  15.840/15.580(avg:158,max:1216)	| 16.500/16.160(avg:15,max:139)
Arndale board:  17.370/16.220(avg:33 max:234)	| 17.480/16.200(avg:11, max:91)
T410: 		21.180/19.820(avg:18 max:160)	| 21.220/19.880(avg:11, max:149)
---------------------------------------------------------------------

2.3 test case3: one uvc streaming test
- uvc device works well(on x86, luvcview can be used too and has
same result with uvc capture)
--------------------------------------------------------------------
		upstream 		| patched
		irq time(us)		| irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  (avg:445, max:873)	| (avg:33, max:44)
Arndale board:  (avg:316, max:630)	| (avg:20, max:27)
T410: 		(avg:39,  max:107)	| (avg:10, max:65)
---------------------------------------------------------------------

2.4 test case4: one uvc streaming plus one mass storage device test
--------------------------------------------------------------------
		upstream 		| patched
		perf(MB/s)+irq time(us)	| perf(MB/s)+irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  20.340(avg:259,max:1704)| 20.390(avg:24, max:101)
Arndale board:  23.460(avg:124,max:726)	| 23.370(avg:15, max:52)
T410: 		28.520(avg:27, max:169)	| 28.630(avg:13, max:160)
---------------------------------------------------------------------

2.5 test case5: read single mass storage device with small transfer
- run below command 10 times and compute the average speed

 dd if=/dev/sdN iflag=direct of=/dev/null bs=4K count=4000

1), test device A:
--------------------------------------------------------------------
		upstream 		| patched
		perf(MB/s)+irq time(us)	| perf(MB/s)+irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  6.5(avg:21, max:64)	| 6.5(avg:10, max:24)
Arndale board:  8.13(avg:12, max:23)	| 8.06(avg:7,  max:17)
T410: 		6.66(avg:13, max:131)   | 6.84(avg:11, max:149)
---------------------------------------------------------------------

2), test device B:
--------------------------------------------------------------------
		upstream 		| patched
		perf(MB/s)+irq time(us)	| perf(MB/s)+irq time(us)
--------------------------------------------------------------------
Pandaboard A1:  5.5(avg:21,max:43)	| 5.49(avg:10, max:24)
Arndale board:  5.9(avg:12, max:22)	| 5.9(avg:7, max:17)
T410: 		5.48(avg:13, max:155)	| 5.48(avg:7, max:140)
---------------------------------------------------------------------

* On T410, sometimes read ehci status register in ehci_irq takes more
than 100us, and the problem has been reported on the link:

	http://marc.info/?t=137065867300001&r=1&w=2

Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-08-12 11:43:49 -07:00
Arnd Bergmann 70b55c2ad0 usb: ehci: mark unlink_empty_async_suspended() as __maybe_unused
Patch 4d053fdac3 "usb: ehci: unlink_empty_async_suspended() only used
with CONFIG_PM" tried to hide the unlink_empty_async_suspended function
inside of an #ifdef to work around an unused function warning.

Unfortunately that had the effect of introducing a new warning:

drivers/usb/host/ehci-q.c:1297:13: warning: 'unlink_empty_async_suspended'
	declared 'static' but never defined [-Wunused-function]

While we could add another #ifdef around the function declaration to avoid
this, a nicer solution is to mark it as __maybe_unused, which will let
gcc silently drop the function definition when it is not needed.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-28 14:49:53 -07:00
Tony Prisk 4d053fdac3 usb: ehci: unlink_empty_async_suspended() only used with CONFIG_PM
Compiling with !CONFIG_PM generates an unused function warning on
unlink_empty_async_suspended().

Enclose the function in a #ifdef CONFIG_PM

Signed-off-by: Tony Prisk <linux@prisktech.co.nz>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-28 11:18:54 -07:00
Alan Stern afc2c9a2c3 USB: EHCI: remove unused variable in unlink_empty_async()
This patch (as1669) removes the check_unlinks_later flag in ehci-hcd's
unlink_empty_async().  It wasn't being used for anything and should
have been removed in an earlier patch, but I forgot about it.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:36:32 -07:00
Alan Stern 214ac7a077 USB: EHCI: improve end_unlink_async()
This patch (as1665) changes the way ehci-hcd's end_unlink_async()
routine works in order to avoid recursive execution and to be more
efficient:

	Now when an IAA cycle ends, a new one gets started up right
	away (if it is needed) instead of waiting until the
	just-unlinked QH has been processed.

	The async_iaa list is renamed to async_idle, which better
	expresses its new purpose: It is now the list of QHs which are
	now completely idle and are waiting to be processed by
	end_unlink_async().

	A new flag is added to track whether an IAA cycle is in
	progress, because the list formerly known as async_iaa no
	longer stores the QHs waiting for the IAA to finish.

	The decision about how many QHs to process when an IAA cycle
	ends is now made at the end of the cycle, when we know the
	current state of the hardware, rather than at the beginning.
	This means a bunch of logic got moved from start_iaa_cycle()
	to end_unlink_async().

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:36:32 -07:00
Alan Stern 6e018751a3 USB: EHCI: convert singly-linked lists to list_heads
This patch (as1664) converts ehci-hcd's async_unlink, async_iaa, and
intr_unlink from singly-linked lists to standard doubly-linked
list_heads.  Originally it didn't seem necessary to use list_heads,
because items are always added to and removed from these lists in FIFO
order.  But now with more list processing going on, it's easier to use
the standard routines than continue with a roll-your-own approach.

I don't know if the code ends up being notably shorter, but the
patterns will be more familiar to any kernel hacker.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:35:05 -07:00
Alan Stern 7bc782d73c USB: EHCI: split needs_rescan into two flags
This patch (as1662) does some more QH-related cleanup in ehci-hcd.
The qh->needs_rescan flag is currently used for two different
purposes; the patch replaces it with two separate flags for greater
clarity: qh->dequeue_during_giveback indicates that a completion
handler dequeued an URB (implying that a rescan is needed), and
qh->exception indicates that the QH is in an exceptional state
requiring an unlink (either it encountered an I/O error or an unlink
was requested).

The new flags get set where the dequeue, exception, or unlink request
occurred, rather than where the unlink is started.  This is so that in
the future, if we need to, we will be able to tell apart unlinks that
truly were required from those that were carried out merely because
the QH wasn't being used.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:35:05 -07:00
Alan Stern 79bcf7b02b USB: EHCI: change return value of qh_completions()
This patch (as1658) cleans up the usage of qh_completions() in
ehci-hcd.  Currently the function's return value indicates whether any
URBs were given back; the idea was that the caller can scan the QH
over again to handle any URBs that were dequeued by a completion
handler.  This is not necessary; when qh_completions() is ready to
give back dequeued URBs, it does its own rescanning.

Therefore the new return value will be a flag indicating whether the
caller needs to unlink the QH.  This is more convenient than forcing
the caller to check qh->needs_rescan, and it makes a lot more sense --
why should "needs_rescan" imply that an unlink is needed?  The callers
are also changed to remove the unneeded rescans.

Lastly, the check for whether qh->qtd_list is non-empty is removed
from the start of qh_completions().  Two of the callers have to make
this test anyway, so the same test can simply be added to the other
two callers.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:35:05 -07:00
Alan Stern c1fdb68e3d USB: EHCI: changes related to qh_refresh()
This patch (as1638) makes several changes to the ehci-hcd driver, all
related to the qh_refresh() function.  This function must be called
whenever an idle QH gets linked back into either the async or the
periodic schedule.

	Change a BUG_ON() in the qh_update routine to a WARN_ON().
	Since this code runs in atomic context, a BUG_ON() would
	immediately freeze the whole system.

	Remove two unneeded calls to qh_refresh(), one when a QH is
	initialized and one when a QH becomes idle.  Adjust the
	adjacent comments accordingly.

	Move the qh_refresh() and qh_link_periodic() calls for new
	interrupt URBs to after the new TDs have been added.

	As a result of the previous two changes, qh_refresh() is never
	called when the qtd_list is empty.  The corresponding check in
	qh_refresh() can be removed, along with an indentation level.

These changes should not cause any alteration of behavior.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-25 13:35:05 -07:00
Alan Stern 2a40f32454 USB: EHCI: fix regression during bus resume
This patch (as1663) fixes a regression caused by commit
6e0c3339a6 (USB: EHCI: unlink one async
QH at a time).  In order to avoid keeping multiple QHs in an unusable
intermediate state, that commit changed unlink_empty_async() so that
it unlinks only one empty QH at a time.

However, when the EHCI root hub is suspended, _all_ async QHs need to
be unlinked.  ehci_bus_suspend() used to do this by calling
unlink_empty_async(), but now this only unlinks one of the QHs, not
all of them.

The symptom is that when the root hub is resumed, USB communications
don't work for some period of time.  This is because ehci-hcd doesn't
realize it needs to restart the async schedule; it assumes that
because some QHs are already on the schedule, the schedule must be
running.

The easiest way to fix the problem is add a new function that unlinks
all the async QHs when the root hub is suspended.

This patch should be applied to all kernels that have the 6e0c3339a6
commit.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-and-tested-by: Adrian Bassett <adrian.bassett@hotmail.co.uk>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-15 12:07:53 -07:00
Alan Stern feca7746d5 USB: EHCI: don't check DMA values in QH overlays
This patch (as1661) fixes a rather obscure bug in ehci-hcd.  In a
couple of places, the driver compares the DMA address stored in a QH's
overlay region with the address of a particular qTD, in order to see
whether that qTD is the one currently being processed by the hardware.
(If it is then the status in the QH's overlay region is more
up-to-date than the status in the qTD, and if it isn't then the
overlay's value needs to be adjusted when the QH is added back to the
active schedule.)

However, DMA address in the overlay region isn't always valid.  It
sometimes will contain a stale value, which may happen by coincidence
to be equal to a qTD's DMA address.  Instead of checking the DMA
address, we should check whether the overlay region is active and
valid.  The patch tests the ACTIVE bit in the overlay, and clears this
bit when the overlay becomes invalid (which happens when the
currently-executing URB is unlinked).

This is the second part of a fix for the regression reported at:

	https://bugs.launchpad.net/bugs/1088733

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: Joseph Salisbury <joseph.salisbury@canonical.com>
Reported-and-tested-by: Stephen Thirlwall <sdt@dr.com>
CC: <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-05 08:45:33 +08:00
Alan Stern 6402c796d3 USB: EHCI: work around silicon bug in Intel's EHCI controllers
This patch (as1660) works around a hardware problem present in some
(if not all) Intel EHCI controllers.  After a QH has been unlinked
from the async schedule and the corresponding IAA interrupt has
occurred, the controller is not supposed access the QH and its qTDs.
There certainly shouldn't be any more DMA writes to those structures.
Nevertheless, Intel's controllers have been observed to perform a
final writeback to the QH's overlay region and to the most recent qTD.
For more information and a test program to determine whether this
problem is present in a particular controller, see

	http://marc.info/?l=linux-usb&m=135492071812265&w=2
	http://marc.info/?l=linux-usb&m=136182570800963&w=2

This patch works around the problem by always waiting for two IAA
cycles when unlinking an async QH.  The extra IAA delay gives the
controller time to perform its final writeback.

Surprisingly enough, the effects of this silicon bug have gone
undetected until quite recently.  More through luck than anything
else, it hasn't caused any apparent problems.  However, it does
interact badly with the path that follows this one, so it needs to be
addressed.

This is the first part of a fix for the regression reported at:

	https://bugs.launchpad.net/bugs/1088733

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Tested-by: Stephen Thirlwall <sdt@dr.com>
CC: <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-03-05 08:45:33 +08:00
Alan Stern 6e0c3339a6 USB: EHCI: unlink one async QH at a time
This patch (as1648) fixes a regression affecting nVidia EHCI
controllers.  Evidently they don't like to have more than one async QH
unlinked at a time.  I can't imagine how they manage to mess it up,
but at least one of them does.

The patch changes the async unlink logic in two ways:

	Each time an IAA cycle is started, only the first QH on the
	async unlink list is handled (rather than all of them).

	Async QHs do not all get unlinked as soon as they have been
	empty for long enough.  Instead, only the last one (i.e., the
	one that has been on the schedule the longest) is unlinked,
	and then only if no other unlinks are in progress at the time.

This means that when multiple QHs are empty, they won't be unlinked as
quickly as before.  That's okay; it won't affect correct operation of
the driver or add an excessive load.  Multiple unlinks tend to be
relatively rare in any case.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-and-tested-by: Piergiorgio Sartor <piergiorgio.sartor@nexgo.de>
Cc: stable <stable@vger.kernel.org> # 3.6
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-01-25 13:58:20 -08:00
Alan Stern 2656a9abcf USB: EHCI: bugfix: urb->hcpriv should not be NULL
This patch (as1632b) fixes a bug in ehci-hcd.  The USB core uses
urb->hcpriv to determine whether or not an URB is active; host
controller drivers are supposed to set this pointer to a non-NULL
value when an URB is queued.  However ehci-hcd sets it to NULL for
isochronous URBs, which defeats the check in usbcore.

In itself this isn't a big deal.  But people have recently found that
certain sequences of actions will cause the snd-usb-audio driver to
reuse URBs without waiting for them to complete.  In the absence of
proper checking by usbcore, the URBs get added to their endpoint list
twice.  This leads to list corruption and a system freeze.

The patch makes ehci-hcd assign a meaningful value to urb->hcpriv for
isochronous URBs.  Improving robustness always helps.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: Artem S. Tashkinov <t.artem@lycos.com>
Reported-by: Christof Meerwald <cmeerw@cmeerw.org>
CC: <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-11-11 18:10:45 -08:00
Pavankumar Kondeti 3d037774b4 EHCI: Update qTD next pointer in QH overlay region during unlink
There is a possibility of QH overlay region having reference to a stale
qTD pointer during unlink.

Consider an endpoint having two pending qTD before unlink process begins.
The endpoint's QH queue looks like this.

qTD1 --> qTD2 --> Dummy

To unlink qTD2, QH is removed from asynchronous list and Asynchronous
Advance Doorbell is programmed.  The qTD1's next qTD pointer is set to
qTD2'2 next qTD pointer and qTD2 is retired upon controller's doorbell
interrupt.  If QH's current qTD pointer points to qTD1, transfer overlay
region still have reference to qTD2. But qtD2 is just unlinked and freed.
This may cause EHCI system error.  Fix this by updating qTD next pointer
in QH overlay region with the qTD next pointer of the current qTD.

Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-09-10 16:01:37 -07:00
Alan Stern 18aafe64d7 USB: EHCI: use hrtimer for the I/O watchdog
This patch (as1586) replaces the kernel timer used by ehci-hcd as an
I/O watchdog with an hrtimer event.

Unlike in the current code, the watchdog event is now always enabled
whenever any isochronous URBs are active.  This will prevent bugs
caused by the periodic schedule wrapping around with no completion
interrupts; the watchdog handler is guaranteed to scan the isochronous
transfers at least once during each iteration of the schedule.  The
extra overhead will be negligible: one timer interrupt every 100 ms.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:56:47 -07:00
Alan Stern 569b394f53 USB: EHCI: always scan each interrupt QH
This patch (as1585) fixes a bug in ehci-hcd's scheme for scanning
interrupt QHs.

Currently a single routine takes care of scanning everything on the
periodic schedule.  Whenever an interrupt occurs, it scans all
isochronous and interrupt URBs scheduled for frames that have elapsed
since the last scan.

This has two disadvantages.  The first is relatively minor: An
interrupt QH is likely to end up getting scanned multiple times,
particularly if the last scan was not fairly recent.  (The current
code avoids this by maintaining a periodic_stamp in each interrupt
QH.)

The second is more serious.  The periodic schedule wraps around.  If
the last scan occurred during frame N, and the next scan occurs when
the schedule has gone through an entire cycle and is back at frame N,
the scanning code won't look at any frames other than N.  Consequently
it won't see any QHs that completed during frame N-1 or earlier.

The patch replaces the entire frame-based approach for scanning
interrupt QHs with a new routine using a list-based approach, the same
as for async QHs.  This has a slight disadvantage, because it means
that all interrupt QHs have to be scanned every time.  But it is more
robust than the current approach.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:56:47 -07:00
Alan Stern 32830f2076 USB: EHCI: use hrtimer for unlinking empty async QHs
This patch (as1583) changes ehci-hcd to use an hrtimer event for
unlinking empty (unused) async QHs instead of using a kernel timer.

The check for empty QHs is moved to a new routine, where it doesn't
require going through an entire scan of both the async and periodic
schedules.  And it can unlink multiple QHs at once, unlike the current
code.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:56:47 -07:00
Alan Stern 3c273a056b USB: EHCI: unlink multiple async QHs together
This patch (as1582) changes ehci-hcd's strategy for unlinking async
QHs.  Currently the driver never unlinks more than one QH at a time.
This can be inefficient and cause unnecessary delays, since a QH
cannot be reused while it is waiting to be unlinked.

The new strategy unlinks all the waiting QHs at once.  In practice the
improvement won't be very big, because it's somewhat uncommon to have
two or more QHs waiting to be unlinked at any time.  But it does
happen, and in any case, doing things this way makes more sense IMO.

The change requires the async unlinking code to be refactored
slightly.  Now in addition to the routines for starting and ending an
unlink, there are new routines for unlinking a single QH and starting
an IAA cycle.  This approach is needed because there are two separate
paths for unlinking async QHs:

	When a transfer error occurs or an URB is cancelled, the QH
	must be unlinked right away;

	When a QH has been idle sufficiently long, it is unlinked
	to avoid consuming DMA bandwidth uselessly.

In the first case we want the unlink to proceed as quickly as
possible, whereas in the second case we can afford to batch several
QHs together and unlink them all at once.  Hence the division of
labor.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:56:47 -07:00
Alan Stern 9d9387475a USB: EHCI: use hrtimer for the IAA watchdog
This patch (as1581) replaces the iaa_watchdog kernel timer used by
ehci-hcd with an hrtimer event, in keeping with the general conversion
to high-res timers.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:56:46 -07:00
Alan Stern 314466101c USB: EHCI: use hrtimer for async schedule
This patch (as1576) adds hrtimer support for managing ehci-hcd's
async schedule.  Just as with the earlier change to the periodic
schedule management, two new hrtimer events take care of everything.

One event polls at 1-ms intervals to see when the Asynchronous
Schedule Status (ASS) flag matches the Asynchronous Schedule Enable
(ASE) value; the schedule's state must not be changed until it does.
The other event delays for 15 ms after the async schedule becomes
empty before turning it off.

The new events replace a busy-wait poll and a kernel timer usage.
They also replace the rather illogical method currently used for
indicating the async schedule should be turned off: attempting to
unlink the dedicated QH at the head of the async list.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:54:25 -07:00
Alan Stern c0c53dbc32 USB: EHCI: add new root-hub state: STOPPING
This patch (as1571) adds a new state for ehci-hcd's root hubs:
EHCI_RH_STOPPING.  This value is used at times when the root hub is
being stopped and we don't know whether or not the hardware has
finished all its DMA yet.

Although the purpose may not be apparent, this distinction will come
in useful later on.  Future patches will avoid actions that depend on
the root hub being operational (like turning on the async or periodic
schedules) when they see the state is EHCI_RH_STOPPING.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:50:14 -07:00
Alan Stern 2f5bb665ba USB: EHCI: add pointer to end of async-unlink list
This patch (as1570) adds a pointer for the end of ehci-hcd's
async-unlink list.  The list (which is actually a queue) is singly
linked, so having a pointer to its end makes adding new entries easier
-- there's no longer any need to scan through the whole list.

In principle it could be changed to a standard doubly-linked list.  It
turns out that doing so actually makes the code less clear, so I'm
leaving it as is.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:50:13 -07:00
Alan Stern 99ac5b1e95 USB: EHCI: rename "reclaim"
This patch (as1569) renames the ehci->reclaim list in ehci-hcd.  The
word "reclaim" is used in the EHCI specification to mean something
quite different, and "unlink_next" is more descriptive of the list's
purpose anyway.

Similarly, the "reclaim" field in the ehci_stats structure is renamed
"iaa", which is more meaningful (to experts, anyway) and is a better
match for the "lost_iaa" field.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:50:13 -07:00
Alan Stern 4c53de7210 USB: EHCI: add symbolic constants for QHs
This patch (as1568) introduces symbolic constants for some of the
less-frequently used bitfields in the QH structure.  This makes the
code a little easier to read and understand.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:50:13 -07:00
Alan Stern c83e1a9ff6 USB: EHCI: don't refcount QHs
This patch (as1567) removes ehci-hcd's reference counting of QH
structures.  It's not necessary to refcount these things because they
always get deallocated at exactly one spot in ehci_endpoint_disable()
(except for two special QHs, ehci->async and ehci->dummy) and are
never used again.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-07-16 16:50:13 -07:00
Greg Kroah-Hartman 82491c2a20 USB: ehci-q.c: remove dbg() usage
dbg() was a very old USB-specific macro that should no longer
be used. This patch removes it from being used in the driver
and uses dev_dbg() instead.

CC: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-05-01 21:33:35 -07:00
Alan Stern 3d9545cc37 EHCI: maintain the ehci->command value properly
The ehci-hcd driver is a little haphazard about keeping track of the
state of the USBCMD register.  The ehci->command field is supposed to
hold the register's value (apart from a few special bits) at all
times, but it isn't maintained properly.

This patch (as1543) cleans up the situation.  It keeps ehci->command
up-to-date, and uses that value rather than reading the register from
the hardware whenever possible.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-04-23 12:05:44 -07:00
Clemens Ladisch bc677d5b64 usb: fix number of mapped SG DMA entries
Add a new field num_mapped_sgs to struct urb so that we have a place to
store the number of mapped entries and can also retain the original
value of entries in num_sgs.  Previously, usb_hcd_map_urb_for_dma()
would overwrite this with the number of mapped entries, which would
break dma_unmap_sg() because it requires the original number of entries.

This fixes warnings like the following when using USB storage devices:
 ------------[ cut here ]------------
 WARNING: at lib/dma-debug.c:902 check_unmap+0x4e4/0x695()
 ehci_hcd 0000:00:12.2: DMA-API: device driver frees DMA sg list with different entry count [map count=4] [unmap count=1]
 Modules linked in: ohci_hcd ehci_hcd
 Pid: 0, comm: kworker/0:1 Not tainted 3.2.0-rc2+ #319
 Call Trace:
  <IRQ>  [<ffffffff81036d3b>] warn_slowpath_common+0x80/0x98
  [<ffffffff81036de7>] warn_slowpath_fmt+0x41/0x43
  [<ffffffff811fa5ae>] check_unmap+0x4e4/0x695
  [<ffffffff8105e92c>] ? trace_hardirqs_off+0xd/0xf
  [<ffffffff8147208b>] ? _raw_spin_unlock_irqrestore+0x33/0x50
  [<ffffffff811fa84a>] debug_dma_unmap_sg+0xeb/0x117
  [<ffffffff8137b02f>] usb_hcd_unmap_urb_for_dma+0x71/0x188
  [<ffffffff8137b166>] unmap_urb_for_dma+0x20/0x22
  [<ffffffff8137b1c5>] usb_hcd_giveback_urb+0x5d/0xc0
  [<ffffffffa0000d02>] ehci_urb_done+0xf7/0x10c [ehci_hcd]
  [<ffffffffa0001140>] qh_completions+0x429/0x4bd [ehci_hcd]
  [<ffffffffa000340a>] ehci_work+0x95/0x9c0 [ehci_hcd]
  ...
 ---[ end trace f29ac88a5a48c580 ]---
 Mapped at:
  [<ffffffff811faac4>] debug_dma_map_sg+0x45/0x139
  [<ffffffff8137bc0b>] usb_hcd_map_urb_for_dma+0x22e/0x478
  [<ffffffff8137c494>] usb_hcd_submit_urb+0x63f/0x6fa
  [<ffffffff8137d01c>] usb_submit_urb+0x2c7/0x2de
  [<ffffffff8137dcd4>] usb_sg_wait+0x55/0x161

Signed-off-by: Clemens Ladisch <clemens@ladisch.de>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-12-09 16:18:19 -08:00
Vikram Pandita 332960bd7e USB: ehci: report Data Buffer Error in debug mode
Data Buffer Error as per spec section 4.15.1.1.2
results when there is Underrun or Overrun condition.

This error is considered non-fatal and never gets reported.
Its a very good indication on things going wrong at system level,
like running memory at much slower speed.

This is a good error to flag allowing system level corrections.

An issue was found with OMAP4460 board where DDR had to be run
at full speed and this logging helped.

Signed-off-by: Vikram Pandita <vikram.pandita@ti.com>
Reviewed-by: Marek Vasut <marek.vasut@gmail.com>
Signed-off-by: Vikram Pandita <vikram.pandita@ti.com>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-11-14 16:46:07 -08:00
Ming Lei 41f05dedea usb: ehci: remove the 1st wmb in qh_append_tds
According to ehci spec 4.10.2, Advance Queue

	If the fetched qTD has its Active bit set to a zero, the
	host controller aborts the queue advance and follows the
	queue head's horizontal pointer to the next schedule data
	structure.

the 'qtd' will be linked into qh hardware queue after the line
below

	*dummy = *qtd;

is executed and observed by EHCI HC, but EHCI HC won't have chance to
fetch the qtd descriptor pointed by 'qtd' in qh_append_tds until the
line below

	dummy->hw_token = token;	#set Active bit here

is executed by CPU and observed by EHCI HC.

There is already one 'wmb' to order writing to 'dummy'/'qtd' descriptors
and writing 'token' to 'dummy' descriptor(set Active bit), so the 1st
wmb is not needed and can be removed.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-09-18 01:38:59 -07:00
Ming Lei 9a971dda82 usb: ehci: only prepare zero packet for out transfer if required
Obviously, ZLP is only required for transfer of OUT direction,
so just take same policy with UHCI for ZLP packet.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-09-18 01:38:59 -07:00
Ming Lei 0412560e18 usb: ehci: remove wmb in qh_update
qh_refresh is always called when the qh is idle and has not been
linked into hardware queue, so EHCI will not access overlay of
the qh at this time. Just before linking qh into hardware queue, there
has already one wmb to order writing qh descriptor and writing dma
address of the qh into hardware queue, so HC can always see
up-to-date qh descriptor once the qh is fetched with its dma address
by EHCI.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-09-18 01:38:59 -07:00
Alan Stern e879990604 USB: EHCI: remove usages of hcd->state
This patch (as1483) improves the ehci-hcd driver family by getting rid
of the reliance on the hcd->state variable.  It has no clear owner and
it isn't protected by the usual HCD locks.  In its place, the patch
adds a new, private ehci->rh_state field to record the state of the
root hub.

Along the way, the patch removes a couple of lines containing
redundant assignments to the state variable.  Also, the QUIESCING
state simply gets changed to the RUNNING state, because the driver
doesn't make any distinction between them.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Acked-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2011-08-22 15:57:01 -07:00