Add a warning message to indicate obsolete/unsupported
BE2 Adapter Family devices
Signed-off-by: Ketan Mukadam <ketan.mukadam@avagotech.com>
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
After TPE recovery, CID table needs to be repopulated as per CIDs in
WRBQ creation responses.
SGL handles table needs to be recreated for posting and its indices need
to be resetted.
This is achieved by calling beiscsi_cleanup_port when disabling and
beiscsi_init_port in enabling port.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Set CID slot to 0xffff to indicate empty.
Check if connection already exists in conn_table before binding.
Check if endpoint already NULL before putting back CID.
Break ep->conn link in free_ep to ignore completions after freeing.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
wq_name is used only to set WQ name when its being allocated.
Remove it from beiscsi_hba structure and define locally.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Fix errors reported in static analysis.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This duplicate code got added during manual merging.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Prepare the IOCTL with appropriate sizes of buffers of V0 and V1.
Set missing chute number in V1 IOCTL.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Added check in beiscsi_process_cq for pio_handle.
pio_handle is cleared in beiscsi_put_wrb_handle.
This catches any case where task gets cleaned up just before completion.
Use back_lock before accessing pio_handle.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
invalid bit in WRB indicates to FW that IO was invalidated before WRB
was fetched from host memory.
For SkyHawk, this invalid bit in WRB is at a different offset.
Use amap_iscsi_wrb_v2 to mark invalid bit for SkyHawk.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Hold the reference of iscsi_task till invalidation completes.
This prevents use of ICD when invalidation of that ICD is being processed.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
System crashes when sg_reset is executed in a loop.
CPU: 13 PID: 7073 Comm: sg_reset Tainted: G E 4.8.0-rc1+ #4
RIP: 0010:[<ffffffffa0825370>] [<ffffffffa0825370>]
beiscsi_eh_device_reset+0x160/0x520 [be2iscsi]
Call Trace:
[<ffffffff814c7c77>] ? scsi_host_alloc_command+0x47/0xc0
[<ffffffff814caafa>] scsi_try_bus_device_reset+0x2a/0x50
[<ffffffff814cb46e>] scsi_ioctl_reset+0x13e/0x260
[<ffffffff814ca477>] scsi_ioctl+0x137/0x3d0
[<ffffffffa05e4ba2>] sg_ioctl+0x572/0xc20 [sg]
[<ffffffff8123f627>] do_vfs_ioctl+0xa7/0x5d0
The accesses to beiscsi_io_task is being protected in device reset handler
with frwd_lock but the freeing of task can happen under back_lock.
Hold the reference of iscsi_task till invalidation completes.
This prevents use of ICD when invalidation of that ICD is being processed.
Use frwd_lock for iscsi_tasks looping and back_lock to access
beiscsi_io_task structures.
Rewrite mgmt_invalidation_icds to handle allocation and freeing of IOCTL
buffer in one place.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Remove shared structure inv_tbl in phba for all sessions to post
invalidation IOCTL.
Always allocate and then free the table after use in reset handler.
Abort handler needs just one instance so define it on stack.
Add checks for BE_INVLDT_CMD_TBL_SZ to not exceed invalidation
command table size in IOCTL.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This update includes the usual round of major driver updates (ncr5380,
lpfc, hisi_sas, megaraid_sas, ufs, ibmvscsis, mpt3sas). There's also
an assortment of minor fixes, mostly in error legs or other not very
user visible stuff. The major change is the pci_alloc_irq_vectors
replacement for the old pci_msix_.. calls; this effectively makes IRQ
mapping generic for the drivers and allows blk_mq to use the
information.
Signed-off-by: James E.J. Bottomley <jejb@linux.vnet.ibm.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAABAgAGBQJYUJOvAAoJEAVr7HOZEZN42B0P/1lj1W2N7y0LOAbR2MasyQvT
fMD/SSip/v+R/zJiTv+5M/IDQT6ez62JnQGWyX3HZTob9VEfoqagbUuHH6y+fmib
tHyqiYc9FC/NZSRX/0ib+rpnSVhC/YRSVV7RrAqilbpOKAzeU25FlN/vbz+Nv/XL
ReVBl+2nGjJtHyWqUN45Zuf74c0zXOWPPUD0hRaNclK5CfZv5wDYupzHzTNSQTkj
nWvwPYT0OgSMNe7mR+IDFyOe3UQ/OYyeJB0yBNqO63IiaUabT8/hgrWR9qJAvWh8
LiH+iSQ69+sDUnvWvFjuls/GzcFuuTljwJbm+FyTsmNHONPVY8JRCLNq7CNDJ6Vx
HwpNuJdTSJpne4lAVBGPwgjs+GhlMvUP/xYVLWAXdaBxU9XGePrwqQDcFu1Rbx3P
yfMiVaY1+e45OEjLRCbDAwFnMPevC3kyymIvSsTySJxhTbYrOsyrrWt5kwWsvE3r
SKANsub+xUnpCkyg57nXRQStJSCiSfGIDsydKmMX+pf1SR4k6gCUQZlcchUX0uOa
dcY6re0c7EJIQQiT7qeGP5TRBblxARocCA/Igx6b5U5HmuQ48tDFlMCps7/TE84V
JBsBnmkXcEi/ALShL/Tui+3YKA1DfOtEnwHtXx/9Ecx/nxP2Sjr9LJwCKiONv8NY
RgLpGfccrix34lQumOk5
=sPXh
-----END PGP SIGNATURE-----
Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
Pull SCSI updates from James Bottomley:
"This update includes the usual round of major driver updates (ncr5380,
lpfc, hisi_sas, megaraid_sas, ufs, ibmvscsis, mpt3sas).
There's also an assortment of minor fixes, mostly in error legs or
other not very user visible stuff. The major change is the
pci_alloc_irq_vectors replacement for the old pci_msix_.. calls; this
effectively makes IRQ mapping generic for the drivers and allows
blk_mq to use the information"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (256 commits)
scsi: qla4xxx: switch to pci_alloc_irq_vectors
scsi: hisi_sas: support deferred probe for v2 hw
scsi: megaraid_sas: switch to pci_alloc_irq_vectors
scsi: scsi_devinfo: remove synchronous ALUA for NETAPP devices
scsi: be2iscsi: set errno on error path
scsi: be2iscsi: set errno on error path
scsi: hpsa: fallback to use legacy REPORT PHYS command
scsi: scsi_dh_alua: Fix RCU annotations
scsi: hpsa: use %phN for short hex dumps
scsi: hisi_sas: fix free'ing in probe and remove
scsi: isci: switch to pci_alloc_irq_vectors
scsi: ipr: Fix runaway IRQs when falling back from MSI to LSI
scsi: dpt_i2o: double free on error path
scsi: cxlflash: Migrate scsi command pointer to AFU command
scsi: cxlflash: Migrate IOARRIN specific routines to function pointers
scsi: cxlflash: Cleanup queuecommand()
scsi: cxlflash: Cleanup send_tmf()
scsi: cxlflash: Remove AFU command lock
scsi: cxlflash: Wait for active AFU commands to timeout upon tear down
scsi: cxlflash: Remove private command pool
...
Variable ret is reset in the loop, and its value will be 0 during the
second and after repeat of the loop. If pci_alloc_consistent() returns a
NULL pointer then, it will leaves with return value 0. 0 means no error,
which is contrary to the fact. This patches fixes the bug, explicitly
assigning "-ENOMEM" to return variable ret on the path that the call to
pci_alloc_consistent() fails.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188951
Signed-off-by: Pan Bian <bianpan2016@163.com>
Reviewed-by: Jitendra Bhivare <Jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Variable ret is reset in the loop, and its value will be 0 during the
second and after repeat of the loop. If pci_alloc_consistent() returns a
NULL pointer then, it will leaves with return value 0. 0 means no error,
which is contrary to the fact. This patches fixes the bug, explicitly
assigning "-ENOMEM" to return variable ret on the path that the call to
pci_alloc_consistent() fails.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188941
Signed-off-by: Pan Bian <bianpan2016@163.com>
Reviewed-by: Jitendra Bhivare <Jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We accidentally allocate sizeof(u32) instead of sizeof(struct
be_cmd_get_session_resp).
Fixes: 50a4b824be ("scsi: be2iscsi: Fix to make boot discovery non-blocking")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We get 6 warnings when building kernel with W=1:
drivers/scsi/be2iscsi/be_main.c:65:1: warning: no previous prototype for 'beiscsi_log_enable_disp' [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:78:1: warning: no previous prototype for 'beiscsi_log_enable_change' [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:97:1: warning: no previous prototype for 'beiscsi_log_enable_store' [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:116:1: warning: no previous prototype for 'beiscsi_log_enable_init' [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:4587:5: warning: no previous prototype for 'beiscsi_iotask_v2' [-Wmissing-prototypes]
drivers/scsi/be2iscsi/be_main.c:4976:6: warning: no previous prototype for 'beiscsi_hba_attrs_init' [-Wmissing-prototypes]
In fact, these functions are only used in the file in which they are
declared and don't need a declaration, but can be made static. So this
patch marks these functions with 'static'.
Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Reviewed-by: Jitendra Bhivare <Jitendra.bhivare@broadcom>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Julia pointed out beiscsi_boot_get_sinfo does not unlock mbox_lock on
nonemb_cmd memory allocation failure.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
alloc_wrb_handle already does memset zero of iscsi_wrb descriptor so
remove redundant memset in WRB submission paths.
Add pwrb_handle NULL check before memsett'ing pwrb.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We know that 'ret' is not an error code because it has been tested a few
lines above. So, if one of these function fails, 0 will be returned
instead of an error code. Return -ENOMEM instead.
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
'ret' needs to be set with error code if hba_setup_cid_tbls fails.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Current EQ delay is set to 0 to receive very high max interrupt per sec.
Set EQ delay to 32 - reducing max interrupt rate from 65K to 20K per sec.
Set TCP connection window size to 64K with scale shift count 2.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In very rare scenario, connection gets killed after throwing this error:
scsi host0: BM_2312 : Event CXN_KILLED_BAD_WRB_INDEX_ERROR[15]... CID : 4
connection1:0: detected conn error (1011)
memset ISCSI_WRB descriptor to zero for all allocations of WRB handle.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
BUG: unable to handle kernel NULL pointer dereference at 000000000000015e
IP: [<ffffffffa0081700>]
hwi_get_async_handle.isra.23.constprop.39+0x90/0x1d0 [be2iscsi]
PGD 0
Oops: 0000 [#1] SMP
...
Call Trace:
<IRQ>
[<ffffffffa00818bc>] hwi_process_default_pdu_ring+0x7c/0x280 [be2iscsi]
[<ffffffffa0088f51>] beiscsi_process_cq+0x321/0xb90 [be2iscsi]
[<ffffffff810af028>] ? __wake_up_common+0x58/0x90
[<ffffffff810b0d84>] ? __wake_up+0x44/0x50
[<ffffffffa0089a2d>] be_iopoll+0x1d/0xb0 [be2iscsi]
[<ffffffff812d1f61>] blk_iopoll_softirq+0xc1/0x100
[<ffffffff81084b0f>] __do_softirq+0xef/0x280
The symptom observed is multiple async handles get queued for same index
thus causing leak in buffers posted to FW.
The root cause is:
- async handle is continued to be used even if it does not match the
completion.
- list_move operation done on already filled index.
1. Remove use of writables, host_write_ptr and ep_read_ptr.
2. Remove consumed logic to update writables. Instead, use only
free_entries to do the accounting of handles to be posted back.
3. Remove busy_list, instead use simple slot to index handles.
4. Added check no data, header less and overflow to make sure
all async_handles are flushed in error cases.
5. Added code to verify gathering of handles to form PDU by
checking final bit before forwarding PDU.
6. Added code to catch mismatch with CQE and handle gracefully.
7. Use AMAP, traverse cri_wait_queue list to post buffers, log
"async PDU" related errors.
8. Rearranged few data structures and added comments in init &
processing path.
9. Added WARN_ONs to catch any HD ring corruption.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Driver unload should call COMMON_FUNCTION_RESET. For TPE feature, this
ensures that FW has knowledge about driver getting unloaded and can
reset its bit vector.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Sessions are no longer valid, so schedule sess_work to fail the sessions
immediately when error is detected. This is done to avoid iSCSI transport
layer to keep sending NOP-Out which driver any ways fail.
Schedule sess_work immediately in case of HBA error. Old sessions are gone
for good and need to be re-established.
iscsi_session_failure needs process context hence this work.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
After UE is detected, check for recoverable error by reading
SLIPORT SEMAPHORE register. If transient parity error i.e. 0xExxx
then schedule recovery work on driver wq.
FLag this error to prevent any transactions for the duration of ue2rp to
restart polling. After that, if FW becomes ready then recover port.
Wake up processes in wq before going offline.
Wait for process to execute before cleaning up.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
mgmt_epfw_cleanup does not implement v1 of OPCODE_COMMON_ISCSI_CLEANUP
IOCTL for SkyHawk.
Replace use of MCCQ with BMBX for issuing the IOCTL.
Remove be_mcc_compl_poll which is no longer needed.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
SLIPORT FUNCTION_RESET does not reset the chip.
So POST status needs to be checked before issuing FUNCTION_RESET.
The completion of FUNCTION_RESET is indicated in BMBX Rdy bit.
be_cmd_fw_initialize too needs to be done before issuing any cmd to FW.
be_cmd_fw_initialize is renamed as beiscsi_cmd_special_wrb.
Rearrange and rename few functions in init and cleanup path.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
beiscsi_fail_session is defined in be_cmds.c: move it to be_iscsi.c
Move card configuration commands to be_cmds.c.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
BE3 and SH cards can recover from transient parity errors treated
earlier as unrecoverable errors.
Add IOCTL to query FW support for this feature.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
UE detection in health check is done in a work scheduled in global wq.
UE caused due to transient parity errors are recoverable and reported
within 1s. If this check for TPE gets delayed, PF0 might initiate
soft-reset and then status of UE recoverable is lost.
Handle UE detection in timer routine. Move out EQ delay update work
from health check. Make the IOCTL for EQ delay update non-blocking
as the completion status is ignored.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Boot work involves:
1. Find and fetch configured boot session and its handle.
2. Attempt to open the session if its not.
3. Get the session details for boot kset creation.
4. Logout of that session owned by FW.
5. Create boot kset for session details.
All these actions were done in blocking call with retries in global wq.
Other works in wq suffered if the IOCTLs stalled or timed out.
This change moves all the boot work to make it non-blocking.
The work queued in global wq just issues the IOCTL depending on the action
to be taken and mcc wq schedules work depending on status of the IOCTL.
Initial boot_work is started on link and ASYNC event.
The other code changes move all boot related functions in one place and
follow naming conventions.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Save ue_detected and fw_timeout errors in state field of beiscsi_hba.
BEISCSI_HBA_RUNNING
BEISCSI_HBA_LINK_UP
BEISCSI_HBA_BOOT_FOUND
BEISCSI_HBA_PCI_ERR
BEISCSI_HBA_FW_TIMEOUT
BEISCSI_HBA_IN_UE
Make sure no PCI transaction happens once in error state.
Add checks in IO path to detect HBA in error.
Skip hwi_purge_eq step which can't be done in error state.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
todo_mcc_cq is not needed as only MCC work is queued.
todo_cq is not used at all.
Rename functions to be consistent.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
alloc_mcc_tag was replaced with alloc_mcc_wrb and is no more used.
beiscsi_pci_soft_reset is not used at all and won't be needed.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Redefine FW IP types.
Before issuing IOCTL to clear IP, check if IP is all zeroes.
All zeroes IP implies IP is not set in FW so FW fails that IOCTL.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Wrong settings displayed for iface:
iface.header_digest = 192.168.197.22
iface.data_digest = 255.255.255.0
iface.immediate_data = 192.168.197.1
Process ISCSI_NET_PARAM only in beiscsi_iface_get_param.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Rename mgmt_get_if_info to be consistent with APIs name.
Rename create/destroy APIs to indicate IFACE operations.
Remove legacy be2iscsi and use beiscsi.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Move mgmt_get_all_if_id before any set param operation.
Rename mgmt_get_all_if_id to beiscsi_if_get_handle.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
VLAN tag is L2 construct, move VLAN code out from configuring IP.
Rearrange and rename the APIs to make it consistent.
Replace ENOSYS with EPERM.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
If BOOTPROTO is changed to static, the DHCP IP address should be released.
All cases are being handled mgmt_set_ip and mgmt_static_ip_modify.
Rearrange IFACE APIs to:
beiscsi_if_clr_ip
beiscsi_if_set_ip
beiscsi_if_en_static
beiscsi_if_en_dhcp
This simplifies release of DHCP IP when BOOTPROTO is set to static.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Gateway APIs assume IP type as IPv4. Modify it to be generic to allow
clearing of IPv6 gateway set using BIOS.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
ipv4_iface and ipv6_iface fields need to be set to NULL when destroyed.
Before creation these are checked. Use these to report correct states.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Driver takes significant time to load 1m:20s and unload 40s.
Checkpatch script threw warning:
WARNING: msleep < 20ms can sleep for up to 20ms; see
Documentation/timers/timers-howto.txt
To eliminate this warning msleep(1) was replaced with msleep(20) before
submitting.
msleep(20) in init and uninit path for creation and destroying of number
of WRBQs, CQs, and EQs is adding to load/unload time.
Replace msleep with schedule_timeout_uninterruptible of 1ms as its
enough in most cases.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This got unnecessarily introduced with other changes in previous
commits. mcc_lock is taken only in process contexts.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Following configuration is created with what driver exports:
iface.vlan_id = 65535
iface.vlan_priority = 255
iface.vlan_state = <empty>
vlan_state is empty as iscsiadm doesn't process "Disabled".
When applying this configuration, iscsiadm checks for if vlan_state is
"disable" if not it enables with value in vlan_id. 65535 not being valid
value, 0 is applied.
Use "enable" or "disable" for ISCSI_NET_PARAM.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In beiscsi_setup_boot_info(), the boot_kset pointer should be set to
NULL in case of failure otherwise an invalid pointer dereference may
occur later.
Cc: <stable@vger.kernel.org>
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.
Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq;
@@
-synchronize_irq(irq);
free_irq(irq, ...);
// </smpl>
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Reviewed-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This adds the missing error check and path for if the call to the
function hwi_init_controller fails as this error path was clearly missed
when writing beiscsi_eeh_resume and thus we must add it now in order to
be able to handle this nonrecoverable failing function call gracefully
in beiscsi_eeh_resume.
Signed-off-by: Nicholas Krause <xerofoify@gmail.com>
Reviewed-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In case of error, the memory allocated for phwi_ctrlr was not freed.
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
FW got into UE after running IO stress test
With kernel change to split session lock in frwd_lock and back_lock for
tx and rx path correspondingly, in the IO path, common resource used in
driver such as WRB was left unprotected.
Add wrb_lock spinlock to protect allocation and freeing of WRB.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Processing of mgmt and IO tasks are done in process context and
softirqs.
Allocation and freeing of sgl_handles needs to be done under
spin_lock_bh/spin_unlock_bh and move the locks to the routines.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Connection resets observed from some targets when NOP-Out with wrong
ExpStatSn is sent.
FW keeps track of StatSn and fills up ExpStatSn accordingly. The header
filled up by the stack needs to be modified by driver to clear
ExpStatSn. If the field is not cleared, FW recalculates ExpStatSn and
wrong offset'ed ExpStatSn is seen in the wire trace.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
WARN_ON(atomic_read(&mccq->used) >= mccq->len) seen when FW gets into
UE.
MCCQ overflow is happening because driver discards any new request and
frees up the tag. The tag allocation controls the number of MCC WRB
posted. It is being replenished but WRBs are not hence the WARN_ON.
Allocation and freeing of WRB and tags for MCC is now done in one place.
This helps to achieve proper accounting of WRB indices and MCC tags.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In open with IP of unknown address family, only tag is freed and error
returned. MCC WRB allocated for the operation is not freed.
Added check for supported family of IP in the beginning before
allocating the tag and WRB.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Remove confusingly named be_mcc_compl_is_new and be_mcc_compl_use
functions in processing of BMBX. Rearrange beiscsi_process_mbox_compl
function.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
be_mcc_compl_poll waits till 'used' count of MCC WRBQ is zero. This is
to determine the completion of an MCC sent.
Change function to poll for the tag of MCC sent, instead, and wait till
its tag_state is cleared.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
be_mbox_notify_wait does exactly same thing as be_mbox_notify.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
be_mcc_compl_process_isr is removed. MCC CQ processing is done only in
beiscsi_process_mcc_cq and MCC CQE processing is done only in
beiscsi_process_mcc_compl.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Rename mcc_numtag to mcc_tag_status. MCC CQE status is processed using
macros already defined in be_cmds.h.
Add MCC_Q_WRB_ and MCC_Q_CMD_TAG_MASK macros to map to already defined
CQE_STATUS_ macros to be consistent when posting MCC.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
mcc_cq_lock spin_lock is used only in beiscsi_process_mcc which is
called only when all interrupts are disabled from mgmt_epfw_cleanup
during unloading of driver. There is no other context where there can be
contention for the processing of CQ.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@broadcom.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
hwi_write_buffer() does not check if mapping dma memory succeed. The
patch adds the check and failure handling.
Found by Linux Driver Verification project (linuxtesting.org).
Signed-off-by: Alexey Khoroshilov <khoroshilov@ispras.ru>
Reviewed-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Acked-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Login/Logout loop was hanging after few hours. /var/log/message showed
that alloc_wrb_handle() function was not able to allocate any new WRB.
Sep 11 11:25:22 Jhelum10 kernel: connection32513:0: Could not send nopout
Sep 11 11:25:22 Jhelum10 kernel: scsi host10: BM_4989 : Alloc of WRB_HANDLE
Failed for the CID : 384
Sep 11 11:25:22 Jhelum10 kernel: connection32513:0: Could not allocate pdu for
mgmt task.
Driver allocates WRB to pass login negotiated parameters information to FW
in beiscsi_offload_connection(). This allocated WRB was not freed so there
was WRB_Leak happening.
Put WRB used for posting the login-negotiated parameters back in pool.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Use only port_link_status only to determine link state change. Only bit
0 is used to get the state. Remove code for processing port_fault.
Fixed get_nic_conf structure definition. Removed rsvd[23] field in
be_cmd_get_nic_conf_resp.
Moved definitions of struct field values below the field.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Async link event provides port_speed info. Cache the port_speed info
and use the same to report in ISCSI_HOST_PARAM_PORT_SPEED query.
Removed link status query IOCTL used to do the same.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
OS not responding when running 2 port traffic on 72 CPUs system.
be2iscsi IRQs gets affined to CPU0 when irqbalancer is disabled.
be_iopoll processing completions in BLOCK_IOPOLL_SOFTIRQ hogged CPU0.
1. Use budget to exit the polling loop. beiscsi_process_cq didn't honour
it.
2. Rearming of EQ is done only after iopoll completes.
[mkp: Fixed up blk_iopoll -> irq_poll transition]
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Change return value of completed MCC EBUSY to EINVAL.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
System crash in I+T card personality.
Fix to add validation for ULP in initiator mode, physical port number,
and supported queue, icd, cid counts.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Log messages for misconfigured transceivers reported by FW.
Register async events that driver handles using MCC_CREATE_EXT ioctl.
Errors messages for faulted/uncertified/unqualified optics are logged.
Added IOCTL to get port_name to be displayed in error message.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Configuring VLAN parameters through IPv6 interface was not supported in
driver.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Null pointer dereference in shutdown path after taking dump.
Shutdown path is not needed as FW comes up clean every time during probe
after issuing FUNCTION reset MBOX command.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Use of mutex_lock_interruptible can return -EINTR, handle and log the
error.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
FW recommended timeout for all mbox command is 30s. Use msleep instead
mdelay to relinquish CPU when polling for mbox completion.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
alloc_mcc_tag needs to be done under mcc_lock.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
beiscsi_mccq_compl sets MCC_TAG_STATE_TIMEOUT before setting up
tag_mem_state. be_mcc_compl_process_isr checks for MCC_TAG_STATE_TIMEOUT
first then accesses tag_mem_state which might be still getting populated
in the process context.
Fix: Set MCC_TAG_STATE_TIMEOUT after tag_mem_state is populated.
Removed MCC_TAG_STATE_COMPLETED. When posted its in running state and
the running state is cleared in be_mcc_compl_process_isr. be_mcc_notify
now takes tag argument to set it to running state. Use bit operations
for tag_state. Use barriers before setting the state.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This is second part of actual fix for soft lockup.
All mbox cmds issued using BMBX and MCC are synchronized using mutex
mbox_lock instead of spin_lock. Used mutex_lock_interruptible where ever
possible.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Shane Seymour <shane.seymour@hpe.com>
Reviewed-by: Hannes Reinicke <hare@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We are taking mbox_lock spinlock which disables pre-emption before we
poll for mbox completion. Waiting there with spinlock held in excess of
20s will cause soft lockup.
Actual fix is to change mbox_lock to mutex. The changes are done in
phases. This is the first part.
1. Changed mgmt_get_all_if_id to use MCC where after posting lock is
released.
2. Changed be_mbox_db_ready_wait to busy wait for 12s max and removed
wait_event_timeout. Added error handling code for IO reads.
OPCODE_COMMON_QUERY_FIRMWARE_CONFIG mbox command takes 8s time when
unreachable boot targets configured.
Signed-off-by: Jitendra Bhivare <jitendra.bhivare@avagotech.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There is no good reason to keep them apart, and this makes using the API
a bit simpler.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
There is no good reason to start out disabled - drivers can control if
the poll instance can be scheduled by simply not scheduling it yet.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
The new name is irq_poll as iopoll is already taken. Better suggestions
welcome.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
drivers/scsi/be2iscsi/be_main.c: In function 'be_sgl_create_contiguous':
drivers/scsi/be2iscsi/be_main.c:3187:18: warning: logical not is only applied to the left hand side of comparison [-Wlogical-not-parentheses]
WARN_ON(!length > 0);
gcc version 5.2.1
Signed-off-by: Tim Gardner <tim.gardner@canonical.com>
Cc: Jayamohan Kallickal <jayamohan.kallickal@avagotech.com>
Cc: Minh Tran <minh.tran@avagotech.com>
Cc: John Soni Jose <sony.john-n@avagotech.com>
Cc: "James E.J. Bottomley" <JBottomley@odin.com>
Reported-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Manoj Kumar <manoj@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: John Soni Jose <sony.john@avagotech.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
While posting WRB the next_pointer of the current WRB should point
to itself and the previous WRB next_pointer should point to the
current WRB.
The next pointer value was retrieved during alloc_pdu and was updated
in wrb before ringing the doorbell. The fix retrieves the
next_pointer just before ringing the doorbell and updates in the WRB.
Signed-off-by: John Soni Jose <sony.john@avagotech.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
We would like to get the following updates in:
Revert ownership to "Emulex" from "Avago Technologies"
Signed-off-by: Ketan Mukadam <ketan.mukadam@avagotech.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Bump the driver version
[jejb: resolve conflict with 4627de9 MAINTAINERS, be2iscsi: change email domain]
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Once be2iscsi driver is loaded and operational close Boot
session established by FW.
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Check DMA memory before it is unmapped.
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Driver was not freeing the DMA memory allocated for EQ/CQ in the
unload path. This patch frees the DMA memory during the driver unload.
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Reserve device PCI I/O and Memory resources.
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Increment the retry count to get the boot target info when
port async event is received by the driver. Update sysfs enteries
with the boot target parameters.
Signed-off-by: Minh Tran <minhduc.tran@emulex.com>
Signed-off-by: John Soni Jose <sony.john-n@emulex.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
be2iscsi change of ownership from Emulex to Avago Technologies recently. We
like to get the following updates in: changed "Emulex" to "Avago
Technologies", changed email addresses from "emulex.com" to "avagotech.com",
updated MAINTAINER list for be2iscsi driver.
Signed-off-by: Minh Tran <minh.tran@avagotech.com>
Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@avagotech.com>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Ketan Mukadam
Jitendra Bhivare
Linus Torvalds
Pan Bian
James Bottomley
Dan Carpenter
Baoyou Xie
Christophe JAILLET
Maurizio Lombardi
Lars-Peter Clausen
Nicholas Krause
Alexey Khoroshilov
Jitendra Bhivare
Christoph Hellwig
James Bottomley
Tim Gardner
John Soni Jose
John Soni Jose
Minh Tran