License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 17:07:57 +03:00
|
|
|
// SPDX-License-Identifier: GPL-2.0
|
2017-01-09 18:55:17 +03:00
|
|
|
/*
|
|
|
|
* Shared Memory Communications over RDMA (SMC-R) and RoCE
|
|
|
|
*
|
|
|
|
* Basic Transport Functions exploiting Infiniband API
|
|
|
|
*
|
|
|
|
* Copyright IBM Corp. 2016
|
|
|
|
*
|
|
|
|
* Author(s): Ursula Braun <ubraun@linux.vnet.ibm.com>
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/socket.h>
|
|
|
|
#include <linux/if_vlan.h>
|
|
|
|
#include <linux/random.h>
|
|
|
|
#include <linux/workqueue.h>
|
|
|
|
#include <net/tcp.h>
|
|
|
|
#include <net/sock.h>
|
|
|
|
#include <rdma/ib_verbs.h>
|
2018-06-05 08:40:19 +03:00
|
|
|
#include <rdma/ib_cache.h>
|
2017-01-09 18:55:17 +03:00
|
|
|
|
|
|
|
#include "smc.h"
|
|
|
|
#include "smc_clc.h"
|
|
|
|
#include "smc_core.h"
|
|
|
|
#include "smc_ib.h"
|
2017-01-09 18:55:19 +03:00
|
|
|
#include "smc_wr.h"
|
2017-01-09 18:55:21 +03:00
|
|
|
#include "smc_llc.h"
|
2017-01-09 18:55:22 +03:00
|
|
|
#include "smc_cdc.h"
|
2017-01-09 18:55:25 +03:00
|
|
|
#include "smc_close.h"
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
#include "smc_ism.h"
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2017-09-21 10:16:31 +03:00
|
|
|
#define SMC_LGR_NUM_INCR 256
|
|
|
|
#define SMC_LGR_FREE_DELAY_SERV (600 * HZ)
|
2018-05-23 17:38:12 +03:00
|
|
|
#define SMC_LGR_FREE_DELAY_CLNT (SMC_LGR_FREE_DELAY_SERV + 10 * HZ)
|
2018-07-25 17:35:33 +03:00
|
|
|
#define SMC_LGR_FREE_DELAY_FAST (8 * HZ)
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2018-05-18 10:34:11 +03:00
|
|
|
static struct smc_lgr_list smc_lgr_list = { /* established link groups */
|
|
|
|
.lock = __SPIN_LOCK_UNLOCKED(smc_lgr_list.lock),
|
|
|
|
.list = LIST_HEAD_INIT(smc_lgr_list.list),
|
|
|
|
.num = 0,
|
|
|
|
};
|
2017-01-09 18:55:21 +03:00
|
|
|
|
2018-05-18 10:34:17 +03:00
|
|
|
static void smc_buf_free(struct smc_link_group *lgr, bool is_rmb,
|
|
|
|
struct smc_buf_desc *buf_desc);
|
2018-05-03 18:57:38 +03:00
|
|
|
|
2019-10-09 11:07:44 +03:00
|
|
|
/* return head of link group list and its lock for a given link group */
|
|
|
|
static inline struct list_head *smc_lgr_list_head(struct smc_link_group *lgr,
|
|
|
|
spinlock_t **lgr_lock)
|
|
|
|
{
|
|
|
|
if (lgr->is_smcd) {
|
|
|
|
*lgr_lock = &lgr->smcd->lgr_lock;
|
|
|
|
return &lgr->smcd->lgr_list;
|
|
|
|
}
|
|
|
|
|
|
|
|
*lgr_lock = &smc_lgr_list.lock;
|
|
|
|
return &smc_lgr_list.list;
|
|
|
|
}
|
|
|
|
|
2018-03-14 13:01:02 +03:00
|
|
|
static void smc_lgr_schedule_free_work(struct smc_link_group *lgr)
|
|
|
|
{
|
|
|
|
/* client link group creation always follows the server link group
|
|
|
|
* creation. For client use a somewhat higher removal delay time,
|
|
|
|
* otherwise there is a risk of out-of-sync link groups.
|
|
|
|
*/
|
2019-10-21 17:13:11 +03:00
|
|
|
if (!lgr->freeing && !lgr->freefast) {
|
|
|
|
mod_delayed_work(system_wq, &lgr->free_work,
|
|
|
|
(!lgr->is_smcd && lgr->role == SMC_CLNT) ?
|
|
|
|
SMC_LGR_FREE_DELAY_CLNT :
|
|
|
|
SMC_LGR_FREE_DELAY_SERV);
|
|
|
|
}
|
2018-03-14 13:01:02 +03:00
|
|
|
}
|
|
|
|
|
2018-07-25 17:35:33 +03:00
|
|
|
void smc_lgr_schedule_free_work_fast(struct smc_link_group *lgr)
|
|
|
|
{
|
2019-10-21 17:13:11 +03:00
|
|
|
if (!lgr->freeing && !lgr->freefast) {
|
|
|
|
lgr->freefast = 1;
|
|
|
|
mod_delayed_work(system_wq, &lgr->free_work,
|
|
|
|
SMC_LGR_FREE_DELAY_FAST);
|
|
|
|
}
|
2018-03-14 13:01:02 +03:00
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
/* Register connection's alert token in our lookup structure.
|
|
|
|
* To use rbtrees we have to implement our own insert core.
|
|
|
|
* Requires @conns_lock
|
|
|
|
* @smc connection to register
|
|
|
|
* Returns 0 on success, != otherwise.
|
|
|
|
*/
|
|
|
|
static void smc_lgr_add_alert_token(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct rb_node **link, *parent = NULL;
|
|
|
|
u32 token = conn->alert_token_local;
|
|
|
|
|
|
|
|
link = &conn->lgr->conns_all.rb_node;
|
|
|
|
while (*link) {
|
|
|
|
struct smc_connection *cur = rb_entry(*link,
|
|
|
|
struct smc_connection, alert_node);
|
|
|
|
|
|
|
|
parent = *link;
|
|
|
|
if (cur->alert_token_local > token)
|
|
|
|
link = &parent->rb_left;
|
|
|
|
else
|
|
|
|
link = &parent->rb_right;
|
|
|
|
}
|
|
|
|
/* Put the new node there */
|
|
|
|
rb_link_node(&conn->alert_node, parent, link);
|
|
|
|
rb_insert_color(&conn->alert_node, &conn->lgr->conns_all);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Register connection in link group by assigning an alert token
|
|
|
|
* registered in a search tree.
|
|
|
|
* Requires @conns_lock
|
|
|
|
* Note that '0' is a reserved value and not assigned.
|
|
|
|
*/
|
|
|
|
static void smc_lgr_register_conn(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_sock *smc = container_of(conn, struct smc_sock, conn);
|
|
|
|
static atomic_t nexttoken = ATOMIC_INIT(0);
|
|
|
|
|
|
|
|
/* find a new alert_token_local value not yet used by some connection
|
|
|
|
* in this link group
|
|
|
|
*/
|
|
|
|
sock_hold(&smc->sk); /* sock_put in smc_lgr_unregister_conn() */
|
|
|
|
while (!conn->alert_token_local) {
|
|
|
|
conn->alert_token_local = atomic_inc_return(&nexttoken);
|
|
|
|
if (smc_lgr_find_conn(conn->alert_token_local, conn->lgr))
|
|
|
|
conn->alert_token_local = 0;
|
|
|
|
}
|
|
|
|
smc_lgr_add_alert_token(conn);
|
|
|
|
conn->lgr->conns_num++;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Unregister connection and reset the alert token of the given connection<
|
|
|
|
*/
|
|
|
|
static void __smc_lgr_unregister_conn(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_sock *smc = container_of(conn, struct smc_sock, conn);
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
|
|
|
rb_erase(&conn->alert_node, &lgr->conns_all);
|
|
|
|
lgr->conns_num--;
|
|
|
|
conn->alert_token_local = 0;
|
|
|
|
sock_put(&smc->sk); /* sock_hold in smc_lgr_register_conn() */
|
|
|
|
}
|
|
|
|
|
2018-10-25 14:25:28 +03:00
|
|
|
/* Unregister connection from lgr
|
2017-01-09 18:55:17 +03:00
|
|
|
*/
|
|
|
|
static void smc_lgr_unregister_conn(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
2019-01-30 20:51:02 +03:00
|
|
|
if (!lgr)
|
|
|
|
return;
|
2017-01-09 18:55:17 +03:00
|
|
|
write_lock_bh(&lgr->conns_lock);
|
|
|
|
if (conn->alert_token_local) {
|
|
|
|
__smc_lgr_unregister_conn(conn);
|
|
|
|
}
|
|
|
|
write_unlock_bh(&lgr->conns_lock);
|
2019-10-21 17:13:13 +03:00
|
|
|
conn->lgr = NULL;
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
2018-07-25 17:35:33 +03:00
|
|
|
/* Send delete link, either as client to request the initiation
|
|
|
|
* of the DELETE LINK sequence from server; or as server to
|
|
|
|
* initiate the delete processing. See smc_llc_rx_delete_link().
|
|
|
|
*/
|
2019-11-14 15:02:45 +03:00
|
|
|
static int smc_link_send_delete(struct smc_link *lnk, bool orderly)
|
2018-07-25 17:35:33 +03:00
|
|
|
{
|
|
|
|
if (lnk->state == SMC_LNK_ACTIVE &&
|
2019-11-14 15:02:45 +03:00
|
|
|
!smc_llc_send_delete_link(lnk, SMC_LLC_REQ, orderly)) {
|
2018-07-25 17:35:33 +03:00
|
|
|
smc_llc_link_deleting(lnk);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
return -ENOTCONN;
|
|
|
|
}
|
|
|
|
|
2018-11-22 12:26:35 +03:00
|
|
|
static void smc_lgr_free(struct smc_link_group *lgr);
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
static void smc_lgr_free_work(struct work_struct *work)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = container_of(to_delayed_work(work),
|
|
|
|
struct smc_link_group,
|
|
|
|
free_work);
|
2019-10-09 11:07:44 +03:00
|
|
|
spinlock_t *lgr_lock;
|
2019-10-21 17:13:11 +03:00
|
|
|
struct smc_link *lnk;
|
2017-01-09 18:55:17 +03:00
|
|
|
bool conns;
|
|
|
|
|
2019-10-09 11:07:44 +03:00
|
|
|
smc_lgr_list_head(lgr, &lgr_lock);
|
|
|
|
spin_lock_bh(lgr_lock);
|
2019-10-21 17:13:11 +03:00
|
|
|
if (lgr->freeing) {
|
|
|
|
spin_unlock_bh(lgr_lock);
|
|
|
|
return;
|
|
|
|
}
|
2017-01-09 18:55:17 +03:00
|
|
|
read_lock_bh(&lgr->conns_lock);
|
|
|
|
conns = RB_EMPTY_ROOT(&lgr->conns_all);
|
|
|
|
read_unlock_bh(&lgr->conns_lock);
|
|
|
|
if (!conns) { /* number of lgr connections is no longer zero */
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_unlock_bh(lgr_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
return;
|
|
|
|
}
|
2019-10-21 17:13:09 +03:00
|
|
|
list_del_init(&lgr->list); /* remove from smc_lgr_list */
|
2018-07-25 17:35:33 +03:00
|
|
|
|
2019-10-21 17:13:11 +03:00
|
|
|
lnk = &lgr->lnk[SMC_SINGLE_LINK];
|
2018-07-25 17:35:33 +03:00
|
|
|
if (!lgr->is_smcd && !lgr->terminating) {
|
|
|
|
/* try to send del link msg, on error free lgr immediately */
|
2018-11-22 12:26:38 +03:00
|
|
|
if (lnk->state == SMC_LNK_ACTIVE &&
|
2019-11-14 15:02:45 +03:00
|
|
|
!smc_link_send_delete(lnk, true)) {
|
2018-07-25 17:35:33 +03:00
|
|
|
/* reschedule in case we never receive a response */
|
|
|
|
smc_lgr_schedule_free_work(lgr);
|
2019-10-21 17:13:11 +03:00
|
|
|
spin_unlock_bh(lgr_lock);
|
2018-07-25 17:35:33 +03:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
}
|
2019-10-21 17:13:11 +03:00
|
|
|
lgr->freeing = 1; /* this instance does the freeing, no new schedule */
|
|
|
|
spin_unlock_bh(lgr_lock);
|
|
|
|
cancel_delayed_work(&lgr->free_work);
|
2018-07-25 17:35:33 +03:00
|
|
|
|
2019-10-21 17:13:11 +03:00
|
|
|
if (!lgr->is_smcd && lnk->state != SMC_LNK_INACTIVE)
|
|
|
|
smc_llc_link_inactive(lnk);
|
2019-11-14 15:02:41 +03:00
|
|
|
if (lgr->is_smcd && !lgr->terminating)
|
2019-10-21 17:13:11 +03:00
|
|
|
smc_ism_signal_shutdown(lgr);
|
|
|
|
smc_lgr_free(lgr);
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
2019-10-21 17:13:14 +03:00
|
|
|
static void smc_lgr_terminate_work(struct work_struct *work)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = container_of(work, struct smc_link_group,
|
|
|
|
terminate_work);
|
|
|
|
|
2019-11-14 15:02:42 +03:00
|
|
|
smc_lgr_terminate(lgr, true);
|
2019-10-21 17:13:14 +03:00
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
/* create a new SMC link group */
|
2019-04-12 13:57:26 +03:00
|
|
|
static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
|
|
|
struct smc_link_group *lgr;
|
2019-10-09 11:07:43 +03:00
|
|
|
struct list_head *lgr_list;
|
2017-01-09 18:55:17 +03:00
|
|
|
struct smc_link *lnk;
|
2019-10-09 11:07:44 +03:00
|
|
|
spinlock_t *lgr_lock;
|
2017-01-09 18:55:17 +03:00
|
|
|
u8 rndvec[3];
|
|
|
|
int rc = 0;
|
2017-01-09 18:55:18 +03:00
|
|
|
int i;
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2019-04-12 13:57:26 +03:00
|
|
|
if (ini->is_smcd && ini->vlan_id) {
|
2019-04-12 13:57:30 +03:00
|
|
|
if (smc_ism_get_vlan(ini->ism_dev, ini->vlan_id)) {
|
|
|
|
rc = SMC_CLC_DECL_ISMVLANERR;
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
goto out;
|
2019-04-12 13:57:30 +03:00
|
|
|
}
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
lgr = kzalloc(sizeof(*lgr), GFP_KERNEL);
|
|
|
|
if (!lgr) {
|
2019-04-12 13:57:30 +03:00
|
|
|
rc = SMC_CLC_DECL_MEM;
|
2019-10-10 11:16:09 +03:00
|
|
|
goto ism_put_vlan;
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
2019-04-12 13:57:26 +03:00
|
|
|
lgr->is_smcd = ini->is_smcd;
|
2018-05-15 18:05:03 +03:00
|
|
|
lgr->sync_err = 0;
|
2019-10-21 17:13:11 +03:00
|
|
|
lgr->terminating = 0;
|
|
|
|
lgr->freefast = 0;
|
|
|
|
lgr->freeing = 0;
|
2019-04-12 13:57:26 +03:00
|
|
|
lgr->vlan_id = ini->vlan_id;
|
2017-01-09 18:55:18 +03:00
|
|
|
rwlock_init(&lgr->sndbufs_lock);
|
|
|
|
rwlock_init(&lgr->rmbs_lock);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
rwlock_init(&lgr->conns_lock);
|
2017-01-09 18:55:18 +03:00
|
|
|
for (i = 0; i < SMC_RMBE_SIZES; i++) {
|
|
|
|
INIT_LIST_HEAD(&lgr->sndbufs[i]);
|
|
|
|
INIT_LIST_HEAD(&lgr->rmbs[i]);
|
|
|
|
}
|
2018-05-18 10:34:11 +03:00
|
|
|
smc_lgr_list.num += SMC_LGR_NUM_INCR;
|
|
|
|
memcpy(&lgr->id, (u8 *)&smc_lgr_list.num, SMC_LGR_ID_SIZE);
|
2017-01-09 18:55:17 +03:00
|
|
|
INIT_DELAYED_WORK(&lgr->free_work, smc_lgr_free_work);
|
2019-10-21 17:13:14 +03:00
|
|
|
INIT_WORK(&lgr->terminate_work, smc_lgr_terminate_work);
|
2017-01-09 18:55:17 +03:00
|
|
|
lgr->conns_all = RB_ROOT;
|
2019-04-12 13:57:26 +03:00
|
|
|
if (ini->is_smcd) {
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
/* SMC-D specific settings */
|
2019-10-09 11:07:45 +03:00
|
|
|
get_device(&ini->ism_dev->dev);
|
2019-04-12 13:57:26 +03:00
|
|
|
lgr->peer_gid = ini->ism_gid;
|
|
|
|
lgr->smcd = ini->ism_dev;
|
2019-10-09 11:07:43 +03:00
|
|
|
lgr_list = &ini->ism_dev->lgr_list;
|
2019-10-09 11:07:44 +03:00
|
|
|
lgr_lock = &lgr->smcd->lgr_lock;
|
2019-11-14 15:02:40 +03:00
|
|
|
lgr->peer_shutdown = 0;
|
2019-11-14 15:02:43 +03:00
|
|
|
atomic_inc(&ini->ism_dev->lgr_cnt);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
} else {
|
|
|
|
/* SMC-R specific settings */
|
2019-10-09 11:07:45 +03:00
|
|
|
get_device(&ini->ib_dev->ibdev->dev);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
lgr->role = smc->listen_smc ? SMC_SERV : SMC_CLNT;
|
2019-04-12 13:57:26 +03:00
|
|
|
memcpy(lgr->peer_systemid, ini->ib_lcl->id_for_peer,
|
|
|
|
SMC_SYSTEMID_LEN);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
|
|
|
|
lnk = &lgr->lnk[SMC_SINGLE_LINK];
|
|
|
|
/* initialize link */
|
|
|
|
lnk->state = SMC_LNK_ACTIVATING;
|
|
|
|
lnk->link_id = SMC_SINGLE_LINK;
|
2019-04-12 13:57:26 +03:00
|
|
|
lnk->smcibdev = ini->ib_dev;
|
|
|
|
lnk->ibport = ini->ib_port;
|
2019-10-09 11:07:43 +03:00
|
|
|
lgr_list = &smc_lgr_list.list;
|
2019-10-09 11:07:44 +03:00
|
|
|
lgr_lock = &smc_lgr_list.lock;
|
2019-04-12 13:57:26 +03:00
|
|
|
lnk->path_mtu =
|
|
|
|
ini->ib_dev->pattr[ini->ib_port - 1].active_mtu;
|
|
|
|
if (!ini->ib_dev->initialized)
|
|
|
|
smc_ib_setup_per_ibdev(ini->ib_dev);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
get_random_bytes(rndvec, sizeof(rndvec));
|
|
|
|
lnk->psn_initial = rndvec[0] + (rndvec[1] << 8) +
|
|
|
|
(rndvec[2] << 16);
|
2018-07-25 17:35:31 +03:00
|
|
|
rc = smc_ib_determine_gid(lnk->smcibdev, lnk->ibport,
|
2019-04-12 13:57:26 +03:00
|
|
|
ini->vlan_id, lnk->gid,
|
|
|
|
&lnk->sgid_index);
|
2018-07-25 17:35:31 +03:00
|
|
|
if (rc)
|
|
|
|
goto free_lgr;
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
rc = smc_llc_link_init(lnk);
|
|
|
|
if (rc)
|
|
|
|
goto free_lgr;
|
|
|
|
rc = smc_wr_alloc_link_mem(lnk);
|
|
|
|
if (rc)
|
|
|
|
goto clear_llc_lnk;
|
|
|
|
rc = smc_ib_create_protection_domain(lnk);
|
|
|
|
if (rc)
|
|
|
|
goto free_link_mem;
|
|
|
|
rc = smc_ib_create_queue_pair(lnk);
|
|
|
|
if (rc)
|
|
|
|
goto dealloc_pd;
|
|
|
|
rc = smc_wr_create_link(lnk);
|
|
|
|
if (rc)
|
|
|
|
goto destroy_qp;
|
|
|
|
}
|
2017-01-09 18:55:17 +03:00
|
|
|
smc->conn.lgr = lgr;
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_lock_bh(lgr_lock);
|
2019-10-09 11:07:43 +03:00
|
|
|
list_add(&lgr->list, lgr_list);
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_unlock_bh(lgr_lock);
|
2017-01-09 18:55:19 +03:00
|
|
|
return 0;
|
|
|
|
|
2017-01-09 18:55:20 +03:00
|
|
|
destroy_qp:
|
|
|
|
smc_ib_destroy_queue_pair(lnk);
|
|
|
|
dealloc_pd:
|
|
|
|
smc_ib_dealloc_protection_domain(lnk);
|
|
|
|
free_link_mem:
|
|
|
|
smc_wr_free_link_mem(lnk);
|
2018-05-15 18:04:59 +03:00
|
|
|
clear_llc_lnk:
|
|
|
|
smc_llc_link_clear(lnk);
|
2017-01-09 18:55:19 +03:00
|
|
|
free_lgr:
|
|
|
|
kfree(lgr);
|
2019-10-10 11:16:09 +03:00
|
|
|
ism_put_vlan:
|
|
|
|
if (ini->is_smcd && ini->vlan_id)
|
|
|
|
smc_ism_put_vlan(ini->ism_dev, ini->vlan_id);
|
2017-01-09 18:55:17 +03:00
|
|
|
out:
|
2019-04-12 13:57:30 +03:00
|
|
|
if (rc < 0) {
|
|
|
|
if (rc == -ENOMEM)
|
|
|
|
rc = SMC_CLC_DECL_MEM;
|
|
|
|
else
|
|
|
|
rc = SMC_CLC_DECL_INTERR;
|
|
|
|
}
|
2017-01-09 18:55:17 +03:00
|
|
|
return rc;
|
|
|
|
}
|
|
|
|
|
2018-10-25 14:25:28 +03:00
|
|
|
static void smc_buf_unuse(struct smc_connection *conn,
|
|
|
|
struct smc_link_group *lgr)
|
2017-01-09 18:55:18 +03:00
|
|
|
{
|
2018-05-18 10:34:10 +03:00
|
|
|
if (conn->sndbuf_desc)
|
2017-01-09 18:55:18 +03:00
|
|
|
conn->sndbuf_desc->used = 0;
|
|
|
|
if (conn->rmb_desc) {
|
2018-05-03 18:57:38 +03:00
|
|
|
if (!conn->rmb_desc->regerr) {
|
2019-10-21 17:13:13 +03:00
|
|
|
if (!lgr->is_smcd && !list_empty(&lgr->list)) {
|
2018-11-22 12:26:43 +03:00
|
|
|
/* unregister rmb with peer */
|
|
|
|
smc_llc_do_delete_rkey(
|
|
|
|
&lgr->lnk[SMC_SINGLE_LINK],
|
|
|
|
conn->rmb_desc);
|
|
|
|
}
|
2019-02-04 15:44:46 +03:00
|
|
|
conn->rmb_desc->used = 0;
|
2018-05-03 18:57:38 +03:00
|
|
|
} else {
|
|
|
|
/* buf registration failed, reuse not possible */
|
|
|
|
write_lock_bh(&lgr->rmbs_lock);
|
|
|
|
list_del(&conn->rmb_desc->list);
|
|
|
|
write_unlock_bh(&lgr->rmbs_lock);
|
|
|
|
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(lgr, true, conn->rmb_desc);
|
2018-05-03 18:57:38 +03:00
|
|
|
}
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
/* remove a finished connection from its link group */
|
|
|
|
void smc_conn_free(struct smc_connection *conn)
|
|
|
|
{
|
2018-10-25 14:25:28 +03:00
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
|
|
|
if (!lgr)
|
2017-01-09 18:55:17 +03:00
|
|
|
return;
|
2018-10-25 14:25:28 +03:00
|
|
|
if (lgr->is_smcd) {
|
2019-11-14 15:02:41 +03:00
|
|
|
if (!list_empty(&lgr->list))
|
|
|
|
smc_ism_unset_conn(conn);
|
2018-06-28 20:05:10 +03:00
|
|
|
tasklet_kill(&conn->rx_tsklet);
|
|
|
|
} else {
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
smc_cdc_tx_dismiss_slots(conn);
|
2018-06-28 20:05:10 +03:00
|
|
|
}
|
2019-10-21 17:13:13 +03:00
|
|
|
if (!list_empty(&lgr->list)) {
|
|
|
|
smc_lgr_unregister_conn(conn);
|
|
|
|
smc_buf_unuse(conn, lgr); /* allow buffer reuse */
|
|
|
|
}
|
2018-10-25 14:25:28 +03:00
|
|
|
|
|
|
|
if (!lgr->conns_num)
|
|
|
|
smc_lgr_schedule_free_work(lgr);
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
static void smc_link_clear(struct smc_link *lnk)
|
|
|
|
{
|
|
|
|
lnk->peer_qpn = 0;
|
2018-05-15 18:04:59 +03:00
|
|
|
smc_llc_link_clear(lnk);
|
2017-01-09 18:55:20 +03:00
|
|
|
smc_ib_modify_qp_reset(lnk);
|
2017-01-09 18:55:19 +03:00
|
|
|
smc_wr_free_link(lnk);
|
2017-01-09 18:55:20 +03:00
|
|
|
smc_ib_destroy_queue_pair(lnk);
|
|
|
|
smc_ib_dealloc_protection_domain(lnk);
|
2017-01-09 18:55:19 +03:00
|
|
|
smc_wr_free_link_mem(lnk);
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
static void smcr_buf_free(struct smc_link_group *lgr, bool is_rmb,
|
|
|
|
struct smc_buf_desc *buf_desc)
|
2017-01-09 18:55:18 +03:00
|
|
|
{
|
2018-05-18 10:34:17 +03:00
|
|
|
struct smc_link *lnk = &lgr->lnk[SMC_SINGLE_LINK];
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
if (is_rmb) {
|
|
|
|
if (buf_desc->mr_rx[SMC_SINGLE_LINK])
|
|
|
|
smc_ib_put_memory_region(
|
|
|
|
buf_desc->mr_rx[SMC_SINGLE_LINK]);
|
|
|
|
smc_ib_buf_unmap_sg(lnk->smcibdev, buf_desc,
|
|
|
|
DMA_FROM_DEVICE);
|
|
|
|
} else {
|
|
|
|
smc_ib_buf_unmap_sg(lnk->smcibdev, buf_desc,
|
|
|
|
DMA_TO_DEVICE);
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
2017-07-28 14:56:20 +03:00
|
|
|
sg_free_table(&buf_desc->sgt[SMC_SINGLE_LINK]);
|
2018-05-03 19:12:38 +03:00
|
|
|
if (buf_desc->pages)
|
|
|
|
__free_pages(buf_desc->pages, buf_desc->order);
|
2017-07-28 14:56:20 +03:00
|
|
|
kfree(buf_desc);
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
static void smcd_buf_free(struct smc_link_group *lgr, bool is_dmb,
|
|
|
|
struct smc_buf_desc *buf_desc)
|
|
|
|
{
|
2018-06-28 20:05:10 +03:00
|
|
|
if (is_dmb) {
|
|
|
|
/* restore original buf len */
|
|
|
|
buf_desc->len += sizeof(struct smcd_cdc_msg);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
smc_ism_unregister_dmb(lgr->smcd, buf_desc);
|
2018-06-28 20:05:10 +03:00
|
|
|
} else {
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
kfree(buf_desc->cpu_addr);
|
2018-06-28 20:05:10 +03:00
|
|
|
}
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
kfree(buf_desc);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void smc_buf_free(struct smc_link_group *lgr, bool is_rmb,
|
|
|
|
struct smc_buf_desc *buf_desc)
|
|
|
|
{
|
|
|
|
if (lgr->is_smcd)
|
|
|
|
smcd_buf_free(lgr, is_rmb, buf_desc);
|
|
|
|
else
|
|
|
|
smcr_buf_free(lgr, is_rmb, buf_desc);
|
|
|
|
}
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
static void __smc_lgr_free_bufs(struct smc_link_group *lgr, bool is_rmb)
|
2017-01-09 18:55:18 +03:00
|
|
|
{
|
2017-07-28 14:56:20 +03:00
|
|
|
struct smc_buf_desc *buf_desc, *bf_desc;
|
|
|
|
struct list_head *buf_list;
|
2017-01-09 18:55:18 +03:00
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < SMC_RMBE_SIZES; i++) {
|
2017-07-28 14:56:20 +03:00
|
|
|
if (is_rmb)
|
|
|
|
buf_list = &lgr->rmbs[i];
|
|
|
|
else
|
|
|
|
buf_list = &lgr->sndbufs[i];
|
|
|
|
list_for_each_entry_safe(buf_desc, bf_desc, buf_list,
|
2017-01-09 18:55:18 +03:00
|
|
|
list) {
|
2017-07-28 14:56:20 +03:00
|
|
|
list_del(&buf_desc->list);
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(lgr, is_rmb, buf_desc);
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
static void smc_lgr_free_bufs(struct smc_link_group *lgr)
|
|
|
|
{
|
|
|
|
/* free send buffers */
|
|
|
|
__smc_lgr_free_bufs(lgr, false);
|
|
|
|
/* free rmbs */
|
|
|
|
__smc_lgr_free_bufs(lgr, true);
|
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
/* remove a link group */
|
2018-11-22 12:26:35 +03:00
|
|
|
static void smc_lgr_free(struct smc_link_group *lgr)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
2017-07-28 14:56:20 +03:00
|
|
|
smc_lgr_free_bufs(lgr);
|
2019-10-09 11:07:45 +03:00
|
|
|
if (lgr->is_smcd) {
|
2019-11-14 15:02:41 +03:00
|
|
|
if (!lgr->terminating) {
|
|
|
|
smc_ism_put_vlan(lgr->smcd, lgr->vlan_id);
|
|
|
|
put_device(&lgr->smcd->dev);
|
|
|
|
}
|
2019-11-14 15:02:43 +03:00
|
|
|
if (!atomic_dec_return(&lgr->smcd->lgr_cnt))
|
|
|
|
wake_up(&lgr->smcd->lgrs_deleted);
|
2019-10-09 11:07:45 +03:00
|
|
|
} else {
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
smc_link_clear(&lgr->lnk[SMC_SINGLE_LINK]);
|
2019-10-09 11:07:45 +03:00
|
|
|
put_device(&lgr->lnk[SMC_SINGLE_LINK].smcibdev->ibdev->dev);
|
|
|
|
}
|
2017-01-09 18:55:17 +03:00
|
|
|
kfree(lgr);
|
|
|
|
}
|
|
|
|
|
2018-03-01 15:51:33 +03:00
|
|
|
void smc_lgr_forget(struct smc_link_group *lgr)
|
|
|
|
{
|
2019-10-09 11:07:44 +03:00
|
|
|
struct list_head *lgr_list;
|
|
|
|
spinlock_t *lgr_lock;
|
|
|
|
|
|
|
|
lgr_list = smc_lgr_list_head(lgr, &lgr_lock);
|
|
|
|
spin_lock_bh(lgr_lock);
|
2018-03-01 15:51:33 +03:00
|
|
|
/* do not use this link group for new connections */
|
2019-10-09 11:07:44 +03:00
|
|
|
if (!list_empty(lgr_list))
|
|
|
|
list_del_init(lgr_list);
|
|
|
|
spin_unlock_bh(lgr_lock);
|
2018-03-01 15:51:33 +03:00
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:41 +03:00
|
|
|
static void smcd_unregister_all_dmbs(struct smc_link_group *lgr)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < SMC_RMBE_SIZES; i++) {
|
|
|
|
struct smc_buf_desc *buf_desc;
|
|
|
|
|
|
|
|
list_for_each_entry(buf_desc, &lgr->rmbs[i], list) {
|
|
|
|
buf_desc->len += sizeof(struct smcd_cdc_msg);
|
|
|
|
smc_ism_unregister_dmb(lgr->smcd, buf_desc);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-10-21 17:13:13 +03:00
|
|
|
static void smc_sk_wake_ups(struct smc_sock *smc)
|
|
|
|
{
|
|
|
|
smc->sk.sk_write_space(&smc->sk);
|
|
|
|
smc->sk.sk_data_ready(&smc->sk);
|
|
|
|
smc->sk.sk_state_change(&smc->sk);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* kill a connection */
|
2019-11-14 15:02:42 +03:00
|
|
|
static void smc_conn_kill(struct smc_connection *conn, bool soft)
|
2019-10-21 17:13:13 +03:00
|
|
|
{
|
|
|
|
struct smc_sock *smc = container_of(conn, struct smc_sock, conn);
|
|
|
|
|
2019-11-14 15:02:40 +03:00
|
|
|
if (conn->lgr->is_smcd && conn->lgr->peer_shutdown)
|
|
|
|
conn->local_tx_ctrl.conn_state_flags.peer_conn_abort = 1;
|
|
|
|
else
|
|
|
|
smc_close_abort(conn);
|
2019-10-21 17:13:13 +03:00
|
|
|
conn->killed = 1;
|
2019-11-14 15:02:40 +03:00
|
|
|
smc->sk.sk_err = ECONNABORTED;
|
2019-10-21 17:13:13 +03:00
|
|
|
smc_sk_wake_ups(smc);
|
2019-11-14 15:02:41 +03:00
|
|
|
if (conn->lgr->is_smcd) {
|
|
|
|
smc_ism_unset_conn(conn);
|
2019-11-14 15:02:42 +03:00
|
|
|
if (soft)
|
|
|
|
tasklet_kill(&conn->rx_tsklet);
|
|
|
|
else
|
|
|
|
tasklet_unlock_wait(&conn->rx_tsklet);
|
2019-11-14 15:02:46 +03:00
|
|
|
} else {
|
|
|
|
smc_cdc_tx_dismiss_slots(conn);
|
2019-11-14 15:02:41 +03:00
|
|
|
}
|
2019-10-21 17:13:13 +03:00
|
|
|
smc_lgr_unregister_conn(conn);
|
2019-10-21 17:13:15 +03:00
|
|
|
smc_close_active_abort(smc);
|
2019-10-21 17:13:13 +03:00
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:41 +03:00
|
|
|
static void smc_lgr_cleanup(struct smc_link_group *lgr)
|
|
|
|
{
|
|
|
|
if (lgr->is_smcd) {
|
|
|
|
smc_ism_signal_shutdown(lgr);
|
|
|
|
smcd_unregister_all_dmbs(lgr);
|
|
|
|
smc_ism_put_vlan(lgr->smcd, lgr->vlan_id);
|
|
|
|
put_device(&lgr->smcd->dev);
|
|
|
|
} else {
|
|
|
|
struct smc_link *lnk = &lgr->lnk[SMC_SINGLE_LINK];
|
|
|
|
|
|
|
|
wake_up(&lnk->wr_reg_wait);
|
2019-11-14 15:02:47 +03:00
|
|
|
if (lnk->state != SMC_LNK_INACTIVE) {
|
|
|
|
smc_link_send_delete(lnk, false);
|
|
|
|
smc_llc_link_inactive(lnk);
|
|
|
|
}
|
2019-11-14 15:02:41 +03:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-10-21 17:13:09 +03:00
|
|
|
/* terminate link group */
|
2019-11-14 15:02:42 +03:00
|
|
|
static void __smc_lgr_terminate(struct smc_link_group *lgr, bool soft)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
|
|
|
struct smc_connection *conn;
|
2017-01-09 18:55:25 +03:00
|
|
|
struct smc_sock *smc;
|
2017-01-09 18:55:17 +03:00
|
|
|
struct rb_node *node;
|
|
|
|
|
2018-05-15 18:05:03 +03:00
|
|
|
if (lgr->terminating)
|
|
|
|
return; /* lgr already terminating */
|
2019-11-14 15:02:42 +03:00
|
|
|
if (!soft)
|
|
|
|
cancel_delayed_work_sync(&lgr->free_work);
|
2018-05-15 18:05:03 +03:00
|
|
|
lgr->terminating = 1;
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!lgr->is_smcd)
|
|
|
|
smc_llc_link_inactive(&lgr->lnk[SMC_SINGLE_LINK]);
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2019-10-21 17:13:10 +03:00
|
|
|
/* kill remaining link group connections */
|
|
|
|
read_lock_bh(&lgr->conns_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
node = rb_first(&lgr->conns_all);
|
|
|
|
while (node) {
|
2019-10-21 17:13:10 +03:00
|
|
|
read_unlock_bh(&lgr->conns_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
conn = rb_entry(node, struct smc_connection, alert_node);
|
2017-01-09 18:55:25 +03:00
|
|
|
smc = container_of(conn, struct smc_sock, conn);
|
2019-10-21 17:13:15 +03:00
|
|
|
sock_hold(&smc->sk); /* sock_put below */
|
2019-10-21 17:13:10 +03:00
|
|
|
lock_sock(&smc->sk);
|
2019-11-14 15:02:42 +03:00
|
|
|
smc_conn_kill(conn, soft);
|
2019-10-21 17:13:10 +03:00
|
|
|
release_sock(&smc->sk);
|
2019-10-21 17:13:15 +03:00
|
|
|
sock_put(&smc->sk); /* sock_hold above */
|
2019-10-21 17:13:10 +03:00
|
|
|
read_lock_bh(&lgr->conns_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
node = rb_first(&lgr->conns_all);
|
|
|
|
}
|
2019-10-21 17:13:10 +03:00
|
|
|
read_unlock_bh(&lgr->conns_lock);
|
2019-11-14 15:02:41 +03:00
|
|
|
smc_lgr_cleanup(lgr);
|
2019-11-14 15:02:42 +03:00
|
|
|
if (soft)
|
|
|
|
smc_lgr_schedule_free_work_fast(lgr);
|
|
|
|
else
|
|
|
|
smc_lgr_free(lgr);
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:42 +03:00
|
|
|
/* unlink and terminate link group
|
|
|
|
* @soft: true if link group shutdown can take its time
|
|
|
|
* false if immediate link group shutdown is required
|
|
|
|
*/
|
|
|
|
void smc_lgr_terminate(struct smc_link_group *lgr, bool soft)
|
2018-05-23 17:38:10 +03:00
|
|
|
{
|
2019-10-09 11:07:44 +03:00
|
|
|
spinlock_t *lgr_lock;
|
|
|
|
|
|
|
|
smc_lgr_list_head(lgr, &lgr_lock);
|
|
|
|
spin_lock_bh(lgr_lock);
|
2019-10-21 17:13:09 +03:00
|
|
|
if (lgr->terminating) {
|
|
|
|
spin_unlock_bh(lgr_lock);
|
|
|
|
return; /* lgr already terminating */
|
|
|
|
}
|
2019-11-14 15:02:42 +03:00
|
|
|
if (!soft)
|
|
|
|
lgr->freeing = 1;
|
2019-10-21 17:13:09 +03:00
|
|
|
list_del_init(&lgr->list);
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_unlock_bh(lgr_lock);
|
2019-11-14 15:02:42 +03:00
|
|
|
__smc_lgr_terminate(lgr, soft);
|
2018-05-23 17:38:10 +03:00
|
|
|
}
|
|
|
|
|
2018-05-18 10:34:11 +03:00
|
|
|
/* Called when IB port is terminated */
|
|
|
|
void smc_port_terminate(struct smc_ib_device *smcibdev, u8 ibport)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr, *l;
|
2019-10-21 17:13:09 +03:00
|
|
|
LIST_HEAD(lgr_free_list);
|
2018-05-18 10:34:11 +03:00
|
|
|
|
2018-05-23 17:38:10 +03:00
|
|
|
spin_lock_bh(&smc_lgr_list.lock);
|
2018-05-18 10:34:11 +03:00
|
|
|
list_for_each_entry_safe(lgr, l, &smc_lgr_list.list, list) {
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!lgr->is_smcd &&
|
|
|
|
lgr->lnk[SMC_SINGLE_LINK].smcibdev == smcibdev &&
|
2019-11-14 15:02:47 +03:00
|
|
|
lgr->lnk[SMC_SINGLE_LINK].ibport == ibport) {
|
2019-10-21 17:13:09 +03:00
|
|
|
list_move(&lgr->list, &lgr_free_list);
|
2019-11-14 15:02:47 +03:00
|
|
|
lgr->freeing = 1;
|
|
|
|
}
|
2018-05-18 10:34:11 +03:00
|
|
|
}
|
2018-05-23 17:38:10 +03:00
|
|
|
spin_unlock_bh(&smc_lgr_list.lock);
|
2019-10-21 17:13:09 +03:00
|
|
|
|
|
|
|
list_for_each_entry_safe(lgr, l, &lgr_free_list, list) {
|
|
|
|
list_del_init(&lgr->list);
|
2019-11-14 15:02:47 +03:00
|
|
|
__smc_lgr_terminate(lgr, false);
|
2019-10-21 17:13:09 +03:00
|
|
|
}
|
2018-05-18 10:34:11 +03:00
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:42 +03:00
|
|
|
/* Called when peer lgr shutdown (regularly or abnormally) is received */
|
2018-11-20 18:46:41 +03:00
|
|
|
void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan)
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
{
|
|
|
|
struct smc_link_group *lgr, *l;
|
|
|
|
LIST_HEAD(lgr_free_list);
|
|
|
|
|
|
|
|
/* run common cleanup function and build free list */
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_lock_bh(&dev->lgr_lock);
|
2019-10-09 11:07:43 +03:00
|
|
|
list_for_each_entry_safe(lgr, l, &dev->lgr_list, list) {
|
|
|
|
if ((!peer_gid || lgr->peer_gid == peer_gid) &&
|
2018-11-20 18:46:41 +03:00
|
|
|
(vlan == VLAN_VID_MASK || lgr->vlan_id == vlan)) {
|
2019-11-14 15:02:40 +03:00
|
|
|
if (peer_gid) /* peer triggered termination */
|
|
|
|
lgr->peer_shutdown = 1;
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
list_move(&lgr->list, &lgr_free_list);
|
|
|
|
}
|
|
|
|
}
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_unlock_bh(&dev->lgr_lock);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
|
|
|
|
/* cancel the regular free workers and actually free lgrs */
|
|
|
|
list_for_each_entry_safe(lgr, l, &lgr_free_list, list) {
|
|
|
|
list_del_init(&lgr->list);
|
2019-11-14 15:02:40 +03:00
|
|
|
schedule_work(&lgr->terminate_work);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:42 +03:00
|
|
|
/* Called when an SMCD device is removed or the smc module is unloaded */
|
|
|
|
void smc_smcd_terminate_all(struct smcd_dev *smcd)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr, *lg;
|
|
|
|
LIST_HEAD(lgr_free_list);
|
|
|
|
|
|
|
|
spin_lock_bh(&smcd->lgr_lock);
|
|
|
|
list_splice_init(&smcd->lgr_list, &lgr_free_list);
|
|
|
|
list_for_each_entry(lgr, &lgr_free_list, list)
|
|
|
|
lgr->freeing = 1;
|
|
|
|
spin_unlock_bh(&smcd->lgr_lock);
|
|
|
|
|
|
|
|
list_for_each_entry_safe(lgr, lg, &lgr_free_list, list) {
|
|
|
|
list_del_init(&lgr->list);
|
|
|
|
__smc_lgr_terminate(lgr, false);
|
|
|
|
}
|
2019-11-14 15:02:43 +03:00
|
|
|
|
|
|
|
if (atomic_read(&smcd->lgr_cnt))
|
|
|
|
wait_event(smcd->lgrs_deleted, !atomic_read(&smcd->lgr_cnt));
|
2019-11-14 15:02:42 +03:00
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:47 +03:00
|
|
|
/* Called when an SMCR device is removed or the smc module is unloaded.
|
|
|
|
* If smcibdev is given, all SMCR link groups using this device are terminated.
|
|
|
|
* If smcibdev is NULL, all SMCR link groups are terminated.
|
|
|
|
*/
|
|
|
|
void smc_smcr_terminate_all(struct smc_ib_device *smcibdev)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr, *lg;
|
|
|
|
LIST_HEAD(lgr_free_list);
|
|
|
|
|
|
|
|
spin_lock_bh(&smc_lgr_list.lock);
|
|
|
|
if (!smcibdev) {
|
|
|
|
list_splice_init(&smc_lgr_list.list, &lgr_free_list);
|
|
|
|
list_for_each_entry(lgr, &lgr_free_list, list)
|
|
|
|
lgr->freeing = 1;
|
|
|
|
} else {
|
|
|
|
list_for_each_entry_safe(lgr, lg, &smc_lgr_list.list, list) {
|
|
|
|
if (lgr->lnk[SMC_SINGLE_LINK].smcibdev == smcibdev) {
|
|
|
|
list_move(&lgr->list, &lgr_free_list);
|
|
|
|
lgr->freeing = 1;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
spin_unlock_bh(&smc_lgr_list.lock);
|
|
|
|
|
|
|
|
list_for_each_entry_safe(lgr, lg, &lgr_free_list, list) {
|
|
|
|
list_del_init(&lgr->list);
|
|
|
|
__smc_lgr_terminate(lgr, false);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:17 +03:00
|
|
|
/* Determine vlan of internal TCP socket.
|
|
|
|
* @vlan_id: address to store the determined vlan id into
|
|
|
|
*/
|
2019-04-12 13:57:26 +03:00
|
|
|
int smc_vlan_by_tcpsk(struct socket *clcsock, struct smc_init_info *ini)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
|
|
|
struct dst_entry *dst = sk_dst_get(clcsock->sk);
|
2018-05-02 17:56:47 +03:00
|
|
|
struct net_device *ndev;
|
|
|
|
int i, nest_lvl, rc = 0;
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2019-04-12 13:57:26 +03:00
|
|
|
ini->vlan_id = 0;
|
2017-01-09 18:55:17 +03:00
|
|
|
if (!dst) {
|
|
|
|
rc = -ENOTCONN;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
if (!dst->dev) {
|
|
|
|
rc = -ENODEV;
|
|
|
|
goto out_rel;
|
|
|
|
}
|
|
|
|
|
2018-05-02 17:56:47 +03:00
|
|
|
ndev = dst->dev;
|
|
|
|
if (is_vlan_dev(ndev)) {
|
2019-04-12 13:57:26 +03:00
|
|
|
ini->vlan_id = vlan_dev_vlan_id(ndev);
|
2018-05-02 17:56:47 +03:00
|
|
|
goto out_rel;
|
|
|
|
}
|
|
|
|
|
|
|
|
rtnl_lock();
|
2019-10-21 21:47:58 +03:00
|
|
|
nest_lvl = ndev->lower_level;
|
2018-05-02 17:56:47 +03:00
|
|
|
for (i = 0; i < nest_lvl; i++) {
|
|
|
|
struct list_head *lower = &ndev->adj_list.lower;
|
|
|
|
|
|
|
|
if (list_empty(lower))
|
|
|
|
break;
|
|
|
|
lower = lower->next;
|
|
|
|
ndev = (struct net_device *)netdev_lower_get_next(ndev, &lower);
|
|
|
|
if (is_vlan_dev(ndev)) {
|
2019-04-12 13:57:26 +03:00
|
|
|
ini->vlan_id = vlan_dev_vlan_id(ndev);
|
2018-05-02 17:56:47 +03:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
rtnl_unlock();
|
2017-01-09 18:55:17 +03:00
|
|
|
|
|
|
|
out_rel:
|
|
|
|
dst_release(dst);
|
|
|
|
out:
|
|
|
|
return rc;
|
|
|
|
}
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
static bool smcr_lgr_match(struct smc_link_group *lgr,
|
|
|
|
struct smc_clc_msg_local *lcl,
|
2018-11-20 18:46:40 +03:00
|
|
|
enum smc_lgr_role role, u32 clcqpn)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
return !memcmp(lgr->peer_systemid, lcl->id_for_peer,
|
|
|
|
SMC_SYSTEMID_LEN) &&
|
|
|
|
!memcmp(lgr->lnk[SMC_SINGLE_LINK].peer_gid, &lcl->gid,
|
|
|
|
SMC_GID_SIZE) &&
|
|
|
|
!memcmp(lgr->lnk[SMC_SINGLE_LINK].peer_mac, lcl->mac,
|
|
|
|
sizeof(lcl->mac)) &&
|
2018-11-20 18:46:40 +03:00
|
|
|
lgr->role == role &&
|
|
|
|
(lgr->role == SMC_SERV ||
|
|
|
|
lgr->lnk[SMC_SINGLE_LINK].peer_qpn == clcqpn);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
}
|
2017-01-09 18:55:17 +03:00
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
static bool smcd_lgr_match(struct smc_link_group *lgr,
|
|
|
|
struct smcd_dev *smcismdev, u64 peer_gid)
|
|
|
|
{
|
|
|
|
return lgr->peer_gid == peer_gid && lgr->smcd == smcismdev;
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
/* create a new SMC connection (and a new link group if necessary) */
|
2019-04-12 13:57:26 +03:00
|
|
|
int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
|
2017-01-09 18:55:17 +03:00
|
|
|
{
|
|
|
|
struct smc_connection *conn = &smc->conn;
|
2019-10-09 11:07:43 +03:00
|
|
|
struct list_head *lgr_list;
|
2017-01-09 18:55:17 +03:00
|
|
|
struct smc_link_group *lgr;
|
|
|
|
enum smc_lgr_role role;
|
2019-10-09 11:07:44 +03:00
|
|
|
spinlock_t *lgr_lock;
|
2017-01-09 18:55:17 +03:00
|
|
|
int rc = 0;
|
|
|
|
|
2019-10-09 11:07:43 +03:00
|
|
|
lgr_list = ini->is_smcd ? &ini->ism_dev->lgr_list : &smc_lgr_list.list;
|
2019-10-09 11:07:44 +03:00
|
|
|
lgr_lock = ini->is_smcd ? &ini->ism_dev->lgr_lock : &smc_lgr_list.lock;
|
2019-04-12 13:57:30 +03:00
|
|
|
ini->cln_first_contact = SMC_FIRST_CONTACT;
|
2017-01-09 18:55:17 +03:00
|
|
|
role = smc->listen_smc ? SMC_SERV : SMC_CLNT;
|
2019-04-12 13:57:26 +03:00
|
|
|
if (role == SMC_CLNT && ini->srv_first_contact)
|
2017-01-09 18:55:17 +03:00
|
|
|
/* create new link group as well */
|
|
|
|
goto create;
|
|
|
|
|
|
|
|
/* determine if an existing link group can be reused */
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_lock_bh(lgr_lock);
|
2019-10-09 11:07:43 +03:00
|
|
|
list_for_each_entry(lgr, lgr_list, list) {
|
2017-01-09 18:55:17 +03:00
|
|
|
write_lock_bh(&lgr->conns_lock);
|
2019-04-12 13:57:26 +03:00
|
|
|
if ((ini->is_smcd ?
|
|
|
|
smcd_lgr_match(lgr, ini->ism_dev, ini->ism_gid) :
|
|
|
|
smcr_lgr_match(lgr, ini->ib_lcl, role, ini->ib_clcqpn)) &&
|
2017-01-09 18:55:17 +03:00
|
|
|
!lgr->sync_err &&
|
2019-04-12 13:57:26 +03:00
|
|
|
lgr->vlan_id == ini->vlan_id &&
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
(role == SMC_CLNT ||
|
|
|
|
lgr->conns_num < SMC_RMBS_PER_LGR_MAX)) {
|
2017-01-09 18:55:17 +03:00
|
|
|
/* link group found */
|
2019-04-12 13:57:30 +03:00
|
|
|
ini->cln_first_contact = SMC_REUSE_CONTACT;
|
2017-01-09 18:55:17 +03:00
|
|
|
conn->lgr = lgr;
|
|
|
|
smc_lgr_register_conn(conn); /* add smc conn to lgr */
|
2019-01-30 20:51:02 +03:00
|
|
|
if (delayed_work_pending(&lgr->free_work))
|
|
|
|
cancel_delayed_work(&lgr->free_work);
|
2017-01-09 18:55:17 +03:00
|
|
|
write_unlock_bh(&lgr->conns_lock);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
write_unlock_bh(&lgr->conns_lock);
|
|
|
|
}
|
2019-10-09 11:07:44 +03:00
|
|
|
spin_unlock_bh(lgr_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
|
2019-04-12 13:57:26 +03:00
|
|
|
if (role == SMC_CLNT && !ini->srv_first_contact &&
|
2019-04-12 13:57:30 +03:00
|
|
|
ini->cln_first_contact == SMC_FIRST_CONTACT) {
|
2017-01-09 18:55:17 +03:00
|
|
|
/* Server reuses a link group, but Client wants to start
|
|
|
|
* a new one
|
|
|
|
* send out_of_sync decline, reason synchr. error
|
|
|
|
*/
|
2019-04-12 13:57:30 +03:00
|
|
|
return SMC_CLC_DECL_SYNCERR;
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
create:
|
2019-04-12 13:57:30 +03:00
|
|
|
if (ini->cln_first_contact == SMC_FIRST_CONTACT) {
|
2019-04-12 13:57:26 +03:00
|
|
|
rc = smc_lgr_create(smc, ini);
|
2017-01-09 18:55:17 +03:00
|
|
|
if (rc)
|
|
|
|
goto out;
|
2019-06-26 18:47:49 +03:00
|
|
|
lgr = conn->lgr;
|
|
|
|
write_lock_bh(&lgr->conns_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
smc_lgr_register_conn(conn); /* add smc conn to lgr */
|
2019-06-26 18:47:49 +03:00
|
|
|
write_unlock_bh(&lgr->conns_lock);
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
2017-01-09 18:55:22 +03:00
|
|
|
conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
|
2018-02-28 14:44:07 +03:00
|
|
|
conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
|
2018-05-23 17:38:11 +03:00
|
|
|
conn->urg_state = SMC_URG_READ;
|
2019-04-12 13:57:26 +03:00
|
|
|
if (ini->is_smcd) {
|
2018-06-28 20:05:10 +03:00
|
|
|
conn->rx_off = sizeof(struct smcd_cdc_msg);
|
|
|
|
smcd_cdc_rx_init(conn); /* init tasklet for this conn */
|
|
|
|
}
|
2017-01-09 18:55:22 +03:00
|
|
|
#ifndef KERNEL_HAS_ATOMIC64
|
|
|
|
spin_lock_init(&conn->acurs_lock);
|
|
|
|
#endif
|
2017-01-09 18:55:17 +03:00
|
|
|
|
|
|
|
out:
|
2019-04-12 13:57:30 +03:00
|
|
|
return rc;
|
2017-01-09 18:55:17 +03:00
|
|
|
}
|
2017-01-09 18:55:18 +03:00
|
|
|
|
2018-05-18 10:34:14 +03:00
|
|
|
/* convert the RMB size into the compressed notation - minimum 16K.
|
|
|
|
* In contrast to plain ilog2, this rounds towards the next power of 2,
|
|
|
|
* so the socket application gets at least its desired sndbuf / rcvbuf size.
|
|
|
|
*/
|
|
|
|
static u8 smc_compress_bufsize(int size)
|
|
|
|
{
|
|
|
|
u8 compressed;
|
|
|
|
|
|
|
|
if (size <= SMC_BUF_MIN_SIZE)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
size = (size - 1) >> 14;
|
|
|
|
compressed = ilog2(size) + 1;
|
|
|
|
if (compressed >= SMC_RMBE_SIZES)
|
|
|
|
compressed = SMC_RMBE_SIZES - 1;
|
|
|
|
return compressed;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* convert the RMB size from compressed notation into integer */
|
|
|
|
int smc_uncompress_bufsize(u8 compressed)
|
|
|
|
{
|
|
|
|
u32 size;
|
|
|
|
|
|
|
|
size = 0x00000001 << (((int)compressed) + 14);
|
|
|
|
return (int)size;
|
|
|
|
}
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
/* try to reuse a sndbuf or rmb description slot for a certain
|
|
|
|
* buffer size; if not available, return NULL
|
2017-01-09 18:55:18 +03:00
|
|
|
*/
|
2018-05-18 10:34:16 +03:00
|
|
|
static struct smc_buf_desc *smc_buf_get_slot(int compressed_bufsize,
|
|
|
|
rwlock_t *lock,
|
|
|
|
struct list_head *buf_list)
|
2017-01-09 18:55:18 +03:00
|
|
|
{
|
2017-07-28 14:56:20 +03:00
|
|
|
struct smc_buf_desc *buf_slot;
|
2017-01-09 18:55:18 +03:00
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
read_lock_bh(lock);
|
|
|
|
list_for_each_entry(buf_slot, buf_list, list) {
|
|
|
|
if (cmpxchg(&buf_slot->used, 0, 1) == 0) {
|
|
|
|
read_unlock_bh(lock);
|
|
|
|
return buf_slot;
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
|
|
|
}
|
2017-07-28 14:56:20 +03:00
|
|
|
read_unlock_bh(lock);
|
2017-01-09 18:55:18 +03:00
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2017-01-09 18:55:24 +03:00
|
|
|
/* one of the conditions for announcing a receiver's current window size is
|
|
|
|
* that it "results in a minimum increase in the window size of 10% of the
|
|
|
|
* receive buffer space" [RFC7609]
|
|
|
|
*/
|
|
|
|
static inline int smc_rmb_wnd_update_limit(int rmbe_size)
|
|
|
|
{
|
|
|
|
return min_t(int, rmbe_size / 10, SOCK_MIN_SNDBUF / 2);
|
|
|
|
}
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
static struct smc_buf_desc *smcr_new_buf_create(struct smc_link_group *lgr,
|
|
|
|
bool is_rmb, int bufsize)
|
2017-07-28 14:56:21 +03:00
|
|
|
{
|
|
|
|
struct smc_buf_desc *buf_desc;
|
|
|
|
struct smc_link *lnk;
|
|
|
|
int rc;
|
|
|
|
|
|
|
|
/* try to alloc a new buffer */
|
|
|
|
buf_desc = kzalloc(sizeof(*buf_desc), GFP_KERNEL);
|
|
|
|
if (!buf_desc)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
2018-05-03 19:12:38 +03:00
|
|
|
buf_desc->order = get_order(bufsize);
|
|
|
|
buf_desc->pages = alloc_pages(GFP_KERNEL | __GFP_NOWARN |
|
|
|
|
__GFP_NOMEMALLOC | __GFP_COMP |
|
|
|
|
__GFP_NORETRY | __GFP_ZERO,
|
|
|
|
buf_desc->order);
|
|
|
|
if (!buf_desc->pages) {
|
2017-07-28 14:56:21 +03:00
|
|
|
kfree(buf_desc);
|
|
|
|
return ERR_PTR(-EAGAIN);
|
|
|
|
}
|
2018-05-03 19:12:38 +03:00
|
|
|
buf_desc->cpu_addr = (void *)page_address(buf_desc->pages);
|
2017-07-28 14:56:21 +03:00
|
|
|
|
|
|
|
/* build the sg table from the pages */
|
|
|
|
lnk = &lgr->lnk[SMC_SINGLE_LINK];
|
|
|
|
rc = sg_alloc_table(&buf_desc->sgt[SMC_SINGLE_LINK], 1,
|
|
|
|
GFP_KERNEL);
|
|
|
|
if (rc) {
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(lgr, is_rmb, buf_desc);
|
2017-07-28 14:56:21 +03:00
|
|
|
return ERR_PTR(rc);
|
|
|
|
}
|
|
|
|
sg_set_buf(buf_desc->sgt[SMC_SINGLE_LINK].sgl,
|
|
|
|
buf_desc->cpu_addr, bufsize);
|
|
|
|
|
|
|
|
/* map sg table to DMA address */
|
|
|
|
rc = smc_ib_buf_map_sg(lnk->smcibdev, buf_desc,
|
|
|
|
is_rmb ? DMA_FROM_DEVICE : DMA_TO_DEVICE);
|
|
|
|
/* SMC protocol depends on mapping to one DMA address only */
|
|
|
|
if (rc != 1) {
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(lgr, is_rmb, buf_desc);
|
2017-07-28 14:56:21 +03:00
|
|
|
return ERR_PTR(-EAGAIN);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* create a new memory region for the RMB */
|
|
|
|
if (is_rmb) {
|
|
|
|
rc = smc_ib_get_memory_region(lnk->roce_pd,
|
|
|
|
IB_ACCESS_REMOTE_WRITE |
|
|
|
|
IB_ACCESS_LOCAL_WRITE,
|
|
|
|
buf_desc);
|
|
|
|
if (rc) {
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(lgr, is_rmb, buf_desc);
|
2017-07-28 14:56:21 +03:00
|
|
|
return ERR_PTR(rc);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-05-18 10:34:10 +03:00
|
|
|
buf_desc->len = bufsize;
|
2017-07-28 14:56:21 +03:00
|
|
|
return buf_desc;
|
|
|
|
}
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
#define SMCD_DMBE_SIZES 7 /* 0 -> 16KB, 1 -> 32KB, .. 6 -> 1MB */
|
|
|
|
|
|
|
|
static struct smc_buf_desc *smcd_new_buf_create(struct smc_link_group *lgr,
|
|
|
|
bool is_dmb, int bufsize)
|
|
|
|
{
|
|
|
|
struct smc_buf_desc *buf_desc;
|
|
|
|
int rc;
|
|
|
|
|
|
|
|
if (smc_compress_bufsize(bufsize) > SMCD_DMBE_SIZES)
|
|
|
|
return ERR_PTR(-EAGAIN);
|
|
|
|
|
|
|
|
/* try to alloc a new DMB */
|
|
|
|
buf_desc = kzalloc(sizeof(*buf_desc), GFP_KERNEL);
|
|
|
|
if (!buf_desc)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
if (is_dmb) {
|
|
|
|
rc = smc_ism_register_dmb(lgr, bufsize, buf_desc);
|
|
|
|
if (rc) {
|
|
|
|
kfree(buf_desc);
|
|
|
|
return ERR_PTR(-EAGAIN);
|
|
|
|
}
|
2018-06-28 20:05:10 +03:00
|
|
|
buf_desc->pages = virt_to_page(buf_desc->cpu_addr);
|
|
|
|
/* CDC header stored in buf. So, pretend it was smaller */
|
|
|
|
buf_desc->len = bufsize - sizeof(struct smcd_cdc_msg);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
} else {
|
|
|
|
buf_desc->cpu_addr = kzalloc(bufsize, GFP_KERNEL |
|
|
|
|
__GFP_NOWARN | __GFP_NORETRY |
|
|
|
|
__GFP_NOMEMALLOC);
|
|
|
|
if (!buf_desc->cpu_addr) {
|
|
|
|
kfree(buf_desc);
|
|
|
|
return ERR_PTR(-EAGAIN);
|
|
|
|
}
|
|
|
|
buf_desc->len = bufsize;
|
|
|
|
}
|
|
|
|
return buf_desc;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int __smc_buf_create(struct smc_sock *smc, bool is_smcd, bool is_rmb)
|
2017-01-09 18:55:18 +03:00
|
|
|
{
|
2018-05-18 10:34:16 +03:00
|
|
|
struct smc_buf_desc *buf_desc = ERR_PTR(-ENOMEM);
|
2017-01-09 18:55:18 +03:00
|
|
|
struct smc_connection *conn = &smc->conn;
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
2017-07-28 14:56:20 +03:00
|
|
|
struct list_head *buf_list;
|
2017-07-28 14:56:14 +03:00
|
|
|
int bufsize, bufsize_short;
|
2017-07-28 14:56:20 +03:00
|
|
|
int sk_buf_size;
|
|
|
|
rwlock_t *lock;
|
2017-01-09 18:55:18 +03:00
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
if (is_rmb)
|
|
|
|
/* use socket recv buffer size (w/o overhead) as start value */
|
|
|
|
sk_buf_size = smc->sk.sk_rcvbuf / 2;
|
|
|
|
else
|
|
|
|
/* use socket send buffer size (w/o overhead) as start value */
|
|
|
|
sk_buf_size = smc->sk.sk_sndbuf / 2;
|
|
|
|
|
2017-11-21 15:23:53 +03:00
|
|
|
for (bufsize_short = smc_compress_bufsize(sk_buf_size);
|
2017-07-28 14:56:14 +03:00
|
|
|
bufsize_short >= 0; bufsize_short--) {
|
2017-07-28 14:56:19 +03:00
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
if (is_rmb) {
|
|
|
|
lock = &lgr->rmbs_lock;
|
|
|
|
buf_list = &lgr->rmbs[bufsize_short];
|
|
|
|
} else {
|
|
|
|
lock = &lgr->sndbufs_lock;
|
|
|
|
buf_list = &lgr->sndbufs[bufsize_short];
|
2017-07-28 14:56:19 +03:00
|
|
|
}
|
2017-07-28 14:56:14 +03:00
|
|
|
bufsize = smc_uncompress_bufsize(bufsize_short);
|
2017-07-28 14:56:15 +03:00
|
|
|
if ((1 << get_order(bufsize)) > SG_MAX_SINGLE_ALLOC)
|
|
|
|
continue;
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
/* check for reusable slot in the link group */
|
2018-05-18 10:34:16 +03:00
|
|
|
buf_desc = smc_buf_get_slot(bufsize_short, lock, buf_list);
|
2017-07-28 14:56:20 +03:00
|
|
|
if (buf_desc) {
|
|
|
|
memset(buf_desc->cpu_addr, 0, bufsize);
|
2017-01-09 18:55:18 +03:00
|
|
|
break; /* found reusable slot */
|
|
|
|
}
|
2017-07-28 14:56:15 +03:00
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (is_smcd)
|
|
|
|
buf_desc = smcd_new_buf_create(lgr, is_rmb, bufsize);
|
|
|
|
else
|
|
|
|
buf_desc = smcr_new_buf_create(lgr, is_rmb, bufsize);
|
|
|
|
|
2017-07-28 14:56:21 +03:00
|
|
|
if (PTR_ERR(buf_desc) == -ENOMEM)
|
|
|
|
break;
|
|
|
|
if (IS_ERR(buf_desc))
|
2017-07-28 14:56:15 +03:00
|
|
|
continue;
|
2017-07-28 14:56:16 +03:00
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
buf_desc->used = 1;
|
|
|
|
write_lock_bh(lock);
|
|
|
|
list_add(&buf_desc->list, buf_list);
|
|
|
|
write_unlock_bh(lock);
|
|
|
|
break; /* found */
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
2017-07-28 14:56:20 +03:00
|
|
|
|
2017-07-28 14:56:21 +03:00
|
|
|
if (IS_ERR(buf_desc))
|
2017-07-28 14:56:20 +03:00
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
if (is_rmb) {
|
|
|
|
conn->rmb_desc = buf_desc;
|
2017-07-28 14:56:14 +03:00
|
|
|
conn->rmbe_size_short = bufsize_short;
|
|
|
|
smc->sk.sk_rcvbuf = bufsize * 2;
|
2017-01-09 18:55:22 +03:00
|
|
|
atomic_set(&conn->bytes_to_rcv, 0);
|
2018-06-28 20:05:10 +03:00
|
|
|
conn->rmbe_update_limit =
|
|
|
|
smc_rmb_wnd_update_limit(buf_desc->len);
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (is_smcd)
|
|
|
|
smc_ism_set_conn(conn); /* map RMB/smcd_dev to conn */
|
2017-01-09 18:55:18 +03:00
|
|
|
} else {
|
2017-07-28 14:56:20 +03:00
|
|
|
conn->sndbuf_desc = buf_desc;
|
|
|
|
smc->sk.sk_sndbuf = bufsize * 2;
|
|
|
|
atomic_set(&conn->sndbuf_space, bufsize);
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
2017-07-28 14:56:20 +03:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2017-07-28 14:56:22 +03:00
|
|
|
void smc_sndbuf_sync_sg_for_cpu(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!conn->lgr || conn->lgr->is_smcd)
|
|
|
|
return;
|
2017-07-28 14:56:22 +03:00
|
|
|
smc_ib_sync_sg_for_cpu(lgr->lnk[SMC_SINGLE_LINK].smcibdev,
|
|
|
|
conn->sndbuf_desc, DMA_TO_DEVICE);
|
|
|
|
}
|
|
|
|
|
|
|
|
void smc_sndbuf_sync_sg_for_device(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!conn->lgr || conn->lgr->is_smcd)
|
|
|
|
return;
|
2017-07-28 14:56:22 +03:00
|
|
|
smc_ib_sync_sg_for_device(lgr->lnk[SMC_SINGLE_LINK].smcibdev,
|
|
|
|
conn->sndbuf_desc, DMA_TO_DEVICE);
|
|
|
|
}
|
|
|
|
|
|
|
|
void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!conn->lgr || conn->lgr->is_smcd)
|
|
|
|
return;
|
2017-07-28 14:56:22 +03:00
|
|
|
smc_ib_sync_sg_for_cpu(lgr->lnk[SMC_SINGLE_LINK].smcibdev,
|
|
|
|
conn->rmb_desc, DMA_FROM_DEVICE);
|
|
|
|
}
|
|
|
|
|
|
|
|
void smc_rmb_sync_sg_for_device(struct smc_connection *conn)
|
|
|
|
{
|
|
|
|
struct smc_link_group *lgr = conn->lgr;
|
|
|
|
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
if (!conn->lgr || conn->lgr->is_smcd)
|
|
|
|
return;
|
2017-07-28 14:56:22 +03:00
|
|
|
smc_ib_sync_sg_for_device(lgr->lnk[SMC_SINGLE_LINK].smcibdev,
|
|
|
|
conn->rmb_desc, DMA_FROM_DEVICE);
|
|
|
|
}
|
|
|
|
|
2017-07-28 14:56:20 +03:00
|
|
|
/* create the send and receive buffer for an SMC socket;
|
|
|
|
* receive buffers are called RMBs;
|
|
|
|
* (even though the SMC protocol allows more than one RMB-element per RMB,
|
|
|
|
* the Linux implementation uses just one RMB-element per RMB, i.e. uses an
|
|
|
|
* extra RMB for every connection in a link group
|
|
|
|
*/
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
int smc_buf_create(struct smc_sock *smc, bool is_smcd)
|
2017-07-28 14:56:20 +03:00
|
|
|
{
|
|
|
|
int rc;
|
|
|
|
|
|
|
|
/* create send buffer */
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
rc = __smc_buf_create(smc, is_smcd, false);
|
2017-07-28 14:56:20 +03:00
|
|
|
if (rc)
|
|
|
|
return rc;
|
|
|
|
/* create rmb */
|
net/smc: add base infrastructure for SMC-D and ISM
SMC supports two variants: SMC-R and SMC-D. For data transport, SMC-R
uses RDMA devices, SMC-D uses so-called Internal Shared Memory (ISM)
devices. An ISM device only allows shared memory communication between
SMC instances on the same machine. For example, this allows virtual
machines on the same host to communicate via SMC without RDMA devices.
This patch adds the base infrastructure for SMC-D and ISM devices to
the existing SMC code. It contains the following:
* ISM driver interface:
This interface allows an ISM driver to register ISM devices in SMC. In
the process, the driver provides a set of device ops for each device.
SMC uses these ops to execute SMC specific operations on or transfer
data over the device.
* Core SMC-D link group, connection, and buffer support:
Link groups, SMC connections and SMC buffers (in smc_core) are
extended to support SMC-D.
* SMC type checks:
Some type checks are added to prevent using SMC-R specific code for
SMC-D and vice versa.
To actually use SMC-D, additional changes to pnetid, CLC, CDC, etc. are
required. These are added in follow-up patches.
Signed-off-by: Hans Wippel <hwippel@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Suggested-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-06-28 20:05:07 +03:00
|
|
|
rc = __smc_buf_create(smc, is_smcd, true);
|
2017-07-28 14:56:20 +03:00
|
|
|
if (rc)
|
2018-05-18 10:34:17 +03:00
|
|
|
smc_buf_free(smc->conn.lgr, false, smc->conn.sndbuf_desc);
|
2017-07-28 14:56:20 +03:00
|
|
|
return rc;
|
2017-01-09 18:55:18 +03:00
|
|
|
}
|
2017-01-09 18:55:20 +03:00
|
|
|
|
|
|
|
static inline int smc_rmb_reserve_rtoken_idx(struct smc_link_group *lgr)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for_each_clear_bit(i, lgr->rtokens_used_mask, SMC_RMBS_PER_LGR_MAX) {
|
|
|
|
if (!test_and_set_bit(i, lgr->rtokens_used_mask))
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
return -ENOSPC;
|
|
|
|
}
|
|
|
|
|
2018-03-01 15:51:30 +03:00
|
|
|
/* add a new rtoken from peer */
|
|
|
|
int smc_rtoken_add(struct smc_link_group *lgr, __be64 nw_vaddr, __be32 nw_rkey)
|
2017-01-09 18:55:20 +03:00
|
|
|
{
|
2018-03-01 15:51:30 +03:00
|
|
|
u64 dma_addr = be64_to_cpu(nw_vaddr);
|
|
|
|
u32 rkey = ntohl(nw_rkey);
|
2017-01-09 18:55:20 +03:00
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < SMC_RMBS_PER_LGR_MAX; i++) {
|
|
|
|
if ((lgr->rtokens[i][SMC_SINGLE_LINK].rkey == rkey) &&
|
2017-05-15 18:33:37 +03:00
|
|
|
(lgr->rtokens[i][SMC_SINGLE_LINK].dma_addr == dma_addr) &&
|
2017-01-09 18:55:20 +03:00
|
|
|
test_bit(i, lgr->rtokens_used_mask)) {
|
2018-03-01 15:51:30 +03:00
|
|
|
/* already in list */
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
i = smc_rmb_reserve_rtoken_idx(lgr);
|
|
|
|
if (i < 0)
|
|
|
|
return i;
|
|
|
|
lgr->rtokens[i][SMC_SINGLE_LINK].rkey = rkey;
|
|
|
|
lgr->rtokens[i][SMC_SINGLE_LINK].dma_addr = dma_addr;
|
|
|
|
return i;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* delete an rtoken */
|
|
|
|
int smc_rtoken_delete(struct smc_link_group *lgr, __be32 nw_rkey)
|
|
|
|
{
|
|
|
|
u32 rkey = ntohl(nw_rkey);
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < SMC_RMBS_PER_LGR_MAX; i++) {
|
|
|
|
if (lgr->rtokens[i][SMC_SINGLE_LINK].rkey == rkey &&
|
|
|
|
test_bit(i, lgr->rtokens_used_mask)) {
|
|
|
|
lgr->rtokens[i][SMC_SINGLE_LINK].rkey = 0;
|
|
|
|
lgr->rtokens[i][SMC_SINGLE_LINK].dma_addr = 0;
|
|
|
|
|
|
|
|
clear_bit(i, lgr->rtokens_used_mask);
|
2017-01-09 18:55:20 +03:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
}
|
2018-03-01 15:51:30 +03:00
|
|
|
return -ENOENT;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* save rkey and dma_addr received from peer during clc handshake */
|
|
|
|
int smc_rmb_rtoken_handling(struct smc_connection *conn,
|
|
|
|
struct smc_clc_msg_accept_confirm *clc)
|
|
|
|
{
|
|
|
|
conn->rtoken_idx = smc_rtoken_add(conn->lgr, clc->rmb_dma_addr,
|
|
|
|
clc->rmb_rkey);
|
2017-01-09 18:55:20 +03:00
|
|
|
if (conn->rtoken_idx < 0)
|
|
|
|
return conn->rtoken_idx;
|
|
|
|
return 0;
|
|
|
|
}
|
2018-05-18 10:34:11 +03:00
|
|
|
|
2019-10-09 11:07:46 +03:00
|
|
|
static void smc_core_going_away(void)
|
|
|
|
{
|
|
|
|
struct smc_ib_device *smcibdev;
|
|
|
|
struct smcd_dev *smcd;
|
|
|
|
|
|
|
|
spin_lock(&smc_ib_devices.lock);
|
|
|
|
list_for_each_entry(smcibdev, &smc_ib_devices.list, list) {
|
|
|
|
int i;
|
|
|
|
|
|
|
|
for (i = 0; i < SMC_MAX_PORTS; i++)
|
|
|
|
set_bit(i, smcibdev->ports_going_away);
|
|
|
|
}
|
|
|
|
spin_unlock(&smc_ib_devices.lock);
|
|
|
|
|
|
|
|
spin_lock(&smcd_dev_list.lock);
|
|
|
|
list_for_each_entry(smcd, &smcd_dev_list.list, list) {
|
|
|
|
smcd->going_away = 1;
|
|
|
|
}
|
|
|
|
spin_unlock(&smcd_dev_list.lock);
|
|
|
|
}
|
|
|
|
|
2019-11-14 15:02:42 +03:00
|
|
|
/* Clean up all SMC link groups */
|
|
|
|
static void smc_lgrs_shutdown(void)
|
2018-05-18 10:34:11 +03:00
|
|
|
{
|
2019-10-09 11:07:43 +03:00
|
|
|
struct smcd_dev *smcd;
|
2018-05-18 10:34:11 +03:00
|
|
|
|
2019-10-09 11:07:46 +03:00
|
|
|
smc_core_going_away();
|
|
|
|
|
2019-11-14 15:02:47 +03:00
|
|
|
smc_smcr_terminate_all(NULL);
|
2019-10-09 11:07:43 +03:00
|
|
|
|
|
|
|
spin_lock(&smcd_dev_list.lock);
|
|
|
|
list_for_each_entry(smcd, &smcd_dev_list.list, list)
|
2019-11-14 15:02:42 +03:00
|
|
|
smc_smcd_terminate_all(smcd);
|
2019-10-09 11:07:43 +03:00
|
|
|
spin_unlock(&smcd_dev_list.lock);
|
2018-05-18 10:34:11 +03:00
|
|
|
}
|
2019-11-14 15:02:42 +03:00
|
|
|
|
|
|
|
/* Called (from smc_exit) when module is removed */
|
|
|
|
void smc_core_exit(void)
|
|
|
|
{
|
|
|
|
smc_lgrs_shutdown();
|
|
|
|
}
|