2019-05-27 09:55:01 +03:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-or-later
|
2007-04-27 02:55:03 +04:00
|
|
|
/* AFS filesystem file handling
|
2005-04-17 02:20:36 +04:00
|
|
|
*
|
2007-04-27 02:55:03 +04:00
|
|
|
* Copyright (C) 2002, 2007 Red Hat, Inc. All Rights Reserved.
|
2005-04-17 02:20:36 +04:00
|
|
|
* Written by David Howells (dhowells@redhat.com)
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/kernel.h>
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/fs.h>
|
|
|
|
#include <linux/pagemap.h>
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 13:33:46 +04:00
|
|
|
#include <linux/writeback.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 11:04:11 +03:00
|
|
|
#include <linux/gfp.h>
|
2017-01-05 13:38:35 +03:00
|
|
|
#include <linux/task_io_accounting_ops.h>
|
2019-01-04 02:29:02 +03:00
|
|
|
#include <linux/mm.h>
|
2020-02-06 17:22:29 +03:00
|
|
|
#include <linux/netfs.h>
|
2005-04-17 02:20:36 +04:00
|
|
|
#include "internal.h"
|
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
static int afs_file_mmap(struct file *file, struct vm_area_struct *vma);
|
2007-05-09 13:33:45 +04:00
|
|
|
static int afs_readpage(struct file *file, struct page *page);
|
2013-05-22 07:17:23 +04:00
|
|
|
static void afs_invalidatepage(struct page *page, unsigned int offset,
|
|
|
|
unsigned int length);
|
2007-05-09 13:33:45 +04:00
|
|
|
static int afs_releasepage(struct page *page, gfp_t gfp_flags);
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static void afs_readahead(struct readahead_control *ractl);
|
2009-04-03 19:42:41 +04:00
|
|
|
|
2007-04-27 02:57:07 +04:00
|
|
|
const struct file_operations afs_file_operations = {
|
|
|
|
.open = afs_open,
|
|
|
|
.release = afs_release,
|
|
|
|
.llseek = generic_file_llseek,
|
2014-04-02 22:33:16 +04:00
|
|
|
.read_iter = generic_file_read_iter,
|
2014-04-03 22:13:46 +04:00
|
|
|
.write_iter = afs_file_write,
|
2017-11-02 18:27:52 +03:00
|
|
|
.mmap = afs_file_mmap,
|
2007-06-01 13:49:19 +04:00
|
|
|
.splice_read = generic_file_splice_read,
|
2020-10-27 12:39:04 +03:00
|
|
|
.splice_write = iter_file_splice_write,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 13:33:46 +04:00
|
|
|
.fsync = afs_fsync,
|
2007-07-16 10:40:12 +04:00
|
|
|
.lock = afs_lock,
|
|
|
|
.flock = afs_flock,
|
2007-04-27 02:57:07 +04:00
|
|
|
};
|
|
|
|
|
2007-02-12 11:55:38 +03:00
|
|
|
const struct inode_operations afs_file_inode_operations = {
|
2007-05-09 13:33:45 +04:00
|
|
|
.getattr = afs_getattr,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 13:33:46 +04:00
|
|
|
.setattr = afs_setattr,
|
2007-04-27 02:57:07 +04:00
|
|
|
.permission = afs_permission,
|
2005-04-17 02:20:36 +04:00
|
|
|
};
|
|
|
|
|
2006-06-28 15:26:44 +04:00
|
|
|
const struct address_space_operations afs_fs_aops = {
|
2007-05-09 13:33:45 +04:00
|
|
|
.readpage = afs_readpage,
|
2020-02-06 17:22:29 +03:00
|
|
|
.readahead = afs_readahead,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 13:33:46 +04:00
|
|
|
.set_page_dirty = afs_set_page_dirty,
|
|
|
|
.launder_page = afs_launder_page,
|
2007-05-09 13:33:45 +04:00
|
|
|
.releasepage = afs_releasepage,
|
|
|
|
.invalidatepage = afs_invalidatepage,
|
2008-10-16 09:04:32 +04:00
|
|
|
.write_begin = afs_write_begin,
|
|
|
|
.write_end = afs_write_end,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 13:33:46 +04:00
|
|
|
.writepage = afs_writepage,
|
|
|
|
.writepages = afs_writepages,
|
2005-04-17 02:20:36 +04:00
|
|
|
};
|
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
static const struct vm_operations_struct afs_vm_ops = {
|
|
|
|
.fault = filemap_fault,
|
|
|
|
.map_pages = filemap_map_pages,
|
|
|
|
.page_mkwrite = afs_page_mkwrite,
|
|
|
|
};
|
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
/*
|
|
|
|
* Discard a pin on a writeback key.
|
|
|
|
*/
|
|
|
|
void afs_put_wb_key(struct afs_wb_key *wbk)
|
|
|
|
{
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
if (wbk && refcount_dec_and_test(&wbk->usage)) {
|
2017-11-02 18:27:52 +03:00
|
|
|
key_put(wbk->key);
|
|
|
|
kfree(wbk);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Cache key for writeback.
|
|
|
|
*/
|
|
|
|
int afs_cache_wb_key(struct afs_vnode *vnode, struct afs_file *af)
|
|
|
|
{
|
|
|
|
struct afs_wb_key *wbk, *p;
|
|
|
|
|
|
|
|
wbk = kzalloc(sizeof(struct afs_wb_key), GFP_KERNEL);
|
|
|
|
if (!wbk)
|
|
|
|
return -ENOMEM;
|
|
|
|
refcount_set(&wbk->usage, 2);
|
|
|
|
wbk->key = af->key;
|
|
|
|
|
|
|
|
spin_lock(&vnode->wb_lock);
|
|
|
|
list_for_each_entry(p, &vnode->wb_keys, vnode_link) {
|
|
|
|
if (p->key == wbk->key)
|
|
|
|
goto found;
|
|
|
|
}
|
|
|
|
|
|
|
|
key_get(wbk->key);
|
|
|
|
list_add_tail(&wbk->vnode_link, &vnode->wb_keys);
|
|
|
|
spin_unlock(&vnode->wb_lock);
|
|
|
|
af->wb = wbk;
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
found:
|
|
|
|
refcount_inc(&p->usage);
|
|
|
|
spin_unlock(&vnode->wb_lock);
|
|
|
|
af->wb = p;
|
|
|
|
kfree(wbk);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-04-27 02:57:07 +04:00
|
|
|
/*
|
|
|
|
* open an AFS file or directory and attach a key to it
|
|
|
|
*/
|
|
|
|
int afs_open(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(inode);
|
2017-11-02 18:27:52 +03:00
|
|
|
struct afs_file *af;
|
2007-04-27 02:57:07 +04:00
|
|
|
struct key *key;
|
2007-04-27 02:59:35 +04:00
|
|
|
int ret;
|
2007-04-27 02:57:07 +04:00
|
|
|
|
2018-10-20 02:57:57 +03:00
|
|
|
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
|
2007-04-27 02:57:07 +04:00
|
|
|
|
|
|
|
key = afs_request_key(vnode->volume->cell);
|
|
|
|
if (IS_ERR(key)) {
|
2017-11-02 18:27:52 +03:00
|
|
|
ret = PTR_ERR(key);
|
|
|
|
goto error;
|
2007-04-27 02:57:07 +04:00
|
|
|
}
|
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
af = kzalloc(sizeof(*af), GFP_KERNEL);
|
|
|
|
if (!af) {
|
|
|
|
ret = -ENOMEM;
|
|
|
|
goto error_key;
|
2007-04-27 02:59:35 +04:00
|
|
|
}
|
2017-11-02 18:27:52 +03:00
|
|
|
af->key = key;
|
2007-04-27 02:59:35 +04:00
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
ret = afs_validate(vnode, key);
|
|
|
|
if (ret < 0)
|
|
|
|
goto error_af;
|
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
if (file->f_mode & FMODE_WRITE) {
|
|
|
|
ret = afs_cache_wb_key(vnode, af);
|
|
|
|
if (ret < 0)
|
|
|
|
goto error_af;
|
|
|
|
}
|
2018-04-06 16:17:26 +03:00
|
|
|
|
|
|
|
if (file->f_flags & O_TRUNC)
|
|
|
|
set_bit(AFS_VNODE_NEW_CONTENT, &vnode->flags);
|
2017-11-02 18:27:52 +03:00
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
file->private_data = af;
|
2007-04-27 02:57:07 +04:00
|
|
|
_leave(" = 0");
|
|
|
|
return 0;
|
2017-11-02 18:27:52 +03:00
|
|
|
|
|
|
|
error_af:
|
|
|
|
kfree(af);
|
|
|
|
error_key:
|
|
|
|
key_put(key);
|
|
|
|
error:
|
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2007-04-27 02:57:07 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* release an AFS file or directory and discard its key
|
|
|
|
*/
|
|
|
|
int afs_release(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(inode);
|
2017-11-02 18:27:52 +03:00
|
|
|
struct afs_file *af = file->private_data;
|
2019-05-15 14:09:17 +03:00
|
|
|
int ret = 0;
|
2007-04-27 02:57:07 +04:00
|
|
|
|
2018-10-20 02:57:57 +03:00
|
|
|
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
|
2007-04-27 02:57:07 +04:00
|
|
|
|
2018-04-06 16:17:26 +03:00
|
|
|
if ((file->f_mode & FMODE_WRITE))
|
2019-05-15 14:09:17 +03:00
|
|
|
ret = vfs_fsync(file, 0);
|
2018-04-06 16:17:26 +03:00
|
|
|
|
2017-11-02 18:27:52 +03:00
|
|
|
file->private_data = NULL;
|
2017-11-02 18:27:52 +03:00
|
|
|
if (af->wb)
|
|
|
|
afs_put_wb_key(af->wb);
|
2017-11-02 18:27:52 +03:00
|
|
|
key_put(af->key);
|
|
|
|
kfree(af);
|
2017-11-02 18:27:52 +03:00
|
|
|
afs_prune_wb_keys(vnode);
|
2019-05-15 14:09:17 +03:00
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2007-04-27 02:57:07 +04:00
|
|
|
}
|
|
|
|
|
2020-02-06 17:22:28 +03:00
|
|
|
/*
|
2020-02-06 17:22:29 +03:00
|
|
|
* Allocate a new read record.
|
2020-02-06 17:22:28 +03:00
|
|
|
*/
|
2020-02-06 17:22:29 +03:00
|
|
|
struct afs_read *afs_alloc_read(gfp_t gfp)
|
2020-02-06 17:22:28 +03:00
|
|
|
{
|
2020-02-06 17:22:29 +03:00
|
|
|
struct afs_read *req;
|
2020-02-06 17:22:28 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
req = kzalloc(sizeof(struct afs_read), gfp);
|
|
|
|
if (req)
|
|
|
|
refcount_set(&req->usage, 1);
|
2020-02-06 17:22:28 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
return req;
|
2020-02-06 17:22:28 +03:00
|
|
|
}
|
|
|
|
|
2017-01-05 13:38:34 +03:00
|
|
|
/*
|
|
|
|
* Dispose of a ref to a read record.
|
|
|
|
*/
|
|
|
|
void afs_put_read(struct afs_read *req)
|
|
|
|
{
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 16:17:25 +03:00
|
|
|
if (refcount_dec_and_test(&req->usage)) {
|
2020-02-06 17:22:28 +03:00
|
|
|
if (req->cleanup)
|
|
|
|
req->cleanup(req);
|
2020-02-06 17:22:27 +03:00
|
|
|
key_put(req->key);
|
2017-01-05 13:38:34 +03:00
|
|
|
kfree(req);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-09-18 11:11:15 +03:00
|
|
|
static void afs_fetch_data_notify(struct afs_operation *op)
|
|
|
|
{
|
|
|
|
struct afs_read *req = op->fetch.req;
|
2020-02-06 17:22:29 +03:00
|
|
|
struct netfs_read_subrequest *subreq = req->subreq;
|
2020-09-18 11:11:15 +03:00
|
|
|
int error = op->error;
|
|
|
|
|
|
|
|
if (error == -ECONNABORTED)
|
|
|
|
error = afs_abort_to_error(op->ac.abort_code);
|
|
|
|
req->error = error;
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
if (subreq) {
|
|
|
|
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
|
|
|
|
netfs_subreq_terminated(subreq, error ?: req->actual_len, false);
|
|
|
|
req->subreq = NULL;
|
|
|
|
} else if (req->done) {
|
2020-09-18 11:11:15 +03:00
|
|
|
req->done(req);
|
2020-02-06 17:22:29 +03:00
|
|
|
}
|
2020-09-18 11:11:15 +03:00
|
|
|
}
|
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
static void afs_fetch_data_success(struct afs_operation *op)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = op->file[0].vnode;
|
|
|
|
|
|
|
|
_enter("op=%08x", op->debug_id);
|
|
|
|
afs_vnode_commit_status(op, &op->file[0]);
|
|
|
|
afs_stat_v(vnode, n_fetches);
|
|
|
|
atomic_long_add(op->fetch.req->actual_len, &op->net->n_fetch_bytes);
|
2020-09-18 11:11:15 +03:00
|
|
|
afs_fetch_data_notify(op);
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
static void afs_fetch_data_put(struct afs_operation *op)
|
|
|
|
{
|
2020-02-06 17:22:28 +03:00
|
|
|
op->fetch.req->error = op->error;
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
afs_put_read(op->fetch.req);
|
|
|
|
}
|
|
|
|
|
|
|
|
static const struct afs_operation_ops afs_fetch_data_operation = {
|
|
|
|
.issue_afs_rpc = afs_fs_fetch_data,
|
|
|
|
.issue_yfs_rpc = yfs_fs_fetch_data,
|
|
|
|
.success = afs_fetch_data_success,
|
2020-06-16 02:34:09 +03:00
|
|
|
.aborted = afs_check_for_remote_deletion,
|
2020-09-18 11:11:15 +03:00
|
|
|
.failed = afs_fetch_data_notify,
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
.put = afs_fetch_data_put,
|
|
|
|
};
|
|
|
|
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
/*
|
|
|
|
* Fetch file data from the volume.
|
|
|
|
*/
|
2020-02-06 17:22:27 +03:00
|
|
|
int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
{
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
struct afs_operation *op;
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
|
2018-10-20 02:57:57 +03:00
|
|
|
_enter("%s{%llx:%llu.%u},%x,,,",
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
vnode->volume->name,
|
|
|
|
vnode->fid.vid,
|
|
|
|
vnode->fid.vnode,
|
|
|
|
vnode->fid.unique,
|
2020-02-06 17:22:27 +03:00
|
|
|
key_serial(req->key));
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
|
2020-02-06 17:22:27 +03:00
|
|
|
op = afs_alloc_operation(req->key, vnode->volume);
|
2020-02-06 17:22:29 +03:00
|
|
|
if (IS_ERR(op)) {
|
|
|
|
if (req->subreq)
|
|
|
|
netfs_subreq_terminated(req->subreq, PTR_ERR(op), false);
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
return PTR_ERR(op);
|
2020-02-06 17:22:29 +03:00
|
|
|
}
|
2019-05-09 17:16:10 +03:00
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
afs_op_set_vnode(op, 0, vnode);
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 22:51:51 +03:00
|
|
|
op->fetch.req = afs_get_read(req);
|
|
|
|
op->ops = &afs_fetch_data_operation;
|
|
|
|
return afs_do_sync_operation(op);
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 18:27:50 +03:00
|
|
|
}
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static void afs_req_issue_op(struct netfs_read_subrequest *subreq)
|
2005-04-17 02:20:36 +04:00
|
|
|
{
|
2020-02-06 17:22:29 +03:00
|
|
|
struct afs_vnode *vnode = AFS_FS_I(subreq->rreq->inode);
|
|
|
|
struct afs_read *fsreq;
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
fsreq = afs_alloc_read(GFP_NOFS);
|
|
|
|
if (!fsreq)
|
|
|
|
return netfs_subreq_terminated(subreq, -ENOMEM, false);
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
fsreq->subreq = subreq;
|
|
|
|
fsreq->pos = subreq->start + subreq->transferred;
|
|
|
|
fsreq->len = subreq->len - subreq->transferred;
|
|
|
|
fsreq->key = subreq->rreq->netfs_priv;
|
|
|
|
fsreq->vnode = vnode;
|
|
|
|
fsreq->iter = &fsreq->def_iter;
|
2020-02-10 13:00:22 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
iov_iter_xarray(&fsreq->def_iter, READ,
|
|
|
|
&fsreq->vnode->vfs_inode.i_mapping->i_pages,
|
|
|
|
fsreq->pos, fsreq->len);
|
2020-02-06 17:22:28 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
afs_fetch_data(fsreq->vnode, fsreq);
|
2007-04-27 02:49:28 +04:00
|
|
|
}
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static int afs_symlink_readpage(struct page *page)
|
2010-05-21 18:27:09 +04:00
|
|
|
{
|
2020-02-06 17:22:29 +03:00
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
|
|
|
struct afs_read *fsreq;
|
2010-05-21 18:27:09 +04:00
|
|
|
int ret;
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
fsreq = afs_alloc_read(GFP_NOFS);
|
|
|
|
if (!fsreq)
|
2017-01-05 13:38:35 +03:00
|
|
|
return -ENOMEM;
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
fsreq->pos = page->index * PAGE_SIZE;
|
|
|
|
fsreq->len = PAGE_SIZE;
|
|
|
|
fsreq->vnode = vnode;
|
|
|
|
fsreq->iter = &fsreq->def_iter;
|
|
|
|
iov_iter_xarray(&fsreq->def_iter, READ, &page->mapping->i_pages,
|
|
|
|
fsreq->pos, fsreq->len);
|
2020-02-06 17:22:28 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
ret = afs_fetch_data(fsreq->vnode, fsreq);
|
|
|
|
page_endio(page, false, ret);
|
|
|
|
return ret;
|
|
|
|
}
|
2017-01-05 13:38:35 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static void afs_init_rreq(struct netfs_read_request *rreq, struct file *file)
|
|
|
|
{
|
|
|
|
rreq->netfs_priv = key_get(afs_file_key(file));
|
|
|
|
}
|
2017-01-05 13:38:35 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static bool afs_is_cache_enabled(struct inode *inode)
|
|
|
|
{
|
|
|
|
struct fscache_cookie *cookie = afs_vnode_cache(AFS_FS_I(inode));
|
|
|
|
|
|
|
|
return fscache_cookie_enabled(cookie) && !hlist_empty(&cookie->backing_objects);
|
|
|
|
}
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static int afs_begin_cache_operation(struct netfs_read_request *rreq)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(rreq->inode);
|
2017-01-05 13:38:35 +03:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
return fscache_begin_read_operation(rreq, afs_vnode_cache(vnode));
|
2017-01-05 13:38:35 +03:00
|
|
|
}
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static int afs_check_write_begin(struct file *file, loff_t pos, unsigned len,
|
|
|
|
struct page *page, void **_fsdata)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
|
|
|
|
|
|
|
|
return test_bit(AFS_VNODE_DELETED, &vnode->flags) ? -ESTALE : 0;
|
|
|
|
}
|
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static void afs_priv_cleanup(struct address_space *mapping, void *netfs_priv)
|
2005-04-17 02:20:36 +04:00
|
|
|
{
|
2020-02-06 17:22:29 +03:00
|
|
|
key_put(netfs_priv);
|
|
|
|
}
|
2010-05-21 18:27:09 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
const struct netfs_read_request_ops afs_req_ops = {
|
2020-02-06 17:22:29 +03:00
|
|
|
.init_rreq = afs_init_rreq,
|
2020-02-06 17:22:29 +03:00
|
|
|
.is_cache_enabled = afs_is_cache_enabled,
|
2020-02-06 17:22:29 +03:00
|
|
|
.begin_cache_operation = afs_begin_cache_operation,
|
2020-02-06 17:22:29 +03:00
|
|
|
.check_write_begin = afs_check_write_begin,
|
2020-02-06 17:22:29 +03:00
|
|
|
.issue_op = afs_req_issue_op,
|
|
|
|
.cleanup = afs_priv_cleanup,
|
|
|
|
};
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static int afs_readpage(struct file *file, struct page *page)
|
|
|
|
{
|
|
|
|
if (!file)
|
|
|
|
return afs_symlink_readpage(page);
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
return netfs_readpage(file, page, &afs_req_ops, NULL);
|
|
|
|
}
|
2009-04-03 19:42:41 +04:00
|
|
|
|
2020-02-06 17:22:29 +03:00
|
|
|
static void afs_readahead(struct readahead_control *ractl)
|
|
|
|
{
|
|
|
|
netfs_readahead(ractl, &afs_req_ops, NULL);
|
2007-04-27 02:49:28 +04:00
|
|
|
}
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2020-10-22 16:08:23 +03:00
|
|
|
/*
|
|
|
|
* Adjust the dirty region of the page on truncation or full invalidation,
|
|
|
|
* getting rid of the markers altogether if the region is entirely invalidated.
|
|
|
|
*/
|
|
|
|
static void afs_invalidate_dirty(struct page *page, unsigned int offset,
|
|
|
|
unsigned int length)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
|
|
|
unsigned long priv;
|
|
|
|
unsigned int f, t, end = offset + length;
|
|
|
|
|
|
|
|
priv = page_private(page);
|
|
|
|
|
|
|
|
/* we clean up only if the entire page is being invalidated */
|
|
|
|
if (offset == 0 && length == thp_size(page))
|
|
|
|
goto full_invalidate;
|
|
|
|
|
|
|
|
/* If the page was dirtied by page_mkwrite(), the PTE stays writable
|
|
|
|
* and we don't get another notification to tell us to expand it
|
|
|
|
* again.
|
|
|
|
*/
|
|
|
|
if (afs_is_page_dirty_mmapped(priv))
|
|
|
|
return;
|
|
|
|
|
|
|
|
/* We may need to shorten the dirty region */
|
2020-10-28 17:23:46 +03:00
|
|
|
f = afs_page_dirty_from(page, priv);
|
|
|
|
t = afs_page_dirty_to(page, priv);
|
2020-10-22 16:08:23 +03:00
|
|
|
|
|
|
|
if (t <= offset || f >= end)
|
|
|
|
return; /* Doesn't overlap */
|
|
|
|
|
|
|
|
if (f < offset && t > end)
|
|
|
|
return; /* Splits the dirty region - just absorb it */
|
|
|
|
|
|
|
|
if (f >= offset && t <= end)
|
|
|
|
goto undirty;
|
|
|
|
|
|
|
|
if (f < offset)
|
|
|
|
t = offset;
|
|
|
|
else
|
|
|
|
f = end;
|
|
|
|
if (f == t)
|
|
|
|
goto undirty;
|
|
|
|
|
2020-10-28 17:23:46 +03:00
|
|
|
priv = afs_page_dirty(page, f, t);
|
2020-10-22 16:08:23 +03:00
|
|
|
set_page_private(page, priv);
|
2020-10-28 17:23:46 +03:00
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("trunc"), page);
|
2020-10-22 16:08:23 +03:00
|
|
|
return;
|
|
|
|
|
|
|
|
undirty:
|
2020-10-28 17:23:46 +03:00
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("undirty"), page);
|
2020-10-22 16:08:23 +03:00
|
|
|
clear_page_dirty_for_io(page);
|
|
|
|
full_invalidate:
|
2020-10-28 17:23:46 +03:00
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("inval"), page);
|
2020-10-20 11:33:45 +03:00
|
|
|
detach_page_private(page);
|
2020-10-22 16:08:23 +03:00
|
|
|
}
|
|
|
|
|
2005-04-17 02:20:36 +04:00
|
|
|
/*
|
2009-04-03 19:42:41 +04:00
|
|
|
* invalidate part or all of a page
|
|
|
|
* - release a page and clean up its private data if offset is 0 (indicating
|
|
|
|
* the entire page)
|
|
|
|
*/
|
2013-05-22 07:17:23 +04:00
|
|
|
static void afs_invalidatepage(struct page *page, unsigned int offset,
|
|
|
|
unsigned int length)
|
2009-04-03 19:42:41 +04:00
|
|
|
{
|
2013-05-22 07:17:23 +04:00
|
|
|
_enter("{%lu},%u,%u", page->index, offset, length);
|
2009-04-03 19:42:41 +04:00
|
|
|
|
|
|
|
BUG_ON(!PageLocked(page));
|
|
|
|
|
2020-10-22 16:08:23 +03:00
|
|
|
if (PagePrivate(page))
|
|
|
|
afs_invalidate_dirty(page, offset, length);
|
2009-04-03 19:42:41 +04:00
|
|
|
|
2020-02-06 17:22:28 +03:00
|
|
|
wait_on_page_fscache(page);
|
2009-04-03 19:42:41 +04:00
|
|
|
_leave("");
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* release a page and clean up its private state if it's not busy
|
|
|
|
* - return true if the page can now be released, false if not
|
2005-04-17 02:20:36 +04:00
|
|
|
*/
|
2007-05-09 13:33:45 +04:00
|
|
|
static int afs_releasepage(struct page *page, gfp_t gfp_flags)
|
2005-04-17 02:20:36 +04:00
|
|
|
{
|
2007-05-09 13:33:45 +04:00
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2018-10-20 02:57:57 +03:00
|
|
|
_enter("{{%llx:%llu}[%lu],%lx},%x",
|
2007-05-09 13:33:45 +04:00
|
|
|
vnode->fid.vid, vnode->fid.vnode, page->index, page->flags,
|
|
|
|
gfp_flags);
|
2005-04-17 02:20:36 +04:00
|
|
|
|
2009-04-03 19:42:41 +04:00
|
|
|
/* deny if page is being written to the cache and the caller hasn't
|
|
|
|
* elected to wait */
|
2020-02-06 17:22:28 +03:00
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
if (PageFsCache(page)) {
|
|
|
|
if (!(gfp_flags & __GFP_DIRECT_RECLAIM) || !(gfp_flags & __GFP_FS))
|
|
|
|
return false;
|
|
|
|
wait_on_page_fscache(page);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2005-04-17 02:20:36 +04:00
|
|
|
if (PagePrivate(page)) {
|
2020-10-28 17:23:46 +03:00
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("rel"), page);
|
2020-10-20 11:33:45 +03:00
|
|
|
detach_page_private(page);
|
2005-04-17 02:20:36 +04:00
|
|
|
}
|
|
|
|
|
2009-04-03 19:42:41 +04:00
|
|
|
/* indicate that the page can be released */
|
|
|
|
_leave(" = T");
|
|
|
|
return 1;
|
2007-04-27 02:49:28 +04:00
|
|
|
}
|
2017-11-02 18:27:52 +03:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Handle setting up a memory mapping on an AFS file.
|
|
|
|
*/
|
|
|
|
static int afs_file_mmap(struct file *file, struct vm_area_struct *vma)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
ret = generic_file_mmap(file, vma);
|
|
|
|
if (ret == 0)
|
|
|
|
vma->vm_ops = &afs_vm_ops;
|
|
|
|
return ret;
|
|
|
|
}
|