vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
/*
|
|
|
|
|
* Licensed under a two-clause BSD-style license.
|
|
|
|
|
* See LICENSE for details.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include "git-compat-util.h"
|
|
|
|
|
#include "sliding_window.h"
|
|
|
|
|
#include "line_buffer.h"
|
|
|
|
|
#include "strbuf.h"
|
|
|
|
|
|
|
|
|
|
static int input_error(struct line_buffer *file)
|
|
|
|
|
{
|
|
|
|
|
if (!buffer_ferror(file))
|
|
|
|
|
return error("delta preimage ends early");
|
|
|
|
|
return error("cannot read delta preimage: %s", strerror(errno));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static int skip_or_whine(struct line_buffer *file, off_t gap)
|
|
|
|
|
{
|
|
|
|
|
if (buffer_skip_bytes(file, gap) != gap)
|
|
|
|
|
return input_error(file);
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static int read_to_fill_or_whine(struct line_buffer *file,
|
|
|
|
|
struct strbuf *buf, size_t width)
|
|
|
|
|
{
|
|
|
|
|
buffer_read_binary(file, buf, width - buf->len);
|
|
|
|
|
if (buf->len != width)
|
|
|
|
|
return input_error(file);
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
2012-02-02 14:28:02 +04:00
|
|
|
static int check_offset_overflow(off_t offset, uintmax_t len)
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
{
|
vcs-svn: rename check_overflow and its arguments for clarity
The canonical interpretation of a range a,b is as an interval [a,b),
not [a,a+b), so this function taking argument names a and b feels
unnatural. Use more explicit names "offset" and "len" to make the
arguments' type and function clearer.
While at it, rename the function to convey that we are making sure
the sum of this offset and length do not overflow an off_t, not a
size_t.
[jn: split out from a patch from Ramsay Jones, then improved with
advice from Thomas Rast, Dmitry Ivankov, and David Barr]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 13:57:23 +04:00
|
|
|
if (len > maximum_signed_value_of_type(off_t))
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
return error("unrepresentable length in delta: "
|
2012-02-02 14:28:02 +04:00
|
|
|
"%"PRIuMAX" > OFF_MAX", len);
|
vcs-svn: rename check_overflow and its arguments for clarity
The canonical interpretation of a range a,b is as an interval [a,b),
not [a,a+b), so this function taking argument names a and b feels
unnatural. Use more explicit names "offset" and "len" to make the
arguments' type and function clearer.
While at it, rename the function to convey that we are making sure
the sum of this offset and length do not overflow an off_t, not a
size_t.
[jn: split out from a patch from Ramsay Jones, then improved with
advice from Thomas Rast, Dmitry Ivankov, and David Barr]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 13:57:23 +04:00
|
|
|
if (signed_add_overflows(offset, (off_t) len))
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
return error("unrepresentable offset in delta: "
|
|
|
|
|
"%"PRIuMAX" + %"PRIuMAX" > OFF_MAX",
|
2012-02-02 14:28:02 +04:00
|
|
|
(uintmax_t) offset, len);
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
int move_window(struct sliding_view *view, off_t off, size_t width)
|
|
|
|
|
{
|
|
|
|
|
off_t file_offset;
|
|
|
|
|
assert(view);
|
|
|
|
|
assert(view->width <= view->buf.len);
|
vcs-svn: rename check_overflow and its arguments for clarity
The canonical interpretation of a range a,b is as an interval [a,b),
not [a,a+b), so this function taking argument names a and b feels
unnatural. Use more explicit names "offset" and "len" to make the
arguments' type and function clearer.
While at it, rename the function to convey that we are making sure
the sum of this offset and length do not overflow an off_t, not a
size_t.
[jn: split out from a patch from Ramsay Jones, then improved with
advice from Thomas Rast, Dmitry Ivankov, and David Barr]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 13:57:23 +04:00
|
|
|
assert(!check_offset_overflow(view->off, view->buf.len));
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
|
vcs-svn: rename check_overflow and its arguments for clarity
The canonical interpretation of a range a,b is as an interval [a,b),
not [a,a+b), so this function taking argument names a and b feels
unnatural. Use more explicit names "offset" and "len" to make the
arguments' type and function clearer.
While at it, rename the function to convey that we are making sure
the sum of this offset and length do not overflow an off_t, not a
size_t.
[jn: split out from a patch from Ramsay Jones, then improved with
advice from Thomas Rast, Dmitry Ivankov, and David Barr]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 13:57:23 +04:00
|
|
|
if (check_offset_overflow(off, width))
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
return -1;
|
|
|
|
|
if (off < view->off || off + width < view->off + view->width)
|
|
|
|
|
return error("invalid delta: window slides left");
|
2012-05-31 18:41:29 +04:00
|
|
|
if (view->max_off >= 0 && view->max_off < off + (off_t) width)
|
2011-05-27 13:07:44 +04:00
|
|
|
return error("delta preimage ends early");
|
vcs-svn: learn to maintain a sliding view of a file
Each section of a Subversion-format delta only requires examining (and
keeping in random-access memory) a small portion of the preimage. At
any moment, this portion starts at a certain file offset and has a
well-defined length, and as the delta is applied, the portion advances
from the beginning to the end of the preimage. Add a move_window
function to keep track of this view into the preimage.
You can use it like this:
buffer_init(f, NULL);
struct sliding_view window = SLIDING_VIEW_INIT(f);
move_window(&window, 3, 7); /* (1) */
move_window(&window, 5, 5); /* (2) */
move_window(&window, 12, 2); /* (3) */
strbuf_release(&window.buf);
buffer_deinit(f);
The data structure is called sliding_view instead of _window to
prevent confusion with svndiff0 Windows.
In this example, (1) reads 10 bytes and discards the first 3;
(2) discards the first 2, which are not needed any more; and (3) skips
2 bytes and reads 2 new bytes to work with.
When move_window returns, the file position indicator is at position
window->off + window->width and the data from positions window->off to
the current file position are stored in window->buf.
This function performs only sequential access from the input file and
never seeks, so it can be safely used on pipes and sockets.
On end-of-file, move_window silently reads less than the caller
requested. On other errors, it prints a message and returns -1.
Helped-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 06:54:58 +03:00
|
|
|
|
|
|
|
|
file_offset = view->off + view->buf.len;
|
|
|
|
|
if (off < file_offset) {
|
|
|
|
|
/* Move the overlapping region into place. */
|
|
|
|
|
strbuf_remove(&view->buf, 0, off - view->off);
|
|
|
|
|
} else {
|
|
|
|
|
/* Seek ahead to skip the gap. */
|
|
|
|
|
if (skip_or_whine(view->file, off - file_offset))
|
|
|
|
|
return -1;
|
|
|
|
|
strbuf_setlen(&view->buf, 0);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (view->buf.len > width)
|
|
|
|
|
; /* Already read. */
|
|
|
|
|
else if (read_to_fill_or_whine(view->file, &view->buf, width))
|
|
|
|
|
return -1;
|
|
|
|
|
|
|
|
|
|
view->off = off;
|
|
|
|
|
view->width = width;
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
