This was mildly fiddly because there's a single vtable structure that
implements two distinct interface types, one for compression and one
for decompression - and I have actually confused them before now
(commit d4304f1b7), so I think it's important to make them actually be
separate types!
Most of these were 'void *' because they weren't even reliably a
structure type underneath - the per-OS storage systems would directly
cast read/write/enum settings handles to and from random things like
FILE *, Unix DIR *, or Windows HKEY. So I've wrapped them in tiny
structs for the sake of having a sensible structure tag visible
elsewhere in the code.
'struct draw_ctx' has a structure tag inside gtkwin.c, so as per this
week's standard practice, let's expose the tag elsewhere so that
pointers declared that way can't be confused with anything else.
hmacmd5_do_hmac and hmac_sha1_simple should be consistently referring
to input memory blocks as 'const void *', but one had pointlessly
typed the pointer as 'const unsigned char *' and the other had missed
out the consts.
The new version of ssh_hash has the same nice property as ssh2_mac,
that I can make the generic interface object type function directly as
a BinarySink so that clients don't have to call h->sink() and worry
about the separate sink object they get back from that.
This piece of tidying-up has come out particularly well in terms of
saving tedious repetition and boilerplate. I've managed to remove
three pointless methods from every MAC implementation by means of
writing them once centrally in terms of the implementation-specific
methods; another method (hmacmd5_sink) vanished because I was able to
make the interface type 'ssh2_mac' be directly usable as a BinarySink
by way of a new delegation system; and because all the method
implementations can now find their own vtable, I was even able to
merge a lot of keying and output functions that had previously
differed only in length parameters by having them look up the lengths
in whatever vtable they were passed.
This is more or less the same job as the SSH-1 case, only more
extensive, because we have a wider range of ciphers.
I'm a bit disappointed about the AES case, in particular, because I
feel as if it ought to have been possible to arrange to combine this
layer of vtable dispatch with the subsidiary one that selects between
hardware and software implementations of the underlying cipher. I may
come back later and have another try at that, in fact.
The interchangeable system of SSH-1 ciphers previously followed the
same pattern as the backends and the public-key algorithms, in that
all the clients would maintain two separate pointers, one to the
vtable and the other to the individual instance / context. Now I've
merged them, just as I did with those other two, so that you only cart
around a single pointer, which has a vtable pointer inside it and a
type distinguishing it from an instance of any of the other
interchangeable sets of algorithms.
ATOFFSET in dialog.h became obsolete when the old 'struct Config' gave
way to the new Conf, because its only use was to identify fields in
struct Config for the generic control handlers to update.
And lenof in ssh.h is redundant because there's also a copy in misc.h.
(Which is already included _everywhere_ that lenof is used - I didn't
even need to add any instances of #include "misc.h" after removing the
copy in ssh.h.)
This was a particularly confusing piece of type-danger, because three
different types were passed outside sshshare.c as 'void *' and only
human vigilance prevented one coming back as the wrong one. Now they
all keep their opaque structure tags when they move through other
parts of the code.
There's now an interface called 'Channel', which handles the local
side of an SSH connection-layer channel, in terms of knowing where to
send incoming channel data to, whether to close the channel, etc.
Channel and the previous 'struct ssh_channel' mutually refer. The
latter contains all the SSH-specific parts, and as much of the common
logic as possible: in particular, Channel doesn't have to know
anything about SSH packet formats, or which SSH protocol version is in
use, or deal with all the fiddly stuff about window sizes - with the
exception that x11fwd.c's implementation of it does have to be able to
ask for a small fixed initial window size for the bodgy system that
distinguishes upstream from downstream X forwardings.
I've taken the opportunity to move the code implementing the detailed
behaviour of agent forwarding out of ssh.c, now that all of it is on
the far side of a uniform interface. (This also means that if I later
implement agent forwarding directly to a Unix socket as an
alternative, it'll be a matter of changing just the one call to
agentf_new() that makes the Channel to plug into a forwarding.)
This is another major source of unexplained 'void *' parameters
throughout the code.
In particular, the currently unused testback.c actually gave the wrong
pointer type to its internal store of the frontend handle - it cast
the input void * to a Terminal *, from which it got implicitly cast
back again when calling from_backend, and nobody noticed. Now it uses
the right type internally as well as externally.
Nearly every part of the code that ever handles a full backend
structure has historically done it using a pair of pointer variables,
one pointing at a constant struct full of function pointers, and the
other pointing to a 'void *' state object that's passed to each of
those.
While I'm modernising the rest of the code, this seems like a good
time to turn that into the same more or less type-safe and less
cumbersome system as I'm using for other parts of the code, such as
Socket, Plug, BinaryPacketProtocol and so forth: the Backend structure
contains a vtable pointer, and a system of macro wrappers handles
dispatching through that vtable.
Same principle again - the more of these structures have globally
visible tags (even if the structure contents are still opaque in most
places), the fewer of them I can mistake for each other.
That's one fewer anonymous 'void *' which might be accidentally
confused with some other pointer type if I misremember the order of
function arguments.
While I'm here, I've made its pointer-nature explicit - that is,
'Ldisc' is now a typedef for the structure type itself rather than a
pointer to it. A stylistic change only, but it feels more natural to
me these days for a thing you're going to eventually pass to a 'free'
function.
This source file isn't actually built as part of even a test binary,
so it hasn't been kept up to date with internal API changes. But it
might still come in useful in the future (I think its original purpose
was to substitute for a normal backend in order to test the GUI side
of a new PuTTY port before the network side was written), so I'll at
least try to carry on keeping it updated.
This commit adds the new ids and fingerprints in the keys appendix of
the manual, and moves the old ones down into the historic-keys
section. I've tweaked a few pieces of wording for ongoing use, so that
they don't imply a specific number of past key rollovers.
The -pgpfp option in all the tools now shows the new Master Key
fingerprint and the previous (2015) one. I've adjusted all the uses of
the #defines in putty.h so that future rollovers should only have to
modify the #defines themselves.
Most importantly, sign.sh bakes in the ids of the current release and
snapshot keys, so that snapshots will automatically be signed with the
new snapshot key and the -r option will invoke the new release key.
I had previously left the platform field (in line 7 of the installer
database's SummaryInformation table) set at "x86" instead of any value
you might expect such as "Arm" or "Arm64", because I found that an MSI
file with either of the latter values was rejected by WoA's msiexec as
invalid.
It turns out this is because I _also_ needed to upgrade the installer
database schema version to a higher value than I even knew existed:
apparently the problem is that those platform fields aren't present in
the older schema. A test confirms that this works.
Unfortunately, WiX 3 doesn't actually know _how_ to write MSIs with
those platform values. But that's OK, because diffing the x86 and x64
MSIs against each other suggested that there were basically no other
changes in the database tables - so I can just generate the installer
as if for x64, and then rewrite that one field after installer
construction using GNOME msitools to take apart the binary file
structure and put it back together. (Those are the same tools I'm
using as part of my system for running WiX on Linux in the first
place.)
This commit introduces a script to do that post-hoc bodging, and calls
it from Buildscr. I've also changed over the choice of Program Files
folder for the Arm installers so that it's ProgramFiles64Folder
instead of ProgramFilesFolder - so now the Windows on Arm installer
doesn't incongruously default to installing in C:\Program Files (x86)!
Now when we construct a packet containing sensitive data, we just set
a field saying '... and make it take up at least this much space, to
disguise its true size', and nothing in the rest of the system worries
about that flag until ssh2bpp.c acts on it.
Also, I've changed the strategy for doing the padding. Previously, we
were following the real packet with an SSH_MSG_IGNORE to make up the
size. But that was only a partial defence: it works OK against passive
traffic analysis, but an attacker proxying the TCP stream and
dribbling it out one byte at a time could still have found out the
size of the real packet by noting when the dribbled data provoked a
response. Now I put the SSH_MSG_IGNORE _first_, which should defeat
that attack.
But that in turn doesn't work when we're doing compression, because we
can't predict the compressed sizes accurately enough to make that
strategy sensible. Fortunately, compression provides an alternative
strategy anyway: if we've got zlib turned on when we send one of these
sensitive packets, then we can pad out the compressed zlib data as
much as we like by adding empty RFC1951 blocks (effectively chaining
ZLIB_PARTIAL_FLUSHes). So both strategies should now be dribble-proof.
The return value wasn't used to indicate failure; it only indicated
whether any compression was being done at all or whether the
compression method was ssh_comp_none, and we can tell the latter just
as well by the fact that its init function returns a null context
pointer.
Yesterday's reinstatement of ssh_free_pktout revealed - via valgrind
spotting the use-after-free - that the code that prefixed sensible
packets with IV-muddling SSH_MSG_IGNOREs was actually sending a second
copy of the sensible packet in place of the IGNORE, due to a typo.
I think ever since commit 679fa90df last month, PuTTY has been
forgetting to free any of its outgoing packet structures after turning
them into their encrypted wire format. And apparently no users of the
development snapshots have noticed - including me!
I just spotted it while I was looking through this module anyway. It
was using %.100s to prevent an sprintf buffer overflow, which hasn't
been necessary since I switched everything over to dupprintf, and also
it was printing the integer value of GetLastError() without using my
convenient translation wrapper win_strerror.
That's the same value as in the OpenSSH source code, so it should be
large enough that anyone needing to sign a larger message will have
other problems too.
I heard recently that at least one third-party client of Pageant
exists, and that it's used to generate signatures to use with TLS
client certificates. Apparently the signature scheme is compatible,
but TLS tends to need signatures over more data than will fit in
AGENT_MAX_MSGLEN.
Before the BinarySink refactor in commit b6cbad89f, this was OK
because the Windows Pageant IPC didn't check the size of the _input_
message against AGENT_MAX_MSGLEN, only the output one. But then we
started checking both, so that third-party TLS client started failing.
Now we use VirtualQuery to find out the actual size of the file
mapping we've been passed, and our only requirement is that the input
and output messages should both fit in _that_. So TLS should work
again, and also, other clients should be able to retrieve longer lists
of public keys if they pass a larger file mapping.
One side effect of this change is that Pageant's reply message is now
written directly into the shared-memory region. Previously, it was
written into a separate buffer and then memcpy()ed over after
pageant_handle_msg returned, but now the buffer is variable-size, it
seems to me to make more sense to avoid that extra not-entirely
controlled malloc. So I've done one very small reordering of
statements in the cross-platform pageant_handle_msg(), which fixes the
only case I could find where that function started writing its output
before it had finished using the contents of the input buffer.
Previously, the code to recover and memory-map the file-mapping object
Pageant uses for its IPC, and the code to convey its contents to and
from the cross-platform agent code, were widely separated, with the
former living in the WM_COPYDATA handler in the window procedure, and
the latter in answer_msg.
Now all of that code lives in answer_filemapping_message; WndProc only
handles the _window message_ contents - i.e. ensures the WM_COPYDATA
message has the right dwData id and that its lpData contains an ASCIZ
string - and answer_filemapping_message goes all the way from that
file-mapping object name to calling pageant_handle_msg.
While I'm here, I've also tidied up the code so that it uses the 'goto
cleanup' idiom rather than nesting everything inconveniently deeply,
and arranged that if anything goes wrong then we at least _construct_
an error message (although as yet we don't use that for anything
unless we're compiled with DEBUG_IPC enabled).
On the post-userauth rekey, when we're specifically rekeying in order
to turn on delayed compression, we shouldn't write the Event Log
"Server supports delayed compression; will try this later" message
that we did in the original key exchange. At this point, it _is_
later, and we're about to turn on compression right now!
The new SSH-2 BPP has two functions ssh2_bpp_new_outgoing_crypto and
ssh2_bpp_new_incoming_crypto, which (due to general symmetry) are
_almost_ identical, except that the code that sets up the compression
context in the two directions has to call compress_init in the former
and decompress_init in the latter.
Except that it called compress_init in both, so compression in SSH-2
has been completely broken for a week. How embarrassing. I _remember_
thinking that I'd better make sure to change that one call between the
two, but apparently it fell out of my head before I committed.
When the whole SSH connection is throttled and then unthrottled, we
need to requeue the callback that transfers data to the Socket from
the new outgoing_data queue introduced in commit 9e3522a97.
The user-visible effect of this missing call was that outgoing SFTP
transactions would lock up, because in SFTP mode we enable the
"simple@putty.projects.tartarus.org" mode and essentially turn off the
per-channel window management, so throttling of the whole connection
becomes the main source of back-pressure.
Thanks to Alex Landau for pointing out that commit 8b98fea4a
introduced two uses of it with the arguments one way round and one
with them the other way round. (Plus a fourth use where it doesn't
matter, because the padding at the end of the encrypted blob of an
OpenSSH PEM private key consists of n bytes with value n. :-)
On the basis of majority vote, I've switched the order in the function
definition to match the two of the three call sites that expressed the
same opinion, and fixed the third.
Now we have the new marshalling system, I think it's outlived its
usefulness, because the new system allows us to directly express
various things (e.g. uint16 and non-zero-terminated strings) that were
actually _more_ awkward to do via the variadic interface. So here's a
rewrite that removes send_packet(), and replaces all its call sites
with something that matches our SSH-2 packet construction idioms.
This diff actually _reduces_ the number of lines of code in ssh.c.
Since the variadic system was trying to save code by centralising
things, that seems like the best possible evidence that it wasn't
pulling its weight!
sshbpp.h now defines a classoid that encapsulates both directions of
an SSH binary packet protocol - that is, a system for reading a
bufchain of incoming data and turning it into a stream of PktIn, and
another system for taking a PktOut and turning it into data on an
outgoing bufchain.
The state structure in each of those files contains everything that
used to be in the 'rdpkt2_state' structure and its friends, and also
quite a lot of bits and pieces like cipher and MAC states that used to
live in the main Ssh structure.
One minor effect of this layer separation is that I've had to extend
the packet dispatch table by one, because the BPP layer can no longer
directly trigger sending of SSH_MSG_UNIMPLEMENTED for a message too
short to have a type byte. Instead, I extend the PktIn type field to
use an out-of-range value to encode that, and the easiest way to make
that trigger an UNIMPLEMENTED message is to have the dispatch table
contain an entry for it.
(That's a system that may come in useful again - I was also wondering
about inventing a fake type code to indicate network EOF, so that that
could be propagated through the layers and be handled by whichever one
currently knew best how to respond.)
I've also moved the packet-censoring code into its own pair of files,
partly because I was going to want to do that anyway sooner or later,
and mostly because it's called from the BPP code, and the SSH-2
version in particular has to be called from both the main SSH-2 BPP
and the bare unencrypted protocol used for connection sharing. While I
was at it, I took the opportunity to merge the outgoing and incoming
censor functions, so that the parts that were common between them
(e.g. CHANNEL_DATA messages look the same in both directions) didn't
need to be repeated.
This mirrors the use of one for incoming wire data: now when we send
raw data (be it the initial greeting, or the output of binary packet
construction), we put it on ssh->outgoing_data, and schedule a
callback to transfer that into the socket.
Partly this is in preparation for delegating the task of appending to
that bufchain to a separate self-contained module that won't have
direct access to the connection's Socket. But also, it has the very
nice feature that I get to throw away the ssh_pkt_defer system
completely! That was there so that we could construct more than one
packet in rapid succession, concatenate them into a single blob, and
pass that blob to the socket in one go so that the TCP headers
couldn't contain any trace of where the boundary between them was. But
now we don't need a separate function to do that: calling the ordinary
packet-send routine twice in the same function before returning to the
main event loop will have that effect _anyway_.
ssh.c has been an unmanageably huge monolith of a source file for too
long, and it's finally time I started breaking it up into smaller
pieces. The first step is to move some declarations - basic types like
packets and packet queues, standard constants, enums, and the
coroutine system - into headers where other files can see them.
It is to put_data what memset is to memcpy. Several places
in the code wanted it already, but not _quite_ enough for me to
have written it with the rest of the BinarySink infrastructure
originally.
This saves a malloc and free every time we add or remove a packet from
a packet queue - it can now be done by pure pointer-shuffling instead
of allocating a separate list node structure.
The body pointer was used after encryption to mark the start of the
fully wire-ready packet by ssh2_pkt_construct, and before encryption
by the log_outgoing_packet functions. Now the former returns a nice
modern ptrlen (it never really needed to store the pointer _in_ the
packet structure anyway), and the latter uses an integer 'prefix'
field, which isn't very different in concept but saves effort on
reallocs.
These were only used in the rdpkt coroutines, during construction of
the incoming packet; once it's complete, they're never touched again.
So really they should have been fields in the rdpkt coroutines' state
- and now they are.
The new memory allocation strategy for incoming packets is to defer
creation of the returned pktin structure until we know how big its
data buffer will really need to be, and then use snew_plus to make the
PktIn and the payload block in the same allocation.
When we have to read and keep some amount of the packet before
allocating the returned structure, we do it by having a persistent
buffer in the rdpkt state, which is retained for the whole connection
and only freed once in ssh_free.
They were duplicating values stored in the BinarySource substructure.
Mostly they're not referred to directly any more (instead, we call
get_foo to access the BinarySource); and when they are, we can switch
to reading the same values back out of the BinarySource anyway.
This is the first stage of massively tidying up this very confused
data structure. In this commit, I replace the unified 'struct Packet'
with two structures PktIn and PktOut, each of which contains only the
fields of struct Packet that are actually used for packets going in
that direction - most notably, PktIn doesn't implement BinarySink, and
PktOut doesn't implement BinarySource.
All uses of the old structure were statically determinable to be one
or the other, so I've done that determination and changed all the
types of variables and function signatures.
Unlike PktIn, PktOut is not reference-counted, so there's a new
ssh_pktout_free function.
The most immediately pleasing thing about this change is that it lets
me finally get rid of the tedious comment explaining how the 'length'
field in struct Packet meant something different depending on
direction. Now it's two fields of the same name in two different
structures, I can comment the same thing much less verbosely!
(I've also got rid of the comment claiming the type field was only
used for incoming packets. That wasn't even true! It might have been
once, because you can write an outgoing packet's type byte straight
into its data buffer, but in fact in the current code pktout->type is
nonetheless used in various other places, e.g. log_outgoing_packet.)
In this commit I've only removed the fields from each structure that
were _already_ unused. There are still quite a few we can get rid of
by _making_ them unused.
It was horrible - even if harmless in practice - that it wrote the
NATed channel id over its input buffer, and I think it's worth the
extra memory management to avoid doing that.
This formalises my occasional habit of using a single malloc to make a
block that contains a header structure and a data buffer that a field
of the structure will point to, allowing it to be freed in one go
later. Previously I had to do this by hand, losing the type-checking
advantages of snew; now I've written an snew-style macro to do the
job, plus an accessor macro to cleanly get the auxiliary buffer
pointer afterwards, and switched existing instances of the pattern
over to using that.
Simon Tatham