This should have been done months ago in commit bf0cf984c, but I've
been indecisive about whether to keep my local dev builds in the
windows subdirectory itself or one level further down...
I've written a new standalone test program which incorporates all of
PuTTY's crypto code, including the mp_int and low-level elliptic curve
layers but also going all the way up to the implementations of the
MAC, hash, cipher, public key and kex abstractions.
The test program itself, 'testcrypt', speaks a simple line-oriented
protocol on standard I/O in which you write the name of a function
call followed by some inputs, and it gives you back a list of outputs
preceded by a line telling you how many there are. Dynamically
allocated objects are assigned string ids in the protocol, and there's
a 'free' function that tells testcrypt when it can dispose of one.
It's possible to speak that protocol by hand, but cumbersome. I've
also provided a Python module that wraps it, by running testcrypt as a
persistent subprocess and gatewaying all the function calls into
things that look reasonably natural to call from Python. The Python
module and testcrypt.c both read a carefully formatted header file
testcrypt.h which contains the name and signature of every exported
function, so it costs minimal effort to expose a given function
through this test API. In a few cases it's necessary to write a
wrapper in testcrypt.c that makes the function look more friendly, but
mostly you don't even need that. (Though that is one of the
motivations between a lot of API cleanups I've done recently!)
I considered doing Python integration in the more obvious way, by
linking parts of the PuTTY code directly into a native-code .so Python
module. I decided against it because this way is more flexible: I can
run the testcrypt program on its own, or compile it in a way that
Python wouldn't play nicely with (I bet compiling just that .so with
Leak Sanitiser wouldn't do what you wanted when Python loaded it!), or
attach a debugger to it. I can even recompile testcrypt for a
different CPU architecture (32- vs 64-bit, or even running it on a
different machine over ssh or under emulation) and still layer the
nice API on top of that via the local Python interpreter. All I need
is a bidirectional data channel.
Now, instead of getting the zlib test/helper program by manually
compiling a source file with unusual options, it gets built as
standard by the ordinary Makefile.
This server is NOT SECURE! If anyone is reading this commit message,
DO NOT DEPLOY IT IN A HOSTILE-FACING ENVIRONMENT! Its purpose is to
speak the server end of everything PuTTY speaks on the client side, so
that I can test that I haven't broken PuTTY when I reorganise its
code, even things like RSA key exchange or chained auth methods which
it's hard to find a server that speaks at all.
(For this reason, it's declared with [UT] in the Recipe file, so that
it falls into the same category as programs like testbn, which won't
be installed by 'make install'.)
Working title is 'Uppity', partly for 'Universal PuTTY Protocol
Interaction Test Yoke', but mostly because it looks quite like the
word 'PuTTY' with part of it reversed. (Apparently 'test yoke' is a
very rarely used term meaning something not altogether unlike 'test
harness', which is a bit of a stretch, but it'll do.)
It doesn't actually _support_ everything I want yet. At the moment,
it's a proof of concept only. But it has most of the machinery
present, and the parts it's missing - such as chained auth methods -
should be easy enough to add because I've built in the required
flexibility, in the form of an AuthPolicy object which can request
them if it wants to. However, the current AuthPolicy object is
entirely trivial, and will let in any user with the password "weasel".
(Another way in which this is not a production-ready server is that it
also has no interaction with the OS's authentication system. In
particular, it will not only let in any user with the same password,
but it won't even change uid - it will open shells and forwardings
under whatever user id you started it up as.)
Currently, the program can only speak the SSH protocol on its standard
I/O channels (using the new FdSocket facility), so if you want it to
listen on a network port, you'll have to run it from some kind of
separate listening program similar to inetd. For my own tests, I'm not
even doing that: I'm just having PuTTY spawn it as a local proxy
process, which also conveniently eliminates the risk of anyone hostile
connecting to it.
The bulk of the actual code reorganisation is already done by previous
commits, so this change is _mostly_ just dropping in a new set of
server-specific source files alongside the client-specific ones I
created recently. The remaining changes in the shared SSH code are
numerous, but all minor:
- a few extra parameters to BPP and PPL constructors (e.g. 'are you
in server mode?'), and pass both sets of SSH-1 protocol flags from
the login to the connection layer
- in server mode, unconditionally send our version string _before_
waiting for the remote one
- a new hook in the SSH-1 BPP to handle enabling compression in
server mode, where the message exchange works the other way round
- new code in the SSH-2 BPP to do _deferred_ compression the other
way round (the non-deferred version is still nicely symmetric)
- in the SSH-2 transport layer, some adjustments to do key derivation
either way round (swapping round the identifying letters in the
various hash preimages, and making sure to list the KEXINITs in the
right order)
- also in the SSH-2 transport layer, an if statement that controls
whether we send SERVICE_REQUEST and wait for SERVICE_ACCEPT, or
vice versa
- new ConnectionLayer methods for opening outgoing channels for X and
agent forwardings
- new functions in portfwd.c to establish listening sockets suitable
for remote-to-local port forwarding (i.e. not under the direction
of a Conf the way it's done on the client side).
After a conversation this week with a user who tried to use it, it's
clear that Borland C can't build the up-to-date PuTTY without having
to make too many compromises of functionality (unsupported API
details, no 'long long' type), even above the issues that could be
worked round with extra porting ifdefs.
This was very strange to write, because it's a bizarre combination of
the GNU-make-isms and rc commands of Makefile.mgw with the cl and link
commands of Makefile.vc (but also the latter thankfully doesn't need
those horrible response files).
I've added a big comment in mkfiles.pl about what the build
requirements for this makefile actually are, which _hopefully_ will be
usable by people other than me.
It's really only useful with MinGW rather than a Cygwin toolchain these
days, as recent versions of the latter insist against linking with the
Cygwin DLL.
(I think it may no longer be possible to build with Cygwin out of the
box at all these days, but I'm not going to say so without having
actually checked that's the case. Settle for listing MinGW first in
various comments and docs.)
cmdgen.c has contained code for ages to build a test main() if you
compile with -DTEST_CMDGEN. But it's painful to do so manually, since
you've still got to link in all the same supporting objects, and also
nobody can have actually done that for a while because the stub test
code hasn't been kept up to date with changes in the internal APIs
(specifically prompt_t).
Now we have the ability to include our test programs in Recipe as [UT]
or [XT] so as to leave them out of 'make install', that seems like a
useful thing to do with cmdgen's test suite. So here's a Recipe change
that builds it as 'cgtest', plus fixes for compiler warnings and bit
rot. Pleasantly, the test suite still _passes_ after those are fixed.
The current state of the OS X GTK port is looking more or less
plausible - it's not finished, of course, but then neither was the old
native Cocoa port. So I'm inclined to advertise it as *the* unfinished
OS X port: it's the one I intend to keep working on, and it's the one
I'd prefer people offered us help with if they're going to offer.
Hence, leaving the old macosx directory around is just confusing; that
directory is long-unmaintained, probably doesn't even compile, and its
only effect will be to mislead people into thinking it's still
relevant. I'm unilaterally deleting it; of course we can always
recover it from source control history if it's ever necessary to do
so.
This is a file that generally seems to turn up when you start using OS
X Finder to interact with directories - which is more likely now that
we're building OS X app bundles into this source tree.
This commit adds two .plist files, which go in the app bundles; two
.bundle files, which are input to gtk-mac-bundler and explain to it
how to _create_ the bundles; and a piece of manual addition to
Makefile.am that actually runs gtk-mac-bundler after building the
gtkapp.c based binaries and the OSX launcher. The latter is
conditionalised on configuring --with-quartz (unlike the binaries
themselves, which you can build on other platforms too, though they
won't do much that's useful).
The big problem with making an OS X application out of a GTK program
is that it won't start unless DYLD_LIBRARY_PATH and several other
environment variables point at all the GTK machinery. So your app
bundle has to contain two programs: a launcher to set up that
environment, and then the real main program that the launcher execs
once it's done so.
But in our case, we also need pterm to start subprocesses _without_
all that stuff in the environment - so our launcher has to be more
complicated than the usual one, because it's also got to save every
detail of how the environment was when it started up. So this is the
launcher program I'm going to use. Comments in the header explain in
more detail how it'll work.
Also in this commit, I add the other end of the same machinery to
gtkapp.c and uxpty.c: the former catches an extra command-line
argument that the launcher used to indicate how it had munged the
environment, and stores it in a global variable where the latter can
pick it up after fork() and use to actually undo the munging.
When it's finished, this will be the backbone of the OS X GTK port:
using a GtkApplication automatically gives us a properly OS X
integrated menu bar.
Using this source file in place of gtkmain.c turns the usual Unix
single-session-per-process PuTTY or pterm into the multi-session-per-
process OS X style one.
Things like Duplicate Session can be done much more simply here - we
just grab the Conf * from the source window and launch a new window
using it, with no fiddly interprocess work needed.
This is still experimental and has a lot of holes, but it's usable
enough to test and improve.
mkicon.py now outputs .pam by hand, rather than using ImageMagick to
go straight to .png. For most purposes the main makefile then uses
ImageMagick anyway, to convert those .pams straight to the .pngs that
the rest of the scripts were expecting. But one script that doesn't do
that is macicon.py, which builds the MacOS .icns file by directly
reading those .pam files back in.
This allows the 'make icns' target in the icons directory to build
from a clean checkout on vanilla MacOS, without requiring a user to
install ImageMagick or any other non-core Python image handling
module.
(I could probably take this change at least a little bit further. I
don't see any reason why icon.pl - generating the Windows .ico files -
couldn't read the .pam files directly, about as easily as macicon.py
did, if anyone had a use case for building the Windows icons in the
presence of Python and Perl but in the absence of ImageMagick. But the
.png files are directly useful outputs for Unix, so _some_ PNG-writing
will have to remain here.)
Now we have licence.pl, it seems to me to make very good sense to have
it generate the Halibut form(s) of the licence and copyright year as
well as the source-code forms.
As a result, I believe _no_ copies of the licence text or copyright
date exist any more except for the master one in LICENCE - so I can
completely remove the checklist section about all the places to update
it, because there's only one. Hooray!
Now all the uses of the licence text or the short copyright notice get
it from a new header "licence.h", which in turn is built by a Perl
script licence.pl invoked by mkfiles.pl, using LICENCE itself as the
source.
Hence, I can completely remove a whole section from the list of
licence locations in CHECKLST.txt :-)
The Xcode icon composer doesn't seem to exist any more in modern
versions of Xcode, or at least if it does then it's well hidden and
certainly doesn't live at the top-level path at /Developer where web
pages still claim it can be found.
There is a free software 'libicns' and associated command-line tools,
but they're large, complicated, picky about the exact format of PNGs
they get as input, and in any case a needless extra build dependency
when it turns out the important parts of the file format can be done
in a few dozen lines of Python. So here's a new macicon.py, and
icons/Makefile additions to build a demo icon for OS X PuTTY, as and
when I finally get it working.
Also I've deleted the static icon file in the neglected 'macosx'
source directory, because this one is better anyway - the old one was
appalling quality, and must have been autogenerated from a single
image in some way.
encodelib.py is a Python library which implements some handy SSH-2
encoding primitives; samplekex.py uses that to fabricate the start of
an SSH connection, up to the point where key exchange totally fails
its crypto.
The idea is that you adapt samplekex.py to construct initial-kex
sequences with particular properties, in order to test robustness and
security fixes that affect the initial-kex sequence. For example, I
used an adaptation of this to test the Diffie-Hellman range check
that's just gone into 0.64.
Simon Tatham
Jacob Nevins