The number of people has been steadily increasing who read our source
code with an editor that thinks tab stops are 4 spaces apart, as
opposed to the traditional tty-derived 8 that the PuTTY code expects.
So I've been wondering for ages about just fixing it, and switching to
a spaces-only policy throughout the code. And I recently found out
about 'git blame -w', which should make this change not too disruptive
for the purposes of source-control archaeology; so perhaps now is the
time.
While I'm at it, I've also taken the opportunity to remove all the
trailing spaces from source lines (on the basis that git dislikes
them, and is the only thing that seems to have a strong opinion one
way or the other).
Apologies to anyone downstream of this code who has complicated patch
sets to rebase past this change. I don't intend it to be needed again.
They're the intermediate product between preprocessing a resource file
and feeding it to the resource compiler proper, so they certainly need
cleaning.
This is for sanitising output that's going to be sent to a terminal,
if you don't want it to be able to send arbitrary escape sequences and
thereby (for example) move the cursor back up to existing text on the
screen and overprint it confusingly.
It works using the standard C library: we convert to a wide-character
string and back, and then use wctype.h to spot control characters in
the intermediate form. This means its idea of the conversion character
set is locale-based rather than any of our own charset library's fixed
settings - which is what you want if the aim is to protect your local
terminal (which we assume the system locale represents accurately).
This also means that the sanitiser strips things that will _act_ as
control characters when sent to the local terminal, whether or not
they were intended as control characters by a server that might have
had a different character set in mind. Since the main aim is to
protect the local terminal rather than to faithfully replicate the
server's intention, I think that's the right criterion.
It only strips control characters at the charset-independent layer,
like backspace, carriage return and the escape character: wctype.h
classifies those as control characters, but classifies as printing all
of the more Unicode-specific controls like bidirectional overrides.
But that's enough to prevent cursor repositioning, for example.
stripctrl.c comes with a test main() of its own, which I wasn't able
to fold into testcrypt and put in the test suite because of its
dependence on the system locale - it wouldn't be guaranteed to work
the same way on different test systems anyway.
A knock-on build tweak: because you can feed data into this sanitiser
in chunks of arbitrary size, including partial multibyte chars, I had
to use mbrtowc() for the decoding, and that means that in the 'old'
Win32 builds I have to link against the Visual Studio C++ library as
well as the C library, because for some reason that's where mbrtowc
lived in VS2003.
In the clang-cl makefile, we run the .rc file through the preprocessor
and the actual resource compiler in two separate steps. But all the
include-file dependencies were being put on the _latter_ step, so
editing a .rc2 file didn't trigger a rebuild of the resource file.
Although C99 introduces variable length array (VLA) feature, it is
not supported by MS Visual Studio (being C++ compiler first of all).
In this commit, we add Clang and GCC options to disable VLA, so they
would be disabled in non-MSVC builds as well.
Some still-supported Linux distributions (Debian jessie and Ubuntu 14.04
at least) still use GCC 4, where C99 isn't the default.
For autoconf, we do it the autotools way. For the standalone Makefiles,
we go for -std=gnu99 rather than c99, to avoid trouble with fdopen().
Now $(CC) is defined to be nothing but the name of the clang-cl binary
itself, which makes it easier to drop in a different one for a special
purpose.
(I tried to use this for static analysis recently - unsuccessfully, as
yet, but I think this change will make anything else along the same
lines easier as well.)
Now we don't have to worry about which windres we're using (or whether
another target architecture's windres will do just as well), this is
very easy - just test for a couple of extra values of $(Platform).
To build on Arm with VS2017 includes and libraries, various blog posts
and websites explain that you have to #define a cumbersome macro
called _ARM_WINAPI_PARTITION_DESKTOP_SDK_AVAILABLE, without which the
headers will #error at you. But if you do that, then everything seems
to compile fine and I actually tested it on an Arm Windows machine
today.
Also, I had to disable stack protection (/GS-), because clang-cl
doesn't yet support the particular form of it for which the VS2017 Arm
C library provides the runtime support. Unfortunate in a security
application, of course, but there we go.
Previously I was using clang-cl as my compiler, lld as my linker, and
GNU windres as my resource compiler, which made a confusing hybrid of
the LLVM and GNU toolchains. This was because llvm-rc had about four
missing features that stopped it being able to handle PuTTY's resource
files. (Some dialog control types; dialog class names; handling
preprocessor output without getting confused by line markers and
snippets of stray C; not complaining about the DISCARDABLE keyword.
Although admittedly I could have dealt with the last of those by just
removing it from my .rc files, because it's pointless anyway.)
In the past month, the llvm-rc developers have been hard at work, and
now it has _all_ those features! So now I can switch over to a purely
LLVM-based toolchain for my Windows builds, which is easier to set up
(and easier to tell other people how to set up, since they get it for
free with the rest of LLVM); doesn't have a nominal architecture
dependency (windres has to built against a particular flavour of
binutils); and produces bit-identical output to Visual Studio's
resource compiler as far as I can see (whereas windres is more in the
'close enough' area).
This needed a small makefile restructuring, because unlike windres,
llvm-rc doesn't have a built-in option to run the resource file
through the preprocessor. So now Makefile.clangcl has separate rules
to preprocess $tool.rc into $tool.rcpp and then compile the latter
into $tool.res.
If you forget the '+' at the start of a continuation line with only
one word on it, then Perl would test $_[1] before checking that it
even existed to test. The test would give the right answer anyway, but
the warning was annoying.
These include an unused variable left over from the command-line
refactoring; an explicit referencing of the module handle for
sspicli.dll which we really do deliberately load and then don't
(directly) use; a missing pointer-type cast in the Windows handle
socket code; and two 32/64 bit integer size mismatches in the types of
functions I was importing from system API DLLs.
The last of those are a bit worrying, and suggest to me that after
going to all that trouble to add type-checking of those runtime
imports in commit 49fb598b0, I might have only checked the resulting
compiler output in a 32-bit build and not a 64-bit one. Oops!
After a conversation this week with a user who tried to use it, it's
clear that Borland C can't build the up-to-date PuTTY without having
to make too many compromises of functionality (unsupported API
details, no 'long long' type), even above the issues that could be
worked round with extra porting ifdefs.
I've just upgraded my build process to a version of lld-link
that knows how to read .res, and I think it's a slightly more
commonly found format, so less confusing to encounter.
I've been experimenting with using clang-cl with older versions of the
Visual Studio libraries and headers, and found that the older VS
libraries have a quirk which can cause link failure in a way that
doesn't happen with the newer one. I assume the corresponding old VS
linker must be doing some kind of special-case handling that lld isn't
mimicking.
The quirk arises because lld tries to pull in more than one of the
*crt0.obj startup objects which set things up before calling main or
WinMain (or whatever), and those objects define some of the same
symbols as each other. The fix is to explicitly ask for the right one
of those objects on the link command line, so that it's already been
loaded _before_ the linker starts searching libraries for unresolved
symbols; then the disputed symbols are never unresolved in the first
place during the library search phase.
But this means you have to pick your crt0 object differently depending
on which subsystem you're compiling for. Accordingly, here's an extra
feature in Makefile.clangcl to make that possible by means of the
right definitions on the make command line.
Thanks to Brian K. White for spotting this straight-up syntax error of
a missing ), in the regex handling the special case of &splitlines
when it findss a word in its input string too long to fit in the
specified output line width. Apparently in all my own uses of
&splitline I'd never exercised that special-case code path before.
I had accidentally included the experimental "XT" app class (the
GtkApplication-based packaging of Unix PuTTY/pterm for OS X) among the
things that should still be built even when GTK is absent. That's
definitely wrong.
This was very strange to write, because it's a bizarre combination of
the GNU-make-isms and rc commands of Makefile.mgw with the cl and link
commands of Makefile.vc (but also the latter thankfully doesn't need
those horrible response files).
I've added a big comment in mkfiles.pl about what the build
requirements for this makefile actually are, which _hopefully_ will be
usable by people other than me.
In commit be586d53b I made empty.h depend on $(allsources), which
unfortunately was defined so as to include empty.h. This was harmless,
because make just ignored the circular dependency, but annoying,
because it constantly mentioned that it was ignoring it.
By default the VS2015 linker produces binaries with the minimum
version fields in the PE header set to 06.00, which causes XP not to
recognise them as valid binaries at all. But there's no other reason
VS2015-built binaries _can't_ run on versions of Windows as old as XP;
so here I add the link option to set those fields to 05.01 which makes
XP like them again.
This only applies to the 32-bit build, because the VS2015 64-bit
linker refuses to lower the min version field to under 06.00.
Originally added in commit 0014ffb70, and promptly reverted in
6bea4b250 when we realised that VS2003 didn't actually understand
them. But now we're building with VS2015, which does understand them,
it's actually useful to put them back in again.
Looking more closely, it turns out that VS2003 didn't actually _fail
to build_ if you passed these flags on the linker command line - it
just printed a warning and ignored them. (So there was no actual need
to revert the original change, except that it would have caused
confusion.) But that means I can add them unconditionally now, without
breaking even the legacy VS2003 build.
I've reset the baseline to be the version of mingw-w64 that comes with
Ubuntu 14.04. Right now, that means no features need to be omitted; all
you need to do is set TOOLPATH to i686-w64-mingw32- .
I've removed -mno-cygwin without comment. Toolchains which don't support
this flag have been around since at least 2012, so we can probably
assume that no-one cares about older toolchains by now.
It's really only useful with MinGW rather than a Cygwin toolchain these
days, as recent versions of the latter insist against linking with the
Cygwin DLL.
(I think it may no longer be possible to build with Cygwin out of the
box at all these days, but I'm not going to say so without having
actually checked that's the case. Settle for listing MinGW first in
various comments and docs.)
Previously, if you tried to set the special cflags for an object file
to the empty string, mkfiles.pl would normalise that to the string
"1". I'm not entirely sure why - that line of code was added without
explanation in commit 64150a5ef which brought in that directive in the
first place - but I have to guess that it was left over from some
earlier design iteration in which I hadn't quite decided whether I was
going to need a string or a boolean to separate version.o from other
objects.
Of course, setting an object's cflags to "" is a bit of a weird thing
to want to do anyway - why not just leave them unset? But in fact I've
now thought of something useful for it to do: this commit arranges
that setting cflags="" has the effect (in the 'am' makefile type) of
separating the object out into its own little automake library but not
actually giving that library any separate cflags. And the point of
_that_, in turn, will be that then you can add cflags to it
_conditionally_ in a "!begin am" snippet, e.g. conditionalised on
something in configure.
With all due respect to Microsoft, a cross-platform program simply
cannot switch to using MS's assorted 'secure' versions of standard C
functions if it wants to continue compiling on platforms other than
Windows. So I might as well squash the warnings, so that any other
more interesting compiler warnings can avoid being swamped in the
mess.
This is to [X] what [UT] is to [U]: that is, it's a program linked
against the GTK libraries, but one which doesn't become part of the
'make install' set. I'll use this for the individual binaries that
will go in the OS X application bundles, and then have another
makefile rule pick those up in turn.
Now all the uses of the licence text or the short copyright notice get
it from a new header "licence.h", which in turn is built by a Perl
script licence.pl invoked by mkfiles.pl, using LICENCE itself as the
source.
Hence, I can completely remove a whole section from the list of
licence locations in CHECKLST.txt :-)
Occurred as a side effect of commit 198bca233, in which I wrote a Perl
loop of the form 'foreach $srcdir (@srcdirs)' inside which I modified
$srcdir - forgetting the Perl gotcha that if you do that, $srcdir
temporarily aliases the actual array element, so you end up modifying
the array you iterated over. Hence, a set of transformations intended
to convert the source directory list into a special form for the nmake
batch-mode inference rule syntax in particular ended up back in
@srcdirs to be reflected in unrelated makefiles output later in the
run.
Now you can run a command like "nmake /f Makefile.vc BUILDDIR=foo\",
which will cause all the generated files to appear in a subdirectory
of putty\windows. This is immediately useful for testing multiple
build configurations against each other by hand; later on I hope it
will also be a convenient way to run multiple build configurations in
the proper bob build.
This enables it to combine the compilation of multiple source files
into a single 'cl' command with multiple input file arguments, which
speeds up the build noticeably.
(I think nmake could be doing a lot more to improve this - for a
start, I haven't found any way to let it aggregate compilations of
source files in more than one directory, and also, it seems to me that
it really ought to be able to reduce down to just _one_ invocation of
cl by choosing the best topological sort of its build operations,
whereas in fact it looks as if it's sorting the operations _before_
doing the aggregation. But even so, it's a big improvement on the
previous build time.)
This is noticeably faster than a sequence of 'echo' commands, because
the file gets created all in one go. The most natural approach to this
job would also hide the file's contents, but doing it this way with a
'type' command lets me see the file on nmake's standard output, so
that the build log should still contain everything useful for
debugging build problems.
It's for regression testing and fuzzing, so there's no use for it if
you're not a developer working on the source.
Leaving it out of the 'make install' target in Makefile.gtk is no
trouble because that's already handled manually in Recipe by inserting
a giant hairy Makefile fragment to do the installation. But
Makefile.am was just setting bin_PROGRAMS to the full set of binaries
built, so for that one, I had to invent a new Recipe program category
[UT] which moves a particular binary into noinst_PROGRAMS.
While I was at it, I've retired the [M] program category, which has
been lying around unused since Ben's old Mac OS pre-X port.
mkfiles.pl was giving a couple of annoying perl warnings, because some
makefile_extra strings were never set by Recipe. We already have the
&def function to convert undefs into "" for this reason, but weren't
using it everywhere. Now I think we are.
I've shifted away from using the SVN revision number as a monotonic
version identifier (replacing it in the Windows version resource with
a count of days since an arbitrary epoch), and I've removed all uses
of SVN keyword expansion (replacing them with version information
written out by Buildscr).
While I'm at it, I've done a major rewrite of the affected code which
centralises all the computation of the assorted version numbers and
strings into Buildscr, so that they're all more or less alongside each
other rather than scattered across multiple source files.
I've also retired the MD5-based manifest file system. A long time ago,
it seemed like a good idea to arrange that binaries of PuTTY would
automatically cease to identify themselves as a particular upstream
version number if any changes were made to the source code, so that if
someone made a local tweak and distributed the result then I wouldn't
get blamed for the results. Since then I've decided the whole idea is
more trouble than it's worth, so now distribution tarballs will have
version information baked in and people can just cope with that.
[originally from svn r10262]
This rearranges the object files so that they each live alongside
their original source file, instead of all being in the same
directory. To my way of thinking this is a more or less neutral change
(perhaps marginally less tidy), but autotools is apparently beginning
to think it's the One True Way and 1.14 will give a warning if you
don't have it enabled.
[originally from svn r10142]
Previously, 'configure' and its assorted machinery lived in the 'unix'
subdir, because that seemed like a clean place to keep it given that
all the other per-platform Makefiles live in their platform
directories. However, this never sat all that happily with autotools,
and even less so now that it likes to have object file pathnames
parallel source file pathnames: if you have Makefile.am refer to
source files outside its subdir as "../terminal.c" and enable
subdir-objects then any out-of-tree build calls the corresponding
object file "../terminal.o" and so your build products mostly end up
at the directory above your build dir! And as of autotools 1.14 my
previous compensatory bodge of prefixing every source file path in
Makefile.am with "$(srcdir)" has stopped working too.
So I'm giving in to necessity, and changing policy by moving the
configure machinery up to the top level of the source tree where
autotools will be less confused by it. This should not be taken as any
indication of the primacy of the Unix port, only of the recalcitrance
of autotools.
Whereas before we had a trivial script called 'configure' at the top
level that invoked unix/configure to effectively do an 'out-of-tree
build' (for make purposes) at the top level of the source tree, we now
have a similar script in unix/configure. So this _should_ make very
little difference: people who were previously running configure from
the top level should still be able to, and likewise people who were
running it from the unix subdir.
[originally from svn r10141]
Thanks to Mike Edenfield for the initial version of this patch; I've
polished it up a bit (in particular inventing a more overengineered
GUID generation strategy) but most of it is his.
[originally from svn r10112]
handle. Revert that when we hackily call it from mkfiles.pl, so that
if I have a need to insert diagnostics in the latter they won't go
into the end of sbcsdat.c.
[originally from svn r10013]
and into AM_CPPFLAGS. This is more conceptually sensible according to
my reading of the automake manual, and also has the specific desirable
effect that they move to the front of the command line, ahead of any
'system' type -I options that autoconf might have felt a need for.
A user reported that autoconf had added -I/usr/local/include to their
command line for the sake of a required header file, but their
/usr/local/include also turned out to include a thing called 'proxy.h'
(from libproxy, nothing to do with us) which shadowed our own proxy.h
and caused a build failure. This should fix that.
[originally from svn r9736]
Simon Tatham
Pavel I. Kryukov
Jacob Nevins
klemens
Owen Dunn