The aim is to let --parseonly succeeds even if the function
is not (yet) present. This is usefull in commit-hooks and
for the inline documentation generation system.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This was necessary because of how I changed file recursion.
The type works much more intelligently now -- files to
be removed have a file resource generated for them, and that
resource handles deletion.
Also fixes#1717; neither age nor size is required now.
Signed-off-by: Luke Kanies <luke@madstop.com>
This makes a lot of sense because source was always
more of a metaparameter than a property -- it affected
the 'should' values of other properties, but it shouldn't
have done any other work.
It will hopefully make everything else much cleaner.
This is such a large commit mostly because of the need
to fix a lot of tests.
Signed-off-by: Luke Kanies <luke@madstop.com>
This is a significant refactor of some very murky code, and it's
all much cleaner and more readable now. All of the 'newvalue' methods
and any value-related code is in a ValueCollection class.
This puts us in a good position to refactor the Property and Parameter
classes more completely.
Signed-off-by: Luke Kanies <luke@madstop.com>
Removing obsolete handleignore method
Removing obsolete FileSource class
Removing a now-obsolete test/unit test
Removing a now-obsolete recursive filebucket test
Signed-off-by: Luke Kanies <luke@madstop.com>
This is an unfortunately messy commit; I should have tried harder
to separate the different tasks into different commits, but it's
not worth going back and doing now.
This is the first commit in the long road of fixing the existing file
tests, and in the process refactoring and better testing the code.
The refactoring in this commit is mostly around the 'stat' instance
variable that tests whether the file exists (and provides its metadata
if it does) and the 'insync?' method in the source property.
Signed-off-by: Luke Kanies <luke@madstop.com>
The following manifest wasn't working:
class one {
notice('class one')
}
class one {
notice('second class one')
}
include one
It all boiled down to class code not being arrays.
Encapsulating code in ASTArray when needed is enough to append code,
because of the property of ASTArray to evaluate all their members in
turn.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
since that method is deprecated.
Conflicts:
CHANGELOG
bin/puppetca
lib/puppet/file_serving/fileset.rb
lib/puppet/network/xmlrpc/client.rb
lib/puppet/type/file/selcontext.rb
spec/unit/file_serving/metadata.rb
spec/unit/type/file.rb
This changesets allow empty if or else branches:
if true {
} else {
}
It works by emitting on the parser stack an AST node that doesn't
do anything (a no-op). This allows the less intrusive code
as no part of the if evaluation code has been touched.
The real problem was that the 'gid' and 'uid' methods didn't handle
the case where 'get_posix_field' didn't return a value, and
the subsequent 'get_posix_field' calls couldn't handle that.
This commit moves the tests for Posix to spec, and fixes the
specific bug.
Signed-off-by: Luke Kanies <luke@madstop.com>
The expressions can be used in if 'test' and in the
right side of assignements.
The expressions can contain any number of sub-expressions
combined by either arithmetic operators, comparison operators,
or boolean operators.
Random Usage Examples:
$result = ((( $two + 2) / $one) + 4 * 5.45) - (6 << 7) + (0x800 + -9)
or
if ($a < 10) and ($a + 10 != 200) {
...
}
variables that the puppet template should otherwise see. Specific changes:
* Added testing of the Scope#to_hash method, which returns a hash of name and
value pairs in a scope or, optionally, in the scope and enclosing scopes.
* Use member variables rather than methods in the function tests.
* Fix up tests that fail once we move over to instance variables rather than
methods: Puppet can no longer detect an undefined variable reference, so we
end up failing any test that expected to get a parser error.
* Several tests have manual checks introduced to simulate an end user manually
writing the checks that used to be automatic, and others drop the validation
that parsing fails when a variable is not in scope.
* Added tests for legacy variable lookup and that the shadowing of local
variables by Kernel methods is still in effect.
Signed-off-by: Daniel Pittman <daniel@rimspace.net>
(cherry picked from commit 5c5f315cceadc52203e53883b77bc01c1d7a2e7f)
Darwin has a case-insensitive FS, so the test was failing
because it was specifically testing case sensitivity.
Signed-off-by: Luke Kanies <luke@madstop.com>
with 'yaml' still being the default but 'marshal' being an option.
This is because testing has shown drastic performance differences
between the two, with up to 70% of compile time being spent
in YAML code. Use the 'catalog_format' setting to choose your format,
and the setting must be set on the client.
Signed-off-by: Luke Kanies <luke@madstop.com>
I.e., you can't do confine :operatingsystem => %w{a b} and then
confine :operatingsystem => %w{b c}; you'd need to do them in one command.
This now-obsolete behaviour does not seem to be used anywhere.
The fix for #1431 is actually just removing the tests that exposed
this change; the change happened when I refactored how confines work.
Signed-off-by: Luke Kanies <luke@madstop.com>
Here were the main changes necessary:
* Fixed the class loader so it only loads mongrel if it's available.
* Fixed the test runner to skip example groups contained in non-runnable
example groups.
* Fixed the Mongrel tests to use quoted class names instead of constants,
since the constants themselves would be absent.
Signed-off-by: Luke Kanies <luke@madstop.com>
The class had a 'transaction' accessor that was assigned
but never used, and it is simple enough that it needed
direct arguments rather than named arguments.
The rest of the code is changing the other classes that use
Events.
Signed-off-by: Luke Kanies <luke@madstop.com>
Also added the fixes to make the certhandler tests pass
even when certs exist; I'll deal with the conflict later.
Conflicts:
CHANGELOG
bin/puppetd
lib/puppet/network/http/handler.rb
lib/puppet/network/http/mongrel/rest.rb
spec/integration/indirector/rest.rb
spec/integration/network/server/mongrel.rb
spec/integration/network/server/webrick.rb
spec/unit/network/http/webrick.rb
The Master handler previously provided the support for the :node_name
setting, and that functionality has now been moved into the Node
class. At the same time, the names to search through have been
changed somewhat: Previously, the certificate name and the
hostname were both used for searching, but now, the cert name
is always searched first (unless node_name == facter), but only
the Facter hostname, domain, and fqdn are used otherwise. We no
longer split the cert name, only the hostname/domain/fqdn.
In the general case, this provides no behaviour change, because
people's hostname is the same as their certname. This only
results in a change in behaviour if you specify a certificate
name that is a normal node name, and you want to look that node
up by something other than the full name in the certificate.
Signed-off-by: Luke Kanies <luke@madstop.com>
Previously, we prefetched then generated, which caused generated
resources that needed prefetching not to work. This just reorders
the calls, so generated resources now get prefetched.
I created a Confiner module for the Provider class methods,
and then I enhanced the interface between it and the Confine
class to make sure binary paths are searched for fresh each time.
This fixes#1197, which was a result of binary paths being
searched for at startup, rather than at execution.
The HttpPool module now removes its cached ssl_host instance
when clearing its cache. This is really only useful
for testing, but it correctly causes the pool to use
new certificates when they're available.
This class provides all of the semantics from puppetca,
and appears to entirely duplicate the behaviour of the existing
executable, with basically all of the code in a library
file, instead of the executable.
As such, I've deleted the test for the executable. We should have
one, but it's not nearly as important.
the [] accessor method (they previously threw an exception when
this method was used with properties). This shouldn't have any
affect functionally; it just makes the method equivalent to 'should'
for properties, but it works for all attribute types now.
compile node configurations, rather than using the Configuration
handler, which was never used directly. I removed the Configuration
handler as a result.
Modified the 'master' handler (responsible for sending configurations
to clients) to always return Time.now as its compile date, so
configurations will always get recompiled.
evaluation time, rather than resource instantiation time. This
means that you don't catch your "errors" as early, but it also
means you should be able to realistically configure a whole host
in one run.
any local variables and added a local variable -- see
http://snurl.com/21zf8. My own testing showed that this
caused memory growth to level off at a reasonable level.
Note that the link above says the problem is only with class
methods, but my own testing showed that it's any method that
meets these criteria. This is not a functional change, but
should hopefully be the last nail in the coffin of #1131.
files, thanks to work by Adam Jacob and Arjuna Christenson
(the finding, not the leak). I'm going to act like this
fixes#1131, at least for now, but I doubt it does,
since that shows general memory growth over time, whereas
the leak here should go away as soon as files are reparsed
(because the parser is holding the reference to the leaking
array).
When :node_name="cert" is specified the 'hostname' fact should be set to the SSL certificate common name instead of the results from facter. I've extended this to also set 'domain' and 'fqdn' since that makes a lot of sense to me. This fixes a regression introduced in SVN#1673
in the Exec resource type -- the log levels are specified
using the 'loglevel' parameter, not 'logoutput'. This never
worked, or at least hasn't for ages, and now the docs are
just correct.
any objects, nor does it use Puppet's RAL resources. In the
process, I fixed#894 (you can now copy links) and refactored
other classes as necessary. Mostly it was fixing tests.
This is a squashed commit of a temporary branch, fwiw,
and it also includes any fixes to the tests that were
necessary to get all tests passing again.
NOTE: This introduces a behaviour change, in that you previously
could realize a resource within a virtual defined resource, and now
you must realize the entire defined resource, rather than just
the contained resource.
*sworn* I did this weekend). In the process, I fixed
a couple of bugs related to differentiating between
nodes and classes, and then cleaned up quite a few
error messages.
just returns a resource from its evaluate() method, and
all of the work is done in the evaluate_code method. This
makes the code cleaner, because it means 1) evaluate() has
the same prototype as all of the other AST classes,
2) evaluate() is no longer called indirectly through
the Parser Resource class, and 3) the classes themselves
are responsible for creating the resources, rather than
it being done in the Compile class.
all of the evaluate() methods only ever accepted a scope,
and sometimes one other option, so I switched them all to
use named arguments instead of a hash.
The only remaining failures are more complicated ones (which I'll
need to not be on a plane to debug, for battery reasons) or those
related to the broken directory_service providers.
We still have about 60 failing tests, but some of them are
the failing directory service tests (probably 20 or so),
and most are simple fixes to the tests themselves.
a central module responsible for managing the http pool
(Puppet::Network::HttpPool), and it also handles
setting certificate information. This gets rid of
what were otherwise long chains of method calls,
and it makes the code paths much clearer.
are running), and #918 (service tests fail when hddtemp is not installed).
Mostly, I just rewrote the service tests, but I cleaned up the cruft from the
Service class, too.
there were enough problems fixing it that I decided something
more drastic needed to be done.
This uses the new Puppet::ResourceReference class to canonize
what a resource reference looks like and how to retrieve resources
via their references. Specifically, it guarantees that resource types
are always capitalized, even when they include '::' in them.
While many files are modified in this commit, the majority of changes are
quite small, and most of the changes are fixing the tests to use
capitalized types.
As we look at consolidating some of our resource types, we could consolidate
the ResourceReference stuff at the same time, but at least the
Puppet::Parser::ResourceReference class subclasses the main Puppet::ResourceReference
class.
so that it correctly just yields if you're not root. It
also no longer tries to set :uid or :gid; just :euid and :egid,
and it once again sets :egid before it sets :euid, which is
important because you usually can't change your group after
you've changed your user id.
what transportable resources get converted to, so
different names don't throw it off.
I also got rid of the Puppet::Type#merge method, which
has been deprecated for ages but was still in there. I
had to fix a few tests that weren't cleaning up after themselves
as a result.
supposed to be doing and to skip the whole network connect thing.
FTR, the reason this is happening is because the keepalive patch
causes http instances to start immediately, which wasn't previously
the case. I have been noticing that not all failures that should
result from this happen consistently.
already migrated all of the tests into rspec but forgot about these
tests -- they were only in the rails/ subdir because people kept
not running the parser/ tests after modifying the Rails code.
mount, fixing #891. The patch was ported to the current
code by David Schmitt, I applied the rest of Matt's
patches, and I then fixed all of the code so that the
tests passed.
The primary change I had to make to the patch was reenabling
host expansion in paths -- his patch had disabled it.
from womble, and rewriting and significantly enhancing the
unit tests for the Puppet::Parser::Collector class; it should
have full coverage now. There are no integration tests for
it, so there's still no guarantee that it works at all, but hey,
we're a lot better off than we were.
part of the suite, but failed when run individually. The
problem was that I changed lib/puppettest/support/utils to
have a separate module and I didn't test the individual files.
from the server. The real problem was getting all of the validation
done before any caching, which required a good bit more refactoring
than I expected.
In actuality, this commit is relatively small even though it covers
many files; most of the changes just make the code clearer or shorter.
switching the node scope to no longer be lazy evaluation,
just like I switched 'main'. When I made all of these
classes and nodes lazy evaluated, I should have decoupled
my real goal (using resources to evaluate them) from the idea
of lazy-evaluating them, and this basically does that.
I also changed the scope heirarchy slightly so that scopes will
tend to be below the node scope, altho this was already generally
the case.
'include' function is used, instead of being lazy-evaluated.
Previous work caused resources to get created to model
these classes, but in the process, I removed the fact
that the classes were evaluated immediately. This meant
that you couldn't guarantee that a class was evaluated
before you went to use its variables.
It was previously using the GRATR::Edge class, which
had wonky overrides that dramatically slowed down
sorting (its hash mechanism hashed the source and
target so that edges with the same source/target got
the same hash, which we actually don't want any more).
This shouldn't change any functionality, just performance.
I didn't retain all functionality from the Edge class, but
a lot of that functionality was, um, horrible, like Edge[]
being equivalent to Edge.new.
to Puppet::SimpleGraph, which should dramatically enhance
performance. It should be largely functionally equivalent,
with the only difference being that edges are no longer deduplicated.
I've provided backward compatibility with the old
handler.
The only terminus type that currently exists for reports
is the 'code' terminus, which is used to process reports
in the style of the old handler. At some point, we should
likely switch at least some of these report types (e.g., 'store')
to terminus types.
This is the first real pass towards using caching. The `puppet`
executable actually uses the indirection work, instead of
handlers and such (and man! is it cleaner).
Most of this work was a result of trying to get the client-side
story working, with correct yaml caching of configurations, which
means this commit also covers converting configurations to yaml,
which was a much bigger PITA than it needed to be.
I still need to write integration tests, and I also need to cover
the server-side story of a normal configuration retrieval.
to the configuration system. 'puppet' itself still
works, even with -e, but I expect that puppetd and
puppetmasterd are broken, and there are still quite a few
broken tests because the default fact store can't write but
that's the default behaviour for a networked configuration
master.
instead of a manifest, and removing all of the ambiguity
around whether an interpreter gets its own file specified
or uses the central setting.
Most of the changes are around fixing existing tests to use this new system.
to work. As a result, it involves a lot of integration-level
testing, and a lot of small design changes to make the code
actually work.
In particular, indirections can now have default termini,
so that configurations and facts default to their code terminus
Also, I've removed the ability to manually control whether
ast nodes are used. I might need to add it back in later,
but if so it will be in the form of a global setting,
rather than the previous system of passing it through 10 different
classes. Instead, the parser detects whether there are AST nodes
defined and requires them if so or ignores them if not.
About 75 tests are still failing in the main set of tests,
but it's going to be a long slog to get them working --
there are significant design issues around them, as most of
the failures are a result of tests trying to emulate both the
client and server sides of a connection, which normally would
have different fact termini but in this case must have the same
terminus just because they're in the same process and are global.
The next step, then, is to figure that process out, thus finding a way
to make this all work.
get a test to show the exception, nor could I figure out how the coding bug
could have actually had an impact, but it's an innocent-enough fix, so I'm fine
applying it.
to sometimes suddenly disappear and thus tests to fail --
Kernel.require was not loading the normal ruby path (e.g.,
'puppet/type/cron'), so if someone else loaded that then
it would replace the in-memory type with a new one, but that
new one couldn't load its own providers, because the Kernel
would ignore the providers, thinking they were already loaded.
This doesn't fix all of the autoloading problems, but at least
we won't suddenly break a ton of tests.
'Puppet::Util::Settings'. This is to clear up
confusion caused by the fact that we now have a
'Configuration' class to model host configurations,
or any set of resources as a "configuration".
it's time to merge it back into the indirection branch.
Considering that this work was what drove me to create the
indirection branch in the first place, i should now be able to
merge both back in the master branch.
in place, which means I'm over the hump in developing
this branch.
I have to fix some design flaws I made in the configurations,
particularly that the 'runner' handler needs to be able to
specify tags and whether to ignore schedules, but otherwise,
I think it's straightforward test- and bug-fixing from here out.
branch. The file recursion code actually works for the first
time in a painful while, but there are still some quirks and design
issues to resolve, particularly around creating implicit resources
that then fail (i.e., the behaviour of the create_implicit_resource
method in Configuration).
I've gone too far down the rabbit hole to turn back now, but the
code is clearly getting more centralized around the Configuration
class, which is the goal.
Things are currently a bit muddy between recursion, dynamic resource
generation, transactions, and the configuration, and I don't expect
to be able to clear it up much until we rewrite all of the tests
for the Transaction class, since that is when we'll actually be
setting its behaviour.
At this point, Files (which are currently the only resources that
generate other resources) are responsible for adding their edges
to the relationship graph. This puts them knowing more than I would
like about how the relationship graph works, but it'll have to do for now.
There are still failing tests, but files seem to work again. Now to
go through the rest of the tests and make them work.
ever converting the Transportable objects into a tree of components
and then converting that into a graph. This is a significant
step, and drastically simplifies the model of how to use a configuration.
The old code might have looked something like this:
file = Puppet::Type.create :path => "/whatever", ...
comp = Puppet::Type.create :name => :whatever
comp.push file
transaction = comp.evaluate
transaction.evaluate
The new code looks like this:
file = Puppet::Type.create :path => "/whatever", ...
config = Puppet::Node::Configuration.new
config.add_resource file
config.apply
I did not really intend to do this much refactoring, but I
found I could not use a Configuration object to do work
without refactoring a lot of the system. The primary problem
was that the Client::Master and the Config classes determined
how the transactions behaved; when I moved to using a Configuration,
this distinction was lost, which meant that configurations were
often needing to create other configurations, which resulted in
a whole lot of infinite recursion (e.g., Config objects that create
directories for Puppet use Configuration objects -- yes, I'm
s/Config/Settings/g soon -- and these Configuration objects would
need to create directories).
Not everything is fixed, but it's very close. I am clearly over
the hump, though, so I wanted to get a commit in.
James Turnbull
Brice Figureau
Luke Kanies
Francois Deppierraz
Andrew Shafer
Daniel Pittman
Francois Deppierraz
Paul Lathrop
Sam Quigley
John Ferlito
Jeffrey J McCune
David Schmitt
Michael V. O'Brien
Marcin Owsiany
Rick Bradley
David Lutterkort