This is mostly just adjusting existing tests to
meet new APIs, but it's a small amount of fixing the
code to meet new standards and an even smaller amount
of porting code over.
Signed-off-by: Luke Kanies <luke@reductivelabs.com>
Events are now queued as they are created, and
the queues are managed through simple interfaces,
rather than collecting events over time and
responding to them inline.
This drastically simplifies event management,
and will make moving it to a separate system
essentially trivial.
Signed-off-by: Luke Kanies <luke@madstop.com>
This bring a new container syntax to the Puppet DSL: hashes.
Hashes are defined like Ruby Hash:
{ key1 => val1, ... }
Hash keys are strings, but hash values can be any possible right
values admitted in Puppet DSL (ie function call, variables access...)
Currently it is possible:
1) to assign hashes to variable
$myhash = { key1 => "myval", key2 => $b }
2) to access hash members (recursively) from a variable containing
a hash (works for array too):
$myhash = { key => { subkey => "b" }}
notice($myhash[key][subjey]]
3) to use hash member access as resource title
4) to use hash in default definition parameter or resource parameter if
the type supports it (known for the moment).
It is not possible to string interpolate an hash access. If it proves
to be an issue it can be added or work-arounded with a string concatenation
operator easily.
It is not possible to use an hash as a resource title. This might be
possible once we support compound resource title.
Unlike the proposed syntax in the ticket it is not possible to assign
individual hash member (mostly to respect write once nature of variable
in puppet).
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Most of these were caused by alias --> host_aliases; one was
caused by the canonicalization of resource references and one
by the removal of puppet_module.
This refactors how reports, catalogs, configurers, and transactions
are all related - the Configurer class manages the report, both
creating and sending it, so the transaction is now just responsible
for adding data to it. I'm still a bit uncomfortable of the coupling
between transactions, the report, and configurer, but it's better than
it was.
This also fixes#2944 and #2973.
Signed-off-by: Luke Kanies <luke@madstop.com>
The following manifest doesn't work:
$foo = undef
case $foo {
undef: { notice("undef") }
default: { notice("defined") }
}
This is because "undef" scope variable are returned as an empty
string.
This patch introduces a behavior change:
Now, unassigned variable usage returns also undef.
This might produce some issues in existing manifests, although
care has been taken to allow correct behavior in the most commonly
used patterns.
For instance:
case $bar {
undef: { notice("undef") }
default: { notice("defined") }
}
will print "undef".
But matching undef in case/selector/if will also match "".
case $bar {
"": { notice("empty") }
default: { notice("defined") }
}
will print "empty".
Of course "" doesn't match undef :-)
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This is technically a duplicate of #1531, I think this change prevents
the problem that appears in #2812, without touching the underlying issues
of #1531.
ssh_authorized_key was failing on keys in ~/.ssh/authorized_keys that
lack a comment field - it would generate a Ssh_authorized_key resource
with the name set to nil, which raises "ArgumentError: Field 'name' is
required."
Fixed by setting such keys' name fields to an empty string.
This prevents the error from being raised and the authorized_keys files
round-trip successfully.
Signed-off-by: Jesse Wolfe <jes5199@gmail.com>
In post processing a Symbol was being passed to StringScanner.
StringScanner was not happy with this. The error message lost
backtrace information and the test coverage was both inadequate
and broken (see #2745).
To be fully effective, this patch needs/assumes the patch for
Signed-off-by: Markus Roberts <Markus@reality.com>
The old fakedata test facility was not playing nicely with the
spec tests; although it looped through all the files failures
in any example file after the first were being ignored because
of the way fakedataparse was interacting with the before blocks.
Signed-off-by: Markus Roberts <Markus@reality.com>
With the path parameter set to false, the server defaults
to Puppet[:server]. This allows to use a remote filebucket without
syncing the servername there with the one used on the config file.
To use the default server, this manifest can be used:
filebucket { main: path => false }
A related bug report is:
http://projects.reductivelabs.com/issues/2456
These tests tried to load something, verified
the loads didn't work, and then created
the thing to load. This is a bit silly,
so I just removed those sections of the tests.
Signed-off-by: Luke Kanies <luke@madstop.com>
This was to fix a failing test/unit test.
Test coverage is now a bit better, more maintainable,
and I refactored the code just slightly to make it a bit
cleaner.
Signed-off-by: Luke Kanies <luke@madstop.com>
This patch uses the unused AST::HostName as the only way to reference
a node in the AST nodes array.
The AST::HostName respect the hash properties of the underlying
string, to keep the O(1) hash properties.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This changeset introduces regexp in if expression with the use of the
=~ (match) and !~ (not match) operator.
Usage:
if $uname =~ /Linux|Debian/ {
...
}
Moreover this patch creates ephemeral variables ($0 to $9) in the current
scope which contains the regex captures:
if $uname =~ /(Linux|Debian)/ {
notice("this is a $1 system")
}
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This fixes the behaviour when you have file recursions
that overlap - we again correctly use the most
specific information.
It's still a bit expensive when you do this, but
at least it behaves correctly, and it should be
a rare circumstance.
Signed-off-by: Luke Kanies <luke@madstop.com>
This feature has been broken since who knows when,
yet no one's noticed. Thus, it's a good
candidate for removal.
Signed-off-by: Luke Kanies <luke@madstop.com>
We have extended the concept of provider specificity so it
now includes both specified defaults and class depth, so:
* We are much more likely to choose the correct provider;
e.g., 'init' will be chosen over 'base'
* We're much less likely to print this warning, because it's
only printed when provider specificities are equal which
is much rarer
lib/puppet/provider.rb | 4 ++--
lib/puppet/type.rb | 4 ++--
spec/unit/type.rb | 17 +++++++++++++++++
test/ral/manager/type.rb | 22 ----------------------
4 files changed, 21 insertions(+), 26 deletions(-)
Signed-off-by: Luke Kanies <luke@madstop.com>
We basically just make sure that we tell Ruby
about files we've loaded, so you can 'require' these
files and doing so will essentially no-op, rather
than clobbering the already-loaded code.
Signed-off-by: Luke Kanies <luke@madstop.com>
This patch removes the limitation of allow/deny which were
only matching ip addresses or hostname (or pattern of).
It makes sure any kind of string can be matched (by strict
equality) while still keeping the old behaviour.
Opaque strings can only contains: alphanumeric characters, -
_ and @.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
I also took the opportunity to clean up and simplify
the interface to the parts of the parser that interact
with this. Mostly it was method renames.
Signed-off-by: Luke Kanies <luke@madstop.com>
The goal of this commit is to fix ordering issues
that could result when the filebuckets are added
to the catalog after the resources that use them.
This condition showed up somewhat arbitrarily.
Signed-off-by: Luke Kanies <luke@madstop.com>
Comments and multi-line comments produces no token per-se during
lexing, so the lexer loops to find another token.
The issue was that we were not skipping whitespace after finding
such non-token.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
We were previously throwing exceptions.
This also ports all of the tests for variable lookup
over to rspec.
Signed-off-by: Luke Kanies <luke@madstop.com>
Mongrel puppet code uses REMOTE_ADDR to set the ip address which will
be use to authenticate the client access.
Since mongrel is always used in a proxy mode with Puppet, REMOTE_ADDR
is always the address of the proxy (usually 127.0.0.1), which defeats
the purpose.
With this changeset, the mongrel code now uses the X-Forwarded-For
HTTP header value if it is passed over the REMOTE_ADDR.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
There's more caching to add, but this simplifies
the interface to the list of paths and then caches
that list so we aren't constantly searching the
filesystem.
Signed-off-by: Luke Kanies <luke@madstop.com>
Previously, when you created a module you had to specify
the path. Now Module instances can use the module path
to look up their paths, and there are methods for determining
whether the module is present (if the path is present).
Also cleaned up the methods for figuring out what's in
the module (plugins, etc.).
Signed-off-by: Luke Kanies <luke@madstop.com>
All the snippets tests were failing because some parser and scope
tests activated storeconfigs without reseting the state.
Activating storeconfigs is not undoable at the moment by just
setting storeconfig=false as some terminus are changed.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This patch moves Type to use Puppet::Util::Tagging as the other
part of Puppet. This brings uniformity and consistency in the
way the tags are used and/or compared to each other.
Type was storing tags in Symbol format, which produced #2207.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
The idea is to have allow/deny authorization directives
that are dynamic: their evaluation is deferred until
we perform the authorization checking in allowed?.
This is done to allow replacing backreferences in allow/deny
directives by parameters of the match that selected this right.
For instance, it is possible to:
allow $1.$2
And using Right::interpolate() with the result of a regex match
using 2 captures, will evaluate $1.$2 to those captures.
For instance, if we captured [host, reductivelabs.com], then the
allow directive is replaced by:
allow host.reductivelabs.com
It is then safe to call allowed?, after which we can reset the
interpolation.
This interpolation is thread-safe.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
authconfig regex support
They don't work with the modified code, and we
rely almost entirely on manual integration testing
for this stuff anyway.
We definitely need to add tests where we can, but
these tests are totally useless.
Signed-off-by: Luke Kanies <luke@madstop.com>
When using recurse and a source, if the client side has many files
it can take a lot of CPU/memory to checksum the whole client
hierarchy. The idea is that it is not necessary to recurse on the
client side if all we want is to manage the files that are sourced
from the server.
This changeset adds the "remote" recurse value which prevents recursing
on the client side when a source is present. Since it also is necessary
to limit the remote side recursion a new File{} parameter has been
added called "recurselimit".
Moreover, the Filetset API is changing to allow the new recurselimit
parameter, and passing the recursion depth limit in the recurse parameter
as an integer is now deprecated and not supported anymore.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
These tests once again largely were caused
by /usr/sbin not being in the path and by
~ not being writable.
The only tests still failing are Rails tests,
and my guess is that they're all failing because
of the recent work by Brice. They should probably
just be removed.
Signed-off-by: Luke Kanies <luke@madstop.com>
Most of these were just obsolete tests that have
been sitting around and broke with recent internal
changes.
Signed-off-by: Luke Kanies <luke@madstop.com>
This replaces the short-lived EventManager class, all of
the service- and timer-related code in puppet.rb, and moves
code from agent.rb, server.rb, and other places into one
class responsible for starting, stopping, pids, and more.
The Daemon module is no longer in existence, so it's been
removed from the classes that were using it.
Signed-off-by: Luke Kanies <luke@madstop.com>
Once I went to add runinterval support to the Agent class,
I realized it's really two classes: One that handles starting,
stopping, running, et al (still called Agent), and one that
handles downloading the catalog, running it, etc. (now
called Configurer).
This commit includes some additional code, but 95% of it is just moving code around.
Signed-off-by: Luke Kanies <luke@madstop.com>
Made minor changes, including removing the parent class.
The functionality hasn't changed yet -- that comes in later patches --
but all but a couple of the older tests pass.
Signed-off-by: Luke Kanies <luke@madstop.com>
Adds an rspec test which demonstrates #1560 and a custom 'process'
method for the aliases provider to fix it.
The default processing uses split() to break the line into records on
the separator, which breaks if records can contain the separator. The
custom method I've added uses a 'limited' split() to break the line on
the first separator only.
This commit fixes#1560
Signed-off-by: Paul Lathrop <paul@tertiusfamily.net>
Most of these are straightforward changes to the tests,
but a couple required small refactorings (e.g., References
can now be created with Puppet::Type instances, and they
know how to extract the type/title from them).
Signed-off-by: Luke Kanies <luke@madstop.com>
This was a complicated project because TransObject had made its
way into too many classes. The usage by Util::Settings
was particularly nefarious. Refactoring and fixing this
exposed some other issues.
The main complication, though, was the extent to which the
Puppet::Type class depended on TransObject.
Signed-off-by: Luke Kanies <luke@madstop.com>
This completely reorganizes how RAL resources are
initialized, and in the process I was able to remove a lot
of code (I removed other apparently obsolete code at
the same time).
Signed-off-by: Luke Kanies <luke@madstop.com>
It is now more likely to fail if there's a problem,
since the yaml should be corrupt, causing a yaml failure.
Also removing the equivalent test from the Storage module.
Signed-off-by: Luke Kanies <luke@madstop.com>
Conflicts:
bin/puppetca
lib/puppet/type/group.rb
lib/puppet/type/tidy.rb
lib/puppet/util/settings.rb
Also edited the following files so tests will pass:
lib/puppet/type/component.rb
spec/unit/ssl/certificate_request.rb
spec/unit/type/computer.rb
spec/unit/type/mcx.rb
spec/unit/type/resources.rb
spec/unit/util/settings.rb
spec/unit/util/storage.rb
test/ral/type/zone.rb
The main problem was that the directory_services user provider was returning
a string for its uid instead of an integer. I also began a 'resources'
spec file.
Signed-off-by: Luke Kanies <luke@madstop.com>
The aim is to let --parseonly succeeds even if the function
is not (yet) present. This is usefull in commit-hooks and
for the inline documentation generation system.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
This was necessary because of how I changed file recursion.
The type works much more intelligently now -- files to
be removed have a file resource generated for them, and that
resource handles deletion.
Also fixes#1717; neither age nor size is required now.
Signed-off-by: Luke Kanies <luke@madstop.com>
This makes a lot of sense because source was always
more of a metaparameter than a property -- it affected
the 'should' values of other properties, but it shouldn't
have done any other work.
It will hopefully make everything else much cleaner.
This is such a large commit mostly because of the need
to fix a lot of tests.
Signed-off-by: Luke Kanies <luke@madstop.com>
This is a significant refactor of some very murky code, and it's
all much cleaner and more readable now. All of the 'newvalue' methods
and any value-related code is in a ValueCollection class.
This puts us in a good position to refactor the Property and Parameter
classes more completely.
Signed-off-by: Luke Kanies <luke@madstop.com>
Removing obsolete handleignore method
Removing obsolete FileSource class
Removing a now-obsolete test/unit test
Removing a now-obsolete recursive filebucket test
Signed-off-by: Luke Kanies <luke@madstop.com>
This is an unfortunately messy commit; I should have tried harder
to separate the different tasks into different commits, but it's
not worth going back and doing now.
This is the first commit in the long road of fixing the existing file
tests, and in the process refactoring and better testing the code.
The refactoring in this commit is mostly around the 'stat' instance
variable that tests whether the file exists (and provides its metadata
if it does) and the 'insync?' method in the source property.
Signed-off-by: Luke Kanies <luke@madstop.com>
The following manifest wasn't working:
class one {
notice('class one')
}
class one {
notice('second class one')
}
include one
It all boiled down to class code not being arrays.
Encapsulating code in ASTArray when needed is enough to append code,
because of the property of ASTArray to evaluate all their members in
turn.
Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
since that method is deprecated.
Conflicts:
CHANGELOG
bin/puppetca
lib/puppet/file_serving/fileset.rb
lib/puppet/network/xmlrpc/client.rb
lib/puppet/type/file/selcontext.rb
spec/unit/file_serving/metadata.rb
spec/unit/type/file.rb
This changesets allow empty if or else branches:
if true {
} else {
}
It works by emitting on the parser stack an AST node that doesn't
do anything (a no-op). This allows the less intrusive code
as no part of the if evaluation code has been touched.
The real problem was that the 'gid' and 'uid' methods didn't handle
the case where 'get_posix_field' didn't return a value, and
the subsequent 'get_posix_field' calls couldn't handle that.
This commit moves the tests for Posix to spec, and fixes the
specific bug.
Signed-off-by: Luke Kanies <luke@madstop.com>
The expressions can be used in if 'test' and in the
right side of assignements.
The expressions can contain any number of sub-expressions
combined by either arithmetic operators, comparison operators,
or boolean operators.
Random Usage Examples:
$result = ((( $two + 2) / $one) + 4 * 5.45) - (6 << 7) + (0x800 + -9)
or
if ($a < 10) and ($a + 10 != 200) {
...
}
variables that the puppet template should otherwise see. Specific changes:
* Added testing of the Scope#to_hash method, which returns a hash of name and
value pairs in a scope or, optionally, in the scope and enclosing scopes.
* Use member variables rather than methods in the function tests.
* Fix up tests that fail once we move over to instance variables rather than
methods: Puppet can no longer detect an undefined variable reference, so we
end up failing any test that expected to get a parser error.
* Several tests have manual checks introduced to simulate an end user manually
writing the checks that used to be automatic, and others drop the validation
that parsing fails when a variable is not in scope.
* Added tests for legacy variable lookup and that the shadowing of local
variables by Kernel methods is still in effect.
Signed-off-by: Daniel Pittman <daniel@rimspace.net>
(cherry picked from commit 5c5f315cceadc52203e53883b77bc01c1d7a2e7f)
Darwin has a case-insensitive FS, so the test was failing
because it was specifically testing case sensitivity.
Signed-off-by: Luke Kanies <luke@madstop.com>
with 'yaml' still being the default but 'marshal' being an option.
This is because testing has shown drastic performance differences
between the two, with up to 70% of compile time being spent
in YAML code. Use the 'catalog_format' setting to choose your format,
and the setting must be set on the client.
Signed-off-by: Luke Kanies <luke@madstop.com>
I.e., you can't do confine :operatingsystem => %w{a b} and then
confine :operatingsystem => %w{b c}; you'd need to do them in one command.
This now-obsolete behaviour does not seem to be used anywhere.
The fix for #1431 is actually just removing the tests that exposed
this change; the change happened when I refactored how confines work.
Signed-off-by: Luke Kanies <luke@madstop.com>
Here were the main changes necessary:
* Fixed the class loader so it only loads mongrel if it's available.
* Fixed the test runner to skip example groups contained in non-runnable
example groups.
* Fixed the Mongrel tests to use quoted class names instead of constants,
since the constants themselves would be absent.
Signed-off-by: Luke Kanies <luke@madstop.com>
The class had a 'transaction' accessor that was assigned
but never used, and it is simple enough that it needed
direct arguments rather than named arguments.
The rest of the code is changing the other classes that use
Events.
Signed-off-by: Luke Kanies <luke@madstop.com>
Also added the fixes to make the certhandler tests pass
even when certs exist; I'll deal with the conflict later.
Conflicts:
CHANGELOG
bin/puppetd
lib/puppet/network/http/handler.rb
lib/puppet/network/http/mongrel/rest.rb
spec/integration/indirector/rest.rb
spec/integration/network/server/mongrel.rb
spec/integration/network/server/webrick.rb
spec/unit/network/http/webrick.rb
The Master handler previously provided the support for the :node_name
setting, and that functionality has now been moved into the Node
class. At the same time, the names to search through have been
changed somewhat: Previously, the certificate name and the
hostname were both used for searching, but now, the cert name
is always searched first (unless node_name == facter), but only
the Facter hostname, domain, and fqdn are used otherwise. We no
longer split the cert name, only the hostname/domain/fqdn.
In the general case, this provides no behaviour change, because
people's hostname is the same as their certname. This only
results in a change in behaviour if you specify a certificate
name that is a normal node name, and you want to look that node
up by something other than the full name in the certificate.
Signed-off-by: Luke Kanies <luke@madstop.com>
Previously, we prefetched then generated, which caused generated
resources that needed prefetching not to work. This just reorders
the calls, so generated resources now get prefetched.
I created a Confiner module for the Provider class methods,
and then I enhanced the interface between it and the Confine
class to make sure binary paths are searched for fresh each time.
This fixes#1197, which was a result of binary paths being
searched for at startup, rather than at execution.
The HttpPool module now removes its cached ssl_host instance
when clearing its cache. This is really only useful
for testing, but it correctly causes the pool to use
new certificates when they're available.
This class provides all of the semantics from puppetca,
and appears to entirely duplicate the behaviour of the existing
executable, with basically all of the code in a library
file, instead of the executable.
As such, I've deleted the test for the executable. We should have
one, but it's not nearly as important.
the [] accessor method (they previously threw an exception when
this method was used with properties). This shouldn't have any
affect functionally; it just makes the method equivalent to 'should'
for properties, but it works for all attribute types now.
compile node configurations, rather than using the Configuration
handler, which was never used directly. I removed the Configuration
handler as a result.
Modified the 'master' handler (responsible for sending configurations
to clients) to always return Time.now as its compile date, so
configurations will always get recompiled.
Luke Kanies
Luke Kanies
Jesse Wolfe
Rein Henrichs
Brice Figureau
Markus Roberts
James Turnbull
Till Maas
Nigel Kersten
Ian Taylor
Paul Lathrop
Francois Deppierraz
Andrew Shafer
Daniel Pittman
Francois Deppierraz