To make sure we do scrub the metadata in the plugin name when installing
it from a local binary, we add a test that does that installation with
both alternatives: 1.0.0-dev and 1.0.0-dev+metadata, which should result
in only one alternative being installed (the last one that succeeded).
The installation with a metadata part in the version for a plugin had
one test that relied on the plugin directories being populated with
packer plugins install --path.
This could change in the future, while the command should remain
functional, so we explicitely call it in the test instead of through the
function that creates/populates a temp plugin dir.
When building a pipeline to count the number of lines returned by
Packer, it can be a bit cumbersome to have to chain the calls to
MkPipeCheck to do that check, so we add one convenience function for the
simplest case: counting the number of lines on stdout, without any kind
of filtering.
When manually installing a plugin to the plugin directory, we compute a
SHA256SUM file from the plugin binary, and install it alongside it so we
can test the loading process for Packer.
In the introduction of the function, we added a check that if we were
running on Windows, we'd remove the extension of the sumfile's name
before writing it.
This is actually not necessary (and breaks the loading logic) as Packer
looks for the name of the plugin with extension, followed by
_SHA256SUM in order to compare the effective digest of the file to the
one written to this file.
Since this prevents the tests that use this function from succeeding in
a Windows environment, we remove this extra step.
Windows relies on the `TMP` (or alternatives) being set in the
environment in order to be able to create temporary directories and
files.
If this is not set, the `os.TempDir` function defaults on the windows
installation root directory (typically C:\Windows), leading to
permission errors when running Packer in the context of a test, as we're
installing plugins in a temporary directory.
To avoid this problem, we get the current setting from the test's
invocation environment, and forward it to the subcommand we execute for
our tests.
Since on Windows extensions are mandatory in order to have something
executable, we compile Packer with a `.exe` suffix during tests, so we
can use the executable afterwards to run tests with.
As we're introducing a --ignore-prerelease-plugins flag to both the
validate and build subcommands, we need to make sure they work as we
expect it to, so we add a test case for that.
Plugins with metadata information in their file name (i.e.
v1.0.0+metadata) should be ignored by Packer as they could introduce
ambiguity since the metadata is free-form, so we add that test to make
sure Packer behaves coherently.
If a plugin is installed with a non-canonical version in its name (e.g.
01.01.01), Packer rejects it with a message to that effect in stderr, so
we add a test for this use-case.
Installing a plugin manually to a directory is something needed for some
tests, especially those not relying on packer commands to install
plugins as they reject/correct the path/version.
Therefore this function is introduced so we have an easy way to install
a binary as a plugin somewhere on the provided plugin directory.
The ExpectedInstalledName function used to compute the expected name of
a plugin binary for a given version, based on the invoker's environment,
used to cleanup the version string passed in parameter of the function,
which could be problematic.
Besides the logic applied would produce some invalid binary names as the
prerelease plugin would not have a `-` separator, so the resulting
plugin would be ignored, and we couldn't test metadata rejection with
this logic.
This commit therefore changes how the function works: the version string
is still parsed to account for manipulation errors, but the string is
left as-is for the final binary name.
If for some reason we only want to run a test on either stream without
doing some manipulation beforehand, we can run a PipeChecker, however
these would error if no pipe gadget was defined, preventing this
use-case.
Instead of errorring then, this commit just ignores if no pipe is
present, as none is required for the test to run.
Since the command object is created using the TestSuite as argument, we
can keep a reference to the test suite's scoped T() instance for the
command's lifecycle, and reuse it later during `Assert`, instead of
needing to pass it as argument when invoking the function.
When troubleshooting a pipeline for a test, it can be useful to print
the input out without necessarily preventing the pipeline to work.
The Tee gadget is exactly made for this purpose, the input of the Tee is
printed out through `t.Log`, and the input is forwarded to the next step
in the pipeline.
When the PACKER_PLUGIN_PATH envvar is defined in the environment, Packer
uses it as the source of truth for the directories in which to look for
plugins to load.
Previously, we used to support multiple directories separated by the
OS-specific path separator (i.e. : on UNIX, ; on Windows).
Since this changed, and Packer returns an error to the user, we make
sure that this is well-documented and tested through this extra test.
The IntCompare checker converts the input string into an int through
strconv.Atoi, which fails when the string isn't valid base 10. This
definition of "valid" also excludes whitespace, which occurs often
through piping gadgets, but shouldn't be a reason to fail that check, so
we trim the whitespace from the input string.
When running the core acceptance tests we compile a temporary Packer
binary to run the tests with.
However in some cases we may want to run a test with another,
pre-compiled version of Packer.
For this use-case, we introduce an environment variable to change the
executable we're running tests with.
When the Run command was introduced, despite the path to the temporary
packer binary being stored in the command, we were invoking Packer
through "packer", which would resolve whatever is in our PATH that
resolves first.
This was not the intention, so we fix this problem with this commit.
When a test has to run in a working directory that is not the current
one we're running tests from, this may imply creating a temporary
directory, moving files into it, and then running the test from this
directory.
This can be a bit verbose to write all this code, so we abstract the
easy case through the TempWorkdir function, which works similarly to the
MakePluginDir function, by creating a temp dir, populating it with the
requested files, and returning a function to clean it up if needed.
For some tests we may need Packer to run in another directory than the
one we're invoking the tests from, so we add a new function to the
packerCommand structure to change that.
The name and semantics were a bit unclear with how they were previously
named, so this commit changes the name of those functions so it's
clearer they are expecting something, and what they're expecting.
As we've introduced pipelines, we can use those to compose a version of
grep that doesn't have specific logic.
Besides, this refactor allows us to expose grep as a function with
variadic options, so this makes it more concise and clear to assert an
input with Grep.
By default PackerCommands are run with PACKER_LOG=1.
If for any reason we don't want that, we can remove it from the
environment so we only see the user-facing logs.
As a common use case in console-oriented pipelines, we check that a
specific command returned a certain number of lines.
With the combination of LineCount and Compare, we can do exactly this.
When running a pipeline on a command's output, a simple check is making
sure the pipeline returned something empty or not.
This is the goal of those two implementations, basically either the
input is empty as expected, or it errors, and the reverse.
When writing tests, one may need to write a one-off checker for a
packer command that ran, without having to completely implement the
Checker interface.
This commit introduces a generic CustomChecker implementation (i.e. a
function) that can be one-off implemented by developers if their test
doesn't fit the existing gadgets, and the need is not generic/reusable
enough to justify introducing a new gadget for other users.
The ExpectedInstalledName function returns the expected full name of a
plugin binary after it's installed.
This is used for tests that need to copy the binary to some place before
they can run commands and ensure the logic for managing plugins conforms
to the docs/specs.
Since sometimes we want to check for matches, and sometimes we want to
check for a lack of match, we add one more option for the Grep gadget:
inverse.
This essentially replicates `grep -v`, and will succeed only if the
regex provided did NOT match on the requested streams.
This commit adds a few scenarios of plugin installations to the test
suite, in order to document and ensure we behave appropriately when
installing pre-releases/metadata.
When building the temporary plugin directory for a test, we didn't check
that the LoadPluginVersion call succeeded and returned a path, which
may cause errors down the line when attempting to install the plugin.
To avoid this problem, we do the check at that time, and immediately
fail if a plugin isn't found.
MakePluginDir used to only load plugins that were precompiled at the
start of the tests, but now when invoked with any list of plugins, this
will attempt to compile plugins one-by-one, so we don't need to modify
the tests in several places when running tests.
There's still value in compiling the plugins in advance though: as they
run in parallel, they all get compiled at once, so we shave off a few
seconds from the test run.
Lucas Bajolet
Wilken Rivera