The source parsing logic was heavily directed towards Github compatible
source URIs, however if we want to support more cases, we need to make
sure we are able to specify those URIs, and to load plugins installed
from those sources.
Right now, since the getters available are only github.com, we will not
support remotely instlling plugins from sources other than github.com,
with the same set of constraints as before. However, we do support now
installing from a local plugin binary to any kind of source, and we
support loading them, including if a template wants this plugin
installed locally with version constraints.
The SDK's Ui interface has had some additions recently for supporting
format-strings and their arguments as paramereters to Say, Ask and
Error.
These changes to the interface means that all the implementations of
that interface need to match the definition, so in this case the
implementations we had in packer/ui.go need to be updated to support
those functions.
When introduced back in January, the version variable extracted from the
binary name had a typo in its name, and was named `protocolVerionStr'
instead of `protocolVersionStr'.
This commit was already merged into main, so it's too late to fix at
introduction site, but we can fix it today as a separate commit ontop
the stack.
If a plugin is installed manually, its version number could be valid but
non-canonical (ex: 1.2.3 vs 01.002.0003).
Since these two versions refer to the same version, but the looks are
different, this may become ambiguous which version should be loaded.
To avoid such a situation, we reject explicitely non-canonical version
numbers in plugins, but only in path, we're aware that because of
metadata, the version from `describe' may already differ from the file
name.
The former way mock plugin sets were created meant that no API version
was set, and since it's private in the SDK, it cannot be set outside of
the package itself.
Fortunately, there is a NewSet function we can call, which initialises a
set properly so we can fill-in the information later.
However, because all the set maps were created in a `var` section, we
cannot create the set with `NewSet`, and then fill the information in
(outside of if there was a fluent interface, but this isn't the case
here).
We therefore opted to keep the variables defined and accessible
globally, but gone through a `init` function to initialise their values
for tests.
As with versions, API versions are useful possibly for ordering plugin
installations in order for Packer to choose which plugin to load.
This could be unnecessary as API versions are stable, and only in dev
plugins this could be a problem normally (there shouldn't be two same
releases of a plugin), but this cements API version in ordering plugins
so we avoid surprises later down the line.
As with the version of the plugin, the API version should also match
between the path and the self-reported API version from the describe
command.
This was not checked before, so users could masquerade a plugin's use of
an API version that may be incompatible with Packer.
To avoid this problem, we make sure both versions are the same, so that
they work as expected.
When Packer discovers binary a bunch of checks are performed, which
ultimately end with a checksum match check.
This however should be the very first thing we do, even before
attempting to run `describe' on the plugin binary we're discovering.
So this commit moves this checksum match to the top of the discovery
process for binaries.
When discovering the installed plugins locally, we perform a couple of
checks on the version, namely that it is valid, if it is a prerelease,
it needs to be a dev, and that the self-reported version matches the one
hinted at through the name of the binary.
This was done through regexes, but those were a wee bit simple when
dealing with versions that have metadata. Those binaries would be
completely ignored by Packer, and never loaded, although they are a
valid use case.
The version library we already used supports those however, and
comparisons are more reliable with them.
So, in order to simplify our code, and make it more reliable, we're
exclusively using this library to perform parsing and comparisons of
versions during the discovery phase.
While migrating to a unified approach for loading plugins, the API
major and minor versions were not added to the constraints for
discovering plugins from the environment, leading to Packer potentially
considering plugins that are not compatible with itself.
This should not be possible, but was due to this omission, which we fix
with this commit.
Since we now support loading pre-releases, we also want Packer to be
able to ignore them by user demand, so we put in place the
infrastructure to modulate this.
When Packer is loaded, we used to perform plugin discovery.
This was done for every call to Packer, including when it is executed as
a plugin, arguably against what the comments document.
Doing this as early in the loading process makes it harder to change
this behaviour, as we'd need to introduce flags aside from the rest, and
handle them manually, which is not optimal.
Therefore, we change this: now when Packer starts executing, it will not
attempt to discover installed plugins anymore, and instead will only try
to load them when a configuration has been parsed, and is being used to
perform actions (typically build/validate).
When a pre-release version of a plugin is locally installed, it may or
may not be loaded depending on the constraints expressed in the template
being executed.
If the template contains constraints for loading the plugin, it would be
ignored, while if that wasn't present, it would be loaded.
This is inconsistent, and deserves to be addressed, which is what this
commit does.
With this change, plugin pre-releases are now loaded, provided the
version reported matches the constraints, independently from its
pre-release suffix `-dev'.
Also, if a release with the same version is also installed alongside the
pre-release version of a plugin, it will have precedence over the
pre-release.
When Packer orders the plugins by their version, we'd assumed it was
ordered from highest to lowest.
However, this was the case because our implementation of `Less' was
actually returning `>=', which is not what it was supposed to do.
This commit therefore fixes the implementation of `Less' to do what it
is documented to do, and ensures the behaviour is correct through
additional testing.
Changing this requires some changes to the loading process as well
because of the aforementioned assumption with regards to ordering.
When a plugin is loaded from Packer, we now check that the version it
reports matches what the name implies. In case there's a mismatch, we
log, and reject the binary.
Right now we had two paths for discovering installed plugins, i.e.
through plugin-getter's `ListInstallations' function, or through the
`Discover' call, which relied on a glob to list the installations.
This was required since we allowed plugins to be installed in multiple
locations, and with different constraints.
Now that we force a certain convention, we can consolidate the logic
into ListInstallations, and rely on that logic in `Discover' to load a
plugin into the PluginConfig for the current Packer run.
The plugin installation list should be sorted according to a name first,
then version second basis.
Right now, we only rely on the glob to add plugin installs to this list,
making the order unreliable since the lexicographical order is not the
order in which we want to see the same plugin ordered (e.g. v1.0.9 >
1.0.10).
To fix this, we implement a logic for sorting a list of installations
that does what's described above with more accuracy.
Since the plugin directory should now be unique instead of a list, we
can use it directly from the receiver's structure instead of as a
parameter to the function.
Since we'll only look in the plugin directory, and not from multiple
sources now, for installing/listing plugins, we can simplify the
structures which used to accept multiple directories so they only accept
one.
Since we'll be only accepting plugins installed locally along with a
shasum, and those that adopt the convention we introduced with
required_plugins, we remove support for plugins that only have a
packer-plugin-.* type name.
Since we're removing the alternative plugin installation directories in
favour of only supporting installing them in the PACKER_PLUGIN_PATH
directory, we only return one directory when getting the known plugin
directories.
Single-component plugins are a relic from the past that has been
deprecated from version 1.7.0 and onwards.
Since we're revisiting how plugins are installed/loaded, and the changes
will be incompatible with those, we remove them in preparation of this
work.
The plugins installed command list installed plugins, and prints out all
their paths and versions.
The results are listed in a list of unique versions, in ascending
order.
When listing plugins outside of a specific identifier however, because
of the insertion in the list works, we only consider the version number,
and ignore all subsequent insertions with the same version.
This causes a problem when we have multiple plugins installed with the
same version, as only the first one that is discovered (typically the
one with lexicographical precedence) gets inserted, and the others are
ignored.
To support such a use case, we change the insertion routine to not only
support versions, but also paths when finding a spot to insert the
installation into.
The --force option for packer init and packer plugins install enforces
installation of a plugin, even if it is already locally installed.
This will become useful if for some reason a pre-existing plugin
binary/version is already installed, and we want to overwrite it.
When a user invokes packer for a build or validation, the template being
processed needs components to be present for Packer to process it
without error.
If the component cannot be found from the plugins loaded (or from the
components bundled with Packer), Packer errors, and the command fails.
This is expected, but the error message does not suggest anything to fix
the error, potantially leaving users confused at the problem.
This commit suggests either a replacement (in case of a typo), or points
to the web documentation for Packer, specifically the integrations, so
they can look for the plugin they're missing, and install it, so
subsequent invocations of Packer work.
When invoking `packer plugins install' to install a plugin, or `packer
init', the checksum file would be installed with 0555 permissions.
This led in turn to further attempts at installing the plugin will
succeed, but the checksum file would not be updated, as it was marked
non-writable by the owner of the file, leading potentially to a
situation where the plugin binary and the checksum would be out-of-sync,
but could not be updated unless the user changed it.
To avoid such a problem, we write the checksum file with 0644
permissions, so the owner can read/write, while the other users can only
read it.
Invoking Packer with the CHECKPOINT_DISABLE env. variable the telemetry reporter is left uninitialized in order to disable telemetry reporting.
Any method calls on the nil reporter is expected to check if the reporter is active or in NOOP mode. This change fixes a crash when calling SetBundledUsage()
on a nil CheckpointTelemetry type that occurs when using a bundled plugin with CHECKPOINT_DISABLE=1.
Since this feature is no longer something we plan to activate later, as
it contradicts with our efforts to remove bundled plugins, and
encouraging users to move to either manually installing plugins, or
managing them through `packer init', we clean-up the code for this
feature.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
* Add test case for loading plugin in CWD
* Add test case to validate checksume files are ignored
* Update Discover to include CWD "." in PluginFolders if KnowPluginFolders is unset
When copying a plugin's checksum file (packer-plugin-*_SHA256SUM) installed by `packer plugins install` or `packer init`
into a separate directory the file may be copied with the executable bit turned out. If unchanged after the copy, Packer would
discover the checksum file as a possible plugin match and error when trying to execute describe on the plugin look a like. This change
adds a checksum file test to the plugin matching logic. If the discovered plugin name is a checksum it is excluded from the discovered plugin list.
Starting with Go 1.19 the loading of binaries from the current working directory was
deemed as a possible security problem. Thus the use of exec.Command or exec.LookPath no longer resolves
an executable within the current working directory. This change updates the discover logic to return absolute
paths for any discovered plugin, which is called directly when passed to exec.Command or exec.LookPath. By doing
this Packer is able to load a custom plugin sitting in the current working directory as it did in version prior to v1.9.2.
Since bundled plugins will be removed in an upcoming version of Packer,
this commit adds a new warning message whenever a template uses one such
plugin.
This warning has been implemented on build, validate, console and the
inspect subcommands.
In addition to warning about the upcoming change and potential issue
this will cause, this warning message proposes solutions to the user so
they know what they'll have to do in order not to rely on those bundled
plugins later.
Packer will try to discover installed plugins in all of the directories
defined by packer.KnowPluginFolders. In a previous release logic was
added to scan nested directories in order to load plugins installed by
`packer plugins install`. This change resulted in a nested directory
scan for each folder within the KnownPluginFolders slice.
This change reduces the nested directory scan to only the directories
where plugins would have been installed using `packer plugins install`
The template type for a builder, provisioner, post-processor or
datasource was not tracked with telemetry data.
This commit adds the infrastructure to track this, bumping the schema
for non-crashes to `beta/packer/6' at the same time.
Packer's telemetry package captures the field names from a configuration
on legacy JSON templates, but did not contain any code for handling HCL2
template configurations.
This commit adds a routine to extract the field names from a HCL2
configuration once flattened, and adds some glue code to propagate the
configurations to the telemetry structures.
This change fixes a regression introduced in Packer 1.8.6, where configurations custom builder names
, via the name attribute, would internally interpolate the proper build names, but would display the
name to STDOUT as uninterpolated values. The change updates the creation of a CoreBuild for legacy JSON templates
to use use the interpolated name as the final build type, which gets rendered to the screen.
Test Results Before Fix
```
~> go test ./... -run='TestCoreBuild_buildNameIntepolation' -v
--- FAIL: TestCoreBuild_buildTypeIntepolation (0.01s)
core_test.go:930: build type interpolation failed; expected "test.mybuild-RandomToken", got "test.{{user `build_name`}}"
core_test.go:930: build type interpolation failed; expected "test.build-vardata", got "test.{{user `build_name`}}"
core_test.go:930: build type interpolation failed; expected "test.build-12345", got "test.{{user `build_name`}}"
FAIL
```
Test Results After Fix
```
~> go test ./... -run='TestCoreBuild_buildNameIntepolation' -v
=== RUN TestCoreBuild_buildTypeIntepolation
--- PASS: TestCoreBuild_buildTypeIntepolation (0.01s)
PASS
```
Closes#12281
This commit irons out one of the pain points of the HCP rework by
introducing a HCPPublisher interface, implemented both by the JSON Core,
and the HCL2 PackerConfig, which keeps a map of the build names used by
Packer to the build names pushed on HCP.
This in turn lets us go back to the old implementation of the GetBuilds
function, which returns a list of (filtered) builds, and eventually an
error if something went wrong while processing.
Devashish
Lucas Bajolet
Wilken Rivera
guoguangwu
hashicorp-copywrite[bot]
Wilken Rivera