Listing installed plugins is a sequential operation that iterates on
each potential plugin binary under PACKER_PLUGIN_PATH.
Each potential candidate may be executed in order to get its
description, and to match that what is reported by the plugin binary
matches what is effectively installed on the filesystem.
This operation may be causing lots of I/O syncs, and if done
sequentially on a rather large number of installed plugins will take
time.
In order to make this faster, we paralellise listing installations, so
we run those commands in parallel in order to minimise idle waiting time
on the main process.
In testing environments we may have concurrent accesses/writes to the
PluginsDetailsStorage global map, as some tests run commands in parallel
from the same process/context, therefore we may have multiple processes
discovering installed plugins at once.
This makes tests crash, and while this does not happen in normal usage
of Packer, we should prevent this by making accesses/stores thread-safe
for this global object.
Since the SDK was updated and now accepts raw version strings in
addition to also supporting metadata officially, we can start using
those functions in our version package as well.
The SDK's Ui interface has had some additions recently for supporting
format-strings and their arguments as paramereters to Say, Ask and
Error.
These changes to the interface means that all the implementations of
that interface need to match the definition, so in this case the
implementations we had in packer/ui.go need to be updated to support
those functions.
Since we have both version/version.go and version/VERSION to specify
version strings, both are a bit redundant.
As version/VERSION is supposed to be the source of truth now, we are
using it to derive the version informaiton we used to rely on in Packer
and its subcommands.
Note: doing this prevents us from changing the version/prerelease
through ldflags though as we derive Version/VersionPrerelease from the
rawVersion variable.
When building the container images for Packer (light and full), we
should only publish them in the version being built is a release, and
not a pre-release.
Changing this means that both light and full latest will always be only
a final release at any time.
When introduced back in January, the version variable extracted from the
binary name had a typo in its name, and was named `protocolVerionStr'
instead of `protocolVersionStr'.
This commit was already merged into main, so it's too late to fix at
introduction site, but we can fix it today as a separate commit ontop
the stack.
If a plugin is installed manually, its version number could be valid but
non-canonical (ex: 1.2.3 vs 01.002.0003).
Since these two versions refer to the same version, but the looks are
different, this may become ambiguous which version should be loaded.
To avoid such a situation, we reject explicitely non-canonical version
numbers in plugins, but only in path, we're aware that because of
metadata, the version from `describe' may already differ from the file
name.
The former way mock plugin sets were created meant that no API version
was set, and since it's private in the SDK, it cannot be set outside of
the package itself.
Fortunately, there is a NewSet function we can call, which initialises a
set properly so we can fill-in the information later.
However, because all the set maps were created in a `var` section, we
cannot create the set with `NewSet`, and then fill the information in
(outside of if there was a fluent interface, but this isn't the case
here).
We therefore opted to keep the variables defined and accessible
globally, but gone through a `init` function to initialise their values
for tests.
As with versions, API versions are useful possibly for ordering plugin
installations in order for Packer to choose which plugin to load.
This could be unnecessary as API versions are stable, and only in dev
plugins this could be a problem normally (there shouldn't be two same
releases of a plugin), but this cements API version in ordering plugins
so we avoid surprises later down the line.
As with the version of the plugin, the API version should also match
between the path and the self-reported API version from the describe
command.
This was not checked before, so users could masquerade a plugin's use of
an API version that may be incompatible with Packer.
To avoid this problem, we make sure both versions are the same, so that
they work as expected.
When installing plugins with the `packer plugins install --path'
command, the metadata is now scrubbed from the file installed locally.
This is as a protection against collisions in the versions, as metadata
is meaningless for version comparison, so if two versions of the same
plugin are installed, the precedence order between them is undefined.
Therefore to avoid such collisions, we remove the metadata from the file
name, that way if two successive versions of a plugin include metadata
in the version, they won't coexist, and the last installed will be the
only installed version locally.
When installing a plugin from a local binary, Packer builds the name of
the plugin from the results of the `describe' command.
Depending on how the plugin is built, the version reported may or may
not contain a leading `v', which was not taken into account beforehand
and the leading `v' was always injected in the path.
This caused plugins that report a leading `v' in their version to
be installed with two v's in their path, making them impossible to load.
Therefore to fix this issue, we count on the version library to print
out a version without the leading v, and we inject that in the resulting
path.
When Packer discovers binary a bunch of checks are performed, which
ultimately end with a checksum match check.
This however should be the very first thing we do, even before
attempting to run `describe' on the plugin binary we're discovering.
So this commit moves this checksum match to the top of the discovery
process for binaries.
When discovering the installed plugins locally, we perform a couple of
checks on the version, namely that it is valid, if it is a prerelease,
it needs to be a dev, and that the self-reported version matches the one
hinted at through the name of the binary.
This was done through regexes, but those were a wee bit simple when
dealing with versions that have metadata. Those binaries would be
completely ignored by Packer, and never loaded, although they are a
valid use case.
The version library we already used supports those however, and
comparisons are more reliable with them.
So, in order to simplify our code, and make it more reliable, we're
exclusively using this library to perform parsing and comparisons of
versions during the discovery phase.
Since v1.11.0 should fix the version ordering to work with semver
comparison instead of reliying on glob (lexicographic), we should call
out the problem in the CHANGELOG.
The test directory contains artifacts from the project's past which are
never used nowadays. All those remains are therefore removed with this
commit, and we can replace that directory's contents with more
up-to-date contents.
The packer plugins remove command allows users to delete plugins
installed locally.
Previous versions of the command only allowed for the plugins to be
removed using the source for a plugin, and the versions to remove,
optionally.
This commit adds the capability for the plugins to be removed using
their local path, in addition to the regular source+version method, that
way we are able to pipe the results of `packer plugins installed' into
the plugins remove command for quick plugin removal.
The plugin and plugins command had a name that was close, and while
plugin is not supposed to be directly called by Packer users, this could
happen by accident while trying to execute packer plugins subcommands,
and when it does, the error messages are far from explicit, so unless
they understand what Packer is doing here, they'll likely be lost.
To reduce the risk of confusion, we rename the command to run packer
embedded components as execute.
The fmt command reformats HCL2 templates, provided it can parse the file
and reformat its contents according to the standards set by the HCL
library's formatters.
However, if the file is malformed for some reason, the command will fail
with a parse error, but while the parse error message is shown, the
actual errors in the template(s) are not forwarded, making it hard for
users to understand what went wrong with the contents of the file
they're trying to format.
In order to be more helpful with those errors, we now forward those
parsing errors to the UI.
This change addresses vulnerabilities reported by govulncheck
```
Vulnerability #1: GO-2024-2610
Errors returned from JSON marshaling may break template escaping in
html/template
More info: https://pkg.go.dev/vuln/GO-2024-2610
Standard library
Found in: html/[email protected]
Fixed in: html/[email protected].1
Example traces found:
#1: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do, which eventually calls template.Template.Execute
#2: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do, which eventually calls template.Template.ExecuteTemplate
Vulnerability #2: GO-2024-2600
Incorrect forwarding of sensitive headers and cookies on HTTP redirect in
net/http
More info: https://pkg.go.dev/vuln/GO-2024-2600
Standard library
Found in: net/[email protected]
Fixed in: net/[email protected].1
Example traces found:
#1: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do
#2: hcl2template/function/aws_secretetkey.go:38:37: function.init calls template.GetAWSSecret, which eventually calls http.Client.Post
#3: hcl2template/function/aws_secretetkey.go:38:37: function.init calls template.GetAWSSecret, which eventually calls http.Client.PostForm
Vulnerability #3: GO-2024-2599
Memory exhaustion in multipart form parsing in net/textproto and net/http
More info: https://pkg.go.dev/vuln/GO-2024-2599
Standard library
Found in: net/[email protected]
Fixed in: net/[email protected].1
Example traces found:
#1: internal/hcp/api/service_build.go:81:48: api.Client.UpdateBuild calls packer_service.Client.PackerServiceUpdateBuild, which eventually calls textproto.Reader.ReadLine
#2: datasource/http/data.go:140:26: http.Datasource.Execute calls io.ReadAll, which eventually calls textproto.Reader.ReadMIMEHeader
Vulnerability #4: GO-2024-2598
Verify panics on certificates with an unknown public key algorithm in
crypto/x509
More info: https://pkg.go.dev/vuln/GO-2024-2598
Standard library
Found in: crypto/[email protected]
Fixed in: crypto/[email protected].1
Example traces found:
#1: datasource/http/data.go:140:26: http.Datasource.Execute calls io.ReadAll, which eventually calls x509.Certificate.Verify
```
Lucas Bajolet
Wilken Rivera
Wilken Rivera
Devashish
dependabot[bot]
pavedroad