The packer plugins remove command allows users to delete plugins
installed locally.
Previous versions of the command only allowed for the plugins to be
removed using the source for a plugin, and the versions to remove,
optionally.
This commit adds the capability for the plugins to be removed using
their local path, in addition to the regular source+version method, that
way we are able to pipe the results of `packer plugins installed' into
the plugins remove command for quick plugin removal.
The plugin and plugins command had a name that was close, and while
plugin is not supposed to be directly called by Packer users, this could
happen by accident while trying to execute packer plugins subcommands,
and when it does, the error messages are far from explicit, so unless
they understand what Packer is doing here, they'll likely be lost.
To reduce the risk of confusion, we rename the command to run packer
embedded components as execute.
The fmt command reformats HCL2 templates, provided it can parse the file
and reformat its contents according to the standards set by the HCL
library's formatters.
However, if the file is malformed for some reason, the command will fail
with a parse error, but while the parse error message is shown, the
actual errors in the template(s) are not forwarded, making it hard for
users to understand what went wrong with the contents of the file
they're trying to format.
In order to be more helpful with those errors, we now forward those
parsing errors to the UI.
This change addresses vulnerabilities reported by govulncheck
```
Vulnerability #1: GO-2024-2610
Errors returned from JSON marshaling may break template escaping in
html/template
More info: https://pkg.go.dev/vuln/GO-2024-2610
Standard library
Found in: html/[email protected]
Fixed in: html/[email protected].1
Example traces found:
#1: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do, which eventually calls template.Template.Execute
#2: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do, which eventually calls template.Template.ExecuteTemplate
Vulnerability #2: GO-2024-2600
Incorrect forwarding of sensitive headers and cookies on HTTP redirect in
net/http
More info: https://pkg.go.dev/vuln/GO-2024-2600
Standard library
Found in: net/[email protected]
Fixed in: net/[email protected].1
Example traces found:
#1: datasource/http/data.go:119:24: http.Datasource.Execute calls http.Client.Do
#2: hcl2template/function/aws_secretetkey.go:38:37: function.init calls template.GetAWSSecret, which eventually calls http.Client.Post
#3: hcl2template/function/aws_secretetkey.go:38:37: function.init calls template.GetAWSSecret, which eventually calls http.Client.PostForm
Vulnerability #3: GO-2024-2599
Memory exhaustion in multipart form parsing in net/textproto and net/http
More info: https://pkg.go.dev/vuln/GO-2024-2599
Standard library
Found in: net/[email protected]
Fixed in: net/[email protected].1
Example traces found:
#1: internal/hcp/api/service_build.go:81:48: api.Client.UpdateBuild calls packer_service.Client.PackerServiceUpdateBuild, which eventually calls textproto.Reader.ReadLine
#2: datasource/http/data.go:140:26: http.Datasource.Execute calls io.ReadAll, which eventually calls textproto.Reader.ReadMIMEHeader
Vulnerability #4: GO-2024-2598
Verify panics on certificates with an unknown public key algorithm in
crypto/x509
More info: https://pkg.go.dev/vuln/GO-2024-2598
Standard library
Found in: crypto/[email protected]
Fixed in: crypto/[email protected].1
Example traces found:
#1: datasource/http/data.go:140:26: http.Datasource.Execute calls io.ReadAll, which eventually calls x509.Certificate.Verify
```
While migrating to a unified approach for loading plugins, the API
major and minor versions were not added to the constraints for
discovering plugins from the environment, leading to Packer potentially
considering plugins that are not compatible with itself.
This should not be possible, but was due to this omission, which we fix
with this commit.
In order to test the Packer subcommands, some tests rely on a plugin:
github.com/sylviamoss/comment.
This plugin is outdated compared to our SDK, and some of the binaries
releases don't match what is expected by Packer, i.e. v1.0.0 vs. v0.2.8.
To fix that, we migrate to use the hashicups plugin, which is our demo
plugin for Packer, and is actually maintained, and up-to-date, which
will make those tests more reliable moving forward.
Since we now support loading pre-releases with Packer subcommands, we
relax the contraints we had placed on the `--path' option, so it will
accept any `-dev' binary, in addition to final releases.
Since we added the capability for the plugin discovery to ignore
installed pre-releases of plugins, we give that capacity to the validate
and build commands, through a new command-line flag: release-only.
Since we now support loading pre-releases, we also want Packer to be
able to ignore them by user demand, so we put in place the
infrastructure to modulate this.
When Packer is loaded, we used to perform plugin discovery.
This was done for every call to Packer, including when it is executed as
a plugin, arguably against what the comments document.
Doing this as early in the loading process makes it harder to change
this behaviour, as we'd need to introduce flags aside from the rest, and
handle them manually, which is not optimal.
Therefore, we change this: now when Packer starts executing, it will not
attempt to discover installed plugins anymore, and instead will only try
to load them when a configuration has been parsed, and is being used to
perform actions (typically build/validate).
When a pre-release version of a plugin is locally installed, it may or
may not be loaded depending on the constraints expressed in the template
being executed.
If the template contains constraints for loading the plugin, it would be
ignored, while if that wasn't present, it would be loaded.
This is inconsistent, and deserves to be addressed, which is what this
commit does.
With this change, plugin pre-releases are now loaded, provided the
version reported matches the constraints, independently from its
pre-release suffix `-dev'.
Also, if a release with the same version is also installed alongside the
pre-release version of a plugin, it will have precedence over the
pre-release.
When Packer orders the plugins by their version, we'd assumed it was
ordered from highest to lowest.
However, this was the case because our implementation of `Less' was
actually returning `>=', which is not what it was supposed to do.
This commit therefore fixes the implementation of `Less' to do what it
is documented to do, and ensures the behaviour is correct through
additional testing.
Changing this requires some changes to the loading process as well
because of the aforementioned assumption with regards to ordering.
Since the plugins remove subcommand now only loads valid plugins, it
cannot run on mock data anymore, and the logic for creating a valid
plugin hierarchy is not present in this repository, but does in another,
so we move those tests from here to that repository in order to continue
testing them.
When a plugin is loaded from Packer, we now check that the version it
reports matches what the name implies. In case there's a mismatch, we
log, and reject the binary.
Right now we had two paths for discovering installed plugins, i.e.
through plugin-getter's `ListInstallations' function, or through the
`Discover' call, which relied on a glob to list the installations.
This was required since we allowed plugins to be installed in multiple
locations, and with different constraints.
Now that we force a certain convention, we can consolidate the logic
into ListInstallations, and rely on that logic in `Discover' to load a
plugin into the PluginConfig for the current Packer run.
The plugin installation list should be sorted according to a name first,
then version second basis.
Right now, we only rely on the glob to add plugin installs to this list,
making the order unreliable since the lexicographical order is not the
order in which we want to see the same plugin ordered (e.g. v1.0.9 >
1.0.10).
To fix this, we implement a logic for sorting a list of installations
that does what's described above with more accuracy.
Since the plugin directory should now be unique instead of a list, we
can use it directly from the receiver's structure instead of as a
parameter to the function.
Since we'll only look in the plugin directory, and not from multiple
sources now, for installing/listing plugins, we can simplify the
structures which used to accept multiple directories so they only accept
one.
Since we'll be only accepting plugins installed locally along with a
shasum, and those that adopt the convention we introduced with
required_plugins, we remove support for plugins that only have a
packer-plugin-.* type name.
Since we're removing the alternative plugin installation directories in
favour of only supporting installing them in the PACKER_PLUGIN_PATH
directory, we only return one directory when getting the known plugin
directories.
With Packer 1.10.1 we started warning when a build failed to complete
because of a potential incompatibility with the builder being used.
This led to cases in which the build failed for other reasons, and
Packer would still warn of potential incompatibilities, even if the
builder was in effect HCP compatible.
We attempted to fix this issue by introducing a new error type, and
checks when we read the artifacts linked to a build, however this loop
would fail when any one of the artifacts is not compatible with HCP
Packer, leading to false failures.
To avoid this problem, we log incompatibilities to the verbose logger,
and only signal the problem if all the artifacts could not be used to
upload data to HCP Packer, in which case it's almost certain that if the
build succeeded and no artifacts are registered to the build, that all
the components used are not compatible with HCP, and should be reported
as such to users.
When running a build with HCP Packer enabled, Packer attempts to push
the build status to HCP.
If the build fails, we update the status to BUILD_FAILED, and that's the
end of it.
If however the build succeeds, Packer attempts to get the HCP artifact
from the builder, which will only succeed if the builder supports it.
Otherwise, we'll get either nil, or an artifact type that is not
compatible with what is expected for HCP support.
When either of those happens, we warn that the builder may not support
HCP Packer at all, so users are aware of the problem.
However, when the error was introduced, it only looked at the fact that
an error was produced, independently of the type of error. This caused
legitimate errors while building to be reported as potential
incompatibility between the builder and HCP, which was confusing to
users.
This commit changes this by introducing a new error type, only produced
when the artifact either is nil, or failed to be deserialised into a HCP
artifact, which lets us produce the incompatibility warning with more
accuracy.
Not validating the bucket's name during parse leads to configurations
being marked as valid, even if the bucket name is not, which will fail
during a real build afterwards.
To avoid this problem and fail with an error as quickly as possible, we
add a check during parsing, so that it gets reported for validate as
well.
This change removes all external plugin docs using the old remote docs framework
from the Packer core documentation in favor of the Packer integration framework.
Remove plugins will be enabled on the integration portal and contacted to merge their PRs to finalize the integration migration.
Lucas Bajolet
Wilken Rivera
pavedroad
dependabot[bot]
Jenna Goldstrich
Heat Hamilton
Wilken Rivera
Heat Hamilton
Devashish