Packer can only install plugins from github.com, with
releases.hashicorp.com consulted first for HashiCorp-published plugins.
This has been a long-standing gap for air-gapped and policy-restricted
environments (#11164): the source address parser already accepts any
hostname, but both existing getters reject non-github.com sources at
install time.
Add a remote plugin getter that installs plugins from the host named in
a required_plugins source address. The host serves the directory
structure of releases.hashicorp.com under the source's path: an
index.json listing versions, and per version a SHA256SUMS file, the
zips it lists, and - when the zip names carry no plugin protocol
version - the version's manifest.json. A plugin published on
releases.hashicorp.com is therefore mirrored as a verbatim copy of its
tree, with every checksum file and signature upstream-authored. A
plugin published as GitHub release assets is mirrored by copying each
release's assets into a version directory, renaming their SHA256SUMS
file to the unprefixed convention with content unchanged, and writing
an index.json listing the versions. Both kinds of content can be
served side by side by one host.
Getter selection happens per source address: github.com sources keep
the release and github getters unchanged, while any other host is
served by the remote getter over HTTPS. Sources with three or more
components are supported, up to the existing 16-component limit, so
nested artifact-repository paths and hosts that embed the upstream
origin in their path all resolve.
Version discovery, constraint solving, checksum verification, and the
binary naming rules match the existing getters. The installed filename
is rebuilt from validated checksum-file fields and never taken from the
server's response, checksum entries matching neither known naming shape
are rejected rather than guessed at, and nothing the remote metadata
supplies is used to fetch from another origin or path. index.json
parsing is covered by fixtures captured from the live releases API, so
a format change there fails tests rather than user installs.
Closes#11164
* added the parser for the enforced block
* Enhance enforced provisioner parsing to support HCL and JSON formats
- Updated ParseProvisionerBlocks to handle both HCL and JSON syntax, including legacy JSON format.
- Added comprehensive test cases for JSON provisioner parsing.
- Improved ExtractBuildProvisionerHCL to merge inline commands from shell provisioners.
- Enhanced logging for enforced block operations in HCP Packer.
* Remove PublishEnforcedBlocks function from Bucket struct
* Remove ExtractBuildProvisionerHCL function and unused imports
* Reverted the version upgrade
* Added the internal-sdk for the enforcedProvsioner api changes
* Enhance enforced provisioner handling and error reporting
- Update error handling in FetchEnforcedBlocks to return detailed errors instead of warnings.
- Modify GetCoreBuildProvisionerFromBlock to accept build name for overrides.
- Add tests for FetchEnforcedBlocks to ensure correct behavior and error handling.
- Implement diagnostics for unsupported legacy JSON templates.
* Implement enforced provisioner parsing and handling
- Introduced a new package `enforcedparser` to handle parsing of enforced provisioner blocks from HCL and JSON formats.
- Refactored existing code to utilize the new `ParseProvisionerBlocks` function from the `enforcedparser` package.
- Updated `GetCoreBuildProvisionerFromEnforcedBlock` method to convert enforced provisioner blocks into core build provisioners.
- Enhanced error handling and logging during the parsing process.
- Added tests for the new parsing functionality and ensured existing tests were updated to reflect changes.
- Modified `InjectEnforcedProvisioners` method in JSON registry to utilize the new parsing logic.
* Add test case for -skip-enforcement flag in BuildArgs
* Refactor sensitive variable handling in provisioners and add related tests
* Refactor enforced provisioner handling: remove internal parser, update tests, and streamline API interactions
* Enhance provisioner block parsing: add error handling for invalid combinations and expand test coverage
* Remove internal SDK replacement for enforced block types in go.mod
* Update dependencies in go.mod and go.sum: bump hcp-sdk-go and packer-plugin-sdk versions, adjust syft version, and update OpenTelemetry packages
* Update hcp-sdk-go dependency to v0.172.0 in go.mod and go.sum
* Fix formatting in TestBuildCommand_ParseArgs and add newline at end of json_enforced_test.go
* Refactor testJSONRegistryWithBuilds: remove environment variable setup and streamline registry initialization
* Rename injected variable for clarity in InjectEnforcedProvisioners function
---------
Co-authored-by: Hari Om <[email protected]>
* add configuration fields to SBOMInternalProvisioner
* add os detection
* move implementation to hcp-sbom
* upload scanner binary and execute
* use latest syft version always
* reduce duplicate code
* rename config fields appropriately
* default to cyclonedx
* add syft dependency
* add support for elevated user for windows
* add retry for download
* add syft dependency
Updates go version
* optimization for windows
* improve docs
* update config usage rules
* add unit tests
* update golang version, fix linter issues
* refactor and improvements
* simplify few lines
* refactor retry for scanner download
* resolved conflicts
* resolve conflicts from main
* stick to syft v1 for compatibility
* fix lint issues
* stricter version check for syft
* fix version eg
* update go version to 1.25.7
* go mod changes
* bump golang.org/x/crypto to 0.43.0
CVE-2025-47913 GO-2025-4116
* fixed go.sum
* fixed multiple warnings that prevented test runs
* make generate
* fix lint errors, update linter version
* fix go vet issues
* changes for pulling binary from releases.hashicorp.com
* cahnges for getting release from release official site
* cahnges for getting release from release official site
* unit test cases
* unit test cases
* unit test coverage
* changes for getter releases.hashicorp.com
* lint fix
* lint fix
* lint fix
* manifest.json related changes
* manifest.json related changes
* manifest.json related changes
* manifest.json related changes
* github getter test cases
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for plugins getter
* unit test cases for getting release from official site
* description to the methods
The GetBuilds function, available on both HCL2 and legacy JSON
configuration objects, used to return the Build interface.
This typing by interface is not useful in this instance, since all the
uses of `GetBuilds' are self-contained within Packer, and we're never
using any other implementation for it than `*CoreBuild`.
We've been relying on the dynamic type for all the builds being
*CoreBuild in some places of the code, so to avoid potential surprises
in the future, we'll change the signature now so it returns only
concrete types.
Since the protos for uploading an SBOM for a build have been changed to
use an enumeration instead of a plain string with the latest revisions
to the HCP Packer SBOM support feature, we update how we reference those
values for the SBOM format to use that enum instead.
Since packer now supports keeping track of SBOMs produced during a
build, we add the code to integrate those changes into the internal/hcp
package, so we do upload them on build completion.
The hcp-sbom provisioner is a provisioner that acts essentially like a
download-only file provisioner, which also verifies the file downloaded
is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to
HCP Packer later on.
As we're trying to move away from gob for serialising data over the
wire, this commit adds the capability for Packer to pick dynamically
between gob or protobuf for the serialisation format to communicate with
plugins.
As it stands, if all the plugins discovered are compatible with
protobuf, and we have not forced gob usage, protobuf will be the
serialisation format picked.
If any plugin is not compatible with protobuf, gob will be used for
communicating with all the plugins that will be used over the course of
a command.
Following up on the DAG work, this commit adds a new option for
initialisation that disables DAG on request.
By default we are going to use the DAG approach, with an option to
fallback to using the older algorithm for evaluation in case users
end-up in an edge-case that prevents them from building a template.
In a couple places in the codebase we didn't check the errors from
functions we execute.
In some cases this is harmless (or at least ignorable), but others may
need to log what went wrong, so for all the reported occurrences we
either ignore explicitly or handle the error with a log.
Some of the variables we create are flagged by our linters as
ineffective assignments, which makes sense as those are generally fed by
code below, so we don't need to use the declaration/assignation syntax
(:=) but instead can fall back to using var with a type to get the zero
value of the declared entity.
Since the enumer implementation we used hadn't been updated for 5+
years, this didn't work with recent linux/go versions, and enumer
crashed while attempting to parse/analyse the source files.
There's another alternative on Github, forked from the one we used,
which seems more maintained now, and does produce the expected files in
Packer.
When Discovering plugins installed through the `Discover` function, we
use the base name of the plugin binary we discovered preliminarly, then
we match its name against a regex to extract the prefix for the plugin's
components.
Extracting the base path used to be done with `path.Base`, which while
working perfectly on UNIX systems, does not on Windows as it uses `\\`
as their path separator.
To circumvent this problem, we use the `filepath` package to extract the
base name of the plugin instead, making the discovery logic work again
on Windows.
Listing installed plugins on Windows requires the extension to be set in
the ListOptions, otherwise they are not discovered.
While working on the discovery code, and consolidating it in a single
location, we've forgotten to pass the argument to ListInstallations, so
that makes it impossible to automatically discover installed components
on Windows.
This commit fixes this issue for the plugins required, and the general
discovery process during build/validate.
If a plugin is installed in the PACKER_PLUGIN_PATH, and its version
contains metadata, we reject it. This is because metadata is free-form
data, which could then make it possible to have multiple conflicting
versions of a plugin installed, so we don't support it and explicitely
reject plugins like those.
A valid plugin with metadata in its version information should be
installed without its metadata part, so there can only be one variant of
the plugin installed at a specific version.
The ParsePluginSource function can be invoked from either a HCL2 context
(when parsing a required_plugins block), or from the command-line
itself.
While in the first context a hcl.Diagnostics is coherent, in case the
source to parse is a command-line argument, for example when installing
or removing a plugin, the error message cannot have an HCL context,
leading to errors that are incorrectly prefixed by a <nil> string dure
to the lack of a reference to attach the diagnostic to.
Therefore, in order to fix this behaviour, the logic that parses plugin
sources now returns an error, and attaching the error to an HCL subject
is done independently, if needed.
When specifying/installing plugins, a source URI is required for Packer
to be able to locate or install a plugin to the local plugin hierarchy.
The plugin hierarchy is based on the plugin source, where each component
in this hierarchy will become a directory.
In order to avoid sources with too many levels of nesting, causing a lot
of mkdirs, we limit the number of sources to 16 in this commit, this
should be long enough for most of our users.
When running a packer command on an HCL2 template, depending on whether
or not there are required_plugin blocks defined, Packer may need to
discover and register a plugin's components multiple times.
This is the behaviour ever since those blocks were introduced to Packer,
but given we are doing the operation multiple times, this is suboptimal.
This commit changes the way things works by first doing the restricted
discovery of plugins (as dictated by required_plugins), then proceeding
to the global discovery, with the change that subsequent component
discoveries will not have precedence over those pre-discovered anymore.
This allows us to invert the call order of both discovery phases safely,
and maintains the constraints described in the templates.
When a user defines PACKER_PLUGIN_PATH in their environment, we need to
error if their path defines multiple directories separated by `:`.
This used to be supported, but this is removed with 1.11 as we're
simplifying the loading process for plugins, so we opted to fall-back to
only one plugin directory supported.
```
~> packer init mondoo_req.pkr.hcl
Failed getting the "github.com/mondoohq/cnspec" plugin:
error:
Remote installation of the plugin version 10.8.1-dev is unsupported.
This is likely an upstream issue with the 10.8.1 release, which should be reported.
If you require this specific version of the plugin, download the binary and install it manually.
packer plugins install --path '<plugin_binary>' github.com/mondoohq/cnspec
```
This change is an attempt to remove the need for additional temporary files, along with
calls to stat the temp files, to reduce the number of files being created, opened, and closed.
In addition to this change, the logic for falling back to a previous version if the highest matched version
is a pre-release has been removed. Instead we will assume that any prior versions will exhibit the same issue and
return immediately. A user can install the version manually if they will or they can modify their version constraint
to a properly released version.
When a checksum file for a release is downloaded and iterated upon to
find the compatible binary for a release, we used to log each
non-compatible entry in the logs.
This is noisy as we know there's going to be multiple entries that don't
match the host's os/arch, and there's no good reason to show those, so
we silence them.
Benjamin Holmes
Tanmay Jain
HariandHari Om
Anurag Sharma
Sebastien Dionne
anshulSharma
Lucas Bajolet
Jenna Goldstrich
Devashish
Lucas Bajolet
huochexizhan
IAMDAVID0920
Wilken Rivera