Packer can only install plugins from github.com, with
releases.hashicorp.com consulted first for HashiCorp-published plugins.
This has been a long-standing gap for air-gapped and policy-restricted
environments (#11164): the source address parser already accepts any
hostname, but both existing getters reject non-github.com sources at
install time.
Add a remote plugin getter that installs plugins from the host named in
a required_plugins source address. The host serves the directory
structure of releases.hashicorp.com under the source's path: an
index.json listing versions, and per version a SHA256SUMS file, the
zips it lists, and - when the zip names carry no plugin protocol
version - the version's manifest.json. A plugin published on
releases.hashicorp.com is therefore mirrored as a verbatim copy of its
tree, with every checksum file and signature upstream-authored. A
plugin published as GitHub release assets is mirrored by copying each
release's assets into a version directory, renaming their SHA256SUMS
file to the unprefixed convention with content unchanged, and writing
an index.json listing the versions. Both kinds of content can be
served side by side by one host.
Getter selection happens per source address: github.com sources keep
the release and github getters unchanged, while any other host is
served by the remote getter over HTTPS. Sources with three or more
components are supported, up to the existing 16-component limit, so
nested artifact-repository paths and hosts that embed the upstream
origin in their path all resolve.
Version discovery, constraint solving, checksum verification, and the
binary naming rules match the existing getters. The installed filename
is rebuilt from validated checksum-file fields and never taken from the
server's response, checksum entries matching neither known naming shape
are rejected rather than guessed at, and nothing the remote metadata
supplies is used to fetch from another origin or path. index.json
parsing is covered by fixtures captured from the live releases API, so
a format change there fails tests rather than user installs.
Closes#11164
* feat(provenance): add SLSA provenance and attestation signing libraries
Add internal/provenance for deriving in-toto subjects from Packer
artifacts, building SLSA Provenance v1 predicates, wrapping in-toto
statements, and best-effort git/CI source detection.
Add internal/attestation for DSSE envelope handling and a pluggable
Signer/Verifier backend supporting key (local PEM), kms (aws/gcp/
azure/hashivault), and keyless (Sigstore Fulcio) modes, plus Sigstore
bundle handling and DSSE/policy verification.
Add the supporting module dependencies in go.mod/go.sum.
* feat(provenance): add provenance post-processor
Add the opt-in "provenance" post-processor that runs after a build,
derives subjects from the artifact, emits DSSE-wrapped SLSA provenance
(and optional SBOM) attestations, signs them via the configured
signing backend, and writes sidecar files (including *.sigstore.json
bundles in keyless mode). Register it in the core post-processor set.
The provenance enable flag is a tri-state so an unset value stays
enabled through HCL2 decoding instead of being silently disabled.
* feat(provenance): add verify-attestation command
Add "packer verify-attestation" to verify signed DSSE attestations
against key, KMS, and keyless policy inputs, including optional
Sigstore bundle checks for Rekor and timestamp evidence. Register the
command in the CLI.
* docs(provenance): add reference CI workflows and changelog
Add reference GitHub Actions workflows under examples/ci for SLSA L2
keyless signing and L3-compatible delegated signing
* fix: lint and tests
* Added docs for Provenance PostProcessor
* added the parser for the enforced block
* Enhance enforced provisioner parsing to support HCL and JSON formats
- Updated ParseProvisionerBlocks to handle both HCL and JSON syntax, including legacy JSON format.
- Added comprehensive test cases for JSON provisioner parsing.
- Improved ExtractBuildProvisionerHCL to merge inline commands from shell provisioners.
- Enhanced logging for enforced block operations in HCP Packer.
* Remove PublishEnforcedBlocks function from Bucket struct
* Remove ExtractBuildProvisionerHCL function and unused imports
* Reverted the version upgrade
* Added the internal-sdk for the enforcedProvsioner api changes
* Enhance enforced provisioner handling and error reporting
- Update error handling in FetchEnforcedBlocks to return detailed errors instead of warnings.
- Modify GetCoreBuildProvisionerFromBlock to accept build name for overrides.
- Add tests for FetchEnforcedBlocks to ensure correct behavior and error handling.
- Implement diagnostics for unsupported legacy JSON templates.
* Implement enforced provisioner parsing and handling
- Introduced a new package `enforcedparser` to handle parsing of enforced provisioner blocks from HCL and JSON formats.
- Refactored existing code to utilize the new `ParseProvisionerBlocks` function from the `enforcedparser` package.
- Updated `GetCoreBuildProvisionerFromEnforcedBlock` method to convert enforced provisioner blocks into core build provisioners.
- Enhanced error handling and logging during the parsing process.
- Added tests for the new parsing functionality and ensured existing tests were updated to reflect changes.
- Modified `InjectEnforcedProvisioners` method in JSON registry to utilize the new parsing logic.
* Add test case for -skip-enforcement flag in BuildArgs
* Refactor sensitive variable handling in provisioners and add related tests
* Refactor enforced provisioner handling: remove internal parser, update tests, and streamline API interactions
* Enhance provisioner block parsing: add error handling for invalid combinations and expand test coverage
* Remove internal SDK replacement for enforced block types in go.mod
* Update dependencies in go.mod and go.sum: bump hcp-sdk-go and packer-plugin-sdk versions, adjust syft version, and update OpenTelemetry packages
* Update hcp-sdk-go dependency to v0.172.0 in go.mod and go.sum
* Fix formatting in TestBuildCommand_ParseArgs and add newline at end of json_enforced_test.go
* Refactor testJSONRegistryWithBuilds: remove environment variable setup and streamline registry initialization
* Rename injected variable for clarity in InjectEnforcedProvisioners function
---------
Co-authored-by: Hari Om <[email protected]>
* changes for pulling binary from releases.hashicorp.com
* cahnges for getting release from release official site
* cahnges for getting release from release official site
* unit test cases
* unit test cases
* unit test coverage
* changes for getter releases.hashicorp.com
* lint fix
* lint fix
* lint fix
* manifest.json related changes
* manifest.json related changes
* manifest.json related changes
* manifest.json related changes
* github getter test cases
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for getters
* added test cases for plugins getter
* unit test cases for getting release from official site
* description to the methods
* formatting multiple hcl files
* recursive formating
* test case fixing
* test case fix
* removed not required code
* existing test code fix
* go formatting
* more test cases for new cases
* doc changes
* added additional check on error message in negative test case
* modifying the for loop to preserver user inputted variable files preference
* adding test cases
* pr comments + refactoring the for loop for better readability.
* pr comments + refactoring the for loop for better readability.
* pr comments | handling default case
* adding additional test case
* fixing test cases
* fixing test cases. creating a subdirectory to add a test case for both hcl and json auto var files
* adding test case for json auto var file
The GetBuilds function, available on both HCL2 and legacy JSON
configuration objects, used to return the Build interface.
This typing by interface is not useful in this instance, since all the
uses of `GetBuilds' are self-contained within Packer, and we're never
using any other implementation for it than `*CoreBuild`.
We've been relying on the dynamic type for all the builds being
*CoreBuild in some places of the code, so to avoid potential surprises
in the future, we'll change the signature now so it returns only
concrete types.
In the current state, a Packer build that succeeds but fails to push its
metadata to HCP for reasons other than a lack of artifact will always
succeed from the perspective of a user invoking `packer build`.
This can be a bit misleading, as users may expect their artifacts to
appear on HCP Packer if their build succeeded on Packer Core, so this
commit changes this behaviour, instead reporting HCP errors as a real
error if the build failed, so packer returns a non-zero error code if
this happens.
The hcp-sbom provisioner is a provisioner that acts essentially like a
download-only file provisioner, which also verifies the file downloaded
is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to
HCP Packer later on.
As we're trying to move away from gob for serialising data over the
wire, this commit adds the capability for Packer to pick dynamically
between gob or protobuf for the serialisation format to communicate with
plugins.
As it stands, if all the plugins discovered are compatible with
protobuf, and we have not forced gob usage, protobuf will be the
serialisation format picked.
If any plugin is not compatible with protobuf, gob will be used for
communicating with all the plugins that will be used over the course of
a command.
`packer validate` would output the same error message four times per
unsupported root block type found in a template (e.g., 'src' instead of
'source'). This behavior was due to a function being called four times
for each file on each stage of the parsing.
The hcl2_upgrade command transforms a JSON template into an HCL2
template for use with Packer.
The command is quite fragile already, but given that this is the last
remaining fragment that causes Packer to depend on the AWS SDK directly,
we can do away with it.
This commit therefore imports the definitions for AWS access config, so
we can extract this information from the JSON template, and include it
in the definition of the output source for AWS, since we manage this one
differently from other sources.
This allows us to not depend on the AWS plugin directly, which in turn
makes Packer not need to link with the AWS plugin when compiling the
executable.
We are still depending on the AWS SDK for now since the SDK exposes a
aws_secretsmanager function that can be used for interpolation (legacy
JSON interpolation to be clear), so this cannot be removed from now, but
we should consider some form of remediation in the future.
For all the commands that call Initialise, we introduce a new flag:
UseSequential.
This disables DAG scheduling for evaluating datasources and locals as a
fallback to the newly introduced DAG scheduling approach.
`hcl2_upgrade` is a special case here, as the template is always JSON,
there cannot be any datasource, so the DAG in this case becomes
meaningless, and is not integrated in this code path.
Since the enumer implementation we used hadn't been updated for 5+
years, this didn't work with recent linux/go versions, and enumer
crashed while attempting to parse/analyse the source files.
There's another alternative on Github, forked from the one we used,
which seems more maintained now, and does produce the expected files in
Packer.
When remotely installing a plugin, constraints are used by Packer to
determine which version of a plugin to install.
These constraints can be arbitrarily complex, including operators and
ranges in which to look for valid versions.
However, the versions specified in those constraints should always be
final releases, and not a pre-release since we don't explicitly support
remotely installing pre-releases.
This commit therefore addds checks to make sure these are reported ASAP,
even before the source is contacted to list releases and picking one to
install.
Listing installed plugins on Windows requires the extension to be set in
the ListOptions, otherwise they are not discovered.
While working on the discovery code, and consolidating it in a single
location, we've forgotten to pass the argument to ListInstallations, so
that makes it impossible to automatically discover installed components
on Windows.
This commit fixes this issue for the plugins required, and the general
discovery process during build/validate.
Taru Garg
Benjamin Holmes
Tanmay Jain
Hari
Sebastien Dionne
anshulSharma
Karthik P
Lucas Bajolet
Devashish
Lucas Bajolet
Gustavo Cruz