* added the parser for the enforced block
* Enhance enforced provisioner parsing to support HCL and JSON formats
- Updated ParseProvisionerBlocks to handle both HCL and JSON syntax, including legacy JSON format.
- Added comprehensive test cases for JSON provisioner parsing.
- Improved ExtractBuildProvisionerHCL to merge inline commands from shell provisioners.
- Enhanced logging for enforced block operations in HCP Packer.
* Remove PublishEnforcedBlocks function from Bucket struct
* Remove ExtractBuildProvisionerHCL function and unused imports
* Reverted the version upgrade
* Added the internal-sdk for the enforcedProvsioner api changes
* Enhance enforced provisioner handling and error reporting
- Update error handling in FetchEnforcedBlocks to return detailed errors instead of warnings.
- Modify GetCoreBuildProvisionerFromBlock to accept build name for overrides.
- Add tests for FetchEnforcedBlocks to ensure correct behavior and error handling.
- Implement diagnostics for unsupported legacy JSON templates.
* Implement enforced provisioner parsing and handling
- Introduced a new package `enforcedparser` to handle parsing of enforced provisioner blocks from HCL and JSON formats.
- Refactored existing code to utilize the new `ParseProvisionerBlocks` function from the `enforcedparser` package.
- Updated `GetCoreBuildProvisionerFromEnforcedBlock` method to convert enforced provisioner blocks into core build provisioners.
- Enhanced error handling and logging during the parsing process.
- Added tests for the new parsing functionality and ensured existing tests were updated to reflect changes.
- Modified `InjectEnforcedProvisioners` method in JSON registry to utilize the new parsing logic.
* Add test case for -skip-enforcement flag in BuildArgs
* Refactor sensitive variable handling in provisioners and add related tests
* Refactor enforced provisioner handling: remove internal parser, update tests, and streamline API interactions
* Enhance provisioner block parsing: add error handling for invalid combinations and expand test coverage
* Remove internal SDK replacement for enforced block types in go.mod
* Update dependencies in go.mod and go.sum: bump hcp-sdk-go and packer-plugin-sdk versions, adjust syft version, and update OpenTelemetry packages
* Update hcp-sdk-go dependency to v0.172.0 in go.mod and go.sum
* Fix formatting in TestBuildCommand_ParseArgs and add newline at end of json_enforced_test.go
* Refactor testJSONRegistryWithBuilds: remove environment variable setup and streamline registry initialization
* Rename injected variable for clarity in InjectEnforcedProvisioners function
---------
Co-authored-by: Hari Om <[email protected]>
Enables specifying channels in the build configuration to automatically update existing HCP Packer registry channels to point to the new build version upon completion. Improves workflow automation and reduces manual channel management for users.
* bump golang.org/x/crypto to 0.43.0
CVE-2025-47913 GO-2025-4116
* fixed go.sum
* fixed multiple warnings that prevented test runs
* make generate
* fix lint errors, update linter version
* fix go vet issues
Before this fix, the error message looked like:
```
A source type is made of three parts that aresplit by a dot `.`; each part must
start with a letter and may contain only letters, digits, underscores, and
dashes.A valid source reference looks like: `source.type.name`
```
* hcl2template: err on malformed local/data dep
When introducing the DAG for locals and datasources, we forgot to handle
one limit case: if a dependency for a local or data is malformed, we
didn't check that a vertex was associated to it, leading to the final
DAG being malformed, and the DAG library would crash in this case.
This commit fixes this problem by checking that the dependency does
exist before attempting to add it to the graph as an edge for a vertex,
so that it is reported accurately, and do that Packer doesn't crash.
* error message change
---------
Co-authored-by: anshul sharma <[email protected]>
* formatting multiple hcl files
* recursive formating
* test case fixing
* test case fix
* removed not required code
* existing test code fix
* go formatting
* more test cases for new cases
* doc changes
* added additional check on error message in negative test case
* modifying the for loop to preserver user inputted variable files preference
* adding test cases
* pr comments + refactoring the for loop for better readability.
* pr comments + refactoring the for loop for better readability.
* pr comments | handling default case
* adding additional test case
* fixing test cases
* fixing test cases. creating a subdirectory to add a test case for both hcl and json auto var files
* adding test case for json auto var file
This commit adds 3 new HCL2 functions:
* `sum`: computes the sum of a collection of numerical values
* `startswith`: checks if a string has another as prefix
* `endswith`: checks if a string has another as suffix
This method will be used to validate and fetch the correct
HCPPackerRefistry block to be used in HCP build
hcl: check for multiple HCP Packer registry block
hcl: support for top level HCP Packer Registry
hcl: deprecation warning for build block based hcp config
The GetBuilds function, available on both HCL2 and legacy JSON
configuration objects, used to return the Build interface.
This typing by interface is not useful in this instance, since all the
uses of `GetBuilds' are self-contained within Packer, and we're never
using any other implementation for it than `*CoreBuild`.
We've been relying on the dynamic type for all the builds being
*CoreBuild in some places of the code, so to avoid potential surprises
in the future, we'll change the signature now so it returns only
concrete types.
The hcp-sbom provisioner is a provisioner that acts essentially like a
download-only file provisioner, which also verifies the file downloaded
is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to
HCP Packer later on.
As the SDK now supports it in the context of legacy templating engine,
we add support in HCL2 for the aws_secretsmanager_raw function, which
gets the raw value of a secret from aws secrets manager.
`packer validate` would output the same error message four times per
unsupported root block type found in a template (e.g., 'src' instead of
'source'). This behavior was due to a function being called four times
for each file on each stage of the parsing.
When a user defines a `hcp_packer_registry` block in their `build`
without a `bucket_name`, but they define it in their environment, Packer
should not report the bucket_name being wrong.
The strcontains function check if a sub string is a indeed a subset of a
given string.
hcl2template: add strcontains function
The strcontains function check if a sub string is a indeed a subset of a
given string.
The filebase64 function aims to read and encode a file's contents into
base64.
This is mostly to support reading the content of a file that is not
valid UTF-8, as is the case with the `file` function.
Evaluating local variables used to be directly written to the
PackerConfig while each variable was created.
This was somewhat of an issue with testing, as we have a bunch of tests
that relied on `PackerConfig.Variables` being set only when we actually
write something.
This is not really a concern for normal use, just for testing, but to
limit the number of changes to the tests in hcl2template, I opted to
change how variables' values are retained, so that evaluating a single
variable returns a Variable in addition to hcl.Diagnostics, so we can
reify the approach and only create the map of variables if there's
something evaluated.
When preparing the datasources to add into the DAG for evaluating the
build prerequisites, we ended-up in a weird situation in which the
datasources for each vertex pointed to the same one.
This is because of the loop semantics of Go, where the same object is
reused over and over again during each loop, so in the end every
datasource vertex pointed to the same instance of a datasource block.
To avoid this, we instead grab them through their reference, making the
reference to the datasource purely local, and pointing to the actual
datasource block, not the one scoped to the function.
Walk uses a reverse topological order to walk on the graph, doing that
visit concurrently if possible.
This is nice as we can speed-up execution of datasources and locals,
however since the `Variables` map stored in the config, and the
production of the context for it, are not meant to be used concurrently,
this means that we end-up in cases where Packer crashes because of
concurrent accesses to that map.
So until we can change this behaviour, we will fallback to using the
sequential visit algorithm for those vertexes, therefore limiting the
risk of those conflicts.
Local variables had an attribute called Name with the name of the local
variable.
However, when producing an error while walking the DAG of
local/datasources, if an error is encountered during validation, the raw
structure of the vertex was printed out, making the error message
produced hard to understand.
Therefore in order to clean it up, we rename the `Name` attribute for
Local variables as `LocalName`, and introduce a `Name()` function for
that block so that the complete name of the variable is clearly
reported.
The implementation of the DAG as extracted from Terraform relied on a
Root vertex being injected into the graph as the last node to visit.
This is used as a sanity check for Terraform, but doesn't apply to our
use-case for now, as we are always executing everything and have no need
for this root node.
Instead, we change how Validate operates so it does not error in case
there is no valid root node for the graph, but enables us calling it to
check for self-referencing edges, and circular dependencies.
Following up on the DAG work, this commit adds a new option for
initialisation that disables DAG on request.
By default we are going to use the DAG approach, with an option to
fallback to using the older algorithm for evaluation in case users
end-up in an edge-case that prevents them from building a template.
As we have finished setting-up the codebase for it, this commit adds the
logic that uses the internal DAG package, and is able to orchestrate
evaluation of datasources and locals in a non-phased way.
Instead, this code acts by first detecting the dependencies for those
components, builds a graph from them, with edges representing the
dependency links between them, and finally walking on the graph
breadth-first to evaluate those components.
This can act as a drop-in replacement for the current phased logic, but
both should be supported until we are confident that the approach works,
and that there are little to no bugs left to squash.
When registering dependencies for datasources and locals, we now use
refString.
This allows for the functions that detect dependencies to not only be
able to register the same types as dependencies, but instead generalises
it to any type that refString supports, so data, local and var.
This can then be leveraged for orchestrating evaluation of those
components in a non-phased way (i.e. with a DAG for dependency
management).
As with variables, this commit introduces a new function whose purpose
is evaluating the datasource directly, without looking at its
dependencies, or recursively trying to execute them.
Instead, we rely on the DAG to determine when it is safe to execute it,
or if using the phased approach, the current logic will apply.
Since we are in the process of integrating a new way to orchestrate
dependency management and evaluation for datsources and local variables,
we need to split the current function that manages the evaluation of
said local variables, so that recursion and the actual evaluation of a
local variable are two separate functions.
This will allow for evaluating a single variable once the dag is ready
to be introduced.
The `Name` function is used by the DAG library to determine which vertex
is associated to a component, so the `Name` attribute/function needs to
replicate the combination of type and name, so the vertex can be
accurately fetched from the graph.
The hcl2template package contains references already, but these are
linked to a particular type.
This becomes problematic if we want to support cross-type references, so
this commit adds a new abstraction: refString.
A refString contains the component type, its type (if applicable), and
its name, so that the combination of those points to a cty object that
can be linked to a block in the configuration.
Right now, only `var`, `local` and `data` are supported, but the type is
extensible enough that anything else that fits this model such as
sources can be supported in the future potentially.
Previously duplicate detection for local variables happened during
`Initialise`, through a call to `checkForDuplicateLocalDefinition`.
This works in a majority of cases, but for commands like `console`, this
was not detected as the return diagnostics for `Initialise` are ignored.
That check can be done as early as during parsing however, as the names
of blocks are not dynamic in the slightest (no interpolation possible),
so we move that detection logic into `Parse`, so that the behaviour is
coherent between all commands.
The logic for evaluating local variables used to rely on their
definition order, with some edge cases. Typically `locals` blocks define
multiple local variables, which don't necessarily appear in the same
order at evaluation as within the template, leading to inconsistent
behaviour, as the order in which those are added to the list of local
variables is non-deterministic.
To avoid this problem, we change how local variables are evaluated, and
we're adopting a workflow similar to datasources, where the local
variables first build a list of direct dependencies. Then when
evaluation happens, we evaluate all the dependencies recursively for
each local variable, which takes care of this issue.
As with Datasources, we add a cap to the recursion: 10. I.e. if the
evaluation of a single variable provokes an infinite recursion, we stop
at that point and return an error to the user, telling them to fix their
template.
GetVarsByType is a function that gets a list of Traversals from a hcl
Block.
This approach works when what we are visiting is indeed one, however
when we can get an immediate list of Traversals, but want to filter them
based on their roots, we have to reimplement parts of that function.
Therefore, we split this function in two, GetVarsByType still keeps its
current behaviour, but the filtering step is exposed as another function
now: FilterTraversalsByType, so we can reuse it elsewhere.
Tanmay Jain
hashicorp-copywrite[bot]
HariandHari Om
Anurag Sharma
Sebastien Dionne
Louis Sautier
Lucas Bajoletandanshul sharma
karthik P
Martin Grogan
Lucas Bajolet
Devashish
Gustavo Cruz
Martin Grogan
Wenfeng Pan