# Description
The output of the example on [uuidv4 Function](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/functions/uuid/uuidv4) is not a valid RFC compliant UUIDv4. It indicates the usage of the `uuidv4()` function and outputs `b5ee72a3-54dd-c4b8-551c-4bdc0204cedb` which is not a valid UUIDv4.
I've corrected the example to output a UUIDv4 conforming to the RFC as such `xxxxxxxx-xxxx-4xxx-Nxxx-xxxxxxxxxxxx`, where:
- The 13th character is always `4` (indicating version 4).
- The 17th character must be either `8`, `9`, `a`, or `b` (indicating the first character of the variant).
# Changes
```diff
- b5ee72a3-54dd-c4b8-551c-4bdc0204cedb
+ 9fc99a70-7cd5-482d-bb2b-03af016e4e94
```
Replaces the old UUID output with a valid RFC compliant UUIDv4.
# References
- [RFC 4122](https://datatracker.ietf.org/doc/html/rfc4122)
- [RFC 9562](https://datatracker.ietf.org/doc/html/rfc9562)
# Misc.
To make sure this wasn't an issue with the `uuidv4()` function within Hashicorp's [packer](https://github.com/hashicorp/packer) I tested the function in the following way:
### Command executed:
```ps
> .\packer.exe inspect .\uuid.pkr.hcl
```
### Contents of the _uuid.pkr.hcl_ file:
```hcl
locals {
uuid_0 = uuidv4()
uuid_1 = uuidv4()
uuid_2 = uuidv4()
uuid_3 = uuidv4()
uuid_4 = uuidv4()
uuid_5 = uuidv4()
uuid_6 = uuidv4()
uuid_7 = uuidv4()
uuid_8 = uuidv4()
uuid_9 = uuidv4()
}
```
### Output:
```ps
Packer Inspect: HCL2 mode
> input-variables:
> local-variables:
local.uuid_0: "90877db8-5519-46ea-ae15-7dfb92594064"
local.uuid_1: "fe6a4649-97d9-4686-b981-3295175f941a"
local.uuid_2: "9944d83d-dab2-4cfb-a1db-572d19271e7a"
local.uuid_3: "547cddb7-c979-4b87-90d0-2bd9b68858b5"
local.uuid_4: "c13dc47a-552c-4dfb-a75d-2f63bb248b41"
local.uuid_5: "3db1ce29-bdde-4642-b010-1a41d47c22a3"
local.uuid_6: "4a020460-edd1-471d-b8a2-5956c0c68257"
local.uuid_7: "1845bf87-6908-4fc0-8f11-b5b4f36c60a7"
local.uuid_8: "f5c7e552-b799-45f3-8172-46162eadfd89"
local.uuid_9: "057c2eaf-6769-4a8d-90c8-775aec80496a"
> builds:
```
`packer validate` would output the same error message four times per
unsupported root block type found in a template (e.g., 'src' instead of
'source'). This behavior was due to a function being called four times
for each file on each stage of the parsing.
Version 5.11.0 of the go-git library is vulnerable to two CVEs as
reported by our scanners.
Both are not impacting Packer since we only use go-git to read values
from a local Git repository, but still we upgrade our version to 5.13.0
so those reports don't apply to us.
The reported AWS S3 vulnerability was inherited from the go-getter
module that Packer uses for downloading files from external sources.
This vulnerability only impacts S3 uploads, therefore Packer is not
vulnerable itself as go-getter only downloads such blobs.
Since the change required to fix this advisory would be to bump the AWS
SDK to v2, this being a major change, is not something to do lightly, so
we opted to ignore this advisory for now so it doesn't block upcoming
releases.
The version of golang.org/x/net that we're using (v0.25.0) is vulnerable
to GO-2024-3333, a DoS vulnerability.
Packer is not particularly vulnerable to this, as we are not a hosted
service, but since security scanners report this, and the fix is rather
simple, we address it today.
The hcl2_upgrade command transforms a JSON template into an HCL2
template for use with Packer.
The command is quite fragile already, but given that this is the last
remaining fragment that causes Packer to depend on the AWS SDK directly,
we can do away with it.
This commit therefore imports the definitions for AWS access config, so
we can extract this information from the JSON template, and include it
in the definition of the output source for AWS, since we manage this one
differently from other sources.
This allows us to not depend on the AWS plugin directly, which in turn
makes Packer not need to link with the AWS plugin when compiling the
executable.
We are still depending on the AWS SDK for now since the SDK exposes a
aws_secretsmanager function that can be used for interpolation (legacy
JSON interpolation to be clear), so this cannot be removed from now, but
we should consider some form of remediation in the future.
The crypto experimental module that the SDK depends on was detected
vulnerable on v0.23.0 to an authentication bypass attack through the
ServerConfig for SSH.
This is only used for locally proxying SSH communications, typically for
use with Ansible to provide an SSH server with which to interact, in
order to provision an instance of an OS that uses a communicator other
than SSH, therefore the vulnerability does not necessarily impact us.
However, in order to resolve the automatic report of that vulnerability,
we bump the crypto module's version to 0.31.0, where that problem was
addressed.
Compiling plugins was originally intended to be an idempotent operation.
This however starts to change as we introduce build customisations,
which have the unfortunate side-effect of changing the state of the
plugin directory, leading to conflicts between concurrent compilation
jobs.
Therefore to mitigate this problem, this commit changes how compilation
jobs are processed, by introducing a global compilation queue, and
processing plugins' compilation one-by-one from this queue.
This however makes such requests asynchronous, so test suites that
require plugins to be compiled will now have to wait on their completion
before they can start their tests.
To this effect, we introduce one more convenience function that
processes those errors, and automatically fails the test should one
compilation job fail for any reason.
When building a plugin, we may want some customisation capabilities
beyond changing the version/pre-release/metadata, and instead run
commands or change files on the filesystem.
To do so, we introduce functions under the BuildCustomisation type,
which have two responsabilities: changing the current state of the
plugin's directory, and cleaning up afterwards.
These customisations are passed as parameters to the BuildSimplePlugin
function, and are called one-by-one, deferring their cleanup after the
build process is finished.
A first implementation of such a customisation is added with this
commit, in order to change the version of a module that the plugin
depends on, which we'll use to change the version of the plugin SDK in
order to test how Packer behaves with different versions of the SDK for
a single plugin.
When trying to validate that a particular file exists after a run of
Packer in a test suite, we can use the FileExists checker that we
provide as part of the gadgets we added for the acceptance test suites.
This approach works well, but only if we can extract a file name
reliably from the output of Packer core, or if we know what to look for
exactly beforehand. For other cases with a generated name however, the
FileExists checker is not enough, and therefore to accomodate for those
cases, we are introducing a new checker for this purpose: FileGlob.
FileGlob, as its name suggests, runs a glob expression on the
filesystem, and returns an error if no match was found regarding this
glob expression.
Some tests will create files and directories as part of the execution
path for Packer, and we need a way to check this, so this commit adds a
new file gadget to do those checks after a command executes.
When a user defines a `hcp_packer_registry` block in their `build`
without a `bucket_name`, but they define it in their environment, Packer
should not report the bucket_name being wrong.
The strcontains function check if a sub string is a indeed a subset of a
given string.
hcl2template: add strcontains function
The strcontains function check if a sub string is a indeed a subset of a
given string.
The acceptance tests for the HTTP datasource had a `Outputs` attribute
for checking the output of the command for a specific regexp pattern.
As pointed out, given the expectative nature of the attribute, naming it
`Outputs` did not make the intent clear, so we rename it to
`ExpectedOutputs` with this commit.
Since the expected error to look for in the output is compiled to a
regexp, the `[]` from the error message were interpreted as a set of
characters, which made the regexp not match the expected output from the
command.
So to avoid this problem, we escape them so they are expected verbatim
in the command output.
Since the DAG package was lifted from Terraform, its contents are more
than what we need for now, so this commit cleans-up the package to keep
only the currently needed parts of code.
If we need to support more in the future, we can revert this commit, or
pickup the changes again from Terraform.
Since we introduce the DAG with this series of commits, only on locals
and data sources, we need to make sure that the behaviour is what we
expect.
Therefore, this commit adds a basic test with Packer build, and packer
validate, to evaluate a template with both locals and data sources
depending on one another.
This is rejected with the sequential evaluation methods, as we process
the different types one-by-one, whereas the DAG allows us to mix the
order between the two, while still rejecting circular dependencies (and
doing that before they even get evaluated), and self-references.
Evaluating local variables used to be directly written to the
PackerConfig while each variable was created.
This was somewhat of an issue with testing, as we have a bunch of tests
that relied on `PackerConfig.Variables` being set only when we actually
write something.
This is not really a concern for normal use, just for testing, but to
limit the number of changes to the tests in hcl2template, I opted to
change how variables' values are retained, so that evaluating a single
variable returns a Variable in addition to hcl.Diagnostics, so we can
reify the approach and only create the map of variables if there's
something evaluated.
When preparing the datasources to add into the DAG for evaluating the
build prerequisites, we ended-up in a weird situation in which the
datasources for each vertex pointed to the same one.
This is because of the loop semantics of Go, where the same object is
reused over and over again during each loop, so in the end every
datasource vertex pointed to the same instance of a datasource block.
To avoid this, we instead grab them through their reference, making the
reference to the datasource purely local, and pointing to the actual
datasource block, not the one scoped to the function.
Walk uses a reverse topological order to walk on the graph, doing that
visit concurrently if possible.
This is nice as we can speed-up execution of datasources and locals,
however since the `Variables` map stored in the config, and the
production of the context for it, are not meant to be used concurrently,
this means that we end-up in cases where Packer crashes because of
concurrent accesses to that map.
So until we can change this behaviour, we will fallback to using the
sequential visit algorithm for those vertexes, therefore limiting the
risk of those conflicts.
Local variables had an attribute called Name with the name of the local
variable.
However, when producing an error while walking the DAG of
local/datasources, if an error is encountered during validation, the raw
structure of the vertex was printed out, making the error message
produced hard to understand.
Therefore in order to clean it up, we rename the `Name` attribute for
Local variables as `LocalName`, and introduce a `Name()` function for
that block so that the complete name of the variable is clearly
reported.
The implementation of the DAG as extracted from Terraform relied on a
Root vertex being injected into the graph as the last node to visit.
This is used as a sanity check for Terraform, but doesn't apply to our
use-case for now, as we are always executing everything and have no need
for this root node.
Instead, we change how Validate operates so it does not error in case
there is no valid root node for the graph, but enables us calling it to
check for self-referencing edges, and circular dependencies.
For all the commands that call Initialise, we introduce a new flag:
UseSequential.
This disables DAG scheduling for evaluating datasources and locals as a
fallback to the newly introduced DAG scheduling approach.
`hcl2_upgrade` is a special case here, as the template is always JSON,
there cannot be any datasource, so the DAG in this case becomes
meaningless, and is not integrated in this code path.
Following up on the DAG work, this commit adds a new option for
initialisation that disables DAG on request.
By default we are going to use the DAG approach, with an option to
fallback to using the older algorithm for evaluation in case users
end-up in an edge-case that prevents them from building a template.
As we have finished setting-up the codebase for it, this commit adds the
logic that uses the internal DAG package, and is able to orchestrate
evaluation of datasources and locals in a non-phased way.
Instead, this code acts by first detecting the dependencies for those
components, builds a graph from them, with edges representing the
dependency links between them, and finally walking on the graph
breadth-first to evaluate those components.
This can act as a drop-in replacement for the current phased logic, but
both should be supported until we are confident that the approach works,
and that there are little to no bugs left to squash.
The dag package is a port over from Terraform to Packer, changing what
little there was to fit our current dependency ecosystem.
Most of the changes are on the type of diagnostics returned, as
Terraform has its own type for them, while we rely on hcl's Diagnostics.
Other than that, the functionality is essentially equivalent, and the
code was barely touched.
When registering dependencies for datasources and locals, we now use
refString.
This allows for the functions that detect dependencies to not only be
able to register the same types as dependencies, but instead generalises
it to any type that refString supports, so data, local and var.
This can then be leveraged for orchestrating evaluation of those
components in a non-phased way (i.e. with a DAG for dependency
management).
As with variables, this commit introduces a new function whose purpose
is evaluating the datasource directly, without looking at its
dependencies, or recursively trying to execute them.
Instead, we rely on the DAG to determine when it is safe to execute it,
or if using the phased approach, the current logic will apply.
Since we are in the process of integrating a new way to orchestrate
dependency management and evaluation for datsources and local variables,
we need to split the current function that manages the evaluation of
said local variables, so that recursion and the actual evaluation of a
local variable are two separate functions.
This will allow for evaluating a single variable once the dag is ready
to be introduced.
The `Name` function is used by the DAG library to determine which vertex
is associated to a component, so the `Name` attribute/function needs to
replicate the combination of type and name, so the vertex can be
accurately fetched from the graph.
The hcl2template package contains references already, but these are
linked to a particular type.
This becomes problematic if we want to support cross-type references, so
this commit adds a new abstraction: refString.
A refString contains the component type, its type (if applicable), and
its name, so that the combination of those points to a cty object that
can be linked to a block in the configuration.
Right now, only `var`, `local` and `data` are supported, but the type is
extensible enough that anything else that fits this model such as
sources can be supported in the future potentially.
In HCP's metadata package, especially the VCS/git parts, we keep the
current HEAD for a repository, along with the state it is in, in order
to report it to HCP Packer when the build completes.
However, when a build is run on a template from an empty Git repository,
and HCP Packer is enabled, the code would crash when trying to get the
information on the current HEAD, as it doesn't exist.
The git library we use returns an error in such a case, but this was
ignored, leading to a crash when attempting to get the hash to this
reference later on.
This commit fixes the problem by NOT ignoring the error to get the head,
and immediately stop processing the git data as it doesn't yet exist.
Joban
Martin Grogan
Gustavo Cruz
dependabot[bot]
hashicorp-tsccr[bot]
Lucas Bajolet
Judith Malnick
Martin Grogan
Nikolay Edigaryev
Henrik Gerdes
huochexizhan
Laurent Goudet
Gregory Bonk
yannic rieger