Since the initialization code was embedded in the sequential logic that
we'll be moving away from soon, we move that to the build block itself,
so we're able to invoke it in any order later on.
When a user specifies a --only/--except option in their command-line,
and it turns-out to be unused, Packer signals it with a Warning
diagnostic.
This diagnostic, in previous versions of Packer, would only state that
one option did not match anything, without stating which option did not
match a build or post-processor.
This commit tracks the usage of each option so we can report later which
of those options was not used in this warning diagnostic.
When processing builds, we convert build blocks/sources to a series of
CoreBuild, which contains the sequence of provisioners and
post-processors to run, and the builder to use to boot the instance and
create the image from.
This process of converting builds used to be completely sequential, and
orchestrated by the `GetBuilds' function.
This gets broken down into a `ToCoreBuild' function on the build block,
whose function is to lift and filter-out (based on --only/--except) the
core builds from a valid build block, and this gets invoked from
`GetBuilds'.
This breakdown is useful as a preparation to the DAG's introduction to
the code later on.
Since these constants and functions will be later useful for the
extraction of the sequential logic in a scheduler, we change the
visibility of those now.
Since hcl.Diagnostics supports extending it through the `Extend' method,
we might as well use it instead of manually appending a series of
diagnostics to it.
Local variables can't have a validation block in their definition, so
this step in not useful and should be removed.
Besides, since the validation was done on the local variables before
evaluation, it did nothing at all, as the PackerConfig.LocalVariables
collection gets populated during evaluation, so this is essentially a
no-op, and can be safely removed.
As with datasources, we break down the local variable evaluation in two
functions, one doing the evaluation for a local variable, and the
sequential/recursive evaluation of all the local variables lifted from
the packer template.
This, plus dependency management will let us later reuse it for another
evaluation strategy, while reusing those components as much as
possible.
In some cases, rather than walking through a full block of HCL in order
to lift the references to other components of a build, we only have the
expression to walk on.
From this expression, we can lift the variables as a collection of
hcl.Traversal, from which we then filter only those that interest us.
So, since this is a use-case, we split the function that lifts the
expressions from a hcl block and filter them based on the root type of
the traversal into two separate functions.
Since we want to allow for independently executing datasources when
needed, we break down the logic to do so.
Each datasource will now hold a list of dependencies. For now, these
dependencies are all datasources themselves, but this may change in the
future.
Then, actual execution of the datasource is self-contained within a
method of the structure, so that we can individually execute them.
Finally, we replace the current sequential execution by a method of the
PackerConfig with an approach akin to local variable evaluation, where
we execute all the datasources one-by-one, and if they cannot be
executed, move along.
We retry this loop for as many times as we don't reach either a state
where all the datasources have been executed; or when we reach a
terminal state where datasources cannot be evaluated because of
dependencies remaining, and no changes have occurred after a try.
The `Datasources' type holds a map of datasource references to their
respective block.
In this map, the blocks were structures rather than pointer to
structures, which makes updating their contents a harder process than it
should be.
To allow for in-place changes for the datasource blocks, we change the
type of the blocks to be a pointer instead.
In previous versions of Packer, the parser would load a HCL file (in
either HCL or JSON format), and process it in phases.
This process meant that if the file was not valid in the first place,
we'd have some duplicate errors first.
Another thing is that sources and build blocks were not being parsed at
the beginning of the parsing phase at all, but were discovered later in
the process, when the data sources have been executed and variables have
been evaluated.
This commit changes the parsing to happen all at once, so we parse the
files with the parser, which will fail should one file be malformatted,
then we visit the body once using the top-level schema for a Packer
build template, handling misplaced blocks once in the process.
Then, since the builds and sources may contain some dynamic blocks that
need to be expanded once their context is ready (i.e. the
variables/datasources they depend on), we can expand the build block for
them, and populate their respective objects.
This last step should also be adapted for datasources later, as they may
need to support dynamic blocks, and be expanded lazily.
The previous implementation of the GetVarsByType function worked only on
top-level attributes, ignoring the nested blocks in the structure.
This implies that if a datasource depends on another through an
expression within a nested block, we may not execute it first, and then
executing this datasource before its dependent is possible, resulting in
an error in the end.
This commit is an attempt at making this more reliable for HCL configs,
but only works on configs lifted from HCL files for now. We need to make
this more reliable for later iterations.
Datasources use their attribute's expressions to determine whether or
not they depend on another datasource, in order to get the list of
dependencies and execute them before executing a datasource.
This code may be useful later on for figuring out the dependencies for
any block, so we move this code to the utils.go file, and use this for
datasources.
When a datasource fails to be evaluated because a cycle has been
detected, we point out one of the links of the chain now so that users
have a better idea of what to look at.
Since datasources are recursively evaluated depending on their
dependencies, we don't need to pre-execute those that depend on nothing,
as the recursive traversal of the datasources will take care of that for
us.
Not sure why this was defined and returned, but the value was set, but
never used, as such this is not useful to keep in the code, so let's
simplify this now.
When users call `packer init' on a template that does not specify a
`required_plugin' block, the command succeeds but does nothing, which is
not helpful for users that may expect their plugins to install.
To remedy that problem, we now output a message like what `packer
plugins required' does on templates without such a block, so that users
have an idea of what to change in order to get the command to work.
The active plugins that resided in a HashiCorp repository have been migrated to
the integrations framework and no longer need to be listed within the plugins-manifest file.
Local variables can't have a validation block in their definition, so
this step in not useful and should be removed.
Besides, since the validation was done on the local variables before
evaluation, it did nothing at all, as the PackerConfig.LocalVariables
collection gets populated during evaluation, so this is essentially a
no-op, and can be safely removed.
The flags -v and --version are handled by the CLI runner, which
outputs the raw version string. This change shortcuts the handling of
the flags to force Packer to run the version subcommand.
Closes: #12565
Lucas Bajolet
Tor Arne Vestbø
Wilken Rivera
Shengjing Zhu
guoguangwu
Sk.Lv
Ilya Voronin