If a variable is defined and overriden in the packer command-line, but
there's a problem during the evaluation of this override (type error
typically), we show an error message which details the problem.
This message points to a temporary in-memory HCL2 "file" that we use for
parsing and evaluating the expression for the variable, but since it's
virtual, there's no point in using this as the source for the error, as
it will always yield "line 0" and no contents.
So, in order to limit confusion here, we remove the source for this
error message.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
* Update validation options for undeclared variables
In an effort to help users move from JSON to HCL2 templates the support for
variable definitions files are being updated to ignore undeclared
variable warnings on build execution. For legacy JSON templates builds no
warnings are displayed when var-files contain undeclared variables.
Since preferred mode HCL2 templates is to be explicit with variable
declarations - they must be declared to be used - validation for
undeclared variables still warns when running `packer validate`. A new
flag has been added to the validate command that can be used to disable
undeclared variable warnings.
* Update validation test for unused variables
Example Run
```
~> go run . validate -no-warn-undeclared-var -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
The configuration is valid.
~> go run . validate -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
Warning: Undefined variable
The variable "unused" was set but was not declared as an input variable.
To declare variable "unused" place this block in one of your .pkr.hcl
files,
such as variables.pkr.hcl
variable "unused" {
type = string
default = null
}
The configuration is valid.
~> go run . build -var-file
command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl
command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
file.chocolate: output will be in this color.
Build 'file.chocolate' finished after 744 microseconds.
==> Wait completed after 798 microseconds
==> Builds finished. The artifacts of successful builds are:
--> file.chocolate: Stored file: chocolate.txt
```
* Rename Strict field to WarnOnUndeclaredVar
The field name Strict is a bit vague since it is only used for
checking against undeclared variables within a var-file definition.
To mitigate against potential overloading of this field it is
being renamed to be more explicit on its usage.
* command/build: Add warn-on-undeclared-var flag
Now that the default behaviour is to not display warnings for undeclared variables
an optional flag has been added to toggle the old behaviour.
```
~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl
Warning: Undefined variable
The variable "unused" was set but was not declared as an input variable.
To declare variable "unused" place this block in one of your .pkr.hcl files,
such as variables.pkr.hcl
variable "unused" {
type = string
default = null
}
file.chocolate: output will be in this color.
Build 'file.chocolate' finished after 762 microseconds.
==> Wait completed after 799 microseconds
==> Builds finished. The artifacts of successful builds are:
--> file.chocolate: Stored file: chocolate.txt
```
When a variable is set in a pkrvars file, but isn't defined, an error
message is output, but does not deliver an example of what is expected
by Packer in order to complete a build.
To remedy that, we improve the error message by giving an example of
variable block to include in the build template.
* evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs
* Make default input variable type the DynamicPseudoType
This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty.
* do not take Empty types from default value
* Update types.variables.go
Co-authored-by: Wilken Rivera <[email protected]>
* prevent duplicate local block creation
* remove duplicate locals block bug
* local variables: first get block, then decode it + simplify retry loop
* Update types.packer_config.go
* revert go get of hcl lib
* HCL2 variables: split validation from getting value, to only
This way we do this only once and log this only once. The errors were being ignored anyways.
* Update types.variables_test.go
* Allow locals to be delcared as individual blocks, and give them the Sensitive flag
* add docs for new local block
* linting
* add tests
* modified parsing to use schema, check for dupes properly
* update comment
fix wording a liiitle
* add tests for duplicate variables definition in two different files
* remove unnecessary slice initialisation
* fix crash by returning when decode error is hit
* parseLocalVariables: only treat a local vars if its not nil
also return in case of error
return locals in case of error too
* fix duplicate_locals test for windows
Co-authored-by: Adrien Delorme <[email protected]>
* hcl2template/addrs.ParseRef will parse a reference and tell for example if we
are referring to a variable and its name, for now it can only do that and in
the future it improved when we need to most of it is from the TF code. This
is used to tell wether a variable vas referenced in a variable validation
condition; for now.
* Added Validations blocks to the hcl2 Variable struct and code to
parse/validate that.
* add the possibility to set the packer.required_version field; to make sure the template file works with that version of Packer
* add tests
* add documentation on packer.required_version
Example:
packer {
required_version = ">= 1.2.0, < 2.0.0"
}
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.
This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.
fix#9478
* hcl2template/types.variables: Update logic for parsing literal value variables
In running tests via the CLI it was determined that when using the variable block with no explicit type assigned the type of the default
value was not being set within the map. This change updates the `decodeConfig` method so that a type is always set for any defined variable if not specified.
The second change is to properly handle the evaluation of basic variable types (e.g String, Number, Bool). Previously variables defined on the
CLI or via PKR_VAR required some additional quoting to for proper evaluation. This change fixes that issue so that it works like it does in Terraform :)
Build results before the change
```
⇶ PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo=home' .
Error: Variables not allowed
on <value for var.foo from arguments> line 1:
(source code not available)
Variables may not be used here.
==> Builds finished but no artifacts were created.
```
Build results after the change
```
⇶ PKR_VAR_example='["one","two"]' ~/bin/packer build -var 'foo="home"' .
null: output will be in this color.
==> null: Running local shell script: /tmp/packer-shell885249462
null: two
null: home
Build 'null' finished.
==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
⇶ ~/bin/packer build -var 'foo=home' -var 'example=["one","another variable"]' .
null: output will be in this color.
==> null: Running local shell script: /tmp/packer-shell123467506
null: another variable
null: home
Build 'null' finished.
==> Builds finished. The artifacts of successful builds are:
--> null: Did not export anything. This is the null builder
```
* tests/hcl2template/types.variables: Update test to use Bool
Turns out a string value won't actually complain if it's given a non
string looking value. It will just covert the value to a string literal
so using a type Bool which should fail if given anything that is not
true or false.
* tests/hcl2template/types.variables: Update unit tests
During testing it was found that by default the variable stanza were defaulting to a cty.NilType, and not the Type of it's default value.
This change sets the default type of the defined variable to ensure variable evaluation behaviors correctly.
* Add a simple cty.Bool test case
* tests/hcl2template/types.variables: Enable quoted_string test case
* Update hcl2template/types.variables.go
space
Co-authored-by: Adrien Delorme <[email protected]>
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.
All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.
## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files
I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields
## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.
This is a breaking change for packer plugins.
a packer component can be a: builder/provisioner/post-processor
each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.
This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:
Provisioner.Prepare(raws ...interface{}) error
Builder.Prepare(raws ...interface{}) ([]string, error)
PostProcessor.Configure(raws ...interface{}) error
close#1768
Example hcl files:
```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
sources = [
"source.amazon-ebs.first",
]
provisioner "shell" {
inline = [
"sleep 5"
]
}
post-processor "shell-local" {
inline = [
"sleep 5"
]
}
}
// amazon-ebs-kms-key/source.pkr.hcl
source "amazon-ebs" "first" {
ami_name = "hcl2-test"
region = "us-east-1"
instance_type = "t2.micro"
kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
encrypt_boot = true
source_ami_filter {
filters {
virtualization-type = "hvm"
name = "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
root-device-type = "ebs"
}
most_recent = true
owners = ["amazon"]
}
launch_block_device_mappings {
device_name = "/dev/xvda"
volume_size = 20
volume_type = "gp2"
delete_on_termination = "true"
}
launch_block_device_mappings {
device_name = "/dev/xvdf"
volume_size = 500
volume_type = "gp2"
delete_on_termination = true
encrypted = true
}
ami_regions = ["eu-central-1"]
run_tags {
Name = "packer-solr-something"
stack-name = "DevOps Tools"
}
communicator = "ssh"
ssh_pty = true
ssh_username = "ec2-user"
associate_public_ip_address = true
}
```
Lucas Bajolet
hashicorp-copywrite[bot]
Wilken Rivera
Adrien DelormeandWilken Rivera
Adrien Delorme
Megan MarshandAdrien Delorme
Jeff Escalante
zac
Wilken RiveraandAdrien Delorme
Sylvia Moss