When Packer is loaded, we used to perform plugin discovery.
This was done for every call to Packer, including when it is executed as
a plugin, arguably against what the comments document.
Doing this as early in the loading process makes it harder to change
this behaviour, as we'd need to introduce flags aside from the rest, and
handle them manually, which is not optimal.
Therefore, we change this: now when Packer starts executing, it will not
attempt to discover installed plugins anymore, and instead will only try
to load them when a configuration has been parsed, and is being used to
perform actions (typically build/validate).
When a user invokes packer for a build or validation, the template being
processed needs components to be present for Packer to process it
without error.
If the component cannot be found from the plugins loaded (or from the
components bundled with Packer), Packer errors, and the command fails.
This is expected, but the error message does not suggest anything to fix
the error, potantially leaving users confused at the problem.
This commit suggests either a replacement (in case of a typo), or points
to the web documentation for Packer, specifically the integrations, so
they can look for the plugin they're missing, and install it, so
subsequent invocations of Packer work.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl.
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Since bundled plugins will be removed in an upcoming version of Packer,
this commit adds a new warning message whenever a template uses one such
plugin.
This warning has been implemented on build, validate, console and the
inspect subcommands.
In addition to warning about the upcoming change and potential issue
this will cause, this warning message proposes solutions to the user so
they know what they'll have to do in order not to rely on those bundled
plugins later.
This change fixes a regression introduced in Packer 1.8.6, where configurations custom builder names
, via the name attribute, would internally interpolate the proper build names, but would display the
name to STDOUT as uninterpolated values. The change updates the creation of a CoreBuild for legacy JSON templates
to use use the interpolated name as the final build type, which gets rendered to the screen.
Test Results Before Fix
```
~> go test ./... -run='TestCoreBuild_buildNameIntepolation' -v
--- FAIL: TestCoreBuild_buildTypeIntepolation (0.01s)
core_test.go:930: build type interpolation failed; expected "test.mybuild-RandomToken", got "test.{{user `build_name`}}"
core_test.go:930: build type interpolation failed; expected "test.build-vardata", got "test.{{user `build_name`}}"
core_test.go:930: build type interpolation failed; expected "test.build-12345", got "test.{{user `build_name`}}"
FAIL
```
Test Results After Fix
```
~> go test ./... -run='TestCoreBuild_buildNameIntepolation' -v
=== RUN TestCoreBuild_buildTypeIntepolation
--- PASS: TestCoreBuild_buildTypeIntepolation (0.01s)
PASS
```
Closes#12281
This commit irons out one of the pain points of the HCP rework by
introducing a HCPPublisher interface, implemented both by the JSON Core,
and the HCL2 PackerConfig, which keeps a map of the build names used by
Packer to the build names pushed on HCP.
This in turn lets us go back to the old implementation of the GetBuilds
function, which returns a list of (filtered) builds, and eventually an
error if something went wrong while processing.
As part of the work to displace everything related to HCP from scattered
places around the Packer code, we move it all to an hcp package.
This in turn reduces the amount of code that the commands have to
integrate, and leaves the HCP details to its own enclave.
To be able to track HCP builds during a Packer build, we need to
propagate the name used for registering the build to HCP.
This works out-of-the-box for JSON templates, as the build's name is
always the builder's, so there's no confusion on that.
For HCP templates however, there's a possibility for a template to
define a name for the build, which is returned by a CoreBuild's Name()
function, in addition to the source's Type/Name.
The builds are registered to HCP with the source.String() function
however, which does not contain the build's name. This prevents any
build containing a name to work, as their name/source.String does not
match.
The name we registered the build with is stored in a CoreBuild as the
Type attribute, but cannot be safely accessed, as builds are types
within the command as packersdk.Build, which only exposes Name() for
this purpose.
In order to circumvent this problem, and as a way to present a prototype
of solution that will likely have to be discussed before we merge it,
this commit hotfixes the issue.
JSON templates used only to report the builder's type in HCP builds,
even if the name was specified.
This commit changes this behaviour to include both if they're available,
in a similar fashion as what is done on the HCL2 templates.
Before this commit, some HCP-related logic was embedded in the parser,
making all of it common to all commands.
This complexifies the logic as some parts rely on the template being in
a HCP-valid environment, and the datasources need to be evaluated in
order for some operations to be executed.
To simplify this and avoid those pitfalls, we move this logic to its own
set of functions that will be performed after the parsing is done, on
any valid HCL2 or JSON template.
* Update handling of registry artifacts
This change uses the github.com/hashicorp/packer-plgin-sdk/packer/registryimage for querying
Artifact State for HCP Registry Image metadata. To handle the conversion
of the RPC response, mapstructure was introduced to conversion state
data into an registryimage.Image before publishing to a image bucket.
* Update to use registry image from packersdk
* Rename internal registry service pkg
* Update vendored plugins to latest version
* The latest release of Amazon, GoogleCompute, and Azure have support
for publishing images to the HCP Packer registry.
* Add Packer version to main context at variable interpolation
Test Results Before Change
```
=== RUN TestCoreBuild_packerVersion
testing.go:30: err: template: root:1:2: executing "root" at <packer_version>: error calling packer_version: packer_version not available
--- FAIL: TestCoreBuild_packerVersion (0.00s)
```
Test Results After Change
```
=== RUN TestCoreBuild_packerVersion
--- PASS: TestCoreBuild_packerVersion (0.00s)
```
* Add packer_core_version to interpolation context for HCL configs
Test Results Before Change
```
=== RUN TestParse_build/provisioner_with_packer_version_interpolation
common_test.go:109: Parser.getBuilds() unexpected diagnostics. testdata/build/provisioner_packer_version_interpolation.pkr.hcl:8,5-24: Failed preparing provisioner-block
"shell" ""; render 'slice_string': template: root:1:2: executing "root" at <packer_version>: error calling packer_version: packer_version not available in:
{{packer_version}}
--- FAIL: TestParse_build (0.01s)
```
Test Results After Change
```
--- PASS: TestParse_build (0.02s)
--- PASS: TestParse_build/provisioner_with_packer_version_interpolation (0.00s)
```
* Add working registry pkg
* Add custom error for handling the loading of PAR environment variables
* Working Publish to Build, with proper error handling for bucket names
* Update hcp-sdk-go to use branch instead of mod replace directive
* Update Packer build status configuration
* Add support for HCP_PACKER_BUILD_FINGERPRINT env
* Add support for publishing one or more PARtifacts from a single build
* add git shas to this branch
* Add ability to set provider name if available
* Add working RegistryBuilder type
* Add RegistryPostProcessor as wrapper post-processor
* When in PAR mode a empty RegistryPostProcessor is added to the end of
the post-processor list to publish all final image data.
* Add support for updating a build from PAR that is not in a DONE state
* Fix a small issue with creation the initial builds for an empty
iteration.
* Add PAR URL to post-processor display
* Implement hcp_packer_registry block (#11168)
* Update vendored Amazon plugin to v1.0.1-dev
* Fix panic when running a Packer registry build in a clean directory
* Remove the publishing of post-processor metadata from the registry
post-processor.
* Remove metadata add from registry_builder
* Update registry builder to skip a build that was found to be DONE
Co-authored-by: Megan Marsh <[email protected]>
Co-authored-by: Sylvia Moss <[email protected]>
This adds the new `required_plugins` block to be nested under the packer block.
Example:
```hcl
packer {
required_plugins {
aws = {
version = ">= 2.7.0"
source = "azr/aws"
}
azure = ">= 2.7.0"
}
}
```
For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"
+ docs
+ tests
The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost.
This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too.
fix#9478
* Update validate command to use FixConfig for checking against known
fixers
* Update validation command flag docs
* Add ConfigFixer method to PackerHandler Interface
* Implement ConfigFixer interface in PackerConfig
* Remove all stdout messaging (i.e calls to c.Ui.Say) in the validate
command. The command will only display hcl.Diagnotic messaging when there is an error or warning.
HCL2 Configs
```
⇶ packer validate docker_centos_shell_provisioner.pkr.hcl
```
JSON Configs
```
⇶ packer validate vmware-iso_ubuntu_minimal/vmware-iso_ubuntu_minimal.json
Error: Failed to prepare build: "vmware-iso"
1 error occurred:
* Deprecated configuration key: 'iso_checksum_type'. Please call `packer fix`
against your template to update your template to be compatable with the current
version of Packer. Visit https://www.packer.io/docs/commands/fix/ for more
detail.
Warning: Fixable configuration found.
You may need to run `packer fix` to get your build to run correctly.
See debug log for more information.
map[string]interface{}{
"builders": []interface{}{
map[string]interface{}{
... // 3 identical entries
"guest_os_type": string("ubuntu-64"),
"http_directory": string("http"),
- "iso_checksum":
string("946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
+ "iso_checksum":
string("sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2"),
- "iso_checksum_type": string("sha256"),
"iso_url":
string("http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04.1-server-amd64.iso"),
"shutdown_command": string("echo 'vagrant' | sudo -S shutdown -P now"),
... // 4 identical entries
},
},
}
```
* refactor so that json and hcl2 templates are both prepared in the same place in the build call, to make code easier to reason about. Remove overly verbose error output which isn't useful in vast majority of cases
* fix tests
* check err msg
* hcl2template.PackerConfig.GetBuilds: raise a diagnostic in case the packer core build perpare call errors
Co-authored-by: Adrien Delorme <[email protected]>
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.
All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.
## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files
I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields
## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.
This is a breaking change for packer plugins.
a packer component can be a: builder/provisioner/post-processor
each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.
This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:
Provisioner.Prepare(raws ...interface{}) error
Builder.Prepare(raws ...interface{}) ([]string, error)
PostProcessor.Configure(raws ...interface{}) error
close#1768
Example hcl files:
```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
sources = [
"source.amazon-ebs.first",
]
provisioner "shell" {
inline = [
"sleep 5"
]
}
post-processor "shell-local" {
inline = [
"sleep 5"
]
}
}
// amazon-ebs-kms-key/source.pkr.hcl
source "amazon-ebs" "first" {
ami_name = "hcl2-test"
region = "us-east-1"
instance_type = "t2.micro"
kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
encrypt_boot = true
source_ami_filter {
filters {
virtualization-type = "hvm"
name = "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
root-device-type = "ebs"
}
most_recent = true
owners = ["amazon"]
}
launch_block_device_mappings {
device_name = "/dev/xvda"
volume_size = 20
volume_type = "gp2"
delete_on_termination = "true"
}
launch_block_device_mappings {
device_name = "/dev/xvdf"
volume_size = 500
volume_type = "gp2"
delete_on_termination = true
encrypted = true
}
ami_regions = ["eu-central-1"]
run_tags {
Name = "packer-solr-something"
stack-name = "DevOps Tools"
}
communicator = "ssh"
ssh_pty = true
ssh_username = "ec2-user"
associate_public_ip_address = true
}
```
It is simply the best/simplest solution and trying to prevent users from passing and integer here would be like opening a can of worms. Because:
* we cannot make mapstructure validate our duration string ( with an UnmarshalJSON func etc.)
* we cannot make mapstructure spit a string instead of a duration and packer will decode-encode-decode config.
* the hcl2 generated code asks for a string, so this will be enforced by default.
Lucas Bajolet
Wilken Rivera
hashicorp-copywrite[bot]
Wilken Rivera
Adrien Delorme
Sathyajith Bhat
Adrien Delorme
Lars Lehtonen