Commit Graph
51 Commits
Author SHA1 Message Date
Megan Marsh 48de1fc7da allow datasources to use other datasources
create a null data source for testing. We can choose to document if we want to, but it's a convenience for us

add a test to catch cyclic datasource dependency, update tests to include out of order data sources, and update the code to clean up the returned diagnostics generated from the recursive evaluation

PR review comments
2021-09-03 16:10:39 -07:00
Megan Marsh 4c22169c90 more test fix 2021-08-23 15:29:58 -07:00
Megan Marsh 2dd8c0dd26 fix tests 2021-08-23 14:18:27 -07:00
Wilken Rivera e8b5b4b516 Fix {{packer_version}} interpolation regression (#11200)
* Add Packer version to main context at variable interpolation

Test Results Before Change
```
=== RUN   TestCoreBuild_packerVersion
    testing.go:30: err: template: root:1:2: executing "root" at <packer_version>: error calling packer_version: packer_version not available
--- FAIL: TestCoreBuild_packerVersion (0.00s)
```

Test Results After Change
```
=== RUN   TestCoreBuild_packerVersion
--- PASS: TestCoreBuild_packerVersion (0.00s)
```

* Add packer_core_version to interpolation context for HCL configs

Test Results Before Change
```
=== RUN   TestParse_build/provisioner_with_packer_version_interpolation
    common_test.go:109: Parser.getBuilds() unexpected diagnostics. testdata/build/provisioner_packer_version_interpolation.pkr.hcl:8,5-24: Failed preparing provisioner-block
 "shell" ""; render 'slice_string': template: root:1:2: executing "root" at <packer_version>: error calling packer_version: packer_version not available in:

        {{packer_version}}
--- FAIL: TestParse_build (0.01s)
```

Test Results After Change
```
--- PASS: TestParse_build (0.02s)
--- PASS: TestParse_build/provisioner_with_packer_version_interpolation (0.00s)
```
2021-08-12 10:55:32 +02:00
a534c743cf HCP Packer registry enabled Packer client (#11154)
* Add working registry pkg

* Add custom error for handling the loading of PAR environment variables

* Working Publish to Build, with proper error handling for bucket names

* Update hcp-sdk-go to use branch instead of mod replace directive

* Update Packer build status configuration

* Add support for HCP_PACKER_BUILD_FINGERPRINT env

* Add support for publishing one or more PARtifacts from a single build

* add git shas to this branch

* Add ability to set provider name if available

* Add working RegistryBuilder type

* Add RegistryPostProcessor as wrapper post-processor

* When in PAR mode a empty RegistryPostProcessor is added to the end of
  the post-processor list to publish all final image data.

* Add support for updating a build from PAR that is not in a DONE state

* Fix a small issue with creation the initial builds for an empty
  iteration.

* Add PAR URL to post-processor display

* Implement hcp_packer_registry block (#11168)

* Update vendored Amazon plugin to v1.0.1-dev

* Fix panic when running a Packer registry build in a clean directory

* Remove the publishing of post-processor metadata from the registry
  post-processor.

* Remove metadata add from registry_builder

* Update registry builder to skip a build that was found to be DONE

Co-authored-by: Megan Marsh <[email protected]>
Co-authored-by: Sylvia Moss <[email protected]>
2021-08-05 09:25:19 -04:00
Adrien Delorme 029729225d tests and fixes 2021-02-15 13:47:39 +01:00
Sylvia Moss 774c5903f6 Add error-cleanup-provisioner to HCL2 (#10604) 2021-02-11 10:23:15 +01:00
Sylvia Moss 3ddb17ad86 Don't allow data sources to be used inside data sources (#10559) 2021-02-04 11:25:44 +01:00
Adrien Delorme ed091163be HCL2 Parse packer.required_plugins block + packer init (#10304)
This adds the new `required_plugins` block to be nested under the packer block.

Example:
```hcl
packer {
  required_plugins {
    aws = {
      version = ">= 2.7.0"
      source = "azr/aws"
    }
    azure = ">= 2.7.0"
  }
}
```

For example on darwin_amd64 Packer will install those under :
* "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64"
* "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5"

+ docs
+ tests
2021-02-02 18:05:04 +01:00
Megan MarshandAdrien Delorme fbbda0f9d9 Sensitive locals (#10509)
* Allow locals to be delcared as individual blocks, and give them the Sensitive flag

* add docs for new local block

* linting

* add tests

* modified parsing to use schema, check for dupes properly

* update comment

fix wording a liiitle

* add tests for duplicate variables definition in two different files

* remove unnecessary slice initialisation

* fix crash by returning when decode error is hit

* parseLocalVariables: only treat a local vars if its not nil

also return in case of error
return locals in case of error too

* fix duplicate_locals test for windows

Co-authored-by: Adrien Delorme <[email protected]>
2021-01-26 10:21:44 +01:00
Sylvia Moss 291121dd55 (2) Implement datasources (#10440) 2021-01-20 10:37:16 +01:00
Wilken Rivera 4e58987026 command/fmt: Ensure all variable files ending in .pkrvars.hcl get formatted (#10377)
Before change
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
Error: Cannot tell whether /tmp/unformatted.pkrvars.hcl contains HCL2 configuration data

⇶  echo $?
1
```

After fix
```
⇶  packer fmt -check /tmp/unformatted.pkrvars.hcl
/tmp/unformatted.pkrvars.hcl

⇶  echo $?
3

⇶  packer fmt -check command/test-fixtures/fmt
command/test-fixtures/fmt/unformatted.pkr.hcl
command/test-fixtures/fmt/unformatted.auto.pkrvars.hcl
command/test-fixtures/fmt/unformatted.pkrvars.hcl

```
2020-12-14 10:29:58 -05:00
Wilken Rivera acabc1c1aa Add packer fmt command (#10225)
* Add packer fmt command

This change adds a new command that allows users to format one or more
HCL2 Packer configuration template files.

Related to: #9174

* command/fmt: Add check flag

Packer's fmt command now supports a check flag that will output the name
of any file that would be changed by the HCL2 formatting engine. The
check flag is mutually exclusive with the write flag and will only check
if formatting is needed.

The update write flag will now overwrite the source files with the newly
formatted HCL2 source unless the `-write=false` or `-check` is passed at
the command line.

* Returns a diagnostic error if Format is unable to show a diff - equivalent to `terraform fmt`
* Updates testing to run against #Format and not the private methods of the HCL2Formatter; fixes ShowDiff test failure on Windows
* Updates comments for exported functions

* Add docs for fmt command
2020-11-11 11:49:39 -05:00
Adrien Delorme 6911495fc4 add VariableAssignment struct that help describe an input var assignment 2020-11-02 15:22:29 +01:00
Adrien Delorme b892414e84 add failing test case 2020-10-30 15:40:31 +01:00
Adrien Delorme 4e08ea6a92 add a test 2020-10-30 12:41:29 +01:00
Adrien Delorme 180dc4a54e add test 2020-09-30 12:02:52 +02:00
Sylvia Moss 9181a435aa Make max_retries a string to allow variable interpolation (#9673) 2020-08-05 17:41:20 +02:00
Adrien Delorme c7b35dd6bc HCL2: add post-processors block to run multiple post-processor after a build (#9638)
added `post-processors` block to run chained post-processors after a build.
Before this, defining multiple `post-processor` blocks after
provisioning steps would run them sequentially, now doing this makes them start
from the build's artifact. To queue post-processors you now have to define them
in a `post-processors` block.

This is a breaking change.
2020-07-28 10:02:37 +02:00
Adrien Delorme 0ed10f921f add test 2020-07-06 11:48:39 +02:00
Adrien Delorme db6c3adbba add hcl2 provisioner type that reprepares itself 2020-07-02 18:02:19 +02:00
Adrien Delorme 284d46a1e0 WIP 2020-07-02 15:56:49 +02:00
Adrien Delorme 3d371a2d5d Add complete HCL2 examples + allow to name a singular build.source blocks (#9490)
* in the examples/hcl folder
* add possibility to name singular build.source blocks to differentiate their output and to filter on them
2020-07-02 11:07:59 +02:00
Adrien Delorme c2975140cf HCL2: allow to use keep_input_artifact in post processors (#9477)
* HCL2: allow to use keep_input_artifact in post processors
* add basic test
* add docs
2020-06-25 09:36:48 +02:00
Jeremiah Snapp 0d7c5dc670 Enable only/except for provisioners and post-processors (#9357) 2020-06-05 11:51:23 +02:00
Adrien Delorme f09fd790b7 Merge branch 'master' into master 2020-06-02 15:12:44 +02:00
Adrien Delorme 125ffe36c3 Allow to override some source fields from the build section
Using `hcl.MergeBodies`
2020-05-25 17:09:37 +02:00
Adrienne Cohea 6c6039798c Add test. 2020-05-17 23:27:06 -07:00
Sylvia Moss 553b1fb9f8 Add RetriedProvisioner to allow retry provisioners (#9061) 2020-04-16 11:58:54 +02:00
Adrien Delorme 840e67f775 HCL2: when we see a map generate an attribute spec instead of a block spec (#9035)
* mapstructure-to-hcl2: when we see a map generate an attribute spec and not a block spec

this will alow to do

tags = {
 key = "value"
}

instead of

tags {
  key = "value"
}

This will also enable using variables directly for those tags

* generate code

* update tests
2020-04-14 16:05:13 +02:00
Adrien Delorme 710ebdcef5 add more test for var files 2020-03-12 17:08:53 +01:00
Adrien Delorme 0ccff0d5b9 all variables must have a value.
A variable's default value can be set to null to force user to set it.
2020-03-09 17:25:56 +01:00
Adrien Delorme 6d8cce501e tweak validation & add tests 2020-03-09 16:16:59 +01:00
Adrien Delorme 9b649594c6 update tests 2020-03-04 13:01:18 +01:00
Adrien Delorme 59cc246942 Merge remote-tracking branch 'origin/master' into fix_8730 2020-03-04 12:30:01 +01:00
Adrien Delorme 50896d4ddf WIP 2020-03-03 11:15:56 +01:00
Sylvia Moss 591b684f08 Fix HCL2 local variables decoding to allow local usage within another local in the same locals block (#8755) 2020-02-21 12:12:30 +01:00
Adrien Delorme a8fcb2d91a HCL2: add support for dynamic blocks, document for loops and splat expressions (#8720) 2020-02-20 10:51:34 +01:00
Adrien Delorme 3c213e6eaf continue and realise and unused undefaulted value triggers an error 2020-02-17 18:05:15 +01:00
Adrien Delorme 79867ca26e add test for unset variable 2020-02-17 16:36:19 +01:00
Adrien Delorme 4b10c5a87c try to reproduce #8730 in tests 2020-02-14 11:54:10 +01:00
Adrien Delorme 193dad46e6 Hcl2 input variables, local variables and functions (#8588)
Mainly redefine or reused what Terraform did.

* allow to used `variables`, `variable` and `local` blocks
* import the following functions and their docs from Terraform: abs, abspath, basename, base64decode, base64encode, bcrypt, can, ceil, chomp, chunklist, cidrhost, cidrnetmask, cidrsubnet, cidrsubnets, coalesce, coalescelist, compact, concat, contains, convert, csvdecode, dirname, distinct, element, file, fileexists, fileset, flatten, floor, format, formatdate, formatlist, indent, index, join, jsondecode, jsonencode, keys, length, log, lookup, lower, max, md5, merge, min, parseint, pathexpand, pow, range, reverse, rsadecrypt, setintersection, setproduct, setunion, sha1, sha256, sha512, signum, slice, sort, split, strrev, substr, timestamp, timeadd, title, trim, trimprefix, trimspace, trimsuffix, try, upper, urlencode, uuidv4, uuidv5, values, yamldecode, yamlencode, zipmap.
2020-02-06 11:49:21 +01:00
Sylvia Moss 08b0bd1d2c Fix HCL2 code generation for slices within another slice (#8669) 2020-02-03 17:03:28 +01:00
Adrien Delorme 2d1a67c6cb hcl2: allow to optionnaly name provisioners and post-processors 2020-01-16 12:08:39 +01:00
Adrien Delorme 0102f5b6fa remove unecessary inexistant block panic + add test
the hcl library will error for us
2020-01-06 16:56:42 +01:00
Adrien Delorme 219c80b1e6 better empty folder test 2020-01-06 16:09:44 +01:00
Adrien Delorme aad90842b4 add test case and fix the breaking ones 2020-01-06 16:06:27 +01:00
Adrien Delorme 5ef1893b00 fix crash when build.sources is set to an invalid name
the `build` body doesn't have any labels and we were trying to display those.
I also added a test.
2020-01-06 14:29:43 +01:00
Adrien Delorme 0785c2f6fc build using HCL2 (#8423)
This follows #8232 which added the code to generate the code required to parse
HCL files for each packer component.

All old config files of packer will keep on working the same. Packer takes one
argument. When a directory is passed, all files in the folder with a name
ending with  “.pkr.hcl” or “.pkr.json” will be parsed using the HCL2 format.
When a file ending with “.pkr.hcl” or “.pkr.json” is passed it will be parsed
using the HCL2 format. For every other case; the old packer style will be used.

## 1. the hcl2template pkg can create a packer.Build from a set of HCL (v2) files

I had to make the packer.coreBuild (which is our one and only packer.Build ) a public struct with public fields

## 2. Components interfaces get a new ConfigSpec Method to read a file from an HCL file.

  This is a breaking change for packer plugins.

a packer component can be a: builder/provisioner/post-processor

each component interface now gets a `ConfigSpec() hcldec.ObjectSpec`
which allows packer to tell what is the layout of the hcl2 config meant
to configure that specific component.

This ObjectSpec is sent through the wire (RPC) and a cty.Value is now
sent through the already existing configuration entrypoints:

 Provisioner.Prepare(raws ...interface{}) error
 Builder.Prepare(raws ...interface{}) ([]string, error)
 PostProcessor.Configure(raws ...interface{}) error

close #1768


Example hcl files:

```hcl
// file amazon-ebs-kms-key/run.pkr.hcl
build {
    sources = [
        "source.amazon-ebs.first",
    ]

    provisioner "shell" {
        inline = [
            "sleep 5"
        ]
    }

    post-processor "shell-local" {
        inline = [
            "sleep 5"
        ]
    }
}

// amazon-ebs-kms-key/source.pkr.hcl

source "amazon-ebs" "first" {

    ami_name = "hcl2-test"
    region = "us-east-1"
    instance_type = "t2.micro"

    kms_key_id = "c729958f-c6ba-44cd-ab39-35ab68ce0a6c"
    encrypt_boot = true
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "amzn-ami-hvm-????.??.?.????????-x86_64-gp2"
          root-device-type = "ebs"
        }
        most_recent = true
        owners = ["amazon"]
    }
    launch_block_device_mappings {
        device_name = "/dev/xvda"
        volume_size = 20
        volume_type = "gp2"
        delete_on_termination = "true"
    }
    launch_block_device_mappings {
        device_name = "/dev/xvdf"
        volume_size = 500
        volume_type = "gp2"
        delete_on_termination = true
        encrypted = true
    }

    ami_regions = ["eu-central-1"]
    run_tags {
        Name = "packer-solr-something"
        stack-name = "DevOps Tools"
    }
    
    communicator = "ssh"
    ssh_pty = true
    ssh_username = "ec2-user"
    associate_public_ip_address = true
}
```
2019-12-17 11:25:56 +01:00
Adrien Delorme 0e4fec03d7 make complete tests files link as windows can't handle synlinks 2019-10-15 12:56:43 +02:00