* Update HCP Packer build labels argument
Previously when running a partial build on multi-cloud build template it
was found that build labels were only being applied at the creation for
the partially executed build. Leaving all other completed builds with
no HCP Packer build labels. This updates how incomplete builds are
loaded from the registry and ensure that any defined
hcp_packer_registry.build_labels are assigned to the build before
starting an actual Packer build.
Related to: #11573
* Add test case for overwriting build labels
* Update tests to call CreateInitialBuild for non-existing builds
* Rename test case to TestBucket_PopulateIteration
* Fix data race in PopulateIteration against mock service
Before Change
```
WARNING: DATA RACE
Write at 0x00c0005421b0 by goroutine 47:
github.com/hashicorp/packer/internal/registry.(*MockPackerClientService).PackerServiceCreateBuild()
/Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6
github.com/hashicorp/packer/internal/registry.(*Client).CreateBuild()
/Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592
github.com/hashicorp/packer/internal/registry.(*Bucket).CreateInitialBuildForIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration.func1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration·dwrap·1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58
Previous write at 0x00c0005421b0 by goroutine 46:
github.com/hashicorp/packer/internal/registry.(*MockPackerClientService).PackerServiceCreateBuild()
/Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6
github.com/hashicorp/packer/internal/registry.(*Client).CreateBuild()
/Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592
github.com/hashicorp/packer/internal/registry.(*Bucket).CreateInitialBuildForIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration.func1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration·dwrap·1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58
Goroutine 47 (running) created at:
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7
github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds()
/Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7
testing.tRunner()
/usr/local/go/src/testing/testing.go:1259 +0x22f
testing.(*T).Run·dwrap·21()
/usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 46 (finished) created at:
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7
github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds()
/Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7
testing.tRunner()
/usr/local/go/src/testing/testing.go:1259 +0x22f
testing.(*T).Run·dwrap·21()
/usr/local/go/src/testing/testing.go:1306 +0x47
==================
```
* Add methods for managing builds on an Iteration
* Update Mock Service to be a bit more concurrent
It is a simple set of changes to make it work for the current testing
use cases. If we need to we can move the Called fields to counters or
re-architect the mock.
This change also fixes a race condition when appending to the Slice of
errs when calling PopulateIteration.
* Update test case descriptions
* Apply suggestions from code review
Only append to errs if err is not nil
Co-authored-by: Sylvia Moss <[email protected]>
* Fix linting issues
Co-authored-by: Sylvia Moss <[email protected]>
* Rename mock bucket test file
* Add failing tests for reproducing #11573
```
=== RUN TestBucket_CreateInitialBuildForIteration
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_CreateInitialBuildForIteration (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
types.bucket_test.go:87: expected the initial build to have an additional build label but thee is no diff: ""
--- FAIL: TestBucket_UpdateLabelsForBuild (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image2
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal
--- FAIL: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s)
FAIL
FAIL github.com/hashicorp/packer/internal/registry 0.646s
```
* Update CreateIntialBuildForIteration to initialize each build with a new map
Previously upon creating the initial build the same map, which was
initialized for the build_labels argument was being shared across all
build images. This was causing an issue with labels being backed by the
same map for all builds. This change ensures that all builds get their
own map with any global build labels copied over during the initial creation.
Closes#11573
Passing tests with changes on branch
```
RUN TestBucket_CreateInitialBuildForIteration
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_CreateInitialBuildForIteration (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_UpdateLabelsForBuild (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image2
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine version:1.7.0]
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:the-original-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine version:1.7.0]
--- PASS: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s)
```
* Handle errors from bucket methods
* Update test cases
Initialize maps for bucket when calling NewBucketWithIteration
* allowing to set shell env vars from a key/value map.
* tests
This new map will work alongside the `environment_vars` but it allows using a
datasource value seamlessly. At validation, and because `environment_vars` was
an array of strings containing `KEY=value`, using datsources would not work,
because these values are not known yet and would evaluate to `<unknown>`. With
this, the value and the key can be unknown and will validate correctly for
datasources.
browsing through the documentation I was searching for the list of builders available. The obvious choice (builders page) didn't link to the list of builders and it took me a few minutes to find that. Adding a link to the plugins might speed up that bit and would avoid having to rescue to the search engine.
* evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs
* Make default input variable type the DynamicPseudoType
This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty.
* do not take Empty types from default value
* Update types.variables.go
Co-authored-by: Wilken Rivera <[email protected]>
* Update HCL2 PackerConfig to call SetDebug mode for CoreBuild when --debug flag is passed
```
~> packer build --debug /tmp/source.pkr.hcl
Debug mode enabled. Builds will not be parallelized.
null.example: output will be in this color.
==> null.example: Pausing after run of step 'StepConnect'. Press enter to continue.
==> null.example: Pausing before the next provisioner . Press enter to continue.
==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell4149319610
null.example: hi
==> null.example: Pausing before the next provisioner . Press enter to continue.
==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell3210691290
null.example: hi 2
==> null.example: Pausing after run of step 'StepProvision'. Press enter to continue.
==> null.example: Pausing before cleanup of step 'StepProvision'. Press enter to continue.
==> null.example: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
Build 'null.example' finished after 8 seconds 284 milliseconds.
==> Wait completed after 8 seconds 284 milliseconds
==> Builds finished. The artifacts of successful builds are:
--> null.example: Did not export anything. This is the null builder
```
* Add call to SetOnError, and SetForce
* prevent duplicate local block creation
* remove duplicate locals block bug
* local variables: first get block, then decode it + simplify retry loop
* Update types.packer_config.go
* revert go get of hcl lib
On a local developer machine there is a good chance the HCP_ORG_ID and
HCP_PROJECT_ID are not set since the HCP Packer client can derive this
information using the token/secret. This change initializes the location
structure with the information obtained from the client and not the
envs.
Results of tests before change with no HCP_ORG_ID or HCP_PROJECT_ID set
```
par_test.go:185: (*packer_service.PackerServiceGetIterationOK)(nil)
par_test.go:185: &{0 [] } (*models.GrpcGatewayRuntimeError) is not supported by the TextConsumer, can be resolved by supporting TextUnmarshaler interface
--- FAIL: TestAcc_PAR_pkr_build (3.94s)
FAIL
FAIL github.com/hashicorp/packer/internal/registry/acctest 6.446s
testing: warning: no tests to run
PASS
ok github.com/hashicorp/packer/internal/registry/env 0.318s [no tests to run]
FAIL
```
Results of tests after change with no HCP_ORG_ID or HCP_PROJECT_ID set
```
2022/01/18 12:23:48 ui: --> null.example: Published metadata to HCP Packer registry packer/pkr-acctest-temp-2/iterations/01FSQ3EPPY69PMXJSTTFYDPQ5R
--- PASS: TestAcc_PAR_pkr_build (3.20s)
PASS
ok github.com/hashicorp/packer/internal/registry/acctest 5.121s
testing: warning: no tests to run
PASS
ok github.com/hashicorp/packer/internal/registry/env 0.155s [no tests to run]
```
Michele Degges
Wilken Rivera
Pekka NurmiandWilken Rivera
Johanan Liebermann
Zachary Shilton
Ygal Blum
Adrien Delorme
Michael Weibel
jugglinmike
Kevin Wang
packer-ci
Jenna Goldstrich
Frank Korving
Xiaoyu Zhong
teddylear
Dylan Staley
Ethan Davidson
Bryce Kalow