* Update HCP Packer build labels argument
Previously when running a partial build on multi-cloud build template it
was found that build labels were only being applied at the creation for
the partially executed build. Leaving all other completed builds with
no HCP Packer build labels. This updates how incomplete builds are
loaded from the registry and ensure that any defined
hcp_packer_registry.build_labels are assigned to the build before
starting an actual Packer build.
Related to: #11573
* Add test case for overwriting build labels
* Update tests to call CreateInitialBuild for non-existing builds
* Rename test case to TestBucket_PopulateIteration
* Fix data race in PopulateIteration against mock service
Before Change
```
WARNING: DATA RACE
Write at 0x00c0005421b0 by goroutine 47:
github.com/hashicorp/packer/internal/registry.(*MockPackerClientService).PackerServiceCreateBuild()
/Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6
github.com/hashicorp/packer/internal/registry.(*Client).CreateBuild()
/Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592
github.com/hashicorp/packer/internal/registry.(*Bucket).CreateInitialBuildForIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration.func1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration·dwrap·1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58
Previous write at 0x00c0005421b0 by goroutine 46:
github.com/hashicorp/packer/internal/registry.(*MockPackerClientService).PackerServiceCreateBuild()
/Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6
github.com/hashicorp/packer/internal/registry.(*Client).CreateBuild()
/Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592
github.com/hashicorp/packer/internal/registry.(*Bucket).CreateInitialBuildForIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration.func1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration·dwrap·1()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58
Goroutine 47 (running) created at:
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7
github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds()
/Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7
testing.tRunner()
/usr/local/go/src/testing/testing.go:1259 +0x22f
testing.(*T).Run·dwrap·21()
/usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 46 (finished) created at:
github.com/hashicorp/packer/internal/registry.(*Bucket).PopulateIteration()
/Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7
github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds()
/Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7
testing.tRunner()
/usr/local/go/src/testing/testing.go:1259 +0x22f
testing.(*T).Run·dwrap·21()
/usr/local/go/src/testing/testing.go:1306 +0x47
==================
```
* Add methods for managing builds on an Iteration
* Update Mock Service to be a bit more concurrent
It is a simple set of changes to make it work for the current testing
use cases. If we need to we can move the Called fields to counters or
re-architect the mock.
This change also fixes a race condition when appending to the Slice of
errs when calling PopulateIteration.
* Update test case descriptions
* Apply suggestions from code review
Only append to errs if err is not nil
Co-authored-by: Sylvia Moss <[email protected]>
* Fix linting issues
Co-authored-by: Sylvia Moss <[email protected]>
* Rename mock bucket test file
* Add failing tests for reproducing #11573
```
=== RUN TestBucket_CreateInitialBuildForIteration
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_CreateInitialBuildForIteration (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
types.bucket_test.go:87: expected the initial build to have an additional build label but thee is no diff: ""
--- FAIL: TestBucket_UpdateLabelsForBuild (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image
2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image2
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal
--- FAIL: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s)
FAIL
FAIL github.com/hashicorp/packer/internal/registry 0.646s
```
* Update CreateIntialBuildForIteration to initialize each build with a new map
Previously upon creating the initial build the same map, which was
initialized for the build_labels argument was being shared across all
build images. This was causing an issue with labels being backed by the
same map for all builds. This change ensures that all builds get their
own map with any global build labels copied over during the initial creation.
Closes#11573
Passing tests with changes on branch
```
RUN TestBucket_CreateInitialBuildForIteration
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_CreateInitialBuildForIteration (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
--- PASS: TestBucket_UpdateLabelsForBuild (0.00s)
=== RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image
2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image2
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine version:1.7.0]
types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:the-original-happycloud-image version:1.7.0]
against global build labels: map[based_off:alpine version:1.7.0]
--- PASS: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s)
```
* Handle errors from bucket methods
* Update test cases
Initialize maps for bucket when calling NewBucketWithIteration
On a local developer machine there is a good chance the HCP_ORG_ID and
HCP_PROJECT_ID are not set since the HCP Packer client can derive this
information using the token/secret. This change initializes the location
structure with the information obtained from the client and not the
envs.
Results of tests before change with no HCP_ORG_ID or HCP_PROJECT_ID set
```
par_test.go:185: (*packer_service.PackerServiceGetIterationOK)(nil)
par_test.go:185: &{0 [] } (*models.GrpcGatewayRuntimeError) is not supported by the TextConsumer, can be resolved by supporting TextUnmarshaler interface
--- FAIL: TestAcc_PAR_pkr_build (3.94s)
FAIL
FAIL github.com/hashicorp/packer/internal/registry/acctest 6.446s
testing: warning: no tests to run
PASS
ok github.com/hashicorp/packer/internal/registry/env 0.318s [no tests to run]
FAIL
```
Results of tests after change with no HCP_ORG_ID or HCP_PROJECT_ID set
```
2022/01/18 12:23:48 ui: --> null.example: Published metadata to HCP Packer registry packer/pkr-acctest-temp-2/iterations/01FSQ3EPPY69PMXJSTTFYDPQ5R
--- PASS: TestAcc_PAR_pkr_build (3.20s)
PASS
ok github.com/hashicorp/packer/internal/registry/acctest 5.121s
testing: warning: no tests to run
PASS
ok github.com/hashicorp/packer/internal/registry/env 0.155s [no tests to run]
```
* Add basic support for build_labels argument
* Update support for build_labels configuration argument
* Update complete test-fixture with a build_labels configuration
* Add test for deprecated labels argument
* Add deprecation for hcp_packer_registry.labels
When using the now deprecated labels argument of the new bucket_labels a
Warning will be presented to the user.
```
~> HCP_PACKER_BUILD_FINGERPRINT=356786543567865456789656789 packer
build source.pkr.hcl
Warning: the argument hcp_packer_registry.labels has been deprecated and will be removed in a future release; please use hcp_packer_registry.bucket_labels
```
When trying to use both bucket_labels and labels together an error is
presented to the user.
```
~> HCP_PACKER_BUILD_FINGERPRINT=ss6786543567865456789656789 packer
build source.pkr.hcl
Error: hcp_packer_registry.labels and hcp_packer_registry.bucket_labels are mutely exclusive; please use the recommended argument hcp_packer_registry.bucket_labels
on source.pkr.hcl line 17:
(source code not available)
```
* Update documentation for build_labels
* Apply suggestions from code review
Co-authored-by: Adrien Delorme <[email protected]>
* Update hcl2template/types.build.hcp_packer_registry.go
Co-authored-by: Adrien Delorme <[email protected]>
* Update handling of registry artifacts
This change uses the github.com/hashicorp/packer-plgin-sdk/packer/registryimage for querying
Artifact State for HCP Registry Image metadata. To handle the conversion
of the RPC response, mapstructure was introduced to conversion state
data into an registryimage.Image before publishing to a image bucket.
* Update to use registry image from packersdk
* Rename internal registry service pkg
* Update vendored plugins to latest version
* The latest release of Amazon, GoogleCompute, and Azure have support
for publishing images to the HCP Packer registry.
* Add working registry pkg
* Add custom error for handling the loading of PAR environment variables
* Working Publish to Build, with proper error handling for bucket names
* Update hcp-sdk-go to use branch instead of mod replace directive
* Update Packer build status configuration
* Add support for HCP_PACKER_BUILD_FINGERPRINT env
* Add support for publishing one or more PARtifacts from a single build
* add git shas to this branch
* Add ability to set provider name if available
* Add working RegistryBuilder type
* Add RegistryPostProcessor as wrapper post-processor
* When in PAR mode a empty RegistryPostProcessor is added to the end of
the post-processor list to publish all final image data.
* Add support for updating a build from PAR that is not in a DONE state
* Fix a small issue with creation the initial builds for an empty
iteration.
* Add PAR URL to post-processor display
* Implement hcp_packer_registry block (#11168)
* Update vendored Amazon plugin to v1.0.1-dev
* Fix panic when running a Packer registry build in a clean directory
* Remove the publishing of post-processor metadata from the registry
post-processor.
* Remove metadata add from registry_builder
* Update registry builder to skip a build that was found to be DONE
Co-authored-by: Megan Marsh <[email protected]>
Co-authored-by: Sylvia Moss <[email protected]>
Sylvia Moss
Wilken RiveraandSylvia Moss
Adrien Delorme
Megan Marsh