Revert "Check that only certain files are executable"

CHANGELOG.md

@@ -1,34 +1,14 @@
-## 1.3.1 (September 13, 2018)
-### IMPROVEMENTS:
-* builder/amazon: automatically decode encoded authorization messages if
-    possible [GH-5415]
-* builder:amazon: Optional cleanup of the authorized keys file [GH-6713]
-
-### BUG FIXES:
-* builder/amazon: fix bugs relating to spot instances provisioning [GH-6697]
-    [GH-6693]
-* builder/openstack: fix ssh keypair not attached [GH-6701]
-* core: progressbar: fix deadlock locking builds afer first display [GH-6698]
-
-## 1.3.0 (September 11, 2018)
+## 1.2.6 (Unreleased)
 
 ### IMPROVEMENTS:
-* azure/arm: Retry cleanup of individual resources on error [GH-6644]
+
 * builder/alicloud: Support source image coming from marketplace [GH-6588]
-* builder/amazon-chroot: Add new `root_volume_type` option. [GH-6669]
-* builder/amazon-chroot: If you have a PV source AMI, with the Amazon Chroot
-    builder, and the destination AMI is type HVM, you can now enable
-    ena_support, example: [GH-6670]
 * builder/amazon-chroot: New feature `root_volume_tags` to tag the created
     volumes. [GH-6504]
-* builder/amazon: Create a random interim AMI name when encrypt_boot is true so
-    that ami name is not searchable. [GH-6657]
 * builder/azure: Implement clean_image_name template engine. [GH-6558]
 * builder/cloudstack: Add option to use a fixed port via public_port. [GH-6532]
 * builder/digitalocean: Add support for tagging to instances [GH-6546]
 * builder/googlecompute: Add new `min_cpu_platform` feature [GH-6607]
-* builder/googlecompute: Update the list of public image projects that we
-    search, based on GCE documentation. [GH-6648]
 * builder/lxc: Allow unplivileged LXC containers. [GH-6279]
 * builder/oci: Add `metadata` feature to Packer config. [GH-6498]
 * builder/openstack: Add support for getting config from clouds-public.yaml.
@@ -38,77 +18,34 @@
 * builder/openstack: Migrate floating IP usage to Network v2 API from Compute
     API. [GH-6373]
 * builder/openstack: Support Block Storage volumes as boot volume. [GH-6596]
-* builder/oracle-oci: Add support for freeform tagging of OCI images [GH-6338]
 * builder/qemu: add ssh agent support. [GH-6541]
 * builder/qemu: New `use_backing_file` feature [GH-6249]
-* builder/vmware-iso: Add support for disk compaction [GH-6411]
 * builder/vmware-iso: Try to use ISO files uploaded to the datastore when
     building remotely instead of uploading them freshly every time [GH-5165]
 * command/validate: Warn users if config needs fixing. [GH-6423]
-* core: Add a 'split' function to parse template variables. [GH-6357]
-* core: Add a template function allowing users to read keys from consul
-    [GH-6577]
-* core: Add a template function allowing users to read keys from vault
-    [GH-6533]
 * core: Add progress-bar to download step. [GH-5851]
-* core: Create a new root-level Packer template option, "sensitive-variables"
-    which allows users to list which variables they would like to have scrubbed
-    from the Packer logs. [GH-6610]
-* core: Create new config options, "boot_keygroup_interval" and
-    "boot_key_interval" that can be set at the builder-level to supercede
-    PACKER_KEY_INTERVAL for the bootcommand. [GH-6616]
 * core: Deduplicate ui and log lines that stream to terminal [GH-6611]
-* core: Refactor and deduplicate ssh code across builders. This should be a no-
-    op but is a big win for maintainability. [GH-6621] [GH-6613]
-* post-processor/compress: Add support for xz compression [GH-6534]
 * post-processor/vagrant: Support for Docker images. [GH-6494]
-* post-processor/vsphere: Add new `esxi_host` option. [GH-5366]
 * postprocessor/vagrant: Add support for Azure. [GH-6576]
-* provisioner/ansible: Add new "extra var", packer_http_addr. [GH-6501]
 * provisioner/ansible: Enable {{.WinRMPassword}} template engine. [GH-6450]
 * provisioner/shell-local: Create PACKER_HTTP_ADDR environment variable
     [GH-6503]
 
-
 ### BUG FIXES:
 * builder/amazon-ebssurrogate: Clean up volumes at end of build. [GH-6514]
-* builder/amazon: Increase default waiter timeout for AWS
-    WaitUntilImageAvailable command [GH-6601]
-* builder/amazon: Increase the MaxRetries in the Amazon client from the default
-    to 20, to work around users who regularly reach their requestlimit and are
-    being throttled. [GH-6641]
-* builder/amazon: Properly apply environment overrides to our custom-written
-    waiters. [GH-6649]
 * builder/azure: Generated password satisfies Azure password requirements
     [GH-6480]
-* builder/hyper-v: Buider no longer errors if skip_compaction isn't true when
-    skip_export is true, and compaction efficiency is improved [GH-6393]
 * builder/lxc: Correctly pass "config" option to "lxc launch". [GH-6563]
-* builder/lxc: Determine lxc root according to the running user [GH-6543]
-* builder/lxc: Fix file copying for unprivileged LXC containers [GH-6544]
-* builder/oracle-oci: Update OCI sdk, fixing validation bug that occurred when
-    RSA key was encrypted. [GH-6492]
 * builder/vmware-iso: Fix crash caused by invalid datacenter url. [GH-6529]
-* builder/vmware: Maintain original boot order during CreateVMX step for
-    vmware-iso builder [GH-6204]
-* communicator/chroot: Fix quote escaping so that ansible provisioner works
-    properly. [GH-6635]
 * core: Better error handling in downloader when connection error occurs.
     [GH-6557]
 * core: Fix broken pathing checks in checksum files. [GH-6525]
-* provisioner/shell Create new template option allowing users to choose to
-    source env vars from a file rather than declaring them inline. This
-    resolves a bug that occurred when users had complex quoting in their
-    `execute_command`s [GH-6636]
-* provisioner/shell-local: Windows inline scripts now default to being appended
-    with ".cmd", fixing a backwards incompatibility in v1.2.5 [GH-6626]
 * provisioner/windows-restart: Provisioner now works when used in conjuction
     with SSH communicator [GH-6606]
 
 ### BACKWARDS INCOMPATIBILITIES:
 * builder/amazon: "owners" field on source_ami_filter is now required for
     secuirty reasons. [GH-6585]
-* builder/vmware-iso: validation will fail for templates using esxi that have the "disk_type_id" set to something other than "thin" or "" and that do not have "skip_compaction": true also set. Use `packer fix` to fix this. [GH-6411]
 
 ## 1.2.5 (July 16, 2018)
 

Makefile

@@ -11,8 +11,6 @@ GOPATH=$(shell go env GOPATH)
 # gofmt
 UNFORMATTED_FILES=$(shell find . -not -path "./vendor/*" -name "*.go" | xargs gofmt -s -l)
 
-EXECUTABLE_FILES=$(shell find . -type f -perm +111 | egrep -v '^\./(website/vendor|vendor/|\.git|bin/|scripts/|pkg/)' | egrep -v '.*(\.sh|\.bats|\.git)' | egrep -v './provisioner/ansible/test-fixtures/exit1')
-
 # Get the git commit
 GIT_DIRTY=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
 GIT_COMMIT=$(shell git rev-parse --short HEAD)
@@ -76,15 +74,6 @@ fmt-check: ## Check go code formatting
 		echo "Check passed."; \
 	fi
 
-mode-check: ## Check that only certain files are executable
-	@echo "==> Checking that only certain files are executable..."
-	@if [ ! -z "$(EXECUTABLE_FILES)" ]; then \
-		echo "These files should not be executable or they must be white listed in the Makefile:"; \
-		echo "$(EXECUTABLE_FILES)" | xargs -n1; \
-		exit 1; \
-	else \
-		echo "Check passed."; \
-	fi
 fmt-docs:
 	@find ./website/source/docs -name "*.md" -exec pandoc --wrap auto --columns 79 --atx-headers -s -f "markdown_github+yaml_metadata_block" -t "markdown_github+yaml_metadata_block" {} -o {} \;
 
@@ -100,7 +89,7 @@ generate: deps ## Generate dynamically generated code
 	goimports -w common/bootcommand/boot_command.go
 	gofmt -w command/plugin.go
 
-test: deps fmt-check mode-check ## Run unit tests
+test: deps fmt-check ## Run unit tests
 	@go test $(TEST) $(TESTARGS) -timeout=2m
 	@go tool vet $(VET)  ; if [ $$? -eq 1 ]; then \
 		echo "ERROR: Vet found problems in the code."; \

builder/alicloud/ecs/builder.go

@@ -157,9 +157,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&stepStopAlicloudInstance{
 			ForceStop: b.config.ForceStopInstance,
 		},

builder/amazon/chroot/builder.go

@@ -42,7 +42,6 @@ type Config struct {
 	PreMountCommands  []string                   `mapstructure:"pre_mount_commands"`
 	RootDeviceName    string                     `mapstructure:"root_device_name"`
 	RootVolumeSize    int64                      `mapstructure:"root_volume_size"`
-	RootVolumeType    string                     `mapstructure:"root_volume_type"`
 	SourceAmi         string                     `mapstructure:"source_ami"`
 	SourceAmiFilter   awscommon.AmiFilterOptions `mapstructure:"source_ami_filter"`
 	RootVolumeTags    awscommon.TagMap           `mapstructure:"root_volume_tags"`
@@ -224,7 +223,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 				EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
 				EnableAMIENASupport:      b.config.AMIENASupport,
 				AmiFilters:               b.config.SourceAmiFilter,
-				AMIVirtType:              b.config.AMIVirtType,
 			},
 			&StepCheckRootDevice{},
 		)
@@ -234,7 +232,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		&StepFlock{},
 		&StepPrepareDevice{},
 		&StepCreateVolume{
-			RootVolumeType: b.config.RootVolumeType,
 			RootVolumeSize: b.config.RootVolumeSize,
 			RootVolumeTags: b.config.RootVolumeTags,
 			Ctx:            b.config.ctx,

builder/amazon/chroot/step_create_volume.go

@@ -2,7 +2,6 @@ package chroot
 
 import (
 	"context"
-	"errors"
 	"fmt"
 	"log"
 
@@ -22,7 +21,6 @@ import (
 type StepCreateVolume struct {
 	volumeId       string
 	RootVolumeSize int64
-	RootVolumeType string
 	RootVolumeTags awscommon.TagMap
 	Ctx            interpolate.Context
 }
@@ -55,21 +53,11 @@ func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) mu
 
 	var createVolume *ec2.CreateVolumeInput
 	if config.FromScratch {
-		rootVolumeType := ec2.VolumeTypeGp2
-		if s.RootVolumeType == "io1" {
-			err := errors.New("Cannot use io1 volume when building from scratch")
-			state.Put("error", err)
-			ui.Error(err.Error())
-			return multistep.ActionHalt
-		} else if s.RootVolumeType != "" {
-			rootVolumeType = s.RootVolumeType
-		}
 		createVolume = &ec2.CreateVolumeInput{
 			AvailabilityZone: instance.Placement.AvailabilityZone,
 			Size:             aws.Int64(s.RootVolumeSize),
-			VolumeType:       aws.String(rootVolumeType),
+			VolumeType:       aws.String(ec2.VolumeTypeGp2),
 		}
-
 	} else {
 		// Determine the root device snapshot
 		image := state.Get("source_image").(*ec2.Image)
@@ -82,13 +70,26 @@ func (s *StepCreateVolume) Run(ctx context.Context, state multistep.StateBag) mu
 			}
 		}
 
-		ui.Say("Creating the root volume...")
-		createVolume, err = s.buildCreateVolumeInput(*instance.Placement.AvailabilityZone, rootDevice)
-		if err != nil {
+		if rootDevice == nil {
+			err := fmt.Errorf("Couldn't find root device!")
 			state.Put("error", err)
 			ui.Error(err.Error())
 			return multistep.ActionHalt
 		}
+
+		ui.Say("Creating the root volume...")
+		vs := *rootDevice.Ebs.VolumeSize
+		if s.RootVolumeSize > *rootDevice.Ebs.VolumeSize {
+			vs = s.RootVolumeSize
+		}
+
+		createVolume = &ec2.CreateVolumeInput{
+			AvailabilityZone: instance.Placement.AvailabilityZone,
+			Size:             aws.Int64(vs),
+			SnapshotId:       rootDevice.Ebs.SnapshotId,
+			VolumeType:       rootDevice.Ebs.VolumeType,
+			Iops:             rootDevice.Ebs.Iops,
+		}
 	}
 
 	if len(tagSpecs) > 0 {
@@ -136,33 +137,3 @@ func (s *StepCreateVolume) Cleanup(state multistep.StateBag) {
 		ui.Error(fmt.Sprintf("Error deleting EBS volume: %s", err))
 	}
 }
-
-func (s *StepCreateVolume) buildCreateVolumeInput(az string, rootDevice *ec2.BlockDeviceMapping) (*ec2.CreateVolumeInput, error) {
-	if rootDevice == nil {
-		return nil, fmt.Errorf("Couldn't find root device!")
-	}
-	createVolumeInput := &ec2.CreateVolumeInput{
-		AvailabilityZone: aws.String(az),
-		Size:             rootDevice.Ebs.VolumeSize,
-		SnapshotId:       rootDevice.Ebs.SnapshotId,
-		VolumeType:       rootDevice.Ebs.VolumeType,
-		Iops:             rootDevice.Ebs.Iops,
-	}
-	if s.RootVolumeSize > *rootDevice.Ebs.VolumeSize {
-		createVolumeInput.Size = aws.Int64(s.RootVolumeSize)
-	}
-
-	if s.RootVolumeType == "" || s.RootVolumeType == *rootDevice.Ebs.VolumeType {
-		return createVolumeInput, nil
-	}
-
-	if s.RootVolumeType == "io1" {
-		return nil, fmt.Errorf("Root volume type cannot be io1, because existing root volume type was %s", *rootDevice.Ebs.VolumeType)
-	}
-
-	createVolumeInput.VolumeType = aws.String(s.RootVolumeType)
-	// non io1 cannot set iops
-	createVolumeInput.Iops = nil
-
-	return createVolumeInput, nil
-}

builder/amazon/chroot/step_create_volume_test.go

@@ -1,73 +0,0 @@
-package chroot
-
-import (
-	"github.com/aws/aws-sdk-go/aws"
-	"github.com/aws/aws-sdk-go/service/ec2"
-	"github.com/stretchr/testify/assert"
-	"testing"
-)
-
-func buildTestRootDevice() *ec2.BlockDeviceMapping {
-	return &ec2.BlockDeviceMapping{
-		Ebs: &ec2.EbsBlockDevice{
-			VolumeSize: aws.Int64(10),
-			SnapshotId: aws.String("snap-1234"),
-			VolumeType: aws.String("gp2"),
-		},
-	}
-}
-
-func TestCreateVolume_Default(t *testing.T) {
-	stepCreateVolume := new(StepCreateVolume)
-	_, err := stepCreateVolume.buildCreateVolumeInput("test-az", buildTestRootDevice())
-	assert.NoError(t, err)
-}
-
-func TestCreateVolume_Shrink(t *testing.T) {
-	stepCreateVolume := StepCreateVolume{RootVolumeSize: 1}
-	testRootDevice := buildTestRootDevice()
-	ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
-	assert.NoError(t, err)
-	// Ensure that the new value is equal to the size of the old root device
-	assert.Equal(t, *ret.Size, *testRootDevice.Ebs.VolumeSize)
-}
-
-func TestCreateVolume_Expand(t *testing.T) {
-	stepCreateVolume := StepCreateVolume{RootVolumeSize: 25}
-	testRootDevice := buildTestRootDevice()
-	ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
-	assert.NoError(t, err)
-	// Ensure that the new value is equal to the size of the value passed in
-	assert.Equal(t, *ret.Size, stepCreateVolume.RootVolumeSize)
-}
-
-func TestCreateVolume_io1_to_io1(t *testing.T) {
-	stepCreateVolume := StepCreateVolume{RootVolumeType: "io1"}
-	testRootDevice := buildTestRootDevice()
-	testRootDevice.Ebs.VolumeType = aws.String("io1")
-	testRootDevice.Ebs.Iops = aws.Int64(1000)
-	ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
-	assert.NoError(t, err)
-	assert.Equal(t, *ret.VolumeType, stepCreateVolume.RootVolumeType)
-	assert.Equal(t, *ret.Iops, *testRootDevice.Ebs.Iops)
-}
-
-func TestCreateVolume_io1_to_gp2(t *testing.T) {
-	stepCreateVolume := StepCreateVolume{RootVolumeType: "gp2"}
-	testRootDevice := buildTestRootDevice()
-	testRootDevice.Ebs.VolumeType = aws.String("io1")
-	testRootDevice.Ebs.Iops = aws.Int64(1000)
-
-	ret, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
-	assert.NoError(t, err)
-	assert.Equal(t, *ret.VolumeType, stepCreateVolume.RootVolumeType)
-	assert.Nil(t, ret.Iops)
-}
-
-func TestCreateVolume_gp2_to_io1(t *testing.T) {
-	stepCreateVolume := StepCreateVolume{RootVolumeType: "io1"}
-	testRootDevice := buildTestRootDevice()
-
-	_, err := stepCreateVolume.buildCreateVolumeInput("test-az", testRootDevice)
-	assert.Error(t, err)
-}

builder/amazon/common/access_config.go

@@ -19,7 +19,6 @@ import (
 type AccessConfig struct {
 	AccessKey            string `mapstructure:"access_key"`
 	CustomEndpointEc2    string `mapstructure:"custom_endpoint_ec2"`
-	DecodeAuthZMessages  bool   `mapstructure:"decode_authorization_messages"`
 	MFACode              string `mapstructure:"mfa_code"`
 	ProfileName          string `mapstructure:"profile"`
 	RawRegion            string `mapstructure:"region"`
@@ -44,9 +43,6 @@ func (c *AccessConfig) Session() (*session.Session, error) {
 		config.WithCredentials(staticCreds)
 	}
 
-	// default is 3, and when it was causing failures for users being throttled
-	config = config.WithMaxRetries(20)
-
 	if c.RawRegion != "" {
 		config = config.WithRegion(c.RawRegion)
 	} else if region := c.metadataRegion(); region != "" {
@@ -92,11 +88,6 @@ func (c *AccessConfig) Session() (*session.Session, error) {
 		}
 		log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName)
 	}
-
-	if c.DecodeAuthZMessages {
-		DecodeAuthZMessages(c.session)
-	}
-
 	return c.session, nil
 }
 

builder/amazon/common/errors.go

@@ -1,58 +0,0 @@
-package common
-
-import (
-	"fmt"
-	"log"
-	"regexp"
-
-	"github.com/aws/aws-sdk-go/aws/awserr"
-
-	"github.com/aws/aws-sdk-go/aws"
-	"github.com/aws/aws-sdk-go/aws/request"
-	"github.com/aws/aws-sdk-go/aws/session"
-	"github.com/aws/aws-sdk-go/service/sts"
-)
-
-var encodedFailureMessagePattern = regexp.MustCompile(`(?i)(.*) Encoded authorization failure message: ([\w-]+) ?( .*)?`)
-
-type stsDecoder interface {
-	DecodeAuthorizationMessage(input *sts.DecodeAuthorizationMessageInput) (*sts.DecodeAuthorizationMessageOutput, error)
-}
-
-// decodeError replaces encoded authorization messages with the
-// decoded results
-func decodeAWSError(decoder stsDecoder, err error) error {
-
-	groups := encodedFailureMessagePattern.FindStringSubmatch(err.Error())
-	if groups != nil && len(groups) > 1 {
-		result, decodeErr := decoder.DecodeAuthorizationMessage(&sts.DecodeAuthorizationMessageInput{
-			EncodedMessage: aws.String(groups[2]),
-		})
-		if decodeErr == nil {
-			msg := aws.StringValue(result.DecodedMessage)
-			return fmt.Errorf("%s Authorization failure message: '%s'%s", groups[1], msg, groups[3])
-		}
-		log.Printf("[WARN] Attempted to decode authorization message, but received: %v", decodeErr)
-	}
-	return err
-}
-
-// DecodeAuthZMessages enables automatic decoding of any
-// encoded authorization messages
-func DecodeAuthZMessages(sess *session.Session) {
-	azd := &authZMessageDecoder{
-		Decoder: sts.New(sess),
-	}
-	sess.Handlers.UnmarshalError.AfterEachFn = azd.afterEachFn
-}
-
-type authZMessageDecoder struct {
-	Decoder stsDecoder
-}
-
-func (a *authZMessageDecoder) afterEachFn(item request.HandlerListRunItem) bool {
-	if err, ok := item.Request.Error.(awserr.Error); ok && err.Code() == "UnauthorizedOperation" {
-		item.Request.Error = decodeAWSError(a.Decoder, err)
-	}
-	return true
-}

builder/amazon/common/errors_test.go

@@ -1,70 +0,0 @@
-package common
-
-import (
-	"fmt"
-	"strings"
-	"testing"
-
-	"github.com/aws/aws-sdk-go/aws"
-	"github.com/aws/aws-sdk-go/service/sts"
-
-	"github.com/aws/aws-sdk-go/aws/awserr"
-)
-
-type mockSTS struct {
-}
-
-func (m *mockSTS) DecodeAuthorizationMessage(input *sts.DecodeAuthorizationMessageInput) (*sts.DecodeAuthorizationMessageOutput, error) {
-	return &sts.DecodeAuthorizationMessageOutput{
-		DecodedMessage: aws.String(`{
-			"allowed": false,
-			"explicitDeny": true,
-			"matchedStatements": {}
-		}`),
-	}, nil
-}
-
-func TestErrorsParsing_RequestFailure(t *testing.T) {
-
-	ae := awserr.New("UnauthorizedOperation",
-		`You are not authorized to perform this operation. Encoded authorization failure message: D9Q7oicjOMr9l2CC-NPP1FiZXK9Ijia1k-3l0siBFCcrK3oSuMFMkBIO5TNj0HdXE-WfwnAcdycFOohfKroNO6toPJEns8RFVfy_M_IjNGmrEFJ6E62pnmBW0OLrMsXxR9FQE4gB4gJzSM0AD6cV6S3FOfqYzWBRX-sQdOT4HryGkFNRoFBr9Xbp-tRwiadwkbdHdfnV9fbRkXmnwCdULml16NBSofC4ZPepLMKmIB5rKjwk-m179UUh2XA-J5no0si6XcRo5GbHQB5QfCIwSHL4vsro2wLZUd16-8OWKyr3tVlTbQe0ERZskqRqRQ5E28QuiBCVV6XstUyo-T4lBSr75Fgnyr3wCO-dS3b_5Ns3WzA2JD4E2AJOAStXIU8IH5YuKkAg7C-dJMuBMPpmKCBEXhNoHDwCyOo5PsV3xMlc0jSb0qYGpfst_TDDtejcZfn7NssUjxVq9qkdH-OXz2gPoQB-hX8ycmZCL5UZwKc3TCLUr7TGnudHjmnMrE9cUo-yTCWfyHPLprhiYhTCKW18EikJ0O1EKI3FJ_b4F19_jFBPARjSwQc7Ut6MNCVzrPdZGYSF6acj5gPaxdy9uSkVQwWXK7Pd5MFP7EBDE1_DgYbzodgwDO2PXeVFUbSLBHKWo_ebZS9ZX2nYPcGss_sYaly0ZVSIJXp7G58B5BoFVhvVH6jYnF9XiAOjMltuP_ycu1pQP1lki500RY3baLvfeYeAsB38XZHKEgWZzq7Fei-uh89q0cjJTmlVyrfRU3q6`,
-		fmt.Errorf("You can't do it!!"))
-	rf := awserr.NewRequestFailure(ae, 400, "abc-def-123-456")
-
-	result := decodeAWSError(&mockSTS{}, rf)
-	if result == nil {
-		t.Error("Expected resulting error")
-	}
-	if !strings.Contains(result.Error(), "Authorization failure message:") {
-		t.Error("Expected authorization failure message")
-	}
-}
-
-func TestErrorsParsing_NonAuthorizationFailure(t *testing.T) {
-
-	ae := awserr.New("BadRequest",
-		`You did something wrong. Try again`,
-		fmt.Errorf("Request was no good."))
-	rf := awserr.NewRequestFailure(ae, 400, "abc-def-123-456")
-
-	result := decodeAWSError(&mockSTS{}, rf)
-	if result == nil {
-		t.Error("Expected resulting error")
-	}
-	if result != rf {
-		t.Error("Expected original error to be returned unchanged")
-	}
-}
-
-func TestErrorsParsing_NonAWSError(t *testing.T) {
-
-	err := fmt.Errorf("Random error occurred")
-
-	result := decodeAWSError(&mockSTS{}, err)
-	if result == nil {
-		t.Error("Expected resulting error")
-	}
-	if result != err {
-		t.Error("Expected original error to be returned unchanged")
-	}
-}

builder/amazon/common/run_config.go

@@ -34,7 +34,6 @@ func (d *AmiFilterOptions) NoOwner() bool {
 type RunConfig struct {
 	AssociatePublicIpAddress          bool              `mapstructure:"associate_public_ip_address"`
 	AvailabilityZone                  string            `mapstructure:"availability_zone"`
-	BlockDurationMinutes              int64             `mapstructure:"block_duration_minutes"`
 	DisableStopInstance               bool              `mapstructure:"disable_stop_instance"`
 	EbsOptimized                      bool              `mapstructure:"ebs_optimized"`
 	EnableT2Unlimited                 bool              `mapstructure:"enable_t2_unlimited"`
@@ -112,11 +111,6 @@ func (c *RunConfig) Prepare(ctx *interpolate.Context) []error {
 		errs = append(errs, fmt.Errorf("An instance_type must be specified"))
 	}
 
-	if c.BlockDurationMinutes%60 != 0 {
-		errs = append(errs, fmt.Errorf(
-			"block_duration_minutes must be multiple of 60"))
-	}
-
 	if c.SpotPrice == "auto" {
 		if c.SpotPriceAutoProduct == "" {
 			errs = append(errs, fmt.Errorf(

builder/amazon/common/state.go

@@ -169,8 +169,6 @@ func WaitForVolumeToBeAttached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeV
 			return req, nil
 		},
 	}
-	w.ApplyOptions(opts...)
-
 	return w.WaitWithContext(ctx)
 }
 
@@ -200,8 +198,6 @@ func WaitForVolumeToBeDetached(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeV
 			return req, nil
 		},
 	}
-	w.ApplyOptions(opts...)
-
 	return w.WaitWithContext(ctx)
 }
 
@@ -231,8 +227,6 @@ func WaitForImageToBeImported(c *ec2.EC2, ctx aws.Context, input *ec2.DescribeIm
 			return req, nil
 		},
 	}
-	w.ApplyOptions(opts...)
-
 	return w.WaitWithContext(ctx)
 }
 
@@ -287,11 +281,11 @@ func getOverride(varInfo envInfo) envInfo {
 	return varInfo
 }
 func getEnvOverrides() overridableWaitVars {
-	// Load env vars from environment.
+	// Load env vars from environment, and use them to override defaults
 	envValues := overridableWaitVars{
-		envInfo{"AWS_POLL_DELAY_SECONDS", 0, false},
+		envInfo{"AWS_POLL_DELAY_SECONDS", 2, false},
 		envInfo{"AWS_MAX_ATTEMPTS", 0, false},
-		envInfo{"AWS_TIMEOUT_SECONDS", 0, false},
+		envInfo{"AWS_TIMEOUT_SECONDS", 300, false},
 	}
 
 	envValues.awsMaxAttempts = getOverride(envValues.awsMaxAttempts)

builder/amazon/common/step_get_password.go

@@ -12,7 +12,6 @@ import (
 	"time"
 
 	"github.com/aws/aws-sdk-go/service/ec2"
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/communicator"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
@@ -95,15 +94,13 @@ WaitLoop:
 			"Password (since debug is enabled): %s", s.Comm.WinRMPassword))
 	}
 	// store so that we can access this later during provisioning
-
-	commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword, s.BuildName)
+	state.Put("winrm_password", s.Comm.WinRMPassword)
 	packer.LogSecretFilter.Set(s.Comm.WinRMPassword)
 
 	return multistep.ActionContinue
 }
 
 func (s *StepGetPassword) Cleanup(multistep.StateBag) {
-	commonhelper.RemoveSharedStateFile("winrm_password", s.BuildName)
 }
 
 func (s *StepGetPassword) waitForPassword(state multistep.StateBag, cancel <-chan struct{}) (string, error) {

builder/amazon/common/step_run_spot_instance.go

@@ -24,7 +24,6 @@ type StepRunSpotInstance struct {
 	AssociatePublicIpAddress          bool
 	AvailabilityZone                  string
 	BlockDevices                      BlockDevices
-	BlockDurationMinutes              int64
 	Debug                             bool
 	Comm                              *communicator.Config
 	EbsOptimized                      bool
@@ -186,15 +185,11 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
 	if s.Comm.SSHKeyPairName != "" {
 		runOpts.KeyName = &s.Comm.SSHKeyPairName
 	}
-	spotInstanceInput := &ec2.RequestSpotInstancesInput{
-		LaunchSpecification: runOpts,
-		SpotPrice:           &spotPrice,
-	}
-	if s.BlockDurationMinutes != 0 {
-		spotInstanceInput.BlockDurationMinutes = &s.BlockDurationMinutes
-	}
 
-	runSpotResp, err := ec2conn.RequestSpotInstances(spotInstanceInput)
+	runSpotResp, err := ec2conn.RequestSpotInstances(&ec2.RequestSpotInstancesInput{
+		SpotPrice:           &spotPrice,
+		LaunchSpecification: runOpts,
+	})
 	if err != nil {
 		err := fmt.Errorf("Error launching source spot instance: %s", err)
 		state.Put("error", err)

builder/amazon/common/step_source_ami_info.go

@@ -21,7 +21,6 @@ type StepSourceAMIInfo struct {
 	SourceAmi                string
 	EnableAMISriovNetSupport bool
 	EnableAMIENASupport      bool
-	AMIVirtType              string
 	AmiFilters               AmiFilterOptions
 }
 
@@ -106,13 +105,11 @@ func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) mul
 
 	// Enhanced Networking can only be enabled on HVM AMIs.
 	// See http://goo.gl/icuXh5
-	if s.EnableAMIENASupport || s.EnableAMISriovNetSupport {
-		err = s.canEnableEnhancedNetworking(image)
-		if err != nil {
-			state.Put("error", err)
-			ui.Error(err.Error())
-			return multistep.ActionHalt
-		}
+	if (s.EnableAMIENASupport || s.EnableAMISriovNetSupport) && *image.VirtualizationType != "hvm" {
+		err := fmt.Errorf("Cannot enable enhanced networking, source AMI '%s' is not HVM", s.SourceAmi)
+		state.Put("error", err)
+		ui.Error(err.Error())
+		return multistep.ActionHalt
 	}
 
 	state.Put("source_image", image)
@@ -120,16 +117,3 @@ func (s *StepSourceAMIInfo) Run(_ context.Context, state multistep.StateBag) mul
 }
 
 func (s *StepSourceAMIInfo) Cleanup(multistep.StateBag) {}
-
-func (s *StepSourceAMIInfo) canEnableEnhancedNetworking(image *ec2.Image) error {
-	if s.AMIVirtType == "hvm" {
-		return nil
-	}
-	if s.AMIVirtType != "" {
-		return fmt.Errorf("Cannot enable enhanced networking, AMIVirtType '%s' is not HVM", s.AMIVirtType)
-	}
-	if *image.VirtualizationType != "hvm" {
-		return fmt.Errorf("Cannot enable enhanced networking, source AMI '%s' is not HVM", s.SourceAmi)
-	}
-	return nil
-}

builder/amazon/common/step_source_ami_info_test.go

@@ -1,42 +0,0 @@
-package common
-
-import (
-	"github.com/aws/aws-sdk-go/aws"
-	"github.com/aws/aws-sdk-go/service/ec2"
-	"github.com/stretchr/testify/assert"
-	"testing"
-)
-
-func TestStepSourceAmiInfo_PVImage(t *testing.T) {
-	err := new(StepSourceAMIInfo).canEnableEnhancedNetworking(&ec2.Image{
-		VirtualizationType: aws.String("paravirtual"),
-	})
-	assert.Error(t, err)
-}
-
-func TestStepSourceAmiInfo_HVMImage(t *testing.T) {
-	err := new(StepSourceAMIInfo).canEnableEnhancedNetworking(&ec2.Image{
-		VirtualizationType: aws.String("hvm"),
-	})
-	assert.NoError(t, err)
-}
-
-func TestStepSourceAmiInfo_PVImageWithAMIVirtPV(t *testing.T) {
-	stepSourceAMIInfo := StepSourceAMIInfo{
-		AMIVirtType: "paravirtual",
-	}
-	err := stepSourceAMIInfo.canEnableEnhancedNetworking(&ec2.Image{
-		VirtualizationType: aws.String("paravirtual"),
-	})
-	assert.Error(t, err)
-}
-
-func TestStepSourceAmiInfo_PVImageWithAMIVirtHVM(t *testing.T) {
-	stepSourceAMIInfo := StepSourceAMIInfo{
-		AMIVirtType: "hvm",
-	}
-	err := stepSourceAMIInfo.canEnableEnhancedNetworking(&ec2.Image{
-		VirtualizationType: aws.String("paravirtual"),
-	})
-	assert.NoError(t, err)
-}

builder/amazon/ebs/builder.go

@@ -126,9 +126,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			AssociatePublicIpAddress:          b.config.AssociatePublicIpAddress,
 			AvailabilityZone:                  b.config.AvailabilityZone,
 			BlockDevices:                      b.config.BlockDevices,
-			BlockDurationMinutes:              b.config.BlockDurationMinutes,
 			Ctx:                               b.config.ctx,
-			Comm:                              &b.config.RunConfig.Comm,
 			Debug:                             b.config.PackerDebug,
 			EbsOptimized:                      b.config.EbsOptimized,
 			ExpectedRootDevice:                "ebs",
@@ -180,7 +178,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
 			EnableAMIENASupport:      b.config.AMIENASupport,
 			AmiFilters:               b.config.SourceAmiFilter,
-			AMIVirtType:              b.config.AMIVirtType,
 		},
 		&awscommon.StepKeyPair{
 			Debug:        b.config.PackerDebug,
@@ -211,9 +208,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&awscommon.StepStopEBSBackedInstance{
 			Skip:                b.config.IsSpotInstance(),
 			DisableStopInstance: b.config.DisableStopInstance,

builder/amazon/ebs/step_create_ami.go

@@ -7,7 +7,6 @@ import (
 
 	"github.com/aws/aws-sdk-go/service/ec2"
 	awscommon "github.com/hashicorp/packer/builder/amazon/common"
-	"github.com/hashicorp/packer/common/random"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -23,17 +22,10 @@ func (s *stepCreateAMI) Run(ctx context.Context, state multistep.StateBag) multi
 	ui := state.Get("ui").(packer.Ui)
 
 	// Create the image
-	amiName := config.AMIName
-	if config.AMIEncryptBootVolume {
-		// to avoid having a temporary unencrypted
-		// image named config.AMIName
-		amiName = random.AlphaNum(7)
-	}
-
-	ui.Say(fmt.Sprintf("Creating unencrypted AMI %s from instance %s", amiName, *instance.InstanceId))
+	ui.Say(fmt.Sprintf("Creating the AMI: %s", config.AMIName))
 	createOpts := &ec2.CreateImageInput{
 		InstanceId:          instance.InstanceId,
-		Name:                &amiName,
+		Name:                &config.AMIName,
 		BlockDeviceMappings: config.BlockDevices.BuildAMIDevices(),
 	}
 

builder/amazon/ebssurrogate/builder.go

@@ -140,9 +140,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			AssociatePublicIpAddress:          b.config.AssociatePublicIpAddress,
 			AvailabilityZone:                  b.config.AvailabilityZone,
 			BlockDevices:                      b.config.BlockDevices,
-			BlockDurationMinutes:              b.config.BlockDurationMinutes,
 			Ctx:                               b.config.ctx,
-			Comm:                              &b.config.RunConfig.Comm,
 			Debug:                             b.config.PackerDebug,
 			EbsOptimized:                      b.config.EbsOptimized,
 			ExpectedRootDevice:                "ebs",
@@ -197,7 +195,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
 			EnableAMIENASupport:      b.config.AMIENASupport,
 			AmiFilters:               b.config.SourceAmiFilter,
-			AMIVirtType:              b.config.AMIVirtType,
 		},
 		&awscommon.StepKeyPair{
 			Debug:        b.config.PackerDebug,
@@ -228,9 +225,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&awscommon.StepStopEBSBackedInstance{
 			Skip:                b.config.IsSpotInstance(),
 			DisableStopInstance: b.config.DisableStopInstance,

builder/amazon/ebsvolume/builder.go

@@ -124,9 +124,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			AssociatePublicIpAddress:          b.config.AssociatePublicIpAddress,
 			AvailabilityZone:                  b.config.AvailabilityZone,
 			BlockDevices:                      b.config.launchBlockDevices,
-			BlockDurationMinutes:              b.config.BlockDurationMinutes,
 			Ctx:                               b.config.ctx,
-			Comm:                              &b.config.RunConfig.Comm,
 			Debug:                             b.config.PackerDebug,
 			EbsOptimized:                      b.config.EbsOptimized,
 			ExpectedRootDevice:                "ebs",
@@ -203,9 +201,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&awscommon.StepStopEBSBackedInstance{
 			Skip:                b.config.IsSpotInstance(),
 			DisableStopInstance: b.config.DisableStopInstance,

builder/amazon/instance/builder.go

@@ -210,9 +210,7 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			AssociatePublicIpAddress: b.config.AssociatePublicIpAddress,
 			AvailabilityZone:         b.config.AvailabilityZone,
 			BlockDevices:             b.config.BlockDevices,
-			BlockDurationMinutes:     b.config.BlockDurationMinutes,
 			Ctx:                      b.config.ctx,
-			Comm:                     &b.config.RunConfig.Comm,
 			Debug:                    b.config.PackerDebug,
 			EbsOptimized:             b.config.EbsOptimized,
 			IamInstanceProfile:       b.config.IamInstanceProfile,
@@ -258,7 +256,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			EnableAMISriovNetSupport: b.config.AMISriovNetSupport,
 			EnableAMIENASupport:      b.config.AMIENASupport,
 			AmiFilters:               b.config.SourceAmiFilter,
-			AMIVirtType:              b.config.AMIVirtType,
 		},
 		&awscommon.StepKeyPair{
 			Debug:        b.config.PackerDebug,
@@ -286,9 +283,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&StepUploadX509Cert{},
 		&StepBundleVolume{
 			Debug: b.config.PackerDebug,

builder/azure/arm/builder.go

@@ -177,9 +177,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 				SSHConfig: b.config.Comm.SSHConfigFunc(),
 			},
 			&packerCommon.StepProvision{},
-			&packerCommon.StepCleanupTempKeys{
-				Comm: &b.config.Comm,
-			},
 			NewStepGetOSDisk(azureClient, ui),
 			NewStepGetAdditionalDisks(azureClient, ui),
 			NewStepPowerOffCompute(azureClient, ui),

builder/azure/arm/config.go

@@ -22,7 +22,6 @@ import (
 	"github.com/hashicorp/packer/builder/azure/common/constants"
 	"github.com/hashicorp/packer/builder/azure/pkcs12"
 	"github.com/hashicorp/packer/common"
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/communicator"
 	"github.com/hashicorp/packer/helper/config"
 	"github.com/hashicorp/packer/packer"
@@ -360,10 +359,7 @@ func setRuntimeValues(c *Config) {
 	var tempName = NewTempName()
 
 	c.tmpAdminPassword = tempName.AdminPassword
-	// store so that we can access this later during provisioning
-	commonhelper.SetSharedState("winrm_password", c.tmpAdminPassword, c.PackerConfig.PackerBuildName)
 	packer.LogSecretFilter.Set(c.tmpAdminPassword)
-
 	c.tmpCertificatePassword = tempName.CertificatePassword
 	if c.TempComputeName == "" {
 		c.tmpComputeName = tempName.ComputeName

builder/azure/arm/step_delete_resource_group.go

@@ -5,7 +5,6 @@ import (
 	"fmt"
 
 	"github.com/hashicorp/packer/builder/azure/common/constants"
-	retry "github.com/hashicorp/packer/common"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -94,19 +93,11 @@ func (s *StepDeleteResourceGroup) deleteDeploymentResources(ctx context.Context,
 			resourceType,
 			resourceName))
 
-		err := retry.Retry(10, 600, 10, func(attempt uint) (bool, error) {
-			err := deleteResource(ctx, s.client,
-				resourceType,
-				resourceName,
-				resourceGroupName)
-			if err != nil {
-				s.reportIfError(err, resourceName)
-				return false, nil
-			}
-
-			return true, nil
-		})
-
+		err := deleteResource(ctx, s.client,
+			resourceType,
+			resourceName,
+			resourceGroupName)
+		s.reportIfError(err, resourceName)
 		if err = deploymentOperations.Next(); err != nil {
 			return err
 		}

builder/azure/arm/step_save_winrm_password.go

@@ -3,7 +3,6 @@ package arm
 import (
 	"context"
 
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -15,11 +14,10 @@ type StepSaveWinRMPassword struct {
 
 func (s *StepSaveWinRMPassword) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
 	// store so that we can access this later during provisioning
-	commonhelper.SetSharedState("winrm_password", s.Password, s.BuildName)
+	state.Put("winrm_password", s.Password)
 	packer.LogSecretFilter.Set(s.Password)
 	return multistep.ActionContinue
 }
 
 func (s *StepSaveWinRMPassword) Cleanup(multistep.StateBag) {
-	commonhelper.RemoveSharedStateFile("winrm_password", s.BuildName)
 }

builder/azure/arm/tempname.go

@@ -4,7 +4,17 @@ import (
 	"fmt"
 	"strings"
 
-	"github.com/hashicorp/packer/common/random"
+	"github.com/hashicorp/packer/builder/azure/common"
+)
+
+const (
+	TempNameAlphabet = "0123456789bcdfghjklmnpqrstvwxyz"
+
+	numbers   = "0123456789"
+	lowerCase = "abcdefghijklmnopqrstuvwxyz"
+	upperCase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+
+	TempPasswordAlphabet = numbers + lowerCase + upperCase
 )
 
 type TempName struct {
@@ -24,7 +34,7 @@ type TempName struct {
 func NewTempName() *TempName {
 	tempName := &TempName{}
 
-	suffix := random.AlphaNumLower(10)
+	suffix := common.RandomString(TempNameAlphabet, 10)
 	tempName.ComputeName = fmt.Sprintf("pkrvm%s", suffix)
 	tempName.DeploymentName = fmt.Sprintf("pkrdp%s", suffix)
 	tempName.KeyVaultName = fmt.Sprintf("pkrkv%s", suffix)
@@ -36,7 +46,7 @@ func NewTempName() *TempName {
 	tempName.ResourceGroupName = fmt.Sprintf("packer-Resource-Group-%s", suffix)
 
 	tempName.AdminPassword = generatePassword()
-	tempName.CertificatePassword = random.AlphaNum(32)
+	tempName.CertificatePassword = common.RandomString(TempPasswordAlphabet, 32)
 
 	return tempName
 }
@@ -50,16 +60,16 @@ func NewTempName() *TempName {
 func generatePassword() string {
 	var s string
 	for i := 0; i < 100; i++ {
-		s := random.AlphaNum(32)
-		if !strings.ContainsAny(s, random.PossibleNumbers) {
+		s := common.RandomString(TempPasswordAlphabet, 32)
+		if !strings.ContainsAny(s, numbers) {
 			continue
 		}
 
-		if !strings.ContainsAny(s, random.PossibleLowerCase) {
+		if !strings.ContainsAny(s, lowerCase) {
 			continue
 		}
 
-		if !strings.ContainsAny(s, random.PossibleUpperCase) {
+		if !strings.ContainsAny(s, upperCase) {
 			continue
 		}
 

builder/azure/arm/tempname_test.go

@@ -3,8 +3,6 @@ package arm
 import (
 	"strings"
 	"testing"
-
-	"github.com/hashicorp/packer/common/random"
 )
 
 func TestTempNameShouldCreatePrefixedRandomNames(t *testing.T) {
@@ -46,14 +44,14 @@ func TestTempNameShouldCreatePrefixedRandomNames(t *testing.T) {
 func TestTempAdminPassword(t *testing.T) {
 	tempName := NewTempName()
 
-	if !strings.ContainsAny(tempName.AdminPassword, random.PossibleNumbers) {
-		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", random.PossibleNumbers)
+	if !strings.ContainsAny(tempName.AdminPassword, numbers) {
+		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", numbers)
 	}
-	if !strings.ContainsAny(tempName.AdminPassword, random.PossibleLowerCase) {
-		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", random.PossibleLowerCase)
+	if !strings.ContainsAny(tempName.AdminPassword, lowerCase) {
+		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", lowerCase)
 	}
-	if !strings.ContainsAny(tempName.AdminPassword, random.PossibleUpperCase) {
-		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", random.PossibleUpperCase)
+	if !strings.ContainsAny(tempName.AdminPassword, upperCase) {
+		t.Errorf("Expected AdminPassword to contain at least one of '%s'!", upperCase)
 	}
 }
 

builder/azure/common/randomstring.go

@@ -0,0 +1,45 @@
+package common
+
+import (
+	"math/rand"
+	"os"
+	"time"
+)
+
+var pwSymbols = []string{
+	"abcdefghijklmnopqrstuvwxyz",
+	"ABCDEFGHIJKLMNOPQRSTUVWXYZ",
+	"0123456789",
+}
+
+var rnd = rand.New(rand.NewSource(time.Now().UnixNano() + int64(os.Getpid())))
+
+func RandomString(chooseFrom string, length int) (randomString string) {
+	cflen := len(chooseFrom)
+	for i := 0; i < length; i++ {
+		randomString += string(chooseFrom[rnd.Intn(cflen)])
+	}
+	return
+}
+
+func RandomPassword() (password string) {
+	pwlen := 15
+	batchsize := pwlen / len(pwSymbols)
+	pw := make([]byte, 0, pwlen)
+	// choose character set
+	for c := 0; len(pw) < pwlen; c++ {
+		s := RandomString(pwSymbols[c%len(pwSymbols)], rnd.Intn(batchsize-1)+1)
+		pw = append(pw, []byte(s)...)
+	}
+	// truncate
+	pw = pw[:pwlen]
+
+	// permute
+	for c := 0; c < pwlen-1; c++ {
+		i := rnd.Intn(pwlen-c) + c
+		x := pw[c]
+		pw[c] = pw[i]
+		pw[i] = x
+	}
+	return string(pw)
+}

builder/azure/common/randomstring_test.go

@@ -0,0 +1,15 @@
+package common
+
+import (
+	"testing"
+)
+
+func TestRandomPassword_generates_15char_passwords(t *testing.T) {
+	for i := 0; i < 100; i++ {
+		pw := RandomPassword()
+		t.Logf("pw: %v", pw)
+		if len(pw) != 15 {
+			t.Fatalf("len(pw)!=15, but %v: %v (%v)", len(pw), pw, i)
+		}
+	}
+}

builder/cloudstack/builder.go

@@ -82,9 +82,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			WinRMPort: commPort,
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		&stepShutdownInstance{},
 		&stepCreateTemplate{},
 	}

builder/digitalocean/builder.go

@@ -90,9 +90,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		new(stepShutdown),
 		new(stepPowerOff),
 		new(stepSnapshot),

builder/docker/builder.go

@@ -54,9 +54,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			},
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 	}
 
 	if b.config.Discard {

builder/docker/communicator_test.go

@@ -68,8 +68,8 @@ func TestUploadDownload(t *testing.T) {
 	hooks[packer.HookProvision] = []packer.Hook{
 		&packer.ProvisionHook{
 			Provisioners: []*packer.HookedProvisioner{
-				{Provisioner: upload, Config: nil, TypeName: ""},
-				{Provisioner: download, Config: nil, TypeName: ""},
+				{upload, nil, ""},
+				{download, nil, ""},
 			},
 		},
 	}
@@ -157,9 +157,9 @@ func TestLargeDownload(t *testing.T) {
 	hooks[packer.HookProvision] = []packer.Hook{
 		&packer.ProvisionHook{
 			Provisioners: []*packer.HookedProvisioner{
-				{Provisioner: shell, Config: nil, TypeName: ""},
-				{Provisioner: downloadCupcake, Config: nil, TypeName: ""},
-				{Provisioner: downloadBigcake, Config: nil, TypeName: ""},
+				{shell, nil, ""},
+				{downloadCupcake, nil, ""},
+				{downloadBigcake, nil, ""},
 			},
 		},
 	}
@@ -266,10 +266,10 @@ func TestFixUploadOwner(t *testing.T) {
 	hooks[packer.HookProvision] = []packer.Hook{
 		&packer.ProvisionHook{
 			Provisioners: []*packer.HookedProvisioner{
-				{Provisioner: fileProvisioner, Config: nil, TypeName: ""},
-				{Provisioner: dirProvisioner, Config: nil, TypeName: ""},
-				{Provisioner: shellProvisioner, Config: nil, TypeName: ""},
-				{Provisioner: verifyProvisioner, Config: nil, TypeName: ""},
+				{fileProvisioner, nil, ""},
+				{dirProvisioner, nil, ""},
+				{shellProvisioner, nil, ""},
+				{verifyProvisioner, nil, ""},
 			},
 		},
 	}

builder/googlecompute/builder.go

@@ -71,9 +71,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			WinRMConfig: winrmConfig,
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 	}
 	if _, exists := b.config.Metadata[StartupScriptKey]; exists || b.config.StartupScriptFile != "" {
 		steps = append(steps, new(StepWaitStartupScript))

builder/googlecompute/driver_gce.go

@@ -168,27 +168,7 @@ func (d *driverGCE) DeleteDisk(zone, name string) (<-chan error, error) {
 }
 
 func (d *driverGCE) GetImage(name string, fromFamily bool) (*Image, error) {
-	projects := []string{
-		d.projectId,
-		// Public projects, drawn from
-		// https://cloud.google.com/compute/docs/images
-		"centos-cloud",
-		"cos-cloud",
-		"coreos-cloud",
-		"debian-cloud",
-		"rhel-cloud",
-		"rhel-sap-cloud",
-		"suse-cloud",
-		"suse-sap-cloud",
-		"ubuntu-os-cloud",
-		"windows-cloud",
-		"windows-sql-cloud",
-		"gce-uefi-images",
-		"gce-nvme",
-		// misc
-		"google-containers",
-		"opensuse-cloud",
-	}
+	projects := []string{d.projectId, "centos-cloud", "coreos-cloud", "cos-cloud", "debian-cloud", "google-containers", "opensuse-cloud", "rhel-cloud", "suse-cloud", "ubuntu-os-cloud", "windows-cloud", "gce-nvme", "windows-sql-cloud", "rhel-sap-cloud"}
 	var errs error
 	for _, project := range projects {
 		image, err := d.GetImageFromProject(project, name, fromFamily)

builder/googlecompute/step_create_windows_password.go

@@ -13,7 +13,6 @@ import (
 	"os"
 	"time"
 
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -114,7 +113,6 @@ func (s *StepCreateWindowsPassword) Run(_ context.Context, state multistep.State
 	}
 
 	state.Put("winrm_password", data.password)
-	commonhelper.SetSharedState("winrm_password", data.password, c.PackerConfig.PackerBuildName)
 	packer.LogSecretFilter.Set(data.password)
 
 	return multistep.ActionContinue

builder/hyperv/iso/builder.go

@@ -460,11 +460,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		// provision requires communicator to be setup
 		&common.StepProvision{},
 
-		// Remove ephemeral key from authorized_hosts if using SSH communicator
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
-
 		&hypervcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/hyperv/vmcx/builder.go

@@ -489,11 +489,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 		// provision requires communicator to be setup
 		&common.StepProvision{},
 
-		// Remove ephemeral SSH keys, if using
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
-
 		&hypervcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/ncloud/builder.go

@@ -54,9 +54,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 				SSHConfig: b.config.Comm.SSHConfigFunc(),
 			},
 			&common.StepProvision{},
-			&common.StepCleanupTempKeys{
-				Comm: &b.config.Comm,
-			},
 			NewStepStopServerInstance(conn, ui),
 			NewStepCreateServerImage(conn, ui, b.config),
 			NewStepDeleteBlockStorageInstance(conn, ui, b.config),

builder/oneandone/builder.go

@@ -48,9 +48,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		new(stepTakeSnapshot),
 	}
 

builder/openstack/builder.go

@@ -110,7 +110,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			ConfigDrive:           b.config.ConfigDrive,
 			InstanceMetadata:      b.config.InstanceMetadata,
 			UseBlockStorageVolume: b.config.UseBlockStorageVolume,
-			Comm:                  &b.config.Comm,
 		},
 		&StepGetPassword{
 			Debug: b.config.PackerDebug,
@@ -133,9 +132,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.RunConfig.Comm,
-		},
 		&StepStopServer{},
 		&StepDetachVolume{
 			UseBlockStorageVolume: b.config.UseBlockStorageVolume,

builder/openstack/step_run_source_server.go

@@ -9,7 +9,6 @@ import (
 	"github.com/gophercloud/gophercloud/openstack/compute/v2/extensions/bootfromvolume"
 	"github.com/gophercloud/gophercloud/openstack/compute/v2/extensions/keypairs"
 	"github.com/gophercloud/gophercloud/openstack/compute/v2/servers"
-	"github.com/hashicorp/packer/helper/communicator"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -28,7 +27,6 @@ type StepRunSourceServer struct {
 	InstanceMetadata      map[string]string
 	UseBlockStorageVolume bool
 	server                *servers.Server
-	Comm                  *communicator.Config
 }
 
 func (s *StepRunSourceServer) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
@@ -109,7 +107,7 @@ func (s *StepRunSourceServer) Run(_ context.Context, state multistep.StateBag) m
 	}
 
 	// Add keypair to the server create options.
-	keyName := s.Comm.SSHKeyPairName
+	keyName := config.Comm.SSHKeyPairName
 	if keyName != "" {
 		serverOptsExt = keypairs.CreateOptsExt{
 			CreateOptsBuilder: serverOptsExt,

builder/oracle/classic/builder.go

@@ -77,12 +77,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		&stepSnapshot{},
 		&stepListImages{},
 	}

builder/oracle/oci/builder.go

@@ -69,9 +69,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		&stepImage{},
 	}
 

builder/oracle/oci/step_get_default_credentials.go

@@ -5,7 +5,6 @@ import (
 	"fmt"
 	"log"
 
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/communicator"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
@@ -52,8 +51,9 @@ func (s *stepGetDefaultCredentials) Run(ctx context.Context, state multistep.Sta
 	}
 
 	// store so that we can access this later during provisioning
-	commonhelper.SetSharedState("winrm_password", s.Comm.WinRMPassword, s.BuildName)
+	state.Put("winrm_password", s.Comm.WinRMPassword)
 	packer.LogSecretFilter.Set(s.Comm.WinRMPassword)
+
 	return multistep.ActionContinue
 }
 

builder/parallels/iso/builder.go

@@ -209,9 +209,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Ctx:                     b.config.ctx,
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&parallelscommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/parallels/pvm/builder.go

@@ -102,9 +102,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,
 		},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&parallelscommon.StepPrlctl{
 			Commands: b.config.PrlctlPost,
 			Ctx:      b.config.ctx,

builder/profitbricks/builder.go

@@ -45,9 +45,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		new(stepTakeSnapshot),
 	}
 

builder/qemu/builder.go

@@ -403,12 +403,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 	steps = append(steps,
 		new(common.StepProvision),
 	)
-
-	steps = append(steps,
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
-	)
 	steps = append(steps,
 		new(stepShutdown),
 	)

builder/scaleway/builder.go

@@ -61,9 +61,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.Comm,
-		},
 		new(stepShutdown),
 		new(stepSnapshot),
 		new(stepImage),

builder/triton/builder.go

@@ -68,9 +68,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			SSHConfig: b.config.Comm.SSHConfigFunc(),
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &config.Comm,
-		},
 		&StepStopMachine{},
 		&StepCreateImageFromMachine{},
 		&StepDeleteMachine{},

builder/virtualbox/iso/builder.go

@@ -267,9 +267,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Ctx:                b.config.ctx,
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&vboxcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/virtualbox/ovf/builder.go

@@ -128,9 +128,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Ctx:                b.config.ctx,
 		},
 		new(common.StepProvision),
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&vboxcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/vmware/iso/builder.go

@@ -139,14 +139,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
 		b.config.DiskAdapterType = "lsilogic"
 	}
 
-	if !b.config.SkipCompaction {
-		if b.config.RemoteType == "esx5" {
-			if b.config.DiskTypeId == "" {
-				b.config.SkipCompaction = true
-			}
-		}
-	}
-
 	if b.config.DiskTypeId == "" {
 		// Default is growable virtual disk split in 2GB files.
 		b.config.DiskTypeId = "1"
@@ -156,13 +148,6 @@ func (b *Builder) Prepare(raws ...interface{}) ([]string, error) {
 		}
 	}
 
-	if b.config.RemoteType == "esx5" {
-		if b.config.DiskTypeId != "thin" && !b.config.SkipCompaction {
-			errs = packer.MultiErrorAppend(
-				errs, fmt.Errorf("skip_compaction must be 'true' for disk_type_id: %s", b.config.DiskTypeId))
-		}
-	}
-
 	if b.config.GuestOSType == "" {
 		b.config.GuestOSType = "other"
 	}
@@ -368,9 +353,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Ctx:               b.config.ctx,
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&vmwcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

builder/vmware/iso/driver_esx5.go

@@ -46,8 +46,7 @@ func (d *ESX5Driver) Clone(dst, src string, linked bool) error {
 }
 
 func (d *ESX5Driver) CompactDisk(diskPathLocal string) error {
-	diskPath := d.datastorePath(diskPathLocal)
-	return d.sh("vmkfstools", "--punchzero", diskPath)
+	return nil
 }
 
 func (d *ESX5Driver) CreateDisk(diskPathLocal string, size string, adapter_type string, typeId string) error {

builder/vmware/iso/driver_esx5_test.go

@@ -45,10 +45,7 @@ func TestESX5Driver_HostIP(t *testing.T) {
 	expected_host := "127.0.0.1"
 
 	//create mock SSH server
-	listen, err := net.Listen("tcp", fmt.Sprintf("%s:0", expected_host))
-	if err != nil {
-		t.Fatalf("Listen: %s", err)
-	}
+	listen, _ := net.Listen("tcp", fmt.Sprintf("%s:0", expected_host))
 	port := listen.Addr().(*net.TCPAddr).Port
 	defer listen.Close()
 

builder/vmware/vmx/builder.go

@@ -111,9 +111,6 @@ func (b *Builder) Run(ui packer.Ui, hook packer.Hook, cache packer.Cache) (packe
 			Ctx:               b.config.ctx,
 		},
 		&common.StepProvision{},
-		&common.StepCleanupTempKeys{
-			Comm: &b.config.SSHConfig.Comm,
-		},
 		&vmwcommon.StepShutdown{
 			Command: b.config.ShutdownCommand,
 			Timeout: b.config.ShutdownTimeout,

command/build.go

@@ -294,16 +294,16 @@ Usage: packer build [options] TEMPLATE
 
 Options:
 
-  -color=false                  Disable color output (on by default)
-  -debug                        Debug mode enabled for builds
-  -except=foo,bar,baz           Build all builds other than these
-  -only=foo,bar,baz             Build only the specified builds
-  -force                        Force a build to continue if artifacts exist, deletes existing artifacts
-  -machine-readable             Machine-readable output
+  -color=false               Disable color output (on by default)
+  -debug                     Debug mode enabled for builds
+  -except=foo,bar,baz        Build all builds other than these
+  -only=foo,bar,baz          Build only the specified builds
+  -force                     Force a build to continue if artifacts exist, deletes existing artifacts
+  -machine-readable          Machine-readable output
   -on-error=[cleanup|abort|ask] If the build fails do: clean up (default), abort, or ask
-  -parallel=false               Disable parallelization (on by default)
-  -var 'key=value'              Variable for templates, can be used multiple times.
-  -var-file=path                JSON file containing user variables.
+  -parallel=false            Disable parallelization (on by default)
+  -var 'key=value'           Variable for templates, can be used multiple times.
+  -var-file=path             JSON file containing user variables.
 `
 
 	return strings.TrimSpace(helpText)

command/fix.go

@@ -157,8 +157,6 @@ Fixes that are run:
                              Hyper-V ISO builder templates
   hyperv-vmxc-typo           Corrects a typo in the "clone_from_vmxc_path"
                              setting. Replaces with "clone_from_vmcx_path".
-  vmware-compaction          Adds "skip_compaction = true" to "vmware-iso"
-                             builders with incompatible disk_type_id
 
 Options:
 

command/fix_test.go

@@ -2,39 +2,18 @@ package command
 
 import (
 	"path/filepath"
-	"strings"
 	"testing"
-
-	"github.com/hashicorp/packer/packer"
-	"github.com/stretchr/testify/assert"
 )
 
 func TestFix(t *testing.T) {
-	s := &strings.Builder{}
-	ui := &packer.BasicUi{
-		Writer: s,
-	}
 	c := &FixCommand{
 		Meta: testMeta(t),
 	}
 
-	c.Ui = ui
-
 	args := []string{filepath.Join(testFixture("fix"), "template.json")}
 	if code := c.Run(args); code != 0 {
 		fatalCommand(t, c.Meta)
 	}
-	expected := `{
-  "builders": [
-    {
-      "type": "dummy"
-    }
-  ],
-  "push": {
-    "name": "foo/bar"
-  }
-}`
-	assert.Equal(t, expected, strings.TrimSpace(s.String()))
 }
 
 func TestFix_invalidTemplate(t *testing.T) {

common/config.go

@@ -9,8 +9,6 @@ import (
 	"runtime"
 	"strings"
 	"time"
-
-	"github.com/hashicorp/packer/packer"
 )
 
 // PackerKeyEnv is used to specify the key interval (delay) between keystrokes
@@ -43,7 +41,7 @@ func SupportedProtocol(u *url.URL) bool {
 
 	// build a dummy NewDownloadClient since this is the only place that valid
 	// protocols are actually exposed.
-	cli := NewDownloadClient(&DownloadConfig{}, new(packer.NoopUi))
+	cli := NewDownloadClient(&DownloadConfig{}, nil)
 
 	// Iterate through each downloader to see if a protocol was found.
 	ok := false
@@ -175,7 +173,7 @@ func FileExistsLocally(original string) bool {
 
 	// First create a dummy downloader so we can figure out which
 	// protocol to use.
-	cli := NewDownloadClient(&DownloadConfig{}, new(packer.NoopUi))
+	cli := NewDownloadClient(&DownloadConfig{}, nil)
 	d, ok := cli.config.DownloaderMap[u.Scheme]
 	if !ok {
 		return false

common/download.go

@@ -10,17 +10,19 @@ import (
 	"errors"
 	"fmt"
 	"hash"
-	"io"
 	"log"
-	"net/http"
 	"net/url"
 	"os"
 	"path"
-	"path/filepath"
 	"runtime"
 	"strings"
+)
 
-	"github.com/hashicorp/packer/packer"
+// imports related to each Downloader implementation
+import (
+	"io"
+	"net/http"
+	"path/filepath"
 )
 
 // DownloadConfig is the configuration given to instantiate a new
@@ -56,7 +58,8 @@ type DownloadConfig struct {
 
 // A DownloadClient helps download, verify checksums, etc.
 type DownloadClient struct {
-	config *DownloadConfig
+	config     *DownloadConfig
+	downloader Downloader
 }
 
 // HashForType returns the Hash implementation for the given string
@@ -78,24 +81,33 @@ func HashForType(t string) hash.Hash {
 
 // NewDownloadClient returns a new DownloadClient for the given
 // configuration.
-func NewDownloadClient(c *DownloadConfig, ui packer.Ui) *DownloadClient {
+func NewDownloadClient(c *DownloadConfig, bar ProgressBar) *DownloadClient {
+	const mtu = 1500 /* ethernet */ - 20 /* ipv4 */ - 20 /* tcp */
+
+	// If bar is nil, then use a dummy progress bar that doesn't do anything
+	if bar == nil {
+		bar = GetDummyProgressBar()
+	}
+
 	// Create downloader map if it hasn't been specified already.
 	if c.DownloaderMap == nil {
 		c.DownloaderMap = map[string]Downloader{
-			"file":  &FileDownloader{Ui: ui, bufferSize: nil},
-			"http":  &HTTPDownloader{Ui: ui, userAgent: c.UserAgent},
-			"https": &HTTPDownloader{Ui: ui, userAgent: c.UserAgent},
-			"smb":   &SMBDownloader{Ui: ui, bufferSize: nil},
+			"file":  &FileDownloader{progress: bar, bufferSize: nil},
+			"http":  &HTTPDownloader{progress: bar, userAgent: c.UserAgent},
+			"https": &HTTPDownloader{progress: bar, userAgent: c.UserAgent},
+			"smb":   &SMBDownloader{progress: bar, bufferSize: nil},
 		}
 	}
 	return &DownloadClient{config: c}
 }
 
-// Downloader defines what capabilities a downloader should have.
+// A downloader implements the ability to transfer a file, and cancel or resume
+//	it.
 type Downloader interface {
 	Resume()
 	Cancel()
-	ProgressBar() packer.ProgressBar
+	Progress() uint64
+	Total() uint64
 }
 
 // A LocalDownloader is responsible for converting a uri to a local path
@@ -138,17 +150,17 @@ func (d *DownloadClient) Get() (string, error) {
 	var finalPath string
 
 	var ok bool
-	downloader, ok := d.config.DownloaderMap[u.Scheme]
+	d.downloader, ok = d.config.DownloaderMap[u.Scheme]
 	if !ok {
 		return "", fmt.Errorf("No downloader for scheme: %s", u.Scheme)
 	}
 
-	remote, ok := downloader.(RemoteDownloader)
+	remote, ok := d.downloader.(RemoteDownloader)
 	if !ok {
-		return "", fmt.Errorf("Unable to treat uri scheme %s as a Downloader. : %T", u.Scheme, downloader)
+		return "", fmt.Errorf("Unable to treat uri scheme %s as a Downloader. : %T", u.Scheme, d.downloader)
 	}
 
-	local, ok := downloader.(LocalDownloader)
+	local, ok := d.downloader.(LocalDownloader)
 	if !ok && !d.config.CopyFile {
 		d.config.CopyFile = true
 	}
@@ -221,9 +233,11 @@ func (d *DownloadClient) VerifyChecksum(path string) (bool, error) {
 // HTTPDownloader is an implementation of Downloader that downloads
 // files over HTTP.
 type HTTPDownloader struct {
+	current   uint64
+	total     uint64
 	userAgent string
 
-	Ui packer.Ui
+	progress ProgressBar
 }
 
 func (d *HTTPDownloader) Cancel() {
@@ -242,7 +256,8 @@ func (d *HTTPDownloader) Download(dst *os.File, src *url.URL) error {
 		return err
 	}
 
-	var current int64
+	// Reset our progress
+	d.current = 0
 
 	// Make the request. We first make a HEAD request so we can check
 	// if the server supports range queries. If the server/URL doesn't
@@ -286,7 +301,7 @@ func (d *HTTPDownloader) Download(dst *os.File, src *url.URL) error {
 					if _, err = dst.Seek(0, os.SEEK_END); err == nil {
 						req.Header.Set("Range", fmt.Sprintf("bytes=%d-", fi.Size()))
 
-						current = fi.Size()
+						d.current = uint64(fi.Size())
 					}
 				}
 			}
@@ -313,22 +328,23 @@ func (d *HTTPDownloader) Download(dst *os.File, src *url.URL) error {
 		return fmt.Errorf("HTTP error: %s", err.Error())
 	}
 
-	total := current + resp.ContentLength
+	d.total = d.current + uint64(resp.ContentLength)
 
-	bar := d.ProgressBar()
-	bar.Start(total)
-	defer bar.Finish()
-	bar.Add(current)
-
-	body := bar.NewProxyReader(resp.Body)
+	bar := d.progress
+	bar.SetTotal64(int64(d.total))
+	progressBar := bar.Start()
+	progressBar.Set64(int64(d.current))
 
 	var buffer [4096]byte
 	for {
-		n, err := body.Read(buffer[:])
+		n, err := resp.Body.Read(buffer[:])
 		if err != nil && err != io.EOF {
 			return err
 		}
 
+		d.current += uint64(n)
+		progressBar.Set64(int64(d.current))
+
 		if _, werr := dst.Write(buffer[:n]); werr != nil {
 			return werr
 		}
@@ -337,16 +353,36 @@ func (d *HTTPDownloader) Download(dst *os.File, src *url.URL) error {
 			break
 		}
 	}
+	progressBar.Finish()
 	return nil
 }
 
+func (d *HTTPDownloader) Progress() uint64 {
+	return d.current
+}
+
+func (d *HTTPDownloader) Total() uint64 {
+	return d.total
+}
+
 // FileDownloader is an implementation of Downloader that downloads
 // files using the regular filesystem.
 type FileDownloader struct {
 	bufferSize *uint
 
-	active bool
-	Ui     packer.Ui
+	active  bool
+	current uint64
+	total   uint64
+
+	progress ProgressBar
+}
+
+func (d *FileDownloader) Progress() uint64 {
+	return d.current
+}
+
+func (d *FileDownloader) Total() uint64 {
+	return d.total
 }
 
 func (d *FileDownloader) Cancel() {
@@ -413,6 +449,7 @@ func (d *FileDownloader) Download(dst *os.File, src *url.URL) error {
 	}
 
 	/* download the file using the operating system's facilities */
+	d.current = 0
 	d.active = true
 
 	f, err := os.Open(realpath)
@@ -426,37 +463,44 @@ func (d *FileDownloader) Download(dst *os.File, src *url.URL) error {
 	if err != nil {
 		return err
 	}
+	d.total = uint64(fi.Size())
 
-	bar := d.ProgressBar()
-
-	bar.Start(fi.Size())
-	defer bar.Finish()
-	fProxy := bar.NewProxyReader(f)
+	bar := d.progress
+	bar.SetTotal64(int64(d.total))
+	progressBar := bar.Start()
+	progressBar.Set64(int64(d.current))
 
 	// no bufferSize specified, so copy synchronously.
 	if d.bufferSize == nil {
-		_, err = io.Copy(dst, fProxy)
+		var n int64
+		n, err = io.Copy(dst, f)
 		d.active = false
 
+		d.current += uint64(n)
+		progressBar.Set64(int64(d.current))
+
 		// use a goro in case someone else wants to enable cancel/resume
 	} else {
 		errch := make(chan error)
 		go func(d *FileDownloader, r io.Reader, w io.Writer, e chan error) {
 			for d.active {
-				_, err := io.CopyN(w, r, int64(*d.bufferSize))
+				n, err := io.CopyN(w, r, int64(*d.bufferSize))
 				if err != nil {
 					break
 				}
 
+				d.current += uint64(n)
+				progressBar.Set64(int64(d.current))
 			}
 			d.active = false
 			e <- err
-		}(d, fProxy, dst, errch)
+		}(d, f, dst, errch)
 
 		// ...and we spin until it's done
 		err = <-errch
 	}
-
+	progressBar.Finish()
+	f.Close()
 	return err
 }
 
@@ -465,8 +509,19 @@ func (d *FileDownloader) Download(dst *os.File, src *url.URL) error {
 type SMBDownloader struct {
 	bufferSize *uint
 
-	active bool
-	Ui     packer.Ui
+	active  bool
+	current uint64
+	total   uint64
+
+	progress ProgressBar
+}
+
+func (d *SMBDownloader) Progress() uint64 {
+	return d.current
+}
+
+func (d *SMBDownloader) Total() uint64 {
+	return d.total
 }
 
 func (d *SMBDownloader) Cancel() {
@@ -515,6 +570,7 @@ func (d *SMBDownloader) Download(dst *os.File, src *url.URL) error {
 	}
 
 	/* Open up the "\\"-prefixed path using the Windows filesystem */
+	d.current = 0
 	d.active = true
 
 	f, err := os.Open(realpath)
@@ -528,39 +584,43 @@ func (d *SMBDownloader) Download(dst *os.File, src *url.URL) error {
 	if err != nil {
 		return err
 	}
+	d.total = uint64(fi.Size())
 
-	bar := d.ProgressBar()
-
-	bar.Start(fi.Size())
-	defer bar.Finish()
-	fProxy := bar.NewProxyReader(f)
+	bar := d.progress
+	bar.SetTotal64(int64(d.total))
+	progressBar := bar.Start()
+	progressBar.Set64(int64(d.current))
 
 	// no bufferSize specified, so copy synchronously.
 	if d.bufferSize == nil {
-		_, err = io.Copy(dst, fProxy)
+		var n int64
+		n, err = io.Copy(dst, f)
 		d.active = false
 
+		d.current += uint64(n)
+		progressBar.Set64(int64(d.current))
+
 		// use a goro in case someone else wants to enable cancel/resume
 	} else {
 		errch := make(chan error)
 		go func(d *SMBDownloader, r io.Reader, w io.Writer, e chan error) {
 			for d.active {
-				_, err := io.CopyN(w, r, int64(*d.bufferSize))
+				n, err := io.CopyN(w, r, int64(*d.bufferSize))
 				if err != nil {
 					break
 				}
 
+				d.current += uint64(n)
+				progressBar.Set64(int64(d.current))
 			}
 			d.active = false
 			e <- err
-		}(d, fProxy, dst, errch)
+		}(d, f, dst, errch)
 
 		// ...and as usual we spin until it's done
 		err = <-errch
 	}
+	progressBar.Finish()
+	f.Close()
 	return err
 }
-
-func (d *HTTPDownloader) ProgressBar() packer.ProgressBar { return d.Ui.ProgressBar() }
-func (d *FileDownloader) ProgressBar() packer.ProgressBar { return d.Ui.ProgressBar() }
-func (d *SMBDownloader) ProgressBar() packer.ProgressBar  { return d.Ui.ProgressBar() }

common/download_test.go

@@ -12,8 +12,6 @@ import (
 	"runtime"
 	"strings"
 	"testing"
-
-	"github.com/hashicorp/packer/packer"
 )
 
 func TestDownloadClientVerifyChecksum(t *testing.T) {
@@ -38,7 +36,7 @@ func TestDownloadClientVerifyChecksum(t *testing.T) {
 		Checksum: checksum,
 	}
 
-	d := NewDownloadClient(config, new(packer.NoopUi))
+	d := NewDownloadClient(config, nil)
 	result, err := d.VerifyChecksum(tf.Name())
 	if err != nil {
 		t.Fatalf("Verify err: %s", err)
@@ -61,7 +59,7 @@ func TestDownloadClient_basic(t *testing.T) {
 		Url:        ts.URL + "/basic.txt",
 		TargetPath: tf.Name(),
 		CopyFile:   true,
-	}, new(packer.NoopUi))
+	}, nil)
 
 	path, err := client.Get()
 	if err != nil {
@@ -97,7 +95,7 @@ func TestDownloadClient_checksumBad(t *testing.T) {
 		Hash:       HashForType("md5"),
 		Checksum:   checksum,
 		CopyFile:   true,
-	}, new(packer.NoopUi))
+	}, nil)
 
 	if _, err := client.Get(); err == nil {
 		t.Fatal("should error")
@@ -123,7 +121,7 @@ func TestDownloadClient_checksumGood(t *testing.T) {
 		Hash:       HashForType("md5"),
 		Checksum:   checksum,
 		CopyFile:   true,
-	}, new(packer.NoopUi))
+	}, nil)
 
 	path, err := client.Get()
 	if err != nil {
@@ -155,7 +153,7 @@ func TestDownloadClient_checksumNoDownload(t *testing.T) {
 		Hash:       HashForType("md5"),
 		Checksum:   checksum,
 		CopyFile:   true,
-	}, new(packer.NoopUi))
+	}, nil)
 	path, err := client.Get()
 	if err != nil {
 		t.Fatalf("err: %s", err)
@@ -185,7 +183,7 @@ func TestDownloadClient_notFound(t *testing.T) {
 	client := NewDownloadClient(&DownloadConfig{
 		Url:        ts.URL + "/not-found.txt",
 		TargetPath: tf.Name(),
-	}, new(packer.NoopUi))
+	}, nil)
 
 	if _, err := client.Get(); err == nil {
 		t.Fatal("should error")
@@ -213,7 +211,7 @@ func TestDownloadClient_resume(t *testing.T) {
 		Url:        ts.URL,
 		TargetPath: tf.Name(),
 		CopyFile:   true,
-	}, new(packer.NoopUi))
+	}, nil)
 
 	path, err := client.Get()
 	if err != nil {
@@ -275,7 +273,7 @@ func TestDownloadClient_usesDefaultUserAgent(t *testing.T) {
 		CopyFile:   true,
 	}
 
-	client := NewDownloadClient(config, new(packer.NoopUi))
+	client := NewDownloadClient(config, nil)
 	_, err = client.Get()
 	if err != nil {
 		t.Fatal(err)
@@ -308,7 +306,7 @@ func TestDownloadClient_setsUserAgent(t *testing.T) {
 		CopyFile:   true,
 	}
 
-	client := NewDownloadClient(config, new(packer.NoopUi))
+	client := NewDownloadClient(config, nil)
 	_, err = client.Get()
 	if err != nil {
 		t.Fatal(err)
@@ -407,7 +405,7 @@ func TestDownloadFileUrl(t *testing.T) {
 		CopyFile: false,
 	}
 
-	client := NewDownloadClient(config, new(packer.NoopUi))
+	client := NewDownloadClient(config, nil)
 
 	// Verify that we fail to match the checksum
 	_, err = client.Get()
@@ -438,7 +436,7 @@ func SimulateFileUriDownload(t *testing.T, uri string) (string, error) {
 	}
 
 	// go go go
-	client := NewDownloadClient(config, new(packer.NoopUi))
+	client := NewDownloadClient(config, nil)
 	path, err := client.Get()
 
 	// ignore any non-important checksum errors if it's not a unc path

common/progress.go

@@ -0,0 +1,145 @@
+package common
+
+import (
+	"fmt"
+	"github.com/cheggaaa/pb"
+	"github.com/hashicorp/packer/helper/multistep"
+	"github.com/hashicorp/packer/packer"
+	"github.com/hashicorp/packer/packer/rpc"
+	"log"
+	"reflect"
+	"time"
+)
+
+// This is the arrow from packer/ui.go -> TargetedUI.prefixLines
+const targetedUIArrowText = "==>"
+
+// The ProgressBar interface is used for abstracting cheggaaa's progress-
+// bar, or any other progress bar. If a UI does not support a progress-
+// bar, then it must return a null progress bar.
+const (
+	DefaultProgressBarWidth = 80
+)
+
+type ProgressBar = *pb.ProgressBar
+
+// Figure out the terminal dimensions and use it to calculate the available rendering space
+func calculateProgressBarWidth(length int) int {
+	// If the UI's width is signed, then this is an interface that doesn't really benefit from a progress bar
+	if length < 0 {
+		log.Println("Refusing to render progress-bar for unsupported UI.")
+		return length
+	}
+
+	// Figure out the terminal width if possible
+	width, _, err := GetTerminalDimensions()
+	if err != nil {
+		newerr := fmt.Errorf("Unable to determine terminal dimensions: %v", err)
+		log.Printf("Using default width (%d) for progress-bar due to error: %s", DefaultProgressBarWidth, newerr)
+		return DefaultProgressBarWidth
+	}
+
+	// If the terminal width is smaller than the requested length, then complain
+	if width < length {
+		newerr := fmt.Errorf("Terminal width (%d) is smaller than UI message width (%d).", width, length)
+		log.Printf("Using default width (%d) for progress-bar due to error: %s", DefaultProgressBarWidth, newerr)
+		return DefaultProgressBarWidth
+	}
+
+	// Otherwise subtract the minimum length and return it
+	return width - length
+}
+
+// Get a progress bar with the default appearance
+func GetDefaultProgressBar() ProgressBar {
+	bar := pb.New64(0)
+	bar.ShowPercent = true
+	bar.ShowCounters = true
+	bar.ShowSpeed = false
+	bar.ShowBar = true
+	bar.ShowTimeLeft = false
+	bar.ShowFinalTime = false
+	bar.SetUnits(pb.U_BYTES)
+	bar.Format("[=>-]")
+	bar.SetRefreshRate(5 * time.Second)
+	return bar
+}
+
+// Return a dummy progress bar that doesn't do anything
+func GetDummyProgressBar() ProgressBar {
+	bar := pb.New64(0)
+	bar.ManualUpdate = true
+	return bar
+}
+
+// Given a packer.Ui, calculate the number of characters that a packer.Ui will
+// prefix a message with. Then we can use this to calculate the progress bar's width.
+func calculateUiPrefixLength(ui packer.Ui) int {
+	var recursiveCalculateUiPrefixLength func(packer.Ui, int) int
+
+	// Define a recursive closure that traverses through all the known packer.Ui types
+	// and aggregates the length of the message prefix from each particular type
+	recursiveCalculateUiPrefixLength = func(ui packer.Ui, agg int) int {
+		switch ui.(type) {
+
+		case *packer.ColoredUi:
+			// packer.ColoredUi is simply a wrapper around .Ui
+			u := ui.(*packer.ColoredUi)
+			return recursiveCalculateUiPrefixLength(u.Ui, agg)
+
+		case *packer.TargetedUI:
+			// A TargetedUI adds the .Target and an arrow by default
+			u := ui.(*packer.TargetedUI)
+			res := fmt.Sprintf("%s %s: ", targetedUIArrowText, u.Target)
+			return recursiveCalculateUiPrefixLength(u.Ui, agg+len(res))
+
+		case *packer.BasicUi:
+			// The standard BasicUi appends only a newline
+			return agg + len("\n")
+
+		// packer.rpc.Ui returns 0 here to trigger the hack described later
+		case *rpc.Ui:
+			return 0
+
+		case *packer.MachineReadableUi:
+			// MachineReadableUi doesn't emit anything...like at all
+			return 0
+		}
+
+		log.Printf("Calculating the message prefix length for packer.Ui type (%T) is not implemented. Using the current aggregated length of %d.", ui, agg)
+		return agg
+	}
+	return recursiveCalculateUiPrefixLength(ui, 0)
+}
+
+func GetPackerConfigFromStateBag(state multistep.StateBag) *PackerConfig {
+	config := state.Get("config")
+	rConfig := reflect.Indirect(reflect.ValueOf(config))
+	iPackerConfig := rConfig.FieldByName("PackerConfig").Interface()
+	packerConfig := iPackerConfig.(PackerConfig)
+	return &packerConfig
+}
+
+func GetProgressBar(ui packer.Ui, config *PackerConfig) ProgressBar {
+	// Figure out the prefix length by quering the UI
+	uiPrefixLength := calculateUiPrefixLength(ui)
+
+	// hack to deal with packer.rpc.Ui courtesy of @Swampdragons
+	if _, ok := ui.(*rpc.Ui); uiPrefixLength == 0 && config != nil && ok {
+		res := fmt.Sprintf("%s %s: \n", targetedUIArrowText, config.PackerBuildName)
+		uiPrefixLength = len(res)
+	}
+
+	// Now we can use the prefix length to calculate the progress bar width
+	width := calculateProgressBarWidth(uiPrefixLength)
+
+	log.Printf("ProgressBar: Using progress bar width: %d\n", width)
+
+	// Get a default progress bar and set some output defaults
+	bar := GetDefaultProgressBar()
+	bar.SetWidth(width)
+	bar.Callback = func(message string) {
+		ui.Message(message)
+	}
+	return bar
+}

common/progress_test.go

@@ -0,0 +1,142 @@
+package common
+
+import (
+	"github.com/hashicorp/packer/packer"
+	"testing"
+)
+
+// test packer.Ui implementation to verify that progress bar is being written
+type testProgressBarUi struct {
+	messageCalled  bool
+	messageMessage string
+}
+
+func (u *testProgressBarUi) Say(string)                {}
+func (u *testProgressBarUi) Error(string)              {}
+func (u *testProgressBarUi) Machine(string, ...string) {}
+
+func (u *testProgressBarUi) Ask(string) (string, error) {
+	return "", nil
+}
+func (u *testProgressBarUi) Message(message string) {
+	u.messageCalled = true
+	u.messageMessage = message
+}
+
+// ..and now let's begin our actual tests
+func TestCalculateUiPrefixLength_Unknown(t *testing.T) {
+	ui := &testProgressBarUi{}
+
+	expected := 0
+	if res := calculateUiPrefixLength(ui); res != expected {
+		t.Fatalf("calculateUiPrefixLength should have returned a length of %d", expected)
+	}
+}
+
+func TestCalculateUiPrefixLength_BasicUi(t *testing.T) {
+	ui := &packer.BasicUi{}
+
+	expected := 1
+	if res := calculateUiPrefixLength(ui); res != expected {
+		t.Fatalf("calculateUiPrefixLength should have returned a length of %d", expected)
+	}
+}
+
+func TestCalculateUiPrefixLength_TargetedUI(t *testing.T) {
+	ui := &packer.TargetedUI{}
+	ui.Target = "TestTarget"
+	arrowText := "==>"
+
+	expected := len(arrowText + " " + ui.Target + ": ")
+	if res := calculateUiPrefixLength(ui); res != expected {
+		t.Fatalf("calculateUiPrefixLength should have returned a length of %d", expected)
+	}
+}
+
+func TestCalculateUiPrefixLength_TargetedUIWrappingBasicUi(t *testing.T) {
+	ui := &packer.TargetedUI{}
+	ui.Target = "TestTarget"
+	ui.Ui = &packer.BasicUi{}
+	arrowText := "==>"
+
+	expected := len(arrowText + " " + ui.Target + ": " + "\n")
+	if res := calculateUiPrefixLength(ui); res != expected {
+		t.Fatalf("calculateUiPrefixLength should have returned a length of %d", expected)
+	}
+}
+
+func TestCalculateUiPrefixLength_TargetedUIWrappingMachineUi(t *testing.T) {
+	ui := &packer.TargetedUI{}
+	ui.Target = "TestTarget"
+	ui.Ui = &packer.MachineReadableUi{}
+
+	expected := 0
+	if res := calculateUiPrefixLength(ui); res != expected {
+		t.Fatalf("calculateUiPrefixLength should have returned a length of %d", expected)
+	}
+}
+func TestDefaultProgressBar(t *testing.T) {
+	var callbackCalled bool
+
+	// Initialize the default progress bar
+	bar := GetDefaultProgressBar()
+	bar.Callback = func(state string) {
+		callbackCalled = true
+		t.Logf("TestDefaultProgressBar emitted %#v", state)
+	}
+	bar.SetTotal64(1)
+
+	// Set it off
+	progressBar := bar.Start()
+	progressBar.Set64(1)
+
+	// Check to see that the callback was hit
+	if !callbackCalled {
+		t.Fatalf("TestDefaultProgressBar.Callback should be called")
+	}
+}
+
+func TestDummyProgressBar(t *testing.T) {
+	var callbackCalled bool
+
+	// Initialize the dummy progress bar
+	bar := GetDummyProgressBar()
+	bar.Callback = func(state string) {
+		callbackCalled = true
+		t.Logf("TestDummyProgressBar emitted %#v", state)
+	}
+	bar.SetTotal64(1)
+
+	// Now we can go
+	progressBar := bar.Start()
+	progressBar.Set64(1)
+
+	// Check to see that the callback was hit
+	if callbackCalled {
+		t.Fatalf("TestDummyProgressBar.Callback should not be called")
+	}
+}
+
+func TestUiProgressBar(t *testing.T) {
+
+	ui := &testProgressBarUi{}
+
+	// Initialize the Ui progress bar
+	bar := GetProgressBar(ui, nil)
+	bar.SetTotal64(1)
+
+	// Ensure that callback has been set to something
+	if bar.Callback == nil {
+		t.Fatalf("TestUiProgressBar.Callback should be initialized")
+	}
+
+	// Now we can go
+	progressBar := bar.Start()
+	progressBar.Set64(1)
+
+	// Check to see that the callback was hit
+	if !ui.messageCalled {
+		t.Fatalf("TestUiProgressBar.messageCalled should be called")
+	}
+	t.Logf("TestUiProgressBar emitted %#v", ui.messageMessage)
+}

common/random/string.go

@@ -1,33 +0,0 @@
-package random
-
-import (
-	"math/rand"
-	"os"
-	"time"
-)
-
-var (
-	PossibleNumbers   = "0123456789"
-	PossibleLowerCase = "abcdefghijklmnopqrstuvwxyz"
-	PossibleUpperCase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
-
-	PossibleAlphaNum      = PossibleNumbers + PossibleLowerCase + PossibleUpperCase
-	PossibleAlphaNumLower = PossibleNumbers + PossibleLowerCase
-	PossibleAlphaNumUpper = PossibleNumbers + PossibleUpperCase
-)
-
-var rnd = rand.New(rand.NewSource(time.Now().UnixNano() + int64(os.Getpid())))
-
-func Numbers(length int) string       { return String(PossibleNumbers, length) }
-func AlphaNum(length int) string      { return String(PossibleAlphaNum, length) }
-func AlphaNumLower(length int) string { return String(PossibleAlphaNumLower, length) }
-func AlphaNumUpper(length int) string { return String(PossibleAlphaNumUpper, length) }
-
-func String(chooseFrom string, length int) (randomString string) {
-	cflen := len(chooseFrom)
-	bytes := make([]byte, length)
-	for i := range bytes {
-		bytes[i] = chooseFrom[rnd.Intn(cflen)]
-	}
-	return string(bytes)
-}

common/shell-local/config.go

@@ -53,6 +53,9 @@ type Config struct {
 	UseLinuxPathing bool `mapstructure:"use_linux_pathing"`
 
 	Ctx interpolate.Context
+
+	// internal use only; for the provisioner.
+	WinRMPassword string
 }
 
 func Decode(config *Config, raws ...interface{}) error {

common/shell-local/run.go

@@ -10,7 +10,6 @@ import (
 	"strings"
 
 	"github.com/hashicorp/packer/common"
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/packer"
 	"github.com/hashicorp/packer/template/interpolate"
 )
@@ -70,6 +69,7 @@ func Run(ui packer.Ui, config *Config) (bool, error) {
 		// buffers and for reading the final exit status.
 		flattenedCmd := strings.Join(interpolatedCmds, " ")
 		cmd := &packer.RemoteCmd{Command: flattenedCmd}
+		packer.LogSecretFilter.Set(config.WinRMPassword)
 		log.Printf("[INFO] (shell-local): starting local command: %s", flattenedCmd)
 		if err := cmd.StartWithUi(comm, ui); err != nil {
 			return false, fmt.Errorf(
@@ -105,7 +105,7 @@ func createInlineScriptFile(config *Config) (string, error) {
 
 	// generate context so you can interpolate the command
 	config.Ctx.Data = &EnvVarsTemplate{
-		WinRMPassword: getWinRMPassword(config.PackerBuildName),
+		WinRMPassword: config.WinRMPassword,
 	}
 
 	for _, command := range config.Inline {
@@ -139,7 +139,7 @@ func createInterpolatedCommands(config *Config, script string, flattenedEnvVars
 		Vars:          flattenedEnvVars,
 		Script:        script,
 		Command:       script,
-		WinRMPassword: getWinRMPassword(config.PackerBuildName),
+		WinRMPassword: config.WinRMPassword,
 	}
 
 	interpolatedCmds := make([]string, len(config.ExecuteCommand))
@@ -169,7 +169,7 @@ func createFlattenedEnvVars(config *Config) (string, error) {
 
 	// interpolate environment variables
 	config.Ctx.Data = &EnvVarsTemplate{
-		WinRMPassword: getWinRMPassword(config.PackerBuildName),
+		WinRMPassword: config.WinRMPassword,
 	}
 	// Split vars into key/value components
 	for _, envVar := range config.Vars {
@@ -196,9 +196,3 @@ func createFlattenedEnvVars(config *Config) (string, error) {
 	}
 	return flattened, nil
 }
-
-func getWinRMPassword(buildName string) string {
-	winRMPass, _ := commonhelper.RetrieveSharedState("winrm_password", buildName)
-	packer.LogSecretFilter.Set(winRMPass)
-	return winRMPass
-}

common/step_cleanup_temp_keys.go

@@ -1,56 +0,0 @@
-package common
-
-import (
-	"context"
-	"fmt"
-	"log"
-
-	"github.com/hashicorp/packer/helper/communicator"
-	"github.com/hashicorp/packer/helper/multistep"
-	"github.com/hashicorp/packer/packer"
-)
-
-type StepCleanupTempKeys struct {
-	Comm *communicator.Config
-}
-
-func (s *StepCleanupTempKeys) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
-	// This step is mostly cosmetic; Packer deletes the ephemeral keys anyway
-	// so there's no realistic situation where these keys can cause issues.
-	// However, it's nice to clean up after yourself.
-
-	comm := state.Get("communicator").(packer.Communicator)
-	ui := state.Get("ui").(packer.Ui)
-
-	if !s.Comm.SSHClearAuthorizedKeys {
-		return multistep.ActionContinue
-	}
-
-	if s.Comm.Type != "ssh" {
-		return multistep.ActionContinue
-	}
-
-	if s.Comm.SSHTemporaryKeyPairName == "" {
-		return multistep.ActionContinue
-	}
-
-	cmd := new(packer.RemoteCmd)
-
-	ui.Say("Trying to remove ephemeral keys from authorized_keys files")
-
-	cmd.Command = fmt.Sprintf("sed -i.bak '/ssh-rsa.*%s$/d' ~/.ssh/authorized_keys; rm ~/.ssh/authorized_keys.bak", s.Comm.SSHTemporaryKeyPairName)
-	if err := cmd.StartWithUi(comm, ui); err != nil {
-		log.Printf("Error cleaning up ~/.ssh/authorized_keys; please clean up keys manually: %s", err)
-	}
-	cmd = new(packer.RemoteCmd)
-	cmd.Command = fmt.Sprintf("sudo sed -i.bak '/ssh-rsa.*%s$/d' /root/.ssh/authorized_keys; sudo rm /root/.ssh/authorized_keys.bak", s.Comm.SSHTemporaryKeyPairName)
-
-	if err := cmd.StartWithUi(comm, ui); err != nil {
-		log.Printf("Error cleaning up /root/.ssh/authorized_keys; please clean up keys manually: %s", err)
-	}
-
-	return multistep.ActionContinue
-}
-
-func (s *StepCleanupTempKeys) Cleanup(state multistep.StateBag) {
-}

common/step_download.go

@@ -63,6 +63,9 @@ func (s *StepDownload) Run(_ context.Context, state multistep.StateBag) multiste
 
 	ui.Say(fmt.Sprintf("Retrieving %s", s.Description))
 
+	// Get a progress bar from the ui so we can hand it off to the download client
+	bar := GetProgressBar(ui, GetPackerConfigFromStateBag(state))
+
 	// First try to use any already downloaded file
 	// If it fails, proceed to regular download logic
 
@@ -96,7 +99,7 @@ func (s *StepDownload) Run(_ context.Context, state multistep.StateBag) multiste
 		}
 		downloadConfigs[i] = config
 
-		if match, _ := NewDownloadClient(config, ui).VerifyChecksum(config.TargetPath); match {
+		if match, _ := NewDownloadClient(config, bar).VerifyChecksum(config.TargetPath); match {
 			ui.Message(fmt.Sprintf("Found already downloaded, initial checksum matched, no download needed: %s", url))
 			finalPath = config.TargetPath
 			break
@@ -140,8 +143,11 @@ func (s *StepDownload) download(config *DownloadConfig, state multistep.StateBag
 	var path string
 	ui := state.Get("ui").(packer.Ui)
 
-	// Create download client with config
-	download := NewDownloadClient(config, ui)
+	// Get a progress bar and hand it off to the download client
+	bar := GetProgressBar(ui, GetPackerConfigFromStateBag(state))
+
+	// Create download client with config and progress bar
+	download := NewDownloadClient(config, bar)
 
 	downloadCompleteCh := make(chan error, 1)
 	go func() {
@@ -153,6 +159,7 @@ func (s *StepDownload) download(config *DownloadConfig, state multistep.StateBag
 	for {
 		select {
 		case err := <-downloadCompleteCh:
+			bar.Finish()
 
 			if err != nil {
 				return "", err, true
@@ -167,6 +174,7 @@ func (s *StepDownload) download(config *DownloadConfig, state multistep.StateBag
 
 		case <-time.After(1 * time.Second):
 			if _, ok := state.GetOk(multistep.StateCancelled); ok {
+				bar.Finish()
 				ui.Say("Interrupt received. Cancelling download...")
 				return "", nil, false
 			}

common/step_provision.go

@@ -22,6 +22,10 @@ type StepProvision struct {
 	Comm packer.Communicator
 }
 
+type ProvisionHookData struct {
+	WinRMPassword string
+}
+
 func (s *StepProvision) Run(_ context.Context, state multistep.StateBag) multistep.StepAction {
 	comm := s.Comm
 	if comm == nil {
@@ -33,12 +37,21 @@ func (s *StepProvision) Run(_ context.Context, state multistep.StateBag) multist
 	hook := state.Get("hook").(packer.Hook)
 	ui := state.Get("ui").(packer.Ui)
 
+	// Save data we need to give to provisioners
+	WinRMPassword, ok := state.GetOk("winrm_password")
+	if !ok {
+		WinRMPassword = ""
+	} else {
+		WinRMPassword = WinRMPassword.(string)
+	}
+	phd := ProvisionHookData{WinRMPassword.(string)}
+
 	// Run the provisioner in a goroutine so we can continually check
 	// for cancellations...
 	log.Println("Running the provision hook")
 	errCh := make(chan error, 1)
 	go func() {
-		errCh <- hook.Run(packer.HookProvision, ui, comm, nil)
+		errCh <- hook.Run(packer.HookProvision, ui, comm, phd)
 	}()
 
 	for {

fix/fixer.go

@@ -38,7 +38,6 @@ func init() {
 		"powershell-escapes":         new(FixerPowerShellEscapes),
 		"hyperv-deprecations":        new(FixerHypervDeprecations),
 		"hyperv-vmxc-typo":           new(FixerHypervVmxcTypo),
-		"vmware-compaction":          new(FixerVMwareCompaction),
 	}
 
 	FixerOrder = []string{
@@ -58,6 +57,7 @@ func init() {
 		"amazon-private-ip",
 		"docker-email",
 		"powershell-escapes",
-		"vmware-compaction",
+		"hyperv-deprecations",
+		"hyperv-vmxc-typo",
 	}
 }

fix/fixer_docker_email.go

@@ -5,14 +5,10 @@ import "github.com/mitchellh/mapstructure"
 type FixerDockerEmail struct{}
 
 func (FixerDockerEmail) Fix(input map[string]interface{}) (map[string]interface{}, error) {
-	if input["post-processors"] == nil {
-		return input, nil
-	}
-
 	// Our template type we'll use for this fixer only
 	type template struct {
-		Builders []map[string]interface{}
-		PP       `mapstructure:",squash"`
+		Builders       []map[string]interface{}
+		PostProcessors []map[string]interface{} `mapstructure:"post-processors"`
 	}
 
 	// Decode the input into our structure, if we can
@@ -31,9 +27,7 @@ func (FixerDockerEmail) Fix(input map[string]interface{}) (map[string]interface{
 	}
 
 	// Go through each post-processor and delete `docker_login` if present
-	pps := tpl.ppList()
-
-	for _, pp := range pps {
+	for _, pp := range tpl.PostProcessors {
 		_, ok := pp["login_email"]
 		if !ok {
 			continue

fix/fixer_pp_manifest_filename.go

@@ -8,13 +8,10 @@ import (
 type FixerManifestFilename struct{}
 
 func (FixerManifestFilename) Fix(input map[string]interface{}) (map[string]interface{}, error) {
-	if input["post-processors"] == nil {
-		return input, nil
-	}
 
 	// Our template type we'll use for this fixer only
 	type template struct {
-		PP `mapstructure:",squash"`
+		PostProcessors []interface{} `mapstructure:"post-processors"`
 	}
 
 	// Decode the input into our structure, if we can
@@ -24,7 +21,20 @@ func (FixerManifestFilename) Fix(input map[string]interface{}) (map[string]inter
 	}
 
 	// Go through each post-processor and get out all the complex configs
-	pps := tpl.ppList()
+	pps := make([]map[string]interface{}, 0, len(tpl.PostProcessors))
+	for _, rawPP := range tpl.PostProcessors {
+		switch pp := rawPP.(type) {
+		case string:
+		case map[string]interface{}:
+			pps = append(pps, pp)
+		case []interface{}:
+			for _, innerRawPP := range pp {
+				if innerPP, ok := innerRawPP.(map[string]interface{}); ok {
+					pps = append(pps, innerPP)
+				}
+			}
+		}
+	}
 
 	for _, pp := range pps {
 		ppTypeRaw, ok := pp["type"]

fix/fixer_pp_manifest_filename_test.go

@@ -1,9 +1,8 @@
 package fix
 
 import (
+	"reflect"
 	"testing"
-
-	"github.com/stretchr/testify/assert"
 )
 
 func TestFixerManifestPPFilename_Impl(t *testing.T) {
@@ -44,7 +43,11 @@ func TestFixerManifestPPFilename_Fix(t *testing.T) {
 	}
 
 	output, err := f.Fix(input)
-	assert.NoError(t, err)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
 
-	assert.Equal(t, expected, output)
+	if !reflect.DeepEqual(output, expected) {
+		t.Fatalf("unexpected: %#v\nexpected: %#v\n", output, expected)
+	}
 }

fix/fixer_pp_vagrant_override.go

@@ -1,6 +1,8 @@
 package fix
 
-import "github.com/mitchellh/mapstructure"
+import (
+	"github.com/mitchellh/mapstructure"
+)
 
 // FixerVagrantPPOverride is a Fixer that replaces the provider-specific
 // overrides for the Vagrant post-processor with the new style introduced
@@ -8,13 +10,9 @@ import "github.com/mitchellh/mapstructure"
 type FixerVagrantPPOverride struct{}
 
 func (FixerVagrantPPOverride) Fix(input map[string]interface{}) (map[string]interface{}, error) {
-	if input["post-processors"] == nil {
-		return input, nil
-	}
-
 	// Our template type we'll use for this fixer only
 	type template struct {
-		PP `mapstructure:",squash"`
+		PostProcessors []interface{} `mapstructure:"post-processors"`
 	}
 
 	// Decode the input into our structure, if we can
@@ -23,7 +21,21 @@ func (FixerVagrantPPOverride) Fix(input map[string]interface{}) (map[string]inte
 		return nil, err
 	}
 
-	pps := tpl.ppList()
+	// Go through each post-processor and get out all the complex configs
+	pps := make([]map[string]interface{}, 0, len(tpl.PostProcessors))
+	for _, rawPP := range tpl.PostProcessors {
+		switch pp := rawPP.(type) {
+		case string:
+		case map[string]interface{}:
+			pps = append(pps, pp)
+		case []interface{}:
+			for _, innerRawPP := range pp {
+				if innerPP, ok := innerRawPP.(map[string]interface{}); ok {
+					pps = append(pps, innerPP)
+				}
+			}
+		}
+	}
 
 	// Go through each post-processor and make the fix if necessary
 	possible := []string{"aws", "digitalocean", "virtualbox", "vmware"}

fix/fixer_pp_vagrant_override_test.go

@@ -1,9 +1,8 @@
 package fix
 
 import (
+	"reflect"
 	"testing"
-
-	"github.com/stretchr/testify/assert"
 )
 
 func TestFixerVagrantPPOverride_Impl(t *testing.T) {
@@ -70,7 +69,11 @@ func TestFixerVagrantPPOverride_Fix(t *testing.T) {
 	}
 
 	output, err := f.Fix(input)
-	assert.NoError(t, err)
+	if err != nil {
+		t.Fatalf("err: %s", err)
+	}
 
-	assert.Equal(t, expected, output)
+	if !reflect.DeepEqual(output, expected) {
+		t.Fatalf("unexpected: %#v\nexpected: %#v\n", output, expected)
+	}
 }

fix/fixer_vmware_compaction.go

@@ -1,91 +0,0 @@
-package fix
-
-import (
-	"github.com/mitchellh/mapstructure"
-)
-
-// FixerVMwareCompaction adds "skip_compaction = true" to "vmware-iso" builders with incompatible disk_type_id
-type FixerVMwareCompaction struct{}
-
-func (FixerVMwareCompaction) Fix(input map[string]interface{}) (map[string]interface{}, error) {
-	// The type we'll decode into; we only care about builders
-	type template struct {
-		Builders []map[string]interface{}
-	}
-
-	// Decode the input into our structure, if we can
-	var tpl template
-	if err := mapstructure.Decode(input, &tpl); err != nil {
-		return nil, err
-	}
-
-	for _, builder := range tpl.Builders {
-		builderTypeRaw, ok := builder["type"]
-		if !ok {
-			continue
-		}
-
-		builderType, ok := builderTypeRaw.(string)
-		if !ok {
-			continue
-		}
-
-		if builderType != "vmware-iso" {
-			continue
-		}
-
-		builderRemoteTypeRaw, ok := builder["remote_type"]
-		if !ok {
-			continue
-		}
-
-		builderRemoteType, ok := builderRemoteTypeRaw.(string)
-		if !ok {
-			continue
-		}
-
-		if builderRemoteType != "esx5" {
-			continue
-		}
-
-		builderDiskTypeIdRaw, ok := builder["disk_type_id"]
-		// set to default when this fixer was added due to incompatibility of defaults
-		if !ok {
-			builderDiskTypeId := "zeroedthick"
-			builder["disk_type_id"] = builderDiskTypeId
-		}
-
-		if ok {
-			builderDiskTypeId, ok := builderDiskTypeIdRaw.(string)
-			if !ok {
-				continue
-			}
-			if builderDiskTypeId == "thin" {
-				continue
-			}
-		}
-
-		builderSkipCompactionRaw, ok := builder["skip_compaction"]
-		// already verified this is not creating a "thin" disk, will need to skip_compaction
-		if ok {
-			builderSkipCompaction, ok := builderSkipCompactionRaw.(bool)
-			if !ok {
-				continue
-			}
-			if !builderSkipCompaction {
-				builder["skip_compaction"] = !builderSkipCompaction
-			}
-			continue
-		}
-
-		builderSkipCompaction := true
-		builder["skip_compaction"] = builderSkipCompaction
-	}
-
-	input["builders"] = tpl.Builders
-	return input, nil
-}
-
-func (FixerVMwareCompaction) Synopsis() string {
-	return `Adds "skip_compaction = true" to "vmware-iso" builders with incompatible disk_type_id`
-}

fix/fixer_vmware_compaction_test.go

@@ -1,112 +0,0 @@
-package fix
-
-import (
-	"reflect"
-	"testing"
-)
-
-func TestFixerVMwareCompaction_impl(t *testing.T) {
-	var _ Fixer = new(FixerVMwareCompaction)
-}
-
-func TestFixerVMwareCompaction_Fix(t *testing.T) {
-	cases := []struct {
-		Input    map[string]interface{}
-		Expected map[string]interface{}
-	}{
-		{
-			Input: map[string]interface{}{
-				"type": "virtualbox-iso",
-			},
-
-			Expected: map[string]interface{}{
-				"type": "virtualbox-iso",
-			},
-		},
-		{
-			Input: map[string]interface{}{
-				"type": "vmware-iso",
-			},
-
-			Expected: map[string]interface{}{
-				"type": "vmware-iso",
-			},
-		},
-		{
-			Input: map[string]interface{}{
-				"type":        "vmware-iso",
-				"remote_type": "esx5",
-			},
-
-			Expected: map[string]interface{}{
-				"type":            "vmware-iso",
-				"remote_type":     "esx5",
-				"disk_type_id":    "zeroedthick",
-				"skip_compaction": true,
-			},
-		},
-		{
-			Input: map[string]interface{}{
-				"type":         "vmware-iso",
-				"remote_type":  "esx5",
-				"disk_type_id": "zeroedthick",
-			},
-
-			Expected: map[string]interface{}{
-				"type":            "vmware-iso",
-				"remote_type":     "esx5",
-				"disk_type_id":    "zeroedthick",
-				"skip_compaction": true,
-			},
-		},
-		{
-			Input: map[string]interface{}{
-				"type":            "vmware-iso",
-				"remote_type":     "esx5",
-				"disk_type_id":    "zeroedthick",
-				"skip_compaction": false,
-			},
-
-			Expected: map[string]interface{}{
-				"type":            "vmware-iso",
-				"remote_type":     "esx5",
-				"disk_type_id":    "zeroedthick",
-				"skip_compaction": true,
-			},
-		},
-		{
-			Input: map[string]interface{}{
-				"type":         "vmware-iso",
-				"remote_type":  "esx5",
-				"disk_type_id": "thin",
-			},
-
-			Expected: map[string]interface{}{
-				"type":         "vmware-iso",
-				"remote_type":  "esx5",
-				"disk_type_id": "thin",
-			},
-		},
-	}
-
-	for _, tc := range cases {
-		var f FixerVMwareCompaction
-
-		input := map[string]interface{}{
-			"builders": []map[string]interface{}{tc.Input},
-		}
-
-		expected := map[string]interface{}{
-			"builders": []map[string]interface{}{tc.Expected},
-		}
-
-		output, err := f.Fix(input)
-		if err != nil {
-			t.Fatalf("err: %s", err)
-		}
-
-		if !reflect.DeepEqual(output, expected) {
-			t.Fatalf("unexpected: %#v\nexpected: %#v\n", output, expected)
-		}
-	}
-}

fix/helpers.go

@@ -1,25 +0,0 @@
-package fix
-
-// PP is a convenient way to interact with the post-processors within a fixer
-type PP struct {
-	PostProcessors []interface{} `mapstructure:"post-processors"`
-}
-
-// postProcessors converts the variable structure of the template to a list
-func (pp *PP) ppList() []map[string]interface{} {
-	pps := make([]map[string]interface{}, 0, len(pp.PostProcessors))
-	for _, rawPP := range pp.PostProcessors {
-		switch pp := rawPP.(type) {
-		case string:
-		case map[string]interface{}:
-			pps = append(pps, pp)
-		case []interface{}:
-			for _, innerRawPP := range pp {
-				if innerPP, ok := innerRawPP.(map[string]interface{}); ok {
-					pps = append(pps, innerPP)
-				}
-			}
-		}
-	}
-	return pps
-}

helper/communicator/config.go

@@ -31,7 +31,6 @@ type Config struct {
 	SSHPrivateKey             []byte        `mapstructure:"ssh_private_key"`
 	SSHKeyPairName            string        `mapstructure:"ssh_keypair_name"`
 	SSHTemporaryKeyPairName   string        `mapstructure:"temporary_key_pair_name"`
-	SSHClearAuthorizedKeys    bool          `mapstructure:"ssh_clear_authorized_keys"`
 	SSHPrivateKeyFile         string        `mapstructure:"ssh_private_key_file"`
 	SSHInterface              string        `mapstructure:"ssh_interface"`
 	SSHIPVersion              string        `mapstructure:"ssh_ip_version"`

packer/progressbar.go

@@ -1,147 +0,0 @@
-package packer
-
-import (
-	"fmt"
-	"io"
-	"sync"
-	"sync/atomic"
-
-	"github.com/cheggaaa/pb"
-)
-
-// ProgressBar allows to graphically display
-// a self refreshing progress bar.
-type ProgressBar interface {
-	Start(total int64)
-	Add(current int64)
-	NewProxyReader(r io.Reader) (proxy io.Reader)
-	Finish()
-}
-
-// StackableProgressBar is a progress bar that
-// allows to track multiple downloads at once.
-// Every call to Start increments a counter that
-// will display the number of current loadings.
-// Every call to Start will add total to an internal
-// total that is the total displayed.
-// First call to Start will start a goroutine
-// that is waiting for every download to be finished.
-// Last call to Finish triggers a cleanup.
-// When all active downloads are finished
-// StackableProgressBar will clean itself to a default
-// state.
-type StackableProgressBar struct {
-	mtx sync.Mutex // locks in Start & Finish
-	BasicProgressBar
-	items int32
-	total int64
-
-	started bool
-}
-
-var _ ProgressBar = new(StackableProgressBar)
-
-func (spb *StackableProgressBar) start() {
-	spb.BasicProgressBar.ProgressBar = pb.New(0)
-	spb.BasicProgressBar.ProgressBar.SetUnits(pb.U_BYTES)
-
-	spb.BasicProgressBar.ProgressBar.Start()
-	spb.started = true
-}
-
-func (spb *StackableProgressBar) Start(total int64) {
-	spb.mtx.Lock()
-
-	spb.total += total
-	spb.items++
-
-	if !spb.started {
-		spb.start()
-	}
-	spb.SetTotal64(spb.total)
-	spb.prefix()
-	spb.mtx.Unlock()
-}
-
-func (spb *StackableProgressBar) prefix() {
-	spb.BasicProgressBar.ProgressBar.Prefix(fmt.Sprintf("%d items: ", atomic.LoadInt32(&spb.items)))
-}
-
-func (spb *StackableProgressBar) Finish() {
-	spb.mtx.Lock()
-	defer spb.mtx.Unlock()
-
-	spb.items--
-	if spb.items == 0 {
-		// slef cleanup
-		spb.BasicProgressBar.ProgressBar.Finish()
-		spb.BasicProgressBar.ProgressBar = nil
-		spb.started = false
-		spb.total = 0
-		return
-	}
-	spb.prefix()
-}
-
-// BasicProgressBar is packer's basic progress bar.
-// Current implementation will always try to keep
-// itself at the bottom of a terminal.
-type BasicProgressBar struct {
-	*pb.ProgressBar
-}
-
-var _ ProgressBar = new(BasicProgressBar)
-
-func (bpb *BasicProgressBar) Start(total int64) {
-	bpb.SetTotal64(total)
-	bpb.ProgressBar.Start()
-}
-
-func (bpb *BasicProgressBar) Add(current int64) {
-	bpb.ProgressBar.Add64(current)
-}
-func (bpb *BasicProgressBar) NewProxyReader(r io.Reader) io.Reader {
-	return &ProxyReader{
-		Reader:      r,
-		ProgressBar: bpb,
-	}
-}
-func (bpb *BasicProgressBar) NewProxyReadCloser(r io.ReadCloser) io.ReadCloser {
-	return &ProxyReader{
-		Reader:      r,
-		ProgressBar: bpb,
-	}
-}
-
-// NoopProgressBar is a silent progress bar.
-type NoopProgressBar struct {
-}
-
-var _ ProgressBar = new(NoopProgressBar)
-
-func (npb *NoopProgressBar) Start(int64)                                      {}
-func (npb *NoopProgressBar) Add(int64)                                        {}
-func (npb *NoopProgressBar) Finish()                                          {}
-func (npb *NoopProgressBar) NewProxyReader(r io.Reader) io.Reader             { return r }
-func (npb *NoopProgressBar) NewProxyReadCloser(r io.ReadCloser) io.ReadCloser { return r }
-
-// ProxyReader implements io.ReadCloser but sends
-// count of read bytes to a progress bar
-type ProxyReader struct {
-	io.Reader
-	ProgressBar
-}
-
-func (r *ProxyReader) Read(p []byte) (n int, err error) {
-	n, err = r.Reader.Read(p)
-	r.ProgressBar.Add(int64(n))
-	return
-}
-
-// Close the reader if it implements io.Closer
-func (r *ProxyReader) Close() (err error) {
-	if closer, ok := r.Reader.(io.Closer); ok {
-		return closer.Close()
-	}
-	return
-}

packer/provisioner.go

@@ -44,6 +44,10 @@ type ProvisionHook struct {
 	runningProvisioner Provisioner
 }
 
+type ProvisionHookData struct {
+	WinRMPassword string
+}
+
 // Runs the provisioners in order.
 func (h *ProvisionHook) Run(name string, ui Ui, comm Communicator, data interface{}) error {
 	// Shortcut
@@ -71,8 +75,15 @@ func (h *ProvisionHook) Run(name string, ui Ui, comm Communicator, data interfac
 		h.lock.Unlock()
 
 		ts := CheckpointReporter.AddSpan(p.TypeName, "provisioner", p.Config)
+		// re-run prepare with builder-generated config variables (e.g. WinRMPassword)
+		// Hack Alert. #SorryNotSorry.
+		err := p.Provisioner.Prepare(data)
+		if err != nil {
+			log.Printf("Error performing secondary Prepare: %s", err)
+		}
 
-		err := p.Provisioner.Provision(ui, comm)
+		// Finally, provision.
+		err = p.Provisioner.Provision(ui, comm)
 
 		ts.End(err)
 		if err != nil {

packer/provisioner_mock.go

@@ -14,8 +14,10 @@ type MockProvisioner struct {
 }
 
 func (t *MockProvisioner) Prepare(configs ...interface{}) error {
-	t.PrepCalled = true
-	t.PrepConfigs = configs
+	if !t.PrepCalled {
+		t.PrepCalled = true
+		t.PrepConfigs = configs
+	}
 	return nil
 }
 

packer/rpc/server.go

@@ -114,8 +114,7 @@ func (s *Server) RegisterProvisioner(p packer.Provisioner) {
 
 func (s *Server) RegisterUi(ui packer.Ui) {
 	s.server.RegisterName(DefaultUiEndpoint, &UiServer{
-		ui:       ui,
-		register: s.server.RegisterName,
+		ui: ui,
 	})
 }
 

packer/rpc/ui.go

@@ -1,7 +1,6 @@
 package rpc
 
 import (
-	"io"
 	"log"
 	"net/rpc"
 
@@ -15,13 +14,10 @@ type Ui struct {
 	endpoint string
 }
 
-var _ packer.Ui = new(Ui)
-
 // UiServer wraps a packer.Ui implementation and makes it exportable
 // as part of a Golang RPC server.
 type UiServer struct {
-	ui       packer.Ui
-	register func(name string, rcvr interface{}) error
+	ui packer.Ui
 }
 
 // The arguments sent to Ui.Machine
@@ -64,31 +60,6 @@ func (u *Ui) Say(message string) {
 	}
 }
 
-func (u *Ui) ProgressBar() packer.ProgressBar {
-	if err := u.client.Call("Ui.ProgressBar", new(interface{}), new(interface{})); err != nil {
-		log.Printf("Error in Ui RPC call: %s", err)
-	}
-	return u // Ui is also a progress bar !!
-}
-
-var _ packer.ProgressBar = new(Ui)
-
-func (pb *Ui) Start(total int64) {
-	pb.client.Call("Ui.Start", total, new(interface{}))
-}
-
-func (pb *Ui) Add(current int64) {
-	pb.client.Call("Ui.Add", current, new(interface{}))
-}
-
-func (pb *Ui) Finish() {
-	pb.client.Call("Ui.Finish", nil, new(interface{}))
-}
-
-func (pb *Ui) NewProxyReader(r io.Reader) io.Reader {
-	return &packer.ProxyReader{Reader: r, ProgressBar: pb}
-}
-
 func (u *UiServer) Ask(query string, reply *string) (err error) {
 	*reply, err = u.ui.Ask(query)
 	return
@@ -120,26 +91,3 @@ func (u *UiServer) Say(message *string, reply *interface{}) error {
 	*reply = nil
 	return nil
 }
-
-func (u *UiServer) ProgressBar(_ *string, reply *interface{}) error {
-	// No-op for now, this function might be
-	// used in the future if we want to use
-	// different progress bars with identifiers.
-	u.ui.ProgressBar()
-	return nil
-}
-
-func (pb *UiServer) Finish(_ string, _ *interface{}) error {
-	pb.ui.ProgressBar().Finish()
-	return nil
-}
-
-func (pb *UiServer) Start(total int64, _ *interface{}) error {
-	pb.ui.ProgressBar().Start(total)
-	return nil
-}
-
-func (pb *UiServer) Add(current int64, _ *interface{}) error {
-	pb.ui.ProgressBar().Add(current)
-	return nil
-}

packer/rpc/ui_test.go

@@ -1,11 +1,8 @@
 package rpc
 
 import (
-	"io"
 	"reflect"
 	"testing"
-
-	"github.com/hashicorp/packer/packer"
 )
 
 type testUi struct {
@@ -20,12 +17,6 @@ type testUi struct {
 	messageMessage string
 	sayCalled      bool
 	sayMessage     string
-
-	progressBarCalled               bool
-	progressBarStartCalled          bool
-	progressBarAddCalled            bool
-	progressBarFinishCalled         bool
-	progressBarNewProxyReaderCalled bool
 }
 
 func (u *testUi) Ask(query string) (string, error) {
@@ -55,28 +46,6 @@ func (u *testUi) Say(message string) {
 	u.sayMessage = message
 }
 
-func (u *testUi) ProgressBar() packer.ProgressBar {
-	u.progressBarCalled = true
-	return u
-}
-
-func (u *testUi) Start(int64) {
-	u.progressBarStartCalled = true
-}
-
-func (u *testUi) Add(int64) {
-	u.progressBarAddCalled = true
-}
-
-func (u *testUi) Finish() {
-	u.progressBarFinishCalled = true
-}
-
-func (u *testUi) NewProxyReader(r io.Reader) io.Reader {
-	u.progressBarNewProxyReaderCalled = true
-	return r
-}
-
 func TestUiRPC(t *testing.T) {
 	// Create the UI to test
 	ui := new(testUi)
@@ -119,26 +88,6 @@ func TestUiRPC(t *testing.T) {
 		t.Fatalf("bad: %#v", ui.errorMessage)
 	}
 
-	bar := uiClient.ProgressBar()
-	if ui.progressBarCalled != true {
-		t.Errorf("ProgressBar not called.")
-	}
-
-	bar.Start(100)
-	if ui.progressBarStartCalled != true {
-		t.Errorf("progressBar.Start not called.")
-	}
-
-	bar.Add(1)
-	if ui.progressBarAddCalled != true {
-		t.Errorf("progressBar.Add not called.")
-	}
-
-	bar.Finish()
-	if ui.progressBarFinishCalled != true {
-		t.Errorf("progressBar.Finish not called.")
-	}
-
 	uiClient.Machine("foo", "bar", "baz")
 	if !ui.machineCalled {
 		t.Fatal("machine should be called")

packer/ui.go

@@ -37,20 +37,8 @@ type Ui interface {
 	Message(string)
 	Error(string)
 	Machine(string, ...string)
-	ProgressBar() ProgressBar
 }
 
-type NoopUi struct{}
-
-var _ Ui = new(NoopUi)
-
-func (*NoopUi) Ask(string) (string, error) { return "", errors.New("this is a noop ui") }
-func (*NoopUi) Say(string)                 { return }
-func (*NoopUi) Message(string)             { return }
-func (*NoopUi) Error(string)               { return }
-func (*NoopUi) Machine(string, ...string)  { return }
-func (*NoopUi) ProgressBar() ProgressBar   { return new(NoopProgressBar) }
-
 // ColoredUi is a UI that is colored using terminal colors.
 type ColoredUi struct {
 	Color      UiColor
@@ -58,8 +46,6 @@ type ColoredUi struct {
 	Ui         Ui
 }
 
-var _ Ui = new(ColoredUi)
-
 // TargetedUI is a UI that wraps another UI implementation and modifies
 // the output to indicate a specific target. Specifically, all Say output
 // is prefixed with the target name. Message output is not prefixed but
@@ -70,8 +56,6 @@ type TargetedUI struct {
 	Ui     Ui
 }
 
-var _ Ui = new(TargetedUI)
-
 // The BasicUI is a UI that reads and writes from a standard Go reader
 // and writer. It is safe to be called from multiple goroutines. Machine
 // readable output is simply logged for this UI.
@@ -82,13 +66,6 @@ type BasicUi struct {
 	l           sync.Mutex
 	interrupted bool
 	scanner     *bufio.Scanner
-	StackableProgressBar
-}
-
-var _ Ui = new(BasicUi)
-
-func (bu *BasicUi) ProgressBar() ProgressBar {
-	return &bu.StackableProgressBar
 }
 
 // MachineReadableUi is a UI that only outputs machine-readable output
@@ -97,8 +74,6 @@ type MachineReadableUi struct {
 	Writer io.Writer
 }
 
-var _ Ui = new(MachineReadableUi)
-
 func (u *ColoredUi) Ask(query string) (string, error) {
 	return u.Ui.Ask(u.colorize(query, u.Color, true))
 }
@@ -125,10 +100,6 @@ func (u *ColoredUi) Machine(t string, args ...string) {
 	u.Ui.Machine(t, args...)
 }
 
-func (u *ColoredUi) ProgressBar() ProgressBar {
-	return u.Ui.ProgressBar() //TODO(adrien): color me
-}
-
 func (u *ColoredUi) colorize(message string, color UiColor, bold bool) string {
 	if !u.supportsColors() {
 		return message
@@ -182,10 +153,6 @@ func (u *TargetedUI) Machine(t string, args ...string) {
 	u.Ui.Machine(fmt.Sprintf("%s,%s", u.Target, t), args...)
 }
 
-func (u *TargetedUI) ProgressBar() ProgressBar {
-	return u.Ui.ProgressBar()
-}
-
 func (u *TargetedUI) prefixLines(arrow bool, message string) string {
 	arrowText := "==>"
 	if !arrow {
@@ -338,7 +305,3 @@ func (u *MachineReadableUi) Machine(category string, args ...string) {
 		}
 	}
 }
-
-func (u *MachineReadableUi) ProgressBar() ProgressBar {
-	return new(NoopProgressBar)
-}

post-processor/vsphere/post-processor.go

@@ -3,13 +3,9 @@ package vsphere
 import (
 	"bytes"
 	"fmt"
-	"io"
 	"log"
 	"net/url"
-	"os"
 	"os/exec"
-	"regexp"
-	"runtime"
 	"strings"
 
 	"github.com/hashicorp/packer/common"
@@ -23,18 +19,6 @@ var builtins = map[string]string{
 	"mitchellh.vmware-esx": "vmware",
 }
 
-var ovftool string = "ovftool"
-
-var (
-	// Regular expression to validate RFC1035 hostnames from full fqdn or simple hostname.
-	// For example "packer-esxi1". Requires proper DNS setup and/or correct DNS search domain setting.
-	hostnameRegex = regexp.MustCompile(`^[[:alnum:]][[:alnum:]\-]{0,61}[[:alnum:]]|[[:alpha:]]$`)
-
-	// Simple regular expression to validate IPv4 values.
-	// For example "192.168.1.1".
-	ipv4Regex = regexp.MustCompile(`^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$`)
-)
-
 type Config struct {
 	common.PackerConfig `mapstructure:",squash"`
 
@@ -43,7 +27,6 @@ type Config struct {
 	Datastore    string   `mapstructure:"datastore"`
 	DiskMode     string   `mapstructure:"disk_mode"`
 	Host         string   `mapstructure:"host"`
-	ESXiHost     string   `mapstructure:"esxi_host"`
 	Insecure     bool     `mapstructure:"insecure"`
 	Options      []string `mapstructure:"options"`
 	Overwrite    bool     `mapstructure:"overwrite"`
@@ -81,11 +64,7 @@ func (p *PostProcessor) Configure(raws ...interface{}) error {
 	// Accumulate any errors
 	errs := new(packer.MultiError)
 
-	if runtime.GOOS == "windows" {
-		ovftool = "ovftool.exe"
-	}
-
-	if _, err := exec.LookPath(ovftool); err != nil {
+	if _, err := exec.LookPath("ovftool"); err != nil {
 		errs = packer.MultiErrorAppend(
 			errs, fmt.Errorf("ovftool not found: %s", err))
 	}
@@ -143,14 +122,6 @@ func (p *PostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact) (pac
 		ovftool_uri += "/Resources/" + p.config.ResourcePool
 	}
 
-	if p.config.ESXiHost != "" {
-		if ipv4Regex.MatchString(p.config.ESXiHost) {
-			ovftool_uri += "/?ip=" + p.config.ESXiHost
-		} else if hostnameRegex.MatchString(p.config.ESXiHost) {
-			ovftool_uri += "/?dns=" + p.config.ESXiHost
-		}
-	}
-
 	args, err := p.BuildArgs(source, ovftool_uri)
 	if err != nil {
 		ui.Message(fmt.Sprintf("Failed: %s\n", err))
@@ -158,26 +129,16 @@ func (p *PostProcessor) PostProcess(ui packer.Ui, artifact packer.Artifact) (pac
 
 	ui.Message(fmt.Sprintf("Uploading %s to vSphere", source))
 
-	log.Printf("Starting ovftool with parameters: %s",
-		strings.Replace(
-			strings.Join(args, " "),
-			password,
-			"<password>",
-			-1))
-
-	var errWriter io.Writer
-	var errOut bytes.Buffer
-	cmd := exec.Command(ovftool, args...)
-	errWriter = io.MultiWriter(os.Stderr, &errOut)
-	cmd.Stdout = os.Stdout
-	cmd.Stderr = errWriter
+	log.Printf("Starting ovftool with parameters: %s", p.filterLog(strings.Join(args, " ")))
 
+	var out bytes.Buffer
+	cmd := exec.Command("ovftool", args...)
+	cmd.Stdout = &out
 	if err := cmd.Run(); err != nil {
-		err := fmt.Errorf("Error uploading virtual machine: %s\n%s\n", err, p.filterLog(errOut.String()))
-		return nil, false, err
+		return nil, false, fmt.Errorf("Failed: %s\n%s\n", err, p.filterLog(out.String()))
 	}
 
-	ui.Message(p.filterLog(errOut.String()))
+	ui.Message(p.filterLog(out.String()))
 
 	artifact = NewArtifact(p.config.Datastore, p.config.VMFolder, p.config.VMName, artifact.Files())
 

provisioner/ansible-local/provisioner_test.go

@@ -8,12 +8,11 @@ import (
 	"testing"
 
 	"fmt"
-	"os/exec"
-
 	"github.com/hashicorp/packer/builder/docker"
 	"github.com/hashicorp/packer/packer"
 	"github.com/hashicorp/packer/provisioner/file"
 	"github.com/hashicorp/packer/template"
+	"os/exec"
 )
 
 func TestProvisioner_Impl(t *testing.T) {
@@ -133,7 +132,7 @@ func TestProvisionerProvision_PlaybookFiles(t *testing.T) {
 	}
 
 	comm := &communicatorMock{}
-	if err := p.Provision(new(packer.NoopUi), comm); err != nil {
+	if err := p.Provision(&uiStub{}, comm); err != nil {
 		t.Fatalf("err: %s", err)
 	}
 
@@ -167,7 +166,7 @@ func TestProvisionerProvision_PlaybookFilesWithPlaybookDir(t *testing.T) {
 	}
 
 	comm := &communicatorMock{}
-	if err := p.Provision(new(packer.NoopUi), comm); err != nil {
+	if err := p.Provision(&uiStub{}, comm); err != nil {
 		t.Fatalf("err: %s", err)
 	}
 
@@ -368,8 +367,8 @@ func testProvisionerProvisionDockerWithPlaybookFiles(t *testing.T, templateStrin
 	hooks[packer.HookProvision] = []packer.Hook{
 		&packer.ProvisionHook{
 			Provisioners: []*packer.HookedProvisioner{
-				{Provisioner: ansible, Config: nil, TypeName: ""},
-				{Provisioner: download, Config: nil, TypeName: ""},
+				{ansible, nil, ""},
+				{download, nil, ""},
 			},
 		},
 	}

provisioner/ansible-local/ui_stub.go

@@ -0,0 +1,15 @@
+package ansiblelocal
+
+type uiStub struct{}
+
+func (su *uiStub) Ask(string) (string, error) {
+	return "", nil
+}
+
+func (su *uiStub) Error(string) {}
+
+func (su *uiStub) Machine(string, ...string) {}
+
+func (su *uiStub) Message(string) {}
+
+func (su *uiStub) Say(msg string) {}

provisioner/ansible/adapter_test.go

@@ -24,7 +24,7 @@ func TestAdapter_Serve(t *testing.T) {
 
 	config := &ssh.ServerConfig{}
 
-	ui := new(packer.NoopUi)
+	ui := new(ui)
 
 	sut := newAdapter(done, &l, config, "", newUi(ui), communicator{})
 	go func() {
@@ -93,6 +93,36 @@ func (a addr) String() string {
 	return "test"
 }
 
+type ui int
+
+func (u *ui) Ask(s string) (string, error) {
+	*u++
+	return s, nil
+}
+
+func (u *ui) Say(s string) {
+	*u++
+	log.Println(s)
+}
+
+func (u *ui) Message(s string) {
+	*u++
+	log.Println(s)
+}
+
+func (u *ui) Error(s string) {
+	*u++
+	log.Println(s)
+}
+
+func (u *ui) Machine(s1 string, s2 ...string) {
+	*u++
+	log.Println(s1)
+	for _, s := range s2 {
+		log.Println(s)
+	}
+}
+
 type communicator struct{}
 
 func (c communicator) Start(*packer.RemoteCmd) error {

provisioner/ansible/provisioner.go

@@ -26,7 +26,6 @@ import (
 	"golang.org/x/crypto/ssh"
 
 	"github.com/hashicorp/packer/common"
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/config"
 	"github.com/hashicorp/packer/packer"
 	"github.com/hashicorp/packer/template/interpolate"
@@ -58,6 +57,8 @@ type Config struct {
 	UseSFTP              bool     `mapstructure:"use_sftp"`
 	InventoryDirectory   string   `mapstructure:"inventory_directory"`
 	InventoryFile        string   `mapstructure:"inventory_file"`
+	// for internal use only; unsettable by user.
+	winrmpassword string
 }
 
 type Provisioner struct {
@@ -73,14 +74,35 @@ type PassthroughTemplate struct {
 }
 
 func (p *Provisioner) Prepare(raws ...interface{}) error {
+	// This is a bit of a hack. For provisioners that need access to
+	// auto-generated WinRMPasswords, the mechanism of keeping provisioner data
+	// and build data totally segregated breaks down. We get around this by
+	// having the builder stash the WinRMPassword in the state bag, then
+	// grabbing it out of the statebag inside of StepProvision.  Then, when
+	// the time comes to provision for real, we run the prepare step one more
+	// time, now with WinRMPassword defined in the raws, and can store the
+	// password on the provisioner config without overwriting the rest of the
+	// work we've already done in the first prepare run.
+	if len(raws) == 1 {
+		for k, v := range raws[0].(map[interface{}]interface{}) {
+			if k.(string) == "WinRMPassword" {
+				p.config.winrmpassword = v.(string)
+
+				// Even if WinRMPassword is not gonna be used, we've stored the
+				// key and pointed it to an empty string. That means we'll
+				// always reach this on our second-run of Prepare()
+				return nil
+			}
+		}
+	}
+
 	p.done = make(chan struct{})
 
-	// Create passthrough for winrm password so we can fill it in once we know
-	// it
+	// don't interpolate winrmpassword yet; if we've made it to this line, then
+	// we have not obtained the winrm password yet.
 	p.config.ctx.Data = &PassthroughTemplate{
 		WinRMPassword: `{{.WinRMPassword}}`,
 	}
-
 	err := config.Decode(&p.config, &config.DecodeOpts{
 		Interpolate:        true,
 		InterpolateContext: &p.config.ctx,
@@ -200,8 +222,9 @@ func (p *Provisioner) getVersion() error {
 func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 	ui.Say("Provisioning with Ansible...")
 	// Interpolate env vars to check for .WinRMPassword
+	packer.LogSecretFilter.Set(p.config.winrmpassword)
 	p.config.ctx.Data = &PassthroughTemplate{
-		WinRMPassword: getWinRMPassword(p.config.PackerBuildName),
+		WinRMPassword: p.config.winrmpassword,
 	}
 	for i, envVar := range p.config.AnsibleEnvVars {
 		envVar, err := interpolate.Render(envVar, &p.config.ctx)
@@ -420,12 +443,7 @@ func (p *Provisioner) executeAnsible(ui packer.Ui, comm packer.Communicator, pri
 
 	// remove winrm password from command, if it's been added
 	flattenedCmd := strings.Join(cmd.Args, " ")
-	sanitized := flattenedCmd
-	if len(getWinRMPassword(p.config.PackerBuildName)) > 0 {
-		sanitized = strings.Replace(sanitized,
-			getWinRMPassword(p.config.PackerBuildName), "*****", -1)
-	}
-	ui.Say(fmt.Sprintf("Executing Ansible: %s", sanitized))
+	ui.Say(fmt.Sprintf("Executing Ansible: %s", flattenedCmd))
 
 	if err := cmd.Start(); err != nil {
 		return err
@@ -553,12 +571,6 @@ func newSigner(privKeyFile string) (*signer, error) {
 	return signer, nil
 }
 
-func getWinRMPassword(buildName string) string {
-	winRMPass, _ := commonhelper.RetrieveSharedState("winrm_password", buildName)
-	packer.LogSecretFilter.Set(winRMPass)
-	return winRMPass
-}
-
 // Ui provides concurrency-safe access to packer.Ui.
 type Ui struct {
 	sem chan int
@@ -600,7 +612,3 @@ func (ui *Ui) Machine(t string, args ...string) {
 	ui.ui.Machine(t, args...)
 	<-ui.sem
 }
-
-func (ui *Ui) ProgressBar() packer.ProgressBar {
-	return new(packer.NoopProgressBar)
-}

provisioner/ansible/provisioner_test.go

@@ -16,8 +16,8 @@ import (
 
 // Be sure to remove the Ansible stub file in each test with:
 //   defer os.Remove(config["command"].(string))
-func testConfig(t *testing.T) map[string]interface{} {
-	m := make(map[string]interface{})
+func testConfig(t *testing.T) map[interface{}]interface{} {
+	m := make((map[interface{}]interface{}))
 	wd, err := os.Getwd()
 	if err != nil {
 		t.Fatalf("err: %s", err)

provisioner/file/provisioner.go

@@ -127,12 +127,12 @@ func (p *Provisioner) ProvisionDownload(ui packer.Ui, comm packer.Communicator)
 		defer f.Close()
 
 		// Get a default progress bar
-		pb := packer.NoopProgressBar{}
-		pb.Start(0) // TODO: find size ? Remove ?
-		defer pb.Finish()
+		pb := common.GetProgressBar(ui, &p.config.PackerConfig)
+		bar := pb.Start()
+		defer bar.Finish()
 
 		// Create MultiWriter for the current progress
-		pf := io.MultiWriter(f)
+		pf := io.MultiWriter(f, bar)
 
 		// Download the file
 		if err = comm.Download(src, pf); err != nil {
@@ -176,8 +176,8 @@ func (p *Provisioner) ProvisionUpload(ui packer.Ui, comm packer.Communicator) er
 		}
 
 		// Get a default progress bar
-		bar := ui.ProgressBar()
-		bar.Start(info.Size())
+		pb := common.GetProgressBar(ui, &p.config.PackerConfig)
+		bar := pb.Start()
 		defer bar.Finish()
 
 		// Create ProxyReader for the current progress

provisioner/file/provisioner_test.go

@@ -1,7 +1,6 @@
 package file
 
 import (
-	"bytes"
 	"io/ioutil"
 	"os"
 	"path/filepath"
@@ -100,6 +99,27 @@ func TestProvisionerPrepare_EmptyDestination(t *testing.T) {
 	}
 }
 
+type stubUi struct {
+	sayMessages string
+}
+
+func (su *stubUi) Ask(string) (string, error) {
+	return "", nil
+}
+
+func (su *stubUi) Error(string) {
+}
+
+func (su *stubUi) Machine(string, ...string) {
+}
+
+func (su *stubUi) Message(string) {
+}
+
+func (su *stubUi) Say(msg string) {
+	su.sayMessages += msg
+}
+
 func TestProvisionerProvision_SendsFile(t *testing.T) {
 	var p Provisioner
 	tf, err := ioutil.TempFile("", "packer")
@@ -121,21 +141,18 @@ func TestProvisionerProvision_SendsFile(t *testing.T) {
 		t.Fatalf("err: %s", err)
 	}
 
-	b := bytes.NewBuffer(nil)
-	ui := &packer.BasicUi{
-		Writer: b,
-	}
+	ui := &stubUi{}
 	comm := &packer.MockCommunicator{}
 	err = p.Provision(ui, comm)
 	if err != nil {
 		t.Fatalf("should successfully provision: %s", err)
 	}
 
-	if !strings.Contains(b.String(), tf.Name()) {
+	if !strings.Contains(ui.sayMessages, tf.Name()) {
 		t.Fatalf("should print source filename")
 	}
 
-	if !strings.Contains(b.String(), "something") {
+	if !strings.Contains(ui.sayMessages, "something") {
 		t.Fatalf("should print destination filename")
 	}
 
@@ -180,21 +197,18 @@ func TestProvisionDownloadMkdirAll(t *testing.T) {
 		if err := p.Prepare(config); err != nil {
 			t.Fatalf("err: %s", err)
 		}
-		b := bytes.NewBuffer(nil)
-		ui := &packer.BasicUi{
-			Writer: b,
-		}
+		ui := &stubUi{}
 		comm := &packer.MockCommunicator{}
 		err = p.ProvisionDownload(ui, comm)
 		if err != nil {
 			t.Fatalf("should successfully provision: %s", err)
 		}
 
-		if !strings.Contains(b.String(), tf.Name()) {
+		if !strings.Contains(ui.sayMessages, tf.Name()) {
 			t.Fatalf("should print source filename")
 		}
 
-		if !strings.Contains(b.String(), "something") {
+		if !strings.Contains(ui.sayMessages, "something") {
 			t.Fatalf("should print destination filename")
 		}
 

provisioner/powershell/provisioner.go

@@ -17,7 +17,6 @@ import (
 
 	"github.com/hashicorp/packer/common"
 	"github.com/hashicorp/packer/common/uuid"
-	commonhelper "github.com/hashicorp/packer/helper/common"
 	"github.com/hashicorp/packer/helper/config"
 	"github.com/hashicorp/packer/packer"
 	"github.com/hashicorp/packer/template/interpolate"
@@ -98,6 +97,9 @@ type Config struct {
 	// Changes will not be effective until the system is rebooted."
 	ValidExitCodes []int `mapstructure:"valid_exit_codes"`
 
+	// internal variable, to be written to at provisioner run time.
+	winrmpassword string
+
 	ctx interpolate.Context
 }
 
@@ -117,8 +119,28 @@ type EnvVarsTemplate struct {
 }
 
 func (p *Provisioner) Prepare(raws ...interface{}) error {
-	// Create passthrough for winrm password so we can fill it in once we know
-	// it
+	// This is a bit of a hack. For provisioners that need access to
+	// auto-generated WinRMPasswords, the mechanism of keeping provisioner data
+	// and build data totally segregated breaks down. We get around this by
+	// having the builder stash the WinRMPassword in the state bag, then
+	// grabbing it out of the statebag inside of StepProvision.  Then, when
+	// the time comes to provision for real, we run the prepare step one more
+	// time, now with WinRMPassword defined in the raws, and can store the
+	// password on the provisioner config without overwriting the rest of the
+	// work we've already done in the first prepare run.
+	if len(raws) == 1 {
+		for k, v := range raws[0].(map[interface{}]interface{}) {
+			if k.(string) == "WinRMPassword" {
+				p.config.winrmpassword = v.(string)
+				// Even if WinRMPassword is not gonna be used, we've stored the
+				// key and pointed it to an empty string. That means we'll
+				// always reach this on our second-run of Prepare()
+				return nil
+			}
+		}
+	}
+
+	// Only get here if it's the first time the Provisioner's Prepare is run
 	p.config.ctx.Data = &EnvVarsTemplate{
 		WinRMPassword: `{{.WinRMPassword}}`,
 	}
@@ -135,7 +157,7 @@ func (p *Provisioner) Prepare(raws ...interface{}) error {
 	}, raws...)
 
 	if err != nil {
-		return err
+		return fmt.Errorf("Error decoding powershell provisioner template: %s", err)
 	}
 
 	if p.config.EnvVarFormat == "" {
@@ -261,7 +283,7 @@ func extractScript(p *Provisioner) (string, error) {
 func (p *Provisioner) Provision(ui packer.Ui, comm packer.Communicator) error {
 	ui.Say(fmt.Sprintf("Provisioning with Powershell..."))
 	p.communicator = comm
-
+	packer.LogSecretFilter.Set(p.config.winrmpassword)
 	scripts := make([]string, len(p.config.Scripts))
 	copy(scripts, p.config.Scripts)
 
@@ -392,7 +414,7 @@ func (p *Provisioner) createFlattenedEnvVars(elevated bool) (flattened string) {
 
 	// interpolate environment variables
 	p.config.ctx.Data = &EnvVarsTemplate{
-		WinRMPassword: getWinRMPassword(p.config.PackerBuildName),
+		WinRMPassword: p.config.winrmpassword,
 	}
 	// Split vars into key/value components
 	for _, envVar := range p.config.Vars {
@@ -467,7 +489,7 @@ func (p *Provisioner) createCommandTextNonPrivileged() (command string, err erro
 	p.config.ctx.Data = &ExecuteCommandTemplate{
 		Path:          p.config.RemotePath,
 		Vars:          p.config.RemoteEnvVarPath,
-		WinRMPassword: getWinRMPassword(p.config.PackerBuildName),
+		WinRMPassword: p.config.winrmpassword,
 	}
 	command, err = interpolate.Render(p.config.ExecuteCommand, &p.config.ctx)
 
@@ -479,12 +501,6 @@ func (p *Provisioner) createCommandTextNonPrivileged() (command string, err erro
 	return command, nil
 }
 
-func getWinRMPassword(buildName string) string {
-	winRMPass, _ := commonhelper.RetrieveSharedState("winrm_password", buildName)
-	packer.LogSecretFilter.Set(winRMPass)
-	return winRMPass
-}
-
 func (p *Provisioner) createCommandTextPrivileged() (command string, err error) {
 	// Prepare everything needed to enable the required env vars within the
 	// remote environment
@@ -496,7 +512,7 @@ func (p *Provisioner) createCommandTextPrivileged() (command string, err error)
 	p.config.ctx.Data = &ExecuteCommandTemplate{
 		Path:          p.config.RemotePath,
 		Vars:          p.config.RemoteEnvVarPath,
-		WinRMPassword: getWinRMPassword(p.config.PackerBuildName),
+		WinRMPassword: p.config.winrmpassword,
 	}
 	command, err = interpolate.Render(p.config.ElevatedExecuteCommand, &p.config.ctx)
 	if err != nil {
@@ -555,7 +571,7 @@ func (p *Provisioner) generateElevatedRunner(command string) (uploadedPath strin
 	}
 	// Replace ElevatedPassword for winrm users who used this feature
 	p.config.ctx.Data = &EnvVarsTemplate{
-		WinRMPassword: getWinRMPassword(p.config.PackerBuildName),
+		WinRMPassword: p.config.winrmpassword,
 	}
 
 	p.config.ElevatedPassword, _ = interpolate.Render(p.config.ElevatedPassword, &p.config.ctx)

provisioner/powershell/provisioner_test.go

@@ -14,8 +14,8 @@ import (
 	"github.com/hashicorp/packer/packer"
 )
 
-func testConfig() map[string]interface{} {
-	return map[string]interface{}{
+func testConfig() map[interface{}]interface{} {
+	return map[interface{}]interface{}{
 		"inline": []interface{}{"foo", "bar"},
 	}
 }

provisioner/shell-local/provisioner.go

@@ -10,6 +10,27 @@ type Provisioner struct {
 }
 
 func (p *Provisioner) Prepare(raws ...interface{}) error {
+	// This is a bit of a hack. For provisioners that need access to
+	// auto-generated WinRMPasswords, the mechanism of keeping provisioner data
+	// and build data totally segregated breaks down. We get around this by
+	// having the builder stash the WinRMPassword in the state bag, then
+	// grabbing it out of the statebag inside of StepProvision.  Then, when
+	// the time comes to provision for real, we run the prepare step one more
+	// time, now with WinRMPassword defined in the raws, and can store the
+	// password on the provisioner config without overwriting the rest of the
+	// work we've already done in the first prepare run.
+	if len(raws) == 1 {
+		for k, v := range raws[0].(map[interface{}]interface{}) {
+			if k.(string) == "WinRMPassword" {
+				p.config.WinRMPassword = v.(string)
+				// Even if WinRMPassword is not gonna be used, we've stored the
+				// key and pointed it to an empty string. That means we'll
+				// always reach this on our second-run of Prepare()
+				return nil
+			}
+		}
+	}
+
 	err := sl.Decode(&p.config, raws...)
 	if err != nil {
 		return err

provisioner/shell-local/provisioner_test.go

@@ -48,8 +48,8 @@ func TestConfigPrepare(t *testing.T) {
 	}
 }
 
-func testConfig(t *testing.T) map[string]interface{} {
-	return map[string]interface{}{
+func testConfig(t *testing.T) map[interface{}]interface{} {
+	return map[interface{}]interface{}{
 		"command": "echo foo",
 	}
 }