Cut version 1.6.2

This reverts commit c35837ee49.

CHANGELOG.md

@@ -1,4 +1,11 @@
-## 1.6.2 (Upcoming)
+## 1.6.2 (August 28, 2020)
+
+### FEATURES:
+* **New command** `hcl2_upgrade` is a JSON to HCL2 transpiler that allows users
+    to transform an existing JSON configuration template into its HCL2 template
+    equivalent. Please see [hcl2_upgrade command
+    docs](https://packer.io/docs/commands/hcl2_upgrade) for more details.
+    [GH-9659]
 
 ### IMPROVEMENTS:
 * builder/amazon:  Add all of the custom AWS template engines to `build`
@@ -8,14 +15,22 @@
 * builder/azure: Add FreeBSD support to azure/chroot builder. [GH-9697]
 * builder/vmware-esx: Add `network_name` option to vmware so that users can set
     a network without using vmx data. [GH-9718]
+* builder/vmware-vmx: Add additional disk configuration option.  Previously
+    only implemented for vmware-iso builder [GH-9815]
 * builder/vmware: Add a `remote_output_directory option` so users can tell
     Packer where on a datastore to create a vm. [GH-9784]
+* builder/vmware: Add option to export to ovf or ova from a local vmware build
+    [GH-9825]
 * builder/vmware: Add progress tracker to vmware-esx5 iso upload. [GH-9779]
+* builder/vsphere-iso:  Add support for building on a single ESXi host
+    [GH-9793]
 * builder/vsphere: Add new `directory_permission` config export option.
     [GH-9704]
 * builder/vsphere: Add option to import OVF templates to the Content Library
     [GH-9755]
 * builder/vsphere: Add step and options to customize cloned VMs. [GH-9665]
+* builder/vsphere: Update `iso_paths` to support reading ISOs from Content
+    Library paths [GH-9801]
 * core/hcl: Add provisioner "override" option to HCL2 templates. [GH-9764]
 * core/hcl: Add vault integration as an HCL2 function function. [GH-9746]
 * core: Add colored prefix to progress bar so it's clearer what build each
@@ -27,6 +42,8 @@
     [GH-9773]
 * post-processor/vsphere: Improve UI to catch bad credentials and print errors.
     [GH-9649]
+* provisioner/ansible-remote: Add `ansible_ssh_extra_args` so users can specify
+    extra arguments to pass to ssh [GH-9821]
 * provisioner/file: Clean up, bugfix, and document previously-hidden `sources`
     option. [GH-9725] [GH-9735]
 * provisioner/salt-masterless: Add option to option to download community
@@ -39,6 +56,11 @@
     binaries. [GH-9706]
 * builder/amazon-ebssurrogate: Make skip_save_build_region option work in the
     ebssurrogate builder, not just the ebs builder. [GH-9666]
+* builder/amazon: Add retry logic to the spot instance creation step to handle
+    "Invalid IAM Instance Profile name" errors [GH-9810]
+* builder/amazon: Update the `aws_secretsmanager` function to read from the AWS
+    credentials file for obtaining default region information; fixes the
+    'MissingRegion' error when AWS_REGION is not set [GH-9781]
 * builder/file: Make sure that UploadDir receives the interpolated destination.
     [GH-9698]
 * builder/googlecompute: Fix bug where startup script hang would cause export
@@ -61,6 +83,10 @@
     interpolation. [GH-9673]
 * post-processor/vsphere-template: Fix ReregisterVM to default to true instead
     of false. [GH-9736]
+* post-processor/yandex-export: Fix issue when validating region_name [GH-9814]
+* provisioner/inspec: Fix the 'Unsupported argument; An argument named
+    "command"' error when using the inspec provisioner in an HCL2 configuration
+    [GH-9800]
 
 ## 1.6.1 (July 30, 2020)
 

Makefile

@@ -25,7 +25,7 @@ export GOLDFLAGS
 .PHONY: bin checkversion ci ci-lint default install-build-deps install-gen-deps fmt fmt-docs fmt-examples generate install-lint-deps lint \
 	releasebin test testacc testrace
 
-default: install-build-deps install-gen-deps generate bin
+default: install-build-deps install-gen-deps generate dev
 
 ci: testrace ## Test in continuous integration
 

builder/amazon/common/run_config.go

@@ -53,14 +53,14 @@ type VpcFilterOptions struct {
 }
 
 type Statement struct {
-	Effect   string
-	Action   []string
-	Resource []string
+	Effect   string   `mapstructure:"Effect" required:"false"`
+	Action   []string `mapstructure:"Action" required:"false"`
+	Resource []string `mapstructure:"Resource" required:"false"`
 }
 
 type PolicyDocument struct {
-	Version   string
-	Statement []Statement
+	Version   string      `mapstructure:"Version" required:"false"`
+	Statement []Statement `mapstructure:"Statement" required:"false"`
 }
 
 type SecurityGroupFilterOptions struct {

builder/amazon/common/run_config.hcl2spec.go

@@ -39,8 +39,8 @@ func (*FlatAmiFilterOptions) HCL2Spec() map[string]hcldec.Spec {
 // FlatPolicyDocument is an auto-generated flat version of PolicyDocument.
 // Where the contents of a field with a `mapstructure:,squash` tag are bubbled up.
 type FlatPolicyDocument struct {
-	Version   *string         `cty:"version" hcl:"version"`
-	Statement []FlatStatement `cty:"statement" hcl:"statement"`
+	Version   *string         `mapstructure:"Version" required:"false" cty:"Version" hcl:"Version"`
+	Statement []FlatStatement `mapstructure:"Statement" required:"false" cty:"Statement" hcl:"Statement"`
 }
 
 // FlatMapstructure returns a new FlatPolicyDocument.
@@ -55,8 +55,8 @@ func (*PolicyDocument) FlatMapstructure() interface{ HCL2Spec() map[string]hclde
 // The decoded values from this spec will then be applied to a FlatPolicyDocument.
 func (*FlatPolicyDocument) HCL2Spec() map[string]hcldec.Spec {
 	s := map[string]hcldec.Spec{
-		"version":   &hcldec.AttrSpec{Name: "version", Type: cty.String, Required: false},
-		"statement": &hcldec.BlockListSpec{TypeName: "statement", Nested: hcldec.ObjectSpec((*FlatStatement)(nil).HCL2Spec())},
+		"Version":   &hcldec.AttrSpec{Name: "Version", Type: cty.String, Required: false},
+		"Statement": &hcldec.BlockListSpec{TypeName: "Statement", Nested: hcldec.ObjectSpec((*FlatStatement)(nil).HCL2Spec())},
 	}
 	return s
 }
@@ -89,9 +89,9 @@ func (*FlatSecurityGroupFilterOptions) HCL2Spec() map[string]hcldec.Spec {
 // FlatStatement is an auto-generated flat version of Statement.
 // Where the contents of a field with a `mapstructure:,squash` tag are bubbled up.
 type FlatStatement struct {
-	Effect   *string  `cty:"effect" hcl:"effect"`
-	Action   []string `cty:"action" hcl:"action"`
-	Resource []string `cty:"resource" hcl:"resource"`
+	Effect   *string  `mapstructure:"Effect" required:"false" cty:"Effect" hcl:"Effect"`
+	Action   []string `mapstructure:"Action" required:"false" cty:"Action" hcl:"Action"`
+	Resource []string `mapstructure:"Resource" required:"false" cty:"Resource" hcl:"Resource"`
 }
 
 // FlatMapstructure returns a new FlatStatement.
@@ -106,9 +106,9 @@ func (*Statement) FlatMapstructure() interface{ HCL2Spec() map[string]hcldec.Spe
 // The decoded values from this spec will then be applied to a FlatStatement.
 func (*FlatStatement) HCL2Spec() map[string]hcldec.Spec {
 	s := map[string]hcldec.Spec{
-		"effect":   &hcldec.AttrSpec{Name: "effect", Type: cty.String, Required: false},
-		"action":   &hcldec.AttrSpec{Name: "action", Type: cty.List(cty.String), Required: false},
-		"resource": &hcldec.AttrSpec{Name: "resource", Type: cty.List(cty.String), Required: false},
+		"Effect":   &hcldec.AttrSpec{Name: "Effect", Type: cty.String, Required: false},
+		"Action":   &hcldec.AttrSpec{Name: "Action", Type: cty.List(cty.String), Required: false},
+		"Resource": &hcldec.AttrSpec{Name: "Resource", Type: cty.List(cty.String), Required: false},
 	}
 	return s
 }

builder/amazon/common/step_run_spot_instance.go

@@ -6,9 +6,11 @@ import (
 	"fmt"
 	"io/ioutil"
 	"log"
+	"strings"
 	"time"
 
 	"github.com/aws/aws-sdk-go/aws"
+	"github.com/aws/aws-sdk-go/aws/request"
 	"github.com/aws/aws-sdk-go/service/ec2"
 	"github.com/hashicorp/packer/common/random"
 	"github.com/hashicorp/packer/common/retry"
@@ -278,23 +280,39 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
 		Type: aws.String("instant"),
 	}
 
+	var createOutput *ec2.CreateFleetOutput
+
+	err = retry.Config{
+		Tries: 11,
+		ShouldRetry: func(err error) bool {
+			if strings.Contains(err.Error(), "Invalid IAM Instance Profile name") {
+				// eventual consistency of the profile. PutRolePolicy &
+				// AddRoleToInstanceProfile are eventually consistent and once
+				// we can wait on those operations, this can be removed.
+				return true
+			}
+			return request.IsErrorRetryable(err)
+		},
+		RetryDelay: (&retry.Backoff{InitialBackoff: 500 * time.Millisecond, MaxBackoff: 30 * time.Second, Multiplier: 2}).Linear,
+	}.Run(ctx, func(ctx context.Context) error {
+		createOutput, err = ec2conn.CreateFleet(createFleetInput)
+
+		if err == nil && createOutput.Errors != nil {
+			err = fmt.Errorf("errors: %v", createOutput.Errors)
+		}
+		if err != nil {
+			log.Printf("create request failed %v", err)
+		}
+		return err
+	})
+
 	// Create the request for the spot instance.
-	req, createOutput := ec2conn.CreateFleetRequest(createFleetInput)
-	ui.Message(fmt.Sprintf("Sending spot request (%s)...", req.RequestID))
-	// Actually send the spot connection request.
-	err = req.Send()
 	if err != nil {
 		if createOutput.FleetId != nil {
 			err = fmt.Errorf("Error waiting for fleet request (%s): %s", *createOutput.FleetId, err)
 		} else {
 			err = fmt.Errorf("Error waiting for fleet request: %s", err)
 		}
-		state.Put("error", err)
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-
-	if len(createOutput.Instances) == 0 {
 		// We can end up with errors because one of the allowed availability
 		// zones doesn't have one of the allowed instance types; as long as
 		// an instance is launched, these errors aren't important.
@@ -308,6 +326,9 @@ func (s *StepRunSpotInstance) Run(ctx context.Context, state multistep.StateBag)
 			ui.Error(err.Error())
 			return multistep.ActionHalt
 		}
+		state.Put("error", err)
+		ui.Error(err.Error())
+		return multistep.ActionHalt
 	}
 
 	instanceId = *createOutput.Instances[0].InstanceIds[0]

builder/vmware/common/disk_config.go

@@ -0,0 +1,69 @@
+//go:generate struct-markdown
+
+package common
+
+import (
+	"github.com/hashicorp/packer/template/interpolate"
+)
+
+type DiskConfig struct {
+	// The size(s) of any additional
+	// hard disks for the VM in megabytes. If this is not specified then the VM
+	// will only contain a primary hard disk. The builder uses expandable, not
+	// fixed-size virtual hard disks, so the actual file representing the disk will
+	// not use the full size unless it is full.
+	AdditionalDiskSize []uint `mapstructure:"disk_additional_size" required:"false"`
+	// The adapter type of the VMware virtual disk to create. This option is
+	// for advanced usage, modify only if you know what you're doing. Some of
+	// the options you can specify are `ide`, `sata`, `nvme` or `scsi` (which
+	// uses the "lsilogic" scsi interface by default). If you specify another
+	// option, Packer will assume that you're specifying a `scsi` interface of
+	// that specified type. For more information, please consult [Virtual Disk
+	// Manager User's Guide](http://www.vmware.com/pdf/VirtualDiskManager.pdf)
+	// for desktop VMware clients. For ESXi, refer to the proper ESXi
+	// documentation.
+	DiskAdapterType string `mapstructure:"disk_adapter_type" required:"false"`
+	// The filename of the virtual disk that'll be created,
+	// without the extension. This defaults to "disk".
+	DiskName string `mapstructure:"vmdk_name" required:"false"`
+	// The type of VMware virtual disk to create. This
+	// option is for advanced usage.
+	//
+	//   For desktop VMware clients:
+	//
+	//   Type ID | Description
+	//   ------- | ---
+	//   `0`     | Growable virtual disk contained in a single file (monolithic sparse).
+	//   `1`     | Growable virtual disk split into 2GB files (split sparse).
+	//   `2`     | Preallocated virtual disk contained in a single file (monolithic flat).
+	//   `3`     | Preallocated virtual disk split into 2GB files (split flat).
+	//   `4`     | Preallocated virtual disk compatible with ESX server (VMFS flat).
+	//   `5`     | Compressed disk optimized for streaming.
+	//
+	//   The default is `1`.
+	//
+	//   For ESXi, this defaults to `zeroedthick`. The available options for ESXi
+	//   are: `zeroedthick`, `eagerzeroedthick`, `thin`. `rdm:dev`, `rdmp:dev`,
+	//   `2gbsparse` are not supported. Due to default disk compaction, when using
+	//   `zeroedthick` or `eagerzeroedthick` set `skip_compaction` to `true`.
+	//
+	//   For more information, please consult the [Virtual Disk Manager User's
+	//   Guide](https://www.vmware.com/pdf/VirtualDiskManager.pdf) for desktop
+	//   VMware clients. For ESXi, refer to the proper ESXi documentation.
+	DiskTypeId string `mapstructure:"disk_type_id" required:"false"`
+}
+
+func (c *DiskConfig) Prepare(ctx *interpolate.Context) []error {
+	var errs []error
+
+	if c.DiskName == "" {
+		c.DiskName = "disk"
+	}
+
+	if c.DiskAdapterType == "" {
+		// Default is lsilogic
+		c.DiskAdapterType = "lsilogic"
+	}
+
+	return errs
+}

builder/vmware/common/driver.go

@@ -77,6 +77,9 @@ type Driver interface {
 
 	// Get the host ip address for the vm
 	HostIP(multistep.StateBag) (string, error)
+
+	// Export the vm to ovf or ova format using ovftool
+	Export([]string) error
 }
 
 // NewDriver returns a new driver implementation for this operating
@@ -600,3 +603,28 @@ func (d *VmwareDriver) HostIP(state multistep.StateBag) (string, error) {
 	}
 	return "", fmt.Errorf("Unable to find host IP from devices %v, last error: %s", devices, lastError)
 }
+
+func GetOVFTool() string {
+	ovftool := "ovftool"
+	if runtime.GOOS == "windows" {
+		ovftool = "ovftool.exe"
+	}
+
+	if _, err := exec.LookPath(ovftool); err != nil {
+		return ""
+	}
+	return ovftool
+}
+
+func (d *VmwareDriver) Export(args []string) error {
+	ovftool := GetOVFTool()
+	if ovftool == "" {
+		return fmt.Errorf("Error: ovftool not found")
+	}
+	cmd := exec.Command(ovftool, args...)
+	if _, _, err := runAndLog(cmd); err != nil {
+		return err
+	}
+
+	return nil
+}

builder/vmware/common/driver_config.go

@@ -84,13 +84,28 @@ func (c *DriverConfig) Prepare(ctx *interpolate.Context) []error {
 }
 
 func (c *DriverConfig) Validate(SkipExport bool) error {
-	if c.RemoteType == "" || SkipExport == true {
+	if SkipExport {
 		return nil
 	}
-	if c.RemotePassword == "" {
-		return fmt.Errorf("exporting the vm (with ovftool) requires that " +
-			"you set a value for remote_password")
+
+	if c.RemoteType != "" && c.RemotePassword == "" {
+		return fmt.Errorf("exporting the vm from esxi with ovftool requires " +
+			"that you set a value for remote_password")
 	}
+
+	if c.RemoteType == "" {
+		// Validate that tool exists, but no need to validate credentials.
+		ovftool := GetOVFTool()
+		if ovftool != "" {
+			return nil
+		} else {
+			return fmt.Errorf("Couldn't find ovftool in path! Please either " +
+				"set `skip_export = true` and remove the `format` option " +
+				"from your template, or make sure ovftool is installed on " +
+				"your build system. ")
+		}
+	}
+
 	if c.SkipValidateCredentials {
 		return nil
 	}

builder/vmware/common/driver_esx5.go

@@ -699,6 +699,10 @@ func (d *ESX5Driver) Download(src, dst string) error {
 	return d.comm.Download(d.datastorePath(src), file)
 }
 
+func (d *ESX5Driver) Export(args []string) error {
+	return d.base.Export(args)
+}
+
 // VerifyChecksum checks that file on the esxi instance matches hash
 func (d *ESX5Driver) VerifyChecksum(hash string, file string) bool {
 	if hash == "none" {

builder/vmware/common/driver_mock.go

@@ -27,6 +27,9 @@ type DriverMock struct {
 	CreateDiskTypeId      string
 	CreateDiskErr         error
 
+	ExportCalled bool
+	ExportArgs   []string
+
 	IsRunningCalled bool
 	IsRunningPath   string
 	IsRunningResult bool
@@ -254,6 +257,12 @@ func (d *DriverMock) Verify() error {
 	return d.VerifyErr
 }
 
+func (d *DriverMock) Export(args []string) error {
+	d.ExportCalled = true
+	d.ExportArgs = args
+	return nil
+}
+
 func (d *DriverMock) GetVmwareDriver() VmwareDriver {
 	var state VmwareDriver
 	state.DhcpLeasesPath = func(string) string {

builder/vmware/common/export_config.go

@@ -10,33 +10,30 @@ import (
 
 type ExportConfig struct {
 	// Either "ovf", "ova" or "vmx", this specifies the output
-	// format of the exported virtual machine. This defaults to "ovf".
-	// Before using this option, you need to install ovftool. This option
-	// currently only works when option remote_type is set to "esx5".
+	// format of the exported virtual machine. This defaults to "ovf" for
+	// remote (esx) builds, and "vmx" for local builds.
+	// Before using this option, you need to install ovftool.
 	// Since ovftool is only capable of password based authentication
-	// remote_password must be set when exporting the VM.
+	// remote_password must be set when exporting the VM from a remote instance.
+	// If you are building locally, Packer will create a vmx and then
+	// export that vm to an ovf or ova. Packer will not delete the vmx and vmdk
+	// files; this is left up to the user if you don't want to keep those
+	// files.
 	Format string `mapstructure:"format" required:"false"`
 	// Extra options to pass to ovftool during export. Each item in the array
 	// is a new argument. The options `--noSSLVerify`, `--skipManifestCheck`,
-	// and `--targetType` are reserved, and should not be passed to this
-	// argument. Currently, exporting the build VM (with ovftool) is only
-	// supported when building on ESXi e.g. when `remote_type` is set to
-	// `esx5`. See the [Building on a Remote vSphere
-	// Hypervisor](/docs/builders/vmware-iso#building-on-a-remote-vsphere-hypervisor)
-	// section below for more info.
+	// and `--targetType` are used by Packer for remote exports, and should not
+	// be passed to this argument. For ovf/ova exports from local builds, Packer
+	// does not automatically set any ovftool options.
 	OVFToolOptions []string `mapstructure:"ovftool_options" required:"false"`
-	// Defaults to `false`. When enabled, Packer will not export the VM. Useful
-	// if the build output is not the resultant image, but created inside the
-	// VM. Currently, exporting the build VM is only supported when building on
-	// ESXi e.g. when `remote_type` is set to `esx5`. See the [Building on a
-	// Remote vSphere
-	// Hypervisor](/docs/builders/vmware-iso#building-on-a-remote-vsphere-hypervisor)
-	// section below for more info.
+	// Defaults to `false`. When true, Packer will not export the VM. This can
+	// be useful if the build output is not the resultant image, but created
+	// inside the VM.
 	SkipExport bool `mapstructure:"skip_export" required:"false"`
-	// Set this to true if you would like to keep
-	// the VM registered with the remote ESXi server. If you do not need to export
-	// the vm, then also set skip_export: true in order to avoid an unnecessary
-	// step of using ovftool to export the vm. Defaults to false.
+	// Set this to true if you would like to keep a remotely-built
+	// VM registered with the remote ESXi server. If you do not need to export
+	// the vm, then also set `skip_export: true` in order to avoid unnecessarily
+	// using ovftool to export the vm. Defaults to false.
 	KeepRegistered bool `mapstructure:"keep_registered" required:"false"`
 	// VMware-created disks are defragmented and
 	// compacted at the end of the build process using vmware-vdiskmanager or
@@ -56,5 +53,6 @@ func (c *ExportConfig) Prepare(ctx *interpolate.Context) []error {
 				errs, fmt.Errorf("format must be one of ova, ovf, or vmx"))
 		}
 	}
+
 	return errs
 }

builder/vmware/common/step_create_disks.go

@@ -1,4 +1,4 @@
-package iso
+package common
 
 import (
 	"context"
@@ -6,7 +6,6 @@ import (
 	"log"
 	"path/filepath"
 
-	vmwcommon "github.com/hashicorp/packer/builder/vmware/common"
 	"github.com/hashicorp/packer/helper/multistep"
 	"github.com/hashicorp/packer/packer"
 )
@@ -20,11 +19,18 @@ import (
 //
 // Produces:
 //   disk_full_paths ([]string) - The full paths to all created disks
-type stepCreateDisk struct{}
+type StepCreateDisks struct {
+	OutputDir          *string
+	CreateMainDisk     bool
+	DiskName           string
+	MainDiskSize       uint
+	AdditionalDiskSize []uint
+	DiskAdapterType    string
+	DiskTypeId         string
+}
 
-func (stepCreateDisk) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
-	config := state.Get("config").(*Config)
-	driver := state.Get("driver").(vmwcommon.Driver)
+func (s *StepCreateDisks) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
+	driver := state.Get("driver").(Driver)
 	ui := state.Get("ui").(packer.Ui)
 
 	ui.Say("Creating required virtual machine disks")
@@ -32,13 +38,15 @@ func (stepCreateDisk) Run(ctx context.Context, state multistep.StateBag) multist
 	// Users can configure disks at several locations in the template so
 	// first collate all the disk requirements
 	var diskFullPaths, diskSizes []string
-	// The 'main' or 'default' disk
-	diskFullPaths = append(diskFullPaths, filepath.Join(config.OutputDir, config.DiskName+".vmdk"))
-	diskSizes = append(diskSizes, fmt.Sprintf("%dM", uint64(config.DiskSize)))
+	// The 'main' or 'default' disk, only used in vmware-iso
+	if s.CreateMainDisk {
+		diskFullPaths = append(diskFullPaths, filepath.Join(*s.OutputDir, s.DiskName+".vmdk"))
+		diskSizes = append(diskSizes, fmt.Sprintf("%dM", uint64(s.MainDiskSize)))
+	}
 	// Additional disks
-	if len(config.AdditionalDiskSize) > 0 {
-		for i, diskSize := range config.AdditionalDiskSize {
-			path := filepath.Join(config.OutputDir, fmt.Sprintf("%s-%d.vmdk", config.DiskName, i+1))
+	if len(s.AdditionalDiskSize) > 0 {
+		for i, diskSize := range s.AdditionalDiskSize {
+			path := filepath.Join(*s.OutputDir, fmt.Sprintf("%s-%d.vmdk", s.DiskName, i+1))
 			diskFullPaths = append(diskFullPaths, path)
 			size := fmt.Sprintf("%dM", uint64(diskSize))
 			diskSizes = append(diskSizes, size)
@@ -50,7 +58,7 @@ func (stepCreateDisk) Run(ctx context.Context, state multistep.StateBag) multist
 		log.Printf("[INFO] Creating disk with Path: %s and Size: %s", diskFullPath, diskSizes[i])
 		// Additional disks currently use the same adapter type and disk
 		// type as specified for the main disk
-		if err := driver.CreateDisk(diskFullPath, diskSizes[i], config.DiskAdapterType, config.DiskTypeId); err != nil {
+		if err := driver.CreateDisk(diskFullPath, diskSizes[i], s.DiskAdapterType, s.DiskTypeId); err != nil {
 			err := fmt.Errorf("Error creating disk: %s", err)
 			state.Put("error", err)
 			ui.Error(err.Error())
@@ -63,4 +71,4 @@ func (stepCreateDisk) Run(ctx context.Context, state multistep.StateBag) multist
 	return multistep.ActionContinue
 }
 
-func (stepCreateDisk) Cleanup(multistep.StateBag) {}
+func (s *StepCreateDisks) Cleanup(multistep.StateBag) {}

builder/vmware/common/step_create_disks_test.go

@@ -0,0 +1,144 @@
+package common
+
+import (
+	"context"
+	"path/filepath"
+	"testing"
+
+	"github.com/hashicorp/packer/helper/multistep"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestStepCreateDisks_impl(t *testing.T) {
+	var _ multistep.Step = new(StepCreateDisks)
+}
+
+func strPtr(s string) *string {
+	return &s
+}
+func NewTestCreateDiskStep() *StepCreateDisks {
+	return &StepCreateDisks{
+		OutputDir:          strPtr("output_dir"),
+		CreateMainDisk:     true,
+		DiskName:           "disk_name",
+		MainDiskSize:       uint(1024),
+		AdditionalDiskSize: []uint{},
+		DiskAdapterType:    "fake_adapter",
+		DiskTypeId:         "1",
+	}
+}
+
+func TestStepCreateDisks_MainOnly(t *testing.T) {
+	state := testState(t)
+	step := NewTestCreateDiskStep()
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	driver := state.Get("driver").(*DriverMock)
+	if !driver.CreateDiskCalled {
+		t.Fatalf("Should have called create disk.")
+	}
+
+	diskFullPaths, ok := state.Get("disk_full_paths").([]string)
+	if !ok {
+		t.Fatalf("Should be able to load disk_full_paths from state")
+	}
+
+	assert.Equal(t, diskFullPaths, []string{filepath.Join("output_dir", "disk_name.vmdk")})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepCreateDisks_MainAndExtra(t *testing.T) {
+	state := testState(t)
+	step := NewTestCreateDiskStep()
+	step.AdditionalDiskSize = []uint{1024, 2048, 4096}
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	driver := state.Get("driver").(*DriverMock)
+	if !driver.CreateDiskCalled {
+		t.Fatalf("Should have called create disk.")
+	}
+
+	diskFullPaths, ok := state.Get("disk_full_paths").([]string)
+	if !ok {
+		t.Fatalf("Should be able to load disk_full_paths from state")
+	}
+
+	assert.Equal(t, diskFullPaths,
+		[]string{
+			filepath.Join("output_dir", "disk_name.vmdk"),
+			filepath.Join("output_dir", "disk_name-1.vmdk"),
+			filepath.Join("output_dir", "disk_name-2.vmdk"),
+			filepath.Join("output_dir", "disk_name-3.vmdk"),
+		})
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepCreateDisks_ExtraOnly(t *testing.T) {
+	state := testState(t)
+	step := NewTestCreateDiskStep()
+	step.CreateMainDisk = false
+	step.AdditionalDiskSize = []uint{1024, 2048, 4096}
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	driver := state.Get("driver").(*DriverMock)
+	if !driver.CreateDiskCalled {
+		t.Fatalf("Should have called create disk.")
+	}
+
+	diskFullPaths, ok := state.Get("disk_full_paths").([]string)
+	if !ok {
+		t.Fatalf("Should be able to load disk_full_paths from state")
+	}
+
+	assert.Equal(t, diskFullPaths,
+		[]string{
+			filepath.Join("output_dir", "disk_name-1.vmdk"),
+			filepath.Join("output_dir", "disk_name-2.vmdk"),
+			filepath.Join("output_dir", "disk_name-3.vmdk"),
+		})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepCreateDisks_Nothing(t *testing.T) {
+	state := testState(t)
+	step := NewTestCreateDiskStep()
+	step.CreateMainDisk = false
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	driver := state.Get("driver").(*DriverMock)
+	if driver.CreateDiskCalled {
+		t.Fatalf("Should not have called create disk.")
+	}
+
+	// Cleanup
+	step.Cleanup(state)
+}

builder/vmware/common/step_export.go

@@ -1,13 +1,11 @@
 package common
 
 import (
-	"bytes"
 	"context"
 	"fmt"
 	"net/url"
 	"os"
-	"os/exec"
-	"runtime"
+	"path/filepath"
 	"strings"
 
 	"github.com/hashicorp/packer/helper/multistep"
@@ -15,30 +13,15 @@ import (
 )
 
 // This step exports a VM built on ESXi using ovftool
-//
-// Uses:
-//   display_name string
 type StepExport struct {
 	Format         string
 	SkipExport     bool
 	VMName         string
 	OVFToolOptions []string
-	OutputDir      string
+	OutputDir      *string
 }
 
-func GetOVFTool() string {
-	ovftool := "ovftool"
-	if runtime.GOOS == "windows" {
-		ovftool = "ovftool.exe"
-	}
-
-	if _, err := exec.LookPath(ovftool); err != nil {
-		return ""
-	}
-	return ovftool
-}
-
-func (s *StepExport) generateArgs(c *DriverConfig, displayName string, hidePassword bool) ([]string, error) {
+func (s *StepExport) generateRemoteExportArgs(c *DriverConfig, displayName string, hidePassword bool, exportOutputPath string) ([]string, error) {
 
 	ovftool_uri := fmt.Sprintf("vi://%s/%s", c.RemoteHost, displayName)
 	u, err := url.Parse(ovftool_uri)
@@ -57,7 +40,15 @@ func (s *StepExport) generateArgs(c *DriverConfig, displayName string, hidePassw
 		"--skipManifestCheck",
 		"-tt=" + s.Format,
 		u.String(),
-		s.OutputDir,
+		filepath.Join(exportOutputPath, s.VMName+"."+s.Format),
+	}
+	return append(s.OVFToolOptions, args...), nil
+}
+
+func (s *StepExport) generateLocalExportArgs(exportOutputPath string) ([]string, error) {
+	args := []string{
+		filepath.Join(exportOutputPath, s.VMName+".vmx"),
+		filepath.Join(exportOutputPath, s.VMName+"."+s.Format),
 	}
 	return append(s.OVFToolOptions, args...), nil
 }
@@ -65,6 +56,7 @@ func (s *StepExport) generateArgs(c *DriverConfig, displayName string, hidePassw
 func (s *StepExport) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
 	c := state.Get("driverConfig").(*DriverConfig)
 	ui := state.Get("ui").(packer.Ui)
+	driver := state.Get("driver").(Driver)
 
 	// Skip export if requested
 	if s.SkipExport {
@@ -72,57 +64,72 @@ func (s *StepExport) Run(ctx context.Context, state multistep.StateBag) multiste
 		return multistep.ActionContinue
 	}
 
-	if c.RemoteType != "esx5" {
-		ui.Say("Skipping export of virtual machine (export is allowed only for ESXi)...")
-		return multistep.ActionContinue
+	// load output path from state. If it doesn't exist, just use the local
+	// outputdir.
+	exportOutputPath, ok := state.Get("export_output_path").(string)
+	if !ok || exportOutputPath == "" {
+		if *s.OutputDir != "" {
+			exportOutputPath = *s.OutputDir
+		} else {
+			exportOutputPath = s.VMName
+		}
 	}
 
-	ovftool := GetOVFTool()
-	if ovftool == "" {
-		err := fmt.Errorf("Error ovftool not found")
-		state.Put("error", err)
+	err := os.MkdirAll(exportOutputPath, 0755)
+	if err != nil {
+		state.Put("error creating export directory", err)
 		ui.Error(err.Error())
 		return multistep.ActionHalt
 	}
 
-	// Export the VM
-	if s.OutputDir == "" {
-		s.OutputDir = s.VMName + "." + s.Format
-	}
-
-	os.MkdirAll(s.OutputDir, 0755)
-
 	ui.Say("Exporting virtual machine...")
 	var displayName string
 	if v, ok := state.GetOk("display_name"); ok {
 		displayName = v.(string)
 	}
-	ui_args, err := s.generateArgs(c, displayName, true)
-	if err != nil {
-		err := fmt.Errorf("Couldn't generate ovftool uri: %s", err)
-		state.Put("error", err)
-		ui.Error(err.Error())
-		return multistep.ActionHalt
+
+	var args, ui_args []string
+
+	ovftool := GetOVFTool()
+	if c.RemoteType == "esx5" {
+		// Generate arguments for the ovftool command, but obfuscating the
+		// password that we can log the command to the UI for debugging.
+		ui_args, err := s.generateRemoteExportArgs(c, displayName, true, exportOutputPath)
+		if err != nil {
+			err := fmt.Errorf("Couldn't generate ovftool export args: %s", err)
+			state.Put("error", err)
+			ui.Error(err.Error())
+			return multistep.ActionHalt
+		}
+		ui.Message(fmt.Sprintf("Executing: %s %s", ovftool,
+			strings.Join(ui_args, " ")))
+		// Re-run the generate command, this time without obfuscating the
+		// password, so we can actually use it.
+		args, err = s.generateRemoteExportArgs(c, displayName, false, exportOutputPath)
+		if err != nil {
+			err := fmt.Errorf("Couldn't generate ovftool export args: %s", err)
+			state.Put("error", err)
+			ui.Error(err.Error())
+			return multistep.ActionHalt
+		}
+	} else {
+		args, err = s.generateLocalExportArgs(exportOutputPath)
+		ui.Message(fmt.Sprintf("Executing: %s %s", ovftool,
+			strings.Join(ui_args, " ")))
 	}
-	ui.Message(fmt.Sprintf("Executing: %s %s", ovftool, strings.Join(ui_args, " ")))
-	var out bytes.Buffer
-	args, err := s.generateArgs(c, displayName, false)
 	if err != nil {
-		err := fmt.Errorf("Couldn't generate ovftool uri: %s", err)
-		state.Put("error", err)
-		ui.Error(err.Error())
-		return multistep.ActionHalt
-	}
-	cmd := exec.Command(ovftool, args...)
-	cmd.Stdout = &out
-	if err := cmd.Run(); err != nil {
-		err := fmt.Errorf("Error exporting virtual machine: %s\n%s\n", err, out.String())
+		err := fmt.Errorf("Couldn't generate ovftool export args: %s", err)
 		state.Put("error", err)
 		ui.Error(err.Error())
 		return multistep.ActionHalt
 	}
 
-	ui.Message(out.String())
+	if err := driver.Export(args); err != nil {
+		err := fmt.Errorf("Error performing ovftool export: %s", err)
+		state.Put("error", err)
+		ui.Error(err.Error())
+		return multistep.ActionHalt
+	}
 
 	return multistep.ActionContinue
 }

builder/vmware/common/step_export_test.go

@@ -2,22 +2,41 @@ package common
 
 import (
 	"context"
+	"path/filepath"
 	"testing"
 
 	"github.com/hashicorp/packer/helper/multistep"
+	"github.com/stretchr/testify/assert"
 )
 
 func TestStepExport_impl(t *testing.T) {
 	var _ multistep.Step = new(StepExport)
 }
 
-func testStepExport_wrongtype_impl(t *testing.T, remoteType string) {
+func stringPointer(s string) *string {
+	return &s
+}
+
+func remoteExportTestState(t *testing.T) multistep.StateBag {
 	state := testState(t)
+	driverConfig := &DriverConfig{
+		RemoteHost:     "123.45.67.8",
+		RemotePassword: "password",
+		RemoteUser:     "user",
+		RemoteType:     "esx5",
+	}
+	state.Put("driverConfig", driverConfig)
+	state.Put("display_name", "vm_name")
+	return state
+}
+
+func TestStepExport_ReturnIfSkip(t *testing.T) {
+	state := testState(t)
+	driverConfig := &DriverConfig{}
+	state.Put("driverConfig", driverConfig)
 	step := new(StepExport)
 
-	var config DriverConfig
-	config.RemoteType = "foo"
-	state.Put("driverConfig", &config)
+	step.SkipExport = true
 
 	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
 		t.Fatalf("bad action: %#v", action)
@@ -26,11 +45,189 @@ func testStepExport_wrongtype_impl(t *testing.T, remoteType string) {
 		t.Fatal("should NOT have error")
 	}
 
+	// We told step to skip so it should not have reached the driver's Export
+	// func.
+	d := state.Get("driver").(*DriverMock)
+	if d.ExportCalled {
+		t.Fatal("Should not have called the driver export func")
+	}
+
 	// Cleanup
 	step.Cleanup(state)
 }
 
-func TestStepExport_wrongtype_impl(t *testing.T) {
-	testStepExport_wrongtype_impl(t, "foo")
-	testStepExport_wrongtype_impl(t, "")
+func TestStepExport_localArgs(t *testing.T) {
+	// even though we aren't overriding the remote args and they are present,
+	// test shouldn't use them since remoteType is not set to esx.
+	state := testState(t)
+	driverConfig := &DriverConfig{}
+	state.Put("driverConfig", driverConfig)
+	step := new(StepExport)
+
+	step.SkipExport = false
+	step.OutputDir = stringPointer("test-output")
+	step.VMName = "test-name"
+	step.Format = "ova"
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	// Check that step ran, and called Export with the expected args.
+	d := state.Get("driver").(*DriverMock)
+	if !d.ExportCalled {
+		t.Fatal("Should have called the driver export func")
+	}
+
+	assert.Equal(t, d.ExportArgs,
+		[]string{
+			filepath.Join("test-output", "test-name.vmx"),
+			filepath.Join("test-output", "test-name.ova")})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepExport_localArgsExportOutputPath(t *testing.T) {
+	// even though we aren't overriding the remote args and they are present,
+	// test shouldn't use them since remoteType is not set to esx.
+	state := testState(t)
+	driverConfig := &DriverConfig{}
+	state.Put("driverConfig", driverConfig)
+	state.Put("export_output_path", "local_output")
+	step := new(StepExport)
+
+	step.SkipExport = false
+	step.OutputDir = stringPointer("test-output")
+	step.VMName = "test-name"
+	step.Format = "ova"
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	// Check that step ran, and called Export with the expected args.
+	d := state.Get("driver").(*DriverMock)
+	if !d.ExportCalled {
+		t.Fatal("Should have called the driver export func")
+	}
+
+	assert.Equal(t, d.ExportArgs,
+		[]string{
+			filepath.Join("local_output", "test-name.vmx"),
+			filepath.Join("local_output", "test-name.ova")})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepExport_localArgs_OvftoolOptions(t *testing.T) {
+	// even though we aren't overriding the remote args and they are present,
+	// test shouldn't use them since remoteType is not set to esx.
+	state := testState(t)
+	driverConfig := &DriverConfig{}
+	state.Put("driverConfig", driverConfig)
+	step := new(StepExport)
+
+	step.SkipExport = false
+	step.OutputDir = stringPointer("test-output")
+	step.VMName = "test-name"
+	step.Format = "ova"
+	step.OVFToolOptions = []string{"--option=value", "--second-option=\"quoted value\""}
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	// Check that step ran, and called Export with the expected args.
+	d := state.Get("driver").(*DriverMock)
+	if !d.ExportCalled {
+		t.Fatal("Should have called the driver export func")
+	}
+
+	assert.Equal(t, d.ExportArgs, []string{"--option=value",
+		"--second-option=\"quoted value\"",
+		filepath.Join("test-output", "test-name.vmx"),
+		filepath.Join("test-output", "test-name.ova")})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepExport_RemoteArgs(t *testing.T) {
+	// Even though we aren't overriding the remote args and they are present,
+	// test shouldn't use them since remoteType is not set to esx.
+	state := remoteExportTestState(t)
+	step := new(StepExport)
+
+	step.SkipExport = false
+	step.OutputDir = stringPointer("test-output")
+	step.VMName = "test-name"
+	step.Format = "ova"
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	// Check that step ran, and called Export with the expected args.
+	d := state.Get("driver").(*DriverMock)
+	if !d.ExportCalled {
+		t.Fatal("Should have called the driver export func")
+	}
+
+	assert.Equal(t, d.ExportArgs, []string{"--noSSLVerify=true",
+		"--skipManifestCheck",
+		"-tt=ova",
+		"vi://user:password@123.45.67.8/vm_name",
+		filepath.Join("test-output", "test-name.ova")})
+
+	// Cleanup
+	step.Cleanup(state)
+}
+
+func TestStepExport_RemoteArgsWithExportOutputPath(t *testing.T) {
+	// Even though we aren't overriding the remote args and they are present,
+	// test shouldn't use them since remoteType is not set to esx.
+	state := remoteExportTestState(t)
+	state.Put("export_output_path", "local_output")
+	step := new(StepExport)
+
+	step.SkipExport = false
+	step.OutputDir = stringPointer("test-output")
+	step.VMName = "test-name"
+	step.Format = "ova"
+
+	if action := step.Run(context.Background(), state); action != multistep.ActionContinue {
+		t.Fatalf("bad action: %#v", action)
+	}
+	if _, ok := state.GetOk("error"); ok {
+		t.Fatal("should NOT have error")
+	}
+
+	// Check that step ran, and called Export with the expected args.
+	d := state.Get("driver").(*DriverMock)
+	if !d.ExportCalled {
+		t.Fatal("Should have called the driver export func")
+	}
+
+	assert.Equal(t, d.ExportArgs, []string{"--noSSLVerify=true",
+		"--skipManifestCheck",
+		"-tt=ova",
+		"vi://user:password@123.45.67.8/vm_name",
+		filepath.Join("local_output", "test-name.ova")})
+
+	// Cleanup
+	step.Cleanup(state)
 }

builder/vmware/iso/builder.go

@@ -83,7 +83,15 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
 			DoCleanup: b.config.DriverConfig.CleanUpRemoteCache,
 			Checksum:  b.config.ISOChecksum,
 		},
-		&stepCreateDisk{},
+		&vmwcommon.StepCreateDisks{
+			OutputDir:          &b.config.OutputDir,
+			CreateMainDisk:     true,
+			DiskName:           b.config.DiskName,
+			MainDiskSize:       b.config.DiskSize,
+			AdditionalDiskSize: b.config.AdditionalDiskSize,
+			DiskAdapterType:    b.config.DiskAdapterType,
+			DiskTypeId:         b.config.DiskTypeId,
+		},
 		&stepCreateVMX{},
 		&vmwcommon.StepConfigureVMX{
 			CustomData:  b.config.VMXData,
@@ -163,6 +171,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
 			SkipExport:     b.config.SkipExport,
 			VMName:         b.config.VMName,
 			OVFToolOptions: b.config.OVFToolOptions,
+			OutputDir:      &b.config.OutputConfig.OutputDir,
 		},
 	}
 

builder/vmware/iso/builder_test.go

@@ -127,6 +127,7 @@ func TestBuilderPrepare_InvalidFloppies(t *testing.T) {
 	var b Builder
 	config := testConfig()
 	config["floppy_files"] = []string{"nonexistent.bat", "nonexistent.ps1"}
+
 	b = Builder{}
 	_, _, errs := b.Prepare(config)
 	if errs == nil {

builder/vmware/iso/config.go

@@ -33,55 +33,12 @@ type Config struct {
 	vmwcommon.ToolsConfig          `mapstructure:",squash"`
 	vmwcommon.VMXConfig            `mapstructure:",squash"`
 	vmwcommon.ExportConfig         `mapstructure:",squash"`
-	// The size(s) of any additional
-	// hard disks for the VM in megabytes. If this is not specified then the VM
-	// will only contain a primary hard disk. The builder uses expandable, not
-	// fixed-size virtual hard disks, so the actual file representing the disk will
-	// not use the full size unless it is full.
-	AdditionalDiskSize []uint `mapstructure:"disk_additional_size" required:"false"`
-	// The adapter type of the VMware virtual disk to create. This option is
-	// for advanced usage, modify only if you know what you're doing. Some of
-	// the options you can specify are `ide`, `sata`, `nvme` or `scsi` (which
-	// uses the "lsilogic" scsi interface by default). If you specify another
-	// option, Packer will assume that you're specifying a `scsi` interface of
-	// that specified type. For more information, please consult [Virtual Disk
-	// Manager User's Guide](http://www.vmware.com/pdf/VirtualDiskManager.pdf)
-	// for desktop VMware clients. For ESXi, refer to the proper ESXi
-	// documentation.
-	DiskAdapterType string `mapstructure:"disk_adapter_type" required:"false"`
-	// The filename of the virtual disk that'll be created,
-	// without the extension. This defaults to packer.
-	DiskName string `mapstructure:"vmdk_name" required:"false"`
+	vmwcommon.DiskConfig           `mapstructure:",squash"`
 	// The size of the hard disk for the VM in megabytes.
 	// The builder uses expandable, not fixed-size virtual hard disks, so the
 	// actual file representing the disk will not use the full size unless it
 	// is full. By default this is set to 40000 (about 40 GB).
 	DiskSize uint `mapstructure:"disk_size" required:"false"`
-	// The type of VMware virtual disk to create. This
-	// option is for advanced usage.
-	//
-	//   For desktop VMware clients:
-	//
-	//   Type ID | Description
-	//   ------- | ---
-	//   `0`     | Growable virtual disk contained in a single file (monolithic sparse).
-	//   `1`     | Growable virtual disk split into 2GB files (split sparse).
-	//   `2`     | Preallocated virtual disk contained in a single file (monolithic flat).
-	//   `3`     | Preallocated virtual disk split into 2GB files (split flat).
-	//   `4`     | Preallocated virtual disk compatible with ESX server (VMFS flat).
-	//   `5`     | Compressed disk optimized for streaming.
-	//
-	//   The default is `1`.
-	//
-	//   For ESXi, this defaults to `zeroedthick`. The available options for ESXi
-	//   are: `zeroedthick`, `eagerzeroedthick`, `thin`. `rdm:dev`, `rdmp:dev`,
-	//   `2gbsparse` are not supported. Due to default disk compaction, when using
-	//   `zeroedthick` or `eagerzeroedthick` set `skip_compaction` to `true`.
-	//
-	//   For more information, please consult the [Virtual Disk Manager User's
-	//   Guide](https://www.vmware.com/pdf/VirtualDiskManager.pdf) for desktop
-	//   VMware clients. For ESXi, refer to the proper ESXi documentation.
-	DiskTypeId string `mapstructure:"disk_type_id" required:"false"`
 	// The adapter type (or bus) that will be used
 	// by the cdrom device. This is chosen by default based on the disk adapter
 	// type. VMware tends to lean towards ide for the cdrom device unless
@@ -155,34 +112,19 @@ func (c *Config) Prepare(raws ...interface{}) ([]string, error) {
 	errs = packer.MultiErrorAppend(errs, c.FloppyConfig.Prepare(&c.ctx)...)
 	errs = packer.MultiErrorAppend(errs, c.VNCConfig.Prepare(&c.ctx)...)
 	errs = packer.MultiErrorAppend(errs, c.ExportConfig.Prepare(&c.ctx)...)
-
-	if c.DiskName == "" {
-		c.DiskName = "disk"
-	}
+	errs = packer.MultiErrorAppend(errs, c.DiskConfig.Prepare(&c.ctx)...)
 
 	if c.DiskSize == 0 {
 		c.DiskSize = 40000
 	}
 
-	if c.DiskAdapterType == "" {
-		// Default is lsilogic
-		c.DiskAdapterType = "lsilogic"
-	}
-
-	if !c.SkipCompaction {
-		if c.RemoteType == "esx5" {
-			if c.DiskTypeId == "" {
-				c.SkipCompaction = true
-			}
-		}
-	}
-
 	if c.DiskTypeId == "" {
 		// Default is growable virtual disk split in 2GB files.
 		c.DiskTypeId = "1"
 
 		if c.RemoteType == "esx5" {
 			c.DiskTypeId = "zeroedthick"
+			c.SkipCompaction = true
 		}
 	}
 
@@ -234,18 +176,19 @@ func (c *Config) Prepare(raws ...interface{}) ([]string, error) {
 		}
 	}
 
-	if c.Format != "" {
+	if c.Format == "" {
 		if c.RemoteType != "esx5" {
-			errs = packer.MultiErrorAppend(errs,
-				fmt.Errorf("format is only valid when RemoteType=esx5"))
+			c.Format = "vmx"
+		} else {
+			c.Format = "ovf"
 		}
-	} else {
-		c.Format = "ovf"
 	}
 
-	if !(c.Format == "ova" || c.Format == "ovf" || c.Format == "vmx") {
-		errs = packer.MultiErrorAppend(errs,
-			fmt.Errorf("format must be one of ova, ovf, or vmx"))
+	if c.RemoteType != "esx5" && c.Format == "vmx" {
+		// if we're building locally and want a vmx, there's nothing to export.
+		// Set skip export flag here to keep the export step from attempting
+		// an unneded export
+		c.SkipExport = true
 	}
 
 	err = c.DriverConfig.Validate(c.SkipExport)

builder/vmware/iso/config.hcl2spec.go

@@ -126,8 +126,8 @@ type FlatConfig struct {
 	AdditionalDiskSize        []uint            `mapstructure:"disk_additional_size" required:"false" cty:"disk_additional_size" hcl:"disk_additional_size"`
 	DiskAdapterType           *string           `mapstructure:"disk_adapter_type" required:"false" cty:"disk_adapter_type" hcl:"disk_adapter_type"`
 	DiskName                  *string           `mapstructure:"vmdk_name" required:"false" cty:"vmdk_name" hcl:"vmdk_name"`
-	DiskSize                  *uint             `mapstructure:"disk_size" required:"false" cty:"disk_size" hcl:"disk_size"`
 	DiskTypeId                *string           `mapstructure:"disk_type_id" required:"false" cty:"disk_type_id" hcl:"disk_type_id"`
+	DiskSize                  *uint             `mapstructure:"disk_size" required:"false" cty:"disk_size" hcl:"disk_size"`
 	CdromAdapterType          *string           `mapstructure:"cdrom_adapter_type" required:"false" cty:"cdrom_adapter_type" hcl:"cdrom_adapter_type"`
 	GuestOSType               *string           `mapstructure:"guest_os_type" required:"false" cty:"guest_os_type" hcl:"guest_os_type"`
 	Version                   *string           `mapstructure:"version" required:"false" cty:"version" hcl:"version"`
@@ -265,8 +265,8 @@ func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec {
 		"disk_additional_size":           &hcldec.AttrSpec{Name: "disk_additional_size", Type: cty.List(cty.Number), Required: false},
 		"disk_adapter_type":              &hcldec.AttrSpec{Name: "disk_adapter_type", Type: cty.String, Required: false},
 		"vmdk_name":                      &hcldec.AttrSpec{Name: "vmdk_name", Type: cty.String, Required: false},
-		"disk_size":                      &hcldec.AttrSpec{Name: "disk_size", Type: cty.Number, Required: false},
 		"disk_type_id":                   &hcldec.AttrSpec{Name: "disk_type_id", Type: cty.String, Required: false},
+		"disk_size":                      &hcldec.AttrSpec{Name: "disk_size", Type: cty.Number, Required: false},
 		"cdrom_adapter_type":             &hcldec.AttrSpec{Name: "cdrom_adapter_type", Type: cty.String, Required: false},
 		"guest_os_type":                  &hcldec.AttrSpec{Name: "guest_os_type", Type: cty.String, Required: false},
 		"version":                        &hcldec.AttrSpec{Name: "version", Type: cty.String, Required: false},

builder/vmware/vmx/builder.go

@@ -74,6 +74,15 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
 			DoCleanup: true,
 			Checksum:  "none",
 		},
+		&vmwcommon.StepCreateDisks{
+			OutputDir:          &b.config.OutputDir,
+			CreateMainDisk:     false,
+			DiskName:           b.config.DiskName,
+			MainDiskSize:       0,
+			AdditionalDiskSize: b.config.AdditionalDiskSize,
+			DiskAdapterType:    b.config.DiskAdapterType,
+			DiskTypeId:         b.config.DiskTypeId,
+		},
 		&StepCloneVMX{
 			Path:      b.config.SourcePath,
 			OutputDir: &b.config.OutputDir,
@@ -161,6 +170,7 @@ func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (pack
 			SkipExport:     b.config.SkipExport,
 			VMName:         b.config.VMName,
 			OVFToolOptions: b.config.OVFToolOptions,
+			OutputDir:      &b.config.OutputConfig.OutputDir,
 		},
 	}
 

builder/vmware/vmx/builder_test.go

@@ -46,7 +46,6 @@ func TestBuilderPrepare_FloppyFiles(t *testing.T) {
 	if err != nil {
 		t.Fatalf("should not have error: %s", err)
 	}
-
 	expected := []string{fmt.Sprintf("%s/bar.bat", floppies_path), fmt.Sprintf("%s/foo.ps1", floppies_path)}
 	if !reflect.DeepEqual(b.config.FloppyFiles, expected) {
 		t.Fatalf("bad: %#v", b.config.FloppyFiles)

builder/vmware/vmx/config.go

@@ -30,6 +30,7 @@ type Config struct {
 	vmwcommon.ToolsConfig          `mapstructure:",squash"`
 	vmwcommon.VMXConfig            `mapstructure:",squash"`
 	vmwcommon.ExportConfig         `mapstructure:",squash"`
+	vmwcommon.DiskConfig           `mapstructure:",squash"`
 	// By default Packer creates a 'full' clone of the virtual machine
 	// specified in source_path. The resultant virtual machine is fully
 	// independant from the parent it was cloned from.
@@ -89,6 +90,7 @@ func (c *Config) Prepare(raws ...interface{}) ([]string, error) {
 	errs = packer.MultiErrorAppend(errs, c.FloppyConfig.Prepare(&c.ctx)...)
 	errs = packer.MultiErrorAppend(errs, c.VNCConfig.Prepare(&c.ctx)...)
 	errs = packer.MultiErrorAppend(errs, c.ExportConfig.Prepare(&c.ctx)...)
+	errs = packer.MultiErrorAppend(errs, c.DiskConfig.Prepare(&c.ctx)...)
 
 	if c.RemoteType == "" {
 		if c.SourcePath == "" {
@@ -112,23 +114,41 @@ func (c *Config) Prepare(raws ...interface{}) ([]string, error) {
 		}
 	}
 
+	if c.DiskTypeId == "" {
+		// Default is growable virtual disk split in 2GB files.
+		c.DiskTypeId = "1"
+
+		if c.RemoteType == "esx5" {
+			c.DiskTypeId = "zeroedthick"
+		}
+	}
+
+	if c.Format == "" {
+		if c.RemoteType != "esx5" {
+			c.Format = "vmx"
+		} else {
+			c.Format = "ovf"
+		}
+	}
+
+	if c.RemoteType != "esx5" && c.Format == "vmx" {
+		// if we're building locally and want a vmx, there's nothing to export.
+		// Set skip export flag here to keep the export step from attempting
+		// an unneded export
+		c.SkipExport = true
+	}
+
 	err = c.DriverConfig.Validate(c.SkipExport)
 	if err != nil {
 		errs = packer.MultiErrorAppend(errs, err)
 	}
 
-	if c.Format != "" {
+	if c.Format == "" {
 		if c.RemoteType != "esx5" {
-			errs = packer.MultiErrorAppend(errs,
-				fmt.Errorf("format is only valid when remote_type=esx5"))
+			c.Format = "vmx"
+		} else {
+			c.Format = "ovf"
 		}
-	} else {
-		c.Format = "ovf"
-	}
-
-	if !(c.Format == "ova" || c.Format == "ovf" || c.Format == "vmx") {
-		errs = packer.MultiErrorAppend(errs,
-			fmt.Errorf("format must be one of ova, ovf, or vmx"))
 	}
 
 	// Warnings

builder/vmware/vmx/config.hcl2spec.go

@@ -108,6 +108,10 @@ type FlatConfig struct {
 	SkipExport                *bool             `mapstructure:"skip_export" required:"false" cty:"skip_export" hcl:"skip_export"`
 	KeepRegistered            *bool             `mapstructure:"keep_registered" required:"false" cty:"keep_registered" hcl:"keep_registered"`
 	SkipCompaction            *bool             `mapstructure:"skip_compaction" required:"false" cty:"skip_compaction" hcl:"skip_compaction"`
+	AdditionalDiskSize        []uint            `mapstructure:"disk_additional_size" required:"false" cty:"disk_additional_size" hcl:"disk_additional_size"`
+	DiskAdapterType           *string           `mapstructure:"disk_adapter_type" required:"false" cty:"disk_adapter_type" hcl:"disk_adapter_type"`
+	DiskName                  *string           `mapstructure:"vmdk_name" required:"false" cty:"vmdk_name" hcl:"vmdk_name"`
+	DiskTypeId                *string           `mapstructure:"disk_type_id" required:"false" cty:"disk_type_id" hcl:"disk_type_id"`
 	Linked                    *bool             `mapstructure:"linked" required:"false" cty:"linked" hcl:"linked"`
 	SourcePath                *string           `mapstructure:"source_path" required:"true" cty:"source_path" hcl:"source_path"`
 	VMName                    *string           `mapstructure:"vm_name" required:"false" cty:"vm_name" hcl:"vm_name"`
@@ -224,6 +228,10 @@ func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec {
 		"skip_export":                    &hcldec.AttrSpec{Name: "skip_export", Type: cty.Bool, Required: false},
 		"keep_registered":                &hcldec.AttrSpec{Name: "keep_registered", Type: cty.Bool, Required: false},
 		"skip_compaction":                &hcldec.AttrSpec{Name: "skip_compaction", Type: cty.Bool, Required: false},
+		"disk_additional_size":           &hcldec.AttrSpec{Name: "disk_additional_size", Type: cty.List(cty.Number), Required: false},
+		"disk_adapter_type":              &hcldec.AttrSpec{Name: "disk_adapter_type", Type: cty.String, Required: false},
+		"vmdk_name":                      &hcldec.AttrSpec{Name: "vmdk_name", Type: cty.String, Required: false},
+		"disk_type_id":                   &hcldec.AttrSpec{Name: "disk_type_id", Type: cty.String, Required: false},
 		"linked":                         &hcldec.AttrSpec{Name: "linked", Type: cty.Bool, Required: false},
 		"source_path":                    &hcldec.AttrSpec{Name: "source_path", Type: cty.String, Required: false},
 		"vm_name":                        &hcldec.AttrSpec{Name: "vm_name", Type: cty.String, Required: false},

command/cli.go

@@ -130,3 +130,15 @@ func (va *InspectArgs) AddFlagSets(flags *flag.FlagSet) {
 type InspectArgs struct {
 	MetaArgs
 }
+
+func (va *HCL2UpgradeArgs) AddFlagSets(flags *flag.FlagSet) {
+	flags.StringVar(&va.OutputFile, "output-file", "", "File where to put the hcl2 generated config. Defaults to JSON_TEMPLATE.pkr.hcl")
+
+	va.MetaArgs.AddFlagSets(flags)
+}
+
+// HCL2UpgradeArgs represents a parsed cli line for a `packer build`
+type HCL2UpgradeArgs struct {
+	MetaArgs
+	OutputFile string
+}

command/exec_test.go

@@ -8,6 +8,7 @@ import (
 	"runtime"
 	"testing"
 
+	"github.com/hashicorp/packer/builder/amazon/ebs"
 	"github.com/hashicorp/packer/builder/file"
 	"github.com/hashicorp/packer/builder/null"
 	"github.com/hashicorp/packer/packer"
@@ -89,6 +90,8 @@ func TestHelperProcess(*testing.T) {
 		os.Exit((&InspectCommand{Meta: commandMeta()}).Run(args))
 	case "build":
 		os.Exit((&BuildCommand{Meta: commandMeta()}).Run(args))
+	case "hcl2_upgrade":
+		os.Exit((&HCL2UpgradeCommand{Meta: commandMeta()}).Run(args))
 	default:
 		fmt.Fprintf(os.Stderr, "Unknown command %q\n", cmd)
 		os.Exit(2)
@@ -115,8 +118,9 @@ func commandMeta() Meta {
 func getBareComponentFinder() packer.ComponentFinder {
 	return packer.ComponentFinder{
 		BuilderStore: packer.MapOfBuilder{
-			"file": func() (packer.Builder, error) { return &file.Builder{}, nil },
-			"null": func() (packer.Builder, error) { return &null.Builder{}, nil },
+			"file":       func() (packer.Builder, error) { return &file.Builder{}, nil },
+			"null":       func() (packer.Builder, error) { return &null.Builder{}, nil },
+			"amazon-ebs": func() (packer.Builder, error) { return &ebs.Builder{}, nil },
 		},
 		ProvisionerStore: packer.MapOfProvisioner{
 			"shell-local": func() (packer.Provisioner, error) { return &shell_local.Provisioner{}, nil },

command/hcl2_upgrade.go

@@ -0,0 +1,432 @@
+package command
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	texttemplate "text/template"
+
+	"github.com/hashicorp/hcl/v2/hclwrite"
+	hcl2shim "github.com/hashicorp/packer/hcl2template/shim"
+	"github.com/hashicorp/packer/template"
+	"github.com/posener/complete"
+	"github.com/zclconf/go-cty/cty"
+)
+
+type HCL2UpgradeCommand struct {
+	Meta
+}
+
+func (c *HCL2UpgradeCommand) Run(args []string) int {
+	ctx, cleanup := handleTermInterrupt(c.Ui)
+	defer cleanup()
+
+	cfg, ret := c.ParseArgs(args)
+	if ret != 0 {
+		return ret
+	}
+
+	return c.RunContext(ctx, cfg)
+}
+
+func (c *HCL2UpgradeCommand) ParseArgs(args []string) (*HCL2UpgradeArgs, int) {
+	var cfg HCL2UpgradeArgs
+	flags := c.Meta.FlagSet("hcl2_upgrade", FlagSetNone)
+	flags.Usage = func() { c.Ui.Say(c.Help()) }
+	cfg.AddFlagSets(flags)
+	if err := flags.Parse(args); err != nil {
+		return &cfg, 1
+	}
+	args = flags.Args()
+	if len(args) != 1 {
+		flags.Usage()
+		return &cfg, 1
+	}
+	cfg.Path = args[0]
+	if cfg.OutputFile == "" {
+		cfg.OutputFile = cfg.Path + ".pkr.hcl"
+	}
+	return &cfg, 0
+}
+
+const (
+	hcl2UpgradeFileHeader = `# This file was autogenerate by the BETA 'packer hcl2_upgrade' command. We
+# recommend double checking that everything is correct before going forward. We
+# also recommend treating this file as disposable. The HCL2 blocks in this
+# file can be moved to other files. For example, the variable blocks could be
+# moved to their own 'variables.pkr.hcl' file, etc. Those files need to be
+# suffixed with '.pkr.hcl' to be visible to Packer. To use multiple files at
+# once they also need to be in the same folder. 'packer inspect folder/'
+# will describe to you what is in that folder.
+
+# All generated input variables will be of string type as this how Packer JSON
+# views them; you can later on change their type. Read the variables type
+# constraints documentation
+# https://www.packer.io/docs/from-1.5/variables#type-constraints for more info.
+`
+
+	sourcesHeader = `
+# source blocks are generated from your builders; a source can be referenced in
+# build blocks. A build block runs provisioner and post-processors onto a
+# source. Read the documentation for source blocks here:
+# https://www.packer.io/docs/from-1.5/blocks/source`
+
+	buildHeader = `
+# a build block invokes sources and runs provisionning steps on them. The
+# documentation for build blocks can be found here:
+# https://www.packer.io/docs/from-1.5/blocks/build
+build {
+`
+)
+
+func (c *HCL2UpgradeCommand) RunContext(buildCtx context.Context, cla *HCL2UpgradeArgs) int {
+
+	out := &bytes.Buffer{}
+	var output io.Writer
+	if err := os.MkdirAll(filepath.Dir(cla.OutputFile), 0); err != nil {
+		c.Ui.Error(fmt.Sprintf("Failed to create output directory: %v", err))
+		return 1
+	}
+	if f, err := os.Create(cla.OutputFile); err == nil {
+		output = f
+		defer f.Close()
+	} else {
+		c.Ui.Error(fmt.Sprintf("Failed to create output file: %v", err))
+		return 1
+	}
+
+	if _, err := output.Write([]byte(hcl2UpgradeFileHeader)); err != nil {
+		c.Ui.Error(fmt.Sprintf("Failed to write to file: %v", err))
+		return 1
+	}
+
+	hdl, ret := c.GetConfigFromJSON(&cla.MetaArgs)
+	if ret != 0 {
+		return ret
+	}
+
+	core := hdl.(*CoreWrapper).Core
+	if err := core.Initialize(); err != nil {
+		c.Ui.Error(fmt.Sprintf("Initialization error, continuing: %v", err))
+	}
+	tpl := core.Template
+
+	// Output variables section
+
+	variables := []*template.Variable{}
+	{
+		// sort variables to avoid map's randomness
+
+		for _, variable := range tpl.Variables {
+			variables = append(variables, variable)
+		}
+		sort.Slice(variables, func(i, j int) bool {
+			return variables[i].Key < variables[j].Key
+		})
+	}
+
+	for _, variable := range variables {
+		variablesContent := hclwrite.NewEmptyFile()
+		variablesBody := variablesContent.Body()
+
+		variableBody := variablesBody.AppendNewBlock("variable", []string{variable.Key}).Body()
+		variableBody.SetAttributeRaw("type", hclwrite.Tokens{&hclwrite.Token{Bytes: []byte("string")}})
+
+		if variable.Default != "" || !variable.Required {
+			variableBody.SetAttributeValue("default", hcl2shim.HCL2ValueFromConfigValue(variable.Default))
+		}
+		if isSensitiveVariable(variable.Key, tpl.SensitiveVariables) {
+			variableBody.SetAttributeValue("sensitive", cty.BoolVal(true))
+		}
+		variablesBody.AppendNewline()
+		out.Write(transposeTemplatingCalls(variablesContent.Bytes()))
+	}
+
+	fmt.Fprintln(out, `# "timestamp" template function replacement`)
+	fmt.Fprintln(out, `locals { timestamp = regex_replace(timestamp(), "[- TZ:]", "") }`)
+
+	// Output sources section
+
+	builders := []*template.Builder{}
+	{
+		// sort builders to avoid map's randomnes
+		for _, builder := range tpl.Builders {
+			builders = append(builders, builder)
+		}
+		sort.Slice(builders, func(i, j int) bool {
+			return builders[i].Type+builders[i].Name < builders[j].Type+builders[j].Name
+		})
+	}
+
+	out.Write([]byte(sourcesHeader))
+
+	for i, builderCfg := range builders {
+		sourcesContent := hclwrite.NewEmptyFile()
+		body := sourcesContent.Body()
+
+		body.AppendNewline()
+		if !c.Meta.CoreConfig.Components.BuilderStore.Has(builderCfg.Type) {
+			c.Ui.Error(fmt.Sprintf("unknown builder type: %q\n", builderCfg.Type))
+			return 1
+		}
+		if builderCfg.Name == "" || builderCfg.Name == builderCfg.Type {
+			builderCfg.Name = fmt.Sprintf("autogenerated_%d", i+1)
+		}
+		sourceBody := body.AppendNewBlock("source", []string{builderCfg.Type, builderCfg.Name}).Body()
+
+		jsonBodyToHCL2Body(sourceBody, builderCfg.Config)
+
+		_, _ = out.Write(transposeTemplatingCalls(sourcesContent.Bytes()))
+	}
+
+	// Output build section
+	out.Write([]byte(buildHeader))
+
+	buildContent := hclwrite.NewEmptyFile()
+	buildBody := buildContent.Body()
+	if tpl.Description != "" {
+		buildBody.SetAttributeValue("description", cty.StringVal(tpl.Description))
+		buildBody.AppendNewline()
+	}
+
+	sourceNames := []string{}
+	for _, builder := range builders {
+		sourceNames = append(sourceNames, fmt.Sprintf("source.%s.%s", builder.Type, builder.Name))
+	}
+	buildBody.SetAttributeValue("sources", hcl2shim.HCL2ValueFromConfigValue(sourceNames))
+	buildBody.AppendNewline()
+	_, _ = buildContent.WriteTo(out)
+
+	for _, provisioner := range tpl.Provisioners {
+		provisionerContent := hclwrite.NewEmptyFile()
+		body := provisionerContent.Body()
+
+		buildBody.AppendNewline()
+		block := body.AppendNewBlock("provisioner", []string{provisioner.Type})
+		cfg := provisioner.Config
+		if len(provisioner.Except) > 0 {
+			cfg["except"] = provisioner.Except
+		}
+		if len(provisioner.Only) > 0 {
+			cfg["only"] = provisioner.Only
+		}
+		if provisioner.MaxRetries != "" {
+			cfg["max_retries"] = provisioner.MaxRetries
+		}
+		if provisioner.Timeout > 0 {
+			cfg["timeout"] = provisioner.Timeout.String()
+		}
+		jsonBodyToHCL2Body(block.Body(), cfg)
+
+		out.Write(transposeTemplatingCalls(provisionerContent.Bytes()))
+	}
+	for _, pps := range tpl.PostProcessors {
+		postProcessorContent := hclwrite.NewEmptyFile()
+		body := postProcessorContent.Body()
+
+		switch len(pps) {
+		case 0:
+			continue
+		case 1:
+		default:
+			body = body.AppendNewBlock("post-processors", nil).Body()
+		}
+		for _, pp := range pps {
+			ppBody := body.AppendNewBlock("post-processor", []string{pp.Type}).Body()
+			if pp.KeepInputArtifact != nil {
+				ppBody.SetAttributeValue("keep_input_artifact", cty.BoolVal(*pp.KeepInputArtifact))
+			}
+			cfg := pp.Config
+			if len(pp.Except) > 0 {
+				cfg["except"] = pp.Except
+			}
+			if len(pp.Only) > 0 {
+				cfg["only"] = pp.Only
+			}
+			if pp.Name != "" && pp.Name != pp.Type {
+				cfg["name"] = pp.Name
+			}
+			jsonBodyToHCL2Body(ppBody, cfg)
+		}
+
+		_, _ = out.Write(transposeTemplatingCalls(postProcessorContent.Bytes()))
+	}
+
+	_, _ = out.Write([]byte("}\n"))
+
+	_, _ = output.Write(hclwrite.Format(out.Bytes()))
+
+	c.Ui.Say(fmt.Sprintf("Successfully created %s ", cla.OutputFile))
+
+	return 0
+}
+
+// transposeTemplatingCalls executes parts of blocks as go template files and replaces
+// their result with their hcl2 variant. If something goes wrong the template
+// containing the go template string is returned.
+func transposeTemplatingCalls(s []byte) []byte {
+	fallbackReturn := func(err error) []byte {
+		return append([]byte(fmt.Sprintf("\n#could not parse template for following block: %q\n", err)), s...)
+	}
+	funcMap := texttemplate.FuncMap{
+		"timestamp": func() string {
+			return "${local.timestamp}"
+		},
+		"isotime": func() string {
+			return "${local.timestamp}"
+		},
+		"user": func(in string) string {
+			return fmt.Sprintf("${var.%s}", in)
+		},
+		"env": func(in string) string {
+			return fmt.Sprintf("${var.%s}", in)
+		},
+		"build": func(a string) string {
+			return fmt.Sprintf("${build.%s}", a)
+		},
+	}
+
+	tpl, err := texttemplate.New("generated").
+		Funcs(funcMap).
+		Parse(string(s))
+
+	if err != nil {
+		return fallbackReturn(err)
+	}
+
+	str := &bytes.Buffer{}
+	v := struct {
+		HTTPIP   string
+		HTTPPort string
+	}{
+		HTTPIP:   "{{ .HTTPIP }}",
+		HTTPPort: "{{ .HTTPPort }}",
+	}
+	if err := tpl.Execute(str, v); err != nil {
+		return fallbackReturn(err)
+	}
+
+	return str.Bytes()
+}
+
+func jsonBodyToHCL2Body(out *hclwrite.Body, kvs map[string]interface{}) {
+	ks := []string{}
+	for k := range kvs {
+		ks = append(ks, k)
+	}
+	sort.Strings(ks)
+
+	for _, k := range ks {
+		value := kvs[k]
+
+		switch value := value.(type) {
+		case map[string]interface{}:
+			var mostComplexElem interface{}
+			for _, randomElem := range value {
+				// HACK: we take the most complex element of that map because
+				// in HCL2, map of objects can be bodies, for example:
+				// map containing object: source_ami_filter {} ( body )
+				// simple string/string map: tags = {} ) ( attribute )
+				//
+				// if we could not find an object in this map then it's most
+				// likely a plain map and so we guess it should be and
+				// attribute. Though now if value refers to something that is
+				// an object but only contains a string or a bool; we could
+				// generate a faulty object. For example a (somewhat invalid)
+				// source_ami_filter where only `most_recent` is set.
+				switch randomElem.(type) {
+				case string, int, float64, bool:
+					if mostComplexElem != nil {
+						continue
+					}
+					mostComplexElem = randomElem
+				default:
+					mostComplexElem = randomElem
+				}
+			}
+
+			switch mostComplexElem.(type) {
+			case string, int, float64, bool:
+				out.SetAttributeValue(k, hcl2shim.HCL2ValueFromConfigValue(value))
+			default:
+				nestedBlockBody := out.AppendNewBlock(k, nil).Body()
+				jsonBodyToHCL2Body(nestedBlockBody, value)
+			}
+		case map[string]string, map[string]int, map[string]float64:
+			out.SetAttributeValue(k, hcl2shim.HCL2ValueFromConfigValue(value))
+		case []interface{}:
+			if len(value) == 0 {
+				continue
+			}
+
+			var mostComplexElem interface{}
+			for _, randomElem := range value {
+				// HACK: we take the most complex element of that slice because
+				// in hcl2 slices of plain types can be arrays, for example:
+				// simple string type: owners = ["0000000000"]
+				// object: launch_block_device_mappings {}
+				switch randomElem.(type) {
+				case string, int, float64, bool:
+					if mostComplexElem != nil {
+						continue
+					}
+					mostComplexElem = randomElem
+				default:
+					mostComplexElem = randomElem
+				}
+			}
+			switch mostComplexElem.(type) {
+			case map[string]interface{}:
+				// this is an object in a slice; so we unwrap it. We
+				// could try to remove any 's' suffix in the key, but
+				// this might not work everywhere.
+				for i := range value {
+					value := value[i].(map[string]interface{})
+					nestedBlockBody := out.AppendNewBlock(k, nil).Body()
+					jsonBodyToHCL2Body(nestedBlockBody, value)
+				}
+				continue
+			default:
+				out.SetAttributeValue(k, hcl2shim.HCL2ValueFromConfigValue(value))
+			}
+		default:
+			out.SetAttributeValue(k, hcl2shim.HCL2ValueFromConfigValue(value))
+		}
+	}
+}
+
+func isSensitiveVariable(key string, vars []*template.Variable) bool {
+	for _, v := range vars {
+		if v.Key == key {
+			return true
+		}
+	}
+	return false
+}
+
+func (*HCL2UpgradeCommand) Help() string {
+	helpText := `
+Usage: packer hcl2_upgrade -output-file=JSON_TEMPLATE.pkr.hcl JSON_TEMPLATE...
+
+  Will transform your JSON template to a HCL2 configuration.
+`
+
+	return strings.TrimSpace(helpText)
+}
+
+func (*HCL2UpgradeCommand) Synopsis() string {
+	return "transform a JSON template into a HCL2 configuration"
+}
+
+func (*HCL2UpgradeCommand) AutocompleteArgs() complete.Predictor {
+	return complete.PredictNothing
+}
+
+func (*HCL2UpgradeCommand) AutocompleteFlags() complete.Flags {
+	return complete.Flags{}
+}

command/hcl2_upgrade_test.go

@@ -0,0 +1,51 @@
+package command
+
+import (
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/google/go-cmp/cmp"
+)
+
+func Test_hcl2_upgrade(t *testing.T) {
+	cwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("Getwd: %v", err)
+	}
+	_ = cwd
+
+	tc := []struct {
+		folder string
+	}{
+		{"hcl2_upgrade_basic"},
+	}
+
+	for _, tc := range tc {
+		t.Run(tc.folder, func(t *testing.T) {
+			inputPath := filepath.Join(testFixture(tc.folder, "input.json"))
+			outputPath := inputPath + ".pkr.hcl"
+			expectedPath := filepath.Join(testFixture(tc.folder, "expected.pkr.hcl"))
+			p := helperCommand(t, "hcl2_upgrade", inputPath)
+			bs, err := p.CombinedOutput()
+			if err != nil {
+				t.Fatalf("%v %s", err, bs)
+			}
+			expected := mustBytes(ioutil.ReadFile(expectedPath))
+			actual := mustBytes(ioutil.ReadFile(outputPath))
+
+			if diff := cmp.Diff(expected, actual); diff != "" {
+				t.Fatalf("unexpected output: %s", diff)
+			}
+			os.Remove(outputPath)
+		})
+	}
+}
+
+func mustBytes(b []byte, e error) []byte {
+	if e != nil {
+		panic(e)
+	}
+	return b
+}

command/meta.go

@@ -66,7 +66,7 @@ func (m *Meta) Core(tpl *template.Template, cla *MetaArgs) (*packer.Core, error)
 // command implements. The exact behavior of FlagSet can be configured
 // using the flags as the second parameter, for example to disable
 // build settings on the commands that don't handle builds.
-func (m *Meta) FlagSet(n string, fs FlagSetFlags) *flag.FlagSet {
+func (m *Meta) FlagSet(n string, _ FlagSetFlags) *flag.FlagSet {
 	f := flag.NewFlagSet(n, flag.ContinueOnError)
 
 	// Create an io.Writer that writes to our Ui properly for errors.

command/test-fixtures/hcl2_upgrade_basic/expected.pkr.hcl

@@ -0,0 +1,115 @@
+# This file was autogenerate by the BETA 'packer hcl2_upgrade' command. We
+# recommend double checking that everything is correct before going forward. We
+# also recommend treating this file as disposable. The HCL2 blocks in this
+# file can be moved to other files. For example, the variable blocks could be
+# moved to their own 'variables.pkr.hcl' file, etc. Those files need to be
+# suffixed with '.pkr.hcl' to be visible to Packer. To use multiple files at
+# once they also need to be in the same folder. 'packer inspect folder/'
+# will describe to you what is in that folder.
+
+# All generated input variables will be of string type as this how Packer JSON
+# views them; you can later on change their type. Read the variables type
+# constraints documentation
+# https://www.packer.io/docs/from-1.5/variables#type-constraints for more info.
+variable "aws_access_key" {
+  type      = string
+  default   = ""
+  sensitive = true
+}
+
+variable "aws_region" {
+  type = string
+}
+
+variable "aws_secret_key" {
+  type      = string
+  default   = ""
+  sensitive = true
+}
+
+# "timestamp" template function replacement
+locals { timestamp = regex_replace(timestamp(), "[- TZ:]", "") }
+
+# source blocks are generated from your builders; a source can be referenced in
+# build blocks. A build block runs provisioner and post-processors onto a
+# source. Read the documentation for source blocks here:
+# https://www.packer.io/docs/from-1.5/blocks/source
+source "amazon-ebs" "autogenerated_1" {
+  access_key      = "${var.aws_access_key}"
+  ami_description = "Ubuntu 16.04 LTS - expand root partition"
+  ami_name        = "ubuntu-16-04-test-${local.timestamp}"
+  encrypt_boot    = true
+  launch_block_device_mappings {
+    delete_on_termination = true
+    device_name           = "/dev/sda1"
+    volume_size           = 48
+    volume_type           = "gp2"
+  }
+  region     = "${var.aws_region}"
+  secret_key = "${var.aws_secret_key}"
+  source_ami_filter {
+    filters = {
+      name                = "ubuntu/images/*/ubuntu-xenial-16.04-amd64-server-*"
+      root-device-type    = "ebs"
+      virtualization-type = "hvm"
+    }
+    most_recent = true
+    owners      = ["099720109477"]
+  }
+  spot_instance_types = ["t2.small", "t2.medium", "t2.large"]
+  spot_price          = "0.0075"
+  ssh_interface       = "session_manager"
+  ssh_username        = "ubuntu"
+  temporary_iam_instance_profile_policy_document {
+    Statement {
+      Action   = ["*"]
+      Effect   = "Allow"
+      Resource = ["*"]
+    }
+    Version = "2012-10-17"
+  }
+}
+
+# a build block invokes sources and runs provisionning steps on them. The
+# documentation for build blocks can be found here:
+# https://www.packer.io/docs/from-1.5/blocks/build
+build {
+  sources = ["source.amazon-ebs.autogenerated_1"]
+
+  provisioner "shell" {
+    inline      = ["echo ${var.secret_account}", "echo ${build.ID}", "echo ${build.SSHPrivateKey}", "sleep 100000"]
+    max_retries = "5"
+    only        = ["amazon-ebs"]
+    timeout     = "5s"
+  }
+  provisioner "shell-local" {
+    except  = ["amazon-ebs"]
+    inline  = ["sleep 100000"]
+    timeout = "5s"
+  }
+  post-processor "amazon-import" {
+    format         = "vmdk"
+    license_type   = "BYOL"
+    region         = "eu-west-3"
+    s3_bucket_name = "hashicorp.adrien"
+    tags = {
+      Description = "packer amazon-import ${local.timestamp}"
+    }
+  }
+  post-processors {
+    post-processor "artifice" {
+      keep_input_artifact = true
+      files               = ["path/something.ova"]
+      name                = "very_special_artifice_post-processor"
+      only                = ["amazon-ebs"]
+    }
+    post-processor "amazon-import" {
+      except         = ["amazon-ebs"]
+      license_type   = "BYOL"
+      s3_bucket_name = "hashicorp.adrien"
+      tags = {
+        Description = "packer amazon-import ${local.timestamp}"
+      }
+    }
+  }
+}

command/test-fixtures/hcl2_upgrade_basic/input.json

@@ -0,0 +1,126 @@
+{
+    "variables": {
+        "aws_region": null,
+        "aws_secret_key": "",
+        "aws_access_key": ""
+    },
+    "sensitive-variables": [
+        "aws_secret_key",
+        "aws_access_key",
+        "potato"
+    ],
+    "builders": [
+        {
+            "type": "amazon-ebs",
+            "region": "{{ user `aws_region` }}",
+            "secret_key": "{{ user `aws_secret_key` }}",
+            "access_key": "{{ user `aws_access_key` }}",
+            "ami_name": "ubuntu-16-04-test-{{ timestamp }}",
+            "ami_description": "Ubuntu 16.04 LTS - expand root partition",
+            "source_ami_filter": {
+                "filters": {
+                    "virtualization-type": "hvm",
+                    "name": "ubuntu/images/*/ubuntu-xenial-16.04-amd64-server-*",
+                    "root-device-type": "ebs"
+                },
+                "owners": [
+                    "099720109477"
+                ],
+                "most_recent": true
+            },
+            "launch_block_device_mappings": [
+                {
+                    "delete_on_termination": true,
+                    "device_name": "/dev/sda1",
+                    "volume_type": "gp2",
+                    "volume_size": 48
+                }
+            ],
+            "spot_price": "0.0075",
+            "spot_instance_types": [
+                "t2.small",
+                "t2.medium",
+                "t2.large"
+            ],
+            "encrypt_boot": true,
+            "ssh_username": "ubuntu",
+            "temporary_iam_instance_profile_policy_document": {
+                "Version": "2012-10-17",
+                "Statement": [
+                    {
+                        "Effect": "Allow",
+                        "Action": [
+                            "*"
+                        ],
+                        "Resource": ["*"]
+                    }
+                ]
+            },
+            "ssh_interface": "session_manager"
+        }
+    ],
+    "provisioners": [
+        {
+            "type": "shell",
+            "only": [
+                "amazon-ebs"
+            ],
+            "max_retries": 5,
+            "timeout": "5s",
+            "inline": [
+                "echo {{ user  `secret_account` }}",
+                "echo {{ build `ID` }}",
+                "echo {{ build `SSHPrivateKey` }}",
+                "sleep 100000"
+            ]
+        },
+        {
+            "type": "shell-local",
+            "except": [
+                "amazon-ebs"
+            ],
+            "timeout": "5s",
+            "inline": [
+                "sleep 100000"
+            ]
+        }
+    ],
+    "post-processors": [
+        [
+            {
+                "type": "amazon-import",
+                "region": "eu-west-3",
+                "s3_bucket_name": "hashicorp.adrien",
+                "license_type": "BYOL",
+                "format": "vmdk",
+                "tags": {
+                    "Description": "packer amazon-import {{timestamp}}"
+                }
+            }
+        ],
+        [
+            {
+                "only": [
+                    "amazon-ebs"
+                ],
+                "files": [
+                    "path/something.ova"
+                ],
+                "keep_input_artifact": true,
+                "name": "very_special_artifice_post-processor",
+                "type": "artifice"
+            },
+            {
+                "except": [
+                    "amazon-ebs"
+                ],
+                "type": "amazon-import",
+                "s3_bucket_name": "hashicorp.adrien",
+                "license_type": "BYOL",
+                "tags": {
+                    "Description": "packer amazon-import {{timestamp}}"
+                }
+            }
+        ]
+    ]
+}

commands.go

@@ -58,5 +58,11 @@ func init() {
 				Meta: *CommandMeta,
 			}, nil
 		},
+
+		"hcl2_upgrade": func() (cli.Command, error) {
+			return &command.HCL2UpgradeCommand{
+				Meta: *CommandMeta,
+			}, nil
+		},
 	}
 }

go.mod

@@ -70,7 +70,7 @@ require (
 	github.com/hashicorp/go-uuid v1.0.2
 	github.com/hashicorp/go-version v1.2.0
 	github.com/hashicorp/golang-lru v0.5.3 // indirect
-	github.com/hashicorp/hcl/v2 v2.3.0
+	github.com/hashicorp/hcl/v2 v2.6.0
 	github.com/hashicorp/serf v0.9.2 // indirect
 	github.com/hashicorp/vault/api v1.0.4
 	github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d

go.sum

@@ -364,6 +364,8 @@ github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
 github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/hashicorp/hcl/v2 v2.3.0 h1:iRly8YaMwTBAKhn1Ybk7VSdzbnopghktCD031P8ggUE=
 github.com/hashicorp/hcl/v2 v2.3.0/go.mod h1:d+FwDBbOLvpAM3Z6J7gPj/VoAGkNe/gm352ZhjJ/Zv8=
+github.com/hashicorp/hcl/v2 v2.6.0 h1:3krZOfGY6SziUXa6H9PJU6TyohHn7I+ARYnhbeNBz+o=
+github.com/hashicorp/hcl/v2 v2.6.0/go.mod h1:bQTN5mpo+jewjJgh8jr0JUguIi7qPHUF6yIfAEN3jqY=
 github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
 github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
 github.com/hashicorp/mdns v1.0.1 h1:XFSOubp8KWB+Jd2PDyaX5xUd5bhSP/+pTDZVDMzZJM8=

hcl2template/shim/values.go

@@ -85,3 +85,48 @@ func ConfigValueFromHCL2(v cty.Value) interface{} {
 	// capsule types, and we don't currently have any such types defined.
 	panic(fmt.Errorf("can't convert %#v to config value", v))
 }
+
+// HCL2ValueFromConfigValue is the opposite of ConfigValueFromHCL2: it takes
+// a value as would be returned from the old interpolator and turns it into
+// a cty.Value so it can be used within, for example, an HCL2 EvalContext.
+func HCL2ValueFromConfigValue(v interface{}) cty.Value {
+	if v == nil {
+		return cty.NullVal(cty.DynamicPseudoType)
+	}
+	if v == UnknownVariableValue {
+		return cty.DynamicVal
+	}
+
+	switch tv := v.(type) {
+	case bool:
+		return cty.BoolVal(tv)
+	case string:
+		return cty.StringVal(tv)
+	case int:
+		return cty.NumberIntVal(int64(tv))
+	case float64:
+		return cty.NumberFloatVal(tv)
+	case []interface{}:
+		vals := make([]cty.Value, len(tv))
+		for i, ev := range tv {
+			vals[i] = HCL2ValueFromConfigValue(ev)
+		}
+		return cty.TupleVal(vals)
+	case []string:
+		vals := make([]cty.Value, len(tv))
+		for i, ev := range tv {
+			vals[i] = cty.StringVal(ev)
+		}
+		return cty.ListVal(vals)
+	case map[string]interface{}:
+		vals := map[string]cty.Value{}
+		for k, ev := range tv {
+			vals[k] = HCL2ValueFromConfigValue(ev)
+		}
+		return cty.ObjectVal(vals)
+	default:
+		// HCL/HIL should never generate anything that isn't caught by
+		// the above, so if we get here something has gone very wrong.
+		panic(fmt.Errorf("can't convert %#v to cty.Value", v))
+	}
+}

post-processor/yandex-export/script.go

@@ -43,7 +43,7 @@ Exit () {
   for i in ${PATHS}; do
     LOGDEST="${i}.exporter.log"
     echo "Uploading exporter log to ${LOGDEST}..."
-    aws s3 --endpoint-url=https://storage.yandexcloud.net cp /var/log/syslog ${LOGDEST}
+    aws s3 --region ru-central1 --endpoint-url=https://storage.yandexcloud.net cp /var/log/syslog ${LOGDEST}
   done
   exit $1
 }
@@ -58,6 +58,28 @@ echo "Disk name - ${DISKNAME}"
 echo "Export paths - ${PATHS}"
 echo "####################################"
 
+echo "Detect Service Account ID..."
+SERVICE_ACCOUNT_ID=$(GetServiceAccountId)
+echo "Use Service Account ID: ${SERVICE_ACCOUNT_ID}"
+
+echo "Create static access key..."
+SEC_json=$(yc iam access-key create --service-account-id ${SERVICE_ACCOUNT_ID} \
+    --description "this key is for export image to storage" --format json)
+
+if [ $? -ne 0 ]; then
+  echo "Failed to create static access key."
+  exit 1
+fi
+
+echo "Setup env variables to access storage..."
+eval "$(jq -r '@sh "export YC_SK_ID=\(.access_key.id); export AWS_ACCESS_KEY_ID=\(.access_key.key_id); export AWS_SECRET_ACCESS_KEY=\(.secret)"' <<<${SEC_json}  )"
+
+echo "Check access to storage..."
+if ! aws s3 --region ru-central1 --endpoint-url=https://storage.yandexcloud.net ls > /dev/null ; then
+  echo "Failed to access storage."
+  exit 1
+fi
+
 echo "Creating disk from image to be exported..."
 if ! yc compute disk create --name ${DISKNAME} --source-image-id ${IMAGE_ID} --zone ${ZONE}; then
   echo "Failed to create disk."
@@ -81,28 +103,6 @@ if ! yc compute instance detach-disk ${INSTANCE_ID}  --disk-name ${DISKNAME} ; t
   echo "Failed to detach disk."
 fi
 
-echo "Detect Service Account ID..."
-SERVICE_ACCOUNT_ID=$(GetServiceAccountId)
-echo "Use Service Account ID: ${SERVICE_ACCOUNT_ID}"
-
-echo "Create static access key..."
-SEC_json=$(yc iam access-key create --service-account-id ${SERVICE_ACCOUNT_ID} \
-    --description "this key is for export image to storage" --format json)
-
-if [ $? -ne 0 ]; then
-  echo "Failed to create static access key."
-  exit 1
-fi
-
-echo "Setup env variables to access storage..."
-eval "$(jq -r '@sh "export YC_SK_ID=\(.access_key.id); export AWS_ACCESS_KEY_ID=\(.access_key.key_id); export AWS_SECRET_ACCESS_KEY=\(.secret)"' <<<${SEC_json}  )"
-
-echo "Check access to storage..."
-if ! aws s3 --endpoint-url=https://storage.yandexcloud.net ls > /dev/null ; then
-  echo "Failed to access storage."
-  exit 1
-fi
-
 FAIL=0
 echo "Deleting disk..."
 if ! yc compute disk delete --name ${DISKNAME} ; then
@@ -111,7 +111,7 @@ if ! yc compute disk delete --name ${DISKNAME} ; then
 fi
 for i in ${PATHS}; do
   echo "Uploading qcow2 disk image to ${i}..."
-  if ! aws s3 --endpoint-url=https://storage.yandexcloud.net cp disk.qcow2 ${i}; then
+  if ! aws s3 --region ru-central1 --endpoint-url=https://storage.yandexcloud.net cp disk.qcow2 ${i}; then
     echo "Failed to upload image to ${i}."
     FAIL=1
   fi

provisioner/ansible/provisioner.go

@@ -89,6 +89,8 @@ type Config struct {
 	AnsibleEnvVars []string `mapstructure:"ansible_env_vars"`
 	// The playbook to be run by Ansible.
 	PlaybookFile string `mapstructure:"playbook_file" required:"true"`
+	// Specifies --ssh-extra-args on command line defaults to -o IdentitiesOnly=yes
+	AnsibleSSHExtraArgs []string `mapstructure:"ansible_ssh_extra_args"`
 	// The groups into which the Ansible host should
 	//  be placed. When unspecified, the host is not associated with any groups.
 	Groups []string `mapstructure:"groups"`
@@ -701,7 +703,15 @@ func (p *Provisioner) createCmdArgs(httpAddr, inventory, playbook, privKeyFile s
 
 	if p.generatedData["ConnType"] == "ssh" && len(privKeyFile) > 0 {
 		// Add ssh extra args to set IdentitiesOnly
-		args = append(args, "--ssh-extra-args", "'-o IdentitiesOnly=yes'")
+		if len(p.config.AnsibleSSHExtraArgs) > 0 {
+			var sshExtraArgs string
+			for _, sshExtraArg := range p.config.AnsibleSSHExtraArgs {
+				sshExtraArgs = sshExtraArgs + sshExtraArg
+			}
+			args = append(args, "--ssh-extra-args", fmt.Sprintf("'%s'", sshExtraArgs))
+		} else {
+			args = append(args, "--ssh-extra-args", "'-o IdentitiesOnly=yes'")
+		}
 	}
 
 	args = append(args, p.config.ExtraArguments...)

provisioner/ansible/provisioner.hcl2spec.go

@@ -20,6 +20,7 @@ type FlatConfig struct {
 	ExtraArguments        []string          `mapstructure:"extra_arguments" cty:"extra_arguments" hcl:"extra_arguments"`
 	AnsibleEnvVars        []string          `mapstructure:"ansible_env_vars" cty:"ansible_env_vars" hcl:"ansible_env_vars"`
 	PlaybookFile          *string           `mapstructure:"playbook_file" required:"true" cty:"playbook_file" hcl:"playbook_file"`
+	AnsibleSSHExtraArgs   []string          `mapstructure:"ansible_ssh_extra_args" cty:"ansible_ssh_extra_args" hcl:"ansible_ssh_extra_args"`
 	Groups                []string          `mapstructure:"groups" cty:"groups" hcl:"groups"`
 	EmptyGroups           []string          `mapstructure:"empty_groups" cty:"empty_groups" hcl:"empty_groups"`
 	HostAlias             *string           `mapstructure:"host_alias" cty:"host_alias" hcl:"host_alias"`
@@ -64,6 +65,7 @@ func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec {
 		"extra_arguments":            &hcldec.AttrSpec{Name: "extra_arguments", Type: cty.List(cty.String), Required: false},
 		"ansible_env_vars":           &hcldec.AttrSpec{Name: "ansible_env_vars", Type: cty.List(cty.String), Required: false},
 		"playbook_file":              &hcldec.AttrSpec{Name: "playbook_file", Type: cty.String, Required: false},
+		"ansible_ssh_extra_args":     &hcldec.AttrSpec{Name: "ansible_ssh_extra_args", Type: cty.List(cty.String), Required: false},
 		"groups":                     &hcldec.AttrSpec{Name: "groups", Type: cty.List(cty.String), Required: false},
 		"empty_groups":               &hcldec.AttrSpec{Name: "empty_groups", Type: cty.List(cty.String), Required: false},
 		"host_alias":                 &hcldec.AttrSpec{Name: "host_alias", Type: cty.String, Required: false},

provisioner/ansible/provisioner_test.go

@@ -490,16 +490,17 @@ func basicGenData(input map[string]interface{}) map[string]interface{} {
 
 func TestCreateCmdArgs(t *testing.T) {
 	type testcase struct {
-		TestName          string
-		PackerBuildName   string
-		PackerBuilderType string
-		UseProxy          confighelper.Trilean
-		generatedData     map[string]interface{}
-		ExtraArguments    []string
-		AnsibleEnvVars    []string
-		callArgs          []string // httpAddr inventory playbook privKeyFile
-		ExpectedArgs      []string
-		ExpectedEnvVars   []string
+		TestName            string
+		PackerBuildName     string
+		PackerBuilderType   string
+		UseProxy            confighelper.Trilean
+		generatedData       map[string]interface{}
+		AnsibleSSHExtraArgs []string
+		ExtraArguments      []string
+		AnsibleEnvVars      []string
+		callArgs            []string // httpAddr inventory playbook privKeyFile
+		ExpectedArgs        []string
+		ExpectedEnvVars     []string
 	}
 	TestCases := []testcase{
 		{
@@ -513,6 +514,18 @@ func TestCreateCmdArgs(t *testing.T) {
 			ExpectedArgs:    []string{"-e", "packer_build_name=\"packerparty\"", "-e", "packer_builder_type=fakebuilder", "-e", "ansible_ssh_private_key_file=/path/to/privkey.pem", "--ssh-extra-args", "'-o IdentitiesOnly=yes'", "-e", "hello-world", "-i", "/var/inventory", "test-playbook.yml"},
 			ExpectedEnvVars: []string{"ENV_1=pancakes", "ENV_2=bananas"},
 		},
+		{
+			// SSH with private key and an extra argument.
+			TestName:            "SSH with private key and an extra argument and a ssh extra argument",
+			PackerBuildName:     "packerparty",
+			generatedData:       basicGenData(nil),
+			ExtraArguments:      []string{"-e", "hello-world"},
+			AnsibleSSHExtraArgs: []string{"-o IdentitiesOnly=no"},
+			AnsibleEnvVars:      []string{"ENV_1=pancakes", "ENV_2=bananas"},
+			callArgs:            []string{common.HttpAddrNotImplemented, "/var/inventory", "test-playbook.yml", "/path/to/privkey.pem"},
+			ExpectedArgs:        []string{"-e", "packer_build_name=\"packerparty\"", "-e", "packer_builder_type=fakebuilder", "--ssh-extra-args", "'-o IdentitiesOnly=no'", "-e", "ansible_ssh_private_key_file=/path/to/privkey.pem", "-e", "hello-world", "-i", "/var/inventory", "test-playbook.yml"},
+			ExpectedEnvVars:     []string{"ENV_1=pancakes", "ENV_2=bananas"},
+		},
 		{
 			TestName:        "SSH with private key and an extra argument and UseProxy",
 			PackerBuildName: "packerparty",
@@ -628,6 +641,17 @@ func TestCreateCmdArgs(t *testing.T) {
 			ExpectedArgs:    []string{"-e", "packer_build_name=\"packerparty\"", "-e", "packer_builder_type=fakebuilder", "-e", "ansible_ssh_private_key_file=/path/to/privkey.pem", "--ssh-extra-args", "'-o IdentitiesOnly=yes'", "-e", "hello-world", "-i", "/var/inventory", "test-playbook.yml"},
 			ExpectedEnvVars: []string{},
 		},
+		{
+			TestName:            "Use PrivateKey and SSH Extra Arg",
+			PackerBuildName:     "packerparty",
+			UseProxy:            confighelper.TriTrue,
+			generatedData:       basicGenData(nil),
+			AnsibleSSHExtraArgs: []string{"-o IdentitiesOnly=no"},
+			ExtraArguments:      []string{"-e", "hello-world"},
+			callArgs:            []string{common.HttpAddrNotImplemented, "/var/inventory", "test-playbook.yml", "/path/to/privkey.pem"},
+			ExpectedArgs:        []string{"-e", "packer_build_name=\"packerparty\"", "-e", "packer_builder_type=fakebuilder", "-e", "ansible_ssh_private_key_file=/path/to/privkey.pem", "--ssh-extra-args", "'-o IdentitiesOnly=no'", "-e", "hello-world", "-i", "/var/inventory", "test-playbook.yml"},
+			ExpectedEnvVars:     []string{},
+		},
 		{
 			TestName:        "Use SSH Agent",
 			UseProxy:        confighelper.TriTrue,
@@ -654,6 +678,7 @@ func TestCreateCmdArgs(t *testing.T) {
 		p.config.PackerBuilderType = "fakebuilder"
 		p.config.PackerBuildName = tc.PackerBuildName
 		p.generatedData = tc.generatedData
+		p.config.AnsibleSSHExtraArgs = tc.AnsibleSSHExtraArgs
 		p.config.ExtraArguments = tc.ExtraArguments
 		p.config.AnsibleEnvVars = tc.AnsibleEnvVars
 

vendor/github.com/apparentlymart/go-textseg/LICENSE

@@ -1,95 +0,0 @@
-Copyright (c) 2017 Martin Atkins
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
----------
-
-Unicode table generation programs are under a separate copyright and license:
-
-Copyright (c) 2014 Couchbase, Inc.
-Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file
-except in compliance with the License. You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software distributed under the
-License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
-either express or implied. See the License for the specific language governing permissions
-and limitations under the License.
-
----------
-
-Grapheme break data is provided as part of the Unicode character database,
-copright 2016 Unicode, Inc, which is provided with the following license:
-
-Unicode Data Files include all data files under the directories
-http://www.unicode.org/Public/, http://www.unicode.org/reports/,
-http://www.unicode.org/cldr/data/, http://source.icu-project.org/repos/icu/, and
-http://www.unicode.org/utility/trac/browser/.
-
-Unicode Data Files do not include PDF online code charts under the
-directory http://www.unicode.org/Public/.
-
-Software includes any source code published in the Unicode Standard
-or under the directories
-http://www.unicode.org/Public/, http://www.unicode.org/reports/,
-http://www.unicode.org/cldr/data/, http://source.icu-project.org/repos/icu/, and
-http://www.unicode.org/utility/trac/browser/.
-
-NOTICE TO USER: Carefully read the following legal agreement.
-BY DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING UNICODE INC.'S
-DATA FILES ("DATA FILES"), AND/OR SOFTWARE ("SOFTWARE"),
-YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
-TERMS AND CONDITIONS OF THIS AGREEMENT.
-IF YOU DO NOT AGREE, DO NOT DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE
-THE DATA FILES OR SOFTWARE.
-
-COPYRIGHT AND PERMISSION NOTICE
-
-Copyright © 1991-2017 Unicode, Inc. All rights reserved.
-Distributed under the Terms of Use in http://www.unicode.org/copyright.html.
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of the Unicode data files and any associated documentation
-(the "Data Files") or Unicode software and any associated documentation
-(the "Software") to deal in the Data Files or Software
-without restriction, including without limitation the rights to use,
-copy, modify, merge, publish, distribute, and/or sell copies of
-the Data Files or Software, and to permit persons to whom the Data Files
-or Software are furnished to do so, provided that either
-(a) this copyright and permission notice appear with all copies
-of the Data Files or Software, or
-(b) this copyright and permission notice appear in associated
-Documentation.
-
-THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
-ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
-WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT OF THIRD PARTY RIGHTS.
-IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS
-NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
-DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
-DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
-TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-PERFORMANCE OF THE DATA FILES OR SOFTWARE.
-
-Except as contained in this notice, the name of a copyright holder
-shall not be used in advertising or otherwise to promote the sale,
-use or other dealings in these Data Files or Software without prior
-written authorization of the copyright holder.

vendor/github.com/apparentlymart/go-textseg/textseg/all_tokens.go

@@ -1,30 +0,0 @@
-package textseg
-
-import (
-	"bufio"
-	"bytes"
-)
-
-// AllTokens is a utility that uses a bufio.SplitFunc to produce a slice of
-// all of the recognized tokens in the given buffer.
-func AllTokens(buf []byte, splitFunc bufio.SplitFunc) ([][]byte, error) {
-	scanner := bufio.NewScanner(bytes.NewReader(buf))
-	scanner.Split(splitFunc)
-	var ret [][]byte
-	for scanner.Scan() {
-		ret = append(ret, scanner.Bytes())
-	}
-	return ret, scanner.Err()
-}
-
-// TokenCount is a utility that uses a bufio.SplitFunc to count the number of
-// recognized tokens in the given buffer.
-func TokenCount(buf []byte, splitFunc bufio.SplitFunc) (int, error) {
-	scanner := bufio.NewScanner(bytes.NewReader(buf))
-	scanner.Split(splitFunc)
-	var ret int
-	for scanner.Scan() {
-		ret++
-	}
-	return ret, scanner.Err()
-}

vendor/github.com/apparentlymart/go-textseg/textseg/generate.go

@@ -1,7 +0,0 @@
-package textseg
-
-//go:generate go run make_tables.go -output tables.go
-//go:generate go run make_test_tables.go -output tables_test.go
-//go:generate ruby unicode2ragel.rb --url=http://www.unicode.org/Public/9.0.0/ucd/auxiliary/GraphemeBreakProperty.txt -m GraphemeCluster -p "Prepend,CR,LF,Control,Extend,Regional_Indicator,SpacingMark,L,V,T,LV,LVT,E_Base,E_Modifier,ZWJ,Glue_After_Zwj,E_Base_GAZ" -o grapheme_clusters_table.rl
-//go:generate ragel -Z grapheme_clusters.rl
-//go:generate gofmt -w grapheme_clusters.go

vendor/github.com/apparentlymart/go-textseg/textseg/grapheme_clusters.rl

@@ -1,132 +0,0 @@
-package textseg
-
-import (
-    "errors"
-    "unicode/utf8"
-)
-
-// Generated from grapheme_clusters.rl. DO NOT EDIT
-%%{
-  # (except you are actually in grapheme_clusters.rl here, so edit away!)
-
-  machine graphclust;
-  write data;
-}%%
-
-var Error = errors.New("invalid UTF8 text")
-
-// ScanGraphemeClusters is a split function for bufio.Scanner that splits
-// on grapheme cluster boundaries.
-func ScanGraphemeClusters(data []byte, atEOF bool) (int, []byte, error) {
-    if len(data) == 0 {
-        return 0, nil, nil
-    }
-
-    // Ragel state
-	cs := 0 // Current State
-	p := 0  // "Pointer" into data
-	pe := len(data) // End-of-data "pointer"
-    ts := 0
-    te := 0
-    act := 0
-    eof := pe
-
-    // Make Go compiler happy
-    _ = ts
-    _ = te
-    _ = act
-    _ = eof
-
-    startPos := 0
-    endPos := 0
-
-    %%{
-        include GraphemeCluster "grapheme_clusters_table.rl";
-
-        action start {
-            startPos = p
-        }
-
-        action end {
-            endPos = p
-        }
-
-        action emit {
-            return endPos+1, data[startPos:endPos+1], nil
-        }
-
-        ZWJGlue = ZWJ (Glue_After_Zwj | E_Base_GAZ Extend* E_Modifier?)?;
-        AnyExtender = Extend | ZWJGlue | SpacingMark;
-        Extension = AnyExtender*;
-        ReplacementChar = (0xEF 0xBF 0xBD);
-
-        CRLFSeq = CR LF;
-        ControlSeq = Control | ReplacementChar;
-        HangulSeq = (
-            L+ (((LV? V+ | LVT) T*)?|LV?) |
-            LV V* T* |
-            V+ T* |
-            LVT T* |
-            T+
-        ) Extension;
-        EmojiSeq = (E_Base | E_Base_GAZ) Extend* E_Modifier? Extension;
-        ZWJSeq = ZWJGlue Extension;
-        EmojiFlagSeq = Regional_Indicator Regional_Indicator? Extension;
-
-        UTF8Cont = 0x80 .. 0xBF;
-        AnyUTF8 = (
-            0x00..0x7F |
-            0xC0..0xDF . UTF8Cont |
-            0xE0..0xEF . UTF8Cont . UTF8Cont |
-            0xF0..0xF7 . UTF8Cont . UTF8Cont . UTF8Cont
-        );
-
-        # OtherSeq is any character that isn't at the start of one of the extended sequences above, followed by extension
-        OtherSeq = (AnyUTF8 - (CR|LF|Control|ReplacementChar|L|LV|V|LVT|T|E_Base|E_Base_GAZ|ZWJ|Regional_Indicator|Prepend)) Extension;
-
-        # PrependSeq is prepend followed by any of the other patterns above, except control characters which explicitly break
-        PrependSeq = Prepend+ (HangulSeq|EmojiSeq|ZWJSeq|EmojiFlagSeq|OtherSeq)?;
-
-        CRLFTok = CRLFSeq >start @end;
-        ControlTok = ControlSeq >start @end;
-        HangulTok = HangulSeq >start @end;
-        EmojiTok = EmojiSeq >start @end;
-        ZWJTok = ZWJSeq >start @end;
-        EmojiFlagTok = EmojiFlagSeq >start @end;
-        OtherTok = OtherSeq >start @end;
-        PrependTok = PrependSeq >start @end;
-
-        main := |*
-            CRLFTok => emit;
-            ControlTok => emit;
-            HangulTok => emit;
-            EmojiTok => emit;
-            ZWJTok => emit;
-            EmojiFlagTok => emit;
-            PrependTok => emit;
-            OtherTok => emit;
-
-            # any single valid UTF-8 character would also be valid per spec,
-            # but we'll handle that separately after the loop so we can deal
-            # with requesting more bytes if we're not at EOF.
-        *|;
-
-        write init;
-        write exec;
-    }%%
-
-    // If we fall out here then we were unable to complete a sequence.
-    // If we weren't able to complete a sequence then either we've
-    // reached the end of a partial buffer (so there's more data to come)
-    // or we have an isolated symbol that would normally be part of a
-    // grapheme cluster but has appeared in isolation here.
-
-    if !atEOF {
-        // Request more
-        return 0, nil, nil
-    }
-
-    // Just take the first UTF-8 sequence and return that.
-    _, seqLen := utf8.DecodeRune(data)
-    return seqLen, data[:seqLen], nil
-}

vendor/github.com/apparentlymart/go-textseg/textseg/unicode2ragel.rb

@@ -1,335 +0,0 @@
-#!/usr/bin/env ruby
-#
-# This scripted has been updated to accept more command-line arguments:
-#
-#    -u, --url                        URL to process
-#    -m, --machine                    Machine name
-#    -p, --properties                 Properties to add to the machine
-#    -o, --output                     Write output to file
-#
-# Updated by: Marty Schoch <marty.schoch@gmail.com>
-# 
-# This script uses the unicode spec to generate a Ragel state machine
-# that recognizes unicode alphanumeric characters.  It generates 5
-# character classes: uupper, ulower, ualpha, udigit, and ualnum.
-# Currently supported encodings are UTF-8 [default] and UCS-4.
-#
-# Usage: unicode2ragel.rb [options]
-#    -e, --encoding [ucs4 | utf8]     Data encoding
-#    -h, --help                       Show this message
-#
-# This script was originally written as part of the Ferret search
-# engine library.
-#
-# Author: Rakan El-Khalil <rakan@well.com>
-
-require 'optparse'
-require 'open-uri'
-
-ENCODINGS = [ :utf8, :ucs4 ]
-ALPHTYPES = { :utf8 => "byte", :ucs4 => "rune" }
-DEFAULT_CHART_URL = "http://www.unicode.org/Public/5.1.0/ucd/DerivedCoreProperties.txt"
-DEFAULT_MACHINE_NAME= "WChar"
-
-###
-# Display vars & default option
-
-TOTAL_WIDTH = 80
-RANGE_WIDTH = 23
-@encoding = :utf8
-@chart_url = DEFAULT_CHART_URL
-machine_name = DEFAULT_MACHINE_NAME
-properties = []
-@output = $stdout
-
-###
-# Option parsing
-
-cli_opts = OptionParser.new do |opts|
-  opts.on("-e", "--encoding [ucs4 | utf8]", "Data encoding") do |o|
-    @encoding = o.downcase.to_sym
-  end
-  opts.on("-h", "--help", "Show this message") do
-    puts opts
-    exit
-  end
-  opts.on("-u", "--url URL", "URL to process") do |o|
-    @chart_url = o 
-  end
-  opts.on("-m", "--machine MACHINE_NAME", "Machine name") do |o|
-    machine_name = o
-  end
-  opts.on("-p", "--properties x,y,z", Array, "Properties to add to machine") do |o|
-    properties = o
-  end
-  opts.on("-o", "--output FILE", "output file") do |o|
-    @output = File.new(o, "w+")
-  end
-end
-
-cli_opts.parse(ARGV)
-unless ENCODINGS.member? @encoding
-  puts "Invalid encoding: #{@encoding}"
-  puts cli_opts
-  exit
-end
-
-##
-# Downloads the document at url and yields every alpha line's hex
-# range and description.
-
-def each_alpha( url, property ) 
-  open( url ) do |file|
-    file.each_line do |line|
-      next if line =~ /^#/;
-      next if line !~ /; #{property} #/;
-
-      range, description = line.split(/;/)
-      range.strip!
-      description.gsub!(/.*#/, '').strip!
-
-      if range =~ /\.\./
-           start, stop = range.split '..'
-      else start = stop = range
-      end
-
-      yield start.hex .. stop.hex, description
-    end
-  end
-end
-
-###
-# Formats to hex at minimum width
-
-def to_hex( n )
-  r = "%0X" % n
-  r = "0#{r}" unless (r.length % 2).zero?
-  r
-end
-
-###
-# UCS4 is just a straight hex conversion of the unicode codepoint.
-
-def to_ucs4( range )
-  rangestr  =   "0x" + to_hex(range.begin)
-  rangestr << "..0x" + to_hex(range.end) if range.begin != range.end
-  [ rangestr ]
-end
-
-##
-# 0x00     - 0x7f     -> 0zzzzzzz[7]
-# 0x80     - 0x7ff    -> 110yyyyy[5] 10zzzzzz[6]
-# 0x800    - 0xffff   -> 1110xxxx[4] 10yyyyyy[6] 10zzzzzz[6]
-# 0x010000 - 0x10ffff -> 11110www[3] 10xxxxxx[6] 10yyyyyy[6] 10zzzzzz[6] 
-
-UTF8_BOUNDARIES = [0x7f, 0x7ff, 0xffff, 0x10ffff]
-
-def to_utf8_enc( n )
-  r = 0
-  if n <= 0x7f
-    r = n
-  elsif n <= 0x7ff
-    y = 0xc0 | (n >> 6)
-    z = 0x80 | (n & 0x3f)
-    r = y << 8 | z
-  elsif n <= 0xffff
-    x = 0xe0 | (n >> 12)
-    y = 0x80 | (n >>  6) & 0x3f
-    z = 0x80 |  n        & 0x3f
-    r = x << 16 | y << 8 | z
-  elsif n <= 0x10ffff
-    w = 0xf0 | (n >> 18)
-    x = 0x80 | (n >> 12) & 0x3f
-    y = 0x80 | (n >>  6) & 0x3f
-    z = 0x80 |  n        & 0x3f
-    r = w << 24 | x << 16 | y << 8 | z
-  end
-
-  to_hex(r)
-end
-
-def from_utf8_enc( n )
-  n = n.hex
-  r = 0
-  if n <= 0x7f
-    r = n
-  elsif n <= 0xdfff
-    y = (n >> 8) & 0x1f
-    z =  n       & 0x3f
-    r = y << 6 | z
-  elsif n <= 0xefffff
-    x = (n >> 16) & 0x0f
-    y = (n >>  8) & 0x3f
-    z =  n        & 0x3f
-    r = x << 10 | y << 6 | z
-  elsif n <= 0xf7ffffff
-    w = (n >> 24) & 0x07
-    x = (n >> 16) & 0x3f
-    y = (n >>  8) & 0x3f
-    z =  n        & 0x3f
-    r = w << 18 | x << 12 | y << 6 | z
-  end
-  r
-end
-
-###
-# Given a range, splits it up into ranges that can be continuously
-# encoded into utf8.  Eg: 0x00 .. 0xff => [0x00..0x7f, 0x80..0xff]
-# This is not strictly needed since the current [5.1] unicode standard
-# doesn't have ranges that straddle utf8 boundaries.  This is included
-# for completeness as there is no telling if that will ever change.
-
-def utf8_ranges( range )
-  ranges = []
-  UTF8_BOUNDARIES.each do |max|
-    if range.begin <= max
-      if range.end <= max
-        ranges << range
-        return ranges
-      end
-
-      ranges << (range.begin .. max)
-      range = (max + 1) .. range.end
-    end
-  end
-  ranges
-end
-
-def build_range( start, stop )
-  size = start.size/2
-  left = size - 1
-  return [""] if size < 1
-
-  a = start[0..1]
-  b = stop[0..1]
-
-  ###
-  # Shared prefix
-
-  if a == b
-    return build_range(start[2..-1], stop[2..-1]).map do |elt|
-      "0x#{a} " + elt
-    end
-  end
-
-  ###
-  # Unshared prefix, end of run
-
-  return ["0x#{a}..0x#{b} "] if left.zero?
-  
-  ###
-  # Unshared prefix, not end of run
-  # Range can be 0x123456..0x56789A
-  # Which is equivalent to:
-  #     0x123456 .. 0x12FFFF
-  #     0x130000 .. 0x55FFFF
-  #     0x560000 .. 0x56789A
-
-  ret = []
-  ret << build_range(start, a + "FF" * left)
-
-  ###
-  # Only generate middle range if need be.
-
-  if a.hex+1 != b.hex
-    max = to_hex(b.hex - 1)
-    max = "FF" if b == "FF"
-    ret << "0x#{to_hex(a.hex+1)}..0x#{max} " + "0x00..0xFF " * left
-  end
-
-  ###
-  # Don't generate last range if it is covered by first range
-  
-  ret << build_range(b + "00" * left, stop) unless b == "FF"
-  ret.flatten!
-end
-
-def to_utf8( range )
-  utf8_ranges( range ).map do |r|   
-    begin_enc = to_utf8_enc(r.begin)
-    end_enc = to_utf8_enc(r.end)
-    build_range begin_enc, end_enc
-  end.flatten!
-end
-
-##
-# Perform a 3-way comparison of the number of codepoints advertised by
-# the unicode spec for the given range, the originally parsed range,
-# and the resulting utf8 encoded range.
-
-def count_codepoints( code )
-  code.split(' ').inject(1) do |acc, elt|
-    if elt =~ /0x(.+)\.\.0x(.+)/
-      if @encoding == :utf8
-        acc * (from_utf8_enc($2) - from_utf8_enc($1) + 1)
-      else
-        acc * ($2.hex - $1.hex + 1)
-      end
-    else
-      acc
-    end
-  end
-end
-
-def is_valid?( range, desc, codes )
-  spec_count  = 1
-  spec_count  = $1.to_i if desc =~ /\[(\d+)\]/
-  range_count = range.end - range.begin + 1
-
-  sum = codes.inject(0) { |acc, elt| acc + count_codepoints(elt) }
-  sum == spec_count and sum == range_count
-end
-
-##
-# Generate the state maching to stdout
-
-def generate_machine( name, property )
-  pipe = " "
-  @output.puts "    #{name} = "
-  each_alpha( @chart_url, property ) do |range, desc|
-
-    codes = (@encoding == :ucs4) ? to_ucs4(range) : to_utf8(range)
-
-    #raise "Invalid encoding of range #{range}: #{codes.inspect}" unless 
-    #  is_valid? range, desc, codes
-
-    range_width = codes.map { |a| a.size }.max
-    range_width = RANGE_WIDTH if range_width < RANGE_WIDTH
-
-    desc_width  = TOTAL_WIDTH - RANGE_WIDTH - 11
-    desc_width -= (range_width - RANGE_WIDTH) if range_width > RANGE_WIDTH
-
-    if desc.size > desc_width
-      desc = desc[0..desc_width - 4] + "..."
-    end
-
-    codes.each_with_index do |r, idx|
-      desc = "" unless idx.zero?
-      code = "%-#{range_width}s" % r
-      @output.puts "      #{pipe} #{code} ##{desc}"
-      pipe = "|"
-    end
-  end
-  @output.puts "      ;"
-  @output.puts ""
-end
-
-@output.puts <<EOF
-# The following Ragel file was autogenerated with #{$0} 
-# from: #{@chart_url}
-#
-# It defines #{properties}.
-#
-# To use this, make sure that your alphtype is set to #{ALPHTYPES[@encoding]},
-# and that your input is in #{@encoding}.
-
-%%{
-    machine #{machine_name};
-    
-EOF
-
-properties.each { |x| generate_machine( x, x ) }
-
-@output.puts <<EOF
-}%%
-EOF

vendor/github.com/apparentlymart/go-textseg/textseg/utf8_seqs.go

@@ -1,19 +0,0 @@
-package textseg
-
-import "unicode/utf8"
-
-// ScanGraphemeClusters is a split function for bufio.Scanner that splits
-// on UTF8 sequence boundaries.
-//
-// This is included largely for completeness, since this behavior is already
-// built in to Go when ranging over a string.
-func ScanUTF8Sequences(data []byte, atEOF bool) (int, []byte, error) {
-	if len(data) == 0 {
-		return 0, nil, nil
-	}
-	r, seqLen := utf8.DecodeRune(data)
-	if r == utf8.RuneError && !atEOF {
-		return 0, nil, nil
-	}
-	return seqLen, data[:seqLen], nil
-}

vendor/github.com/hashicorp/hcl/v2/CHANGELOG.md

@@ -1,5 +1,39 @@
 # HCL Changelog
 
+## v2.6.0 (June 4, 2020)
+
+### Enhancements
+
+* hcldec: Add a new `Spec`, `ValidateSpec`, which allows custom validation of values at decode-time. ([#387](https://github.com/hashicorp/hcl/pull/387))
+
+### Bugs Fixed
+
+* hclsyntax: Fix panic with combination of sequences and null arguments ([#386](https://github.com/hashicorp/hcl/pull/386))
+* hclsyntax: Fix handling of unknown values and sequences ([#386](https://github.com/hashicorp/hcl/pull/386))
+
+## v2.5.1 (May 14, 2020)
+
+### Bugs Fixed
+
+* hclwrite: handle legacy dot access of numeric indexes. ([#369](https://github.com/hashicorp/hcl/pull/369))
+* hclwrite: Fix panic for dotted full splat (`foo.*`) ([#374](https://github.com/hashicorp/hcl/pull/374))
+
+## v2.5.0 (May 6, 2020)
+
+### Enhancements
+
+* hclwrite: Generate multi-line objects and maps. ([#372](https://github.com/hashicorp/hcl/pull/372))
+
+## v2.4.0 (Apr 13, 2020)
+
+### Enhancements
+
+* The Unicode data tables that HCL uses to produce user-perceived "column" positions in diagnostics and other source ranges are now updated to Unicode 12.0.0, which will cause HCL to produce more accurate column numbers for combining characters introduced to Unicode since Unicode 9.0.0.
+
+### Bugs Fixed
+
+* json: Fix panic when parsing malformed JSON. ([#358](https://github.com/hashicorp/hcl/pull/358))
+
 ## v2.3.0 (Jan 3, 2020)
 
 ### Enhancements

vendor/github.com/hashicorp/hcl/v2/diagnostic.go

@@ -22,14 +22,14 @@ const (
 )
 
 // Diagnostic represents information to be presented to a user about an
-// error or anomoly in parsing or evaluating configuration.
+// error or anomaly in parsing or evaluating configuration.
 type Diagnostic struct {
 	Severity DiagnosticSeverity
 
 	// Summary and Detail contain the English-language description of the
 	// problem. Summary is a terse description of the general problem and
 	// detail is a more elaborate, often-multi-sentence description of
-	// the probem and what might be done to solve it.
+	// the problem and what might be done to solve it.
 	Summary string
 	Detail  string
 

vendor/github.com/hashicorp/hcl/v2/go.mod

@@ -1,9 +1,11 @@
 module github.com/hashicorp/hcl/v2
 
+go 1.12
+
 require (
 	github.com/agext/levenshtein v1.2.1
 	github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3
-	github.com/apparentlymart/go-textseg v1.0.0
+	github.com/apparentlymart/go-textseg/v12 v12.0.0
 	github.com/davecgh/go-spew v1.1.1
 	github.com/go-test/deep v1.0.3
 	github.com/google/go-cmp v0.3.1

vendor/github.com/hashicorp/hcl/v2/go.sum

@@ -4,6 +4,8 @@ github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3 h1:ZSTrOEhi
 github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3/go.mod h1:oL81AME2rN47vu18xqj1S1jPIPuN7afo62yKTNn3XMM=
 github.com/apparentlymart/go-textseg v1.0.0 h1:rRmlIsPEEhUTIKQb7T++Nz/A5Q6C9IuX2wFoYVvnCs0=
 github.com/apparentlymart/go-textseg v1.0.0/go.mod h1:z96Txxhf3xSFMPmb5X/1W05FF/Nj9VFpLOpjS5yuumk=
+github.com/apparentlymart/go-textseg/v12 v12.0.0 h1:bNEQyAGak9tojivJNkoqWErVCQbjdL7GzRt3F8NvfJ0=
+github.com/apparentlymart/go-textseg/v12 v12.0.0/go.mod h1:S/4uRK2UtaQttw1GenVJEynmyUenKwP++x/+DdGV/Ec=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/go-test/deep v1.0.3 h1:ZrJSEWsXzPOxaZnFteGEfooLba+ju3FYIbOrS+rQd68=

vendor/github.com/hashicorp/hcl/v2/gohcl/doc.go

@@ -17,6 +17,7 @@
 //    attr (the default) indicates that the value is to be populated from an attribute
 //    block indicates that the value is to populated from a block
 //    label indicates that the value is to populated from a block label
+//    optional is the same as attr, but the field is optional
 //    remain indicates that the value is to be populated from the remaining body after populating other fields
 //
 // "attr" fields may either be of type *hcl.Expression, in which case the raw
@@ -34,6 +35,9 @@
 // the blocks being decoded. In this case, the name token is used only as
 // an identifier for the label in diagnostic messages.
 //
+// "optional" fields behave like "attr" fields, but they are optional
+// and will not give parsing errors if they are missing.
+//
 // "remain" can be placed on a single field that may be either of type
 // hcl.Body or hcl.Attributes, in which case any remaining body content is
 // placed into this field for delayed processing. If no "remain" field is

vendor/github.com/hashicorp/hcl/v2/hcldec/spec.go

@@ -1565,6 +1565,52 @@ func (s *TransformFuncSpec) sourceRange(content *hcl.BodyContent, blockLabels []
 	return s.Wrapped.sourceRange(content, blockLabels)
 }
 
+// ValidateFuncSpec is a spec that allows for extended
+// developer-defined validation. The validation function receives the
+// result of the wrapped spec.
+//
+// The Subject field of the returned Diagnostic is optional. If not
+// specified, it is automatically populated with the range covered by
+// the wrapped spec.
+//
+type ValidateSpec struct {
+	Wrapped Spec
+	Func    func(value cty.Value) hcl.Diagnostics
+}
+
+func (s *ValidateSpec) visitSameBodyChildren(cb visitFunc) {
+	cb(s.Wrapped)
+}
+
+func (s *ValidateSpec) decode(content *hcl.BodyContent, blockLabels []blockLabel, ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
+	wrappedVal, diags := s.Wrapped.decode(content, blockLabels, ctx)
+	if diags.HasErrors() {
+		// We won't try to run our function in this case, because it'll probably
+		// generate confusing additional errors that will distract from the
+		// root cause.
+		return cty.UnknownVal(s.impliedType()), diags
+	}
+
+	validateDiags := s.Func(wrappedVal)
+	// Auto-populate the Subject fields if they weren't set.
+	for i := range validateDiags {
+		if validateDiags[i].Subject == nil {
+			validateDiags[i].Subject = s.sourceRange(content, blockLabels).Ptr()
+		}
+	}
+
+	diags = append(diags, validateDiags...)
+	return wrappedVal, diags
+}
+
+func (s *ValidateSpec) impliedType() cty.Type {
+	return s.Wrapped.impliedType()
+}
+
+func (s *ValidateSpec) sourceRange(content *hcl.BodyContent, blockLabels []blockLabel) hcl.Range {
+	return s.Wrapped.sourceRange(content, blockLabels)
+}
+
 // noopSpec is a placeholder spec that does nothing, used in situations where
 // a non-nil placeholder spec is required. It is not exported because there is
 // no reason to use it directly; it is always an implementation detail only.

vendor/github.com/hashicorp/hcl/v2/hclsyntax/expression.go

@@ -260,6 +260,20 @@ func (e *FunctionCallExpr) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnosti
 		}
 
 		switch {
+		case expandVal.Type().Equals(cty.DynamicPseudoType):
+			if expandVal.IsNull() {
+				diags = append(diags, &hcl.Diagnostic{
+					Severity:    hcl.DiagError,
+					Summary:     "Invalid expanding argument value",
+					Detail:      "The expanding argument (indicated by ...) must not be null.",
+					Subject:     expandExpr.Range().Ptr(),
+					Context:     e.Range().Ptr(),
+					Expression:  expandExpr,
+					EvalContext: ctx,
+				})
+				return cty.DynamicVal, diags
+			}
+			return cty.DynamicVal, diags
 		case expandVal.Type().IsTupleType() || expandVal.Type().IsListType() || expandVal.Type().IsSetType():
 			if expandVal.IsNull() {
 				diags = append(diags, &hcl.Diagnostic{
@@ -406,22 +420,39 @@ func (e *FunctionCallExpr) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnosti
 			} else {
 				param = varParam
 			}
-			argExpr := e.Args[i]
 
-			// TODO: we should also unpick a PathError here and show the
-			// path to the deep value where the error was detected.
-			diags = append(diags, &hcl.Diagnostic{
-				Severity: hcl.DiagError,
-				Summary:  "Invalid function argument",
-				Detail: fmt.Sprintf(
-					"Invalid value for %q parameter: %s.",
-					param.Name, err,
-				),
-				Subject:     argExpr.StartRange().Ptr(),
-				Context:     e.Range().Ptr(),
-				Expression:  argExpr,
-				EvalContext: ctx,
-			})
+			// this can happen if an argument is (incorrectly) null.
+			if i > len(e.Args)-1 {
+				diags = append(diags, &hcl.Diagnostic{
+					Severity: hcl.DiagError,
+					Summary:  "Invalid function argument",
+					Detail: fmt.Sprintf(
+						"Invalid value for %q parameter: %s.",
+						param.Name, err,
+					),
+					Subject:     args[len(params)].StartRange().Ptr(),
+					Context:     e.Range().Ptr(),
+					Expression:  e,
+					EvalContext: ctx,
+				})
+			} else {
+				argExpr := e.Args[i]
+
+				// TODO: we should also unpick a PathError here and show the
+				// path to the deep value where the error was detected.
+				diags = append(diags, &hcl.Diagnostic{
+					Severity: hcl.DiagError,
+					Summary:  "Invalid function argument",
+					Detail: fmt.Sprintf(
+						"Invalid value for %q parameter: %s.",
+						param.Name, err,
+					),
+					Subject:     argExpr.StartRange().Ptr(),
+					Context:     e.Range().Ptr(),
+					Expression:  argExpr,
+					EvalContext: ctx,
+				})
+			}
 
 		default:
 			diags = append(diags, &hcl.Diagnostic{

vendor/github.com/hashicorp/hcl/v2/hclsyntax/parser.go

@@ -6,7 +6,7 @@ import (
 	"strconv"
 	"unicode/utf8"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 	"github.com/hashicorp/hcl/v2"
 	"github.com/zclconf/go-cty/cty"
 )
@@ -670,6 +670,7 @@ Traversal:
 				trav := make(hcl.Traversal, 0, 1)
 				var firstRange, lastRange hcl.Range
 				firstRange = p.NextRange()
+				lastRange = marker.Range
 				for p.Peek().Type == TokenDot {
 					dot := p.Read()
 

vendor/github.com/hashicorp/hcl/v2/hclsyntax/parser_template.go

@@ -5,7 +5,7 @@ import (
 	"strings"
 	"unicode"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 	"github.com/hashicorp/hcl/v2"
 	"github.com/zclconf/go-cty/cty"
 )

vendor/github.com/hashicorp/hcl/v2/hclsyntax/spec.md

@@ -273,7 +273,7 @@ tuple = "[" (
 object = "{" (
     (objectelem ("," objectelem)* ","?)?
 ) "}";
-objectelem = (Identifier | Expression) "=" Expression;
+objectelem = (Identifier | Expression) ("=" | ":") Expression;
 ```
 
 Only tuple and object values can be directly constructed via native syntax.

vendor/github.com/hashicorp/hcl/v2/hclsyntax/token.go

@@ -4,7 +4,7 @@ import (
 	"bytes"
 	"fmt"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 	"github.com/hashicorp/hcl/v2"
 )
 

vendor/github.com/hashicorp/hcl/v2/hclwrite/generate.go

@@ -119,15 +119,15 @@ func appendTokensForValue(val cty.Value, toks Tokens) Tokens {
 			Type:  hclsyntax.TokenOBrace,
 			Bytes: []byte{'{'},
 		})
+		if val.LengthInt() > 0 {
+			toks = append(toks, &Token{
+				Type:  hclsyntax.TokenNewline,
+				Bytes: []byte{'\n'},
+			})
+		}
 
 		i := 0
 		for it := val.ElementIterator(); it.Next(); {
-			if i > 0 {
-				toks = append(toks, &Token{
-					Type:  hclsyntax.TokenComma,
-					Bytes: []byte{','},
-				})
-			}
 			eKey, eVal := it.Element()
 			if hclsyntax.ValidIdentifier(eKey.AsString()) {
 				toks = append(toks, &Token{
@@ -142,6 +142,10 @@ func appendTokensForValue(val cty.Value, toks Tokens) Tokens {
 				Bytes: []byte{'='},
 			})
 			toks = appendTokensForValue(eVal, toks)
+			toks = append(toks, &Token{
+				Type:  hclsyntax.TokenNewline,
+				Bytes: []byte{'\n'},
+			})
 			i++
 		}
 

vendor/github.com/hashicorp/hcl/v2/hclwrite/parser.go

@@ -88,6 +88,16 @@ func (it inputTokens) PartitionType(ty hclsyntax.TokenType) (before, within, aft
 	panic(fmt.Sprintf("didn't find any token of type %s", ty))
 }
 
+func (it inputTokens) PartitionTypeOk(ty hclsyntax.TokenType) (before, within, after inputTokens, ok bool) {
+	for i, t := range it.writerTokens {
+		if t.Type == ty {
+			return it.Slice(0, i), it.Slice(i, i+1), it.Slice(i+1, len(it.nativeTokens)), true
+		}
+	}
+
+	return inputTokens{}, inputTokens{}, inputTokens{}, false
+}
+
 func (it inputTokens) PartitionTypeSingle(ty hclsyntax.TokenType) (before inputTokens, found *Token, after inputTokens) {
 	before, within, after := it.PartitionType(ty)
 	if within.Len() != 1 {
@@ -404,6 +414,19 @@ func parseTraversalStep(nativeStep hcl.Traverser, from inputTokens) (before inpu
 		children = step.inTree.children
 		before, from, after = from.Partition(nativeStep.SourceRange())
 
+		if inBefore, dot, from, ok := from.PartitionTypeOk(hclsyntax.TokenDot); ok {
+			children.AppendUnstructuredTokens(inBefore.Tokens())
+			children.AppendUnstructuredTokens(dot.Tokens())
+
+			valBefore, valToken, valAfter := from.PartitionTypeSingle(hclsyntax.TokenNumberLit)
+			children.AppendUnstructuredTokens(valBefore.Tokens())
+			key := newNumber(valToken)
+			step.key = children.Append(key)
+			children.AppendUnstructuredTokens(valAfter.Tokens())
+
+			return before, newNode(step), after
+		}
+
 		var inBefore, oBrack, keyTokens, cBrack inputTokens
 		inBefore, oBrack, from = from.PartitionType(hclsyntax.TokenOBrack)
 		children.AppendUnstructuredTokens(inBefore.Tokens())
@@ -498,8 +521,8 @@ func writerTokens(nativeTokens hclsyntax.Tokens) Tokens {
 // The tokens are assumed to be in source order and non-overlapping, which
 // will be true if the token sequence from the scanner is used directly.
 func partitionTokens(toks hclsyntax.Tokens, rng hcl.Range) (start, end int) {
-	// We us a linear search here because we assume tha in most cases our
-	// target range is close to the beginning of the sequence, and the seqences
+	// We use a linear search here because we assume that in most cases our
+	// target range is close to the beginning of the sequence, and the sequences
 	// are generally small for most reasonable files anyway.
 	for i := 0; ; i++ {
 		if i >= len(toks) {

vendor/github.com/hashicorp/hcl/v2/hclwrite/tokens.go

@@ -4,7 +4,7 @@ import (
 	"bytes"
 	"io"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 	"github.com/hashicorp/hcl/v2"
 	"github.com/hashicorp/hcl/v2/hclsyntax"
 )

vendor/github.com/hashicorp/hcl/v2/json/scanner.go

@@ -3,7 +3,7 @@ package json
 import (
 	"fmt"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 	"github.com/hashicorp/hcl/v2"
 )
 
@@ -107,6 +107,15 @@ func scan(buf []byte, start pos) []token {
 			})
 			// If we've encountered an invalid then we might as well stop
 			// scanning since the parser won't proceed beyond this point.
+			// We insert a synthetic EOF marker here to match the expectations
+			// of consumers of this data structure.
+			p.Pos.Column++
+			p.Pos.Byte++
+			tokens = append(tokens, token{
+				Type:  tokenEOF,
+				Bytes: nil,
+				Range: posRange(p, p),
+			})
 			return tokens
 		}
 	}

vendor/github.com/hashicorp/hcl/v2/pos_scanner.go

@@ -4,7 +4,7 @@ import (
 	"bufio"
 	"bytes"
 
-	"github.com/apparentlymart/go-textseg/textseg"
+	"github.com/apparentlymart/go-textseg/v12/textseg"
 )
 
 // RangeScanner is a helper that will scan over a buffer using a bufio.SplitFunc

vendor/modules.txt

@@ -94,8 +94,6 @@ github.com/antchfx/xquery/xml
 github.com/antihax/optional
 # github.com/apparentlymart/go-cidr v1.0.1
 github.com/apparentlymart/go-cidr/cidr
-# github.com/apparentlymart/go-textseg v1.0.0
-github.com/apparentlymart/go-textseg/textseg
 # github.com/apparentlymart/go-textseg/v12 v12.0.0
 github.com/apparentlymart/go-textseg/v12/textseg
 # github.com/approvals/go-approval-tests v0.0.0-20160714161514-ad96e53bea43
@@ -354,7 +352,7 @@ github.com/hashicorp/hcl/hcl/token
 github.com/hashicorp/hcl/json/parser
 github.com/hashicorp/hcl/json/scanner
 github.com/hashicorp/hcl/json/token
-# github.com/hashicorp/hcl/v2 v2.3.0
+# github.com/hashicorp/hcl/v2 v2.6.0
 github.com/hashicorp/hcl/v2
 github.com/hashicorp/hcl/v2/ext/customdecode
 github.com/hashicorp/hcl/v2/ext/dynblock

version/version.go

@@ -14,7 +14,7 @@ const Version = "1.6.2"
 // A pre-release marker for the version. If this is "" (empty string)
 // then it means that it is a final release. Otherwise, this is a pre-release
 // such as "dev" (in development), "beta", "rc1", etc.
-const VersionPrerelease = "dev"
+const VersionPrerelease = ""
 
 func FormattedVersion() string {
 	var versionString bytes.Buffer

website/data/docs-navigation.js

@@ -160,7 +160,7 @@ export default [
   'terminology',
   {
     category: 'commands',
-    content: ['build', 'console', 'fix', 'inspect', 'validate'],
+    content: ['build', 'console', 'fix', 'inspect', 'validate', 'hcl2_upgrade'],
   },
   {
     category: 'templates',

website/data/version.js

@@ -1 +1 @@
-export default '1.6.1'
+export default '1.6.2'

website/pages/docs/builders/googlecompute.mdx

@@ -25,64 +25,80 @@ information.
 
 ## Authentication
 
-Authenticating with Google Cloud services requires at most one JSON file,
-called the _account file_. The _account file_ is **not** required if you are
-running the `googlecompute` Packer builder from a GCE instance with a
-properly-configured [Compute Engine Service
+Authenticating with Google Cloud services requires either a User Application Default Credentials
+or a JSON Service Account Key. These are **not** required if you are
+running the `googlecompute` Packer builder on Google Cloud with a
+properly-configured [Google Service
 Account](https://cloud.google.com/compute/docs/authentication).
 
-### Running With a Compute Engine Service Account
+### Running locally on your workstation.
 
-If you run the `googlecompute` Packer builder from a GCE instance, you can
-configure that instance to use a [Compute Engine Service
+If you run the `googlecompute` Packer builder locally on your workstation, you will 
+need to install the Google Cloud SDK and authenticate using [User Application Default
+Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default).
+You don't need to specify an _account file_ if you are using this method. Your user
+must have at least `Compute Instance Admin (v1)` & `Service Account User` roles 
+to use Packer succesfully.
+
+
+### Running on Google Cloud
+
+If you run the `googlecompute` Packer builder on GCE or GKE, you can
+configure that instance or cluster to use a [Google Service
 Account](https://cloud.google.com/compute/docs/authentication). This will allow
 Packer to authenticate to Google Cloud without having to bake in a separate
 credential/authentication file.
 
-To create a GCE instance that uses a service account, provide the required
-scopes when launching the instance.
-
-For `gcloud`, do this via the `--scopes` parameter:
+It is recommended that you create a custom service account for Packer and assign it
+ `Compute Instance Admin (v1)` & `Service Account User` roles.
 
+For `gcloud`, you can run the following commands:
 ```shell-session
+$ gcloud iam service-accounts create packer \
+  --project YOUR_GCP_PROJECT \
+  --description="Packer Service Account" \
+  --display-name="Packer Service Account"
+
+$ gcloud projects add-iam-policy-binding YOUR_GCP_PROJECT \
+    --member=serviceAccount:packer@YOUR_GCP_PROJECT.iam.gserviceaccount.com \
+    --role=roles/compute.instanceAdmin.v1
+
+$ gcloud projects add-iam-policy-binding YOUR_GCP_PROJECT \
+    --member=serviceAccount:packer@YOUR_GCP_PROJECT.iam.gserviceaccount.com \
+    --role=roles/iam.serviceAccountUser
+
 $ gcloud compute instances create INSTANCE-NAME \
   --project YOUR_GCP_PROJECT \
-  --image-family ubuntu-1804-lts \
-  --image-project gce-uefi-images \
+  --image-family ubuntu-2004-lts \
+  --image-project ubuntu-os-cloud \
   --network YOUR_GCP_NETWORK \
   --zone YOUR_GCP_ZONE \
-  --scopes "https://www.googleapis.com/auth/compute,https://www.googleapis.com/auth/devstorage.full_control"
+  --service-account=packer@YOUR_GCP_PROJECT.iam.gserviceaccount.com \
+  --scopes="https://www.googleapis.com/auth/cloud-platform"
 ```
 
-For the [Google Developers Console](https://console.developers.google.com):
-
-1.  Choose "Show advanced options"
-2.  Tick "Enable Compute Engine service account"
-3.  Choose "Read Write" for Compute
-4.  Choose "Full" for "Storage"
-
 **The service account will be used automatically by Packer as long as there is
 no _account file_ specified in the Packer configuration file.**
 
-### Running Without a Compute Engine Service Account
+### Running outside of Google Cloud
 
-The [Google Developers Console](https://console.developers.google.com) allows
+The [Google Cloud Console](https://console.cloud.google.com) allows
 you to create and download a credential file that will let you use the
 `googlecompute` Packer builder anywhere. To make the process more
 straightforwarded, it is documented here.
 
-1.  Log into the [Google Developers
-    Console](https://console.developers.google.com) and select a project.
+1.  Log into the [Google Cloud
+    Console](https://console.cloud.google.com/iam-admin/serviceaccounts) and select a project.
 
-2.  Under the "API Manager" section, click "Credentials."
+2.  Click Select a project, choose your project, and click Open.
 
-3.  Click the "Create credentials" button, select "Service account key"
+3.  Click Create Service Account.
 
-4.  Create a new service account that at least has
-    `Compute Engine Instance Admin (v1)` and `Service Account User` roles.
+4.  Enter a service account name (friendly display name), an optional description, select the `Compute Engine Instance Admin (v1)` and `Service Account User` roles, and then click Save.
 
-5.  Choose `JSON` as the Key type and click "Create". A JSON file will be
-    downloaded automatically. This is your _account file_.
+5.  Generate a JSON Key and save it in a secure location.
+
+5.  Set the Environment Variable `GOOGLE_APPLICATION_CREDENTIALS` to point to the path of the service account key.
 
 ### Precedence of Authentication Methods
 
@@ -115,9 +131,7 @@ location found:
 
 Below is a fully functioning example. It doesn't do anything useful since no
 provisioners or startup-script metadata are defined, but it will effectively
-repackage an existing GCE image. The account_file is obtained in the previous
-section. If it parses as JSON it is assumed to be the file itself, otherwise,
-it is assumed to be the path to the file containing the JSON.
+repackage an existing GCE image.
 
 <Tabs>
 <Tab heading="JSON">
@@ -127,9 +141,8 @@ it is assumed to be the path to the file containing the JSON.
   "builders": [
     {
       "type": "googlecompute",
-      "account_file": "account.json",
       "project_id": "my project",
-      "source_image": "debian-7-wheezy-v20150127",
+      "source_image": "debian-9-stretch-v20200805",
       "ssh_username": "packer",
       "zone": "us-central1-a"
     }
@@ -142,9 +155,8 @@ it is assumed to be the path to the file containing the JSON.
 
 ```hcl
 source "googlecompute" "basic-example" {
-  account_file = "account.json"
   project_id = "my project"
-  source_image = "debian-7-wheezy-v20150127"
+  source_image = "debian-9-stretch-v20200805"
   ssh_username = "packer"
   zone = "us-central1-a"
 }
@@ -168,18 +180,20 @@ using the gcloud command.
 Or alternatively by navigating to [https://console.cloud.google.com/networking/firewalls/list](https://console.cloud.google.com/networking/firewalls/list).
 
 Once this is set up, the following is a complete working packer config after
-setting a valid `account_file` and `project_id`:
+setting a valid `project_id`:
+
+<Tabs>
+<Tab heading="JSON">
 
 ```json
 {
   "builders": [
     {
       "type": "googlecompute",
-      "account_file": "account.json",
       "project_id": "my project",
-      "source_image": "windows-server-2016-dc-v20170227",
+      "source_image": "windows-server-2019-dc-v20200813",
       "disk_size": "50",
-      "machine_type": "n1-standard-1",
+      "machine_type": "n1-standard-2",
       "communicator": "winrm",
       "winrm_username": "packer_user",
       "winrm_insecure": true,
@@ -193,6 +207,33 @@ setting a valid `account_file` and `project_id`:
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+source "googlecompute" "windows-example" {
+  project_id = "MY_PROJECT"
+  source_image = "windows-server-2019-dc-v20200813"
+  zone = "us-central1-a"
+  disk_size = 50,
+  machine_type = "n1-standard-2",
+  communicator ="winrm",
+  winrm_username = "packer_user",
+  winrm_insecure = true,
+  winrm_use_ssl = true,
+  metadata {
+    windows-startup-script-cmd = "winrm quickconfig -quiet & net user /add packer_user & net localgroup administrators packer_user /add & winrm set winrm/config/service/auth @{Basic=\"true\"}"
+  }
+}
+
+build {
+  sources = ["sources.googlecompute.windows-example"]
+}
+```
+
+</Tab>
+</Tabs>
+
 -> **Warning:** Please note that if you're setting up WinRM for provisioning, you'll probably want to turn it off or restrict its permissions as part of a shutdown script at the end of Packer's provisioning process. For more details on the why/how, check out this useful blog post and the associated code:
 https://cloudywindows.io/post/winrm-for-provisioning-close-the-door-on-the-way-out-eh/
 
@@ -206,12 +247,14 @@ Virtualization for VM
 Instances](https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances)
 for details.
 
+<Tabs>
+<Tab heading="JSON">
+
 ```json
 {
   "builders": [
     {
       "type": "googlecompute",
-      "account_file": "account.json",
       "project_id": "my project",
       "source_image_family": "centos-7",
       "ssh_username": "packer",
@@ -222,6 +265,77 @@ for details.
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+source "googlecompute" "basic-example" {
+  project_id = "my project"
+  source_image_family = "centos-7"
+  ssh_username = "packer"
+  zone = "us-central1-a"
+  image_licenses = ["projects/vm-options/global/licenses/enable-vmx"]
+}
+
+build {
+  sources = ["sources.googlecompute.basic-example"]
+}
+```
+
+</Tab>
+</Tabs>
+
+### Shared VPC Example
+
+This is an example of using the `network_project_id` configuration option to create
+a GCE instance in a Shared VPC Network. See [Creating a GCE Instance using Shared
+VPC](https://cloud.google.com/vpc/docs/provisioning-shared-vpc#creating_an_instance_in_a_shared_subnet)
+for details. The user/service account running Packer must have `Compute Network User` role on
+the Shared VPC Host Project to create the instance in addition to the other roles mentioned in the
+Running on Google Cloud section.
+
+<Tabs>
+<Tab heading="JSON">
+
+```json
+{
+  "builders": [
+    {
+      "type": "googlecompute",
+      "project_id": "my project",
+      "subnetwork": "default",
+      "source_image_family": "centos-7",
+      "network_project_id": "SHARED_VPC_PROJECT",
+      "ssh_username": "packer",
+      "zone": "us-central1-a",
+      "image_licenses": ["projects/vm-options/global/licenses/enable-vmx"]
+    }
+  ]
+}
+```
+
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+source "googlecompute" "sharedvpc-example" {
+  project_id = "my project"
+  source_image_family = "centos-7"
+  subnetwork = "default"
+  network_project_id = "SHARED_VPC_PROJECT"
+  ssh_username = "packer"
+  zone = "us-central1-a"
+  image_licenses = ["projects/vm-options/global/licenses/enable-vmx"]
+}
+
+build {
+  sources = ["sources.googlecompute.sharedvpc-example"]
+}
+```
+
+</Tab>
+</Tabs>
+
 ## Configuration Reference
 
 Configuration options are organized below into two categories: required and
@@ -252,9 +366,9 @@ will vary depending on the duration of the startup script. If
 `startup_script_file` is set, the `startup-script` `metadata` field will be
 overwritten. In other words, `startup_script_file` takes precedence.
 
-The builder does not check for a pass/fail/error signal from the startup
-script, at this time. Until such support is implemented, startup scripts should
-be robust, as an image will still be built even when a startup script fails.
+The builder does check for a pass/fail/error signal from the startup
+script by tracking the `startup-script-status` metadata. Packer will check if this key
+is set to done and if it not set to done before the timeout, Packer will fail the build.
 
 ### Windows
 
@@ -267,8 +381,7 @@ that it finishes before the instance shuts down.
 
 Startup script logs can be copied to a Google Cloud Storage (GCS) location
 specified via the `startup-script-log-dest` instance creation `metadata` field.
-The GCS location must be writeable by the credentials provided in the builder
-config's `account_file`.
+The GCS location must be writeable by the service account of the instance that Packer created.
 
 ## Gotchas
 

website/pages/docs/builders/vmware/iso.mdx

@@ -95,6 +95,12 @@ necessary for this build to succeed and can be found further down the page.
 
 @include 'builder/vmware/iso/Config-not-required.mdx'
 
+### Extra Disk Configuration
+
+#### Optional:
+
+@include 'builder/vmware/common/DiskConfig-not-required.mdx'
+
 ### ISO Configuration
 
 @include 'common/ISOConfig.mdx'

website/pages/docs/builders/vmware/vmx.mdx

@@ -72,12 +72,6 @@ There are many configuration options available for the VMware builder. They are
 organized below into two categories: required and optional. Within each
 category, the available options are alphabetized and described.
 
-In addition to the options listed here, a
-[communicator](/docs/templates/communicator) can be configured for this
-builder.
-
-## Configuration Reference
-
 There are many configuration options available for the builder. In addition to
 the items listed here, you will want to look at the general configuration
 references for
@@ -103,6 +97,12 @@ necessary for this build to succeed and can be found further down the page.
 
 @include 'builder/vmware/vmx/Config-not-required.mdx'
 
+### Extra Disk Configuration
+
+#### Optional:
+
+@include 'builder/vmware/common/DiskConfig-not-required.mdx'
+
 ### Http directory configuration
 
 @include 'common/HTTPConfig.mdx'

website/pages/docs/commands/hcl2_upgrade.mdx

@@ -0,0 +1,80 @@
+---
+description: |
+  The `packer hcl2_upgrade` Packer command is used to transpile a JSON
+  configuration template to it's formatted HCL2 counterpart. The command will
+  return a zero exit status on success, and a non-zero exit status on failure.
+layout: docs
+page_title: packer hcl2_upgrade - Commands
+sidebar_title: <tt>hcl2_upgrade</tt>
+---
+
+-> **Note:** This command is Beta, and currently being improved upon; do not
+hesitate [opening a new
+issue](https://github.com/hashicorp/packer/issues/new/choose) if you find
+something wrong.
+
+# `hcl2_upgrade` Command
+
+The `packer hcl2_upgrade` Packer command is used to transpile a JSON
+configuration template to it's formatted HCL2 counterpart. The command will
+return a zero exit status on success, and a non-zero exit status on failure.
+
+Example usage:
+
+```shell-session
+$ packer hcl2_upgrade my-template.json
+
+Successfully created my-template.json.pkr.hcl
+```
+
+## User variables using other user variables
+
+Packer JSON recently started allowing using user variables from variables. In
+HCL2, input variables cannot use functions nor other variables and are
+virtually static, local variables must be used instead to craft more dynamic
+variables. For that reason `hcl2_upgrade` cannot decide for you what local
+variables to create and the `hcl2_upgrade` command will simply output all seen
+variables as an input variable, it is now up to you to create a local variable.
+
+Here is an example of a local variable using a string input variables:
+
+```hcl
+variable "foo" {
+  default = "Hello,"
+}
+
+variable "bar" {
+  default = "World!"
+}
+
+locals {
+  baz = "${var.foo} ${var.bar}"
+}
+```
+
+## Go template functions
+
+`hcl2_upgrade` will do its best to transform your go _template calls_ to HCL2,
+here is the list of calls that should get transformed:
+  - ```{{ user `my_var` }}``` becomes ```${var.my_var}```.
+  - ```{{ env `my_var` }}``` becomes ```${var.my_var}```. Packer HCL2 supports 
+    environment variables through input variables. See
+    [docs](http://packer.io/docs/from-1.5/variables#environment-variables)
+    for more info.
+  - ```{{ timestamp }}``` becomes ```${local.timestamp}```, the local variable
+    will be created for all generated files.
+  - ```{{ build `ID` }}``` becomes ```${build.ID}```.
+
+The rest of the calls should remain go template calls for now, this will be
+improved over time.
+
+-> **Note**: The `hcl2_upgrade` command does its best to transform template
+calls to their JSON counterpart, but it might fail. In that case the
+`hcl2_upgrade` command will simply output the local HCL2 block without
+transformation and with the error message in a comment. We are currently
+working on improving this part of the transformer.
+
+## Options
+
+- `-output-file` - File where to put the hcl2 generated config. Defaults to
+  JSON_TEMPLATE.pkr.hcl

website/pages/docs/from-1.5/variables.mdx

@@ -168,6 +168,72 @@ $ packer build -var='image_id_map={"us-east-1":"ami-abc123","us-east-2":"ami-def
 
 The `-var` option can be used any number of times in a single command.
 
+If you plan to assign variables via the command line, we strongly recommend that
+you at least set a default type instead of using empty blocks; this helps the
+HCL parser understand what is being set.
+
+For example:
+
+```hcl
+variable "pizza" {
+  type = string
+}
+source "null" "example" {
+    communicator = "none"
+}
+build {
+  sources = [
+    "source.null.example"
+  ]
+  provisioner "shell-local" {
+    inline = ["echo $PIZZA"]
+    environment_vars = ["PIZZA=${var.pizza}"]
+  }
+}
+```
+
+If you call the above template using the command
+
+```sh
+packer build -var pizza=pineapple shell_local_variables.pkr.hcl
+```
+
+then the Packer build will run successfully. However, if you define the variable
+using an empty block, the parser will not know what type the variable is, and it
+cannot infer the type from the command line, as shown in this example:
+
+```hcl
+variable "pizza" {}
+source "null" "example" {
+    communicator = "none"
+}
+build {
+  sources = [
+    "source.null.example"
+  ]
+  provisioner "shell-local" {
+    inline = ["echo $PIZZA"]
+    environment_vars = ["PIZZA=${var.pizza}"]
+  }
+}
+```
+The above template will result in the error:
+
+```
+Error: Variables not allowed
+   on <value for var.pizza from arguments> line 1:
+   (source code not available)
+Variables may not be used here.
+```
+
+You can work around this either by quoting the variable on the command line, or
+by adding the type to the variable block as shown in the previous example.
+Setting the expected type is the more resilient option.
+
+```sh
+packer build -var 'pizza="pineapple"' shell_local_variables.pkr.hcl
+```
+
 ### Variable Definitions (`.pkrvars.hcl`) Files
 
 To set lots of variables, it is more convenient to specify their values in a

website/pages/docs/provisioners/inspec.mdx

@@ -16,6 +16,10 @@ target configured to use SSH, runs an SSH server, executes `inspec exec`, and
 marshals InSpec tests through the SSH server to the machine being provisioned
 by Packer.
 
+-> **Note:** Inspec is required to be installed on the host machine and 
+available in it's corresponding path. It is not required to be installed 
+on the provisioned image.
+
 ## Basic Example
 
 This is a fully functional template that will test an image on DigitalOcean.

website/pages/docs/provisioners/salt-masterless.mdx

@@ -21,6 +21,9 @@ master.
 
 The example below is fully functional.
 
+<Tabs>
+<Tab heading="JSON">
+
 ```json
 {
   "type": "salt-masterless",
@@ -28,6 +31,18 @@ The example below is fully functional.
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+provisioner "salt-masterless" {
+  local_state_tree = "/Users/me/salt"
+}
+```
+
+</Tab>
+</Tabs>
+
 ## Configuration Reference
 
 The reference of available configuration options is listed below. The only
@@ -101,7 +116,7 @@ Optional:
   "windows".
 
 - `formulas` (array of strings) - An array of git source formulas to be downloaded to the local
-  state tree prior to moving to the remote state tree. Note: `//directory` must be included in 
+  state tree prior to moving to the remote state tree. Note: `//directory` must be included in
   the URL to download the appropriate formula directory. Example:
   `git::https://github.com/saltstack-formulas/vault-formula.git//vault?ref=v1.2.3`
 

website/pages/docs/provisioners/windows-restart.mdx

@@ -26,12 +26,25 @@ so Windows must be completely booted in order to continue.
 
 The example below is fully functional.
 
+<Tabs>
+<Tab heading="JSON">
+
 ```json
 {
   "type": "windows-restart"
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+provisioner "windows-restart" {}
+```
+
+</Tab>
+</Tabs>
+
 ## Configuration Reference
 
 The reference of available configuration options is listed below.
@@ -64,6 +77,9 @@ Optional parameters:
 - `restart_check_command` (string) - A command to execute to check if the
   restart succeeded. This will be done in a loop. Example usage:
 
+<Tabs>
+<Tab heading="JSON">
+
 ```json
 {
   "type": "windows-restart",
@@ -71,6 +87,18 @@ Optional parameters:
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+provisioner "windows-restart" {
+  restart_check_command = "powershell -command \"& {Write-Output 'restarted.'}\""
+}
+```
+
+</Tab>
+</Tabs>
+
 - `restart_timeout` (string) - The timeout to wait for the restart. By
   default this is 5 minutes. Example value: `5m`. If you are installing
   updates or have a lot of startup services, you will probably need to

website/pages/docs/provisioners/windows-shell.mdx

@@ -18,6 +18,9 @@ The windows-shell Packer provisioner runs commands on a Windows machine using
 
 The example below is fully functional.
 
+<Tabs>
+<Tab heading="JSON">
+
 ```json
 {
   "type": "windows-shell",
@@ -25,6 +28,18 @@ The example below is fully functional.
 }
 ```
 
+</Tab>
+<Tab heading="HCL2">
+
+```hcl
+provisioner "windows-shell" {
+  inline = ["dir c:\\"]
+}
+```
+
+</Tab>
+</Tabs>
+
 ## Configuration Reference
 
 @include 'provisioners/shell-config.mdx'

website/pages/partials/builder/amazon/common/PolicyDocument-not-required.mdx

@@ -0,0 +1,5 @@
+<!-- Code generated from the comments of the PolicyDocument struct in builder/amazon/common/run_config.go; DO NOT EDIT MANUALLY -->
+
+- `Version` (string) - Version
+
+- `Statement` ([]Statement) - Statement

website/pages/partials/builder/amazon/common/Statement-not-required.mdx

@@ -0,0 +1,7 @@
+<!-- Code generated from the comments of the Statement struct in builder/amazon/common/run_config.go; DO NOT EDIT MANUALLY -->
+
+- `Effect` (string) - Effect
+
+- `Action` ([]string) - Action
+
+- `Resource` ([]string) - Resource

website/pages/partials/builder/vmware/common/DiskConfig-not-required.mdx

@@ -0,0 +1,45 @@
+<!-- Code generated from the comments of the DiskConfig struct in builder/vmware/common/disk_config.go; DO NOT EDIT MANUALLY -->
+
+- `disk_additional_size` ([]uint) - The size(s) of any additional
+  hard disks for the VM in megabytes. If this is not specified then the VM
+  will only contain a primary hard disk. The builder uses expandable, not
+  fixed-size virtual hard disks, so the actual file representing the disk will
+  not use the full size unless it is full.
+
+- `disk_adapter_type` (string) - The adapter type of the VMware virtual disk to create. This option is
+  for advanced usage, modify only if you know what you're doing. Some of
+  the options you can specify are `ide`, `sata`, `nvme` or `scsi` (which
+  uses the "lsilogic" scsi interface by default). If you specify another
+  option, Packer will assume that you're specifying a `scsi` interface of
+  that specified type. For more information, please consult [Virtual Disk
+  Manager User's Guide](http://www.vmware.com/pdf/VirtualDiskManager.pdf)
+  for desktop VMware clients. For ESXi, refer to the proper ESXi
+  documentation.
+
+- `vmdk_name` (string) - The filename of the virtual disk that'll be created,
+  without the extension. This defaults to "disk".
+
+- `disk_type_id` (string) - The type of VMware virtual disk to create. This
+  option is for advanced usage.
+  
+    For desktop VMware clients:
+  
+    Type ID | Description
+    ------- | ---
+    `0`     | Growable virtual disk contained in a single file (monolithic sparse).
+    `1`     | Growable virtual disk split into 2GB files (split sparse).
+    `2`     | Preallocated virtual disk contained in a single file (monolithic flat).
+    `3`     | Preallocated virtual disk split into 2GB files (split flat).
+    `4`     | Preallocated virtual disk compatible with ESX server (VMFS flat).
+    `5`     | Compressed disk optimized for streaming.
+  
+    The default is `1`.
+  
+    For ESXi, this defaults to `zeroedthick`. The available options for ESXi
+    are: `zeroedthick`, `eagerzeroedthick`, `thin`. `rdm:dev`, `rdmp:dev`,
+    `2gbsparse` are not supported. Due to default disk compaction, when using
+    `zeroedthick` or `eagerzeroedthick` set `skip_compaction` to `true`.
+  
+    For more information, please consult the [Virtual Disk Manager User's
+    Guide](https://www.vmware.com/pdf/VirtualDiskManager.pdf) for desktop
+    VMware clients. For ESXi, refer to the proper ESXi documentation.

website/pages/partials/builder/vmware/common/ExportConfig-not-required.mdx

@@ -1,33 +1,30 @@
 <!-- Code generated from the comments of the ExportConfig struct in builder/vmware/common/export_config.go; DO NOT EDIT MANUALLY -->
 
 - `format` (string) - Either "ovf", "ova" or "vmx", this specifies the output
-  format of the exported virtual machine. This defaults to "ovf".
-  Before using this option, you need to install ovftool. This option
-  currently only works when option remote_type is set to "esx5".
+  format of the exported virtual machine. This defaults to "ovf" for
+  remote (esx) builds, and "vmx" for local builds.
+  Before using this option, you need to install ovftool.
   Since ovftool is only capable of password based authentication
-  remote_password must be set when exporting the VM.
+  remote_password must be set when exporting the VM from a remote instance.
+  If you are building locally, Packer will create a vmx and then
+  export that vm to an ovf or ova. Packer will not delete the vmx and vmdk
+  files; this is left up to the user if you don't want to keep those
+  files.
 
 - `ovftool_options` ([]string) - Extra options to pass to ovftool during export. Each item in the array
   is a new argument. The options `--noSSLVerify`, `--skipManifestCheck`,
-  and `--targetType` are reserved, and should not be passed to this
-  argument. Currently, exporting the build VM (with ovftool) is only
-  supported when building on ESXi e.g. when `remote_type` is set to
-  `esx5`. See the [Building on a Remote vSphere
-  Hypervisor](/docs/builders/vmware-iso#building-on-a-remote-vsphere-hypervisor)
-  section below for more info.
+  and `--targetType` are used by Packer for remote exports, and should not
+  be passed to this argument. For ovf/ova exports from local builds, Packer
+  does not automatically set any ovftool options.
 
-- `skip_export` (bool) - Defaults to `false`. When enabled, Packer will not export the VM. Useful
-  if the build output is not the resultant image, but created inside the
-  VM. Currently, exporting the build VM is only supported when building on
-  ESXi e.g. when `remote_type` is set to `esx5`. See the [Building on a
-  Remote vSphere
-  Hypervisor](/docs/builders/vmware-iso#building-on-a-remote-vsphere-hypervisor)
-  section below for more info.
+- `skip_export` (bool) - Defaults to `false`. When true, Packer will not export the VM. This can
+  be useful if the build output is not the resultant image, but created
+  inside the VM.
 
-- `keep_registered` (bool) - Set this to true if you would like to keep
-  the VM registered with the remote ESXi server. If you do not need to export
-  the vm, then also set skip_export: true in order to avoid an unnecessary
-  step of using ovftool to export the vm. Defaults to false.
+- `keep_registered` (bool) - Set this to true if you would like to keep a remotely-built
+  VM registered with the remote ESXi server. If you do not need to export
+  the vm, then also set `skip_export: true` in order to avoid unnecessarily
+  using ovftool to export the vm. Defaults to false.
 
 - `skip_compaction` (bool) - VMware-created disks are defragmented and
   compacted at the end of the build process using vmware-vdiskmanager or

website/pages/partials/builder/vmware/iso/Config-not-required.mdx

@@ -1,54 +1,10 @@
 <!-- Code generated from the comments of the Config struct in builder/vmware/iso/config.go; DO NOT EDIT MANUALLY -->
 
-- `disk_additional_size` ([]uint) - The size(s) of any additional
-  hard disks for the VM in megabytes. If this is not specified then the VM
-  will only contain a primary hard disk. The builder uses expandable, not
-  fixed-size virtual hard disks, so the actual file representing the disk will
-  not use the full size unless it is full.
-
-- `disk_adapter_type` (string) - The adapter type of the VMware virtual disk to create. This option is
-  for advanced usage, modify only if you know what you're doing. Some of
-  the options you can specify are `ide`, `sata`, `nvme` or `scsi` (which
-  uses the "lsilogic" scsi interface by default). If you specify another
-  option, Packer will assume that you're specifying a `scsi` interface of
-  that specified type. For more information, please consult [Virtual Disk
-  Manager User's Guide](http://www.vmware.com/pdf/VirtualDiskManager.pdf)
-  for desktop VMware clients. For ESXi, refer to the proper ESXi
-  documentation.
-
-- `vmdk_name` (string) - The filename of the virtual disk that'll be created,
-  without the extension. This defaults to packer.
-
 - `disk_size` (uint) - The size of the hard disk for the VM in megabytes.
   The builder uses expandable, not fixed-size virtual hard disks, so the
   actual file representing the disk will not use the full size unless it
   is full. By default this is set to 40000 (about 40 GB).
 
-- `disk_type_id` (string) - The type of VMware virtual disk to create. This
-  option is for advanced usage.
-  
-    For desktop VMware clients:
-  
-    Type ID | Description
-    ------- | ---
-    `0`     | Growable virtual disk contained in a single file (monolithic sparse).
-    `1`     | Growable virtual disk split into 2GB files (split sparse).
-    `2`     | Preallocated virtual disk contained in a single file (monolithic flat).
-    `3`     | Preallocated virtual disk split into 2GB files (split flat).
-    `4`     | Preallocated virtual disk compatible with ESX server (VMFS flat).
-    `5`     | Compressed disk optimized for streaming.
-  
-    The default is `1`.
-  
-    For ESXi, this defaults to `zeroedthick`. The available options for ESXi
-    are: `zeroedthick`, `eagerzeroedthick`, `thin`. `rdm:dev`, `rdmp:dev`,
-    `2gbsparse` are not supported. Due to default disk compaction, when using
-    `zeroedthick` or `eagerzeroedthick` set `skip_compaction` to `true`.
-  
-    For more information, please consult the [Virtual Disk Manager User's
-    Guide](https://www.vmware.com/pdf/VirtualDiskManager.pdf) for desktop
-    VMware clients. For ESXi, refer to the proper ESXi documentation.
-
 - `cdrom_adapter_type` (string) - The adapter type (or bus) that will be used
   by the cdrom device. This is chosen by default based on the disk adapter
   type. VMware tends to lean towards ide for the cdrom device unless

website/pages/partials/from-1.5/beta-hcl2-note.mdx

@@ -2,4 +2,5 @@
 Packer is still in Beta. Please see the [Packer Issue
 Tracker](https://github.com/hashicorp/packer/issues/9176) for a list of
 supported features. For the old-style stable configuration language see
-[template docs](/docs/templates).
+[template docs](/docs/templates). You can now transform your JSON file into an
+HCL2 config file using the [hcl2_upgrade command](/docs/commands/hcl2_upgrade).

website/pages/partials/provisioner/ansible/Config-not-required.mdx

@@ -46,6 +46,8 @@
     "ansible_env_vars": [ "WINRM_PASSWORD={{.WinRMPassword}}" ],
     ```
 
+- `ansible_ssh_extra_args` ([]string) - Specifies --ssh-extra-args on command line defaults to -o IdentitiesOnly=yes
+
 - `groups` ([]string) - The groups into which the Ansible host should
    be placed. When unspecified, the host is not associated with any groups.